| \documentclass{article} |
| \usepackage{microtype,graphicx,booktabs,xurl,hyperref} |
| \newcommand{\theHalgorithm}{\arabic{algorithm}} |
| \usepackage{icml2026} |
| \usepackage{amsmath,amssymb,amsthm,mathtools} |
| \usepackage{algorithm,algorithmic,float,multirow,tabularx,xcolor,enumitem,placeins} |
| \usepackage[capitalize,noabbrev]{cleveref} |
| \definecolor{gold}{HTML}{C68B25} |
| \definecolor{goldpale}{HTML}{FFF4DA} |
| \definecolor{muted}{HTML}{666666} |
| \newcommand{\method}{\textsc{DooABLe}} |
| \newcommand{\E}{\mathbb E} |
| \newcommand{\R}{\mathbb R} |
| \newcommand{\T}{\mathcal T} |
| \newcommand{\Y}{\mathcal Y} |
| \newcommand{\KL}{\operatorname{KL}} |
| \newcommand{\TV}{\operatorname{TV}} |
| \newcommand{\lse}{\operatorname{LSE}} |
| \newcommand{\pending}{\ensuremath{\cdots}} |
| \newcommand{\algcomment}[1]{\hfill\textcolor{muted}{\(\triangleright\) #1}} |
| \DeclareMathOperator*{\argmin}{arg\,min} |
| \DeclareMathOperator*{\argmax}{arg\,max} |
| \newtheorem{proposition}{Proposition} |
| \newtheorem{theorem}{Theorem} |
| \setlist[itemize]{leftmargin=*,itemsep=2pt,topsep=3pt} |
| \setlist[enumerate]{leftmargin=*,itemsep=2pt,topsep=3pt} |
| \newcommand{\tablefont}{\fontsize{8}{10}\selectfont} |
|
|
|
|
| \icmltitlerunning{DooABLe: Learning Target Distributions with Budgeted Executable Paths} |
| \begin{document} |
| \twocolumn[ |
| \icmltitle{DooABLe: Learning Target Distributions\\with Budgeted Executable Paths} |
| \begin{icmlauthorlist} |
| \icmlauthor{Anonymous Authors}{anon} |
| \end{icmlauthorlist} |
| \icmlaffiliation{anon}{Anonymous Institution} |
| \icmlcorrespondingauthor{Anonymous Author}{anonymous@example.com} |
| \icmlkeywords{generative modeling, executable paths, molecular design, constrained generation} |
| \vskip 0.3in |
| ] |
| \printAffiliationsAndNotice{} |
| \begin{abstract} |
| Generating an outcome with desired properties often requires a short sequence of admissible operations. In molecular design, each candidate must be reachable from available compounds through a specified number of chemical reactions. Existing approaches support reward-guided generation and synthesis-aware construction, while the joint choice of outcome probabilities and execution costs requires an explicit distribution over complete routes. We introduce \method{} (Distributional Optimization Over Admissible Budget-Limited Executions), a learning objective for a prescribed distribution of reachable outcomes with cost-dependent conditional paths. An entropy-regularized objective within each endpoint yields a normalized route distribution, and a backward free-energy recursion permits its estimation through local predecessor sets. Forward trajectory-balance learning then matches the desired endpoint probabilities while retaining the full executable path. We derive the optimal joint law and separate bounds for endpoint error and conditional route error. The evaluation covers budgeted discrete generation, reaction-based molecular design from an Enamine parent catalog, multi-objective property control, and computational scaling. |
| \end{abstract} |
|
|
|
|
| \section{Introduction} |
| Generative design connects a desired outcome to the operations required to produce it. A molecule may satisfy a binding or permeability objective while requiring unavailable reagents or a long synthetic route \citep{synthesizable,synthemol}. Similar constraints arise when editing a biological sequence, constructing a program, or planning a sequence of discrete interventions. In these settings, the output includes a final state and a finite sequence of admissible operations. The available starting states, operation set, and execution budget determine which outcomes can be generated. |
|
|
| \begin{figure*}[t] |
| \centering |
| \includegraphics[width=\textwidth]{figures/overview.pdf} |
| \caption{\textbf{\method{}.} Admissible actions connect available parents to outcomes within budget $B$. The prefix free energy $v$ determines normalized backward probabilities $q$, including the choice of terminal route length. Forward trajectory balance matches the target weight $r_z(y)$ and the conditional route probability. At the optimum, outcome probabilities are proportional to $r_z$, and routes to a fixed outcome have probability proportional to $\exp(-C/\tau)$.} |
| \label{fig:overview} |
| \end{figure*} |
|
|
| Diffusion and flow matching support flexible distributions over continuous and discrete states \citep{ho2020ddpm,song2021scorebased,lipman2023flowmatching,austin2021d3pm,gat2024discreteflowmatching}. With consistency models and learned flow maps, sampling can use a small number of network evaluations \citep{song2023consistencymodels,frans2025shortcutmodels,geng2025meanflows}. Recent stochastic extensions further support pathwise simulation and reward alignment \citep{kiyohara2025neuralstochasticflows,holderrieth2026glass,potaptchik2026metaflowmaps,holderrieth2026diamondmaps,mccallum2026strongstochasticflowmaps}. The steps in these samplers follow the learned generative dynamics. A constraint on physical construction requires an additional operation model and explicit accounting of its costs. |
|
|
| For molecules, synthesis-aware generation connects structures to available reactants and reaction templates. Early approaches generated synthetic construction plans or searched through reaction outcomes \citep{synthesizable,synthesisdags,chembo}. More recent work combines these actions with generative flow networks, including SynFlowNet and RxnFlow \citep{synflownet,rxnflow}. Compositional generation also permits joint synthesis and three-dimensional pose design \citep{cgflow}. Experimental synthesis of generated antibiotics further supports the value of restricting generation to chemically grounded operations \citep{synthemol}. These capabilities motivate a specific design setting in which an orderable parent compound can undergo at most one or two additional reactions. |
|
|
| Reward-proportional endpoint sampling is well established in generative flow networks \citep{gfn,gfnfoundations,tb}. Multiple construction routes can terminate at the same outcome, and normalized backward policies account for this multiplicity. Trajectory and subtrajectory objectives permit learning from sampled paths \citep{tb,subtb}, while learned backward policies affect route allocation and training efficiency \citep{backward}. Prescribed endpoint marginals also arise in Schr\"odinger bridges, where path probabilities depend on a chosen reference process \citep{dsb,imf,sf2m}. The question here concerns the execution distribution that accompanies a specified outcome law. Changing route costs should change how a selected outcome is constructed while preserving its specified sampling probability. |
|
|
| An explicit conditional objective gives this separation. For each reachable outcome, the route distribution minimizes expected execution cost with conditional entropy regularization. The corresponding normalization occurs within the set of routes to that outcome. A global normalization then determines probabilities across outcomes. This distinction matters whenever routes differ in length, cost, or multiplicity. For two equally weighted outcomes, adding alternative realizations of one outcome changes its conditional path distribution while leaving the desired endpoint probabilities equal. |
|
|
| Here, we introduce \method{}, a joint distribution of outcomes and budgeted executable paths (\cref{fig:overview}). We derive an endpoint-conditioned path objective, estimate its free energies with a backward recursion, and train a forward sampler through trajectory balance. A single route temperature controls the preference for economical realizations, while property weights determine the target outcome distribution. The molecular evaluation starts from a fixed parent catalog and counts additional reactions explicitly. Our contributions are summarized below. |
| \begin{enumerate} |
| \item We derive the optimal joint law for prescribed outcome probabilities and entropy-regularized execution costs under a finite budget. |
| \item We develop backward free-energy learning with forward trajectory balance, and bound endpoint and conditional route errors separately. |
| \item We provide a reproducible evaluation across exact discrete systems, reaction-based molecular design, property trade-offs, and catalog scaling. |
| \end{enumerate} |
|
|
|
|
|
|
|
|
| \section{Preliminaries} |
| \label{sec:prelim} |
| \paragraph{Executable paths.} |
| Consider a directed acyclic graph with initial node $s_0$, admissible edges $e$, and terminal outcomes $y$. A path $\xi=(e_1,\ldots,e_n)$ starts at $s_0$ and terminates at $y(\xi)$. Its additive cost is |
| \begin{equation} |
| C(\xi)=\sum_{i=1}^{n}c(e_i),\qquad c(e_i)\geq0. |
| \label{eq:cost} |
| \end{equation} |
| The path cost is the sum of its edge costs. A step counter can be included in the state to represent bounded trajectories through an underlying graph with cycles. Distinct paths may have the same terminal outcome. |
|
|
| \paragraph{Forward and backward distributions.} |
| A forward policy $p_\theta(e\mid s)$ assigns normalized probabilities to outgoing edges. Its path probability is |
| \begin{equation} |
| P_\theta(\xi)=\prod_{i=1}^{n}p_\theta(e_i\mid s_{i-1}). |
| \label{eq:forward} |
| \end{equation} |
| A backward policy $q_\phi(e\mid s')$ is normalized over edges entering $s'$. Reversing a terminal path gives |
| \begin{equation} |
| Q_\phi(\xi\mid y)=\prod_{i=1}^{n}q_\phi(e_i\mid s_i). |
| \label{eq:backward} |
| \end{equation} |
| For a finite graph in which every node is reachable from $s_0$, backward traversal terminates at $s_0$, so $Q_\phi(\cdot\mid y)$ is a conditional probability distribution \citep{gfnfoundations}. |
|
|
| \paragraph{Conditional entropy and relative entropy.} |
| For a joint path law $P$ with endpoint marginal $P_Y$, the conditional entropy is |
| \begin{equation} |
| H_P(\Xi\mid Y)=-\sum_\xi P(\xi)\log P(\xi\mid y(\xi)). |
| \label{eq:entropy} |
| \end{equation} |
| It measures the uncertainty over routes after fixing the outcome. Relative entropy is $\KL(P\|Q)=\sum_\xi P(\xi)\log[P(\xi)/Q(\xi)]$. The chain rule separates endpoint divergence from the mean conditional path divergence. Entropy regularization also appears in stochastic control and probabilistic generation \citep{sac,gfnfoundations}. |
|
|
|
|
| \section{DooABLe} |
| \label{sec:method} |
| \subsection{Reachable Outcome Distributions} |
| Let $\mathcal C$ be a set of available starting states and $B$ an execution budget. A root edge selects $x_0\in\mathcal C$. Subsequent actions apply a declared executor $x_{i+1}=F(x_i,a_i)$ and consume nonnegative resources $b(a_i)$. The feasible path set is |
| \begin{equation} |
| \T_B=\left\{\xi:\ x_0\in\mathcal C,\ a_i\in\mathcal A(x_i),\ \sum_i b(a_i)\leq B\right\}. |
| \label{eq:support} |
| \end{equation} |
| Paths end with an explicit stop action. The terminal set $\Y_B=\{y(\xi):\xi\in\T_B\}$ contains each outcome once. For a user specification $z$ and positive outcome weight $r_z(y)$, define |
| \begin{equation} |
| \pi_{B,z}(y)=\frac{r_z(y)}{Z_{B,z}},\qquad |
| Z_{B,z}=\sum_{y'\in\Y_B}r_z(y'). |
| \label{eq:target} |
| \end{equation} |
| Thus, each reachable outcome receives probability proportional to its specified weight. Multiple parents, route lengths, or action sequences leading to the same outcome contribute to one terminal node. Hard outcome constraints can be imposed by restricting $\Y_B$ before defining the positive weights. |
|
|
| For multiple bounded utility functions $u_j(y)\in[0,1]$, one choice is |
| \begin{equation} |
| \log r_z(y)=\beta\sum_{j=1}^{J}w_j u_j(y),\qquad |
| w_j\geq0,\quad\sum_jw_j=1. |
| \label{eq:reward} |
| \end{equation} |
| The weights $w_j$ specify property preferences, and $\beta>0$ controls their concentration. The execution budget $B$ determines support, while the additive cost $C$ distinguishes feasible routes. |
|
|
| \subsection{Conditional Route Optimization} |
| Among path laws with endpoint marginal $\pi_{B,z}$, minimize |
| \begin{equation} |
| \begin{aligned} |
| P^*_{B,z,\tau}=\argmin_{P:\ P_Y=\pi_{B,z}} |
| \bigl\{\E_P[C(\Xi)]-\tau H_P(\Xi\mid Y)\bigr\}, |
| \end{aligned} |
| \label{eq:primal} |
| \end{equation} |
| where $\tau>0$ is the route temperature. The objective balances execution cost and route diversity at the prescribed endpoint marginal. Define the conditional path partition |
| \begin{equation} |
| A_{B,\tau}(y)=\sum_{\xi\in\T_B:\ y(\xi)=y} |
| \exp\{-C(\xi)/\tau\}. |
| \label{eq:partition} |
| \end{equation} |
| This sum assigns greater weight to less costly routes ending at $y$. The optimal joint law is |
| \begin{equation} |
| \boxed{P^*_{B,z,\tau}(\xi)= |
| \pi_{B,z}(y(\xi))\, |
| \frac{\exp\{-C(\xi)/\tau\}}{A_{B,\tau}(y(\xi))}.} |
| \label{eq:joint} |
| \end{equation} |
| The probability of a path equals its endpoint probability multiplied by its normalized conditional route weight. The endpoint law remains $\pi_{B,z}$ for every $\tau>0$. |
|
|
| \begin{proposition}[Conditional optimum] |
| \label{prop:optimum} |
| On a finite nonempty feasible path set with positive endpoint weights, \cref{eq:joint} uniquely minimizes \cref{eq:primal}. For any feasible $P$ with the same endpoint marginal, |
| \begin{equation} |
| \mathcal J(P)-\mathcal J(P^*)= |
| \tau\E_{y\sim\pi_{B,z}}\KL(P(\cdot\mid y)\|Q^*(\cdot\mid y)), |
| \label{eq:gap} |
| \end{equation} |
| where $Q^*(\xi\mid y)=\exp\{-C(\xi)/\tau\}/A_{B,\tau}(y)$ and $\mathcal J$ denotes the objective in \cref{eq:primal}. |
| \end{proposition} |
| The objective gap is the mean conditional divergence scaled by the route temperature. As $\tau\downarrow0$, probability within each endpoint concentrates on minimum-cost realizations. The proof appears in \cref{app:proofs}. |
|
|
| \subsection{Backward Free Energies} |
| Augment execution states with consumed resources and a finite step counter. For each outcome $y$, connect all eligible stopping states to one terminal node $t_y$. These stop edges have zero cost. Let $A(s)$ be the sum of exponential cost weights over root-to-$s$ prefixes. Its recursion is |
| \begin{equation} |
| A(s_0)=1,\qquad A(s)=\sum_{e:u\to s}A(u)e^{-c(e)/\tau}. |
| \label{eq:prefix} |
| \end{equation} |
| Each incoming prefix contributes its weight multiplied by the weight of its final edge. In particular, $A(t_y)=A_{B,\tau}(y)$. Writing $v(s)=\log A(s)$ gives |
| \begin{equation} |
| v(s_0)=0,\qquad |
| v(s)=\lse_{e:u\to s}\{v(u)-c(e)/\tau\}. |
| \label{eq:bellman} |
| \end{equation} |
| Here, $\lse$ is log-sum-exp. The backward edge probability is |
| \begin{equation} |
| q^*(e\mid s)=\exp\{v(u)-c(e)/\tau-v(s)\}. |
| \label{eq:qstar} |
| \end{equation} |
| Multiplication along a route cancels its intermediate free energies, giving $Q^*(\xi\mid y)$ in \cref{eq:joint}. |
|
|
| A learned value $v_\phi(s;B,\tau)$ can share parameters across states and budgets. Enforce $v_\phi(s_0)=0$ and fit the local recursion with |
| \begin{equation} |
| \begin{aligned} |
| \widehat v_\phi(s)&=\lse_{e:u\to s}[v_\phi(u)-c(e)/\tau],\\ |
| \mathcal L_{\rm route}(\phi)&=\E_{s\sim\nu}\bigl[v_\phi(s)-\operatorname{sg}(\widehat v_\phi(s))\bigr]^2, |
| \end{aligned} |
| \label{eq:route_loss} |
| \end{equation} |
| where $\nu$ samples nonroot states and $\operatorname{sg}$ stops gradients through the fitted target. Normalize the learned backward policy locally, |
| \begin{equation} |
| q_\phi(e\mid s)= |
| \frac{e^{v_\phi(u)-c(e)/\tau}} |
| {\sum_{e':u'\to s}e^{v_\phi(u')-c(e')/\tau}}. |
| \label{eq:qlearned} |
| \end{equation} |
| This normalization defines a conditional path distribution throughout training. Complete predecessor sets are required for the stated support. On a stored graph, the prefix values can also be computed by one topological pass. |
|
|
| \subsection{Forward Learning and Sampling} |
| The forward policy is trained against the locally normalized backward law. With a learned scalar $\zeta_\theta$ for $\log Z_{B,z}$, define the trajectory residual |
| \begin{equation} |
| \Delta_{\theta,\phi}(\xi)= |
| \zeta_\theta+\log P_\theta(\xi) |
| -\log r_z(y)-\log Q_\phi(\xi\mid y). |
| \label{eq:tb_residual} |
| \end{equation} |
| The residual compares forward path mass with the outcome weight and conditional backward path mass. The sampling loss is |
| \begin{equation} |
| \mathcal L_{\rm sample}(\theta)= |
| \E_{\xi\sim\mu}\bigl[\Delta_{\theta,\phi}(\xi)^2\bigr]. |
| \label{eq:tb_loss} |
| \end{equation} |
| This is the trajectory-balance objective \citep{tb}, with the backward law learned from execution costs. Sampling each action from a mixture of the current and uniform policies defines $\mu$. Route and sampling updates alternate, with $Q_\phi$ fixed during each sampling update. A conditional implementation includes $(B,z,\tau)$ in the value, policy, and normalizer inputs. |
|
|
| At inference, sample a parent and successive admissible actions until stop. Mask any action that exceeds the remaining budget. Every selected edge is retained, including reagent identifiers and reaction metadata in the molecular application. Sampling therefore returns $(y,\xi)$, with the full route available for replay. Algorithm~\ref{alg:training} gives the learning procedure, and Algorithm~\ref{alg:inference} gives inference. |
|
|
| \begin{theorem}[Endpoint and route error] |
| \label{thm:error} |
| Suppose every complete path has at most $H$ edges, $v_\phi(s_0)=0$, and the full feasible support satisfies |
| \begin{align} |
| |\Delta_{\theta,\phi}(\xi)|&\leq\eta,\label{eq:balance_bound}\\ |
| \left|v_\phi(s)-\lse_{e:u\to s}[v_\phi(u)-c(e)/\tau]\right|&\leq\epsilon.\label{eq:route_bound} |
| \end{align} |
| Then the endpoint and joint path distributions satisfy |
| \begin{align} |
| \TV(P_{\theta,Y},\pi_{B,z})&\leq\tanh\eta,\label{eq:tv}\\ |
| \KL(P_\theta\|P^*_{B,z,\tau})&\leq2\eta+2H\epsilon.\label{eq:joint_bound} |
| \end{align} |
| At exact balance, the conditional objective gap is at most $2\tau H\epsilon$. |
| \end{theorem} |
| The endpoint bound depends on forward balance error. The joint bound additionally includes accumulated route-recursion error. The proof is given in \cref{app:errorproof}. |
|
|
| \subsection{Exact Forward Construction} |
| \label{sec:exact_forward} |
| On a stored graph, the joint optimum can be sampled through a forward policy without enumerating complete paths. Start with terminal masses $D(t_y)=\pi_{B,z}(y)$ and propagate them backward, |
| \begin{equation} |
| D(u)=\sum_{e:u\to s}q^*(e\mid s)D(s). |
| \label{eq:demand} |
| \end{equation} |
| The mass at $u$ is the probability of visiting that node when first drawing an endpoint and then reversing a conditional route. Reversing the corresponding edge masses gives |
| \begin{equation} |
| p^*(e\mid u)=\frac{q^*(e\mid s)D(s)}{D(u)}. |
| \label{eq:exact_forward} |
| \end{equation} |
| The outgoing probabilities sum to one by \cref{eq:demand}. Their product along a complete path is $D(t_y)Q^*(\xi\mid y)$ because the intermediate node masses cancel and $D(s_0)=1$. This equals the joint law in \cref{eq:joint}. |
|
|
| Parent selection follows from the same computation. For a root action that selects parent $x$, its probability is |
| \begin{equation} |
| p^*(x\mid s_0)=\sum_{y\in\Y_B}\pi_{B,z}(y) |
| Q^*(X_0=x\mid y). |
| \label{eq:parent} |
| \end{equation} |
| Thus, the starting-state distribution depends on the target outcome weights and the costs of their realizations. Parent-selection preferences can be incorporated through additional root-edge costs. The endpoint marginal remains fixed after conditional normalization. |
|
|
| Prefix evaluation and backward mass propagation each require one pass over the graph. Their time and storage costs are $O(|V|+|E|)$ after expansion. Exact reference distributions for the discrete experiments and small reaction inventories are computed by these passes. The learned forward policy approximates these path probabilities through sampled trajectory constraints, with network parameters shared across states. Both procedures retain the sequence of selected operations. |
|
|
| \subsection{Route Multiplicity and Temperature} |
| \label{sec:route_effects} |
| Consider two outcomes with equal weights. Suppose $y_1$ has $m$ feasible routes of costs $c_1,\ldots,c_m$ and $y_2$ has one route of cost $d$. The prescribed joint law satisfies |
| \begin{equation} |
| \begin{aligned} |
| P^*(\xi_i)&=\frac12\frac{e^{-c_i/\tau}}{\sum_{j=1}^{m}e^{-c_j/\tau}},\\ |
| P^*(\xi_{y_2})&=\frac12. |
| \end{aligned} |
| \label{eq:multiplicity_example} |
| \end{equation} |
| Adding realizations of $y_1$ redistributes its conditional mass while preserving $P^*(Y=y_1)=1/2$. This behavior also holds when routes begin at different parents or terminate after different numbers of operations. Canonical endpoint merging defines the set over which the outcome normalization is computed. |
|
|
| For a fixed endpoint, differentiating its conditional partition gives |
| \begin{equation} |
| \frac{\partial}{\partial\tau}\E_{Q^*(\cdot\mid y)}[C] |
| =\frac{\operatorname{Var}_{Q^*(\cdot\mid y)}[C]}{\tau^2}\geq0. |
| \label{eq:temperature} |
| \end{equation} |
| Increasing the route temperature therefore increases mean execution cost whenever feasible routes have different costs. At low temperature, sampling concentrates on economical realizations. At high temperature, the conditional distribution approaches the uniform distribution over feasible routes to that endpoint. The target weights $r_z$ determine how frequently the outcome itself is selected throughout this sweep. |
|
|
| The declared action representation determines conditional route entropy. Duplicate descriptions of an identical operation should be consolidated before graph construction. Distinct reagent choices or physically different procedures can remain separate when their execution records matter. Outcome probabilities are invariant to these route-level choices under exact balance. Conditional route probabilities use the resulting execution set. |
|
|
| \subsection{Reaction-Budgeted Molecular Design} |
| For the molecular application, $\mathcal C$ contains canonical structures with supplier identifiers. A root action selects one complete parent. Each subsequent action applies an allowed reaction template with an eligible reagent, and consumes one unit of the additional-reaction budget. A parent already in the catalog can be returned through the zero-reaction path. Its supplier construction metadata are retained separately. |
|
|
| The state contains molecular structure, reaction count, and any resource needed to determine future admissibility. Stop edges merge identical canonical outcomes across parents and route lengths. Edge records contain the input structure, template, reagent, product, and cost. Re-execution of those records tests consistency with the declared reaction rules. Laboratory synthesis establishes reaction success and yield. |
|
|
| Affinity and permeability preferences are specified through separately evaluated property scores in $r_z$. The main molecular protocol uses target-specific docking, held-out permeability prediction, and molecular quality objectives. These scores specify computational selection criteria. Experimental binding and permeability measurements are reported separately when available. The same mathematical construction applies to other finite executable systems by replacing the parent set, actions, resources, and outcome weights. |
|
|
| \subsection{Support Coverage} |
| \label{sec:coverage} |
| A finite inventory defines the computational domain for graph construction. Reaction expansion starts from every selected parent, applies eligible templates and reagents, and records each canonical product at its consumed budget. The same product can appear at several budget states because its remaining actions depend on the resources already used. Those states remain distinct until their stop edges connect to a common terminal outcome. |
|
|
| For a stored path set $\widehat\T_B\subseteq\T_B$, let $\widehat\Y_B$ be its reachable outcomes. The exact endpoint law on this graph is |
| \begin{equation} |
| \widehat\pi_{B,z}(y)=\frac{r_z(y)\,\mathbf 1\{y\in\widehat\Y_B\}} |
| {\sum_{y'\in\widehat\Y_B}r_z(y')}. |
| \label{eq:restricted_target} |
| \end{equation} |
| This is the full target conditioned on the retained outcomes. Its total-variation distance from the full target is the excluded probability mass $\pi_{B,z}(\Y_B\setminus\widehat\Y_B)$. Retaining at least one route to every outcome preserves the endpoint target. Additional routes then improve the domain over which conditional execution cost is optimized. The derivation is given in \cref{app:proofs}. |
|
|
| The computational record therefore includes the parent subset, reagent inventory, template versions, budget, and graph size. An expansion limit is checked during construction. A completed graph defines the support used for training and sampling. Catalog-size comparisons report the cost of creating this graph together with the subsequent learning and generation costs. The distinction between graph expansion and repeated sampling is relevant when many property preferences are evaluated over the same inventory. |
|
|
| \section{Experiments and Results} |
| \label{sec:results} |
| The evaluation progresses from exact endpoint distributions to molecular generation under a fixed reaction budget. Each stage tests a different requirement of the joint law. The discrete experiments measure distributional error and route cost. Molecular experiments then measure executable support, target-specific scores, permeability prediction, diversity, and the cost of increasing catalog size. |
|
|
| \subsection{Endpoint Accuracy and Execution Cost} |
| \label{sec:discrete} |
| Finite systems permit direct comparison with $P^*$ and $\pi_{B,z}$. The benchmark includes a lattice with costly passages, an edit system with multiple action orders, and layered graphs with controlled route multiplicity. Training and test graphs use disjoint initial states or graph instances, with the instance as the unit of independence. Each task uses five seeds and 10,000 generated paths per seed. The target marginals and conditional optimum for each stored graph are computed by exact dynamic programming. |
|
|
| Comparisons include uniform executable sampling, reward tilting of the same reference process, trajectory balance with uniform backward probabilities, learned backward trajectory balance, and \method{}. All sequential methods use the same graph, rewards, and path budget. The principal metrics are endpoint total variation, conditional free-energy gap, mean execution cost, and the fraction of sampled paths within budget (\cref{tab:discrete}). |
| \begin{table}[H] |
| \centering\tablefont |
| \caption{\textbf{Exact discrete generation.} Mean across held-out graph instances and five seeds. TV is endpoint total variation. Gap is conditional free-energy excess in cost units. Feas. is budget feasibility.} |
| \label{tab:discrete} |
| \begin{tabular}{lcccc} |
| \toprule |
| Method & TV $\downarrow$ & Gap $\downarrow$ & Cost $\downarrow$ & Feas. $\uparrow$\\ |
| \midrule |
| Uniform executable & \pending&\pending&\pending&\pending\\ |
| Reference tilt & \pending&\pending&\pending&\pending\\ |
| TB, uniform backward & \pending&\pending&\pending&\pending\\ |
| TB, learned backward & \pending&\pending&\pending&\pending\\ |
| SubTB & \pending&\pending&\pending&\pending\\ |
| \method{} & \pending&\pending&\pending&\pending\\ |
| Exact joint law & \pending&\pending&\pending&\pending\\ |
| \bottomrule |
| \end{tabular} |
| \end{table} |
| For route multiplicity, retain two equally weighted outcomes and duplicate admissible realizations of one outcome by factors of 2, 4, 8, and 16. Endpoint probability drift measures the change from the original graph. A separate temperature sweep measures the conditional route-cost distribution. Together, these measurements test the separation between endpoint weights and route preferences in \cref{eq:joint} (\cref{fig:diagnostics}). |
| \begin{figure}[t] |
| \centering |
| \includegraphics[width=\columnwidth]{figures/diagnostics.pdf} |
| \caption{\textbf{Distribution and route diagnostics.} Endpoint probability against route multiplicity, conditional mean cost against route temperature, and endpoint TV against training steps. Curves report the mean and standard error across five seeds.} |
| \label{fig:diagnostics} |
| \end{figure} |
|
|
| \subsection{Generation Within a Reaction Budget} |
| \label{sec:molecular} |
| The molecular benchmark fixes a parent catalog, reagent inventory, and reaction set before generation. Parent scaffolds define disjoint training, validation, and test partitions. For $B\in\{0,1,2\}$, the feasible products include each parent and all products obtainable through the allowed additional reactions. Canonical structure determines endpoint identity. Parent identifiers and original supplier reaction metadata remain associated with each returned path. |
|
|
| SynFlowNet, RxnFlow, and \method{} receive the same parents, reagents, templates, and property scores \citep{synflownet,rxnflow}. A SyntheMol search comparator uses the same action inventory \citep{synthemol}. The structure-conditioned CGFlow comparison additionally uses pocket coordinates \citep{cgflow}. For an endpoint generator followed by route retrieval, the evaluation includes retrieval time and unresolved outputs. Comparisons report each method's native settings alongside the matched-inventory results. |
|
|
| The primary construction measurements are the proportion of generated structures with a replayable route, budget compliance, additional reaction count, and the number of unique feasible outcomes per 10,000 oracle calls (\cref{tab:routes}). Reaction replay uses the recorded substrate, reagent, and template. The supplier parent ID is checked against the fixed catalog manifest. |
| \begin{table}[H] |
| \centering\tablefont |
| \caption{\textbf{Molecular construction, $B=2$.} Replay and budget compliance are fractions of all requested candidates. Steps count reactions after parent selection. Unique is the number of distinct feasible products per 10,000 oracle calls. Five seeds per target.} |
| \label{tab:routes} |
| \begin{tabular}{lcccc} |
| \toprule |
| Method & Replay $\uparrow$ & Budget $\uparrow$ & Steps $\downarrow$ & Unique $\uparrow$\\ |
| \midrule |
| Catalog search & \pending&\pending&\pending&\pending\\ |
| SyntheMol & \pending&\pending&\pending&\pending\\ |
| SynFlowNet & \pending&\pending&\pending&\pending\\ |
| RxnFlow & \pending&\pending&\pending&\pending\\ |
| CGFlow & \pending&\pending&\pending&\pending\\ |
| \method{} & \pending&\pending&\pending&\pending\\ |
| \bottomrule |
| \end{tabular} |
| \end{table} |
| The zero- and one-reaction comparisons appear in \cref{tab:budgets}. Errors are grouped by parent resolution, reaction execution, canonical product identity, and budget accounting. This breakdown identifies which operation prevents a candidate from entering the feasible set. |
|
|
| \subsection{Target Scores and Permeability Control} |
| \label{sec:properties} |
| After establishing construction feasibility, the property evaluation measures the quality and diversity of feasible outcomes. LIT-PCBA provides a set of target-specific screening tasks \citep{litpcba}. The docking protocol fixes receptor preparation, grid coordinates, software version, and random seeds across methods \citep{vina,vinaupdate}. Reported docking energies retain their units of kcal/mol. Caco-2 permeability prediction uses the Wang dataset through Therapeutics Data Commons with scaffold-disjoint fitting and evaluation \citep{tdc,caco2}. The predictor's held-out error accompanies generation results. |
|
|
| Property weights are swept over a fixed grid using the same number of oracle calls for every setting. The reported measurements include docking score, predicted Caco-2 permeability, QED, internal diversity, and dominated hypervolume of normalized utilities (\cref{tab:properties}). Hypervolume uses the affinity and permeability utilities with reference point zero after clipping to $[0,1]^2$. The top-100 subset is selected by the declared scalar reward. Feasibility rates are computed over all requested candidates, and property summaries use the feasible candidates. Target-level scores and seed-level variation are retained in the supplementary tables. |
| \begin{table}[H] |
| \centering\tablefont |
| \caption{\textbf{Multi-objective molecular generation.} Top-100 feasible candidates per target and seed at $B=2$. Dock is mean docking energy. Perm. is predicted $\log_{10}$ Caco-2 permeability in cm/s. Div. is mean pairwise Tanimoto distance. HV is normalized hypervolume.} |
| \label{tab:properties} |
| \begin{tabular}{lccccc} |
| \toprule |
| Method & Dock $\downarrow$ & Perm. $\uparrow$ & QED $\uparrow$ & Div. $\uparrow$ & HV $\uparrow$\\ |
| \midrule |
| Catalog search &\pending&\pending&\pending&\pending&\pending\\ |
| SyntheMol &\pending&\pending&\pending&\pending&\pending\\ |
| SynFlowNet &\pending&\pending&\pending&\pending&\pending\\ |
| RxnFlow &\pending&\pending&\pending&\pending&\pending\\ |
| CGFlow &\pending&\pending&\pending&\pending&\pending\\ |
| \method{} &\pending&\pending&\pending&\pending&\pending\\ |
| \bottomrule |
| \end{tabular} |
| \end{table} |
| Molecular optimization also depends on scoring efficiency. A complementary comparison uses the Practical Molecular Optimization protocol to report top-10 reward area under the oracle-call curve \citep{pmo}. The same feasibility restriction is applied to every scored molecule. Additional predictor and target results are specified in \cref{app:properties}. |
|
|
| \subsection{Sensitivity to Property Preferences} |
| \label{sec:preferences} |
| Changing the property weights modifies the desired outcome distribution while preserving the action inventory. The preference experiment fixes each parent and reagent set, then varies the affinity weight over $\{0,0.25,0.5,0.75,1\}$ with the remaining weight assigned to permeability. The concentration parameter uses $\beta\in\{1,2,5\}$. For each setting, the evaluation records the achieved utilities, the number of distinct feasible outcomes, and the spread of predicted properties among sampled candidates (\cref{fig:preferences}). |
|
|
| A second comparison changes route temperature over $\tau\in\{0.1,0.3,0.7,1,2\}$ at fixed outcome weights. Exact graphs permit direct evaluation of endpoint probability drift and the conditional cost derivative in \cref{eq:temperature}. Molecular runs report the distribution of reaction counts and the frequencies of different parent choices. Together, these experiments quantify how outcome preferences affect candidate selection and how execution preferences affect the routes used to produce those candidates. |
|
|
| For repeated preference queries, the backward prefix values can be reused at a fixed graph and temperature. The exact computation updates the terminal demands and propagates their masses through \cref{eq:demand}. Neural runs initialize the forward network from the preceding preference setting and record the additional updates required to reach the selected validation tolerance. Runtime includes this adaptation and the property evaluations required by the new query. |
| \begin{figure}[t] |
| \centering |
| \includegraphics[width=\columnwidth]{figures/preferences.pdf} |
| \caption{\textbf{Property preferences and catalog scaling.} The upper plot reports mean affinity and permeability utilities for the preference sweep at each reaction budget. The lower plot separates graph construction, forward-policy training, and generation time as the parent count increases. Error bars summarize variation across seeds.} |
| \label{fig:preferences} |
| \end{figure} |
|
|
| \subsection{Component Contributions} |
| The ablation study tests the conditional route objective, endpoint merging, and backward normalization under identical graph and oracle budgets. Uniform backward probabilities isolate the effect of cost-dependent routes. Setting $C=0$ removes the cost preference. Removing canonical terminal merging assigns reward separately to route endpoints with duplicate outcomes. Replacing the normalized backward law with an unnormalized learned weight tests the role of conditional normalization (\cref{tab:ablation}). |
| \begin{table}[H] |
| \centering\tablefont |
| \caption{\textbf{Ablations.} Endpoint TV and conditional gap use exact graphs. Replay uses molecular graphs at $B=2$. Runtime is measured under a fixed sampling count. Means and standard errors are recorded over five seeds.} |
| \label{tab:ablation} |
| \begin{tabular}{lcccc} |
| \toprule |
| Variant & TV $\downarrow$ & Gap $\downarrow$ & Replay $\uparrow$ & Time $\downarrow$\\ |
| \midrule |
| \method{} &\pending&\pending&\pending&\pending\\ |
| Uniform backward &\pending&\pending&\pending&\pending\\ |
| Zero route cost &\pending&\pending&\pending&\pending\\ |
| Duplicate endpoints &\pending&\pending&\pending&\pending\\ |
| Unnormalized backward &\pending&\pending&\pending&\pending\\ |
| Exact prefix values &\pending&\pending&\pending&\pending\\ |
| \bottomrule |
| \end{tabular} |
| \end{table} |
|
|
| \subsection{Support Size and Computational Cost} |
| Catalog scaling is measured by increasing the number of parents while keeping the reaction inventory and budget fixed. Construction time, stored states and edges, peak memory, training time, and candidate generation time are reported separately. The exact implementation requires linear work in the stored graph for each forward or backward dynamic-programming pass. Expanding the reaction graph can require substantially more work than a sampling pass, especially when the budget increases. |
|
|
| The scaling study uses parent subsets of $10^2$, $10^3$, and $10^4$ compounds, with the realized graph sizes recorded for each subset. A second sweep fixes the parent set and increases $B$ from zero to two (\cref{tab:scaling}). Reward changes are evaluated after graph construction to measure the cost of new property preferences. For learned policies, the experiment records adaptation updates and total oracle calls along with the final sampling time. |
| \begin{table}[H] |
| \centering\tablefont |
| \caption{\textbf{Catalog scaling at $B=2$.} Graph construction and training use seconds, memory uses GB, and sampling uses seconds per 1,000 paths. Parent counts refer to the supplied subset.} |
| \label{tab:scaling} |
| \begin{tabular}{rccccc} |
| \toprule |
| Parents & Edges & Build & Train & Memory & Sample\\ |
| \midrule |
| $10^2$ &\pending&\pending&\pending&\pending&\pending\\ |
| $10^3$ &\pending&\pending&\pending&\pending&\pending\\ |
| $10^4$ &\pending&\pending&\pending&\pending&\pending\\ |
| \bottomrule |
| \end{tabular} |
| \end{table} |
|
|
|
|
| \section{Discussion} |
| We introduced \method{} to learn a desired distribution over feasible outcomes together with budgeted executable paths. The conditional entropy objective gives an explicit route law for each outcome, and the backward free-energy recursion permits computation through local graph operations. With normalized backward probabilities, forward trajectory balance determines the endpoint marginal. The derived error bounds distinguish imperfect endpoint alignment from imperfect route-cost optimization. This separation permits changing route preferences while retaining the same target outcome weights. |
|
|
| The principal computational limitation is construction and coverage of the executable graph. For a restricted parent and reagent inventory, the exact implementation provides a reference distribution and complete route records. Larger inventories require broader action discovery, reliable predecessor retrieval, and evaluation of support coverage. Molecular conclusions also depend on reaction scope and property-prediction accuracy. Prospective synthesis, binding measurements, and permeability assays are needed to connect the generated routes and computational scores to experimental performance. The proposed benchmarks quantify distributional accuracy, execution cost, and molecular selection under those stated inputs. |
|
|
| \label{mainend} |
| \clearpage |
| \section*{Impact Statement} |
| This work concerns computational generation under explicit construction rules. Molecular applications require experimental assessment of synthesis, binding, permeability, and safety. Catalog access and redistribution follow the corresponding supplier agreements. |
| \bibliography{references} |
| \bibliographystyle{icml2026} |
| \clearpage |
| \appendix |
| \onecolumn |
| \raggedbottom |
| \section{Proofs} |
| \label{app:proofs} |
| \subsection{Conditional Optimum} |
| For a fixed outcome $y$, let $\T_B(y)=\{\xi\in\T_B:y(\xi)=y\}$. For any conditional law $Q(\cdot\mid y)$, |
| \begin{align} |
| \E_Q[C]+\tau\E_Q[\log Q] |
| &=\tau\sum_{\xi\in\T_B(y)}Q(\xi\mid y) |
| \log\frac{Q(\xi\mid y)}{e^{-C(\xi)/\tau}/A_{B,\tau}(y)} |
| -\tau\log A_{B,\tau}(y)\\ |
| &=\tau\KL(Q(\cdot\mid y)\|Q^*(\cdot\mid y)) |
| -\tau\log A_{B,\tau}(y). |
| \label{eq:conditional_proof} |
| \end{align} |
| The conditional cost plus negative entropy equals a relative entropy and a constant. Relative entropy is nonnegative and vanishes only at $Q=Q^*$. Averaging over the fixed endpoint marginal gives \cref{eq:gap} and proves \cref{prop:optimum}. All sums are finite, and the endpoint weights and conditional exponential weights are positive. |
|
|
| Let $C_{\min}(y)=\min_{\xi\in\T_B(y)}C(\xi)$. Dividing each conditional weight by $e^{-C_{\min}(y)/\tau}$ gives |
| \begin{equation} |
| Q^*(\xi\mid y)= |
| \frac{e^{-[C(\xi)-C_{\min}(y)]/\tau}} |
| {\sum_{\xi'\in\T_B(y)}e^{-[C(\xi')-C_{\min}(y)]/\tau}}. |
| \end{equation} |
| As the route temperature decreases to zero, positive-cost gaps receive vanishing mass. The limiting conditional law is uniform over the minimum-cost routes. The endpoint marginal remains $\pi_{B,z}$ because each conditional law sums to one. |
|
|
| \subsection{Backward Normalization and Prefix Recursion} |
| The prefix partition at a nonroot state is the sum over the disjoint possible final edges into that state. This gives \cref{eq:prefix}. For a complete path $\xi=(e_1,\ldots,e_n)$ with states $(s_0,\ldots,t_y)$, |
| \begin{equation} |
| \prod_{i=1}^{n}q^*(e_i\mid s_i) |
| =\exp\left\{\sum_{i=1}^{n}[v(s_{i-1})-c(e_i)/\tau-v(s_i)]\right\} |
| =\frac{e^{-C(\xi)/\tau}}{A_{B,\tau}(y)}. |
| \label{eq:telescoping} |
| \end{equation} |
| The prefix terms cancel along the path, leaving its total cost and terminal partition. Local backward normalization also establishes a probability distribution for any finite learned values. Every backward traversal reaches the root because the graph is finite, acyclic, and root-reachable. |
|
|
| For exact forward balance, $e^{\zeta_\theta}P_\theta(\xi)=r_z(y)Q_\phi(\xi\mid y)$. Summing over the paths to $y$ and then over outcomes gives |
| \begin{equation} |
| P_{\theta,Y}(y)=e^{-\zeta_\theta}r_z(y),\qquad |
| e^{\zeta_\theta}=\sum_{y\in\Y_B}r_z(y)=Z_{B,z}. |
| \end{equation} |
| Thus the endpoint law is fixed by the target weights for every normalized backward policy. The role of the route objective is to determine its conditional execution distribution. |
|
|
| \subsection{Endpoint and Joint Error Bounds} |
| \label{app:errorproof} |
| Write $\widetilde P(\xi)=\pi_{B,z}(y)Q_\phi(\xi\mid y)$ and let $d(\xi)=\Delta_{\theta,\phi}(\xi)$. Normalization implies |
| \begin{equation} |
| P_\theta(\xi)= |
| \frac{\widetilde P(\xi)e^{d(\xi)}}{\E_{\widetilde P}[e^d]}. |
| \label{eq:residual_tilt} |
| \end{equation} |
| When $|d|\leq\eta$, the likelihood ratio $P_\theta/\widetilde P$ lies in $[e^{-2\eta},e^{2\eta}]$. A normalized likelihood ratio in $[a,b]$ has total variation at most $(b-1)(1-a)/(b-a)$. Substitution gives $\TV(P_\theta,\widetilde P)\leq\tanh\eta$. Marginalization cannot increase total variation, and $\widetilde P_Y=\pi_{B,z}$, proving \cref{eq:tv}. The same ratio bound gives $\KL(P_\theta\|\widetilde P)\leq2\eta$. |
|
|
| Define the prefix residual |
| \begin{equation} |
| \delta(s)=v_\phi(s)-\lse_{e:u\to s}\{v_\phi(u)-c(e)/\tau\}. |
| \end{equation} |
| The log-sum-exp function is 1-Lipschitz in the maximum norm. Induction over graph depth therefore gives $|v_\phi(s)-v(s)|\leq h(s)\epsilon$, where $h(s)$ is the maximum number of edges in a root-to-$s$ path. For a complete route, |
| \begin{equation} |
| \log Q_\phi(\xi\mid y)=-C(\xi)/\tau-v_\phi(t_y)+\sum_{i=1}^{n}\delta(s_i). |
| \end{equation} |
| Consequently, $|\log[Q_\phi(\xi\mid y)/Q^*(\xi\mid y)]|\leq2H\epsilon$. Adding this bound to the forward likelihood-ratio bound gives |
| \begin{equation} |
| \KL(P_\theta\|P^*) |
| =\KL(P_\theta\|\widetilde P) |
| +\E_{P_\theta}\log\frac{Q_\phi(\Xi\mid Y)}{Q^*(\Xi\mid Y)} |
| \leq2\eta+2H\epsilon. |
| \end{equation} |
| At exact balance, the endpoint marginal is $\pi_{B,z}$, so \cref{eq:gap} yields the conditional objective bound. These conclusions prove \cref{thm:error}. |
|
|
| \subsection{Restricted Support} |
| Let $\widehat\T_B\subseteq\T_B$ be a stored or discovered subset of executable paths, with endpoint set $\widehat\Y_B$. All preceding identities hold on this declared graph. If every full-support endpoint has at least one retained route, exact balance recovers the same endpoint law, while conditional route optimization uses the retained routes. If endpoints are missing, the normalized endpoint target changes. Its total variation from the full target equals |
| \begin{equation} |
| \TV\bigl(\pi_{B,z}(\cdot\mid\widehat\Y_B),\pi_{B,z}\bigr) |
| =\pi_{B,z}(\Y_B\setminus\widehat\Y_B). |
| \end{equation} |
| The missing target mass quantifies the effect of incomplete endpoint coverage. This quantity is computable in the exact benchmarks. Large-catalog evaluation records the supplied parent and reagent inventory together with expansion limits. |
|
|
| \section{Algorithms} |
| \label{app:algorithms} |
| \begin{algorithm}[H] |
| \caption{DooABLe training on an executable graph} |
| \label{alg:training} |
| \begin{algorithmic}[1] |
| \REQUIRE Root-reachable DAG $G_B$, positive weights $r_z$, temperature $\tau$, exploration rate $\alpha$ |
| \STATE Merge equivalent outcomes into terminal nodes $t_y$ \algcomment{one target weight per outcome} |
| \STATE Initialize $v_\phi$, $p_\theta$, and $\zeta_\theta$ with $v_\phi(s_0)=0$ |
| \FOR{each optimization step} |
| \STATE Sample nonroot states $S\sim\nu$ |
| \FOR{$s\in S$} |
| \STATE $\widehat v(s)\gets\lse_{e:u\to s}[v_\phi(u)-c(e)/\tau]$ \algcomment{complete incoming edge set} |
| \ENDFOR |
| \STATE Update $\phi$ on $\sum_{s\in S}[v_\phi(s)-\operatorname{sg}(\widehat v(s))]^2$ |
| \STATE Normalize $q_\phi(e\mid s)\propto\exp[v_\phi(u)-c(e)/\tau]$ on incoming edges |
| \STATE Collect paths using $(1-\alpha)p_\theta+\alpha p_{\rm uniform}$ \algcomment{mask inadmissible actions} |
| \FOR{each collected path $\xi$ with outcome $y$} |
| \STATE $\ell_P\gets\sum_{e:u\to s\in\xi}\log p_\theta(e\mid u)$ |
| \STATE $\ell_Q\gets\sum_{e:u\to s\in\xi}\log q_\phi(e\mid s)$ |
| \STATE $\Delta(\xi)\gets\zeta_\theta+\ell_P-\log r_z(y)-\ell_Q$ |
| \ENDFOR |
| \STATE Update $\theta,\zeta_\theta$ on mean $\Delta(\xi)^2$ \algcomment{hold backward probabilities fixed} |
| \ENDFOR |
| \STATE \textbf{return} Forward sampler $p_\theta$ and route value $v_\phi$ |
| \end{algorithmic} |
| \end{algorithm} |
|
|
| \begin{algorithm}[H] |
| \caption{DooABLe generation with route recovery} |
| \label{alg:inference} |
| \begin{algorithmic}[1] |
| \REQUIRE Forward policy $p_\theta$, executor $F$, starting set $\mathcal C$, budget $B$ |
| \STATE Sample parent $x_0\in\mathcal C$ from the root policy |
| \STATE $x\gets x_0$, $b\gets B$, $\xi\gets[\operatorname{select}(x_0)]$ |
| \WHILE{stop has not been selected} |
| \STATE Enumerate admissible actions $a$ with resource use $b(a)\leq b$ |
| \STATE Include stop if the current state is an eligible outcome |
| \STATE Sample $a$ from the masked policy $p_\theta(\cdot\mid x,b)$ |
| \IF{$a=\operatorname{stop}$} |
| \STATE Append stop to $\xi$ and terminate |
| \ELSE |
| \STATE $x'\gets F(x,a)$ \algcomment{execute the recorded operation} |
| \STATE Append $(x,a,x')$ to $\xi$ |
| \STATE $x\gets x'$, $b\gets b-b(a)$ |
| \ENDIF |
| \ENDWHILE |
| \STATE \textbf{return} Canonical outcome $y(x)$ and complete execution path $\xi$ |
| \end{algorithmic} |
| \end{algorithm} |
|
|
| \begin{algorithm}[H] |
| \caption{Exact reference computation on a stored DAG} |
| \label{alg:exact} |
| \begin{algorithmic}[1] |
| \REQUIRE Graph $G_B$, log rewards $\log r_z$, temperature $\tau$ |
| \STATE Compute prefix values $v$ in topological order using \cref{eq:bellman} |
| \STATE Compute normalized backward probabilities $q^*$ using \cref{eq:qstar} |
| \STATE Set terminal mass $D(t_y)\gets r_z(y)/\sum_{y'}r_z(y')$ |
| \STATE Initialize all nonterminal masses to zero |
| \FOR{nodes $s$ in reverse topological order} |
| \FOR{incoming edges $e:u\to s$} |
| \STATE $D(u)\gets D(u)+q^*(e\mid s)D(s)$ \algcomment{accumulate terminal demand} |
| \ENDFOR |
| \ENDFOR |
| \STATE Set $p^*(e\mid u)\gets q^*(e\mid s)D(s)/D(u)$ |
| \STATE \textbf{return} Exact forward and backward policies |
| \end{algorithmic} |
| \end{algorithm} |
| The implementation stores logarithmic prefix values and logarithmic demand during normalization. These passes require $O(|V|+|E|)$ operations and $O(|V|+|E|)$ storage after graph construction. |
|
|
| \section{Experimental Specification} |
| \label{app:experimental} |
| \subsection{Discrete Systems} |
| The lattice benchmark uses grid width 8 and step budgets 6, 8, and 12. Four directional moves are permitted inside the grid. Visiting the designated wall column costs an additional 0.7 units except at its opening. All reachable coordinates are eligible outcomes, and visits at different times connect to the same terminal node. The edit benchmark uses fixed-length categorical strings and a finite set of substitutions, with terminal identity determined by the final string. For multiplicity tests, every duplicated route retains its edge costs and final outcome. Distinct action identifiers permit separate route accounting. |
|
|
| Endpoint TV is $\frac12\sum_y|\widehat p(y)-\pi(y)|$. Exact forward marginals are used for stored graphs, with empirical frequencies also reported at fixed sample counts. For an imperfect endpoint marginal, the conditional free-energy excess is $\tau\sum_y P_{\theta,Y}(y)\KL(P_\theta(\cdot\mid y)\|Q^*(\cdot\mid y))$. This equals $\tau$ times joint KL minus endpoint KL. At exact endpoint alignment it agrees with \cref{eq:gap}. Endpoint drift is the absolute change in outcome probability after duplicating routes. Training comparisons retain the number of optimizer updates, sampled paths, scored outcomes, and elapsed time. |
|
|
| \subsection{Molecular Data and Splits} |
| Parent records contain canonical isomeric SMILES, supplier identifiers, and source provenance. Enamine identifiers of the form \texttt{m\_22\_654222\_29794186} are parsed into chemistry class, parent reaction ID, and parent reagent IDs. These fields describe the purchased parent. The additional-reaction budget starts after parent selection. |
|
|
| Molecules are canonicalized before assigning Bemis--Murcko scaffold groups \citep{scaffolds}. Duplicate structures share an endpoint, with supplier aliases retained in metadata. Scaffold groups are assigned to training, validation, and test sets before model fitting. All generated descendants retain parent provenance so that parent-held-out analyses can be reproduced. Products reachable from multiple parent partitions are assigned to a common partition or removed from cross-partition evaluation. The full inventory manifest includes reaction-template versions, reagent structures, stock filters, and expansion limits. |
|
|
| The public integration dataset contains twelve named parent compounds, thirteen reagents, and four directional coupling templates. It supports amide and ester construction over zero, one, and two additional reactions. This set is used to test parsing, graph expansion, learning, sampling, and reaction replay. The Enamine comparisons use the supplied licensed parent inventory. |
|
|
| \subsection{Property Prediction and Molecular Scores} |
| \label{app:properties} |
| The public property workflow fits separate predictors for Caco-2 transport and BACE inhibition. Caco-2 labels are $\log_{10}$ permeability in cm/s. BACE labels are pIC$_{50}$ measurements from MoleculeNet \citep{moleculenet}. Each dataset is canonicalized, repeated structures are averaged, and scaffold groups are partitioned before fitting. The reference predictors use 1,024-bit Morgan fingerprints and 256-tree extremely randomized tree ensembles \citep{morgan,extratrees}. Mean absolute error and $R^2$ are computed on the held-out scaffold partition. Generated-molecule scores are predictions from the fitted training models. |
|
|
| For target-pocket evaluation, docking uses a fixed receptor and grid for each LIT-PCBA target. All methods share the same preparation and scoring calls. Affinity-like docking energies and predicted permeability remain separate columns. The utility transformation, clipping bounds, property weights, and concentration $\beta$ are fixed using the validation split. For public integration, BACE utility is $\operatorname{clip}[(\widehat{\mathrm{pIC}}_{50}-4)/5,0,1]$, and permeability utility is $\operatorname{clip}[(\widehat\ell+7)/3,0,1]$. These explicit transforms permit reproduction of the generation distribution. |
|
|
| Internal diversity is mean pairwise Tanimoto distance over Morgan fingerprints. QED uses the published weighted desirability calculation \citep{qed}. Top-$k$ summaries report the actual number of feasible candidates when fewer than $k$ are available. Oracle-call counts include every unique scored structure, and a cache avoids charging the same score repeatedly. Hypervolume is computed over nondominated, normalized utility vectors with reference point zero. |
|
|
| \begin{table}[H] |
| \centering\small |
| \caption{\textbf{Additional reaction budgets.} Matched parent inventory and five seeds. Scores are mean top-100 feasible candidate values. Perm. is predicted Caco-2 transport.} |
| \label{tab:budgets} |
| \begin{tabular}{lcccccc} |
| \toprule |
| Method & $B$ & Replay $\uparrow$ & Steps $\downarrow$ & Dock $\downarrow$ & Perm. $\uparrow$ & Unique $\uparrow$\\ |
| \midrule |
| Catalog search & 0 &\pending&\pending&\pending&\pending&\pending\\ |
| RxnFlow & 1 &\pending&\pending&\pending&\pending&\pending\\ |
| \method{} & 1 &\pending&\pending&\pending&\pending&\pending\\ |
| RxnFlow & 2 &\pending&\pending&\pending&\pending&\pending\\ |
| \method{} & 2 &\pending&\pending&\pending&\pending&\pending\\ |
| \bottomrule |
| \end{tabular} |
| \end{table} |
|
|
| Target-specific docking summaries and permeability-predictor validation are given in \cref{tab:targets,tab:predictors}. Each target uses the same docking grid across compared methods. Seed-level measurements are retained in the source CSV files. |
| \begin{table}[H] |
| \centering\small |
| \caption{\textbf{Target-specific docking.} Mean top-100 feasible docking energy in kcal/mol at $B=2$. Lower is better. Each entry records the mean and standard error across five seeds.} |
| \label{tab:targets} |
| \begin{tabular}{lccccc} |
| \toprule |
| Target & SyntheMol & SynFlowNet & RxnFlow & CGFlow & \method{}\\ |
| \midrule |
| ADRB2 &\pending&\pending&\pending&\pending&\pending\\ |
| ALDH1 &\pending&\pending&\pending&\pending&\pending\\ |
| ESR1 agonism &\pending&\pending&\pending&\pending&\pending\\ |
| ESR1 antagonism &\pending&\pending&\pending&\pending&\pending\\ |
| FEN1 &\pending&\pending&\pending&\pending&\pending\\ |
| GBA &\pending&\pending&\pending&\pending&\pending\\ |
| IDH1 &\pending&\pending&\pending&\pending&\pending\\ |
| KAT2A &\pending&\pending&\pending&\pending&\pending\\ |
| MAPK1 &\pending&\pending&\pending&\pending&\pending\\ |
| MTORC1 &\pending&\pending&\pending&\pending&\pending\\ |
| OPRK1 &\pending&\pending&\pending&\pending&\pending\\ |
| PKM2 &\pending&\pending&\pending&\pending&\pending\\ |
| PPARG &\pending&\pending&\pending&\pending&\pending\\ |
| TP53 &\pending&\pending&\pending&\pending&\pending\\ |
| VDR &\pending&\pending&\pending&\pending&\pending\\ |
| \bottomrule |
| \end{tabular} |
| \end{table} |
|
|
| \begin{table}[H] |
| \centering\small |
| \caption{\textbf{Property-predictor validation.} Held-out scaffold groups, with five split seeds. MAE retains label units. Molecular structures are canonicalized before grouping.} |
| \label{tab:predictors} |
| \begin{tabular}{llrrrr} |
| \toprule |
| Dataset & Predictor & Train molecules & Test molecules & MAE $\downarrow$ & $R^2\uparrow$\\ |
| \midrule |
| Caco-2 Wang & Morgan + extra trees &\pending&\pending&\pending&\pending\\ |
| Caco-2 Wang & Molecular encoder + head &\pending&\pending&\pending&\pending\\ |
| BACE & Morgan + extra trees &\pending&\pending&\pending&\pending\\ |
| BACE & Molecular encoder + head &\pending&\pending&\pending&\pending\\ |
| \bottomrule |
| \end{tabular} |
| \end{table} |
|
|
| \subsection{Implementation and Benchmark Records} |
| The reference implementation includes a validated DAG representation, log-domain dynamic programs, learned prefix values, a forward trajectory-balance sampler, reaction execution, and public property prediction. The neural policy uses concatenated source and destination features together with edge cost. Both value and policy networks have two hidden layers of width 64 with SiLU activations. The Adam learning rate is 0.002, the batch size is 64, and the uniform-action exploration fraction is 0.15. The default run uses 2,000 updates. Separate training runs specify each budget, property setting, and route temperature. |
|
|
| Full-benchmark measurements are stored in a common schema with method, target, budget, seed, oracle calls, sample count, elapsed time, and metric values. External baseline outputs include the same candidate and route fields. The repository retains completed integration measurements and lists the remaining main-benchmark runs. Exact and neural samplers use the same graph serialization so that target distributions, sampled paths, and action logs can be compared directly. |
|
|
| \section{Additional Related Work} |
| \label{app:related} |
| \paragraph{Continuous generation and learned maps.} |
| Diffusion training connects denoising with learned stochastic dynamics \citep{sohldickstein2015nonequilibrium,ho2020ddpm,song2021scorebased}. Continuous generative trajectories can be parameterized with neural differential equations, rectified flows, and stochastic interpolants \citep{chen2018neuralode,liu2023rectifiedflow,albergo2025stochasticinterpolants}. Minibatch transport couplings affect the conditional paths used for flow matching \citep{tong2024minibatchot}. Consistency trajectories, flow-map matching, self-distillation, and mean flows permit direct prediction across finite intervals \citep{kim2024ctm,boffi2025flowmapmatching,boffi2025selfdistillation,geng2025meanflows,sabour2025alignyourflow}. The operation graph in \method{} specifies executable state changes and their resource use. Its conditional path objective can be evaluated independently of the numerical steps used by a continuous generator. |
|
|
| \paragraph{Discrete probability paths.} |
| Discrete denoising can use transition matrices, continuous-time jump processes, or masked-token conditionals \citep{austin2021d3pm,campbell2022continuoustimediscrete,lou2024sedd,sahoo2024mdlm,shi2024simplifiedmasked}. Discrete flow matching permits specified probability paths over categorical states, including simplex representations and general discrete interpolants \citep{gat2024discreteflowmatching,stark2024dirichletflow,campbell2024generativeflows,shaul2025generaldiscretepaths}. Generator matching extends this construction to more general Markov processes \citep{holderrieth2025generatormatching}. Endpoint-generation comparisons can use these formulations with feasibility assessed through a common executor and route-retrieval procedure. |
|
|
| \paragraph{Molecular construction and optimization.} |
| Grammar and junction-tree representations constrain molecular assembly, while graph policies optimize molecular properties through sequential actions \citep{grammarvae,jtvae,gcpn}. Retrosynthetic planning and synthesis-graph generation explicitly connect products to construction steps \citep{retrostar,synthesizable,synthesisdags}. Three-dimensional molecular generation additionally requires consistency among atomic identities, bonds, and target-pocket geometry \citep{molopt,targetdiff,cgflow}. For large molecular spaces, benchmark conclusions depend on oracle budgets and the behavior of property predictors away from their training distribution \citep{pmo,designbench,coms,cbas,min}. Molecular benchmarks and synthesis studies provide complementary measurements of structural quality, predicted properties, and experimental accessibility \citep{moleculenet,litpcba,synthemol,guacamol,moses}. |
|
|
| \paragraph{Distributional comparisons.} |
| Generative flow networks support reward-proportional sampling, conditional generation, and entropy-related quantities \citep{gfn,gfnfoundations,tb,subtb}. Multi-objective sampling changes the reward through user-selected preference weights \citep{mogfn}. Learned backward policies can be fitted through trajectory likelihood objectives \citep{backward}. Here, the conditional cost objective gives a specified target for that backward policy. Kernel two-sample statistics offer additional endpoint comparisons on larger outcome spaces \citep{mmd,rff}, with the feature map and sample count fixed across methods. |
|
|
| \end{document} |
|
|
|
|
