new

Get trending papers in your email inbox!

Subscribe

Daily Papers

byAK and the research community

Oct 16

Nonreciprocal black holes with primary hair in beyond Horndeski gravity: Exact solutions and stability

We develop a novel systematic method for constructing exact, static, and asymptotically flat black-hole solutions in shift-symmetric and Z_2-symmetric beyond Horndeski theories. In contrast to previous studies, our approach does not rely on the restrictive reciprocal condition g_{tt}g_{rr}=-1, allowing for more general geometries characterized by two independent metric functions. The resulting solution space contains two qualitatively distinct types of hair originating from different branches of the theory. One branch is characterized by khronometric hair, while the other by genuine scalar hair. In both, the scalar profile defines a nontrivial preferred foliation, but the shift-symmetry Noether current and the associated propagating degree of freedom are trivial for the former and nonvanishing for the latter. Focusing mainly on the second branch, we derive several distinct families of exact black-hole solutions. In general, these solutions are singular and exhibit primary scalar hair associated with an independent scalar charge. Nevertheless, we demonstrate that they can be regularized to yield regular black-hole or solitonic configurations, for which the scalar hair becomes secondary. We further investigate the stability properties of these solutions under linear perturbations by analyzing the axial sector and the monopole (ell=0) mode of the even-parity sector. For representative solutions with scalar hair, the perturbative analysis provides evidence for linear stability under the modes considered.

  • 3 authors
·
Oct 7

Polynomial Acceleration of Alternating Projections: Chebyshev Acceleration, Convergence Rates, and Retractions

Alternating projections (AP) near a clean intersection of smooth manifolds can converge slowly when the Friedrichs angle is small. We develop the Fixed-Point Manifold Acceleration (FPMA) framework for {local convergence and polynomial acceleration of} smooth maps that fix a manifold pointwise. {Under suitable regularity and uniform normal spectral stability, FPMA establishes local R-linear convergence and a smooth limiting map inducing a local retraction. At fixed points, consistent polynomial acceleration with p(1)=1 preserves the selected limit's first-order response to normal initial perturbations whenever the transformed normal blocks are uniformly spectrally stable. Under sufficient smoothness, we give a curvature criterion for second-order induced retractions and show that stable consistent polynomial acceleration preserves both the quadratic retraction expansion and this criterion. {As a special case, we establish new local R-linear convergence guarantees for fixed-degree Chebyshev-accelerated exact AP and second-order induced retractions. For uniform Friedrichs angle θ_Fin(0,π/2), acceleration strictly improves the local convergence rate per AP evaluation. The large-degree limiting spectral factor at fixed θ_F predicts O(θ_F^{-1}) evaluations for fixed relative error reduction as θ_Fdownarrow0, versus O(θ_F^{-2}) for ordinary AP.}} {The framework also covers} inexact, relaxed, and generalized AP {under their respective regularity and stability conditions.}

  • 2 authors
·
Oct 4

The Implicit Regularization of Dynamical Stability in Stochastic Gradient Descent

In this paper, we study the implicit regularization of stochastic gradient descent (SGD) through the lens of {\em dynamical stability} (Wu et al., 2018). We start by revising existing stability analyses of SGD, showing how the Frobenius norm and trace of Hessian relate to different notions of stability. Notably, if a global minimum is linearly stable for SGD, then the trace of Hessian must be less than or equal to 2/eta, where eta denotes the learning rate. By contrast, for gradient descent (GD), the stability imposes a similar constraint but only on the largest eigenvalue of Hessian. We then turn to analyze the generalization properties of these stable minima, focusing specifically on two-layer ReLU networks and diagonal linear networks. Notably, we establish the {\em equivalence} between these metrics of sharpness and certain parameter norms for the two models, which allows us to show that the stable minima of SGD provably generalize well. By contrast, the stability-induced regularization of GD is provably too weak to ensure satisfactory generalization. This discrepancy provides an explanation of why SGD often generalizes better than GD. Note that the learning rate (LR) plays a pivotal role in the strength of stability-induced regularization. As the LR increases, the regularization effect becomes more pronounced, elucidating why SGD with a larger LR consistently demonstrates superior generalization capabilities. Additionally, numerical experiments are provided to support our theoretical findings.

  • 2 authors
·
May 27, 2023

Extensions of Schoen--Simon--Yau and Schoen--Simon theorems via iteration à la De Giorgi

We give an alternative proof of the Schoen--Simon--Yau curvature estimates and associated Bernstein-type theorems (1975), and extend the original result by including the case of 6-dimensional (stable minimal) immersions. The key step is an ε-regularity theorem, that assumes smallness of the scale-invariant L^2 norm of the second fundamental form. Further, we obtain a graph description, in the Lipschitz multi-valued sense, for any stable minimal immersion of dimension ngeq 2, that may have a singular set Σ of locally finite H^{n-2}-measure, and that is weakly close to a hyperplane. (In fact, if H^{n-2}(Σ)=0, the conclusion is strengthened to a union of smooth graphs.) This follows directly from an ε-regularity theorem, that assumes smallness of the scale-invariant L^2 tilt-excess (verified when the hypersurface is weakly close to a hyperplane). Specialising the multi-valued decomposition to the case of embeddings, we recover the Schoen--Simon theorem (1981). In both ε-regularity theorems the relevant quantity (respectively, length of the second fundamental form and tilt function) solves a non-linear PDE on the immersed minimal hypersurface. The proof is carried out intrinsically (without linearising the PDE) by implementing an iteration method à la De Giorgi (from the linear De Giorgi--Nash--Moser theory). Stability implies estimates (intrinsic weak Caccioppoli inequalities) that make the iteration effective despite the non-linear framework. (In both ε-regularity theorems the method gives explicit constants that quantify the required smallness.)

  • 1 authors
·
Sep 11, 2025

Analysis of grid instabilities in particle-in-cell codes based on a meshfree approach

A new method of analyzing grid, or aliasing, instabilities is described, in particle-in-cell (PIC) codes for plasma kinetic theory. It starts with a meshfree approach with N_{p} macroparticles, with N_{p}<infty. Linearization of these equations is done about a uniform density non-drifting equilibrium prescribed by macroparticles on a uniform lattice. Macroparticle positions perturbations are induced about this lattice. The resulting linear equations are analyzed for stability and dispersion relations. The linearized equations are then discretized on a grid of N_{g} points with N_{g} le N_{p}, in both momentum conserving (MCP) and energy conserving (ECP) formulations. For a cold stationary plasma, this leads a dynamical matrix. For ECP, the matrix is symmetric and positive definite (SPD) and immediately shows stability. For MCP, the dynamical matrix is neither symmetric nor positive definite, preventing immediate conclusions on stability. For both MCP and ECP and for a cold stationary plasma, the resulting matrix elements vary relative to the meshfree value, a result of aliasing. This deviation is periodic in the displacement of the particle lattice relative to the grid. In the MCP discretization, eigenvalues occur in complex conjugate pairs for general placement of the particle lattice relative to the grid. These conjugate pairs indicate nonnegative growth rates. The nature of the aliasing is studied and shown to be related to the trapezoidal rule error over the grid. The scaling of the linear growth rates with respect to N_{g} and especially with respect to N_{ppc} is studied. These analytical results are compared with PIC simulations and found to be in excellent agreement. The connection with a cold drifting beam is discussed briefly, specifically, grid-induced instabilities for both the MCP and ECP.

  • 2 authors
·
Sep 30

Extended Detailed Balance for Systems with Irreversible Reactions

The principle of detailed balance states that in equilibrium each elementary process is equilibrated by its reverse process. For many real physico-chemical complex systems (e.g. homogeneous combustion, heterogeneous catalytic oxidation, most enzyme reactions etc), detailed mechanisms include both reversible and irreversible reactions. In this case, the principle of detailed balance cannot be applied directly. We represent irreversible reactions as limits of reversible steps and obtain the principle of detailed balance for complex mechanisms with some irreversible elementary processes. We proved two consequences of the detailed balance for these mechanisms: the structural condition and the algebraic condition that form together the extended form of detailed balance. The algebraic condition is the principle of detailed balance for the reversible part. The structural condition is: the convex hull of the stoichiometric vectors of the irreversible reactions has empty intersection with the linear span of the stoichiometric vectors of the reversible reaction. Physically, this means that the irreversible reactions cannot be included in oriented pathways. The systems with the extended form of detailed balance are also the limits of the reversible systems with detailed balance when some of the equilibrium concentrations (or activities) tend to zero. Surprisingly, the structure of the limit reaction mechanism crucially depends on the relative speeds of this tendency to zero.

  • 2 authors
·
Jan 27, 2011

Synchrony and Oscillatory Dynamics for a 2-D PDE-ODE Model of Diffusion-Sensing with Small Signaling Compartments

We analyze a class of cell-bulk coupled PDE-ODE models, motivated by quorum and diffusion sensing phenomena in microbial systems, that characterize communication between localized spatially segregated dynamically active signaling compartments that have a permeable boundary. Each cell secretes a signaling chemical into the bulk region at a constant rate and receives a feedback of the bulk chemical from the entire collection of cells. This global feedback, which activates signaling pathways within the cells, modifies the intracellular dynamics according to the external environment. The cell secretion and global feedback are regulated by permeability parameters across the cell membrane. For arbitrary reaction-kinetics within each cell, the method of matched asymptotic expansions is used in the limit of small cell radius to construct steady-state solutions of the PDE-ODE model, and to derive a globally coupled nonlinear matrix eigenvalue problem (GCEP) that characterizes the linear stability properties of the steady-states. In the limit of large bulk diffusivity an asymptotic analysis of the PDE-ODE model leads to a limiting ODE system for the spatial average of the concentration in the bulk region that is coupled to the intracellular dynamics within the cells. Results from the linear stability theory and ODE dynamics are illustrated for Sel'kov reaction-kinetics, where the kinetic parameters are chosen so that each cell is quiescent when uncoupled from the bulk medium. For various specific spatial configurations of cells, the linear stability theory is used to construct phase diagrams in parameter space characterizing where a switch-like emergence of intracellular oscillations can occur through a Hopf bifurcation.

  • 2 authors
·
Jul 16, 2020

Conflict-Predictive Variable Horizons in Multi-Drone Distributed Model Predictive Control

In distributed model predictive control for multi-drone collision avoidance, a fixed prediction horizon forces a compromise: a short horizon is inexpensive but reacts late to approaching neighbors, whereas a long one anticipates conflicts at a per-step cost that grows superlinearly with its length. We propose a conflict-predictive variable horizon that each drone sets locally, leaving the distributed model predictive control itself unchanged. From a short history of observed positions, a drone extrapolates the flight lines of its neighbors, tests each against its own using confidence funnels that narrow with prediction range, and obtains each time to conflict in closed form. The horizon is then the smallest admissible value whose planning window covers the farthest predicted conflict. It collapses to its minimum in clear airspace and grows only when a conflict lies ahead. Provided this minimum meets a single computable feasibility bound, we prove that recursive feasibility and asymptotic stability are preserved for every horizon the policy can select. These guarantees hold for a linear model, and a cascaded inner loop reduces each quadrotor's translational dynamics to a perturbed double integrator, so they carry over to the linearized quadrotor model and, as practical stability, to the full nonlinear one. In simulation on dense antipodal-swap benchmarks, the variable horizon reduces both per-step solver cost and total computation well below those of a long fixed horizon, and it maintains separation in every run, which a short fixed horizon of comparable per-step cost does not.

  • 4 authors
·
Sep 6

A Cumulative Framework for Solid Deformation

A cumulative variational framework for solid deformation is proposed. Its central organizing principle is the recursive combination of three structural components of one material body: a compatible Cosserat component, a distributed-incompatibility component, and a defect-source component. At any prescribed external condition these components constitute a combined material state. They are governed under common physical boundary and initial conditions and are coupled through one cumulative energy and the associated field equations. They are not separate boundary-value problems and need not appear in a prescribed temporal sequence. The cumulative character of the framework concerns the response of the combined state to variation of externally prescribed conditions. For the special case of an initially perfect body, loss of stability may produce successive branches with defect and then source participation. Such a sequence is a bifurcation property of the selected initial state and constitutive energy, rather than a defining hierarchy of the framework. A one-dimensional specialization illustrates structural bifurcation, residual deformation and residual stress produced by retained structure, dependence on the initial combined state, directional memory under reverse loading, parametric cumulative hysteresis, and changes of observable tangent stiffness associated with structural stabilization. A spatial extension introduces defect and source gradient terms and yields a size-dependent structural stability operator. For the stable quartic energy considered, the gradient terms regularize nonuniform perturbations but do not by themselves generate a finite-wavelength localization instability on an already stable three-component branch. The examples thereby distinguish consequences of the cumulative structure and energy from phenomena requiring additional constitutive evolution laws.

  • 1 authors
·
Sep 16

Necessary Conditions for Free End-Time Sweeping Process Problems

This paper derives necessary optimality conditions for free end-time optimal control problems governed by sweeping process dynamics. In contrast to classical optimal control problems, the presence of the normal cone term in the state equation introduces discontinuities with respect to the state variable and prevents the standard Lipschitz regularity assumptions typically used in optimal control theory. As a result, both the analysis of trajectories and the formulation of optimality conditions require refined tools. We address problems with general endpoint constraints and time-dependent data, considering both the case in which the dynamics are Lipschitz continuous in time and the more challenging setting of merely measurable time dependence. Special attention is devoted to the additional transversality conditions associated with the free end-time variable. In the Lipschitz case, these conditions are expressed in terms of a function of bounded variation that coincides almost everywhere with the maximized Hamiltonian. In the measurable case, we show that a suitable interpretation can be obtained via the notions of sub- and super-essential values of the maximized Hamiltonian, recently introduced in the literature, leading to strengthened transversality conditions. These conditions also involve an additional term reflecting the interaction between the costate arc and the normal cone to the moving set at the optimal endpoints. Our approach combines suitable perturbation techniques with a penalization method tailored to sweeping processes. The moving set is modelled as a finite intersection of inequality-defined sets satisfying suitable constraint qualifications, including a positive linear independence condition and a diagonal dominance property of the associated Gramian matrix. We also introduce a local version of the constraint qualification condition. Our results apply to a broad class of problems, including those involving unbounded moving sets such as polyhedra. The resulting optimality system extends and sharpens recent results for classical free end-time problems to the non-Lipschitz, discontinuous framework of sweeping processes. The relevance of the transversality conditions obtained in this article is illustrated through two simple examples.

  • 3 authors
·
Oct 6

Data-Driven Policy Iteration Without an Initial Stabilizing Policy: A Finite-Horizon Bootstrap Method

This article investigates data-driven policy iteration (PI) for continuous-time linear systems without requiring an initially stabilizing policy. Standard infinite-horizon PI is not self-starting because its policy-evaluation step is well posed only when the feedback gain is stabilizing. However, verifying this property is difficult when the system matrices are unknown. To remove this requirement, we develop a finite-horizon bootstrap method. The key idea is to perform policy evaluation over a compact interval for a shifted system, where the evaluation equation is well defined for arbitrary bounded time-varying policies. We show that, for a sufficiently long horizon, the initial-time optimal gain of the shifted finite-horizon problem, when applied as a constant feedback gain, achieves a prescribed stability margin for the original system. We then derive a data-driven implementation from an off-policy identity evaluated along trajectories of the original plant. We use basis-function approximations to reconstruct the finite-horizon value matrix and policy, and we characterize the resulting error through a perturbed policy-improvement recursion. A data-driven Lyapunov certificate is further introduced to verify admissibility of the candidate gain before it is used to initialize infinite-horizon PI. Numerical studies of a batch reactor and a two-mass-spring system demonstrate the effectiveness of the proposed bootstrap method.

  • 2 authors
·
Sep 14

On error bounds and noncritical Lagrange multipliers

Critical Lagrange multipliers are known to be responsible for the potentially slow convergence of Newton-type methods, and noncriticality of a given multiplier has been tied to an error bound which estimates the distance of a primal-dual pair associated with a perturbed problem to the primal solution and the multiplier set of the unperturbed one, and is decisive for a local convergence analysis. We investigate this relation in an abstract setting where merely a set-valued mapping assigning primal-dual pairs to a parameter is available, and characterize the error bound of interest in terms of noncriticality of the underlying multiplier and two calmness-type properties of the associated multiplier mapping, one of which is a novel weak inner calmness in the fuzzy sense. No structural assumptions on the underlying problem are needed for that, and the two calmness-type conditions are not only sufficient but also necessary. Afterwards, we specify these findings for composite optimization problems. Whenever the subdifferential of the outer function is a polyhedral mapping, both conditions hold automatically, which explains the equivalence of noncriticality and the error bound observed in the polyhedral setting. Whenever the outer function is C^2-decomposable, they are secured by calmness of a restricted multiplier mapping together with a closedness condition, which recovers and extends the known characterization for C^2-cone reducible constraint systems.

  • 2 authors
·
Oct 11

Predictor-Feedback CACC for Vehicular Platoons with Actuation and Communication Delays Based on a Multiple-Predecessor-Following CTH Nominal Strategy

We develop a predictor-feedback cooperative adaptive cruise control (CACC) design relying on a multiple-predecessor-following (MPF) topology-based nominal delay-free CACC law. We consider vehicular platoons with heterogeneous vehicles, whose dynamics are described by a third-order linear system subject to actuation delay, along with vehicle-to-vehicle (V2V) communication delay. The design achieves individual vehicle stability, string stability, and zero, steady-state speed/spacing tracking errors, for any value of the actuation delay. The proofs of individual vehicle stability, string stability, and regulation rely on employment of an input-output approach on the frequency domain, capitalizing on the delay-compensating property of the design, which enables as to derive explicit string stability conditions on control and vehicle models parameters. The theoretical guarantees of string stability and the respective conditions on parameters are illustrated also numerically. We present consistent simulation results, for a ten-vehicle platoon, illustrating the potential of the design in traffic throughput improvement, as compared with a predictor-feedback CACC design in which, each ego vehicle's controller utilizes information only from a single preceding vehicle. We also present simulation results in a realistic scenario in which the leading vehicle's trajectory is obtained from NGSIM data.

  • 3 authors
·
Apr 6

Hierarchical Melnikov Realization: Logarithmic Factor Improvement to Hilbert Number Lower Bounds

The second part of Hilbert's 16th problem is one of the most fundamental open problems in the qualitative theory of ordinary differential equations and dynamical systems. A central unresolved question is whether the Hilbert number H(d) is finite for arbitrary polynomial degree d. (A very recent manuscript from OpenAI asserts the finiteness of H(d); see the remark at the end of this paper.) In the absence of a general upper bound theory, most existing work constructs explicit perturbed polynomial systems to obtain improved lower bounds for H(d), while rigorous upper bound estimates remain largely unavailable. This paper focuses on improving the quantitative lower bound for polynomial systems through a systematic Hierarchical Melnikov Realization framework. We construct perturbed dynamics based on an anisotropic Chebyshev Hamiltonian, where hierarchically selected perturbation coefficients generate simple zeros of the first-order Melnikov function simultaneously across multiple disjoint period annuli. Two distinct 3-adic filtrations supply the necessary logarithmic correction factors, and a Borel--Gauss analysis establishes the required local rank condition under sufficiently strong anisotropy. As the main result of this work, we prove that H(d)=Ωbigl(d^2ln^2 dbigr), which improves the previously known lower bound by a factor of ln d. The full construction of limit cycles and the resulting dynamical bounds are formally certified using Lean 4 formal verification.

  • 2 authors
·
Oct 7

Leslie Population Models in Predator-prey and Competitive populations: theory and applications by machine learning

We introduce a new predator-prey model by replacing the growth and predation constant by a square matrix, and the population density as a population vector. The classical Lotka-Volterra model describes a population that either modulates or converges. Stability analysis of such models have been extensively studied by the works of Merdan (https://doi.org/10.1016/j.chaos.2007.06.062). The new model adds complexity by introducing an age group structure where the population of each age group evolves as prescribed by the Leslie matrix. The added complexity changes the behavior of the model such that the population either displays roughly an exponential growth or decay. We first provide an exact equation that describes a time evolution and use analytic techniques to obtain an approximate growth factor. We also discuss the variants of the Leslie model, i.e., the complex value predator-prey model and the competitive model. We then prove the Last Species Standing theorem that determines the dominant population in the large time limit. The recursive structure of the model denies the application of simple regression. We discuss a machine learning scheme that allows an admissible fit for the population evolution of Paramecium Aurelia and Paramecium Caudatum. Another potential avenue to simplify the computation is to use the machinery of quantum operators. We demonstrate the potential of this approach by computing the Hamiltonian of a simple Leslie system.

  • 5 authors
·
Dec 20, 2024

Data-driven Kernel-based Predictive Control with Stability and Robustness Guarantees

In this paper, we provide a theoretical analysis of the closed-loop properties of a data-driven kernel-based predictive control (DDKPC) scheme developed solely from input-output data. The proposed formulation integrates a robust data-driven predictive control framework with a multi-step predictor for nonlinear systems constructed via kernel-based methods. This predictor implicitly captures the system's nonlinear behavior using the representer theorem. For the nominal case with noise-free data, we prove that the DDKPC scheme guarantees recursive feasibility and closed-loop stability, provided that the prediction horizon is sufficiently long and the kernel representation error is sufficiently small. To facilitate real-time implementation, we introduce a penalty relaxation formulation to alleviate the computational burden inherently caused by nonconvex implicit constraints. Furthermore, the framework is robustified against measurement noise by aggregating the representation mismatch and the bounded noise into a unified uncertainty bound. Finally, we extend the DDKPC framework to slowly time-varying nonlinear systems by periodically reconstructing the kernel predictor from a fixed-budget online dictionary managed by the approximate linear dependency (ALD) criterion. Under suitable conditions on the rate of variation of the input-output evolution and the online prediction error, recursive feasibility and practical closed-loop stability are preserved. The effectiveness of the proposed approach is illustrated through numerical examples.

  • 4 authors
·
Jul 2

Horizon-Dependent Tube MPC for Spacecraft Rendezvous on Elliptical Orbits with Conditional Robust Constraint Satisfaction

Spacecraft rendezvous on elliptical orbits must hold a safety corridor under navigation noise, unmodelled perturbations, and thrust errors driven by propellant mass uncertainty. This paper develops a tube-based model predictive controller for the Yamanaka-Ankersen linear time-varying dynamics, carrying constraint-tightening tube methods from circular to elliptical orbits. A horizon-dependent, element-wise error bound propagates the actual closed-loop matrices along the prediction window, so early prediction steps keep nearly the full corridor that a constant-width tube would surrender. A multiplicative-to-additive conversion folds mass and thrust uncertainty into the same tightening recursion, and a Perron-Frobenius spectral-radius condition supplies a computable certificate that the tightening converges, with an explicit input-to-state stability gain. In a paired Monte Carlo campaign on a Mars sample-return orbit, the tube controller cuts mean corridor violations by more than an order of magnitude against a nominal predictive baseline at comparable fuel cost. A nonlinear truth-model test with oblateness perturbations well beyond the assumed disturbance budget leaves the corridor unviolated, and when the design envelope is exceeded the tightened problem becomes infeasible and the controller reverts to a saturated linear fallback, making the loss of guarantee explicit rather than silent. The construction requires only element-wise arithmetic and an open-source quadratic-programming solver.

  • 1 authors
·
Aug 8

Emergent Transfer of a Physics Foundation Model from Simulation to Laboratory Turbulence

Whether physics foundation models can be usefully deployed on laboratory experiments remains an open question for scientific machine learning (ML). We test this question on the Rayleigh-Taylor instability (RTI), a ubiquitous and demanding fluid instability seen from tabletop flows to supernova explosions, in which small perturbations at a density interface grow into chaotic, multiscale mixing as a lighter fluid accelerates into a heavier one. Standard ML models struggle with RTI, and despite over a century of theoretical, numerical, and experimental work, it carries an unresolved discrepancy between simulation and experiment: the late-time mixing growth rate, α, measured in most laboratory experiments (sim 0.06-0.07), is roughly three times the value from idealized direct numerical simulations (DNS, sim 0.02). The gap's origin remains debated. These properties make RTI a stringent test for a question that matters well beyond RTI: can foundation models trained only on simulations generalise to sparse, messy, and noisy laboratory settings? We finetune Walrus, a foundation model for continuum dynamics, on three or fewer DNS realizations and recover key RTI physics over long rollouts. Applied zero-shot to sliding-barrier laboratory data, the finetuned model leaves the DNS-like regime and enters the observed growth band, having never seen a single experimental sample. These results provide independent, data-driven evidence that initial conditions play a crucial role in the longstanding sim-experiment gap in α. The model also generalises zero-shot to stable stratification, a buoyancy regime absent from training, correctly slowing mixing-layer growth. Together, our results show that foundation models can generalise well beyond their training data, predicting laboratory behavior and unseen physical regimes, opening new ways to probe longstanding simulation-experiment gaps.

polymathic-ai Polymathic AI
·
May 30

An Idealized Delay-Differential Model of Scuba Diver Porpoising and Runaway Ascent

A scuba diver holding constant depth balances on an unstable equilibrium: the gas carried in the suit and buoyancy compensator compresses with depth, so the buoyant force falls as the diver sinks and rises as the diver ascends. We represent the diver as a proportional-derivative controller that regulates this compressible-buoyancy saddle after a finite reaction delay, and we derive the governing delay differential equation from the vertical force balance and the isothermal gas law, reducing it to a damping ratio, two control gains, and a dimensionless delay. The characteristic spectrum, obtained by pseudospectral collocation of the semigroup generator and checked against a direct Newton solution of the characteristic equation, locates the Hopf boundary that separates stable hovering from sustained porpoising; for the baseline diver the critical reaction delay is 3.36 s and the onset period is 28.8 s. The bifurcation is supercritical, and because the saturating force is the quadratic hydrodynamic drag, the limit-cycle amplitude grows in proportion to the delay excess rather than as its square root. The safe-operating envelope shows that runaway ascent is triggered by saturation of the compensator, not by loss of linear stability, so a stable and an unstable diver can share the same escape threshold. As onset is approached, the lag-one autocorrelation and variance rise while the fitted recovery rate falls and matches the spectral abscissa, giving an eigenvalue-exact early warning of the transition.

  • 8 authors
·
Aug 14

Transition from decaying to decayless kink oscillations of solar coronal loops

The transition of an impulsively excited kink oscillation of a solar coronal loop to an oscillation with a stationary amplitude, i.e., the damping pattern, is determined using the low-dimensional self-oscillation model. In the model, the decayless kink oscillations are sustained by the interaction of the oscillating loop with an external quasi-steady flow. The analytical solution is based on the assumption that the combined effect of the effective dissipation, for example, by resonant absorption, and interaction with an external flow, is weak. The effect is characterised by a dimensionless coupling parameter. The damping pattern is found to depend upon the initial amplitude and the coupling parameter. The approximate expression shows a good agreement with a numerical solution of the self-oscillation equation. The plausibility of the established damping pattern is demonstrated by an observational example. Notably, the damping pattern is not exponential, and the characteristic decay time is different from the time determined by the traditionally used exponential damping fit. Implications of this finding for seismology of the solar coronal plasmas are discussed. In particular, it is suggested that a very rapid, in less than the oscillation period, decay of the oscillation to the stationary level, achieved for larger values of the coupling parameter, can explain the relative rareness of the kink oscillation events.

  • 3 authors
·
Jun 10, 2024

MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs

We present the design, implementation and engineering experience in building and deploying MegaScale, a production system for training large language models (LLMs) at the scale of more than 10,000 GPUs. Training LLMs at this scale brings unprecedented challenges to training efficiency and stability. We take a full-stack approach that co-designs the algorithmic and system components across model block and optimizer design, computation and communication overlapping, operator optimization, data pipeline, and network performance tuning. Maintaining high efficiency throughout the training process (i.e., stability) is an important consideration in production given the long extent of LLM training jobs. Many hard stability issues only emerge at large scale, and in-depth observability is the key to address them. We develop a set of diagnosis tools to monitor system components and events deep in the stack, identify root causes, and derive effective techniques to achieve fault tolerance and mitigate stragglers. MegaScale achieves 55.2% Model FLOPs Utilization (MFU) when training a 175B LLM model on 12,288 GPUs, improving the MFU by 1.34x compared to Megatron-LM. We share our operational experience in identifying and fixing failures and stragglers. We hope by articulating the problems and sharing our experience from a systems perspective, this work can inspire future LLM systems research.

  • 32 authors
·
Feb 23, 2024 2

Sparse Linear Regression is Easy on Random Supports

Sparse linear regression is one of the most basic questions in machine learning and statistics. Here, we are given as input a design matrix X in R^{N times d} and measurements or labels {y} in R^N where {y} = {X} {w}^* + {xi}, and {xi} is the noise in the measurements. Importantly, we have the additional constraint that the unknown signal vector {w}^* is sparse: it has k non-zero entries where k is much smaller than the ambient dimension. Our goal is to output a prediction vector {w} that has small prediction error: 1{N}cdot |{X} {w}^* - {X} {w}|^2_2. Information-theoretically, we know what is best possible in terms of measurements: under most natural noise distributions, we can get prediction error at most epsilon with roughly N = O(k log d/epsilon) samples. Computationally, this currently needs d^{Omega(k)} run-time. Alternately, with N = O(d), we can get polynomial-time. Thus, there is an exponential gap (in the dependence on d) between the two and we do not know if it is possible to get d^{o(k)} run-time and o(d) samples. We give the first generic positive result for worst-case design matrices {X}: For any {X}, we show that if the support of {w}^* is chosen at random, we can get prediction error epsilon with N = poly(k, log d, 1/epsilon) samples and run-time poly(d,N). This run-time holds for any design matrix {X} with condition number up to 2^{poly(d)}. Previously, such results were known for worst-case {w}^*, but only for random design matrices from well-behaved families, matrices that have a very low condition number (poly(log d); e.g., as studied in compressed sensing), or those with special structural properties.

  • 3 authors
·
Nov 8, 2025

SNIC bifurcation and its Application to MEMS

This project focuses on a method to extract a frequency comb in mechanical means, for general interest and numerous practical applications in MEMS. The method of execution is the implementation of a beam that is exhibiting non-linear dynamics that is perturbed and analyzed for its transverse vibrations. The perturbation is an external harmonic driver with a chosen small amplitude and frequency (which is slightly detuned from the beam eigenfrequency), that when engaged with the unperturbed beam oscillations, causes it reach a state of "injection pulling" - an effect that occurs when one harmonic oscillator is coupled with a second one and causes it to oscillate in a frequency near its own. This causes the beam to reach SNIC bifurcation, rendering a frequency comb as desired. Theoretical analysis showed that the problem can be modelled using a non-linear equation of the beam, that translates to a form of the non-linear Duffing equation. While a solution to the dynamics function of the beam is hard to obtain in practice due to mathematical difficulties, a slow evolution model is suggested that is composed of functions of a amplitude and phase. Using several additional mathematical assumptions, the amplitude is seen to be related to the phase, while the phase equation solution is seen to be of the form of Adler's equation. These assumptions ultimately reduce the entire behaviour of the beam to a relatively simple solution to the Adler equation, which has a known analytical solution. Computerized numerical simulations are run on it to check the results and compare them to the theory and desired outcome. The results agreed with the theory and produce the expected frequency comb, showing the assumptions to be valid in extracting the comb.

  • 1 authors
·
Aug 24, 2025

Bounded Reachability & Jailbreak Detection via Contraction-Constrained State Space Models

Safety heads are lightweight classifiers attached to pretrained language models for flagging harmful inputs before generation. Their empirical detection performance has been studied, but their formal robustness properties remain largely unexplored. We ask when a State Space Model (SSM)-based safety head can be certified to produce the same prediction for all inputs within a bounded embedding-space perturbation. We prove that the answer turns on a single condition: the l_infty norm of the state transition matrix must satisfy A_infty<1 (the contraction condition), which enables exact interval bound propagation (IBP) certification for linear time-invariant classifiers. When the contraction condition holds, the reachable output interval has bounded steady-state width and examples can be certified as robustly classified. When it fails, the interval grows exponentially with sequence length and certification is impossible at any practical perturbation radius. We enforce contraction with a hinge penalty and show on toxic comment data that certified fraction improves from 41\% to 59\%, with a sharp empirical phase transition at A_infty=1 matching the theory. Applying a contraction-regularized S4 head to jailbreak detection on JailbreakBench, we achieve a zero-shot transfer to AdvBench (DR=0.994) and HarmBench (DR=0.988). A logistic regression on mean-pooled Mamba-130M embeddings matches or exceeds the S4 head on every detection metric, confirming that harmful intent is already linearly separable in the embedding space. The S4 safety head's contribution is not superior discrimination but the formal certification that no probe-based approach provides.

  • 1 authors
·
Oct 1

On the Effectiveness of Parameter-Efficient Fine-Tuning

Fine-tuning pre-trained models has been ubiquitously proven to be effective in a wide range of NLP tasks. However, fine-tuning the whole model is parameter inefficient as it always yields an entirely new model for each task. Currently, many research works propose to only fine-tune a small portion of the parameters while keeping most of the parameters shared across different tasks. These methods achieve surprisingly good performance and are shown to be more stable than their corresponding fully fine-tuned counterparts. However, such kind of methods is still not well understood. Some natural questions arise: How does the parameter sparsity lead to promising performance? Why is the model more stable than the fully fine-tuned models? How to choose the tunable parameters? In this paper, we first categorize the existing methods into random approaches, rule-based approaches, and projection-based approaches based on how they choose which parameters to tune. Then, we show that all of the methods are actually sparse fine-tuned models and conduct a novel theoretical analysis of them. We indicate that the sparsity is actually imposing a regularization on the original model by controlling the upper bound of the stability. Such stability leads to better generalization capability which has been empirically observed in a lot of recent research works. Despite the effectiveness of sparsity grounded by our theory, it still remains an open problem of how to choose the tunable parameters. To better choose the tunable parameters, we propose a novel Second-order Approximation Method (SAM) which approximates the original problem with an analytically solvable optimization function. The tunable parameters are determined by directly optimizing the approximation function. The experimental results show that our proposed SAM model outperforms many strong baseline models and it also verifies our theoretical analysis.

  • 6 authors
·
Nov 27, 2022

Small-scale proxies for large-scale Transformer training instabilities

Teams that have trained large Transformer-based models have reported training instabilities at large scale that did not appear when training with the same hyperparameters at smaller scales. Although the causes of such instabilities are of scientific interest, the amount of resources required to reproduce them has made investigation difficult. In this work, we seek ways to reproduce and study training stability and instability at smaller scales. First, we focus on two sources of training instability described in previous work: the growth of logits in attention layers (Dehghani et al., 2023) and divergence of the output logits from the log probabilities (Chowdhery et al., 2022). By measuring the relationship between learning rate and loss across scales, we show that these instabilities also appear in small models when training at high learning rates, and that mitigations previously employed at large scales are equally effective in this regime. This prompts us to investigate the extent to which other known optimizer and model interventions influence the sensitivity of the final loss to changes in the learning rate. To this end, we study methods such as warm-up, weight decay, and the muParam (Yang et al., 2022), and combine techniques to train small models that achieve similar losses across orders of magnitude of learning rate variation. Finally, to conclude our exploration we study two cases where instabilities can be predicted before they emerge by examining the scaling behavior of model activation and gradient norms.

  • 16 authors
·
Sep 25, 2023 2

JAWS: Enhancing Long-term Rollout of Neural Operators via Spatially-Adaptive Jacobian Regularization

Data-driven surrogate models improve the efficiency of simulating continuous dynamical systems, yet their autoregressive rollouts are often limited by instability and spectral blow-up. While global regularization techniques can enforce contractive dynamics, they uniformly damp high-frequency features, introducing a contraction-dissipation dilemma. Furthermore, long-horizon trajectory optimization methods that explicitly correct drift are bottlenecked by memory constraints. In this work, we propose Jacobian-Adaptive Weighting for Stability (JAWS), a probabilistic regularization strategy designed to mitigate these limitations. By framing operator learning as Maximum A Posteriori (MAP) estimation with spatially heteroscedastic uncertainty, JAWS dynamically modulates the regularization strength based on local physical complexity. This allows the model to enforce contraction in smooth regions to suppress noise, while relaxing constraints near singular features to preserve gradients, effectively realizing a behavior similar to numerical shock-capturing schemes. Experiments demonstrate that this spatially-adaptive prior serves as an effective spectral pre-conditioner, which reduces the base operator's burden of handling high-frequency instabilities. This reduction enables memory-efficient, short-horizon trajectory optimization to match or exceed the long-term accuracy of long-horizon baselines. Evaluated on the 1D viscous Burgers' equation, our hybrid approach improves long-term stability, shock fidelity, and out-of-distribution generalization while reducing training computational costs.

  • 2 authors
·
Mar 4

Which Invariance Should We Transfer? A Causal Minimax Learning Approach

A major barrier to deploying current machine learning models lies in their non-reliability to dataset shifts. To resolve this problem, most existing studies attempted to transfer stable information to unseen environments. Particularly, independent causal mechanisms-based methods proposed to remove mutable causal mechanisms via the do-operator. Compared to previous methods, the obtained stable predictors are more effective in identifying stable information. However, a key question remains: which subset of this whole stable information should the model transfer, in order to achieve optimal generalization ability? To answer this question, we present a comprehensive minimax analysis from a causal perspective. Specifically, we first provide a graphical condition for the whole stable set to be optimal. When this condition fails, we surprisingly find with an example that this whole stable set, although can fully exploit stable information, is not the optimal one to transfer. To identify the optimal subset under this case, we propose to estimate the worst-case risk with a novel optimization scheme over the intervention functions on mutable causal mechanisms. We then propose an efficient algorithm to search for the subset with minimal worst-case risk, based on a newly defined equivalence relation between stable subsets. Compared to the exponential cost of exhaustively searching over all subsets, our searching strategy enjoys a polynomial complexity. The effectiveness and efficiency of our methods are demonstrated on synthetic data and the diagnosis of Alzheimer's disease.

  • 5 authors
·
Jul 5, 2021

High-Fidelity Digital Twin Dataset Generation for Inverter-Based Microgrids Under Multi-Scenario Disturbances

Public power-system datasets often lack electromagnetic transient (EMT) waveforms, inverter control dynamics, and diverse disturbance coverage, which limits their usefulness for training surrogate models and studying cyber-physical behavior in inverter-based microgrids. This paper presents a high-fidelity digital twin dataset generated from a MATLAB/Simulink EMT model of a low-voltage AC microgrid with ten inverter-based distributed generators. The dataset records synchronized three-phase PCC voltages and currents, per-DG active power, reactive power, and frequency, together with embedded scenario labels, producing 38 aligned channels sampled at Δt = 2~μs over T = 1~s (N = 500{,}001 samples) per scenario. Eleven operating and disturbance scenarios are included: normal operation, load step, voltage sag (temporary three-phase fault), load ramp, frequency ramp, DG trip, tie-line trip, reactive power step, single-line-to-ground faults, measurement noise injection, and communication delay. To ensure numerical stability without altering sequence length, invalid samples (NaN, Inf, and extreme outliers) are repaired using linear interpolation. Each scenario is further validated using system-level evidence from mean frequency, PCC voltage magnitude, total active power, voltage unbalance, and zero-sequence current to confirm physical observability and correct timing. The resulting dataset provides a consistent, labeled EMT benchmark for surrogate modeling, disturbance classification, robustness testing under noise and delay, and cyber-physical resilience analysis in inverter-dominated microgrids. The dataset and processing scripts will be released upon acceptance

  • 3 authors
·
Mar 9