Func-JEPA

Pretrained weights of Func-JEPA: Learning a Sobolev Space of Functions as Latent Vectors (Alejandro Moreno Arcas, 2026). Code: https://github.com/amorarc/Func-JEPA. Data: https://huggingface.co/datasets/amorarc/func-jepa-data.

Func-JEPA encodes the symbolic form of a univariate function into a vector of R^256 whose norm is the exact H¹ norm of the function and whose dot products approximate H¹ inner products. Differentiation is a learned latent operator. Small modules on the frozen encoder decode latents to formulas, compose and integrate in latent space, and map sampled values to latents.

Usage

pip install git+https://github.com/amorarc/Func-JEPA

import numpy as np
from fjepa import FuncJEPA

m = FuncJEPA.from_pretrained("amorarc/func-jepa")
z = m.encode("4*sin(7*x) - 3*cos(7*x)")
m.norm(z)                          # 12.87, the H1 norm of f on [-pi, pi]
m.decode(m.derivative(z))[0]       # 21*sin(7*x) + 28*cos(7*x)
m.regress(32 * np.cos(6 * m.nodes) + 8)     # 32*cos(6*x) + 8, from 64 samples

Files

Folder	Parameters	Role
encoder_pi	35.1M	encoder + derivative operator on [-π, π] (all results of the paper)
encoder_05	35.1M	the same model trained on [0, 5]
decoder	17.7M	latent → prefix tokens
compose	52.9M	composition operators: sin, cos, log, sqrt, square, cube, quadratic
integrator	7.6M	latent of f → latent of the zero-mean antiderivative
value_encoder	7.4M	64 samples → latent direction
decoder_lc, value_encoder_lc	17.7M, 7.4M	variants fine-tuned for the Lample–Charton ODE benchmark

Each folder holds model.safetensors and config.json; SHA256.json lists the hashes.

Training

Synthetic data only: 2.0M functions from 17 parameterised families and short random expression trees, generated with SymPy. Objective: H¹ angle matching, linearity, and a JEPA term predicting the latent of f' from that of f. One training run of 40k steps on a single 8 GB GPU (6.6 hours); the modules were trained afterwards on the frozen encoder.

Results (from the paper, single run)

• Latent cosines and dot products correlate 0.993 and 0.998 with their H¹ counterparts.
• The derivative operator retrieves f' among 3,956 candidates for 82.0% of test functions.
• Exact symbolic regression from 64 samples: 76% / 76% / 62% at noise 0 / 0.1% / 1% on 50 test functions.
• Symbolic integration on the paper's test set: 91.7% verified by SymPy (full solver, which needs the training data).

Limitations

• Latent operators are approximate (median relative error of the derivative: 7%). Predicted latents decode to the right shape more often than to exact constants; refit and verify the candidates.
• Weak outside the pretraining families: 20–25% on the integration test sets of Lample and Charton and 16–25% on their ODE test sets, where task-specific models exceed 90%.
• The symbolic-regression advantage is measured on functions of the pretraining families.
• Univariate functions on a fixed interval, 37-token vocabulary without special functions, at most 126 tokens.
• Single training run: variability across seeds was not measured.

Citation

@article{morenoarcas2026funcjepa,
  title  = {Func-JEPA: Learning a Sobolev Space of Functions as Latent Vectors},
  author = {Moreno Arcas, Alejandro},
  year   = {2026}
}