license: mit
library_name: pytorch
tags:
- symbolic-mathematics
- jepa
- representation-learning
- symbolic-regression
- function-encoder
Func-JEPA
Pretrained weights of Func-JEPA: Learning a Sobolev Space of Functions as Latent Vectors (Alejandro Moreno Arcas, 2026). Code: https://github.com/amorarc/Func-JEPA. Data: https://huggingface.co/datasets/amorarc/func-jepa-data.
Func-JEPA encodes the symbolic form of a univariate function into a vector of R^256 whose norm is the exact H¹ norm of the function and whose dot products approximate H¹ inner products. Differentiation is a learned latent operator. Small modules on the frozen encoder decode latents to formulas, compose and integrate in latent space, and map sampled values to latents.
Usage
pip install git+https://github.com/amorarc/Func-JEPA
import numpy as np
from fjepa import FuncJEPA
m = FuncJEPA.from_pretrained("amorarc/func-jepa")
z = m.encode("4*sin(7*x) - 3*cos(7*x)")
m.norm(z) # 12.87, the H1 norm of f on [-pi, pi]
m.decode(m.derivative(z))[0] # 21*sin(7*x) + 28*cos(7*x)
m.regress(32 * np.cos(6 * m.nodes) + 8) # 32*cos(6*x) + 8, from 64 samples
Files
| Folder | Parameters | Role |
|---|---|---|
encoder_pi |
35.1M | encoder + derivative operator on [-π, π] (all results of the paper) |
encoder_05 |
35.1M | the same model trained on [0, 5] |
decoder |
17.7M | latent → prefix tokens |
compose |
52.9M | composition operators: sin, cos, log, sqrt, square, cube, quadratic |
integrator |
7.6M | latent of f → latent of the zero-mean antiderivative |
value_encoder |
7.4M | 64 samples → latent direction |
decoder_lc, value_encoder_lc |
17.7M, 7.4M | variants fine-tuned for the Lample–Charton ODE benchmark |
Each folder holds model.safetensors and config.json; SHA256.json lists the hashes.
Training
Synthetic data only: 2.0M functions from 17 parameterised families and short random expression trees, generated with
SymPy. Objective: H¹ angle matching, linearity, and a JEPA term predicting the latent of f' from that of f. One
training run of 40k steps on a single 8 GB GPU (6.6 hours); the modules were trained afterwards on the frozen encoder.
Results (from the paper, single run)
- Latent cosines and dot products correlate 0.993 and 0.998 with their H¹ counterparts.
- The derivative operator retrieves
f'among 3,956 candidates for 82.0% of test functions. - Exact symbolic regression from 64 samples: 76% / 76% / 62% at noise 0 / 0.1% / 1% on 50 test functions.
- Symbolic integration on the paper's test set: 91.7% verified by SymPy (full solver, which needs the training data).
Limitations
- Latent operators are approximate (median relative error of the derivative: 7%). Predicted latents decode to the right shape more often than to exact constants; refit and verify the candidates.
- Weak outside the pretraining families: 20–25% on the integration test sets of Lample and Charton and 16–25% on their ODE test sets, where task-specific models exceed 90%.
- The symbolic-regression advantage is measured on functions of the pretraining families.
- Univariate functions on a fixed interval, 37-token vocabulary without special functions, at most 126 tokens.
- Single training run: variability across seeds was not measured.
Citation
@article{morenoarcas2026funcjepa,
title = {Func-JEPA: Learning a Sobolev Space of Functions as Latent Vectors},
author = {Moreno Arcas, Alejandro},
year = {2026}
}