physics-ai999x — offline, verified physics solver with step-by-step answers

Install · Capability guide · ONNX models · Evaluation · AI999X on Hugging Face · Website · Support: Buy Me a Coffee

License: Apache-2.0 · Author: Ibnul Jaif Farabi (creator and engineer) · Organization: AI999X · Version 2.0.0

physics-ai999x is an offline, deterministic Python solver that answers physics questions with verified results from 148 certified capabilities, gives 859 documented answers identically on Windows and Linux, and refuses rather than guesses when an answer cannot be verified.

physics-ai999x is a compact, deterministic physics engine for Python. It reads a question in plain English (or a typed call), picks a certified physics capability, computes the answer exactly, checks it, and returns the result with its steps, units, LaTeX and assumptions — or an honest refusal that says what is missing. It runs fully offline on any CPU, needs no GPU, no API key and no internet, and is built to give zero wrong answers: when it is not sure, it refuses instead of guessing.

Buy me a coffee License Python Offline

Highlights

  • Verified, not generated. Every answer is computed by a certified physics capability and checked (substituted back, dimension-checked or compared with an independent computation) before it is returned.
  • Zero wrong answers by design. Seven explicit outcomes — SOLVED, NEEDS_INPUT, AMBIGUOUS, INVALID_INPUT, OUT_OF_SCOPE, UNRECOGNIZED_FORMAT, INTERNAL_ERROR — never a silent guess. Measured: 0 wrong answers on 274 benchmark questions and on 1,567 verified natural-language questions.
  • Plain-English questions and a guaranteed typed interface. ask() reads documented question forms; solve(capability_id, **inputs) always solves a supported task with valid inputs.
  • Explains itself. Steps, formulas in LaTeX, units, assumptions and the model used, ready for notebooks, tutoring apps and homework checkers.
  • ONNX included. 365 deterministic formula graphs (onnx/formulas/, 94.7 KB) + AI999X capability encoder (onnx/search/encoder.int8.onnx, 4.9 MB int8) + decision model (onnx/decision.int8.onnx, 37.4 MB int8): run the exact formulas without Python — onnxruntime in C++, C#, Java, JavaScript or the browser; 13,256 of 13,256 checks agree with the Python engine.
  • Small and portable. Pure Python (sympy, mpmath); identical answers on Windows and Linux (Python 3.10, 3.11, 3.12, 3.13, 3.14).
  • Coverage: kinematics, forces and Newton's laws, work and energy, momentum and collisions, circular motion and gravitation, rotation, oscillations and waves, fluids, heat and thermodynamics, electricity and circuits, magnetism and induction, optics and modern physics.

Model overview

Property Value
Package physics-ai999x 2.0.0 (import physics_ai999x)
Type Deterministic neuro-symbolic physics engine (rule-based reading + certified symbolic capabilities)
Capabilities 23 certified derivations, 16 certified model_familys, 2 certified procedures, 107 certified relations
Knowledge pack version 0.6.0
Golden answers 859 documented answers, identical on every tested platform
Platforms tested Windows (Python 3.13), Linux x86-64 (Python 3.10, 3.11, 3.12, 3.13, 3.14)
ONNX 365 deterministic formula graphs (onnx/formulas/, 94.7 KB) + AI999X capability encoder (onnx/search/encoder.int8.onnx, 4.9 MB int8) + decision model (onnx/decision.int8.onnx, 37.4 MB int8)
Runtime CPU only, offline; no GPU, no network, no API key
License Apache-2.0
Creator / engineer Ibnul Jaif Farabi
Organization AI999X

Quickstart

pip install physics-ai999x
# or straight from this repository:
pip install https://huggingface.co/AI999X/physics-ai999x/resolve/main/dist/physics_ai999x-2.0.0-py3-none-any.whl
import physics_ai999x as physics

physics.ask("A 1000 kg car moving at 20 m/s hits a stationary 500 kg car and they stick together. What is their final velocity?").show()
# -> SOLVED — 13.3333 m/s  (collision_1d)
physics.ask("How much heat boils away 0.5 kg of water? Its latent heat of vaporization is 2260000 J/kg.").show()
# -> SOLVED — 1.13e+06 J  (latent_heat_vaporization)
physics.ask("A hockey puck slides across ice at 10 m/s and slows down at a constant rate of 0.5 m/s^2. What is its speed after 8 seconds?").show()
# -> SOLVED — 6 m/s  (velocity_time)
physics.solve("kinetic_energy", mass="3 kg", speed="4 m/s").show()
# -> SOLVED — 24 J  (kinetic_energy)
physics.ask("What is the kinetic energy of a car moving at 20 m/s?").show()
# -> NEEDS_INPUT — Translational kinetic energy applies, but it needs: mass (kg). Add it to the question with its unit, or use so
physics.ask("Who created you?").message
# -> physics-ai999x was created and engineered by Ibnul Jaif Farabi at AI999X.

Command line: physics-ai999x "<question>", physics-ai999x --capabilities. The outputs above were produced by running this exact code on physics-ai999x 2.0.0.

Find the right capability, and draw it

physics.search() finds which certified capability a question is about — across every AI999X model, so a question for another model points you to it. It runs ai999x-encoder, AI999X's own capability encoder (4.7M parameters, a 4.9 MB ONNX model, offline; the root model.safetensors of this repository): pip install "physics-ai999x[search]". ask(question, suggest=True) adds the same pointers to a question it does not answer; an answer itself never changes.

import physics_ai999x as physics

physics.search("How long does a pendulum take to swing back and forth?", k=2)
# -> Period of a simple pendulum (physics-ai999x, 0.78); Precession period of a Foucault pendulum (mechanics-ai999x, 0.68)
physics.search("Solve x^2 - 5x + 6 = 0", k=2)
# -> Roots of a quadratic equation (algebra-ai999x, 0.67); Solution of two linear equations in two unknowns (algebra-ai999x, 0.57)

How every answer is verified

  1. Reading — the question is turned into typed quantities (value, unit, kind, role) and one asked quantity; every stated number must be used or the question is refused.
  2. Planning — only certified capabilities may answer; a capability applies only under the conditions the wording states (for example a pendulum, a liquid at rest, boiling).
  3. Computing — exact symbolic or high-precision numeric evaluation.
  4. Checking — dimensions, substitution back into the defining equation, and agreement between independent readings; any disagreement becomes a refusal, never a number.

Capabilities are certified before they ship: each must reproduce hand-worked reference cases, reject a deliberately wrong variant (a control), and leave every benchmark answer unchanged.

Benchmarks

Evaluation set Questions Correct Refused (with reason) Wrong
Development probes 112 68 44 0
Held-out: fresh questions (2026-10-15) 42 32 10 0
Held-out: probes 34 30 4 0
Held-out: promotion checks (2026-10-13) 27 14 13 0
Held-out: multi-body systems (2026-10-15) 43 33 10 0
Held-out: unused-input traps 16 9 7 0
Verified natural-language questions (physics-questions-1) 1567 478 1086 0

Wrong means a number that disagrees with the verified answer. On the verified-question set, 3 of 481 answers differ from the record: two are unit spellings (ohm/ohms) or a self-contradictory question ('angular frequency in Hz'), one pairs two records that share an id; none is a reading error. Refusals are expected for questions outside the certified capabilities or with unusual wording; solve() covers every capability with typed inputs.

Optional decision model (auto (gated) + suggest)

This repository also holds a compact ONNX decision model (onnx/, 37.4 MB int8, 33.4M parameters, CPU; fine-tuned from BAAI/bge-small-en-v1.5): it reads a question the rule-based reader refuses, names the capability it needs and copies the input values from the words (it never writes a digit itself).

When it answers by itself: only through a gate — confidence at or above the zero-wrong cut chosen on validation with this very file, every required input read, every number in the question used, and the engine's verification; the asked quantity read by the rule-based reader (never inferred from which value is missing), every input copied with its unit, no direction or sign word in the question, and the relation's context conditions. The answer says it was read by the model. Otherwise it suggests the capability and a pre-filled call for you to check.

Measured (held-out questions the rule-based reader refuses) Value
Right capability named 157 of 159 named right (227 held-out questions the rule-based reader refuses)
Input values copied through the gate: 19 answered automatically on test, 0 wrong; 201 get a suggestion
CPU latency, p95 47.2 ms (4 threads)

Through the gate it answered 19 held-out test and 22 validation questions that the rule-based reader refuses, with 0 wrong. On 1,377 rejected teacher questions it answered 41 distinct questions by itself, all correct by hand calculation (in every disagreement the teacher's key was wrong), and the 88 benchmark questions the reader refuses stay refused.

pip install "physics-ai999x[onnx]"
hf download AI999X/physics-ai999x --include "onnx/*" --local-dir physics_model
import physics_ai999x as physics
reader = physics.DecisionReader("physics_model/onnx")
physics.ask("A 0.25 m rod generates 5 V while sliding in a 1 T field. How fast is the rod moving?", reader=reader).value   # -> 20.0 (m/s), read by the model, verified by the engine
physics.ask("quick one: spring 500 N/m, mass 0.01 kg, amplitude 0.004 m, top speed?", reader=reader).message
# -> ... Suggestion (decision model, not verified): this looks like `shm_max_speed` ... run physics_ai999x.shm_max_speed(k='500 N/m', m='0.01 kg', X='0.004 m')

Why the gate: the engine verifies a computation, not a reading. A copied number carries no sign, and the model does not decide what is asked — the rule-based reader names the asked quantity, and a question that sets directions against each other goes to a suggestion. Before that check, one held-out question ('downward at 2.0 m/s ... upward acceleration') was answered 8 m/s instead of 4 m/s upward.

ONNX formula pack (deterministic, no Python needed)

onnx/formulas/ holds 365 tiny ONNX graphs (94.7 KB in total): every certified closed-form capability compiled from its exact formula — not learned, so there is nothing approximate in them. Float64, standard operators only; run them with onnxruntime in Python, C++, C#, Java or in the browser (onnxruntime-web).

Verified against the Python engine: 13,256 of 13,256 checks agree, worst relative error 9.9e-15. onnx/formulas/formulas.json lists each graph's inputs (SI units), outputs and, where a formula has several solution branches, which one to take.

import numpy as np, onnxruntime as ort
s = ort.InferenceSession("onnx/formulas/uniform_motion__d.onnx")
print(s.run(None, {"v": np.array(2.0), "t": np.array(2.0)}))
import * as ort from "onnxruntime-web";
const s = await ort.InferenceSession.create("onnx/formulas/uniform_motion__d.onnx");
const out = await s.run({ v: new ort.Tensor("float64", new Float64Array([2.0]), []), t: new ort.Tensor("float64", new Float64Array([2.0]), []) });

Find the exact formula, never guess (autocomplete): onnx/formulas/catalog.json is a search index of every graph (names, keywords, the unknown it solves for, inputs with SI units, LaTeX). formula-pack.js searches it, describes an entry and runs the exact graph with the branch rule; formula-pack.d.ts gives your editor every formula id and input name.

import { loadPack } from "./formula-pack.js";
const pack = await loadPack("https://huggingface.co/AI999X/physics-ai999x/resolve/main/onnx/formulas");
pack.search("kinetic energy speed");   // best match first: the formula and the unknown it solves for

Guidebook: docs/physics_ai999x_onnx_formulas.md (also HTML and PDF) documents every graph — LaTeX, inputs and units, branches, conditions — each with an example executed on the ONNX file and compared with the Python engine.

Reading questions in words stays in the Python package (and the optional decision model); the formula pack is the compute half, for apps, calculators and edge devices.

Capabilities

23 certified derivations, 16 certified model_familys, 2 certified procedures, 107 certified relations. A few of them:

  • Derivations: acceleration with friction level; angular acceleration from tangential; angular frequency period; average speed equal instantaneous; centripetal acceleration from period; dielectric increases capacitance; displacement from velocity time; distance fallen time; energy via resistance; final speed from work; focal length; max height from velocity displacement; net work to speed; period from wave speed; power from voltage and resistance; solve for g; speed after height; speed after time dropped; speed from uniform motion; temp change inverse mass; time to stop; work to speed rest; zero initial speed
  • Model_familys: A gas between two states; Apparent weight in an elevator; Block on a surface pulled over a pulley; Collisions along a line; Doppler effect for sound; Ideal banking of a curve; Maximum speed on a flat curve; Mixing to thermal equilibrium; Motion in stages; Projectile motion; RC circuits; Radioactive decay; Recoil from rest; Rolling from a height; Series and parallel resistor and capacitor networks; Two masses over a pulley
  • Procedures: Two resistors in series and in parallel; Velocity and displacement after constant acceleration
  • Relations: Absolute pressure at a depth in a fluid; Acceleration down a frictionless incline; Angular frequency and frequency; Angular momentum of a point mass on a circle; Angular momentum of a rotating body; Angular velocity and angle turned under constant angular acceleration; Angular velocity under constant angular acceleration; Average net force from a change of momentum; Average power; Capacitance of parallel plates; Carnot (maximum) efficiency; Centripetal acceleration; Centripetal acceleration from angular velocity; Centripetal force; Charge carried by a steady current; Charge on a capacitor; Coefficient of performance of a heat pump; Coefficient of performance of a refrigerator; Component of the weight along an incline; Current dying away in an RL circuit; De Broglie wavelength; Displacement from the mean of initial and final velocity; Displacement under constant acceleration; Efficiency of a heat engine …

physics_ai999x.templates() lists every question form ask() reads; the capability guide (PDF, every example executed) is included in this repository.

All 109 formulas

Show every formula, by topic

AC circuits

Reactance of a capacitor

X=12πfC X = \frac{1}{2 \pi f C}

Reactance of an inductor

X=2πfL X = 2 \pi f L

Resonant frequency of an LC circuit

f=12πLC f = \frac{1}{2 \pi \sqrt{L C}}

Rms current of a sinusoidal supply

Irms=I02 I_{\text{rms}} = \frac{I_{0}}{\sqrt{2}}

Rms voltage of a sinusoidal supply

Vrms=V02 V_{\text{rms}} = \frac{V_{0}}{\sqrt{2}}

Capacitors

Capacitance of parallel plates

C=ε0Ad C = \frac{\varepsilon_{0} A}{d}

Charge on a capacitor

Q=CV Q = C V

Energy stored in a capacitor

Ecap=CV22 \text{Ecap} = \frac{C V^{2}}{2}

Circuits

Power from current and resistance

P=I2R P = I^{2} R

Power from voltage and resistance

P=V2R P = \frac{V^{2}}{R}

Resistance of a uniform wire

R=ρeLA R = \frac{\rho_{e} L}{A}

Two resistors in series and in parallel

rs=r1+r2,rp(r1+r2)=r1r2 r_{s} = r_{1} + r_{2},\quad r_{p} \left(r_{1} + r_{2}\right) = r_{1} r_{2}

Circular

Angular frequency and frequency

ω=2πf \omega = 2 \pi f

Centripetal acceleration

a=v2r a = \frac{v^{2}}{r}

Centripetal force

F=mv2r F = \frac{m v^{2}}{r}

Period and frequency

T=1f T = \frac{1}{f}

Speed on a circle from angular speed

v=ωr v = \omega r

Dynamics

Acceleration down a frictionless incline

a=gsin⁡(θ) a = g \sin\left(\theta\right)

Average net force from a change of momentum

F=p2−p1t F = \frac{p_2 - p_1}{t}

Component of the weight along an incline

F=mgsin⁡(θ) F = m g \sin\left(\theta\right)

Impulse of a constant force

J=Ft J = F t

Linear momentum

p=mv p = m v

Net force and acceleration

F=ma F = m a

Net force with an opposing friction force

F=P−f F = P - f

Normal force on a level surface

N=W N = W

Normal force on an incline

N=mgcos⁡(θ) N = m g \cos\left(\theta\right)

Sliding friction

f=μN f = \mu N

Tension in a rope holding a hanging body

T=mg T = m g

Weight in a uniform gravitational field

W=mg W = m g

Electricity

Charge carried by a steady current

Q=It Q = I t

Electric power

P=VI P = V I

Voltage across an ohmic resistance

V=IR V = I R

Electrostatics

Electric field of a point charge

Ef=keQr2 E_{f} = \frac{k_{e} Q}{r^{2}}

Electric potential of a point charge

V=keQr V = \frac{k_{e} Q}{r}

Force on a charge in an electric field

F=qEf F = q E_{f}

Potential difference in a uniform field

V=Efd V = E_{f} d

Energy

Average power

P=Et P = \frac{E}{t}

Energy stored in a linear spring

U=kx22 U = \frac{k x^{2}}{2}

Force of a linear spring

F=kx F = k x

Gravitational potential energy near the surface

U=mgh U = m g h

Power of a force on a moving body

P=Fv P = F v

Speed of a body launched by a spring

kx22=mv22 \frac{k x^{2}}{2} = \frac{m v^{2}}{2}

Translational kinetic energy

K=mv22 K = \frac{m v^{2}}{2}

Work by a constant force

W=Fdcos⁡(θ) W = F d \cos\left(\theta\right)

Work-energy theorem

W=m(v2−v02)2 W = \frac{m \left(v^{2} - v_0^{2}\right)}{2}

Fluids

Absolute pressure at a depth in a fluid

P=P0+ρgh P = P_{0} + \rho g h

Flow rate as volume per time

Q=Vt Q = \frac{V}{t}

Flow rate through a cross section

Q=Av Q = A v

Mass density

ρ=mV \rho = \frac{m}{V}

Pressure of a column of fluid

P=ρgh P = \rho g h

Pressure of a force on an area

P=FA P = \frac{F}{A}

Gravitation

Gravitational acceleration at a distance from a mass

g=GMr2 g = \frac{G M}{r^{2}}

Period of a circular orbit

T=2πr3GM T = 2 \pi \sqrt{\frac{r^{3}}{G M}}

Speed in a circular orbit

v=GMr v = \sqrt{\frac{G M}{r}}

Induction

Current dying away in an RL circuit

I=I0e−tτ I = I_{0} e^{\frac{-t}{\tau}}

Emf of a rod moving through a magnetic field

emf=Blv \text{emf} = B l v

Time constant of an RL circuit

τ=LR \tau = \frac{L}{R}

Kinematics

Displacement from the mean of initial and final velocity

x=(v0+v)t2 x = \frac{\left(v_0 + v\right) t}{2}

Displacement under constant acceleration

x=v0t+at22 x = v_0 t + \frac{a t^{2}}{2}

Maximum height of a vertical throw

h=v022g h = \frac{v_0^{2}}{2 g}

Motion at constant velocity

d=vt d = v t

Speed after a displacement under constant acceleration

v2=v02+2ax v^{2} = v_0^{2} + 2 a x

Speed after falling from rest

v=2gh v = \sqrt{2 g h}

Time to the top of a vertical throw

t=v0g t = \frac{v_0}{g}

Velocity and displacement after constant acceleration

v=u+at,2s=(u+v)t v = u + a t,\quad 2 s = \left(u + v\right) t

Velocity under constant acceleration

v=v0+at v = v_0 + a t

Magnetism

Magnetic field at the centre of a circular loop

B=μ0I2r B = \frac{\mu_{0} I}{2 r}

Magnetic field inside a long solenoid

B=μ0ntI B = \mu_{0} n_{t} I

Magnetic field of a long straight wire

B=μ0I2πr B = \frac{\mu_{0} I}{2 \pi r}

Magnetic force on a current-carrying wire

F=ILBsin⁡(θ) F = I L B \sin\left(\theta\right)

Magnetic force on a moving charge

F=qvBsin⁡(θ) F = q v B \sin\left(\theta\right)

Radius of a charged particle's circle in a magnetic field

r=mvqBsin⁡(θ) r = \frac{m v}{q B \sin\left(\theta\right)}

Optics

Focal length of a spherical mirror

f=R2 f = \frac{R}{2}

Index of refraction

n=cv n = \frac{c}{v}

Thin lens equation

1do+1di=1f \frac{1}{d_{o}} + \frac{1}{d_{i}} = \frac{1}{f}

Wavelength of light in a medium

λn=λn \lambda_{n} = \frac{\lambda}{n}

Oscillations

Maximum speed of a mass on a spring

v=Xkm v = X \sqrt{\frac{k}{m}}

Period of a mass on a spring

T=2πmk T = 2 \pi \sqrt{\frac{m}{k}}

Period of a simple pendulum

T=2πLpg T = 2 \pi \sqrt{\frac{L_{p}}{g}}

Quantum

De Broglie wavelength

λ=hp \lambda = \frac{h}{p}

Energy of a photon

E=hf E = h f

Momentum of a photon

p=hλ p = \frac{h}{\lambda}

Photon energy from wavelength

E=hcλ E = \frac{h c}{\lambda}

RC circuits

Time constant of an RC circuit

τ=RC \tau = R C

Voltage of a charging capacitor

V=emf(1−e−tRC) V = \text{emf} \left(1 - e^{\frac{-t}{R C}}\right)

Voltage of a discharging capacitor

V=V0e−tRC V = V_{0} e^{\frac{-t}{R C}}

Relativity

Rest energy

E0=mc2 E_{0} = m c^{2}

Rotation

Angular momentum of a point mass on a circle

L=mvr L = m v r

Angular momentum of a rotating body

L=Irω L = I_{r} \omega

Angular velocity and angle turned under constant angular acceleration

ω2=ω02+2αθ \omega^{2} = \omega_{0}^{2} + 2 \alpha \theta

Angular velocity under constant angular acceleration

ω=ω0+αt \omega = \omega_{0} + \alpha t

Centripetal acceleration from angular velocity

a=rω2 a = r \omega^{2}

Rotational kinetic energy

KR=Irω22 K_{R} = \frac{I_{r} \omega^{2}}{2}

Tangential acceleration of a point on a rotating body

a=rα a = r \alpha

Torque of a force

τ=rFsin⁡(θ) \tau = r F \sin\left(\theta\right)

Thermal

Carnot (maximum) efficiency

eff=1−TcTh \text{eff} = 1 - \frac{T_{c}}{T_{h}}

Coefficient of performance of a heat pump

cop=QhW \text{cop} = \frac{Q_{h}}{W}

Coefficient of performance of a refrigerator

cop=QcW \text{cop} = \frac{Q_{c}}{W}

Efficiency of a heat engine

eff=WQh \text{eff} = \frac{W}{Q_{h}}

Energy balance of a cyclic heat engine

W=Qh−Qc W = Q_{h} - Q_{c}

Heat for a temperature change

Q=mcΔT Q = m c \Delta T

Heat to boil or condense

Q=mLv Q = m L_{v}

Heat to melt or freeze

Q=mLf Q = m L_{f}

Ideal gas law

PV=nRT P V = n R T

Linear thermal expansion

dL=αtLdT d_{L} = \alpha_{t} L d_{T}

Thermal expansion of an area

dA=2αtAdT d_{A} = 2 \alpha_{t} A d_{T}

Thermal expansion of volume

dV=βV0dT d_{V} = \beta V_{0} d_{T}

Waves

Intensity as power per area

I=PA I = \frac{P}{A}

Speed of a periodic wave

v=fλ v = f \lambda

Out of scope

  • Open-ended explanation, essay writing or tutoring dialogue: it is a solver, not a chatbot.
  • Questions outside its certified physics capabilities, and any question whose stated numbers it cannot all use — it refuses with the reason instead.
  • High-stakes engineering, medical or legal decisions without independent review.

Limitations

  • It answers what its certified capabilities cover (kinematics, forces and Newton's laws, work and energy, momentum and collisions, circular motion and gravitation, rotation, oscillations and waves, fluids, heat and thermodynamics, electricity and circuits, magnetism and induction, optics and modern physics); anything else is refused with a reason.
  • ask() reads documented question forms and common phrasings; unusual wording may be refused — use solve() for guaranteed results.
  • Results are as correct as the certified relations and their stated conditions; a certified relation is a checked model, not a claim about every physical situation.

Citation

@software{physics_ai999x_2_0_0,
  title  = {physics-ai999x: an offline, verified physics solver},
  author = {Farabi, Ibnul Jaif}, organization = {AI999X},
  year   = {2026}, version = {2.0.0},
  url    = {https://huggingface.co/AI999X/physics-ai999x}
}

Contact

Questions, bug reports and requests: open a discussion on https://huggingface.co/AI999X/physics-ai999x/discussions or visit https://ai999x.com.

Support the project

physics-ai999x is free and open source. If it helps you, you can support further development on Buy Me a Coffee.

Created and engineered by Ibnul Jaif Farabi · AI999X · ai999x.com

Downloads last month
64
Safetensors
Model size
4.72M params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Collection including AI999X/physics-ai999x

Evaluation results

  • correct (of all questions) on Development probes
    self-reported
    0.607
  • wrong answers on Development probes
    self-reported
    0.000
  • correct (of all questions) on Held-out: fresh questions (2026-10-15)
    self-reported
    0.762
  • wrong answers on Held-out: fresh questions (2026-10-15)
    self-reported
    0.000
  • correct (of all questions) on Held-out: probes
    self-reported
    0.882
  • wrong answers on Held-out: probes
    self-reported
    0.000
  • correct (of all questions) on Held-out: promotion checks (2026-10-13)
    self-reported
    0.518
  • wrong answers on Held-out: promotion checks (2026-10-13)
    self-reported
    0.000