ai999x-encoder — a 4.9 MB encoder that finds the right capability for a science, mathematics or engineering question

AI999X on Hugging Face · Capability orb · Website

License: Apache-2.0 · Author: Ibnul Jaif Farabi (creator and engineer) · Organization: AI999X · 4,715,520 parameters · version 3

ai999x-encoder turns a question written in plain English into a 256-number vector and finds which AI999X capability it is about — a formula, a procedure, an algorithm — among 2,696 capabilities of 31 models at once. It is small enough to run offline on any CPU, on a phone and in the browser, and it powers search() in every AI999X package and the search on the capability orb.

Results

Held-out questions. 12,998 verified questions it never saw in training, across all 31 models; the task: put the right one of 2,696 capabilities first.

Model	Right capability first	In the top five
ai999x-encoder v3 (4.7M parameters, 4.9 MB)	91.8%	98.8%
A commercial embedding API (its teacher)	85.0%	97.6%
bge-small-en-v1.5 (33M parameters)	80.1%	95.0%

Stress set. 10,800 questions from 27 models, written in 12 different styles (beginner, expert, story, terse, two-part, negation, …) and never used for training or tuning.

Model	Right capability first	In the top five
ai999x-encoder v3	89.2%	98.8%
ai999x-encoder v2 (previous version)	50.9%	73.8%

By wording style (v3)

Style	First	Top five
asked-first	93.8%	100.0%
asked-last	90.9%	99.3%
beginner	78.8%	94.6%
comparison	88.6%	99.1%
expert	88.5%	98.9%
imperative	92.4%	99.8%
list-like	91.5%	99.4%
negation	91.0%	100.0%
story	87.9%	98.6%
terse	87.4%	98.2%
two-part	91.6%	98.9%
verbose	87.3%	98.7%

By field (held-out questions)

Field	Questions	First	Top five
Mathematics & Statistics	4,089	88.4%	97.3%
Physical Sciences	2,721	96.1%	100.0%
Engineering	4,006	94.9%	99.8%
Life Sciences	803	94.1%	99.9%
Economics & Finance	416	86.3%	98.3%
Geospatial	448	78.8%	95.1%
Data Science & Machine Learning	515	85.0%	97.5%

By model (held-out questions)

Model	Field	First	Top five
bayesian-statistics-ai999x	Mathematics & Statistics	94.9%	99.2%
calculus-ai999x	Mathematics & Statistics	95.5%	100.0%
differential-equations-ai999x	Mathematics & Statistics	93.1%	99.5%
discrete-math-ai999x	Mathematics & Statistics	88.3%	98.2%
geometry-ai999x	Mathematics & Statistics	88.9%	98.3%
linear-algebra-ai999x	Mathematics & Statistics	95.0%	99.5%
math-foundations-ai999x	Mathematics & Statistics	88.1%	98.5%
optimization-ai999x	Mathematics & Statistics	93.2%	99.4%
statistics-ai999x	Mathematics & Statistics	83.3%	97.0%
time-series-ai999x	Mathematics & Statistics	64.2%	82.3%
astronomy-ai999x	Physical Sciences	97.9%	100.0%
chemistry-ai999x	Physical Sciences	92.0%	100.0%
classical-physics-ai999x	Physical Sciences	98.2%	100.0%
earth-science-ai999x	Physical Sciences	96.2%	100.0%
materials-science-ai999x	Physical Sciences	94.6%	100.0%
quantum-physics-ai999x	Physical Sciences	98.3%	99.7%
aerospace-engineering-ai999x	Engineering	94.5%	100.0%
chemical-engineering-ai999x	Engineering	95.9%	99.8%
civil-engineering-ai999x	Engineering	94.2%	99.8%
electrical-engineering-ai999x	Engineering	94.4%	100.0%
energy-systems-ai999x	Engineering	88.1%	100.0%
mechanical-engineering-ai999x	Engineering	96.9%	99.8%
nuclear-engineering-ai999x	Engineering	97.1%	100.0%
signal-processing-ai999x	Engineering	93.0%	98.9%
thermofluids-ai999x	Engineering	97.4%	100.0%
transportation-ai999x	Engineering	96.0%	100.0%
biomedical-science-ai999x	Life Sciences	93.4%	99.8%
computational-biology-ai999x	Life Sciences	94.9%	100.0%
economics-finance-ai999x	Economics & Finance	86.3%	98.3%
geospatial-analytics-ai999x	Geospatial	78.8%	95.1%
data-science-ml-ai999x	Data Science & Machine Learning	85.0%	97.5%

How it was made

Its own 8,000-token vocabulary was learned from AI999X's science and mathematics texts. A 4-layer, 256-wide encoder first learned general meaning from a larger embedding model's vectors of those texts, then learned the task itself: a question close to its own capability's description and far from the others. The held-out and stress questions were removed from every training stage, the vocabulary included.

Use it

The easiest way is through any AI999X package:

# pip install "classical-physics-ai999x[search]"
import classical_physics_ai999x as physics
physics.search("How long does a pendulum take to swing back and forth?", k=3)

Or directly with ONNX Runtime (Python, JavaScript, C#, Java, C++):

import numpy as np, onnxruntime as ort
from tokenizers import Tokenizer
tok = Tokenizer.from_file("tokenizer.json"); tok.enable_truncation(96)
sess = ort.InferenceSession("onnx/encoder.int8.onnx")
e = tok.encode("A rocket burns its fuel; how much does its speed change?")
v = sess.run(None, {"input_ids": np.array([e.ids], dtype=np.int64),
                    "attention_mask": np.array([e.attention_mask], dtype=np.int64)})[0][0]
# v is a unit vector (256 numbers); capability_index.json holds the vectors of every capability

In the browser with onnxruntime-web (this is how the capability orb searches):

import * as ort from 'onnxruntime-web';
const session = await ort.InferenceSession.create('onnx/encoder.int8.onnx');
// ids and mask from tokenizer.json (WordPiece, lower-cased, max 96 tokens)
const out = await session.run({ input_ids: new ort.Tensor('int64', ids, [1, ids.length]),
                                attention_mask: new ort.Tensor('int64', mask, [1, mask.length]) });
// out[Object.keys(out)[0]].data: the 256-number vector; rank capability_index.json by dot product

Versions

Version	Covers	Held-out: first / top five
3 (current)	2,696 capabilities, 31 models	91.8% / 98.8%
2	171 capabilities of the first 3 models	89.0% / 97.7% (on those models only)

Files

File	What it is
model.safetensors, config.json	the weights (BERT-style encoder: 4 layers, 256 wide, mean pooling)
onnx/encoder.int8.onnx	4.9 MB int8 graph — ids and attention mask in, a unit vector out
onnx/encoder.onnx	the float32 graph
tokenizer.json	its own 8,000-token WordPiece vocabulary, learned from science and mathematics text
capability_index.json	every AI999X capability with its vector, name, model and an example question
evaluation/report.json	the measurements above

Limits

• It finds capabilities; it does not answer. Answers come from the AI999X engines, which compute and check them.
• English, science and mathematics questions; it has not been trained for other text.

Created and engineered by Ibnul Jaif Farabi · AI999X · ai999x.com