ConeML/coneml-810m-alpha-arithmetic

ConeML 810M Alpha-Arithmetic

ConeML 810M Alpha-Arithmetic is an 810,112,512-parameter, decoder-only language model trained from scratch and instruction-tuned for a narrow small-number arithmetic profile. It shares its pretrained base with coneml-810m-alpha and differs in fine-tuning composition.

This variant records the stronger carry/borrow profile of the pair. The companion coneml-810m-alpha has substantially stronger function-writing and two-step results. Results are specific to the documented prompt formats and task families, not claims of general mathematical, coding, reasoning, or factual-knowledge performance.

• Architecture: LLaMA-style; 28 layers, d=1536, 12 heads (3 KV), tied embeddings, 8192 context, 32,768-token BPE tokenizer
• Pretraining corpus: 27.87B curated tokens
• Selected pretrained-base exposure: approximately 12.32B token positions, or 15.2 tokens per parameter
• Pretraining wall time: approximately 11 days on one NVIDIA RTX 5090; fine-tuning and evaluation are excluded
• Release model.safetensors SHA-256: e1de27f5f91ac43130ce2a57a4bf9a036b84b2e5b589789b5981ecf79489e0d1

1. Measured profile

Across four held-out core lane sets—1-digit addition, 1-digit multiplication, carry addition, and borrow subtraction—the model scores 1,093/1,116 (97.9%). Two of those four held-out sets scored 100%.

This is a result on four explicitly named, task-family-matched sets. It is not a claim of 97.9% general arithmetic accuracy. ConeML internal certification is not independent third-party certification.

held-out task family	result	n
2–3-digit addition with carry	96.8%	500
2-digit subtraction with borrow	98.6%	500
1-digit addition, including 19 zero-operand items	100%	71
1-digit multiplication	100%	45
"Which is bigger, X or Y?"	93.7%	300
comparison word problems	79.0%	500
missing addend	67.4%	500
two-step word problems	43.8%	500
2-digit × 1-digit multiplication	83.6%	500
missing factor	18.0%	500
Python function writing (single function, executed)	33.7%	300

Behavioral screens:

• Designated-refusal probe: 17/17 refusal prompts produced refusals, and 4/5 in-scope contrast prompts were answered correctly. The miss was an incorrect comparison answer, not an over-refusal. This is a small targeted policy test, not general epistemic calibration.
• Conversation-v2: 8/8 turns passed in one scripted dialogue covering cross-turn arithmetic, story revision, comparison explanation, and one unknown-personal-fact refusal. This is a single dialogue, not a general conversation benchmark.
• Everyday-reasoning screen: 13/20 by manual adjudication; the phrase-sensitive automatic scorer recorded 6/20.

2. Peer comparison and efficiency

The pair was tested against five post-trained peers on the same reduced diagnostic screen, through each model's native instruction interface, with greedy decoding, repetition penalty 1.15, and matched short-answer budgets.

model	parameters	mixed arithmetic (n=585)	executed functions (n=100)	transitive names d1/d3/d5 (n=32 each)	designated refusals (n=17)
ConeML 810M Alpha	0.81B	421 (72.0%)	83	23/15/15	13
ConeML 810M Alpha-Arithmetic	0.81B	442 (75.6%)	35	26/17/16	11
Qwen3.5	0.8B	154 (26.3%)	93	23/22/10	0
Qwen3	0.6B	223 (38.1%)	98	18/10/9	1
Llama 3.2 Instruct	1.24B	343 (58.6%)	79	16/7/5	1
TinyLlama Chat	1.1B	113 (19.3%)	51	8/18/22	0
SmolLM2 Instruct	1.7B	380 (65.0%)	96	16/6/5	1

This is not a general leaderboard: the task families match ConeML's trained surfaces. On this screen, the ConeML pair led mixed arithmetic; ConeML 810M Alpha exceeded Llama 3.2 and TinyLlama on function writing but trailed Qwen3.5, Qwen3, and SmolLM2. All models recorded zero over-refusals on five in-scope contrasts.

On the question-form name-chain depth screen, ConeML 810M Alpha-Arithmetic recorded 26/32, 17/32, and 16/32 at depths 1, 3, and 5; ConeML 810M Alpha recorded 23/32, 15/32, and 15/32. The arithmetic variant exceeded Qwen3 and Llama 3.2 at all three shown depths and Qwen3.5 at depths 1 and 5, while Qwen3.5 led it at depth 3 and TinyLlama led the group at depth 5. Entity-chain controls were mixed and are reported in the full table. Chance is 1/(depth+1); this is an exact-selection surface test, not a claim of general reasoning.

Qwen3.5 with thinking enabled reached 451/585 (77.1%), compared with this release's 442/585 (75.6%), while using at least 17.6 times the generated-token budget per item and approximately 24 times the recorded wall time. Runtime ratios are implementation-specific.

ConeML's selected base consumed 12.32B token positions—15.2 tokens per parameter and approximately 5.99e19 training FLOPs under the 6ND convention. Pretraining took approximately 11 days on one RTX 5090. Assuming, rather than claiming to have metered, a 0.70 kW average wall draw gives 184.8 kWh; at CHF 0.14–0.30/kWh, that is approximately CHF 26–55 of marginal pretraining electricity. This excludes hardware, labor, SFT, evaluation, conversion, and emissions accounting.

The complete category table, peer wins, adverse base result, evaluated revisions, assumptions, and frozen evidence commitments are in eval/PEER_COMPARISON.md.

3. Standard benchmark limits

Zero-shot, greedy decoding with a 256-token generation cap. GSM8K uses strict final-number matching; HumanEval reports pass@1 from executed tests.

benchmark	this model	same base, no fine-tune
GSM8K (test, first 200)	10/200 (5.0%)	7/200 (3.5%)
HumanEval (pass@1, tests executed)	3/164 (1.8%)	0/164 (0.0%)

These results are weak and define the boundary of the specialization: the four-lane arithmetic profile does not transfer to broad GSM8K or HumanEval performance.

4. Supported prompt formats

Evaluated decoding is greedy with repetition penalty 1.15, as pinned in generation_config.json.

# task format (single turn)
Question: <task>
Answer:

# dialogue format (chat_template.jinja)
User:
<message>
Assistant:

Dialogue generation ends at <|endoftext|>. Prompting outside these formats, including paraphrases, was not systematically evaluated except for the published raw-completion spot check.

5. Quantized variants

Q8_0 GGUF was evaluated separately under llama.cpp. Its runtime and stop handling differ from the bf16 harness, so score differences cannot be attributed solely to quantization.

held-out task family	Q8_0
subtraction with borrow	98.6% (n=500)
addition with carry	92.6% (n=500)
1-digit multiplication	100% (n=45)
two-step word problems	32.6% (n=500)

For local GGUF inference:

llama-cli \
  -m coneml-810m-alpha-arithmetic-Q8_0.gguf \
  -p $'Question: What is 47 + 38?\nAnswer:' \
  -n 32 --repeat-penalty 1.15 --no-conversation --single-turn

For Ollama, download Modelfile beside the Q8_0 file, then run:

ollama create coneml-810m-alpha-arithmetic -f Modelfile
ollama run coneml-810m-alpha-arithmetic "What is 47 + 38?"

6. Loading with Transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "ConeML/coneml-810m-alpha-arithmetic"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(
    repo_id,
    dtype=torch.bfloat16,
    device_map="auto",
)

prompt = "Question: What is 47 + 38?\nAnswer:"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(
    **inputs,
    max_new_tokens=32,
    do_sample=False,
    repetition_penalty=1.15,
)
answer = tokenizer.decode(
    output[0][inputs["input_ids"].shape[-1]:],
    skip_special_tokens=True,
)
print(answer)

This is a text-only causal language model. Load it with AutoModelForCausalLM or LlamaForCausalLM, not a multimodal model class.

7. Intended use and safety

This release is intended for research and constrained local workflows inside its measured small-number arithmetic envelope. It is not a general mathematics system. Validate outputs whenever exactness matters.

It is not intended for factual retrieval, high-stakes medical, legal, or financial decisions, autonomous action, or unsandboxed execution of generated code. Generated code may be incomplete or unsafe; inspect and test it in an isolated environment.

8. Known limitations

• The measured strength is narrow. Missing factor scored 18.0%, two-step word problems 43.8%, 2-digit × 1-digit multiplication 83.6%, and executed single-function writing 33.7%.
• Not a code model: complete programs scored 0/10, executable SQL 1/10, and bash 1/5.
• The model was not evaluated as a factual-QA system and refused all six real-world factual prompts in the designated-refusal probe.
• Money/decimal change-making scored 0/3 on the held-out screen; the model also showed yes-bias on feasibility questions.
• Multi-step GSM-class reasoning remained weak.
• Frameless prompting was spot-checked, not systematically certified.

9. Evidence and reproducibility

The eval/ directory publishes:

• summary.json: all aggregate bf16 results used above;
• representative-samples.json: deterministic, rule-selected examples from ConeML-owned probes;
• EVALUATION_METHODOLOGY.md: evaluation definitions and scope;
• separate Q8_0 and focused aggregate results; and
• PRIVATE_EVIDENCE_SHA256SUMS.txt: commitments to the complete row-level evidence retained by ConeML.
• peer-comparison-summary.json, PEER_COMPARISON.md, and PEER_EVIDENCE_SHA256SUMS.txt: the peer results, interpretation, and commitments to the retained peer-generation evidence.

Complete proprietary probe rows and generations are retained privately to avoid releasing the full measurement instruments. The hashes permit later disclosure to be checked against the evidence frozen at release time. Public GSM8K and HumanEval rows are not redistributed; only aggregate results are published. SHA256SUMS.txt covers every shipped file.

10. Research license and commercial work

Released for non-commercial use under CC BY-NC 4.0. Commercial use is not granted by this release. See LICENSE.md.

For commercial evaluation, model-engineering, or licensing enquiries, contact contact@coneml.com or visit ConeML for Organizations. Commercial availability does not imply that this alpha checkpoint is suitable for an unvalidated production workflow.