tiny-codemode / README.md
erayack's picture
feat: release Tiny Codemode CNN with evaluation and inference package
659433c verified
|
Raw History Blame Contribute Delete
6.58 kB
metadata
license: mit
language:
  - en
library_name: numpy
tags:
  - semantic-parsing
  - tool-calling
  - numpy
  - cpu
  - synthetic-data
  - research

Tiny Codemode CNN

A 67,485-parameter research model for bounded English tool requests. It maps an instruction and an explicit tool catalog to a typed plan, then deterministically compiles the plan into JavaScript. It does not execute tools and is not an agent, general language model or reliable authorization system.

This release is the original CNN checkpoint, not a later attention candidate. Unsupported instructions can be incorrectly accepted. Read the limitations before using any generated plan.

Quickstart

Download this model repository, preserving its directory structure. With Python 3.10+ and uv 0.9+ installed, run from the downloaded directory:

uv sync --locked --no-dev
uv run --locked --no-dev tiny-codemode --help
uv run --locked --no-dev tiny-codemode --js < examples/request.jsonl

These commands install NumPy-only inference, not Torch. --js returns generated JavaScript as text; it does not execute it. Input is JSONL: one complete JSON object per line. Output is a plan (and optional JavaScript), or a diagnostic. python -m pip install . is also supported but is not a locked installation.

Python API, after installation:

from core import Diagnostic, compile_js
from model import Model

request = {
    "instruction": 'Call list_users with team="blue"; return the last result.',
    "tools": [
        {
            "name": "list_users",
            "arguments": {"team": "string"},
            "required": ["team"],
            "returns": {"kind": "array", "properties": ["id", "name"]},
        }
    ],
}

try:
    plan = Model().predict(request)
    print(plan)
    print(compile_js(plan, request["tools"]))
except Diagnostic as error:
    print({"diagnostic": str(error)})

The expected plan for this example is:

{
  "mode": "single",
  "calls": [{ "tool": "list_users", "args": { "team": "blue" } }],
  "output": { "kind": "last" }
}

This is custom NumPy inference, not a Transformers/AutoModel checkpoint. No trust_remote_code flag, hosted inference widget or generic text-generation API is required or promised. The bundled Python code runs locally; review it before use. The NPZ contains numeric arrays and is loaded without pickle.

Scope and model details

The contract allows 1–8 catalog tools, 1–3 calls, up to four primitive arguments per tool and 128 input tokens. It supports single, sequential or parallel schedules and all/last outputs. Single-array projection and prefix limits are bounded. Dependencies, mixed schedule groups, loops, branching, nested arguments and arbitrary JavaScript requests are outside the contract.

Architecture: 1,024 fixed hash buckets, a learned 32-channel embedding, 12 mechanical schema/primitive features, three width-5 48-channel ReLU convolutions, and learned token-role, schedule, output and rejection heads. Catalog/literal identities are normalized for the model; exact names and values are retained by the deterministic frontend/compiler. No pretrained model was used.

  • Parameters: 67,485, float32 (269,940 raw parameter bytes).
  • Weight file: tiny_codemode_weights/model.npz (253,781 bytes compressed).
  • SHA-256: a6b944d56580b64dfb526da5b3888a137b6783ea29d8a719a0fea3f4e3f7ee76.
  • Inference: Python + NumPy; CPU only is sufficient.
  • License: MIT, supplied with the code and checkpoint.

SHA256SUMS covers every exported file except itself. release.json identifies the checkpoint and scope. The manifest checks integrity, not publisher authenticity.

Training and data

The selected CNN was randomly initialized and CPU-trained on 26,000 synthetic requests, with 1,600 validation requests. Plans and token-role spans were AI-authored/generated before rendering instructions; no real-user traffic is represented. Synthetic schemas and phrasing are not a representative sample of English tool use. There is no external pretrained base model or Hub dataset ID. The full training corpus and experiment infrastructure are not part of this inference-only upload; it is not a standalone training-reproduction package.

Seed 715, Adam learning rate 0.002, batch size 96, four CPU threads, 55 epochs. The earliest best validation full-decision checkpoint was selected at epoch 12 (99.0625% validation full-decision accuracy). Training used 25% neutral-wrapper hash-bucket perturbation; no runtime wrapper stripping is performed. Historical training took 137.60 seconds for the selected run on its recorded CPU environment; this is not a hardware-independent speed guarantee.

Evaluation

See EVALUATION.md for counts, denominators, definitions and limitations, and evaluation.json for the curated numeric summary. These are historical results bound to the exact shipped checkpoint, not new benchmarks performed while packaging. They include false accepts and rejected supported requests, not only successful examples.

The authored bank scored 40/50 supported plans, with 13/70 negative requests falsely accepted. Its 120 requests are transformations of only 24 AI-authored semantic bases—not 120 independent users. All reported banks are now exposed; they must not be relabeled as future held-out evaluation.

Intended use, risks and limitations

Use for research, education and constrained offline demonstrations. Do not use as an autonomous tool executor, policy gate, authorization mechanism or production natural-language parser. Structural validity does not establish that a plan matches the user's intent or permission.

Known failures include unfamiliar prohibitions, mixed ordering, unknown projected fields, loop/sorting contexts and valid unseen phrasings. The model can reject valid requests or silently drop unsupported intent. Neither a successful plan nor a diagnostic should be treated as a security guarantee. Independent authorization, semantic review and appropriate execution isolation are required before real side effects. Mock execution tests are not a security sandbox.

No broad-English, multilingual, human-population, demographic fairness, worst-case latency or production-safety evaluation has been established. Dataset diversity and labels are synthetic and limited. Later experimental architectures were not promoted into this release, and their scores must not be attributed to this CNN.