Inkling-2-Small-Riggs1890-lexgym

A LoRA adapter that teaches Inkling 2 Small one book: Stephen R. Riggs, A Dakota-English Dictionary (1890). After three training passes the model recalls 84% of the tested entries from memory, and it still uses a dictionary lookup tool correctly on 100% of the tested lookup tasks.

Try it	Live demo Space: base model against this adapter on any word
Method	Cookbook in the repository
Data	Tasks and lookup index · evaluation traces
Charts	W&B project
Base model	thinkingmachines/Inkling-2-Small
Trained with	Tinker by Thinking Machines, for $33.70

This is a research checkpoint. It shows a training method. It is not a Dakota language assistant, and its output is not authoritative Dakota.

The idea in one paragraph

A model can answer a question about a dictionary in two ways: it can know the entry, or it can look the entry up. Earlier runs in this project showed that training a model to know the book made it stop looking things up. On two Qwen models, three passes of ordinary fine-tuning took the lookup score from about 0.75 to about 0.03. This adapter was trained on a file that mixes plain question-and-answer rows with solved lookup conversations. The model learned the book and kept the tool.

The source

All training data comes from the public-domain scan of Riggs' dictionary. Each entry was parsed into a headword, its syllable division and accent, a part of speech, a definition, the inflected forms that the entry lists, and the word it is derived from. These fields make eight kinds of task.

Task	The question	Training tasks
gloss	English definition of a Dakota word	13,562
en_to_dak	Dakota word for an English word	3,369
syllabify	Syllable division and accent mark	13,990
pos	Part-of-speech abbreviation	13,985
inflect	Inflected forms that the entry lists	4,936
root	The word that this word is derived from	3,060
lookup_gloss	Read the entry with the tool, give the definition	13,562
lookup_root_gloss	Read the entry, read the root's entry, give the root's definition	2,447

One word family in ten is held out with all its derived words. The model never sees those families in training.

Results

Exact match on the same 1,500 tasks at four points. Temperature 0, one seed.

Measure	Base	Pass 1	Pass 2	Pass 3 (this adapter)
Recall from memory (600)	0.030	0.295	0.637	0.835
– Dakota word → English definition	0.007	0.223	0.510	0.817
– English word → Dakota word	0.053	0.367	0.763	0.853
Held-out word families (900)	0.443	0.814	0.820	0.816
– Lookup, one hop	0.767	1.000	1.000	1.000
– Lookup, two hops	0.893	1.000	1.000	1.000
– Syllables and accent	0.327	0.867	0.913	0.907
– Root word	0.380	0.820	0.740	0.773
– Part of speech	0.293	0.693	0.753	0.720
– Inflected forms	0.000	0.507	0.513	0.493
All 1,500 tasks	0.278	0.607	0.747	0.823

How to read the table:

• Recall rises with each pass. Recall tasks ask about words that were in training, so they measure memory of the book. More passes give more memory.
• Held-out scores are flat after pass 1. These tasks use word families the model did not see. They depend on rules, such as where Riggs puts the accent, and the model learns the rules in one pass.
• The base model already called the tool correctly. Its lookup errors were answers in its own words. Training taught it to give the entry text as the book prints it.
• Inflected forms are the weak task at 0.49. The model must produce a form that the book lists for a word it has not seen.

The same question before and after

Question	Riggs, 1890	Base model	This adapter
Definition of iwiśtaŋ	to treat well, as a sick person	to be ashamed; to be bashful	to treat well, as a sick person
Definition of ćunksiya	to have for a daughter, count as a daughter	To have a daughter.	to have for a daughter, count as a daughter
Dakota word for depute (v. a.)	yeśi	Iyuŋkapi	yeśi
Syllables and accent of hunapte (held out)	hu-na'-pte	hu-nap'-te	hu-na'-pte
Inflected form of yaaśdadaŋ (held out)	mdaaśdadaŋ	yaaśdadaŋke, yaaśdadaŋpi	mdaaśdadaŋ

All rows are from the public evaluation traces.

How the tool is kept

The training file has 69,461 rows: 56,120 plain rows and 13,341 rows from solved lookup conversations. The conversations are built from the lookup index with no teacher model, because the index already holds the entry text and the correct answer. Each conversation is replayed against the rules of the evaluation environment and kept only if it scores 1.0. It is then written as one row for each model turn, so that training always puts the loss on the last model message of a row.

About one training row in five shows the tool. That is enough for the model to keep the habit while it memorises the book.

What this result does not prove. Two things changed between the Qwen runs and this run: the model and the training file. The chart at the top compares them, but it cannot say which change protects tool use. A Qwen run on the mixed file is the control.

Training

The loss falls in a step at the start of each pass. That shape is the mark of memorisation: each time the model sees a row again, it knows it better.

Setting	Value
Method	Supervised fine-tuning, LoRA rank 32
Service	Tinker, tinker 0.39.0 and tinker-cookbook 0.6.1
Passes	3 (1,620 steps of 128 rows)
Learning rate	2e-4 with linear decay, set by hand
Loss	Last model message of each row
Reasoning	Effort 0 in training and evaluation
Tokens trained	22.47 million
Cost	$33.70 for training, about $35 with the four evaluations
Code	cookbook/launch_sft_curve_job.py, repository commit a281807
Tinker checkpoint	tinker://551aa04a-f385-5fdc-825c-7f47cf3a769d:train:0/sampler_weights/final

One effect of training is easy to miss. At the lowest reasoning setting, the base model still reasons before it answers and writes about 600 tokens for one definition. This adapter writes about 15. On the 1,500 evaluation tasks the base model sampled 1.27 million tokens and the adapter sampled 23,000.

Prompt format

The adapter expects the prompts it was trained on.

Plain tasks, system message:

You answer from Stephen R. Riggs, A Dakota-English Dictionary (1890). Use the spelling of that book. Give only the answer on one line, starting with 'ANSWER:'.

Lookup tasks, system message:

You answer from Stephen R. Riggs, A Dakota-English Dictionary (1890). You can read the dictionary. To read the entry for a Dakota word, reply with exactly one line: 'LOOKUP: <word>'. You get the entry in the next message. You have 3 lookups. When you know the answer, reply with one line that starts with 'ANSWER:'.

The user messages are in lexgym/tasks.py, for example Give the English definition of the Dakota word: iwiśtaŋ. The tool is not a function-calling schema. The model writes a LOOKUP: line, your code returns the entry as the next user message, and the model continues. The demo's engine.py is a complete loop in about 40 lines.

Use

Through Tinker. This path is tested. It samples from the checkpoint with any OpenAI client.

from openai import OpenAI
import os

client = OpenAI(
    base_url="https://tinker.thinkingmachines.dev/services/tinker-prod/oai/api/v1",
    api_key=os.environ["TINKER_API_KEY"],
)
out = client.chat.completions.create(
    model="tinker://551aa04a-f385-5fdc-825c-7f47cf3a769d:train:0/sampler_weights/final",
    messages=[
        {"role": "system", "content": "You answer from Stephen R. Riggs, A Dakota-English Dictionary (1890). "
         "Use the spelling of that book. Give only the answer on one line, starting with 'ANSWER:'."},
        {"role": "user", "content": "Give the English definition of the Dakota word: iwiśtaŋ"},
    ],
    max_tokens=96, temperature=0.0, reasoning_effort="none",
)
print(out.choices[0].message.content)   # ANSWER: to treat well, as a sick person

Tinker checkpoints expire. If the path above no longer works, the files in this repository are the permanent copy.

From the files in this repository. The files are a PEFT LoRA adapter, converted from the Tinker checkpoint with tinker checkpoint push-hf. The base model is large (594 billion parameters, with about 17 billion active for each token), and it needs a server with several hundred gigabytes of GPU memory. We evaluated the checkpoint on Tinker. We have not load-tested the converted files in peft, vLLM, or SGLang.

Limits

• The source is a missionary dictionary from 1890. It records one written variety, with the errors and the views of its time. It is not a description of Dakota as it is spoken now.
• A correct score means agreement with the book. Dakota speakers are the authority on the language. A model that is correct for the 1890 book is a starting point for correction by them.
• The training data comes from a vision-model reading of page scans and contains reading errors.
• Recall covers words that were in training. For a word outside the book, the model still writes a confident answer.
• The model does not choose between memory and the tool. The prompt decides if the tool is available.
• All numbers are from one run with one seed.

Citation

@misc{cooper2026riggslexgym,
  author = {Cooper, Christian H.},
  title  = {Inkling-2-Small-Riggs1890-lexgym: teaching a model one dictionary while keeping its lookup tool},
  year   = {2026},
  url    = {https://huggingface.co/HarleyCooper/Inkling-2-Small-Riggs1890-lexgym}
}

Source text: Stephen R. Riggs, A Dakota-English Dictionary, ed. James Owen Dorsey, 1890. Public domain.

An independent project by Christian H. Cooper. It is not a publication of Thinking Machines Lab.