Instructions to use deburky/inkling-small-claude-code with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use deburky/inkling-small-claude-code with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("thinkingmachines/Inkling-Small") model = PeftModel.from_pretrained(base_model, "deburky/inkling-small-claude-code") - Notebooks
- Google Colab
- Kaggle
Restore model card with evaluation results
Browse files
README.md
CHANGED
|
@@ -9,29 +9,80 @@ tags:
|
|
| 9 |
tinker_path: tinker://0e3342b7-6f34-5a67-9009-f94aad6ae787:train:0/sampler_weights/claude-code-r32
|
| 10 |
---
|
| 11 |
|
| 12 |
-
#
|
| 13 |
|
| 14 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
|
| 16 |
## Usage
|
| 17 |
|
| 18 |
```python
|
|
|
|
| 19 |
from transformers import AutoModelForCausalLM
|
| 20 |
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
model = AutoModelForCausalLM.from_pretrained(adapter_id, device_map="auto")
|
| 25 |
```
|
| 26 |
|
| 27 |
-
|
| 28 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 29 |
```
|
| 30 |
-
tinker://0e3342b7-6f34-5a67-9009-f94aad6ae787:train:0/sampler_weights/claude-code-r32
|
| 31 |
-
```
|
| 32 |
|
| 33 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
|
|
|
| 9 |
tinker_path: tinker://0e3342b7-6f34-5a67-9009-f94aad6ae787:train:0/sampler_weights/claude-code-r32
|
| 10 |
---
|
| 11 |
|
| 12 |
+
# Inkling-Small LoRA: claude-code
|
| 13 |
|
| 14 |
+
Rank-32 LoRA adapter for [`thinkingmachines/Inkling-Small`](https://huggingface.co/thinkingmachines/Inkling-Small),
|
| 15 |
+
trained on Tinker from [`deburky/gpt-oss-claude-code`](https://huggingface.co/datasets/deburky/gpt-oss-claude-code).
|
| 16 |
+
|
| 17 |
+
## Training
|
| 18 |
+
|
| 19 |
+
| | |
|
| 20 |
+
| --- | --- |
|
| 21 |
+
| Base | `thinkingmachines/Inkling-Small` (276B total / 12B active MoE) |
|
| 22 |
+
| Method | LoRA, rank 32, alpha 32, `target_modules: all-linear` |
|
| 23 |
+
| Modules | attention + MoE expert MLPs (`train_mlp=True`) |
|
| 24 |
+
| Optimizer | Adam, lr 1e-4, 2 epochs, batch 8 |
|
| 25 |
+
| Data | 284 train / 71 validation chat records |
|
| 26 |
+
| Platform | Tinker |
|
| 27 |
+
|
| 28 |
+
Held-out loss went from **1.5374** (base) to **0.7486**.
|
| 29 |
+
|
| 30 |
+
The source dataset is in gpt-oss harmony format with tool calls encoded as JSON
|
| 31 |
+
inside the final channel. It was parsed back to structured messages and
|
| 32 |
+
re-rendered with Inkling's renderer, converting those to native tool calls.
|
| 33 |
|
| 34 |
## Usage
|
| 35 |
|
| 36 |
```python
|
| 37 |
+
from peft import PeftModel
|
| 38 |
from transformers import AutoModelForCausalLM
|
| 39 |
|
| 40 |
+
base = AutoModelForCausalLM.from_pretrained("thinkingmachines/Inkling-Small")
|
| 41 |
+
model = PeftModel.from_pretrained(base, "deburky/inkling-small-claude-code")
|
|
|
|
|
|
|
| 42 |
```
|
| 43 |
|
| 44 |
+
Serving with vLLM, which supports MoE expert LoRA via `FusedMoEWithLoRA`:
|
| 45 |
|
| 46 |
+
```bash
|
| 47 |
+
vllm serve thinkingmachines/Inkling-Small-NVFP4 \
|
| 48 |
+
--enable-lora \
|
| 49 |
+
--lora-modules claude-code=deburky/inkling-small-claude-code \
|
| 50 |
+
--max-lora-rank 32
|
| 51 |
```
|
|
|
|
|
|
|
| 52 |
|
| 53 |
+
The adapter uses the "shared-outer" MoE layout: `w1`/`w3` `lora_A` and `w2`
|
| 54 |
+
`lora_B` are stored once with expert-dim 1, the rest per-expert across 256
|
| 55 |
+
experts.
|
| 56 |
+
|
| 57 |
+
## Honest evaluation
|
| 58 |
+
|
| 59 |
+
Held-out loss more than halved, but **this adapter does not beat the base model
|
| 60 |
+
on behaviour**, and it is not recommended over base for agentic use.
|
| 61 |
+
|
| 62 |
+
Scored in real Claude Code sessions (3 tasks x 6 repeats, tool access, checkable
|
| 63 |
+
ground truth):
|
| 64 |
+
|
| 65 |
+
| Model | Held-out loss | Agentic (n=18) |
|
| 66 |
+
| --- | --- | --- |
|
| 67 |
+
| base `Inkling-Small` | 1.5374 | 18/18 |
|
| 68 |
+
| this adapter | 0.7486 | 18/18 |
|
| 69 |
+
| attention-only variant | 0.8175 | 12/18 |
|
| 70 |
+
|
| 71 |
+
On a harder multi-step task (discover files, read several, judge a bug) the
|
| 72 |
+
adapter needed 2-3x the turns and wall-clock of base, dropped the `file:line`
|
| 73 |
+
citations base gives consistently, and confabulated a path outside the repo in
|
| 74 |
+
one run of three.
|
| 75 |
+
|
| 76 |
+
The base model already handled tool calling and prose style well, so most of the
|
| 77 |
+
loss reduction reflects fitting the corpus's phrasing rather than new capability.
|
| 78 |
+
|
| 79 |
+
## Size
|
| 80 |
+
|
| 81 |
+
7.9 GB in F32, because `train_mlp=True` puts a LoRA pair on all 256 experts per
|
| 82 |
+
layer: 96% of the bytes are expert MLPs, 2% attention. An attention-only variant
|
| 83 |
+
is 185 MB but scores worse (12/18 above). Casting to bf16 would roughly halve
|
| 84 |
+
this file.
|
| 85 |
|
| 86 |
+
Note the asymmetry: the adapter is 8 GB, but the base it attaches to is 532 GB in
|
| 87 |
+
BF16. Serving realistically means a quantized base (`Inkling-Small-NVFP4` at
|
| 88 |
+
171 GB) on multi-GPU.
|