deburky commited on
Commit
eddbb4e
·
verified ·
1 Parent(s): 3b2003a

Restore model card with evaluation results

Browse files
Files changed (1) hide show
  1. README.md +64 -13
README.md CHANGED
@@ -9,29 +9,80 @@ tags:
9
  tinker_path: tinker://0e3342b7-6f34-5a67-9009-f94aad6ae787:train:0/sampler_weights/claude-code-r32
10
  ---
11
 
12
- # Tinker LoRA Adapter
13
 
14
- This repository contains a LoRA adapter exported from Tinker.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
15
 
16
  ## Usage
17
 
18
  ```python
 
19
  from transformers import AutoModelForCausalLM
20
 
21
- adapter_id = "deburky/inkling-small-claude-code"
22
- base_model = "thinkingmachines/Inkling-Small"
23
-
24
- model = AutoModelForCausalLM.from_pretrained(adapter_id, device_map="auto")
25
  ```
26
 
27
- ## Source
28
 
 
 
 
 
 
29
  ```
30
- tinker://0e3342b7-6f34-5a67-9009-f94aad6ae787:train:0/sampler_weights/claude-code-r32
31
- ```
32
 
33
- ## Details
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
34
 
35
- - Base model: thinkingmachines/Inkling-Small
36
- - LoRA rank: 32
37
- - Trained modules: attn=True, mlp=True, unembed=True
 
9
  tinker_path: tinker://0e3342b7-6f34-5a67-9009-f94aad6ae787:train:0/sampler_weights/claude-code-r32
10
  ---
11
 
12
+ # Inkling-Small LoRA: claude-code
13
 
14
+ Rank-32 LoRA adapter for [`thinkingmachines/Inkling-Small`](https://huggingface.co/thinkingmachines/Inkling-Small),
15
+ trained on Tinker from [`deburky/gpt-oss-claude-code`](https://huggingface.co/datasets/deburky/gpt-oss-claude-code).
16
+
17
+ ## Training
18
+
19
+ | | |
20
+ | --- | --- |
21
+ | Base | `thinkingmachines/Inkling-Small` (276B total / 12B active MoE) |
22
+ | Method | LoRA, rank 32, alpha 32, `target_modules: all-linear` |
23
+ | Modules | attention + MoE expert MLPs (`train_mlp=True`) |
24
+ | Optimizer | Adam, lr 1e-4, 2 epochs, batch 8 |
25
+ | Data | 284 train / 71 validation chat records |
26
+ | Platform | Tinker |
27
+
28
+ Held-out loss went from **1.5374** (base) to **0.7486**.
29
+
30
+ The source dataset is in gpt-oss harmony format with tool calls encoded as JSON
31
+ inside the final channel. It was parsed back to structured messages and
32
+ re-rendered with Inkling's renderer, converting those to native tool calls.
33
 
34
  ## Usage
35
 
36
  ```python
37
+ from peft import PeftModel
38
  from transformers import AutoModelForCausalLM
39
 
40
+ base = AutoModelForCausalLM.from_pretrained("thinkingmachines/Inkling-Small")
41
+ model = PeftModel.from_pretrained(base, "deburky/inkling-small-claude-code")
 
 
42
  ```
43
 
44
+ Serving with vLLM, which supports MoE expert LoRA via `FusedMoEWithLoRA`:
45
 
46
+ ```bash
47
+ vllm serve thinkingmachines/Inkling-Small-NVFP4 \
48
+ --enable-lora \
49
+ --lora-modules claude-code=deburky/inkling-small-claude-code \
50
+ --max-lora-rank 32
51
  ```
 
 
52
 
53
+ The adapter uses the "shared-outer" MoE layout: `w1`/`w3` `lora_A` and `w2`
54
+ `lora_B` are stored once with expert-dim 1, the rest per-expert across 256
55
+ experts.
56
+
57
+ ## Honest evaluation
58
+
59
+ Held-out loss more than halved, but **this adapter does not beat the base model
60
+ on behaviour**, and it is not recommended over base for agentic use.
61
+
62
+ Scored in real Claude Code sessions (3 tasks x 6 repeats, tool access, checkable
63
+ ground truth):
64
+
65
+ | Model | Held-out loss | Agentic (n=18) |
66
+ | --- | --- | --- |
67
+ | base `Inkling-Small` | 1.5374 | 18/18 |
68
+ | this adapter | 0.7486 | 18/18 |
69
+ | attention-only variant | 0.8175 | 12/18 |
70
+
71
+ On a harder multi-step task (discover files, read several, judge a bug) the
72
+ adapter needed 2-3x the turns and wall-clock of base, dropped the `file:line`
73
+ citations base gives consistently, and confabulated a path outside the repo in
74
+ one run of three.
75
+
76
+ The base model already handled tool calling and prose style well, so most of the
77
+ loss reduction reflects fitting the corpus's phrasing rather than new capability.
78
+
79
+ ## Size
80
+
81
+ 7.9 GB in F32, because `train_mlp=True` puts a LoRA pair on all 256 experts per
82
+ layer: 96% of the bytes are expert MLPs, 2% attention. An attention-only variant
83
+ is 185 MB but scores worse (12/18 above). Casting to bf16 would roughly halve
84
+ this file.
85
 
86
+ Note the asymmetry: the adapter is 8 GB, but the base it attaches to is 532 GB in
87
+ BF16. Serving realistically means a quantized base (`Inkling-Small-NVFP4` at
88
+ 171 GB) on multi-GPU.