Card fact-check: usage snippet, long inputs, 4090 follow-up, limits, privacy, sources

124bf0b verified 4 days ago

2.17 kB

# Rebuild and serve (d3-mini-agentcode, pre-release)

## Use the merged weights (this repo)
The repo root is a complete d3 package: the base layout, runtime code, tokenizer and config, with the fine-tuned
backbone weights and answer readout.
```python
from transformers import AutoModel
model = AutoModel.from_pretrained("stonefish-labs/d3-mini-agentcode", trust_remote_code=True)
```
`MODEL_MANIFEST.json` lists every file's SHA-256 and size. On every load the runtime checks the files it reads
(`verify="fast"`: it hashes files up to 64 MB and checks the size of larger ones). `verify="full"` hashes them all, and
`verify="release"` also checks the card and other files the runtime doesn't read. `SHA256SUMS` covers every file except
itself and the manifest.

## Rebuild the merged weights from the adapter
`adapter/` holds the LoRA adapter (`adapter_model.safetensors`, `adapter_config.json`), the trained answer readout
(`readout.safetensors`) and the training settings (`training_args.json`).
1. Download the base package at the pinned revision:
   `huggingface-cli download vllm-sr/d3-mini --revision df99bf4ca4418bc6908ee68e9b7479af1f3b8800 --local-dir d3-mini`
2. Merge (requires torch and safetensors; runs on CPU or GPU):
   `python adapter/merge.py --base d3-mini --adapter adapter --out merged --name d3-mini-agentcode`
   - Each adapted weight becomes W ← bf16(W + (α/r)·B·A), with α/r = 2.
   - The trained readout replaces the base readout.
   - Everything else is copied from the base, and a new `MODEL_MANIFEST.json` is written.
3. Check: the rebuilt `model-0000{1,2}-of-00002.safetensors` and `readout.safetensors` match the hashes in this repo's
   `SHA256SUMS`. They did on our rebuild.

## Serve
- **transformers:** as above. `model.system_one(state=..., questions=...)`; see `examples/usage.py`.
- **vLLM Semantic Router:** the package keeps d3-mini's format (`d3-code-readout-v1`), so runtimes that serve d3-mini
  should serve it too (see the d3-mini card). We only ran it with the bundled runtime above.
- **Hardware:** the bf16 weights take about 9 GB. On an RTX 4090, batches of 8 at up to 8,192 tokens peaked at 12.7 GiB
  allocated.

	# Rebuild and serve (d3-mini-agentcode, pre-release)

	## Use the merged weights (this repo)
	The repo root is a complete d3 package: the base layout, runtime code, tokenizer and config, with the fine-tuned
	backbone weights and answer readout.
	```python
	from transformers import AutoModel
	model = AutoModel.from_pretrained("stonefish-labs/d3-mini-agentcode", trust_remote_code=True)
	```
	`MODEL_MANIFEST.json` lists every file's SHA-256 and size. On every load the runtime checks the files it reads
	(`verify="fast"`: it hashes files up to 64 MB and checks the size of larger ones). `verify="full"` hashes them all, and
	`verify="release"` also checks the card and other files the runtime doesn't read. `SHA256SUMS` covers every file except
	itself and the manifest.

	## Rebuild the merged weights from the adapter
	`adapter/` holds the LoRA adapter (`adapter_model.safetensors`, `adapter_config.json`), the trained answer readout
	(`readout.safetensors`) and the training settings (`training_args.json`).
	1. Download the base package at the pinned revision:
	`huggingface-cli download vllm-sr/d3-mini --revision df99bf4ca4418bc6908ee68e9b7479af1f3b8800 --local-dir d3-mini`
	2. Merge (requires torch and safetensors; runs on CPU or GPU):
	`python adapter/merge.py --base d3-mini --adapter adapter --out merged --name d3-mini-agentcode`
	- Each adapted weight becomes W ← bf16(W + (α/r)·B·A), with α/r = 2.
	- The trained readout replaces the base readout.
	- Everything else is copied from the base, and a new `MODEL_MANIFEST.json` is written.
	3. Check: the rebuilt `model-0000{1,2}-of-00002.safetensors` and `readout.safetensors` match the hashes in this repo's
	`SHA256SUMS`. They did on our rebuild.

	## Serve
	- transformers: as above. `model.system_one(state=..., questions=...)`; see `examples/usage.py`.
	- vLLM Semantic Router: the package keeps d3-mini's format (`d3-code-readout-v1`), so runtimes that serve d3-mini
	should serve it too (see the d3-mini card). We only ran it with the bundled runtime above.
	- Hardware: the bf16 weights take about 9 GB. On an RTX 4090, batches of 8 at up to 8,192 tokens peaked at 12.7 GiB
	allocated.