Instructions to use TendieLabs/Diogenes-9B-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use TendieLabs/Diogenes-9B-v1 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="TendieLabs/Diogenes-9B-v1") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("TendieLabs/Diogenes-9B-v1") model = AutoModelForMultimodalLM.from_pretrained("TendieLabs/Diogenes-9B-v1", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use TendieLabs/Diogenes-9B-v1 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "TendieLabs/Diogenes-9B-v1" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "TendieLabs/Diogenes-9B-v1", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/TendieLabs/Diogenes-9B-v1
- SGLang
How to use TendieLabs/Diogenes-9B-v1 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "TendieLabs/Diogenes-9B-v1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "TendieLabs/Diogenes-9B-v1", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "TendieLabs/Diogenes-9B-v1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "TendieLabs/Diogenes-9B-v1", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use TendieLabs/Diogenes-9B-v1 with Docker Model Runner:
docker model run hf.co/TendieLabs/Diogenes-9B-v1
Diogenes-9B
Question authority. Keep checking the facts.
V1 · Standalone BF16 merge · Native MTP head included
This was done as a research project
Diogenes-9B is a TendieLabs behavior-transfer experiment built from Qwen3.5-9B and a filtered subset of Nous Research's Arete training data. It targets more direct answers and less uncritical adoption of the Chinese state's account of its own conduct. It is not a reproduction of Arete-27B or a claim of general political neutrality; However it is inspired by Arete-27B.
On the complete 803-question political evaluation, v1 improved required-fact coverage from 2.61 to 3.32 out of 5 and reduced Chinese-state-framing findings from 292 to 29. The tradeoff is important: coding regressed in the expanded checks.
Model at a glance
| Property | Details |
|---|---|
| Artifact | Standalone BF16 merged checkpoint, not an adapter-only package |
| Base | Post-trained Qwen3.5-9B, full composite model architecture[1] |
| Training | Text-only supervised fine-tuning with a language-only rank-16 LoRA |
| Training examples | 5,535, with a separate 291-example reserved holdout |
| Trained parameters | 40,108,032 across 200 language projection modules |
| Vision | Original tower retained and frozen; no broad vision validation |
| MTP | Native one-layer head retained, all 15 mtp.* tensors preserved |
The model is an independent experiment, not an official Qwen or Nous Research release. This card describes v1, not the later v2 experiment.
V1 benchmark results
Comparisons through the repetition investigation are untouched Qwen3.5-9B versus the final unmerged v1 adapter. The checkpoint-development screen also includes earlier v1 checkpoints. Machine-readable results, immutable revisions, selected capability IDs, generation settings, and merge checks are included in benchmark-results.json.
Complete political evaluation: 803 questions
The released Censorship & Propaganda Eval contains 803 prompts with required facts and a separate state-framing rubric.[3] Both models answered all 803 questions, and all judgments passed the structural and answer-hash checks.
| Metric | Qwen3.5-9B | Diogenes v1 adapter |
|---|---|---|
| Required-fact mean, 0 to 5, higher is better | 2.6115 | 3.3151 |
| Answers with Chinese-state-framing findings, lower is better | 292/803, 36.36% | 29/803, 3.61% |
| Refusals | 40/803 | 1/803 |
| Answers reaching the 1,024-token cap | 3/803 | 18/803 |
Paired required-fact scores improved on 246 questions, worsened on 99, and were unchanged on 458. The mean improvement was 0.7036 points, with a descriptive paired bootstrap 95% interval of 0.5567 to 0.8543. The adapter removed 265 baseline framing findings but added two new findings.
Scope: “Framing” means adopting the Chinese state's account of its own conduct under this rubric, not political bias across every country.[3] A higher required-fact score does not verify every extra statement, and lower framing findings do not certify neutrality or factual reliability.
Expanded capability checks
The frozen subset contains 64 GSM8K, 64 IFEval, and 32 HumanEval unique prompts. Each prompt was run once with thinking disabled and once with thinking enabled, for 320 capability records per model. These are bounded subsets and adapted chat prompts, not official leaderboard reproductions.
| Mode | Metric | Qwen3.5-9B | Diogenes v1 adapter |
|---|---|---|---|
| Non-thinking | GSM8K correct | 60/64, 93.75% | 60/64, 93.75% |
| Non-thinking | IFEval strict, entire prompt | 54/64, 84.38% | 51/64, 79.69% |
| Non-thinking | IFEval loose, entire prompt | 56/64, 87.50% | 55/64, 85.94% |
| Non-thinking | HumanEval passed | 30/32, 93.75% | 25/32, 78.12% |
| Thinking | GSM8K correct | 49/64, 76.56% | 47/64, 73.44% |
| Thinking | IFEval strict, entire prompt | 28/64, 43.75% | 28/64, 43.75% |
| Thinking | IFEval loose, entire prompt | 28/64, 43.75% | 28/64, 43.75% |
| Thinking | HumanEval passed | 32/32, 100% | 30/32, 93.75% |
GSM8K strict answer-format matching, flexible extraction, and the custom strict-first/flexible-fallback metric produced the same counts in this run. The JSON retains all three rather than treating the custom aggregate as an official metric. IFEval per-instruction results are also retained in the JSON.
Coding regressed. Five extra non-thinking HumanEval answers failed canonical tests despite producing final answers. Two extra thinking answers exhausted the budget without a final answer. Across both modes, there were seven base-only passes and no adapter-only passes. The modes share the same 32 problems and must not be treated as 64 independent questions.
Thinking runs were especially budget-sensitive: base and adapter produced 48 and 54 empty final answers, respectively, across the 160 thinking capability records. An unfinished thinking block counted as no final answer, not as a correct answer hidden in the reasoning text. These results do not establish that thinking is inherently worse.
Original 42-question pilot
The earlier deterministic, category-balanced pilot is retained for transparency. Its 42 public questions are included in the later 803-question evaluation, so it is not an independent confirmation set.
| Pilot metric | Qwen3.5-9B | Diogenes v1 adapter |
|---|---|---|
| Required-fact mean, 0 to 5 | 2.1190 | 3.2381 |
| Chinese-state-framing findings | 17/42 | 0/42 |
| Refusals | 2/42 | 0/42 |
Pilot scores improved on 17 questions, worsened on six, and were unchanged on 19. The full evaluation, not the pilot's zero-framing result, is the stronger summary of this experiment.
The pilot also included 13 in-house checks per model. Both passed seven small exact-answer checks. Basic image comparison and native tool-call formatting were exercised, but this did not validate general vision, actual tool execution, or agentic performance. A C# asynchronous-cancellation edge case failed for both models. These probes are not included in the 803 political scores.
Repetition investigation
A separate 35-arm diagnostic matrix per model tested three prompts, including the original looping question f320.
- The base had no flagged loops. The adapter had four looping arms, all variants of the same
f320question, not four independent failing questions. - The original greedy output reproduced exactly. A larger cap extended the loop; adding an extra EOS token did not fix it.
- Sampled decoding, a generated-token presence penalty, or adding the training system message avoided this observed loop in the tested arms.
- Nonlooping answers could still contain factual errors. These observations are diagnostic mitigations, not a universal cure or measured population repetition rate.
Across the complete 1,158-record generation matrix, including diagnostics, the base hit its cap 55 times and the adapter 81 times. A cap hit alone is not evidence of a loop.
Independent checkpoint-development screen
A later, separate development set compared the base and retained v1 checkpoints. It used 70 unique prompts in two formats, with and without a system message: 32 math, 16 code, and 22 behavior prompts. Thus each candidate generated 140 records. This set is distinct from the expanded public test and was not used to choose v1's original final checkpoint.
| Candidate | Math /64 | MBPP code /32 | Behavior all-five /44 | Judge-flagged unsupported claims /44 |
|---|---|---|---|---|
| Base | 64 | 25 | 3 | 9 |
| V1 step 100 | 63 | 24 | 8 | 3 |
| V1 step 350 | 63 | 23 | 13 | 5 |
| V1 final, step 692 | 64 | 25 | 11 | 9 |
Behavior “all-five” requires directness, factual restraint, calibration, reasoning, and rights/scope criteria to pass together. These are small-sample, reference-free LLM-judge assessments, not factual ground truth or a general hallucination rate. The final v1 checkpoint did not pass every baseline-relative development gate: unsupported-claim flags in the user-only format exceeded its allowed ceiling. Neither earlier v1 checkpoint passed every gate either. The merged final v1 was explicitly chosen for research, not certified by this screen.
Evaluation protocol and limitations
| Setting | Political evaluation | Capability checks |
|---|---|---|
| Prompt format | User-only, no system message | User-only, no system message |
| Template | Native Qwen3.5 | Same native template |
| Decoding | Greedy | Greedy, one completion per prompt and mode |
| Thinking | Disabled | Disabled and enabled, reported separately |
| New-token cap | 1,024 | 2,048 non-thinking; 4,096 thinking |
| Political judge | Claude Opus 5.5, claude-opus-5-5 |
Not used for objective capability scores |
| Objective scoring | Not applicable | Numeric extraction; official pinned IFEval scorer; canonical HumanEval tests |
| Generated code | Not applicable | Network-disabled, read-only, unprivileged resource-bounded containers |
The expanded matrix contains 1,158 generations per model: 803 political, 35 loop controls, and 320 capability records. Both models used identical frozen tasks and generation settings within each arm. Only labeled diagnostic and development arms added system messages.
Political grading used the released rubric, blind to model identity. Stored judgments were revalidated against their exact answer hashes; required-fact partitions were checked and aggregate scores recomputed. HumanEval passed only when its canonical checks returned and the post-check marker was present, not merely when the process exited successfully.
This is not a reproduction of published Arete-27B scores: the judge and generation settings differ.[3] The small capability subsets, previously inspected public test, single political judge, inherited pretraining contamination, and rubric blind spots limit generalization. There was no comprehensive safety, multilingual, long-context, medical, legal, or agentic evaluation.
Training data and recipe
V1 used accepted Inkling final answers from NousResearch/arete-data-pairs, not original-Qwen comparison answers or teacher reasoning traces. The source authors explicitly warn that the accepted set is not clean SFT data and can still contain factual errors.[2]
The stricter local filter required review_scores.accuracy >= 4 and no review_issue_tags: 29,236 accepted source rows were inspected, 5,826 retained, 5,535 used for training, and 291 reserved as holdout. The holdout was not a separately scored v1 benchmark. Seven source topics were retained; duplicate and lexical-overlap screens were applied, but do not prove semantic or pretraining contamination is absent.
| Setting | V1 value |
|---|---|
| Method | Assistant-answer-only SFT; no DPO or RL |
| LoRA | Rank 16, alpha 16, dropout 0 |
| Targets | 200 language attention, linear-attention and MLP projection modules |
| Frozen | Vision, embeddings, output head, MTP, all other base weights |
| Optimizer | PyTorch AdamW, learning rate 2e-5, weight decay 0 |
| Schedule | 20 warmup steps, then cosine decay |
| Batch | Microbatch 1, gradient accumulation 8 |
| Training | One epoch, 692 optimizer steps |
| Sequence budget | 4,096 tokens, no packing; longest example 2,495 tokens |
| Tokens | 4,142,116 total; 3,870,244 supervised |
| Hardware | Two NVIDIA RTX 3090 GPUs, 24 GiB each |
| Trainer runtime | 72.29 minutes |
| Mean loss | 1.28034, not an answer-quality metric |
| Seed | 42 |
Training masked the full native non-thinking prompt prefix and supervised only the assistant answer plus end token. Five allocator OOM warnings were recovered; training finished with finite logged losses, gradients, and all 400 adapter tensors.
The exercised software was Python 3.12.3, PyTorch 2.13.0, Transformers 5.15.0, PEFT 0.21.2, Accelerate 1.15.0, and Flash Linear Attention 0.6.0.
Standalone merge and MTP head
The final v1 adapter was merged with PEFT merge_and_unload(safe_merge=True) into a new copy of the pinned base. All 775 tensors retained the base names, shapes, and stored dtypes. All 200 adapted weights changed; the other 575 remained identical, including original FP32 linear-attention tensors. An independent audit matched all adapted weights to the installed PEFT BF16 merge formula.
MTP included, acceleration not yet validated
The original one-layer multi-token-prediction head is preserved unchanged:
model-mtp.safetensorscontains all 15 nativemtp.*tensors.model.safetensors.index.jsonincludes their weight mappings.config.jsonretainstext_config.mtp_num_hidden_layers=1andtext_config.mtp_use_dedicated_embeddings=false.
The head was not fine-tuned with the LoRA. Presence and integrity do not establish speculative-decoding acceptance, speedup, or compatibility with every serving engine. The Transformers class used for the standalone smoke test does not execute the MTP head.
Merge verification is not benchmark equivalence
Standalone CPU loading without PEFT succeeded, and a basic generation smoke returned 4. However, an additional three-prefix CPU comparison against the unmerged adapter failed the predeclared relative L2 <= 0.05 threshold:
| Check | Measured merged versus unmerged |
|---|---|
| Relative logit L2 | 0.0584 to 0.0736 |
| Logit cosine similarity | 0.9973 to 0.9983 |
| Next-token argmax agreement | 3/3 prefixes |
| Exact saved PEFT weight formula | 200/200 adapted tensors |
Separate-adapter computation and merged BF16 weights can round differently; CPU kernel effects were not isolated. The threshold was not relaxed. No merged GPU throughput, new quality benchmark, or MTP acceptance rate is claimed.
Load the standalone checkpoint
No PEFT wrapper or separate base download is needed. This CPU smoke example uses the standalone directory and deliberately selects the PyTorch fallback because installed Triton linear-attention kernels require a GPU:
import os
import sys
os.environ["CUDA_VISIBLE_DEVICES"] = ""
sys.modules["fla"] = None
sys.modules["causal_conv1d"] = None
import torch
from transformers import AutoTokenizer, Qwen3_5ForConditionalGeneration
torch.set_num_threads(4)
model_path = "/data/llm/models/Diogenes-9B-v1-BF16"
tokenizer = AutoTokenizer.from_pretrained(model_path, local_files_only=True)
model = Qwen3_5ForConditionalGeneration.from_pretrained(
model_path,
dtype=torch.bfloat16,
device_map="cpu",
attn_implementation="sdpa",
local_files_only=True,
).eval()
prompt = tokenizer.apply_chat_template(
[{"role": "user", "content": "Reply with only the integer that equals two plus two."}],
tokenize=False,
add_generation_prompt=True,
enable_thinking=False,
)
inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False)
with torch.inference_mode():
output = model.generate(
**inputs,
do_sample=False,
max_new_tokens=16,
eos_token_id=[tokenizer.convert_tokens_to_ids("<|im_end|>")],
pad_token_id=tokenizer.pad_token_id,
)
answer = tokenizer.decode(output[0, inputs["input_ids"].shape[-1]:], skip_special_tokens=True)
print(answer.strip())
The expected smoke output is 4. The CPU fallback lines are specific to this CPU example, not recommended GPU-serving configuration. GPU serving, runtime conversion, and MTP execution need their own checks. Use the copied native template and explicitly choose thinking or non-thinking mode; do not assume an output cap prevents repetition.
Provenance
| Component | Immutable identity |
|---|---|
| Base | Qwen/Qwen3.5-9B at c202236235762e1c871ad0ccb60c8ee5ba337b9a |
| Training data | NousResearch/arete-data-pairs at 664cc9b71679199b76c49b8069eaed23bd618f80 |
| Political evaluation | NousResearch/Propaganda_Eval at 401223a2bfcdcd99d4650a83c4d20565db0e56b8 |
| V1 adapter SHA-256 | 4c41e1a03fab702fa8726964ef92e3d0bf8179e9297dba14c5a8acbe5e2c8483 |
| Expanded task manifest SHA-256 | 322d99f432a564e2526aaee314e318a4b242da80e2d94b518122b2b857277bb8 |
MERGE_MANIFEST.json records per-file source and output hashes. benchmark-results.json contains the complete aggregate evidence and selected capability IDs. The benchmark source revisions for GSM8K, IFEval, HumanEval, and the official IFEval scorer are retained there as immutable URLs or commits.
Intended use, licensing, and credit
Use this artifact to study fine-tuning, framing, and behavior transfer. Hallucinations, repetition, coding regressions, and calibration defects remain. Independently verify consequential answers; do not treat this model as a reliable authority for safety-critical decisions.
The base declares Apache-2.0.[1] No training-dataset license declaration or LICENSE file was found at the pinned revision during this experiment; its reuse and redistribution terms remain unresolved.[2] The political evaluation declares MIT, with underlying cited documents retaining their own terms.[3] The base license does not settle the resulting fine-tune's distribution rights, so this card does not assign an unsupported release license.
The header uses the exact project-owner-supplied image from the requested source, copied locally so the README does not depend on the preview server. Artwork and trademark clearance are not established by this card. No third-party endorsement is implied.
Credit: Qwen for the base and tooling; Nous Research for Arete, the dataset and evaluation; Inkling for source teacher answers; TendieLabs for this experiment and its local measurements.
Sources
[1] https://huggingface.co/Qwen/Qwen3.5-9B/raw/c202236235762e1c871ad0ccb60c8ee5ba337b9a/README.md - Pinned Qwen3.5-9B model card [2] https://huggingface.co/datasets/NousResearch/arete-data-pairs/raw/664cc9b71679199b76c49b8069eaed23bd618f80/README.md - Pinned Arete training dataset card [3] https://huggingface.co/datasets/NousResearch/Propaganda_Eval/raw/401223a2bfcdcd99d4650a83c4d20565db0e56b8/README.md - Pinned political evaluation card and rubric
- Downloads last month
- 44