vcruz305/DeepNemotron-3.5-Lightning · Hugging Face

Hugging Face

vcruz305 / DeepNemotron-3.5-Lightning like 0

Text Generation

Transformers

Safetensors

nemotron_h

nemotron

lightning

deeplightning

lora

Merge

conversational

License: other

Deploy

Copy to bucket new

Use this model

DeepNemotron-3.5-Lightning v4 fullrun (DeepLightning corrected v4 LoRA merged, MTP-aware)

Merged checkpoint of the DeepLightning v4 fullrun LoRA trained on nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 (NeMo Automodel, corrected v4 config, r=16 / alpha=32, two repeated MTP layers, dataset 19,573 rows, one epoch = 2,447 steps, LR=2e-5, sequence length=4096, GBS/LBS=8/1, seed=1111, final adapter at epoch_0_step_2446).

The LoRA was merged into the base weights (architecture-aware merge preserving MTP tensors) so this repo is a standalone dense BF16 model — no PEFT adapter needed at inference.

Architecture

• Nemotron-H hybrid: Mamba + MoE + repeated MTP layers (GB10 / DGX Spark class). Query the model with vLLM + --mamba-backend flashinfer for correct Mamba kernel dispatch. MTP tensors verified present (6513 keys, 270 MTP keys).
• dtype : bfloat16, max_model_len : 8192+.

Serving (recommended)

vllm serve . --trust-remote-code --mamba-backend flashinfer \
  --enable-auto-tool-choice --tool-call-parser hermes \
  --max-model-len 8192 --gpu-memory-utilization 0.85

(vLLM 0.1.dev1+; causal-conv1d / mamba-ssm fused kernels are NOT required when using the flashinfer mamba backend.)

Benchmarks (merged BF16, Sixcat v0.4.0 strict/thinking-off, limit 20)

sixcat (limit=20, ~180 items), minus tools (harness expects OpenAI tool-call JSON; this Hermes-trained model emits Hermes-XML <tool_call> blocks — a wire-format mismatch, not a capability gap):

Category	Score	n
knowledge	60
math	40
truth	85
instruct	45
code	30
tools*	80
overall	56.67

hermes-agentic-bench loop_gate (20 tasks): 19/20 pass, 0 caps, 0 parse-fail, 0 dup.

*tools/agentic parse failures are a harness wire-format artifact (Hermes-XML vs OpenAI JSON), not a model defect. The model selects the correct tool and args.

Comparison vs stock baseline (v040-stock.json, same sixcat v0.4.0 identity): +4.17 overall (52.5 → 56.67). All categories at or above stock (K+15, M+5, T=0, I+5, C=0 ceiling, T=0 floor).

Files

• model.safetensors (single 65.8 GB file)
• config.json , generation_config.json , tokenizer*.json , chat_template.jinja

Downloads last month

Safetensors

Model size

32B params

Tensor type

F32

·

BF16

·

Chat template

Files info

Inference Providers NEW

Text Generation

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for vcruz305/DeepNemotron-3.5-Lightning

Base model

nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16

Adapter

( 5 )

this model

Adapters

2 models

Quantizations

1 model

---

Note: Parallel NVFP4 quantization (lightning_w4a16_nvfp4_4o6 Four-Over-Six recipe) is in progress on the same merge. Accepted NVFP4 will be uploaded to vcruz305/DeepNemotron-3.5-Lightning-NVFP4 only after Gate 5 passes. This repo contains the BF16 merge only.

Training details (corrected v4 fullrun): 19,573-row dataset (SHA-256 verified), one shuffled epoch = 2,447 steps, LR=2e-5, LoRA rank/alpha=16/32, sequence length=4096, GBS/LBS=8/1, seed=1111, two repeated MTP layers, checkpoint every 250 steps, recovery retention latest two checkpoints. Final adapter epoch_0_step_2446.

Sixcat identity (both stock and candidate): sixcat-eval==0.4.0, git tag v0.4.0 commit 82b937fdb369df9a013f2ed4ca96dc78f75205af, parser v4, challenge-v1 (fingerprint 05c04833fcdb), policy fingerprint 8ad5460a8ed5, strict, thinking off, limit 20, max-minutes 30.

NVFP4 Quantization Plan (in parallel)

See the full plan at the source workspace for Phase 0–5 gates, official lightning_w4a16_nvfp4_4o6 recipe, layer policy (MoE experts W4A16 NVFP4 Four-Over-Six, Mamba FP8, Attention BF16, KV FP8), and success criteria (≥98–99% recovery of BF16 agentic metrics).

End of README