README.md

---
base_model:
- whitecircle/GLM-4.7-Flash-Coder
license: apache-2.0
pipeline_tag: text-generation
library_name: transformers
tags:
- moe
- mixture-of-experts
- glm
- glm-4.7-flash
- code
- coder
- nvfp4
---
# Model Card



Source Model: [whitecircle/GLM-4.7-Flash-Coder](https://huggingface.co/whitecircle/GLM-4.7-Flash-Coder)




<div style="max-width: 70%; margin: 0 auto; text-align: center; font-weight: bold;">
  THIS REPO:
</div>
<div style="max-width: 70%; margin: 0 auto; text-align: center; font-weight: bold;">
 NVFP4 quantization for use on Blackwell gpus sm100 sm120 sm121.
</div>


Ready to serve on latest vLLM branch. 

Please take note of the server configuration below.

For MTP repo, check our [cleyesode/GLM-4.7-Flash-Coder-MTP-NVFP4](https://huggingface.co/cleyesode/GLM-4.7-Flash-Coder-MTP-NVFP4)


Quantization via NVIDIA/Model-Optimizer custom workflow:
| Component | Precision |
|---|---|
| Routed experts | NVFP4 W4A4 (group 16, FP8 block scales, FP32 global scales) |
| Shared expert, router, attention, embeddings, `lm_head` | BF16 |
| MTP | no MTP in this repo |
| KV cache | Not quantized |
Please see post-quantization [verification_report](https://huggingface.co/cleyesode/GLM-4.7-Flash-Coder-NVFP4/blob/main/verification_report.txt) for details. 



Calibrated on curated samples from [Nickyang/RazorCal](https://huggingface.co/datasets/Nickyang/RazorCal):


| Domain                    | Count   | Percentage | Description / Sub-sources                                                           |
| :------------------------ | :------ | :--------- | :---------------------------------------------------------------------------------- |
| **Coding**                | 57      | \~24.1%    | Algorithmic Python/C++, competitive programming, and GitHub SWE issue resolution.   |
| **Math**                  | 41      | \~17.3%    | High-level math derivations, AIME-style problems, calculus, and geometry.           |
| **Chinese-STEM**          | 33      | \~13.9%    | Multilingual math, coding, and science MCQs written in Chinese.                     |
| **Instruction Following** | 32      | \~13.5%    | Strict formatting constraints (e.g., word count, excluded words, required letters). |
| **World Knowledge**       | 30      | \~12.7%    | Trivia, history, reading comprehension, and translation tasks.                      |
| **Tool Calling**          | 25      | \~10.5%    | Multi-turn agentic workflows using external APIs (weather, stocks, ecommerce).      |
| **Science (STEM)**        | 19      | \~8.0%     | Chemistry, physics, biology, and thermodynamics (both MCQs and long-form).          |




## Serving with vLLM

Example: 4 x 5060Ti 16GB | tensor parallel 4 with expert parallelism | single sequence at 175k context

**Note:** kv-cache-dtype quants not well supported on this architecture in vllm - BF16 kv config shown here

```bash
export CUDA_DEVICE_ORDER=PCI_BUS_ID
export CUDA_VISIBLE_DEVICES=0,1,2,3
export PYTORCH_CUDA_ALLOC_CONF="expandable_segments:True"
export VLLM_USE_RUST_FRONTEND=0


vllm serve cleyesode/GLM-4.7-Flash-Coder-NVFP4 \
    --served-model-name GLM-4.7-Flash-Coder-NVFP4 \
    --host 0.0.0.0 \
    --port 8000 \
    --distributed-executor-backend mp \
    --tensor-parallel-size 4 \
    --enable-expert-parallel \
    --kv-cache-memory 9567537324 \
    --max-model-len 175000 \
    --max-num-seqs 1 \
    --max-num-batched-tokens 1024 \
    --enable-prefix-caching \
    --enable-chunked-prefill \
    --tool-call-parser glm47 \
    --reasoning-parser glm45 \
    --enable-auto-tool-choice \
    --attention-backend triton_mla \
    --moe-backend flashinfer_cutlass \
    --kv-cache-dtype bfloat16 \
    --no-enable-flashinfer-autotune \
    --trust-remote-code
```


## Memory Requirements

  - VRAM: ~63GiB (per the above tp4 example)


## Metrics (compute-only)

Very impressive generation speed absent MTP.

Throughput metrics collected on **real prompts**:


| Input tokens | Output tokens | TTFT (ms) | TTFT p99 (ms) | Prefill (tok/s) | TPOT (ms) | ITL p99 (ms) | **Decode (tok/s)** | E2E latency (s) | Draft acceptance | Mean accepted length |
|-------------:|--------------:|----------:|--------------:|----------------:|----------:|-------------:|---------------:|----------------:|-----------------:|---------------------:|
|          956 |           512 |       104 |           135 |           9,158 |       9.0 |          9.8 |          **111.0** |             4.7 |               -- |                   -- |
|        8,127 |           512 |     1,085 |         1,094 |           7,493 |      10.3 |         11.1 |           **96.9** |             6.4 |               -- |                   -- |
|       32,705 |           512 |     8,759 |         8,769 |           3,734 |      14.4 |         15.0 |           **69.3** |            16.1 |               -- |                   -- |
|       65,451 |           512 |    29,482 |        29,484 |           2,220 |      19.8 |         20.5 |           **50.4** |            39.5 |               -- |                   -- |
|      126,975 |           512 |   101,749 |       101,809 |           1,248 |      28.9 |         29.7 |           **34.6** |           116.6 |               -- |                   -- |

## Benchmarks

  **No performance benchmarks completed for NVFP4 model at this time**




---

Source Repo Model Card:

# Introducing ⚡ GLM-4.7-Flash-Coder ⚡

We fine-tuned GLM-4.7-Flash to push its agentic coding capabilities further: from 33.15% to 41.65% on SWE-rebench-V2 (+26% relative improvement) at 14% lower inference cost.

This 30B Mixture-of-Experts model beats competitors 3.5-4× its size — Qwen3.5-122B-A10B and Devstral-2 — and surpasses Ling-2.6-Flash by a narrow margin.

We trained it with [Halo](https://github.com/whitecircle/halo), the post-training framework we’re open-sourcing today — the full story of the framework [here](https://whitecircle.com/research).

![Score vs cost-per-task scatter plot on SWE-rebench-V2, 500 Python tasks: GLM-4.7-Flash-Coder at 0.417 sits 26% above GLM-4.7-Flash at 0.332, at a lower cost per task](media/score_vs_cost.png)

## More reasoning, fewer turns

Although GLM-4.7-Flash-Coder does 2.4× more reasoning than the base model, it completes tasks faster: 45% fewer turns, which translates to 14% lower inference cost.

![Box plots for base vs SFT model: median reasoning tokens per response 63 vs 150, assistant turns per task 82.5 vs 44, trace token length 52k vs 39k, average cost per task $0.0443 vs $0.0382](media/trace_overview.png)

## The key to turn efficiency is concurrent tool calls

The fine-tuned model learned to batch its tool use instead of working one call at a time. Turns with two or more parallel `read` calls jumped from 1.6% to 9.7% of all turns, parallel `bash` from 0.6% to 7.2%, and parallel `edit`, which is almost nonexistent in the base model, grew ~90×. Reading three files in one turn instead of three turns is exactly the kind of habit that compounds into 45% fewer turns overall.

![Frequency of responses with 2+ read, bash, and edit tool calls: the SFT model issues simultaneous tool calls 6–90× more often than the base model](media/simultaneous_tool_calls.png)

## SWE-rebench-V2

We trained and evaluated models on [nebius/SWE-rebench-V2](https://huggingface.co/datasets/nebius/SWE-rebench-V2), a dataset of 32k agentic coding tasks. It consists of real issues from public repositories, closely representing real-world SWE challenges.

![SWE-rebench-V2 pipeline: a repository and an issue enter an agentic loop in a sandbox, the generated patch is checked by the test-suite verifier, and the reward is 1 if all tests pass](media/SWE-rebench-V2-pipeline.png)

## Teacher — GLM-5.1

The training data comes from GLM-5.1 acting as a teacher. We ran it on the train split with 4 rollouts per task and kept only trajectories that passed the tests: 7,777 successful multi-turn traces after rejection sampling.

We’re releasing the [training dataset](https://huggingface.co/datasets/whitecircle/swe-rebench-v2-glm-5.1-pi-agent-successful-traces) alongside the [train/test splits](https://huggingface.co/datasets/whitecircle/swe-rebench-v2-clean-python-tasks).

All models run inside PI — a lightweight agent harness with four tools: `bash`, `read`, `edit`, `write`.

## Training

We trained Coder with Supervised Fine-Tuning (SFT) using [Halo](https://github.com/whitecircle/halo), our post-training framework we’re open-sourcing today 🎉

- Parallelism mode: DP=16 (multi-node training)
- Context length: 75k
- Packing: Enabled
- Global batch: 16
- Learning rate scheduling: Cosine, with linear warm-up
- Training time: 3 hours on 16 B300 GPUs

### Halo 😇 is driven entirely by a single YAML config

```yaml
model_name_or_path: zai-org/GLM-4.7-Flash
trust_remote_code: true
attn_implementation: flash_attention_4_hybrid

dataset:
- whitecircle/swe-rebench-v2-glm-5.1-pi-agent-successful-traces
test_size: 0.01

conversation_field: messages
tools_field: tools
interleaved_thinking: true

max_length: 75000
packing: true

learning_rate: 1e-5
lr_scheduler_type: cosine
warmup_steps: 25
per_device_train_batch_size: 1
per_device_eval_batch_size: 1

moe_balancing: none
liger_kernel_config:
  fused_linear_cross_entropy: true

assistant_message_template: "<|assistant|>"

output_dir: /mnt/research/GLM-4.7-Flash-Coder
save_steps: 50
save_only_model: true

eval_strategy: steps
eval_steps: 50
logging_steps: 1
logging_first_step: true
log_decoded_samples: true

report_to: wandb
project_name: agentic-sft
run_name: glm-4.7-flash-coder

enable_efficiency_metrics: true
```

## Emergent skills

The model picked up several of the teacher’s skills and patterns during fine-tuning:

- A 10× higher rate of self-corrections in reasoning,
- Self-directed questions (“But wait, could this ever be True?”),
- Advanced tool calling patterns, e.g. setting timeout for the `bash` tool.

![Bar charts of self-corrections, question marks in reasoning, git stash usage, and timeouts passed to bash across base, SFT, and teacher models](media/learned_patterns.png)

Perhaps the most surprising transfer was the `git stash` usage:

> *Hmm, actually these look like pre-existing test differences unrelated to my fix. Let me revert my fix and run the test again.*
>
> ```bash
> git stash && python -m pytest tests/test_markdown.py::test_markdown_table -xvs 2>&1 | tail -20
> ```

## Loss decomposition into Reasoning vs. Tool Calls

We only train on assistant responses. Of the 177 million tokens in the dataset, only 46 million carry loss — the remaining 131 million (system prompts and tool responses) are masked.

Within an assistant message there are two very different kinds of tokens, mainly reasoning and tool calls. Tool-call tokens are far lower-entropy:

![A packed training row rendered with per-token probabilities: reasoning tokens in grey, tool call tokens in pink, masked tokens untrained, dimmer shades meaning higher probability](media/reasoning-tool-calls-logprobs.png)

Although tool-call tokens make up 43.3% of input tokens, they contribute only 16.1% of the overall loss. To track convergence separately, we decompose the loss into two components: negative log-likelihood on reasoning tokens and on tool-call tokens.

![Train and eval loss curves decomposed into reasoning and tool-call components with epoch boundaries; every eval loss reaches its global minimum at step 300](media/train_loss_decomposition.png)

The training-loss curves show 10–20% cliffs at epoch boundaries — the usual signature of memorizing repeated data. We read it as benign: eval losses keep improving through the second epoch.

It’s possible to set different learning rates for the two components, but on our data both eval losses reach their global minima at the same moment (the end of epoch 2) so we didn’t need to.

## Test scores

Below is the comparison of GLM-4.7-Flash, GLM-4.7-Flash-Coder, and its teacher GLM-5.1 on test set of SWE-rebench-V2:

| metric | GLM-4.7-Flash | GLM-4.7-Flash-Coder | Δ | Teacher GLM-5.1 |
| --- | --- | --- | --- | --- |
| pass@1 | 0.3315 | 0.4165 | +25.6% | 0.5650 |
| pass@2 | 0.4087 | 0.5023 | +22.9% | 0.6063 |
| pass@4 | 0.4620 | 0.5720 | +23.8% | 0.6300 |

## How we enabled FA4

FlashAttention-4 is a fast attention kernel — but it’s incompatible with GLM-4.7-Flash’s architecture:

> *FlashAttention-4 produces NaN gradients for this model’s head_dim-256 partial-rotary attention; falling back to `attn_implementation='sdpa'`.*

We found that the incompatibility only appears in the backward pass; the forward computation is correct. So we designed a hybrid scheme: FA4 handles the forward pass, while FA2 takes over for the backward. The hybrid approach gave a 1.36× increase in attention throughput.

*We’ll have a separate post on FA4-fwd-FA2-bwd in [our research blog](https://whitecircle.com/#research); stay tuned! 💚*

## Kernels

GLM-4.7-Flash is a Mixture-of-Experts model. Tokens are distributed unevenly across experts, so the MLP blocks receive tensors of varying width. The Grouped-GEMM kernel batches matrix multiplications of varying shapes into a single kernel launch. Enabling it gave a 1.6× speedup on the FFN step.

Additionally, with Liger Fused Linear Cross-Entropy, we were able to compute loss without full logit matrix materialization in VRAM, preventing OOM on long sequences.

Together, these improvements yield up to a 1.63× training speedup over TRL, raising throughput to 11.1k tokens/s on a single B300 GPU on context length 4k:

![GLM-4.7-Flash SFT throughput on a single B300 across context lengths from 4k to 128k: Halo FA4-hybrid leads Halo FA2 and TRL at every length; TRL OOMs at 128k](media/glm47flash_throughput.png)

## Agentic infra challenge

Agentic evaluation at scale turned out to be a real infrastructure problem.

With thousands of concurrent agents, Docker brings high startup overhead and heavy disk pressure, and quickly saturates the local network — producing a high rate of environment startup errors.

Kubernetes struggled at high concurrency on a single host as well: it couldn’t set up and tear down pods fast enough, triggering timeouts. The problem was worst on powerful hosts: they could support enormous concurrency, but pod scheduling couldn’t keep up, so we never reached full CPU or RAM utilization.

## Overlay sandboxes

Our solution is the Overlay Sandbox: instead of a container image, the agent runs on the bare host, inside an isolated filesystem and process group.

It has three layers: a base filesystem with the OS and repository, a hidden layer with the verifier suite, and an upper layer holding all edits and pip-installed packages.

Because overlay mounts are lazy (they don’t even index the filesystem!) and track only modified files, sandbox startup is essentially free.

![Overlay sandbox diagram: a read-only base filesystem, hidden verifier files not visible to the agent, and a writable lazy overlay mount holding edits and new files](media/overlay_sandbox.png)

Since SWE-rebench-V2 is built from public repositories, agents must run without network access — otherwise, a model can simply fetch a real fix from GitHub. Before restricting network, we caught Fable 5 and Sonnet 5 doing exactly this — retrieving the upstream patches directly on 20-40% of tasks.

## Traffic recorder

Our sandbox also includes a traffic recorder — a tiny proxy that forwards requests to the provider (OpenRouter / vLLM / SGLang) and records every response. It’s a harness-agnostic way to assemble full trajectories in OpenAI format and collect performance metrics.

![Traffic recorder diagram: a proxy between the agent and the provider records raw requests and assembles them into an OpenAI-format trace](media/traffic_recorder.png)

## Evaluation

Every agent runs with `max_turns = 200`, medium reasoning effort, and default sampling parameters.

Inference costs are calculated from OpenRouter pricing tables. We report the theoretical price under the assumption of perfect caching — actual OpenRouter costs were higher due to inconsistent routing. For locally served GLM-4.7-Flash-Coder, we applied the per-request pricing of GLM-4.7-Flash via OpenRouter’s DeepInfra provider.

<details>
<summary>OpenRouter providers per model</summary>

| model | provider | input $/Mtok | output $/Mtok | cache_read $/Mtok | cache_write $/Mtok |
| --- | --- | --- | --- | --- | --- |
| Fable-5 | google-vertex/global | 10 | 50 | 1 | 12.5 |
| Opus-4.8 | anthropic | 5 | 25 | 0.5 | 6.25 |
| GLM-5.1 | wafer/fp4 | 1 | 3.2 | 0.1 |  |
| MiniMax-M3 | minimax/fp8 | 0.3 | 1.2 | 0.06 |  |
| DeepSeek-V4-Pro | deepinfra/fp4 | 1.3 | 2.6 | 0.1 |  |
| GPT-5.5 | azure | 5 | 30 | 0.5 |  |
| Kimi-K2.6 | siliconflow/fp8 | 0.77 | 3.4 | 0.14 |  |
| Haiku-4.5 | anthropic | 1 | 5 | 0.1 | 1.25 |
| Qwen3.5-397B-A17B | deepinfra/fp8 | 0.45 | 3 | 0.22 |  |
| GPT-5.4-Mini | azure | 0.75 | 4.5 | 0.075 |  |
| Gemini-3.1-Pro | google-ai-studio | 2 | 12 | 0.2 | 0.375 |
| GLM-4.7-Flash-Coder | deepinfra/bf16 | 0.06 | 0.4 | 0.01 |  |
| Ling-2.6-Flash | novita | 0.1 | 0.3 | 0.02 |  |
| Devstral-2 | mistral | 0.4 | 2 | 0.04 |  |
| Qwen3.5-122B-A10B | alibaba | 0.4 | 3.2 |  |  |
| GLM-4.7-Flash | deepinfra/bf16 | 0.06 | 0.4 | 0.01 |  |
| Gemini-3.5-Flash | google-vertex/global | 1.5 | 9 | 0.15 | 0.08333 |
| Qwen3-32B | deepinfra/fp8 | 0.08 | 0.28 |  |  |
| Qwen3.6-35B-A3B | parasail/fp8 | 0.15 | 1 | 0.05 |  |
| Gemini-2.5-Flash | google-vertex/global | 0.3 | 2.5 | 0.03 | 0.08333 |
| Nemotron-3-Super | digitalocean | 0.21 | 0.455 | 0.06 |  |
| Solar-Pro-3 | upstage | 0.15 | 0.6 | 0.015 |  |

</details>

<details>
<summary>Prompt template</summary>

```markdown
You are solving a real GitHub issue in the `{repo}` repository. The repo is already cloned and set up in the current working directory.

## Workflow

1. **Understand the issue.** Read the problem statement carefully. Identify the expected vs actual behavior.
2. **Explore the codebase.** Find the relevant files using grep, find, or by reading the project structure. Read the code that needs to change.
3. **Reproduce the bug** if possible — run the relevant test file or write a small script to confirm the issue exists.
4. **Implement a minimal fix.** Change only what is necessary. Do not refactor unrelated code, do not add features beyond what the issue asks for.
5. **Verify your fix.** Run the project's existing test suite to make sure you haven't introduced regressions. Look at the project's test configuration (Makefile, tox.ini, pyproject.toml, CI config) to find the right test command.
6. **Mark the task complete** only after you've verified existing tests pass with your changes.

## Issue

{problem_statement}

## Expected Interfaces

The following interfaces are expected by the test suite. Pay close attention to function names, parameter signatures, and return types — the tests verify these exactly.

{interface}
```

</details>

<details>
<summary>Inference with SGLang 0.5.14, CUDA 13</summary>

```bash
python3 -m sglang_router.launch_server \
  --model-path whitecircle/GLM-4.7-Flash-Coder \
  --trust-remote-code \
  --host 0.0.0.0 --port 30000 \
  --context-length 202752 \
  --prefill-attention-backend trtllm_mla \
  --decode-attention-backend triton \
  --page-size 64 \
  --tool-call-parser glm47 --reasoning-parser glm45 \
  --router-policy cache_aware \
  --tp-size 1 --dp-size 8
```

</details>

## Team

- Project lead: Oleg Smirnov
- Agentic sandboxes: Konstantin Korolev, Sergei Bratchnikov
- Contributors: Mohamed Mekkouri, Dmitrii Kharlapenko, Daniil Oblakov

A [White Circle](https://whitecircle.com) project