Qwen3.8-27B-CTL

Qwen3.8-27B fine-tuned for CTL2, the CloverDX Transformation Language: the language in which CloverDX data integration graphs implement their business logic (Reformat, Rollup, Denormalizer, Normalizer, joiners, DataGenerator, Partition and other transform components).

The model:

  • writes CTL2 transform code from a task description and record metadata;
  • explains the language, its built-in functions and component contracts;
  • reviews CTL2 code and reports findings as [ERROR] / [WARNING] lines;
  • fixes code from compiler or runtime error messages.

It is meant to run with thinking on, with the system prompt shipped in this repository, and inside a loop that compiles and runs the generated code and feeds errors back (see How to use it).

Training

Everything is merged into the weights: this is a full bf16 checkpoint, and no adapter is needed at serving time.

stage what
base Qwen/Qwen3.8-27B
SFT LoRA SFT on dpavlis/ctl_lora_sft_data: answer-only examples mixed with thinking traces
DPO LoRA DPO on the dataset's preference pairs (invented functions, Java/Python habits, wrong component contracts, false language beliefs)
on-policy context distillation the model learns, on its own samples, to behave as it does when CTL2 rules and review guidance are in its prompt; the last round trained code review on a mix of defective and clean code, which reduced false alarms on correct code

CTL2 facts in the training data and in the system prompt were checked against the CTL compiler and a live CloverDX server.

The checkpoint keeps the base model's vision tower weights so that it loads with the base model's architecture, but it was trained and evaluated on text only.

How to use it

Serving with vLLM

vllm serve dpavlis/Qwen3.8-27B-CTL --served-model-name ctl2 \
  --dtype bfloat16 --max-model-len 16384 --max-num-seqs 64 --enable-prefix-caching

The bf16 weights take about 51 GB, so the model needs one GPU with 80 GB or more, or tensor parallelism.

Set --max-num-seqs explicitly. vLLM's default of 1024 exceeds the Mamba cache blocks of this hybrid architecture, and the server refuses to start. All evaluations ran with 64.

Request settings

setting value
system prompt the full text of system_prompt.txt, verbatim, for both code generation and review
thinking on: chat_template_kwargs: {"enable_thinking": true, "reasoning_effort": "medium"}
sampling temperature 0.4, top_p 0.95, top_k 20
max tokens 16384

The system prompt is part of the deployed configuration, not decoration. It carries CTL2 rules that the model follows more reliably in context than from training alone: reserved words, regex and string-escape syntax, exception handling, Rollup accumulators, the DataIntersection transform, strict date parsing and day arithmetic on UTC dates.

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="none")
system_prompt = open("system_prompt.txt").read()
response = client.chat.completions.create(
    model="ctl2",
    messages=[
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": "Write a CTL2 Reformat transform that copies all fields and parses the string "
                                    "field amount_text (values like \"$(1,250.00)\") into the decimal field amount."},
    ],
    temperature=0.4, top_p=0.95, max_tokens=16384,
    extra_body={"top_k": 20, "chat_template_kwargs": {"enable_thinking": True, "reasoning_effort": "medium"}},
)
print(response.choices[0].message.content)

Compile and retry

Validate the generated code (CTL compiler or a CloverDX server), run it on test data, and when something fails send the error message back as the next user turn. Allow up to two retries.

Evaluation

These are internal measurements. Generated code was run on a CloverDX server and scored on whether it produced the expected output rows, not by an LLM judge.

test result
new-domain code generation tasks (rewritten tasks unlike the training prompts), thinking on 0.93 pass on the first attempt, 0.94 after the compile-and-retry loop
code review of known-clean code, thinking on almost no false [ERROR] findings
code review of code with planted, server-confirmed defects, thinking on about 0.95 of the defects found

With thinking off, the model is markedly weaker at code generation and flags clean code as wrong far more often. Use thinking on.

Limitations

  • CTL2 only: it knows CloverDX graphs as context for CTL, but it is not trained to design whole graphs or to write component XML.
  • Rare components and their contracts (for example job-control components such as GetJobInput, SetJobOutput or ExecuteScript) are weak without a description of the component in the prompt.
  • Like any code model, it can produce code that compiles and runs but does not do what was asked. Test generated code on real data before using it.
Downloads last month
-
Safetensors
Model size
27B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for dpavlis/Qwen3.8-27B-CTL

Base model

Qwen/Qwen3.8-27B
Finetuned
(910)
this model

Dataset used to train dpavlis/Qwen3.8-27B-CTL