Lamapi commited on
Commit
49f50e4
路
verified 路
1 Parent(s): 913f0e5

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +154 -16
README.md CHANGED
@@ -1,23 +1,161 @@
1
- # Next 3 Pro TurboQuant
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
 
3
- Q8_0 weights packaged for llama-tq compatibility with MTP disabled.
4
- TurboQuant compresses the KV cache during inference.
5
- Tested llama-tq commit: \`f572e7348edafba2df338a50e3fd8403b8855d4b\`.
6
 
7
- \`\`\`bash
8
- llama-completion -m Next-3-Pro-Q8_0-TurboQuant-Compat-NoMTP.gguf -ngl 999 \
9
- -c 8192 -fa on -ctk ktq4_1 -ctv vtq4_1 -p "Merhaba" -n 128
10
- \`\`\`
11
 
12
- ## A100 80 GB
13
 
14
- Median of three runs per mode, 8,192-token context capacity,
15
- a short prompt and a 128-token output budget.
16
 
17
- | KV cache | Decode tok/s | GPU context memory |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
18
  |---|---:|---:|
19
- | q8_0 / q8_0 | 42.43 | 421 MiB |
20
- | ktq4_1 / vtq4_1 | 42.24 | 565 MiB |
 
 
 
 
 
 
 
 
 
21
 
22
- TurboQuant used 144 MiB more GPU context memory and decoded
23
- 0.45% slower in this test.
 
1
+ ---
2
+ base_model: Qwen/Qwen3.8-27B
3
+ pipeline_tag: image-text-to-text
4
+ library_name: transformers
5
+ language:
6
+ - en
7
+ - tr
8
+ tags:
9
+ - lamapi
10
+ - next-3-pro
11
+ - multimodal
12
+ - vision-language
13
+ - coding
14
+ - reasoning
15
+ - tool-use
16
+ - long-context
17
+ - instruction-tuned
18
+ datasets:
19
+ - openai/gsm8k
20
+ - google-research-datasets/mbpp
21
+ ---
22
 
23
+ ![image](https://cdn-uploads.huggingface.co/production/uploads/67d46bc5fe6ad6f6511d6f44/Xw3ra-HPaKbWXd8KueFsz.png)
 
 
24
 
25
+ # Next 3 Pro
 
 
 
26
 
27
+ **A 27B multimodal model from Lamapi for building software, working through complex problems, and understanding documents, images, and video.**
28
 
29
+ Give Next 3 Pro a feature brief, a screenshot of a broken interface, or a failing test. Ask it for a working first draft, then bring back the browser error or the edge case it missed. The model supports technical conversations in English and Turkish, including projects where the discussion and the code use different languages.
 
30
 
31
+ [Main checkpoint](https://huggingface.co/thelamapi/Next-3-Pro) 路 [GGUF builds](https://huggingface.co/thelamapi/Next-3-Pro-GGUF) 路 [TurboQuant compatibility build](https://huggingface.co/thelamapi/Next-3-Pro-TurboQuant)
32
+
33
+ ## Model overview
34
+
35
+ | | |
36
+ |---|---|
37
+ | **Developer** | [Lamapi](https://huggingface.co/thelamapi) |
38
+ | **Model family** | Next 3 |
39
+ | **Base model** | [Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) |
40
+ | **Model size** | 27B class |
41
+ | **Release format** | Merged Transformers checkpoint |
42
+ | **Modalities** | Text, images, and video |
43
+ | **Fine-tuning focus** | Coding, reasoning, technical instruction following, and English and Turkish interaction |
44
+
45
+ ## What it supports
46
+
47
+ The full Next 3 Pro checkpoint retains the Qwen3.8-27B architecture and its supported input modalities and interfaces. The sections below describe inherited capabilities. See the [upstream model card](https://huggingface.co/Qwen/Qwen3.8-27B) for the base model鈥檚 runtime configuration and examples.
48
+
49
+ | Capability | What you can do with it |
50
+ |---|---|
51
+ | **Coding and debugging** | Draft features, inspect errors, propose patches, write tests, and revise code as requirements change. |
52
+ | **Mathematical and technical reasoning** | Work through calculations, compare alternatives under constraints, and explain solutions with concrete examples. |
53
+ | **Image understanding** | Analyze screenshots, diagrams, scanned pages, and visual layouts. Use the multimodal processor with the full checkpoint. |
54
+ | **Video understanding** | Summarize a clip, track changes across frames, and answer questions about visual events. Video preprocessing and practical limits depend on the runtime. |
55
+ | **Tool calling and agent workflows** | Supply tool schemas, let your application execute permitted calls, and return tool results to the model. |
56
+ | **Thinking controls** | Choose direct answers or thinking mode where the serving stack exposes controls such as `enable_thinking`, `reasoning_effort`, and `preserve_thinking`. |
57
+ | **Long-context input** | Use the upstream context configuration for large inputs. Qwen documents a native context length of 262,144 tokens and an extended configuration up to 1,000,000 tokens. Lamapi has not benchmarked this fine-tuned checkpoint at those lengths. |
58
+
59
+ ## Try a two-turn coding workflow
60
+
61
+ **First prompt**
62
+
63
+ > Build a task board that runs from a single `index.html` file. I need three columns, task creation, title editing, and drag-and-drop between columns. Persist the board in `localStorage`. Use vanilla HTML, CSS, and JavaScript. Return the complete file.
64
+
65
+ **Follow-up in the same conversation**
66
+
67
+ > Add search and undo for the last move. Moving a card while a search filter is active must move the correct card. If I rename a card after moving it, undo must restore its position and keep the new title. Existing saved boards must still load. Return the complete updated file.
68
+
69
+ **Acceptance test:** Add two cards with similar titles, filter the board, move one card, edit its title, undo the move, and refresh. The edited card should return to its original column, retain the new title, and stay there after reload.
70
+
71
+ ## Quickstart with Transformers
72
+
73
+ Use a Transformers release that supports Qwen3.8-27B and a runtime with sufficient memory for the BF16 weights and inference context. If repository access requires authentication, sign in to Hugging Face or set `HF_TOKEN` in your environment.
74
+
75
+ ```python
76
+ import os
77
+
78
+ import torch
79
+ import transformers
80
+ from transformers import AutoConfig, AutoProcessor
81
+
82
+ model_id = "thelamapi/Next-3-Pro"
83
+ token = os.environ.get("HF_TOKEN")
84
+
85
+ config = AutoConfig.from_pretrained(model_id, token=token)
86
+ model_class = getattr(transformers, config.architectures[0])
87
+ model = model_class.from_pretrained(
88
+ model_id,
89
+ token=token,
90
+ dtype=torch.bfloat16,
91
+ device_map="auto",
92
+ ).eval()
93
+
94
+ processor = AutoProcessor.from_pretrained(model_id, token=token)
95
+ messages = [{
96
+ "role": "user",
97
+ "content": [{
98
+ "type": "text",
99
+ "text": (
100
+ "Write a Python function that sorts a list of dictionaries by "
101
+ "date without mutating the input. Include one small test."
102
+ ),
103
+ }],
104
+ }]
105
+
106
+ inputs = processor.apply_chat_template(
107
+ messages,
108
+ tokenize=True,
109
+ add_generation_prompt=True,
110
+ enable_thinking=False,
111
+ return_dict=True,
112
+ return_tensors="pt",
113
+ )
114
+
115
+ device = model.get_input_embeddings().weight.device
116
+ inputs = inputs.to(device)
117
+ prompt_length = inputs["input_ids"].shape[-1]
118
+
119
+ with torch.inference_mode():
120
+ output = model.generate(
121
+ **inputs,
122
+ max_new_tokens=512,
123
+ do_sample=False,
124
+ )
125
+
126
+ print(processor.tokenizer.decode(
127
+ output[0, prompt_length:], skip_special_tokens=True,
128
+ ).strip())
129
+ ```
130
+
131
+ For image and video inputs, the processor accepts multimodal message content in the format documented by the [Qwen3.8-27B quickstart](https://huggingface.co/Qwen/Qwen3.8-27B). For full-file code generation, increase `max_new_tokens` to fit the expected output.
132
+
133
+ ## Fine-tuning and evaluation
134
+
135
+ Lamapi developed Next 3 Pro through LoRA fine-tuning and exported a merged checkpoint. The training workflow included English and Turkish instructions, model identity examples, programmatically checked coding and reasoning tasks, and a benchmark-focused continuation.
136
+
137
+ The benchmark continuation used training examples from [GSM8K](https://huggingface.co/datasets/openai/gsm8k) and [MBPP Sanitized](https://huggingface.co/datasets/google-research-datasets/mbpp), along with synthetic repository-editing tasks that pair an issue description with a unified diff.
138
+
139
+ One observed failure is useful when designing tests. The model can write code that computes the correct value while stating a different value in prose. Check both the program output and the final answer.
140
+
141
+ ## Local inference
142
+
143
+ The [GGUF repository](https://huggingface.co/thelamapi/Next-3-Pro-GGUF) includes a Q8_0 weight export. Use a recent llama.cpp build that recognizes the checkpoint鈥檚 tensor layout. Image and video input through a GGUF runtime depends on the matching vision components and the runtime鈥檚 supported preprocessing.
144
+
145
+ The [TurboQuant compatibility repository](https://huggingface.co/thelamapi/Next-3-Pro-TurboQuant) contains a Q8_0 compatibility build with MTP disabled. TurboQuant cache settings are selected at inference time.
146
+
147
+ | A100 80 GB 路 8,192-token context capacity | Median decode speed | Reported GPU context memory |
148
  |---|---:|---:|
149
+ | Q8 KV cache | 42.43 tok/s | 421 MiB |
150
+ | TurboQuant `ktq4_1/vtq4_1` | 42.24 tok/s | 565 MiB |
151
+
152
+ These figures are medians from three runs per setting on the same compatibility build. Each run used the same short prompt and a 128-token output budget. In this setup, TurboQuant increased reported context memory and produced nearly the same decode speed.
153
+
154
+ ## Use and limitations
155
+
156
+ - Test generated code against your acceptance criteria. The model can omit edge cases or introduce regressions.
157
+ - Check numerical answers, citations, and factual claims against the underlying evidence.
158
+ - Vision, video, tool use, and extended context behavior depend on the serving stack and input configuration. Lamapi has not published separate evaluations for those capabilities on this checkpoint.
159
+ - Review the [upstream model terms](https://huggingface.co/Qwen/Qwen3.8-27B) and applicable dataset terms when adapting or redistributing the checkpoint.
160
 
161
+ [Explore Lamapi models on Hugging Face](https://huggingface.co/thelamapi)