Text Generation
Transformers
Safetensors
Chinese
English
lfm3
tool-calling
function-calling
agent
chinese
conversational
custom_code
Instructions to use whcl412/FinchAI-Rune2-f16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use whcl412/FinchAI-Rune2-f16 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="whcl412/FinchAI-Rune2-f16", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("whcl412/FinchAI-Rune2-f16", trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained("whcl412/FinchAI-Rune2-f16", trust_remote_code=True, device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use whcl412/FinchAI-Rune2-f16 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "whcl412/FinchAI-Rune2-f16" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "whcl412/FinchAI-Rune2-f16", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/whcl412/FinchAI-Rune2-f16
- SGLang
How to use whcl412/FinchAI-Rune2-f16 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "whcl412/FinchAI-Rune2-f16" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "whcl412/FinchAI-Rune2-f16", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "whcl412/FinchAI-Rune2-f16" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "whcl412/FinchAI-Rune2-f16", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use whcl412/FinchAI-Rune2-f16 with Docker Model Runner:
docker model run hf.co/whcl412/FinchAI-Rune2-f16
补「多轮必须做防护」等 8 条已知问题
Browse files
README.md
CHANGED
|
@@ -108,19 +108,62 @@ print(tok.decode(out[0], skip_special_tokens=True))
|
|
| 108 |
|
| 109 |
思考内容在 `<think>…</think>` 里,和正文一起出现在同一条 assistant 输出中。
|
| 110 |
|
| 111 |
-
## 四、已知问题
|
| 112 |
-
|
| 113 |
-
1.
|
| 114 |
-
|
| 115 |
-
|
| 116 |
-
|
| 117 |
-
3
|
| 118 |
-
|
| 119 |
-
|
| 120 |
-
|
| 121 |
-
|
| 122 |
-
|
| 123 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 124 |
|
| 125 |
## 五、怎么训的
|
| 126 |
|
|
|
|
| 108 |
|
| 109 |
思考内容在 `<think>…</think>` 里,和正文一起出现在同一条 assistant 输出中。
|
| 110 |
|
| 111 |
+
## 四、已知问题(重要)
|
| 112 |
+
|
| 113 |
+
### 1. 多轮对话必须做防护 ⚠️
|
| 114 |
+
|
| 115 |
+
**这是目前最需要宿主处理的一点。**
|
| 116 |
+
|
| 117 |
+
实测:连续聊到第 3~4 轮,模型会**开始复读上一句**。比如问"说几个 emoji"、解释完之后
|
| 118 |
+
再问"哦""拜拜""?",它会一直回"😊 笑脸。"——上下文里重复的模式会自我强化。
|
| 119 |
+
|
| 120 |
+
**这是模型本身的问题,工程侧只能兜住,不能根治。** 宿主必须做两件事:
|
| 121 |
+
|
| 122 |
+
```python
|
| 123 |
+
# ① 历史窗口别开太长:只留最近 4 轮
|
| 124 |
+
if len(msgs) > 9:
|
| 125 |
+
msgs = [msgs[0]] + msgs[-8:]
|
| 126 |
+
|
| 127 |
+
# ② 复读检测:新回答和上一条完全一样 → 清空上下文重来
|
| 128 |
+
if ans and last_ans and ans.strip() == last_ans:
|
| 129 |
+
msgs = [msgs[0]] # 只留 system
|
| 130 |
+
last_ans = None
|
| 131 |
+
```
|
| 132 |
+
|
| 133 |
+
另外在 system prompt 里加一句「针对当前这句话回答,不要重复上一条回复的内容」,能明显缓解。
|
| 134 |
+
|
| 135 |
+
### 2. 多轮承接是短板
|
| 136 |
+
|
| 137 |
+
上文已经给出具体信息("10:02、10:05、10:08 三次超时"),追问"那什么时候开始的"时,
|
| 138 |
+
它仍可能回答"我没有查日志的工具,给我路径"——倾向于找工具,而不是读上文。
|
| 139 |
+
|
| 140 |
+
### 3. 反问会问偏一层
|
| 141 |
+
|
| 142 |
+
问"把那个文件改名",可能反问"哪个目录?"而不是"哪个文件?"。
|
| 143 |
+
|
| 144 |
+
### 4. 工具顺序敏感
|
| 145 |
+
|
| 146 |
+
同一组工具只换顺序、问题一字不改,选择会变。**把最可能用到的工具排在前面**能明显改善。
|
| 147 |
+
|
| 148 |
+
### 5. 结果不足时会编数字
|
| 149 |
+
|
| 150 |
+
让工具返回**直接可得答案**。别指望它从列表里数个数——给 `count_files(pattern, path)`
|
| 151 |
+
这类专用工具最稳。
|
| 152 |
+
|
| 153 |
+
### 6. 长上下文未验证
|
| 154 |
+
|
| 155 |
+
窗口是 256K,但训练样本最长 850 token。**超过约 1K 之后的表现没有任何保证**。
|
| 156 |
+
|
| 157 |
+
### 7. 安全边界是"拒绝 + 给替代"
|
| 158 |
+
|
| 159 |
+
让它删 `/Users` 全部文件,它会拒绝,但可能给出 `find /Users -name "*.tmp" -delete`
|
| 160 |
+
这类限定类型的替代命令——按设计这是允许的,如果你要绝对禁止任何删除命令,
|
| 161 |
+
在宿主侧再拦一层。
|
| 162 |
+
|
| 163 |
+
### 8. 极短输入会答偏
|
| 164 |
+
|
| 165 |
+
用户只发"哦""?""嗯"时,它可能给出过长或略偏的回答(但不会复读了)。
|
| 166 |
+
属于不知道该怎么接,不是错误。
|
| 167 |
|
| 168 |
## 五、怎么训的
|
| 169 |
|