Document Rec MLX HTTP output budgets and ignored reasoning options

#2
by alphabell-research - opened
Files changed (1) hide show
  1. README.md +20 -0
README.md CHANGED
@@ -27,6 +27,26 @@ rec launch pi
27
  rec launch opencode
28
  ```
29
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
30
  ## Download the weights separately
31
 
32
  With the [Hugging Face CLI](https://huggingface.co/docs/huggingface_hub/guides/cli) installed:
 
27
  rec launch opencode
28
  ```
29
 
30
+ ### HTTP output budgets in Rec MLX v0.1.0
31
+
32
+ For OpenAI-compatible HTTP requests, set `max_tokens` or `max_completion_tokens` to bound total generated output while retaining `grammar: "flint"`:
33
+
34
+ ```json
35
+ {
36
+ "model": "flint",
37
+ "messages": [{"role": "user", "content": "Hello"}],
38
+ "grammar": "flint",
39
+ "max_tokens": 2048,
40
+ "stream": false
41
+ }
42
+ ```
43
+
44
+ The cap includes reasoning and answer or tool-call tokens. Reaching it can return `finish_reason: "length"` before an answer or tool call is complete. Non-null `max_completion_tokens` takes precedence over `max_tokens`. Omitting both permits generation through the remaining context; the terminal chat token default is separate.
45
+
46
+ This version has no separate reasoning-only allowance. Unsupported fields such as `reasoning_effort`, `thinking_token_budget`, `reasoning_budget_tokens` and `chat_template_kwargs` are removed and logged. Nonempty custom stop strings cannot be combined with Flint grammar. Token limits must use integer JSON values.
47
+
48
+ See the pinned [request allowlist and validation](https://github.com/recursive-oss/rec/blob/8af99abaf50b33903147d89eed91d1bb3ac642dd/engine/src/rec_mlx/server.py#L134-L224) and [output-limit handling](https://github.com/recursive-oss/rec/blob/8af99abaf50b33903147d89eed91d1bb3ac642dd/engine/src/rec_mlx/server.py#L249-L262).
49
+
50
  ## Download the weights separately
51
 
52
  With the [Hugging Face CLI](https://huggingface.co/docs/huggingface_hub/guides/cli) installed: