Instructions to use RecursiveSI/flint with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use RecursiveSI/flint with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="RecursiveSI/flint")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("RecursiveSI/flint")
model = AutoModelForCausalLM.from_pretrained("RecursiveSI/flint", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use RecursiveSI/flint with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "RecursiveSI/flint"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RecursiveSI/flint",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/RecursiveSI/flint

SGLang

How to use RecursiveSI/flint with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "RecursiveSI/flint" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RecursiveSI/flint",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "RecursiveSI/flint" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RecursiveSI/flint",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use RecursiveSI/flint with Docker Model Runner:
```
docker model run hf.co/RecursiveSI/flint
```

Tool-use planning loops with rec despite attached tool schemas

#12

by rachpradhan - opened about 13 hours ago

Discussion

rachpradhan

about 13 hours ago

With Flint served through rec's OpenAI-compatible endpoint, an agent can repeatedly reconsider how to invoke tools whose definitions are already supplied, inspecting unrelated workspace or harness files instead of making the tool call that advances the task.

This appears distinct from discussion #11's argument-formatting issue: the concern here is choosing and using an available tool, not just making its arguments conform to a schema.

Could you document:

Recommended system-prompt or tool-description conventions for reliable direct tool use.
Whether rec supports a per-request reasoning budget while retaining its tool-call grammar, or another supported way to bound these planning loops.

A small integration example would help distinguish prompting requirements from runtime configuration. A prompt adjustment is being investigated, but no fix or performance improvement has been established yet.

alphabell-research

about 3 hours ago

I checked rec's public MLX server at revision 8af99ab (October 22). It accepts per-request max_tokens or max_completion_tokens alongside grammar:"flint", so you can cap total generation while keeping the tool grammar. This counts all generated tokens, including reasoning; it can end with finish_reason:"length" before a tool call is complete.

A potentially confusing detail: the handler removes unsupported request fields and logs them. reasoning_effort, thinking_token_budget, reasoning_budget_tokens and chat_template_kwargs are not supported controls in this version. See the pinned request allowlist and grammar/output-limit handling.

For a direct API reproduction, explicitly set the total output limit. Omitting both limits permits generation through the remaining context. Custom stop strings are rejected with Flint grammar. I did not find a supported reasoning-only allowance that reserves tokens for the answer or tool call afterward. That would be distinct from the current total-output cap.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment