Text Generation
Transformers
Safetensors
qwen3
reasoning
code
inference
chat
conversational
text-generation-inference
Instructions to use LucidityAI/Astral-4B-Preview with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use LucidityAI/Astral-4B-Preview with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="LucidityAI/Astral-4B-Preview") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("LucidityAI/Astral-4B-Preview") model = AutoModelForCausalLM.from_pretrained("LucidityAI/Astral-4B-Preview", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use LucidityAI/Astral-4B-Preview with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "LucidityAI/Astral-4B-Preview" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "LucidityAI/Astral-4B-Preview", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/LucidityAI/Astral-4B-Preview
- SGLang
How to use LucidityAI/Astral-4B-Preview with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "LucidityAI/Astral-4B-Preview" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "LucidityAI/Astral-4B-Preview", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "LucidityAI/Astral-4B-Preview" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "LucidityAI/Astral-4B-Preview", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use LucidityAI/Astral-4B-Preview with Docker Model Runner:
docker model run hf.co/LucidityAI/Astral-4B-Preview
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,46 +1,75 @@
|
|
|
|
|
|
|
|
| 1 |
---
|
|
|
|
|
|
|
| 2 |
license: apache-2.0
|
| 3 |
base_model: Qwen/Qwen3-4b-thinking-2507
|
| 4 |
tags:
|
| 5 |
-
-
|
| 6 |
-
-
|
| 7 |
-
-
|
| 8 |
-
-
|
|
|
|
| 9 |
library_name: transformers
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 10 |
---
|
| 11 |
|
| 12 |
-
#
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
|
| 14 |
-
|
| 15 |
|
| 16 |
-
##
|
| 17 |
|
| 18 |
-
|
| 19 |
-
- **Training**: LoRA fine-tuning on multi-domain reasoning tasks
|
| 20 |
-
- **Merged**: Yes, this is the merged model (no adapter needed)
|
| 21 |
|
| 22 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
-
|
| 25 |
-
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 26 |
|
| 27 |
-
|
| 28 |
-
"LucidityAI/Astral-400",
|
| 29 |
-
torch_dtype="auto",
|
| 30 |
-
device_map="auto"
|
| 31 |
-
)
|
| 32 |
-
tokenizer = AutoTokenizer.from_pretrained("LucidityAI/Astral-400")
|
| 33 |
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
```
|
| 40 |
|
| 41 |
-
|
| 42 |
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
-
|
| 46 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Certainly! Here's a more professional, polished, and clearly structured version of your model card—refined for tone, clarity, and technical precision while preserving all original information:
|
| 2 |
+
|
| 3 |
---
|
| 4 |
+
|
| 5 |
+
```yaml
|
| 6 |
license: apache-2.0
|
| 7 |
base_model: Qwen/Qwen3-4b-thinking-2507
|
| 8 |
tags:
|
| 9 |
+
- reasoning
|
| 10 |
+
- code
|
| 11 |
+
- inference
|
| 12 |
+
- chat
|
| 13 |
+
- multimodal (if applicable)
|
| 14 |
library_name: transformers
|
| 15 |
+
datasets:
|
| 16 |
+
- nvidia/AceReason-1.1-SFT
|
| 17 |
+
```
|
| 18 |
+
|
| 19 |
+
# Astral-4B-Preview
|
| 20 |
+
|
| 21 |
+
|
| 22 |
+
Astral-4B is a specialized reasoning-focused language model developed as part of the Astral series, designed to deliver high-fidelity, step-by-step reasoning with configurable depth. Built upon the Qwen3-4b-thinking-2507 foundation, this variant has been fine-tuned on the `nvidia/AceReason-1.1-SFT` dataset to enhance logical coherence, problem-solving capability, and structured thinking.
|
| 23 |
+
|
| 24 |
+
This model is currently in **preview** and intended for research, evaluation, and development use. Feedback is encouraged to guide future iterations.
|
| 25 |
+
|
| 26 |
---
|
| 27 |
|
| 28 |
+
## Key Features
|
| 29 |
+
|
| 30 |
+
- **Configurable Reasoning Depth**: Supports explicit control over reasoning granularity via system-level prompts.
|
| 31 |
+
- **ChatML Template Compliance**: Fully compatible with the ChatML format for seamless integration into existing inference pipelines.
|
| 32 |
+
- **Structured Output**: Generates detailed reasoning traces (`<think>` blocks) when enabled, enabling auditability and interpretability.
|
| 33 |
+
- **Optimized for Complex Tasks**: Strong performance on mathematical reasoning, code generation, and multi-step logic problems.
|
| 34 |
|
| 35 |
+
---
|
| 36 |
|
| 37 |
+
## Usage Instructions
|
| 38 |
|
| 39 |
+
To invoke the model correctly, include a **reasoning-level indicator** in the system prompt using the `Reasoning-level:` directive. The available levels are:
|
|
|
|
|
|
|
| 40 |
|
| 41 |
+
| Level | Behavior |
|
| 42 |
+
|----------|--------|
|
| 43 |
+
| `none` | No reasoning trace generated; direct response only. |
|
| 44 |
+
| `low` | Minimal internal reasoning; concise output. |
|
| 45 |
+
| `medium` | Balanced reasoning with intermediate steps. |
|
| 46 |
+
| `high` | Detailed step-by-step reasoning with full justification. |
|
| 47 |
+
| `ultra` | Maximal depth reasoning; exhaustive exploration of alternatives and edge cases. |
|
| 48 |
|
| 49 |
+
> **Note**: The absence of a valid reasoning level will result in undefined behavior. Always specify one.
|
|
|
|
| 50 |
|
| 51 |
+
### Example Prompt (ChatML Format):
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 52 |
|
| 53 |
+
```xml
|
| 54 |
+
<|im_start|>system
|
| 55 |
+
Reasoning-level: high
|
| 56 |
+
<|im_end|>
|
| 57 |
+
<|im_start|>user
|
| 58 |
+
What is the capital of France?
|
| 59 |
+
<|im_end|>
|
| 60 |
+
<|im_start|>assistant
|
| 61 |
+
<think>
|
| 62 |
+
To determine the capital of France, we recall that France is a country in Western Europe. Its major cities include Paris, Lyon, Marseille, and Toulouse. Among these, Paris is historically and politically recognized as the capital. It hosts the French government, national institutions, and the President’s residence. Therefore, the capital of France is Paris.
|
| 63 |
+
</think>
|
| 64 |
+
Paris
|
| 65 |
```
|
| 66 |
|
| 67 |
+
---
|
| 68 |
|
| 69 |
+
## Important Notes
|
| 70 |
+
|
| 71 |
+
- This is a **preview release**. Performance may vary across edge cases or non-standard inputs.
|
| 72 |
+
- For production applications, please wait for the official release and corresponding documentation.
|
| 73 |
+
- Model outputs should be validated for accuracy, especially in safety-critical domains.
|
| 74 |
+
|
| 75 |
+
---
|