Text Generation
Transformers
Chinese
English
character-level
chinese
gru
mamba
Mixture of Experts
titans
Instructions to use masterofthestars/tianxuan with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use masterofthestars/tianxuan with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="masterofthestars/tianxuan")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("masterofthestars/tianxuan", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use masterofthestars/tianxuan with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "masterofthestars/tianxuan" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "masterofthestars/tianxuan", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/masterofthestars/tianxuan
- SGLang
How to use masterofthestars/tianxuan with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "masterofthestars/tianxuan" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "masterofthestars/tianxuan", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "masterofthestars/tianxuan" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "masterofthestars/tianxuan", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use masterofthestars/tianxuan with Docker Model Runner:
docker model run hf.co/masterofthestars/tianxuan
Tianxuan: full project + data (checkpoints, world_model, data without raw) with correct directory structure
db7bc06 Download scripts/prepare_training_data.ts from masterofthestars/tianxuan: direct link, hf CLI and curl.
- Browser
- Download file 1.56 kB
-
https://huggingface.co/masterofthestars/tianxuan/resolve/main/scripts/prepare_training_data.ts
- Command line
-
hf download hf://masterofthestars/tianxuan/scripts/prepare_training_data.ts
-
curl -L -o prepare_training_data.ts https://huggingface.co/masterofthestars/tianxuan/resolve/main/scripts/prepare_training_data.ts
1.56 kB
| /** | |
| * 准备训练数据:将清理后的语料放入 data/raw/ 供训练脚本使用 | |
| */ | |
| import * as fs from "fs"; | |
| import * as path from "path"; | |
| function main() { | |
| const rawDir = "data/raw"; | |
| // 确保 raw 目录存在 | |
| if (!fs.existsSync(rawDir)) { | |
| fs.mkdirSync(rawDir, { recursive: true }); | |
| } | |
| // 读取清理后的语料(从 large 版本) | |
| const largePath = "data/corpus_large.txt"; | |
| console.log(`读取: ${largePath}`); | |
| const raw = fs.readFileSync(largePath, "utf-8"); | |
| console.log(`原始字符数: ${raw.length}`); | |
| // 清理非CJK字符 | |
| const cleaned = raw.replace(/[^\u4E00-\u9FFF\u3000-\u303F\uFF00-\uFFEF\u2014\u2018-\u201D\u2026\n\r]/g, ""); | |
| console.log(`清理后字符数: ${cleaned.length}`); | |
| // 分成多个文件(每个约 30KB,符合训练脚本的 rawPerFile 限制) | |
| const chunkSize = 30_000; | |
| let fileIndex = 0; | |
| let offset = 0; | |
| while (offset < cleaned.length) { | |
| const chunk = cleaned.slice(offset, offset + chunkSize); | |
| const fileName = `event_${String(fileIndex).padStart(3, '0')}.txt`; | |
| const filePath = path.join(rawDir, fileName); | |
| fs.writeFileSync(filePath, chunk, "utf-8"); | |
| console.log(` 写入: ${filePath} (${chunk.length} 字符)`); | |
| fileIndex++; | |
| offset += chunkSize; | |
| } | |
| // 同时复制一个到 corpus.txt(供 --resume 使用) | |
| fs.writeFileSync("data/corpus.txt", cleaned, "utf-8"); | |
| console.log(`\n总文件数: ${fileIndex} 个 (在 ${rawDir}/)`); | |
| console.log(`已创建: data/corpus.txt (${cleaned.length} 字符)`); | |
| } | |
| main(); | |