Instructions to use fromziro/50M-pro with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use fromziro/50M-pro with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="fromziro/50M-pro", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("fromziro/50M-pro", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use fromziro/50M-pro with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "fromziro/50M-pro" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "fromziro/50M-pro", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/fromziro/50M-pro
- SGLang
How to use fromziro/50M-pro with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "fromziro/50M-pro" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "fromziro/50M-pro", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "fromziro/50M-pro" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "fromziro/50M-pro", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use fromziro/50M-pro with Docker Model Runner:
docker model run hf.co/fromziro/50M-pro
Commit History
9 days ago
-
Fix generate(): prompt was truncated to its last token
Verified signatureFLA prepare_inputs_for_generation sliced input_ids to the last token when transformers passed its own non-empty DynamicCache, so generate() ignored the prompt. Now only an FLA Cache is trusted and input_ids are sliced by tokens actually seen. Verified on GPU: generate logits match teacher-forced forward (worst cos 0.9996, was 0.68). Weights unchanged.0da1f2c -
Support fp32/fp16 loading (BF16 autocast) and padded batches (unpad via cu_seqlens)
Verified signatureFLA chunk_delta_rule rejects fp32, so non-BF16 weights now run the blocks under BF16 autocast (training numerics). attention_mask with padding is now handled by unpacking rows into cu_seqlens varlen sequences instead of raising. Weights unchanged.b127a78
10 days ago
-
Update README.md
Verified signature8a50338 -
step 80000 | val loss 2.9141
46c9706