Text Generation
Transformers
Safetensors
Korean
English
nova_gdn2
nova
gdn2
gated-deltanet
hybrid
korean
custom_code
Instructions to use Insightnova/Nova-1B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Insightnova/Nova-1B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Insightnova/Nova-1B", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("Insightnova/Nova-1B", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Insightnova/Nova-1B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Insightnova/Nova-1B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Insightnova/Nova-1B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/Insightnova/Nova-1B
- SGLang
How to use Insightnova/Nova-1B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Insightnova/Nova-1B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Insightnova/Nova-1B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Insightnova/Nova-1B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Insightnova/Nova-1B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use Insightnova/Nova-1B with Docker Model Runner:
docker model run hf.co/Insightnova/Nova-1B
File size: 2,199 Bytes
84a4b93 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 | {
"architecture.py": {
"source": "model source: model.py",
"source_sha256": "ab2fded12d841dbd7c393cffb8e1c0542564bf69f1e0d0f057f5c074c6d7925a",
"release_sha256": "df27fa8146cebf2d9655acd8b0571f774d113f6911ea3f9ea5b4e3da70b0615d",
"release_filename": "architecture.py"
},
"gdn2.py": {
"source": "model source: gdn2.py",
"source_sha256": "4f94ccff363504ed236293a1691f3e489a78b41defc1896aae560dc48d031b57",
"release_sha256": "5408fa540c75e88d1930dfb6a711b342a662a1c84e0884267026d4de483ad4d9",
"release_filename": "gdn2.py"
},
"fast_decode.py": {
"source": "model source: fast_decode.py",
"source_sha256": "955c6386a7e20a3b6b376c0634659d87b7c79ac48c48261baf22b36c4ad1507e",
"release_sha256": "07d8fda5b92bdd96180a265b6ca66fcd19ab2e110576ce9e34fa2a98d19ad5d8",
"release_filename": "fast_decode.py"
},
"sequence_utils.py": {
"source": "model source: gdn_varlen.py",
"source_sha256": "5e678923a253de1491f47eb73cc6c8a428e3789f10834901541007aaaeeeaa32",
"release_sha256": "8214a6627b12a96d42e48427faeb4f17b5eafda0528847f643907247f442ffbe",
"release_filename": "sequence_utils.py"
},
"gdn2_spec.py": {
"source": "model source: spec.py",
"source_sha256": "c61ddeaaa18821d3b706be56f042a077f4154f4e4d4657e9e6002babcc7e15ab",
"release_sha256": "276ff07404fec37ae3e36e96871c09b30959bf12850d6a26ce363de3c4fbe20f",
"release_filename": "gdn2_spec.py"
},
"architecture_config.py": {
"source": "model source: config.py",
"source_sha256": "c3759dfd46cfc6296699b8567d13d035c1579dbc19d972a24d9db2fcbe542194",
"adaptation": "Only model shape and architecture options retained; training settings removed",
"release_sha256": "bc2f96f5e88dbbffe83f571e093764d5e7ca3312f5623042111aa34f44eb9424",
"release_filename": "architecture_config.py"
},
"cpu_backend.py": {
"reference": "FLA 0.5.2 GDN2 layer/recurrence and OLMo-core architecture semantics",
"adaptation": "Inference-only ordinary PyTorch CPU/CUDA implementation, original checkpoint keys retained",
"release_filename": "cpu_backend.py",
"release_sha256": "4330dc7349d6008d943e9ae5f463258bf97c32b74fc054772caddc84f633d253"
}
}
|