Text Generation
Transformers
Safetensors
Korean
English
nova_gdn2
nova
gdn2
gated-deltanet
hybrid
korean
custom_code
Instructions to use Insightnova/Nova-1B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Insightnova/Nova-1B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Insightnova/Nova-1B", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("Insightnova/Nova-1B", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Insightnova/Nova-1B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Insightnova/Nova-1B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Insightnova/Nova-1B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/Insightnova/Nova-1B
- SGLang
How to use Insightnova/Nova-1B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Insightnova/Nova-1B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Insightnova/Nova-1B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Insightnova/Nova-1B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Insightnova/Nova-1B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use Insightnova/Nova-1B with Docker Model Runner:
docker model run hf.co/Insightnova/Nova-1B
Download gdn2_spec.py from Insightnova/Nova-1B: direct link, hf CLI and curl.
- Browser
- Download file 2.64 kB
-
https://huggingface.co/Insightnova/Nova-1B/resolve/main/gdn2_spec.py
- Command line
-
hf download hf://Insightnova/Nova-1B/gdn2_spec.py
-
curl -L -o gdn2_spec.py https://huggingface.co/Insightnova/Nova-1B/resolve/main/gdn2_spec.py
2.64 kB
| """GDN2 dimensions and exact FLA 0.5.2 parameter counts; no GPU imports.""" | |
| from dataclasses import dataclass | |
| import math | |
| class GDN2Spec: | |
| d_model: int | |
| n_heads: int | |
| head_dim: int = 128 | |
| expand_v: float = 2.0 | |
| n_v_heads: int | None = None | |
| conv_size: int = 4 | |
| conv_bias: bool = False | |
| def __post_init__(self) -> None: | |
| for name in ("d_model", "n_heads", "head_dim", "conv_size"): | |
| value = getattr(self, name) | |
| if not isinstance(value, int) or isinstance(value, bool) or value <= 0: | |
| raise ValueError(f"{name} must be a positive integer") | |
| if self.head_dim > 256: | |
| raise ValueError("FLA GDN2 requires head_dim <= 256") | |
| if not math.isfinite(self.expand_v) or self.expand_v <= 0: | |
| raise ValueError("expand_v must be finite and positive") | |
| expanded = self.head_dim * self.expand_v | |
| if not math.isclose(expanded, int(expanded), rel_tol=1e-5) or int(expanded) < 1: | |
| raise ValueError("head_dim * expand_v must be a positive integer") | |
| if self.n_v_heads is not None: | |
| if not isinstance(self.n_v_heads, int) or self.n_v_heads < self.n_heads: | |
| raise ValueError("n_v_heads must be an integer >= n_heads") | |
| if self.n_v_heads % self.n_heads: | |
| raise ValueError("n_v_heads must be divisible by n_heads") | |
| def value_heads(self) -> int: | |
| return self.n_heads if self.n_v_heads is None else self.n_v_heads | |
| def head_v_dim(self) -> int: | |
| return int(self.head_dim * self.expand_v) | |
| def key_dim(self) -> int: | |
| return self.n_heads * self.head_dim | |
| def value_dim(self) -> int: | |
| return self.value_heads * self.head_v_dim | |
| def num_params(self) -> int: | |
| d, k, v, r = self.d_model, self.key_dim, self.value_dim, self.head_v_dim | |
| return ( | |
| d * (2 * k + v) # q/k/v | |
| + r * (d + k) # low-rank decay f_proj | |
| + d * (k + v) # erase b_proj, write w_proj | |
| + self.n_heads + k # A_log, channel-wise dt_bias | |
| + r * (d + v) + v # low-rank output gate, including bias | |
| + r + v * d # output norm and o_proj | |
| + (self.conv_size + int(self.conv_bias)) * (2 * k + v) | |
| ) | |
| def state_elements(self) -> int: | |
| return self.value_heads * self.head_dim * self.head_v_dim | |