Instructions to use igorktech/nanofly-decoder-ru with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use igorktech/nanofly-decoder-ru with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="igorktech/nanofly-decoder-ru", trust_remote_code=True)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("igorktech/nanofly-decoder-ru", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use igorktech/nanofly-decoder-ru with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "igorktech/nanofly-decoder-ru"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "igorktech/nanofly-decoder-ru",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/igorktech/nanofly-decoder-ru

SGLang

How to use igorktech/nanofly-decoder-ru with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "igorktech/nanofly-decoder-ru" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "igorktech/nanofly-decoder-ru",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "igorktech/nanofly-decoder-ru" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "igorktech/nanofly-decoder-ru",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use igorktech/nanofly-decoder-ru with Docker Model Runner:
```
docker model run hf.co/igorktech/nanofly-decoder-ru
```

nanofly-decoder-ru

File size: 1,091 Bytes

36b95ca

{
  "arch": "decoder",
  "architectures": [
    "FlyForCausalLM"
  ],
  "auto_map": {
    "AutoConfig": "configuration_fly.FlyConfig",
    "AutoModel": "modeling_fly.FlyModel",
    "AutoModelForCausalLM": "modeling_fly.FlyForCausalLM"
  },
  "bos_token_id": 1,
  "connectome": "MaleCNS v1.0, FlyEM / HHMI Janelia, University of Cambridge, MRC LMB, Google Research. CC BY 4.0.",
  "d_emb": 256,
  "delay": 8,
  "dtype": "float32",
  "eos_token_id": 2,
  "head_type": "lowrank",
  "min_syn": 1,
  "mode": "gains",
  "model_type": "fly",
  "modulatory_sign": 0.0,
  "n_edges": 9055280,
  "n_neurons": 49393,
  "n_news_input": 0,
  "n_reserved": 2635,
  "n_token_input": 11434,
  "news_dim": 0,
  "news_encoder": "none",
  "news_glom": 0,
  "news_group": "orn",
  "news_mode": "glomeruli",
  "news_prefix": "",
  "pad_token_id": 0,
  "readout": "all",
  "readout_rank": 256,
  "readout_size": 49393,
  "source_repo": "",
  "ticks": 2,
  "tie_word_embeddings": false,
  "token_input": "cb_sensory,visual_projection",
  "transformers_version": "5.17.0",
  "use_cache": true,
  "vocab_size": 4096
}