Instructions to use DRUPEs/CherryPit-1.2-2B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use DRUPEs/CherryPit-1.2-2B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="DRUPEs/CherryPit-1.2-2B") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("DRUPEs/CherryPit-1.2-2B") model = AutoModelForCausalLM.from_pretrained("DRUPEs/CherryPit-1.2-2B", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use DRUPEs/CherryPit-1.2-2B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "DRUPEs/CherryPit-1.2-2B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "DRUPEs/CherryPit-1.2-2B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/DRUPEs/CherryPit-1.2-2B
- SGLang
How to use DRUPEs/CherryPit-1.2-2B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "DRUPEs/CherryPit-1.2-2B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "DRUPEs/CherryPit-1.2-2B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "DRUPEs/CherryPit-1.2-2B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "DRUPEs/CherryPit-1.2-2B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use DRUPEs/CherryPit-1.2-2B with Docker Model Runner:
docker model run hf.co/DRUPEs/CherryPit-1.2-2B
CherryPit-1.2-2B in the CMF format (cortiq: Vulkan, Metal, CPU) + a Turbo file for long prompts
Hi DRUPEs team, thank you for CherryPit-1.2!
CherryPit-1.2-2B is now available in the CMF format for cortiq, a Rust inference engine with no Python and no ML framework: NVIDIA, AMD and Intel GPUs through Vulkan/DX12, Apple silicon through Metal, and the CPU.
- infosave/CherryPit-1.2-2B-cmf:
q8_2f(2.54 GB, WikiText-2 ppl 18.46 against 18.45 for bf16) andq4tp(1.77 GB, GPTQ calibrated with chats in your template, ppl 19.40). - infosave/CherryPit-1.2-2B-Turbo-cmf: long prompts at a fixed memory ceiling. At 64 000 tokens on an RTX 5090 it takes 2.9 GB of VRAM and decodes at 155 tok/s, where the standard file takes 7.4 GB and runs at 54 tok/s; facts planted in 100 000 characters are recalled 8 of 8.
Your chat template, the thinking switch and the XML tool calls work as in transformers (the prompt tokens match on seven test conversations, and the server returns tool calls as message.tool_calls). Decode speed: about 190 tok/s on an RTX 5090, 43 tok/s (q4tp) on a MacBook Air M4.
cargo install cortiq-cli
hf download infosave/CherryPit-1.2-2B-cmf CherryPit-1.2-2B-q8_2f.cmf --local-dir .
cortiq run CherryPit-1.2-2B-q8_2f.cmf --prompt "Build a React component with a search box and a list that loads on open." --no-think --max-tokens 1024
The files run on cortiq 0.8.61 and later; the speeds above come with the next release.