Instructions to use fingerclose/Shelly with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use fingerclose/Shelly with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="fingerclose/Shelly")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("fingerclose/Shelly")
model = AutoModelForCausalLM.from_pretrained("fingerclose/Shelly", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use fingerclose/Shelly with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "fingerclose/Shelly"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fingerclose/Shelly",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/fingerclose/Shelly

SGLang

How to use fingerclose/Shelly with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "fingerclose/Shelly" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fingerclose/Shelly",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "fingerclose/Shelly" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fingerclose/Shelly",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use fingerclose/Shelly with Docker Model Runner:
```
docker model run hf.co/fingerclose/Shelly
```

Shelly

File size: 1,285 Bytes

---
license: apache-2.0
language:
- hi
- en
---
name: Shelly AI

description: >
  Shelly is an advanced AI companion designed by Ankush. 
  She blends technology and creativity to assist users 
  in various tasks, providing engaging and natural conversations.
  Perfect for developers and creators, Shelly aims to inspire and innovate 
  while maintaining a friendly and approachable persona.

model: 
  name: Shelly 3.1
  version: "1B"
  capabilities:
    - Natural language understanding
    - Contextual conversation
    - Task assistance
    - Creative collaboration
  usage:
    - Personal projects
    - Development assistance
    - Content creation
    - Learning support

author: Ankush Saini
author_instagram: "@ankush_saini_100"
repository: 
  name: fingerclose/Shelly
  type: private

metrics:
  - name: accuracy
    value: 0.85  # Update with your accuracy value
    description: "The model's accuracy in understanding and responding to queries."
  - name: response_time
    value: "50ms"  # Update with your average response time
    description: "Average time taken for the model to generate a response."
  - name: user_satisfaction
    value: 4.9  # Update with your average user satisfaction rating
    description: "Average user satisfaction rating based on feedback."