Instructions to use Corianas/Quokka_1.3b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Corianas/Quokka_1.3b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Corianas/Quokka_1.3b")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Corianas/Quokka_1.3b")
model = AutoModelForCausalLM.from_pretrained("Corianas/Quokka_1.3b", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Corianas/Quokka_1.3b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Corianas/Quokka_1.3b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Corianas/Quokka_1.3b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Corianas/Quokka_1.3b

SGLang

How to use Corianas/Quokka_1.3b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Corianas/Quokka_1.3b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Corianas/Quokka_1.3b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Corianas/Quokka_1.3b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Corianas/Quokka_1.3b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Corianas/Quokka_1.3b with Docker Model Runner:
```
docker model run hf.co/Corianas/Quokka_1.3b
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Model Card for Cerebras 1.3b Dollyfied.

This is a finetuned model of Cerebras 1.3b model. using DataBricksLabs Dolly Framework

Model Details

Model Description

This is a finetuned version of cerebras' 1.3Billion paramater model that has been trained to follow instructions.

It was accomplished using DataBricks Dolly training tools, and was trained for 2 epochs.

Developed by: Finetuned by Corianas (me) using open source tools
Shared by [optional]: [More Information Needed]
Model type: [More Information Needed]
Language(s) (NLP): EN
License: cc-by-nc-4.0
Finetuned from model: https://huggingface.co/cerebras/Cerebras-GPT-111m
Finetuned using: https://www.databricks.com/blog/2023/03/24/hello-dolly-democratizing-magic-chatgpt-open-models.html

Uses

This is a simple GPT chatbot that has been finetuned to understand instructions. Its knowledge about facts about the world is should be considered suspect at best.

Direct Use

If you have a use you put it to, Please let me know.

[More Information Needed]

Downstream Use [optional]

[More Information Needed]

Out-of-Scope Use

Any form of use where any form of accuracy is needed. FOR THE LOVE OF GOD DO NOT FOLLOW MEDICAL ADVICE FROM THIS. or financial advice.

[More Information Needed]

Bias, Risks, and Limitations

Limitations... Yes, I am sure there are so so many.

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Hardware Type: 8xA100s (accomplished while I was downloading the model I was actually training.)
Minutes used: 17
Cloud Provider: LambdaGPU
Compute Region: USA
Carbon Emitted: [More Information Needed]

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	27.1
ARC (25-shot)	27.73
HellaSwag (10-shot)	37.91
MMLU (5-shot)	26.66
TruthfulQA (0-shot)	40.14
Winogrande (5-shot)	52.72
GSM8K (5-shot)	0.0
DROP (3-shot)	4.54

Downloads last month: 102

Safetensors

Model size

1B params

Tensor type

BF16

Model tree for Corianas/Quokka_1.3b

Quantizations

2 models

Datasets used to train Corianas/Quokka_1.3b

Spaces using Corianas/Quokka_1.3b 31

Paper for Corianas/Quokka_1.3b

Quantifying the Carbon Emissions of Machine Learning

Paper • 1910.09700 • Published Oct 21, 2019 • 61