Instructions to use SaikatM/Code-Gemma-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use SaikatM/Code-Gemma-v1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="SaikatM/Code-Gemma-v1")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("SaikatM/Code-Gemma-v1")
model = AutoModelForCausalLM.from_pretrained("SaikatM/Code-Gemma-v1", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use SaikatM/Code-Gemma-v1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "SaikatM/Code-Gemma-v1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SaikatM/Code-Gemma-v1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/SaikatM/Code-Gemma-v1

SGLang

How to use SaikatM/Code-Gemma-v1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "SaikatM/Code-Gemma-v1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SaikatM/Code-Gemma-v1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "SaikatM/Code-Gemma-v1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SaikatM/Code-Gemma-v1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use SaikatM/Code-Gemma-v1 with Docker Model Runner:
```
docker model run hf.co/SaikatM/Code-Gemma-v1
```

Model Card for Model ID

This model is a fine-tuned version of google/gemma-2b on an SaikatM/Code-Platypus dataset.

Model Details

Model Description

Finetuned from model: google/gemma-2b

Model Sources

Training code can be found here: https://github.com/Saikat-M/LLM-Finetuning

Direct Use

Code generation tasks

Training Data

Dataset: https://huggingface.co/datasets/SaikatM/Code-Platypus
Source Dataset: https://huggingface.co/datasets/garage-bAInd/Open-Platypus

Training Procedure

Used QLoRA from PEFT and used SFTTrainer.

Preprocessing

From the Open-Platypus dataset filtering-out rows which has leetcode_ne in it's data_source column.

Training Hyperparameters

LoraConfig(

r=4,
lora_alpha=2,
target_modules=modules,
lora_dropout=0.05,
bias="none",
task_type="CAUSAL_LM"

)

TrainingArguments(

output_dir="gemma-2b-code-platypus",      
num_train_epochs=1,                    
per_device_train_batch_size=4,            
gradient_accumulation_steps=4,            
gradient_checkpointing=True,              
optim="paged_adamw_8bit",                 
logging_steps=1,                          
save_strategy="epoch",                    
bf16=False,
tf32=False,
learning_rate=2e-4,                       
max_steps= 100,
max_grad_norm=0.3,                        
warmup_ratio=0.03,                        
lr_scheduler_type="constant",             
push_to_hub=False,                        
report_to="tensorboard",

)

SFTTrainer(

model=model,
train_dataset=train_data,
eval_dataset=test_data,
dataset_text_field="text",
peft_config=lora_config,
max_seq_length=512,
tokenizer=tokenizer,
args=training_arguments,

)

Speeds, Sizes, Times

Took around 1 hour to train.

Results

Test Result 1:

Write a fucntion to sort a list in python

Answer:

def sort_list(list):
    return sorted(list)<eos>
Response:  None

Test Result 2:

Write a function to count Consonants in a Given Word in Python

Response:  None

Test Result 3:

Write a function to count the number of vowels in a given string in Python.

Example 1:

Input:  s =  "leetcodeisgreat"
Output:  5
Explanation:  The vowels are  'e', 'i', 'a', 'o', and 'u'.

Example 2:

Input:  s =  "leetcodeisgreat"
Output:  0
Explanation:  The vowels are  'e', 'i', 'a', 'o', and 'u'.

Constraints:

*   1 <= s.length <= 100
*   s consists of lowercase English letters.


def countVowels(s):
    count = 0
    for c in s:
        if c in 'aeiou':
            count += 1
    return count
<eos>
Response:  None

Compute Infrastructure

Trained in Google Colab

Hardware

T4 GPU Hardware accelerator.

Downloads last month: 33

Safetensors

Model size

3B params

Tensor type

F16

SaikatM
/

Code-Gemma-v1