Instructions to use Stoned-Code/Opusculum-1.5B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Local Apps Settings
- Unsloth Desktop
|
Download README.md from Stoned-Code/Opusculum-1.5B: direct link, hf CLI and curl.
- Browser
- Download file 1.74 kB
-
https://huggingface.co/Stoned-Code/Opusculum-1.5B/resolve/main/README.md
- Command line
-
hf download hf://Stoned-Code/Opusculum-1.5B/README.md
-
curl -L -o README.md https://huggingface.co/Stoned-Code/Opusculum-1.5B/resolve/main/README.md
1.74 kB
metadata
datasets:
- Stoned-Code/magnum-v4
- Delta-Vector/Hydrus-Claude-Instruct-5K
- anthracite-org/kalo-opus-instruct-22k-no-refusal
- Epiculous/SynthRP-Gens-v1.1-Filtered-n-Cleaned
- lodrick-the-lafted/kalo-opus-instruct-3k-filtered
- anthracite-org/nopm_claude_writing_fixed
- Epiculous/Synthstruct-Gens-v1.1-Filtered-n-Cleaned
- anthracite-org/kalo_opus_misc_240827
- anthracite-org/kalo_misc_part2
base_model:
- unsloth/Qwen2.5-1.5B-bnb-4bit
pipeline_tag: text-generation
license: apache-2.0
language:
- en
tags:
- unsloth
A model finetuned from unsloth/Qwen2.5-1.5B-bnb-4bit using the datasets used to train anthracite-org/magnum-v4-9b.
The model is a full finetune using the following configuration on a single 5060ti with 16GB of VRAM.
training:
max_seq_length: 8192
num_epochs: 3
learning_rate: 0.00002
embedding_learning_rate: null
batch_size: 2
gradient_accumulation_steps: 4
warmup_steps: 3
max_steps: 0
save_steps: 100
eval_steps: 100
early_stopping_patience: 0
weight_decay: 0.001
random_seed: 3407
packing: true
train_on_completions: true
gradient_checkpointing: unsloth
offload_layers: 0
offload_vram_gb: null
offload_vram_gb_per_device: {}
prefetch_depth: 2
optim: adamw_bnb_8bit
lr_scheduler_type: cosine
lora:
lora_r: 16
lora_alpha: 16
lora_dropout: 0
target_modules:
- q_proj
- k_proj
- v_proj
- o_proj
- gate_proj
- up_proj
- down_proj
use_rslora: false
use_loftq: false
use_dora: false
logging:
enable_wandb: false
wandb_project: ''
enable_tensorboard: false
tensorboard_dir: ''
log_frequency: 10