Sebestianmek commited on
Commit
e04ed8b
·
verified ·
1 Parent(s): fae6b42

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +22 -49
README.md CHANGED
@@ -1,58 +1,31 @@
1
  ---
2
- library_name: transformers
3
  license: apache-2.0
4
- base_model: Sebestianmek/Seb-AI-S1-Mini-350M
 
5
  tags:
6
- - generated_from_trainer
7
- model-index:
8
- - name: Seb-AI-S1-Mini-Instruct
9
- results: []
 
 
10
  ---
11
 
12
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
13
- should probably proofread and complete it, then remove this comment. -->
14
 
15
- # Seb-AI-S1-Mini-Instruct
16
 
17
- This model is a fine-tuned version of [Sebestianmek/Seb-AI-S1-Mini-350M](https://huggingface.co/Sebestianmek/Seb-AI-S1-Mini-350M) on the None dataset.
 
18
 
19
- ## Model description
 
 
 
 
20
 
21
- More information needed
22
-
23
- ## Intended uses & limitations
24
-
25
- More information needed
26
-
27
- ## Training and evaluation data
28
-
29
- More information needed
30
-
31
- ## Training procedure
32
-
33
- ### Training hyperparameters
34
-
35
- The following hyperparameters were used during training:
36
- - learning_rate: 0.00015
37
- - train_batch_size: 4
38
- - eval_batch_size: 16
39
- - seed: 42
40
- - gradient_accumulation_steps: 32
41
- - total_train_batch_size: 128
42
- - optimizer: Use OptimizerNames.ADAFACTOR and the args are:
43
- No additional optimizer arguments
44
- - lr_scheduler_type: linear
45
- - lr_scheduler_warmup_steps: 40
46
- - training_steps: 600
47
- - mixed_precision_training: Native AMP
48
-
49
- ### Training results
50
-
51
-
52
-
53
- ### Framework versions
54
-
55
- - Transformers 5.20.2
56
- - Pytorch 2.11.0+cu130
57
- - Datasets 4.8.5
58
- - Tokenizers 0.23.3
 
1
  ---
 
2
  license: apache-2.0
3
+ language:
4
+ - en
5
  tags:
6
+ - seb-ai
7
+ - text-generation
8
+ - causal-lm
9
+ - llama
10
+ - trained-from-scratch
11
+ pipeline_tag: text-generation
12
  ---
13
 
14
+ # Seb AI S1 Mini Instruct (409M)
 
15
 
16
+ **Seb AI S1 Mini Instruct** is an open-source 409M parameter instruction-tuned language model trained completely from scratch by **Seb AI Inc.**
17
 
18
+ - **Creator:** Seb AI Inc.
19
+ - **Licence:** Apache 2.0 (Open-source for anyone to download, use, and fine-tune)
20
 
21
+ ## Model Details
22
+ - **Parameters:** 409 Million
23
+ - **Architecture:** Llama-style Causal Language Model (`LlamaForCausalLM`)
24
+ - **Context Length:** 512 tokens
25
+ - **Training:** Pre-trained from scratch and full-weight instruction-tuned across multi-turn conversational datasets, casual dialogue, and custom identity data by Seb AI Inc.
26
 
27
+ ## Prompt Format
28
+ ```text
29
+ User: Who made you?
30
+ Assistant:
31
+ ```