Update model card
Browse files
README.md
CHANGED
|
@@ -1,9 +1,71 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
tags:
|
| 4 |
- petitgpt
|
| 5 |
- native-pytorch
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 6 |
- research
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
---
|
| 8 |
|
| 9 |
# PetitGPT
|
|
@@ -115,3 +177,16 @@ The native export preserves all 213 named state entries and 60 non-persistent ro
|
|
| 115 |
Author-controlled code, the model weights, and the tokenizer are released under [Apache-2.0](LICENSE). Author-written documentation is licensed separately under [CC BY 4.0](DOCUMENTATION_LICENSE.md). Third-party content retains its own terms and notices.
|
| 116 |
|
| 117 |
The [source notice](SOURCE_NOTICE.md) records the dataset and collection identities, tokenizer linkage, differences between component notices, and publisher observations that reflect only current pages. See the [third-party notices](THIRD_PARTY_NOTICES.md), the [run guide](RUN_GUIDE.md), and the [commit-pinned technical report](https://github.com/yangqi0/petitgpt/blob/157ef969de38167dbf0dce04e9095344a4a4b177/docs/petitgpt-v1/TECHNICAL_REPORT.md) for the detailed research record.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
pipeline_tag: text-generation
|
| 6 |
+
datasets:
|
| 7 |
+
- HuggingFaceTB/smollm-corpus
|
| 8 |
+
- HuggingFaceTB/dclm-edu
|
| 9 |
+
- HuggingFaceFW/finewiki
|
| 10 |
+
- HuggingFaceFW/finephrase
|
| 11 |
+
- allenai/dolmino-mix-1124
|
| 12 |
+
- common-pile/stackv2_edu_filtered
|
| 13 |
+
- HuggingFaceTB/smol-smoltalk
|
| 14 |
tags:
|
| 15 |
- petitgpt
|
| 16 |
- native-pytorch
|
| 17 |
+
- from-scratch
|
| 18 |
+
- small-language-model
|
| 19 |
+
- pretraining
|
| 20 |
+
- sft
|
| 21 |
+
- model-interpolation
|
| 22 |
+
- gqa
|
| 23 |
+
- single-gpu
|
| 24 |
- research
|
| 25 |
+
model-index:
|
| 26 |
+
- name: petitgpt-alpha075
|
| 27 |
+
results:
|
| 28 |
+
- task:
|
| 29 |
+
type: text-generation
|
| 30 |
+
name: Text Generation
|
| 31 |
+
dataset:
|
| 32 |
+
type: allenai/ai2_arc
|
| 33 |
+
name: ARC-Easy
|
| 34 |
+
config: ARC-Easy
|
| 35 |
+
split: test
|
| 36 |
+
revision: 210d026faf9955653af8916fad021475a3f00453
|
| 37 |
+
metrics:
|
| 38 |
+
- type: acc
|
| 39 |
+
value: 57.74
|
| 40 |
+
name: accuracy (zero-shot likelihood, FP32)
|
| 41 |
+
verified: false
|
| 42 |
+
- type: acc_norm
|
| 43 |
+
value: 52.36
|
| 44 |
+
name: normalized accuracy (zero-shot likelihood, FP32)
|
| 45 |
+
verified: false
|
| 46 |
+
source:
|
| 47 |
+
name: PetitGPT technical report (native protocol-compatible evaluator; not an installed lm-evaluation-harness run)
|
| 48 |
+
url: https://github.com/yangqi0/petitgpt/blob/157ef969de38167dbf0dce04e9095344a4a4b177/docs/petitgpt-v1/TECHNICAL_REPORT.md
|
| 49 |
+
- task:
|
| 50 |
+
type: text-generation
|
| 51 |
+
name: Text Generation
|
| 52 |
+
dataset:
|
| 53 |
+
type: baber/piqa
|
| 54 |
+
name: PIQA
|
| 55 |
+
split: validation
|
| 56 |
+
revision: 142f6d7367fd9877f0fb3b5734ea6a545f54cdd1
|
| 57 |
+
metrics:
|
| 58 |
+
- type: acc
|
| 59 |
+
value: 63.49
|
| 60 |
+
name: accuracy (zero-shot likelihood, FP32)
|
| 61 |
+
verified: false
|
| 62 |
+
- type: acc_norm
|
| 63 |
+
value: 62.30
|
| 64 |
+
name: normalized accuracy (zero-shot likelihood, FP32)
|
| 65 |
+
verified: false
|
| 66 |
+
source:
|
| 67 |
+
name: PetitGPT technical report (native protocol-compatible evaluator; not an installed lm-evaluation-harness run)
|
| 68 |
+
url: https://github.com/yangqi0/petitgpt/blob/157ef969de38167dbf0dce04e9095344a4a4b177/docs/petitgpt-v1/TECHNICAL_REPORT.md
|
| 69 |
---
|
| 70 |
|
| 71 |
# PetitGPT
|
|
|
|
| 177 |
Author-controlled code, the model weights, and the tokenizer are released under [Apache-2.0](LICENSE). Author-written documentation is licensed separately under [CC BY 4.0](DOCUMENTATION_LICENSE.md). Third-party content retains its own terms and notices.
|
| 178 |
|
| 179 |
The [source notice](SOURCE_NOTICE.md) records the dataset and collection identities, tokenizer linkage, differences between component notices, and publisher observations that reflect only current pages. See the [third-party notices](THIRD_PARTY_NOTICES.md), the [run guide](RUN_GUIDE.md), and the [commit-pinned technical report](https://github.com/yangqi0/petitgpt/blob/157ef969de38167dbf0dce04e9095344a4a4b177/docs/petitgpt-v1/TECHNICAL_REPORT.md) for the detailed research record.
|
| 180 |
+
|
| 181 |
+
## Citation
|
| 182 |
+
|
| 183 |
+
If you use this model or its documentation, please cite:
|
| 184 |
+
|
| 185 |
+
```bibtex
|
| 186 |
+
@misc{qi2026petitgpt,
|
| 187 |
+
title = {PetitGPT: A 124.6M-Parameter Language Model Trained from Scratch on a Single GPU},
|
| 188 |
+
author = {Qi, Yang},
|
| 189 |
+
year = {2026},
|
| 190 |
+
url = {https://huggingface.co/yqi0/petitgpt}
|
| 191 |
+
}
|
| 192 |
+
```
|