Spaces:
Running on Zero
Running on Zero
Add Apache-2.0 license tag and data/model attribution note
Browse files
README.md
CHANGED
|
@@ -7,6 +7,7 @@ sdk: gradio
|
|
| 7 |
sdk_version: 5.49.1
|
| 8 |
app_file: app.py
|
| 9 |
pinned: false
|
|
|
|
| 10 |
---
|
| 11 |
|
| 12 |
# Predicting Memorization Before Fine-Tuning — demo
|
|
@@ -57,3 +58,18 @@ Locally the app loads Pythia-1.4B (~3 GB) at startup on a GPU if present, otherw
|
|
| 57 |
- The classifier uses only base-model (pre-fine-tuning) features and was trained on a separate
|
| 58 |
Run-1 fine-tuning run, then evaluated here on a disjoint Run-2 run.
|
| 59 |
- Qwen and Llama can be added to the Explore tab once their run-2 token sequences are regenerated.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
sdk_version: 5.49.1
|
| 8 |
app_file: app.py
|
| 9 |
pinned: false
|
| 10 |
+
license: apache-2.0
|
| 11 |
---
|
| 12 |
|
| 13 |
# Predicting Memorization Before Fine-Tuning — demo
|
|
|
|
| 58 |
- The classifier uses only base-model (pre-fine-tuning) features and was trained on a separate
|
| 59 |
Run-1 fine-tuning run, then evaluated here on a disjoint Run-2 run.
|
| 60 |
- Qwen and Llama can be added to the Explore tab once their run-2 token sequences are regenerated.
|
| 61 |
+
|
| 62 |
+
## Data and model attribution
|
| 63 |
+
The demo code in this repository is released under Apache-2.0. The text excerpts shown in the
|
| 64 |
+
Explore tab are short passages drawn from public datasets and are displayed only to illustrate
|
| 65 |
+
research findings; each dataset remains under its own upstream license, held by its original
|
| 66 |
+
authors.
|
| 67 |
+
|
| 68 |
+
- FineWeb, [HuggingFaceFW/fineweb](https://huggingface.co/datasets/HuggingFaceFW/fineweb), under ODC-By 1.0.
|
| 69 |
+
- PG-19, [deepmind/pg19](https://huggingface.co/datasets/deepmind/pg19), public-domain books from Project Gutenberg.
|
| 70 |
+
- The Stack, [bigcode/the-stack](https://huggingface.co/datasets/bigcode/the-stack), permissively licensed source code collected by the BigCode project.
|
| 71 |
+
- OpenWebMath, [open-web-math/open-web-math](https://huggingface.co/datasets/open-web-math/open-web-math), openly released mathematical web text.
|
| 72 |
+
- Base model Pythia-1.4B, [EleutherAI/pythia-1.4b](https://huggingface.co/EleutherAI/pythia-1.4b), under Apache-2.0.
|
| 73 |
+
|
| 74 |
+
We thank the authors and maintainers of these datasets and of Pythia. If you are a rights holder
|
| 75 |
+
and want an excerpt removed, please open a discussion on the Space and we will take it down.
|