Upload 3 files
Browse files- README.md +61 -3
- README_RU.md +61 -0
- model.safetensors +3 -0
README.md
CHANGED
|
@@ -1,3 +1,61 @@
|
|
| 1 |
-
|
| 2 |
-
|
| 3 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Claude Fable 6.9
|
| 2 |
+
|
| 3 |
+
A tiny 69-parameter neural network (fully connected, two tanh hidden layers) trained to approximate the function `sin(x) + 0.5·cos(2x)`.
|
| 4 |
+
|
| 5 |
+
## Architecture
|
| 6 |
+
|
| 7 |
+
A multilayer perceptron 1 → 6 → 7 → 1: one number in, one number out, two hidden layers with the tanh activation.
|
| 8 |
+
|
| 9 |
+
```
|
| 10 |
+
x ──► Linear(1→6) ──► tanh ──► Linear(6→7) ──► tanh ──► Linear(7→1) ──► ŷ
|
| 11 |
+
```
|
| 12 |
+
|
| 13 |
+
| Layer | Weights | Biases | Parameters |
|
| 14 |
+
|---|---|---|---|
|
| 15 |
+
| Linear(1 → 6) | 6 × 1 | 6 | 12 |
|
| 16 |
+
| Linear(6 → 7) | 7 × 6 | 7 | 49 |
|
| 17 |
+
| Linear(7 → 1) | 1 × 7 | 1 | 8 |
|
| 18 |
+
| **Total** | | | **69** |
|
| 19 |
+
|
| 20 |
+
| Property | Value |
|
| 21 |
+
|---|---|
|
| 22 |
+
| input / output size | 1 / 1 |
|
| 23 |
+
| hidden layers | 2 (sizes 6 and 7) |
|
| 24 |
+
| activation function | tanh on hidden layers, linear output |
|
| 25 |
+
| weight dtype | `float32` |
|
| 26 |
+
| initialization | weights ~ N(0, 1/fan_in), biases = 0 |
|
| 27 |
+
|
| 28 |
+
## Training
|
| 29 |
+
|
| 30 |
+
**Data.** A synthetic dataset:
|
| 31 |
+
- `x` is uniformly distributed on [−π, π];
|
| 32 |
+
- `y = sin(x) + 0.5·cos(2x) + ε`, where `ε` is Gaussian noise with a standard deviation of 0.05;
|
| 33 |
+
- 100 points for training and 50 points for validation, the random number generator is seeded (`numpy.random.default_rng(0)`).
|
| 34 |
+
|
| 35 |
+
**Loss function.** Mean squared error (MSE): `L = mean((ŷ − y)²)`.
|
| 36 |
+
Gradients are computed by hand with backpropagation (NumPy implementation, see `train.py`).
|
| 37 |
+
|
| 38 |
+
**Optimization.** Full-batch Adam (all 100 training points at every step).
|
| 39 |
+
|
| 40 |
+
| Hyperparameter | Value |
|
| 41 |
+
|---|---|
|
| 42 |
+
| optimizer | Adam (β₁ = 0.9, β₂ = 0.999, ε = 1e-8) |
|
| 43 |
+
| learning rate | 0.02 (constant) |
|
| 44 |
+
| iterations | 5000 |
|
| 45 |
+
| batch size | 100 (full batch) |
|
| 46 |
+
|
| 47 |
+
## Training result
|
| 48 |
+
|
| 49 |
+
| Iteration | train MSE | validation MSE |
|
| 50 |
+
|---|---|---|
|
| 51 |
+
| 1 | 1.39811 | 1.29456 |
|
| 52 |
+
| 500 | 0.00737 | 0.00772 |
|
| 53 |
+
| 1000 | 0.00714 | 0.00771 |
|
| 54 |
+
| 2000 | 0.00687 | 0.00729 |
|
| 55 |
+
| 3000 | 0.00674 | 0.00700 |
|
| 56 |
+
| 4000 | 0.00656 | 0.00688 |
|
| 57 |
+
| 5000 (final) | 0.00629 | 0.00682 |
|
| 58 |
+
|
| 59 |
+
The noise variance in the data (σ²) is 0.0025, so the final error is about 2.5 times higher. The model has not reached the noise level:
|
| 60 |
+
the loss is still decreasing slowly at iteration 5000. With only 69 parameters the network most likely lacks the capacity to follow the function exactly,
|
| 61 |
+
and it may also need more iterations; neither explanation was tested. Train and validation MSE are close, so there is no overfitting.
|
README_RU.md
ADDED
|
@@ -0,0 +1,61 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Claude Fable 6.9
|
| 2 |
+
|
| 3 |
+
Крошечная нейросеть на 69 параметров (полносвязная, два скрытых слоя с tanh), обученная аппроксимировать функцию `sin(x) + 0.5·cos(2x)`.
|
| 4 |
+
|
| 5 |
+
## Архитектура
|
| 6 |
+
|
| 7 |
+
Многослойный перцептрон 1 → 6 → 7 → 1: вход и выход — по одному числу, два скрытых слоя с функцией активации tanh.
|
| 8 |
+
|
| 9 |
+
```
|
| 10 |
+
x ──► Linear(1→6) ──► tanh ──► Linear(6→7) ──► tanh ──► Linear(7→1) ──► ŷ
|
| 11 |
+
```
|
| 12 |
+
|
| 13 |
+
| Слой | Вес | Смещение | Параметров |
|
| 14 |
+
|---|---|---|---|
|
| 15 |
+
| Linear(1 → 6) | 6 × 1 | 6 | 12 |
|
| 16 |
+
| Linear(6 → 7) | 7 × 6 | 7 | 49 |
|
| 17 |
+
| Linear(7 → 1) | 1 × 7 | 1 | 8 |
|
| 18 |
+
| **Итого** | | | **69** |
|
| 19 |
+
|
| 20 |
+
| Характеристика | Значение |
|
| 21 |
+
|---|---|
|
| 22 |
+
| размер входа / выхода | 1 / 1 |
|
| 23 |
+
| скрытые слои | 2 (размеры 6 и 7) |
|
| 24 |
+
| функция активации | tanh на скрытых слоях, на выходе линейный |
|
| 25 |
+
| тип весов | `float32` |
|
| 26 |
+
| инициализация | веса ~ N(0, 1/fan_in), смещения = 0 |
|
| 27 |
+
|
| 28 |
+
## Обучение
|
| 29 |
+
|
| 30 |
+
**Данные.** Синтетическая выборка:
|
| 31 |
+
- `x` — равномерное распределение на [−π, π];
|
| 32 |
+
- `y = sin(x) + 0.5·cos(2x) + ε`, где `ε` — гауссовский шум со стандартным отклонением 0.05;
|
| 33 |
+
- 100 точек для обучения и 50 точек для проверки (validation), генератор зафиксирован (`numpy.random.default_rng(0)`).
|
| 34 |
+
|
| 35 |
+
**Функция потерь.** Среднеквадратичная ошибка (MSE): `L = mean((ŷ − y)²)`.
|
| 36 |
+
Градиенты считаются вручную обратным распространением ошибки (реализация на NumPy, см. `train.py`).
|
| 37 |
+
|
| 38 |
+
**Оптимизация.** Adam на полном батче (все 100 обучающих точек на каждом шаге).
|
| 39 |
+
|
| 40 |
+
| Гиперпараметр | Значение |
|
| 41 |
+
|---|---|
|
| 42 |
+
| оптимизатор | Adam (β₁ = 0.9, β₂ = 0.999, ε = 1e-8) |
|
| 43 |
+
| скорость обучения | 0.02 (постоянная) |
|
| 44 |
+
| число итераций | 5000 |
|
| 45 |
+
| размер батча | 100 (полный) |
|
| 46 |
+
|
| 47 |
+
## Результат обучения
|
| 48 |
+
|
| 49 |
+
| Итерация | train MSE | validation MSE |
|
| 50 |
+
|---|---|---|
|
| 51 |
+
| 1 | 1.39811 | 1.29456 |
|
| 52 |
+
| 500 | 0.00737 | 0.00772 |
|
| 53 |
+
| 1000 | 0.00714 | 0.00771 |
|
| 54 |
+
| 2000 | 0.00687 | 0.00729 |
|
| 55 |
+
| 3000 | 0.00674 | 0.00700 |
|
| 56 |
+
| 4000 | 0.00656 | 0.00688 |
|
| 57 |
+
| 5000 (итог) | 0.00629 | 0.00682 |
|
| 58 |
+
|
| 59 |
+
Дисперсия шума в данных (σ²) равна 0.0025, то есть итоговая ошибка примерно в 2.5 раза выше. До уровня шума модель не дошла:
|
| 60 |
+
к 5000-й итерации loss ещё медленно снижается. Скорее всего, 69 параметров не хватает, чтобы точно повторить функцию, и возможно, нужно больше
|
| 61 |
+
итераций; ни одно из объяснений не проверялось. Train и validation MSE близки, переобучения нет.
|
model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:bc4aba8b6c7b2f13ca5ab6770d64cafda83e7431d8ba63c2c2226ff4b3d99448
|
| 3 |
+
size 812
|