Upload 3 files
Browse files- README.md +64 -3
- README_RU.md +64 -0
- model.safetensors +3 -0
README.md
CHANGED
|
@@ -1,3 +1,64 @@
|
|
| 1 |
-
-
|
| 2 |
-
|
| 3 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# GPT-6 Cyber
|
| 2 |
+
|
| 3 |
+
A 60,000-parameter neural network (fully connected, four tanh hidden layers) trained to approximate the function `sin(3x)·cos(x) + 0.1·x²`.
|
| 4 |
+
|
| 5 |
+
## Architecture
|
| 6 |
+
|
| 7 |
+
A multilayer perceptron 1 → 144 → 128 → 127 → 192 → 1: one number in, one number out, four hidden layers with the tanh activation.
|
| 8 |
+
|
| 9 |
+
```
|
| 10 |
+
x ──► Linear(1→144) ──► tanh ──► Linear(144→128) ──► tanh ──► Linear(128→127) ──► tanh ──► Linear(127→192) ──► tanh ──► Linear(192→1) ──► ŷ
|
| 11 |
+
```
|
| 12 |
+
|
| 13 |
+
| Layer | Weights | Biases | Parameters |
|
| 14 |
+
|---|---|---|---|
|
| 15 |
+
| Linear(1 → 144) | 144 × 1 | 144 | 288 |
|
| 16 |
+
| Linear(144 → 128) | 128 × 144 | 128 | 18,560 |
|
| 17 |
+
| Linear(128 → 127) | 127 × 128 | 127 | 16,383 |
|
| 18 |
+
| Linear(127 → 192) | 192 × 127 | 192 | 24,576 |
|
| 19 |
+
| Linear(192 → 1) | 1 × 192 | 1 | 193 |
|
| 20 |
+
| **Total** | | | **60,000** |
|
| 21 |
+
|
| 22 |
+
| Property | Value |
|
| 23 |
+
|---|---|
|
| 24 |
+
| input / output size | 1 / 1 |
|
| 25 |
+
| hidden layers | 4 (sizes 144, 128, 127, 192) |
|
| 26 |
+
| activation function | tanh on hidden layers, linear output |
|
| 27 |
+
| weight dtype | `float32` |
|
| 28 |
+
| initialization | weights ~ N(0, 1/fan_in), biases = 0 |
|
| 29 |
+
|
| 30 |
+
The hidden layer sizes were chosen so that the total parameter count is exactly 60,000.
|
| 31 |
+
|
| 32 |
+
## Training
|
| 33 |
+
|
| 34 |
+
**Data.** A synthetic dataset:
|
| 35 |
+
- `x` is uniformly distributed on [−3, 3];
|
| 36 |
+
- `y = sin(3x)·cos(x) + 0.1·x² + ε`, where `ε` is Gaussian noise with a standard deviation of 0.05;
|
| 37 |
+
- 512 points for training and 256 points for validation, the random number generator is seeded (`numpy.random.default_rng(0)`).
|
| 38 |
+
|
| 39 |
+
**Loss function.** Mean squared error (MSE): `L = mean((ŷ − y)²)`.
|
| 40 |
+
Gradients are computed by hand with backpropagation (NumPy implementation, see `train.py`).
|
| 41 |
+
|
| 42 |
+
**Optimization.** Full-batch Adam (all 512 training points at every step).
|
| 43 |
+
|
| 44 |
+
| Hyperparameter | Value |
|
| 45 |
+
|---|---|
|
| 46 |
+
| optimizer | Adam (β₁ = 0.9, β₂ = 0.999, ε = 1e-8) |
|
| 47 |
+
| learning rate | 0.002 at the start, cosine decay to 0 |
|
| 48 |
+
| iterations | 4000 |
|
| 49 |
+
| batch size | 512 (full batch) |
|
| 50 |
+
|
| 51 |
+
## Training result
|
| 52 |
+
|
| 53 |
+
| Iteration | train MSE | validation MSE |
|
| 54 |
+
|---|---|---|
|
| 55 |
+
| 1 | 3.17048 | 3.02635 |
|
| 56 |
+
| 400 | 0.00699 | 0.00620 |
|
| 57 |
+
| 800 | 0.00229 | 0.00300 |
|
| 58 |
+
| 1200 | 0.00208 | 0.00286 |
|
| 59 |
+
| 2000 | 0.00204 | 0.00286 |
|
| 60 |
+
| 4000 (final) | 0.00202 | 0.00286 |
|
| 61 |
+
|
| 62 |
+
The noise variance in the data (σ²) is 0.0025. The validation MSE is about 14% above it, and the train MSE is slightly below it (0.0020),
|
| 63 |
+
which indicates mild fitting to the noise in the training points: the network has far more parameters (60,000) than training points (512).
|
| 64 |
+
The gap between train and validation MSE is small, and both stay almost flat from iteration 1200 onwards.
|
README_RU.md
ADDED
|
@@ -0,0 +1,64 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# GPT-6 Cyber
|
| 2 |
+
|
| 3 |
+
Нейросеть на 60 000 параметров (полносвязная, четыре скрытых слоя с tanh), обученная аппроксимировать функцию `sin(3x)·cos(x) + 0.1·x²`.
|
| 4 |
+
|
| 5 |
+
## Архитектура
|
| 6 |
+
|
| 7 |
+
Многослойный перцептрон 1 → 144 → 128 → 127 → 192 → 1: вход и выход — по одному числу, четыре скрытых слоя с функцией активации tanh.
|
| 8 |
+
|
| 9 |
+
```
|
| 10 |
+
x ──► Linear(1→144) ──► tanh ──► Linear(144→128) ──► tanh ──► Linear(128→127) ──► tanh ──► Linear(127→192) ──► tanh ──► Linear(192→1) ──► ŷ
|
| 11 |
+
```
|
| 12 |
+
|
| 13 |
+
| Слой | Вес | Смещение | Параметров |
|
| 14 |
+
|---|---|---|---|
|
| 15 |
+
| Linear(1 → 144) | 144 × 1 | 144 | 288 |
|
| 16 |
+
| Linear(144 → 128) | 128 × 144 | 128 | 18 560 |
|
| 17 |
+
| Linear(128 → 127) | 127 × 128 | 127 | 16 383 |
|
| 18 |
+
| Linear(127 → 192) | 192 × 127 | 192 | 24 576 |
|
| 19 |
+
| Linear(192 → 1) | 1 × 192 | 1 | 193 |
|
| 20 |
+
| **Итого** | | | **60 000** |
|
| 21 |
+
|
| 22 |
+
| Характеристика | Значение |
|
| 23 |
+
|---|---|
|
| 24 |
+
| размер входа / выхода | 1 / 1 |
|
| 25 |
+
| скрытые слои | 4 (размеры 144, 128, 127, 192) |
|
| 26 |
+
| функция активации | tanh на скрытых слоях, на выходе линейный |
|
| 27 |
+
| тип весов | `float32` |
|
| 28 |
+
| инициализация | веса ~ N(0, 1/fan_in), смещения = 0 |
|
| 29 |
+
|
| 30 |
+
Размеры скрытых слоёв подобраны так, чтобы общее число параметров равнялось ровно 60 000.
|
| 31 |
+
|
| 32 |
+
## Обучение
|
| 33 |
+
|
| 34 |
+
**Данные.** Синтетическая выборка:
|
| 35 |
+
- `x` — равномерное распределение на [−3, 3];
|
| 36 |
+
- `y = sin(3x)·cos(x) + 0.1·x² + ε`, где `ε` — гауссовский шум со стандартным отклонением 0.05;
|
| 37 |
+
- 512 точек для обучения и 256 точек для проверки (validation), генератор зафиксирован (`numpy.random.default_rng(0)`).
|
| 38 |
+
|
| 39 |
+
**Функция потерь.** Среднеквадратичная ошибка (MSE): `L = mean((ŷ − y)²)`.
|
| 40 |
+
Градиенты считаются вручную обратным распространением ошибки (реализация на NumPy, см. `train.py`).
|
| 41 |
+
|
| 42 |
+
**Оптимизация.** Adam на полном батче (все 512 обучающих точек на каждом шаге).
|
| 43 |
+
|
| 44 |
+
| Гиперпараметр | Значение |
|
| 45 |
+
|---|---|
|
| 46 |
+
| оптимизатор | Adam (β₁ = 0.9, β₂ = 0.999, ε = 1e-8) |
|
| 47 |
+
| скорость обучения | 0.002 в начале, косинусное затухание до 0 |
|
| 48 |
+
| число итераций | 4000 |
|
| 49 |
+
| размер батча | 512 (полный) |
|
| 50 |
+
|
| 51 |
+
## Результат обучения
|
| 52 |
+
|
| 53 |
+
| Итерация | train MSE | validation MSE |
|
| 54 |
+
|---|---|---|
|
| 55 |
+
| 1 | 3.17048 | 3.02635 |
|
| 56 |
+
| 400 | 0.00699 | 0.00620 |
|
| 57 |
+
| 800 | 0.00229 | 0.00300 |
|
| 58 |
+
| 1200 | 0.00208 | 0.00286 |
|
| 59 |
+
| 2000 | 0.00204 | 0.00286 |
|
| 60 |
+
| 4000 (итог) | 0.00202 | 0.00286 |
|
| 61 |
+
|
| 62 |
+
Дисперсия шума в данных (σ²) равна 0.0025. Validation MSE примерно на 14% выше неё, а train MSE немного ниже (0.0020),
|
| 63 |
+
что говорит о лёгкой подгонке под шум в обучающих точках: параметров (60 000) намного больше, чем обучающих точек (512).
|
| 64 |
+
Разрыв между train и validation MSE небольшой, а обе величины с 1200-й итерации почти не меняются.
|
model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7da7e540f2ff689a86310bc0299a78f5db84791dc48a26d70d85dd5bfdcf36b0
|
| 3 |
+
size 240896
|