Claude Opus 5.5
A 55,000-parameter neural network (fully connected, four tanh hidden layers) trained to approximate the function sin(2x) + 0.5Β·cos(5x).
Architecture
A multilayer perceptron 1 β 64 β 151 β 208 β 64 β 1: one number in, one number out, four hidden layers with the tanh activation.
x βββΊ Linear(1β64) βββΊ tanh βββΊ Linear(64β151) βββΊ tanh βββΊ Linear(151β208) βββΊ tanh βββΊ Linear(208β64) βββΊ tanh βββΊ Linear(64β1) βββΊ Ε·
| Layer | Weights | Biases | Parameters |
|---|---|---|---|
| Linear(1 β 64) | 64 Γ 1 | 64 | 128 |
| Linear(64 β 151) | 151 Γ 64 | 151 | 9,815 |
| Linear(151 β 208) | 208 Γ 151 | 208 | 31,616 |
| Linear(208 β 64) | 64 Γ 208 | 64 | 13,376 |
| Linear(64 β 1) | 1 Γ 64 | 1 | 65 |
| Total | 55,000 |
| Property | Value |
|---|---|
| input / output size | 1 / 1 |
| hidden layers | 4 (sizes 64, 151, 208, 64) |
| activation function | tanh on hidden layers, linear output |
| weight dtype | float32 |
| initialization | weights ~ N(0, 1/fan_in), biases = 0 |
The hidden layer sizes were chosen so that the total parameter count is exactly 55,000.
Training
Data. A synthetic dataset:
xis uniformly distributed on [β3, 3];y = sin(2x) + 0.5Β·cos(5x) + Ξ΅, whereΞ΅is Gaussian noise with a standard deviation of 0.05;- 512 points for training and 256 points for validation, the random number generator is seeded (
numpy.random.default_rng(0)).
Loss function. Mean squared error (MSE): L = mean((Ε· β y)Β²).
Gradients are computed by hand with backpropagation (NumPy implementation, see train.py).
Optimization. Full-batch Adam (all 512 training points at every step).
| Hyperparameter | Value |
|---|---|
| optimizer | Adam (Ξ²β = 0.9, Ξ²β = 0.999, Ξ΅ = 1e-8) |
| learning rate | 0.002 at the start, cosine decay to 0 |
| iterations | 4000 |
| batch size | 512 (full batch) |
An earlier run with a constant learning rate of 0.002 ended with a worse validation MSE (0.00327 versus 0.00290 here) because the loss drifted upwards at the end of training; the cosine decay was added to fix this and is what produced the final model.
Training result
| Iteration | train MSE | validation MSE |
|---|---|---|
| 1 | 1.26461 | 1.31803 |
| 400 | 0.00270 | 0.00333 |
| 800 | 0.00210 | 0.00295 |
| 1200 | 0.00203 | 0.00290 |
| 2000 | 0.00201 | 0.00289 |
| 4000 (final) | 0.00200 | 0.00290 |
The noise variance in the data (ΟΒ²) is 0.0025. The validation MSE is about 16% above it, and the train MSE is slightly below it (0.0020), which indicates mild fitting to the noise in the training points: the network has far more parameters (55,000) than training points (512). The gap between train and validation MSE is small, and the validation MSE stays flat from iteration 1200 onwards.