Claude Opus 5.5

A 55,000-parameter neural network (fully connected, four tanh hidden layers) trained to approximate the function sin(2x) + 0.5Β·cos(5x).

Architecture

A multilayer perceptron 1 β†’ 64 β†’ 151 β†’ 208 β†’ 64 β†’ 1: one number in, one number out, four hidden layers with the tanh activation.

x ──► Linear(1β†’64) ──► tanh ──► Linear(64β†’151) ──► tanh ──► Linear(151β†’208) ──► tanh ──► Linear(208β†’64) ──► tanh ──► Linear(64β†’1) ──► Ε·
Layer Weights Biases Parameters
Linear(1 β†’ 64) 64 Γ— 1 64 128
Linear(64 β†’ 151) 151 Γ— 64 151 9,815
Linear(151 β†’ 208) 208 Γ— 151 208 31,616
Linear(208 β†’ 64) 64 Γ— 208 64 13,376
Linear(64 β†’ 1) 1 Γ— 64 1 65
Total 55,000
Property Value
input / output size 1 / 1
hidden layers 4 (sizes 64, 151, 208, 64)
activation function tanh on hidden layers, linear output
weight dtype float32
initialization weights ~ N(0, 1/fan_in), biases = 0

The hidden layer sizes were chosen so that the total parameter count is exactly 55,000.

Training

Data. A synthetic dataset:

  • x is uniformly distributed on [βˆ’3, 3];
  • y = sin(2x) + 0.5Β·cos(5x) + Ξ΅, where Ξ΅ is Gaussian noise with a standard deviation of 0.05;
  • 512 points for training and 256 points for validation, the random number generator is seeded (numpy.random.default_rng(0)).

Loss function. Mean squared error (MSE): L = mean((Ε· βˆ’ y)Β²). Gradients are computed by hand with backpropagation (NumPy implementation, see train.py).

Optimization. Full-batch Adam (all 512 training points at every step).

Hyperparameter Value
optimizer Adam (β₁ = 0.9, Ξ²β‚‚ = 0.999, Ξ΅ = 1e-8)
learning rate 0.002 at the start, cosine decay to 0
iterations 4000
batch size 512 (full batch)

An earlier run with a constant learning rate of 0.002 ended with a worse validation MSE (0.00327 versus 0.00290 here) because the loss drifted upwards at the end of training; the cosine decay was added to fix this and is what produced the final model.

Training result

Iteration train MSE validation MSE
1 1.26461 1.31803
400 0.00270 0.00333
800 0.00210 0.00295
1200 0.00203 0.00290
2000 0.00201 0.00289
4000 (final) 0.00200 0.00290

The noise variance in the data (σ²) is 0.0025. The validation MSE is about 16% above it, and the train MSE is slightly below it (0.0020), which indicates mild fitting to the noise in the training points: the network has far more parameters (55,000) than training points (512). The gap between train and validation MSE is small, and the validation MSE stays flat from iteration 1200 onwards.

Downloads last month

-

Downloads are not tracked for this model. How to track
Safetensors
Model size
55k params
Tensor type
F32
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Collection including salgr/Claude-Opus-5.5