Claude Fable 6.9

A tiny 69-parameter neural network (fully connected, two tanh hidden layers) trained to approximate the function sin(x) + 0.5Β·cos(2x).

Architecture

A multilayer perceptron 1 β†’ 6 β†’ 7 β†’ 1: one number in, one number out, two hidden layers with the tanh activation.

x ──► Linear(1β†’6) ──► tanh ──► Linear(6β†’7) ──► tanh ──► Linear(7β†’1) ──► Ε·
Layer Weights Biases Parameters
Linear(1 β†’ 6) 6 Γ— 1 6 12
Linear(6 β†’ 7) 7 Γ— 6 7 49
Linear(7 β†’ 1) 1 Γ— 7 1 8
Total 69
Property Value
input / output size 1 / 1
hidden layers 2 (sizes 6 and 7)
activation function tanh on hidden layers, linear output
weight dtype float32
initialization weights ~ N(0, 1/fan_in), biases = 0

Training

Data. A synthetic dataset:

  • x is uniformly distributed on [βˆ’Ο€, Ο€];
  • y = sin(x) + 0.5Β·cos(2x) + Ξ΅, where Ξ΅ is Gaussian noise with a standard deviation of 0.05;
  • 100 points for training and 50 points for validation, the random number generator is seeded (numpy.random.default_rng(0)).

Loss function. Mean squared error (MSE): L = mean((Ε· βˆ’ y)Β²). Gradients are computed by hand with backpropagation (NumPy implementation, see train.py).

Optimization. Full-batch Adam (all 100 training points at every step).

Hyperparameter Value
optimizer Adam (β₁ = 0.9, Ξ²β‚‚ = 0.999, Ξ΅ = 1e-8)
learning rate 0.02 (constant)
iterations 5000
batch size 100 (full batch)

Training result

Iteration train MSE validation MSE
1 1.39811 1.29456
500 0.00737 0.00772
1000 0.00714 0.00771
2000 0.00687 0.00729
3000 0.00674 0.00700
4000 0.00656 0.00688
5000 (final) 0.00629 0.00682

The noise variance in the data (σ²) is 0.0025, so the final error is about 2.5 times higher. The model has not reached the noise level: the loss is still decreasing slowly at iteration 5000. With only 69 parameters the network most likely lacks the capacity to follow the function exactly, and it may also need more iterations; neither explanation was tested. Train and validation MSE are close, so there is no overfitting.

Downloads last month

-

Downloads are not tracked for this model. How to track
Safetensors
Model size
69 params
Tensor type
F32
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Collection including salgr/Claude-Fable-6.9