Gemini 4 Carbon

Today the weights of Gemini 4 Carbon were leaked by the hacker Salgr. Below is all the information about this model:

Architecture

A single linear layer without a bias term: one number in, one number out.

x  ──►  [ w ]  ──►  ŷ = w · x
Property Value
input / output size 1 / 1
layers 1 linear (equivalent to Linear(1, 1, bias=False))
activation functions none
trainable parameters 1 (w)
weight dtype and shape float32, [1, 1]
initial value w = 0

Training

Data. A synthetic dataset of 100 points:

  • x is uniformly distributed on [−1, 1];
  • y = 3·x + ε, where ε is Gaussian noise with a standard deviation of 0.1;
  • the random number generator is seeded (seed = 0).

There is no train/test split; all 100 points are used for training.

Loss function. Mean squared error:

L(w) = mean((w·x − y)²)

Optimization. Full-batch gradient descent (all 100 points at every step):

dL/dw = mean(2·(w·x − y)·x)
w ← w − lr · dL/dw
Hyperparameter Value
learning rate (lr) 0.5
iterations 201
batch size 100 (full batch)

Training result

Quantity Value
trained w 2.9594
final loss (MSE) 0.00848
w used to generate the data 3.0

The loss stops changing at around iteration 40: the gradient at that point is zero, so additional iterations do not change the result. The gap between 2.9594 and 3.0 is caused by the noise in the data: on a given sample, the MSE minimum is at w = Σxy / Σx², not at the "true" value.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Collection including salgr/Gemini-4-Carbon