ritvik-su/smai-a4-mlp-p0

MLP (3072-128-64-32-10, ReLU)

Trained for Task 1.4 of SMAI Assignment 4 (CIFAR-10, true labels, noise level p = 0) and compared in Task 1.5.

Architecture

• Layers: 3072 -> 128 -> 64 -> 32 -> 10, hidden activation ReLU, no activation on the output (logits).
• 404,010 parameters.
• No constraints on the weights. Initialisation of Task 1.2.2: normal with variance 2/n_in in the ReLU layers, 1/n_in in the output layer, biases 0.

Preprocessing

Flatten the image to 3072 numbers (row-major, X.reshape(n, 3072)), convert to float64, divide by 255, then (x - mean) / std with the arrays in preprocessing.npz (mean and standard deviation of every column of the 50,000-image train part; divisor n, and 1 where the standard deviation is 0).

Files

• weights.npz: arrays w0..w3 with shape (n_in, n_out) and b0..b3. Forward pass: h = act(h @ w_i + b_i) for the first three layers and logits = h @ w3 + b3.
• preprocessing.npz: arrays mean and std, both of shape (3072,).

Training

PyTorch, SGD (no momentum), learning rate 0.05, batch size 64, gradient accumulation 1, cross-entropy loss, 100 epochs, no early stopping. Seed 22012 (run r = 0): torch.manual_seed(22013) before building the models and torch.Generator().manual_seed(22014) for the batch order.

Results (CIFAR-10 test part, 10,000 images)

train loss	test loss	train accuracy	test accuracy	mean F1
0.2740	3.4904	0.9031	0.4828	0.4813