federation

Next Federation

Next Federation trains one digit classifier across 10 clients whose data are distributed differently (Dirichlet label skew) and who sometimes drop out. It compares FedAvg, FedProx and local-only training on global accuracy, the worst client's accuracy on its own data, and communication traffic. This is a simulation: all clients run in one process; nothing was deployed to real devices.

Space: https://huggingface.co/spaces/thelamapi/Next-Federation — change skew, dropout, clients and rounds; all three methods retrain in the browser.

Results (5 seeds per setting, 40 rounds, 2 local epochs, lr 0.3, FedProx μ 0.01)

Setting Method Global test accuracy Worst client Mean client Traffic (MB)
alpha=0.1, dropout=0.0 FedAvg 0.934 ± 0.011 0.784 ± 0.100 0.940 7.7
alpha=0.1, dropout=0.0 FedProx 0.934 ± 0.011 0.784 ± 0.100 0.940 7.7
alpha=0.1, dropout=0.0 Local only 0.357 ± 0.011 0.448 ± 0.391 0.920 0.0
alpha=0.1, dropout=0.3 FedAvg 0.928 ± 0.019 0.776 ± 0.090 0.934 6.5
alpha=0.1, dropout=0.3 FedProx 0.929 ± 0.018 0.769 ± 0.090 0.932 6.5
alpha=0.1, dropout=0.3 Local only 0.357 ± 0.011 0.448 ± 0.391 0.920 0.0
alpha=0.5, dropout=0.0 FedAvg 0.953 ± 0.004 0.866 ± 0.031 0.947 7.7
alpha=0.5, dropout=0.0 FedProx 0.952 ± 0.003 0.866 ± 0.031 0.948 7.7
alpha=0.5, dropout=0.0 Local only 0.657 ± 0.019 0.674 ± 0.092 0.881 0.0
alpha=0.5, dropout=0.3 FedAvg 0.952 ± 0.005 0.860 ± 0.025 0.949 6.5
alpha=0.5, dropout=0.3 FedProx 0.952 ± 0.006 0.860 ± 0.025 0.949 6.5
alpha=0.5, dropout=0.3 Local only 0.657 ± 0.019 0.674 ± 0.092 0.881 0.0
alpha=5.0, dropout=0.0 FedAvg 0.961 ± 0.004 0.880 ± 0.013 0.956 7.7
alpha=5.0, dropout=0.0 FedProx 0.960 ± 0.005 0.880 ± 0.013 0.955 7.7
alpha=5.0, dropout=0.0 Local only 0.871 ± 0.006 0.755 ± 0.074 0.883 0.0
alpha=5.0, dropout=0.3 FedAvg 0.960 ± 0.005 0.890 ± 0.013 0.955 6.5
alpha=5.0, dropout=0.3 FedProx 0.959 ± 0.005 0.890 ± 0.013 0.955 6.5
alpha=5.0, dropout=0.3 Local only 0.871 ± 0.006 0.755 ± 0.074 0.883 0.0

Global test set: 360 digits never used for training or tuning. "Worst client" is the lowest accuracy over clients on their own held-out 20%. Local-only global accuracy is the mean of the 10 client models. Traffic counts float32 model downloads to every client every round plus uploads that arrive.

Benchmark

Method

  • Model: MLP 64→32→10 with ReLU, softmax cross-entropy, minibatch SGD (batch 20).
  • FedAvg: sample-weighted average of the updates that arrive. FedProx adds (μ/2)‖w − w_global‖² to every local step.
  • Dropout: a client's update is lost with probability p (its download still counts).
  • lr and μ were chosen on a separate validation set with development seeds 100–101 (lr scores {0.05: 0.9027777777777778, 0.1: 0.9361111111111111, 0.3: 0.9527777777777777}, μ scores {0.01: 0.9527777777777777, 0.1: 0.95, 1.0: 0.9333333333333333}).
  • The Space runs a line-by-line JavaScript port; Node parity on 4 fixed runs: max abs difference 0.
  • Released weights (model.json): FedProx global model from α=0.5, dropout 0.3, seed 0.

Usage

from inference import load, predict
spec = load()
probs = predict(spec, pixels)   # pixels: (N, 64) ints 0..16

Limitations

  • Simulated clients on a small, clean dataset; real federations have system heterogeneity, stragglers and privacy mechanisms (none here).
  • Dropout is independent per client and round; real outages are correlated.
  • Recorded on NVIDIA L4. The numbers describe this simulation only.
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Space using thelamapi/Next-Federation 1

Collection including thelamapi/Next-Federation