hltv-tick-encoder / STAGE2.md
leminhohoho's picture
Upload STAGE2.md with huggingface_hub
9148022 verified
|
Raw History Blame Contribute Delete
4.46 kB

Stage 2 - do tick-level SSL embeddings predict demo events better than hand features?

Data: 1313 maps, 27525 rounds, ~13k rounds/... (see PHASE notes). Encoder = t1_model.py trained 14,000 steps on 364 maps (SSL: next view, next displacement, next alive, masked-other) - the "ck14k" checkpoint. Probe = fixed-capacity MLP per target, refit on 100% disjoint data.

Protocol (t2_probe2.py) - the thing that was broken before

  • Iteration 1/2 (t2_probe.py): test matches were only excluded from a third of the TRAIN windows, so ~2/3 of test windows shared matches with train, and run1 vs run2 disagreed in sign. Both discarded.
  • Iteration 3: 20% of MATCHES held out globally by match id, decided once with a fixed seed. Encoder-pretrain matches (364) removed from the test pool entirely -> test matches (71) are unseen by both the encoder and the probe.
  • One row per (window, player) at the last observed tick; labels = event in the next 1 s. 29,284 train rows / 11,121 test rows. 3 probe seeds averaged. 400-rep cluster bootstrap over windows for every CI, and PAIRED bootstrap for every difference.

Results - 30k-step encoder (re-probe, same protocol)

target hand trained enc (alone) random enc + hand enc + hand paired delta vs random+hand
enemy sees me .613 .680 .753 .761 +.009 [-.020, +.037]
I kill or die .619 .690 .761 .783 +.021 [-.015, +.065]
I take damage .663 .630 .728 .762 +.033 [+.012, +.062]
I fire .607 .594 .653 .668 +.016 [+.006, +.026]

The 30k checkpoint is essentially identical to the 14k one (+/-1 AUC point, no consistent direction, loss curves flat after ~10k). SSL budget is not the constraint; the objective is.

Three things that are solid:

  1. A single frozen encoder, trained only to predict the next 1 s of raw state, supports 4 different exact demo-event probes at AUC .67-.78 on unseen matches. Nothing aggregate was used as input.
  2. The joint 10-player state at 8 Hz is worth +15 to +17 AUC points over the same features computed per-player by hand (that's the enc+hand - hand row).
  3. The self-supervision itself is worth about +1 to +3 points over an identically-shaped encoder with random weights - real but small, and only 2/4 targets get a CI that excludes zero.

Results (AUC, 71 unseen matches)

target base rate hand untrained encoder + hand trained enc + hand paired delta (enc+hand - untrained)
enemy sees me 1.9% .613 .753 .772 +.020 [-.003, +.042]
I kill or die 1.9% .619 .761 .781 +.019 [-.012, +.055]
I take damage 2.9% .663 .728 .760 +.033 [+.012, +.052]
I fire 8.8% .607 .653 .667 +.015 [+.004, +.026]

Controls that matter:

  • raw window flat (last 16/32/48 ticks x 10 players x 9 channels, flattened -> 1440/2880/4320 dims, same MLP): .656/.635/.640 + hand on "spotted", i.e. WAY below enc+hand .772. => the encoder is not just "a summary of the window"; a same-capacity MLP on the flattened window cannot get there. (Note raw_w64 = 5760 dims is degenerate at this n - MLP can't fit, AUC .50.)
  • untrained encoder (same architecture, random weights) + hand = .753/.761/.728/.653. This is a strong baseline: random nonlinear projection of the 10-player state already buys a lot over hand. The trained encoder's advantage over it is small (+1.5 to +3.3 AUC points) and marginal.
  • hand alone = .607-.663. So "raw joint state + any nonlinear head" >> hand features, but only a little of that is learned by SSL.

Honest reading

  1. Predicting the next second of a CS2 round from raw per-tick state is EASY: AUC .78 for duel onset, .77 for being seen. That is the actual new capability, not a P4 signal.
  2. The self-supervised objective contributes ~+1.5-3.3 AUC points over a randomly-weighted encoder of identical capacity. Sign is consistently positive (4/4 targets), significant on 2/4 (damaged, fired) with paired CIs.
  3. Everything that "beat HLTV features" here is mostly the joint 10-player state at 8 Hz, not the aggregate-stat-free representation. Worth stating plainly because it was the premise.

Files

tools/t2_probe2.py (protocol), /root/t2_pair.py (paired bootstrap), /root/t2_add.py, /root/t2_add2.py (controls), tools/t4_fig_stage2.py (figure), data/ticks/pr14k*.json, notes/fig_stage2.png