MoE_detector / RESEARCH_LOG.md
assa8945's picture
Research log section 16 (2026-11-15): MoE (QL r7, D3) table, round-0 ablation, fusion / frame-selection / temporal-key tests (STOP), second-detector flywheels, VideoFACT feedback (VFFB, VFFB2 running), last-checkpoint rule
367dba8 verified
|
Raw History Blame Contribute Delete
49.5 kB

MoE detector: research log (Setting G)

Last update: 2026-11-15 (section 16). This is a private working log. All numbers come from local runs on one test set (Setting G). There is no peer review yet. The text follows ASD-STE100 Simplified Technical English.

Working title: Route by Reliability: a degradation-aware mixture of forensic experts for the detection of AI-generated video.

0. Target test set: mix_setting_v1 (added 2026-10-25)

Since 2026-10-24 the only target test set is mix_setting_v1. Sections 1–7 report Setting G and stay as development history. The term "re-encodes" replaces "hops" for the length of a repost chain.

  • Set: batch_infer/combination_test/mix_setting_v1/manifest.csv. 10,000 slots; 9,939 videos are ready (2,949 real, 6,990 fake). 61 SkyReels-V3 extension slots are not encoded yet.
  • Re-encodes: every video has 1, 5, 10 or 20 re-encodes (4:3:2:1). There is no set with 0 re-encodes.
  • Platforms: 12 profiles (6 old + 6 measured *_real). Chains with 5 or more re-encodes mix the platforms. No mix platform is a router platform.
  • Protocol: as in section 2. Platt on the router calib_fit, Youden threshold on the router calib_select. No mix labels are fitted.

0.1 Main result (n = 9,939; 95% CI from 1,000 paired bootstrap resamples grouped by source video, 4,689 groups)

Model AUC [95% CI] Macro F1 [95% CI] TPR TNR
E2 D3 0.693 [0.682, 0.705] 0.554 [0.543, 0.564] 0.476 0.757
E3β€² WaveRep-DINOv3 0.803 [0.794, 0.811] 0.695 [0.685, 0.705] 0.841 0.540
Mean of z (E2, E3β€²) 0.842 [0.834, 0.849] 0.735 [0.726, 0.745] 0.844 0.627
  • The mean of z is better than E3β€² alone: Ξ”AUC +0.039 [0.034, 0.043], Ξ”F1 +0.041 [0.034, 0.049].
  • E1 (released WaveRep) is not scored on mix_setting_v1 yet.
Re-encodes (mean of z) 1 5 10 20
AUC / Macro F1 0.904 / 0.803 0.819 / 0.711 0.792 / 0.694 0.736 / 0.588
Real TNR 0.71 0.68 0.57 0.25
Fake family vs all reals (mean of z) AUC / Macro F1
HunyuanVideo-1.5 0.963 / 0.746
Wan2.2 0.874 / 0.716
LTX-2 0.832 / 0.720
Wan-animate / replace 0.826 / 0.476
SkyReels-V3 0.700 / 0.613
Relight-Video 0.641 / 0.455
Platforms (mean of z) AUC / Macro F1
Old profiles only 0.896 / 0.792
Measured profiles only 0.914 / 0.813
Mixed chains 0.790 / 0.690
  • The weak points are the same as on G: false alarms on real videos after many re-encodes, SkyReels-V3 and Relight-Video.
  • Mixed-platform chains are harder than chains on one profile group.

0.2 Studies on mix_setting_v1 (training-free, no mix labels fitted)

Study Model AUC / Macro F1 Key detail
Problem 1: real-reference score (Β§7.5, one bin) Real reference alone 0.798 / 0.685 20 re-encodes: 0.718 / 0.658, real TNR 0.63 (E3β€²: 0.676 / 0.540, TNR 0.18)
Mean of z (E2, E3β€², real reference) 0.837 / 0.723 no overall gain
Problem 2: change-point scores on the E3β€² frame logits Mean of z (E2, E3β€², CUSUM) 0.839 / 0.730 SkyReels-V3 0.700 β†’ 0.761; LTX-2 0.832 β†’ 0.798
Mean of z (E2, E3β€², late βˆ’ early) 0.839 / 0.731 SkyReels-V3 0.739
Feature change, distance to start 0.499, 0.480 no signal
Design 2: periodic VAE pattern (F4, frames de-duplicated) F4 alone 0.611 / 0.515 no help on SkyReels-V3 (0.508)
Mean of z (E2, E3β€², F4) 0.844 / 0.733 +0.002 AUC, no clear gain
Design 3 pilot: change-point transformer on feature splices Transformer alone 0.749 / 0.668 trained on router_train feature splices
Mean of z (E2, E3β€², transformer) 0.829 / 0.728 worse; SkyReels-V3 0.721
  • Problem 2: on mix, the SkyReels-V3 frame logit also goes up over the clip (βˆ’4.0 at frame 3, βˆ’0.5 at frame 15). A fixed temporal statistic helps SkyReels-V3 but makes the other generators worse.
  • Real videos with shot cuts do not give more false alarms with these scores (TNR 0.67 vs 0.56 for the other real videos).
  • Design 2 (analysis on our own videos, no mix):
    • Native gen4 videos have a clear 4-frame pattern in the noise residual: median F4 3–12; period 4 is the strongest period in 57–78% of the videos.
    • Platform frame-rate conversion adds a period-5 pattern to all videos.
    • With duplicate frames removed, the F4 AUC on router videos is 0.72 at 0 re-encodes and 0.59–0.66 after 1–20 re-encodes. The cue is real but weak.
  • Design 3 pilot:
    • The model finds the seam in router_val feature splices to within about 2 frames (router_val AUC 0.90).
    • It does not transfer to the new generators in mix. The router fakes are old generators, and feature splices have no real VAE seam.
    • Design 3 needs real extension clips from current generators.
  • Data: claude_workspace/csv/{mix_moe_*,mix_bootstrap,problem1_mix,problem2_changepoint_mix,mix_extra_expert_design2,design2_periodic*,design3_pilot_mix}.csv.

0.3 Expert studies on mix_setting_v1 (added 2026-10-26; GPU runs, no mix labels fitted)

Each new model replaces E3β€² (or is added to it) in the mean of z with E2. Same protocol as Β§0.1. Reference: E3β€² 0.803 / 0.695; mean of z (E2, E3β€²) 0.842 / 0.735.

Study Model alone AUC / Macro F1 Mean of z with E2 AUC / Macro F1 20 re-encodes (with E2) Key detail
E1 released WaveRep, added to (E2, E3β€²) 0.637 / 0.495 0.837 / 0.709 (E2, E3β€², E1) 0.731 / 0.654 E1 is near chance after 5+ re-encodes; no gain
E3β€² without HunyuanVideo-1.5 0.787 / 0.687 0.834 / 0.719 0.715 / 0.633 HunyuanVideo-1.5 (E3β€² alone) 0.940 β†’ 0.891
E3β€² without LTX-2 0.763 / 0.667 0.810 / 0.708 0.705 / 0.546 LTX-2 (E3β€² alone) 0.791 β†’ 0.723
E3β€² without SkyReels-V3 0.796 / 0.700 0.840 / 0.715 0.737 / 0.667 SkyReels-V3 (E3β€² alone) 0.681 β†’ 0.651
E3β€² without Wan2.2 0.798 / 0.696 0.839 / 0.737 0.720 / 0.611 Wan2.2 (E3β€² alone) 0.810 β†’ 0.809
Paired views, Ξ» = 0 (BCE on clean + re-encoded view) 0.790 / 0.690 0.833 / 0.718 0.696 / 0.606 control for Ξ» = 1
Paired views, Ξ» = 1 (+ consistency loss) 0.776 / 0.679 0.824 / 0.721 0.707 / 0.624 worse than E3β€² and than Ξ» = 0
  • E1: the released WaveRep adds no information to (E2, E3β€²) on mix.

  • Leave one generator out: overall AUC with E2 drops by 0.002–0.032. The left-out family loses the most (LTX-2 βˆ’0.068 for E3β€² alone). No single training generator is necessary; together they give the full result.

  • Paired consistency (problem 1): an explicit compression-invariance loss does not help. Both paired runs are below E3β€² at all re-encode levels, also at 20 re-encodes. The stochastic re-encode augmentation of E3β€² (p 0.3) is sufficient.

  • Re-encode-aware gate (crf_hat): the router CRF head (router_train only) predicts the re-encode level of each mix video (Spearman 0.65 with the true count; it underestimates mix chains: median crf_hat 1.46 at 5 re-encodes vs 2.80 on router). Six designs on (z_d, z_v), all fitted on router calib only, 3 or 5 crf_hat bins:

    Design (3 bins) AUC / Macro F1 Ξ”AUC [95% CI] vs mean of z 20 re-encodes AUC / F1 / TNR
    Mean of z (E2, E3β€²), one threshold (current) 0.842 / 0.735 β€” 0.736 / 0.588 / 0.25
    Threshold per bin 0.842 / 0.737 +0.000 [+0.000, +0.000] 0.736 / 0.618 / 0.33
    Expert weights per bin (E2 weight 0.42 β†’ 0.59 with depth) 0.842 / 0.736 +0.001 [βˆ’0.001, +0.002] 0.736 / 0.631 / 0.38
    Weights + threshold per bin 0.842 / 0.735 +0.001 [βˆ’0.001, +0.002] 0.736 / 0.633 / 0.38
    Stacker with crf_hat interactions 0.841 / 0.734 βˆ’0.001 [βˆ’0.003, +0.001] 0.738 / 0.654 / 0.47
    Soft gate w(crf_hat) 0.842 / 0.730 +0.000 [βˆ’0.001, +0.001] 0.743 / 0.656 / 0.45

    No design changes the overall AUC or Macro F1 (all CIs include 0). They only move the false-alarm / miss balance at 20 re-encodes (TNR 0.25 β†’ 0.47). This agrees with Β§7.3 on G: the plain mean of z stays the MoE. Data: csv/mix_crf_gate.csv, moe_mix/crf_hat_mix.csv.

  • UCF-Net-style feature fusion on mix (Β§7.1 method, frozen features, trained on router_train, 5 seeds; ucf_fusion_mix.py):

    Method (E2 + E3β€²) AUC / Macro F1 (5-seed ensemble) Mean Β± sd over seeds 20 re-encodes AUC / F1
    Mean of z (score level, current) 0.842 / 0.735 β€” 0.736 / 0.588
    Logistic stacker on z (score level) 0.843 / 0.737 β€” 0.739 / 0.605
    UCF: LEA + UAF 0.842 / 0.740 0.838 Β± 0.006 / 0.736 Β± 0.008 0.720 / 0.620
    LEA + sum 0.841 / 0.743 0.839 Β± 0.004 / 0.736 Β± 0.005 0.722 / 0.638
    LEA + concatenation 0.841 / 0.739 0.838 Β± 0.005 / 0.735 Β± 0.005 0.721 / 0.631
    LEA + cross-attention 0.840 / 0.737 0.835 Β± 0.006 / 0.732 Β± 0.006 0.716 / 0.626
    UAF without LEA 0.815 / 0.713 0.812 Β± 0.004 / 0.710 Β± 0.002 β€”

    Same result as on G: feature fusion equals score fusion (AUC 0.838–0.842 vs 0.842); UAF equals a plain sum; the UAF weights stay near 0.43 / 0.57 at all re-encode levels. Feature fusion is worse at 20 re-encodes in AUC (0.720 vs 0.736). Data: csv/ucf_fusion_mix*.csv.

  • Data: claude_workspace/csv/mix_extra_expert_{e1,logo,pair}.csv; logs claude_workspace/log/mix_extra_expert_{e1,logo,pair}.log.

1. Test set

  • The test set is Setting G. It has 9,468 videos that the experts can score: 3,010 real and 6,458 fake.
  • The fakes come from 7 generators.
  • Each video has a re-encode depth of 0, 1, 5, 10 or 20 hops. One hop is one re-encode by a simulated platform. The chains use 6 platforms.
  • The benchmark log gives more data about Setting G.

2. Protocol (all MoE results)

  • We do not use G labels in a part that we fit.
  • We fit Platt scaling for each expert on calib_fit.
  • We train the router heads and the stackers on router_train.
  • We select the temperature Ο„ on router_val.
  • The decision threshold is the Youden J point on calib_select. We test on G.
  • The router and calibration set has 9,577 videos:
    • 5,246 real videos from ActivityNet and InternVid.
    • Fakes from CogVideoX (3,064), HunyuanVideo (767), Kling (250), OpenSora (199) and Sora (51). 500 of these videos come from GenVidBench.
    • Each video is re-encoded 0, 1, 5, 10 or 20 hops. The platforms are not the G platforms: whatsapp_hd, wechat_chat, generic and their bad-connection variants.
  • Metrics: ACC, Macro F1, AUC, TPR (fake), TNR (real) and TPR at 1% FPR. We always show AUC and Macro F1 together.
  • Bootstrap: 1,000 paired resamples, grouped by source video name. We give the 95% percentile interval.

3. Experts

ID Expert Training data G AUC / Macro F1 (alone)
E1 WaveRep: the released checkpoint (DINOv2 ViT-B/14 reg4, weights_dinov2_G4.ckpt). Score = mean frame logit. Data of the authors; frozen 0.749 / 0.563 (threshold 0.5)
E2 D3: X-CLIP-B/16 features; std of the second-order frame-feature distance. Real if dis_std > 5. No training 0.695 / 0.646
E3 (old) Frozen DINOv3 ViT-L/16 temporal-difference features and a linear probe expert_train (CogVideoX fakes only) 0.606 / 0.560
E3β€² (new) WaveRep-DINOv3: our re-implementation of the WaveRep recipe on DINOv3 ViT-L/16. Full fine-tune, 5 s clips, re-encode augmentation. gen4 (4 modern generators, see section 8) 0.871 / 0.756

4. Main result: Setting G (n = 9,468)

Method ACC Macro F1 AUC TPR fake TNR real TPR@1%FPR
Old router E1–E3 (Ο„ = 2) 0.746 0.718 0.826 0.774 0.684 0.349
Old mean of z (E1, E2, E3) 0.740 0.713 0.819 0.768 0.679 0.340
E3β€² WaveRep-DINOv3 alone 0.794 0.751 0.871 0.887 0.594 0.432
Router E1, E2, E3β€² (Ο„ = 2) 0.805 0.785 0.887 0.816 0.783 0.432
Mean of z (E2, E3β€²) 0.816 0.783 0.892 0.885 0.668 0.449
Mean of z (E1, E2, E3β€²) 0.799 0.784 0.894 0.778 0.846 0.513
Logistic stacker on z (E1, E2, E3β€²): best model 0.813 0.795 0.896 0.818 0.804 0.494
[reference, uses G labels] in-G logistic on z 0.820 0.792 0.896 0.866 0.720 0.506

The best model is the logistic stacker on the Platt scores z of E1, E2 and E3β€². It is equal to the in-G reference that uses G labels.

4.1 Results for each hop (AUC / Macro F1)

Model 0 1 5 10 20
E1 WaveRep (released) 0.926 / 0.826 0.772 / 0.627 0.556 / 0.326 0.477 / 0.307 0.490 / 0.334
E2 D3 0.700 / 0.580 0.681 / 0.582 0.712 / 0.560 0.678 / 0.535 0.702 / 0.510
E3β€² WaveRep-DINOv3 0.936 / 0.824 0.879 / 0.770 0.766 / 0.676 0.706 / 0.566 0.683 / 0.483
Mean of z (E2, E3β€²) 0.941 / 0.839 0.894 / 0.794 0.821 / 0.727 0.764 / 0.652 0.752 / 0.584
Mean of z (E1, E2, E3β€²) 0.952 / 0.872 0.904 / 0.795 0.815 / 0.646 0.747 / 0.644 0.753 / 0.640
Old router E1–E3 0.927 / 0.815 0.820 / 0.726 0.682 / 0.582 0.665 / 0.580 0.632 / 0.549
  • Each cell is the real-vs-fake result on the G videos with that hop count. Each cell contains real and fake videos.
  • The threshold is the same at all hops.
  • The fusion degrades less than each single expert. But it still degrades: AUC goes from 0.952 to 0.753 between 0 and 20 hops.

4.2 Findings

  1. E3β€² in place of the old E3: AUC increases by 0.06–0.07 and Macro F1 by 0.07–0.08 at all hop depths.
  2. E3β€² in place of E1 is worse: router 0.884 / 0.761, mean of z 0.886 / 0.755. Keep WaveRep.
  3. Routing gives no gain over simple fusion: router 0.887, mean of z 0.894. The gates stay almost uniform (0.28–0.40).
  4. Contribution of each expert. We use a paired bootstrap on G at threshold 0, with E3β€² seed 0 / seed 1.
    • Add WaveRep to (D3, E3β€²): Ξ”AUC +0.002 / +0.006; Ξ”TPR@1%FPR +0.057 [+0.026, +0.079] / +0.089 [+0.047, +0.113].
    • Add E3β€² to (WaveRep, D3): Ξ”AUC +0.073 [+0.065, +0.080] / +0.071 [+0.063, +0.078]; Ξ”Macro F1 +0.050 / +0.059.
    • Add D3 to (WaveRep, E3β€²): Ξ”AUC +0.019 [+0.017, +0.021] / +0.018 [+0.016, +0.021].
  5. E1 is not necessary for AUC or F1. Mean of z (E2, E3β€²) gives 0.892 / 0.783. E1 adds TPR@1%FPR (+0.064) and results at 0–1 hops.
  6. WaveRep and E3β€² are complementary.
    • On GenVideo (clean video, older generators, n = 977): WaveRep 0.998 / 0.955; E3β€² 0.948 / 0.818. WaveRep wins on all 10 generators.
    • On G (modern generators, repost chains): E3β€² wins at all hops, on all platforms and on 6 of 7 generators.
    • WaveRep wins only on Relight-Video (0.664 vs 0.578). E3β€² did not see this generator in training.

5. Failure analysis: mean of z (E2, E3β€²)

The model makes 1,740 errors on G: 999 false alarms on real videos and 741 missed fakes.

5.1 False alarms on real videos (57% of errors)

Hops 0 1 5 10 20
Error rate on real videos 0.26 0.27 0.34 0.56 0.79
  • Deep re-encoding makes real videos look fake.
  • All platforms have this problem (0.36–0.44). Both real sources have it: "cuts" 30%, "crawl" 38%.

5.2 Missed fakes (43% of errors)

Generator Miss rate Part of missed fakes
SkyReels-V3 0.32 51%
Relight-Video 0.45 12%
LTX-Video 0.10 (0.19–0.25 at 5–10 hops) 24%
HunyuanVideo, Wan-2.2, Wan-Lightning ≀ 0.06 small

5.3 Which expert is wrong

Case Part of errors
Both experts are wrong (a better fusion cannot fix these) 41%
Only D3 is correct 39%
Only E3β€² is correct 20%

6. SkyReels-V3: real start, generated end

SkyReels-V3 videos in G start with real frames, and the generated part comes later. The per-frame E3β€² logits show this pattern (16 frames, uniform over the clip, 0 hops):

Frame position 0 4 8 12 15
SkyReels-V3 βˆ’3.3 βˆ’5.2 βˆ’2.7 βˆ’0.3 +1.3
Real βˆ’4.8 βˆ’5.5 βˆ’5.5 βˆ’5.5 βˆ’5.6
Wan-2.2 βˆ’3.0 +5.1 +5.4 +5.6 +5.4

The mean over all frames adds the real start to the score. Thus the score goes toward "real". We tested other temporal pooling methods on E3β€² alone, with no retraining and the same protocol:

E3β€² pooling G AUC / Macro F1 SkyReels AUC Relight AUC LTX AUC
Mean (current) 0.871 / 0.751 0.724 0.578 0.885
Maximum of sliding 4-frame windows 0.877 / 0.764 0.764 0.580 0.883
Mean of the 8 highest frames 0.876 / 0.759 0.749 0.574 0.889
Mean of the last quarter 0.880 / 0.765 0.800 0.575 0.862
  • The last-quarter mean is the best, but it is correct only for one format: a generated end. The sliding-window maximum does not use a position, so we recommend it.
  • Possible next steps:
    1. Multiple-instance scoring at inference: more frames and the maximum window score, for E3β€² and D3.
    2. Training on partly fake clips: a real start and a generated end, with a multiple-instance loss.
    3. A seam feature: a change point in the per-frame score. The late-minus-early score alone gives SkyReels AUC 0.740.

7. Studies with negative or small results

7.1 UCF-Net-style feature fusion (E2, E3β€²)

  • We adapted the fusion of UCF-Net (Zou et al., arXiv 2609.07670) to E2 and E3β€².
  • Branches: both backbones are frozen, with no LoRA.
    • E3β€² branch: the CLS token of the 25 layers, mean over 16 frames.
    • E2 branch: one temporal token per X-CLIP layer (13 layers). The token is the per-channel std over time of the frame differences.
  • LEA: a layer gate and a shared bottleneck expert (d = 256), into a common dimension of 1024.
  • UAF: branch weight = 1 βˆ’ normalized channel entropy.
  • Training: focal loss. We train on router_train (5,548 videos), with 5 seeds.
Method (E2 + E3β€²) ACC Macro F1 AUC TPR TNR TPR@1%
Mean of z (score level) 0.816 0.783 0.892 0.885 0.668 0.449
Logistic stacker (score level) 0.817 0.786 0.892 0.878 0.687 0.446
UCF: LEA + UAF 0.816 0.790 0.888 0.854 0.734 0.390
LEA + sum (average) 0.816 0.791 0.889 0.855 0.734 0.389
LEA + concatenation 0.814 0.789 0.886 0.850 0.737 0.380
LEA + cross-attention 0.816 0.793 0.891 0.844 0.756 0.408
UAF without LEA 0.798 0.769 0.873 0.848 0.693 0.386
  • UAF is equal to a plain sum (0.888 vs 0.889). This does not agree with the +2.03 mAUC gain in the paper.
  • The UAF weights stay near 0.42 / 0.58 at all hops.
  • Feature fusion does not do better than score fusion. At 20 hops it is worse (AUC 0.711 vs 0.752).

7.2 Light stability as a fusion input (for Relight-Video)

  • We used the smoothness score of relit_video_evaluation.py:
    • Signals: the mean of the bright pixels (gray > 125), the number of bright pixels, and the frame-to-frame change.
    • S = exp(βˆ’a Β· mean|Ξ”x| / range(x)). The final score is the mean of the 3 values of S.
  • Clip length is a confound:
    • On G, S gives relit-vs-real AUC 0.801. But relit videos are short: median 85 frames, real videos 301 frames.
    • Clip length alone gives AUC 0.973.
    • Against real videos of the same length, S gives only 0.598.
  • The direction is the opposite of the hypothesis:
    • We controlled the length: 24 fps, 64-frame windows, mean over the windows.
    • With this control, relit videos have more stable light than real videos (AUC 0.42 with "low stability = fake").
  • In-G 7:3 split (uses G labels):
    • We split the 173 relit source videos by source video and repeated the split 20 times.
    • We added the windowed stability to the stacker.
    • Relit AUC goes from 0.717 to 0.744, and it increases in 18 of 20 splits. All-G AUC goes from 0.895 to 0.902.
    • The original metric does not help (relit AUC 0.701).
    • Clip length alone gives relit AUC 0.855. This is a shortcut in the benchmark.
  • Data limit: all relit videos on the disk are in G. A correct test needs new relit videos from expert_train sources, with the same length and fps as the real videos.

7.3 Degradation descriptor (contribution 2 of the plan)

  • The descriptor is d = [crf_hat, log motion, log Laplacian variance, log height].

  • crf_hat follows the hop depth on G (Spearman 0.66) and does not show the label:

    Hops 0 1 5 10 20
    Median crf_hat, real 0.93 0.98 2.23 3.34 4.66
    Median crf_hat, fake 0.62 0.80 2.15 3.25 4.73
  • We compared the stacker with and without d. We used the same protocol and a paired bootstrap.

Experts Stacker input Ξ”AUC [95% CI] Ξ”Macro F1 [95% CI]
E1, E2, E3β€² + crf_hat +0.001 [+0.001, +0.002] βˆ’0.005 [βˆ’0.009, βˆ’0.000]
E1, E2, E3β€² + crf_hat + zΒ·crf_hat +0.001 [βˆ’0.001, +0.002] βˆ’0.003 [βˆ’0.007, +0.002]
E1, E2, E3β€² + all 4 descriptors βˆ’0.009 [βˆ’0.011, βˆ’0.007] βˆ’0.039 [βˆ’0.046, βˆ’0.033]
E1, E2, E3β€² + all 4 descriptors + interactions βˆ’0.010 [βˆ’0.012, βˆ’0.006] βˆ’0.021 [βˆ’0.027, βˆ’0.015]
E2, E3β€² + crf_hat βˆ’0.001 [βˆ’0.001, βˆ’0.000] βˆ’0.003 [βˆ’0.005, +0.000]
E2, E3β€² + all 4 descriptors + interactions βˆ’0.011 [βˆ’0.014, βˆ’0.008] βˆ’0.012 [βˆ’0.017, βˆ’0.005]
  • The descriptor does not improve the ranking. It only moves the balance between false alarms and misses.
  • Example (all 4 descriptors, E1, E2, E3β€²):
    • TNR at 20 hops goes from 0.335 to 0.742.
    • But all-G TPR goes from 0.818 to 0.688.
  • Conclusion: the current results do not support degradation-aware routing. The router and the condition-aware logistic (0.884) agree with this.

7.5 Repost-chain compression: layer probes and a real-reference score (added 2026-10-21)

Question: after deep compression, do real and fake videos still have different distributions in the feature space? Data: the stored frozen features in wr3/ucf_feat (16 frames for each video, router and G). No G labels are fitted.

(a) Linear probe on each layer (router_train; C selected on router_val; Youden threshold on calib_select).

Token G AUC / Macro F1 Hop 0 AUC Hop 20 AUC Hop 20 TNR
E3β€² last layer (pooler, layer 24) 0.868 / 0.763 0.932 0.681 0.245
E3β€² middle layers (15–17) 0.84 / 0.73 0.93 0.61–0.63 0.64–0.72
E2 X-CLIP layer 0, temporal token (std of frame differences) 0.673 / 0.504 0.684 0.673 0.761
  • Our hypothesis was that the middle layers keep more separation. This is not correct for E3β€². The last layer is the best at all hops.
  • The low-level temporal token of E2 does not change with compression (0.684 β†’ 0.673), but it is weak.
  • Main finding: at 20 hops, E3β€² still ranks the videos (AUC 0.68), but its TNR is only 0.08. Most of the failure is a shift of all scores toward "fake". The separation is not fully lost.

(b) Distance to real videos at the same degradation level.

  • The score is the Mahalanobis distance (Ledoit-Wolf, PCA to 64 dimensions) from the E3β€² pooler token to the router real videos (router_train + calib_fit) in the same crf_hat quantile bin. It does not use fake videos.
  • Platt on calib_fit. Paired bootstrap (1,000 resamples, grouped by source video) against the mean of z (E2, E3β€²).
Model (G) AUC / Macro F1 Hop 0 AUC / F1 Hop 20 AUC / F1 Hop 20 TNR Ξ”AUC vs mean (E2, E3β€²) [95% CI]
E1 WaveRep (released, DINOv2-B) 0.749 / 0.563 0.926 / 0.826 0.490 / 0.334 not calculated not calculated
E3β€² alone 0.871 / 0.751 0.936 / 0.824 0.683 / 0.483 0.084 βˆ’0.020 [βˆ’0.023, βˆ’0.017]
Real reference, 1 bin 0.858 / 0.751 0.920 / 0.831 0.733 / 0.651 0.452 βˆ’0.034 [βˆ’0.038, βˆ’0.029]
Real reference, 5 crf_hat bins 0.860 / 0.755 0.925 / 0.834 0.743 / 0.653 0.645 βˆ’0.032 [βˆ’0.037, βˆ’0.027]
Mean of z (E2, E3β€²) 0.892 / 0.783 0.941 / 0.839 0.752 / 0.584 0.213 0
Mean of z (E2, real reference 5 bins) 0.866 / 0.756 0.923 / 0.816 0.779 / 0.685 0.626 βˆ’0.026 [βˆ’0.030, βˆ’0.021]
Mean of z (E2, E3β€², real reference 5 bins) 0.888 / 0.788 0.938 / 0.856 0.760 / 0.668 0.400 βˆ’0.004 [βˆ’0.006, βˆ’0.001]
  • The real-reference score is better than E3β€² at deep hops (hop 20: AUC +0.06, F1 +0.17). It is a little worse at hop 0.
  • The crf_hat bins raise the TNR at 20 hops from 0.45 to 0.65, but they do not change the AUC. Thus the degradation descriptor helps the calibration, not the ranking (this agrees with Β§7.3).
  • In the 3-expert mean, Macro F1 increases (0.783 β†’ 0.788; hop 20: 0.584 β†’ 0.668), but AUC decreases by 0.004. There is no overall AUC gain.
  • Conclusion: after compression, real and fake videos are still separable. The detector features are not invariant to compression, so all scores move toward "fake". The next step is to train for this invariance (Β§10, item 1).

7.4 Older negative results

  • The old E3 does not improve the ranking of (E1, E2): Ξ”AUC 0.000. It moves the threshold: F1 +0.021 [0.013, 0.030], TNR βˆ’0.16, TPR@1% βˆ’0.053. It calls G real videos fake at all hops (TNR 0.17–0.31).
  • The router E1–E3 with seed 0 gives 0.816 AUC. The mean of 5 seeds is 0.798 Β± 0.015. A router that knows the true hop count gives 0.811.
  • Label-free accuracy estimation (SML, GNB-EM) gives no gain. A self-probe by re-encoding fails, because the compression does not change after one re-encode.
  • E3 v2 (fixed 125 ms step): 0.594 vs 0.606. Frame rate is not a shortcut in G (AUC 0.50).
  • Training-free trajectory statistics (dense DINOv3, Wan2.1 and CogVideoX VAE latents): best G AUC 0.703. They give no fusion gain.
  • Other frozen E3 candidates (linear probe, CogVideoX-only training data): SigLIP 2 0.733 / 0.612, DINOv3 RGB 0.677 / 0.591, V-JEPA 2 0.598 / 0.516. None of them improves the (WaveRep, D3) fusion.

8. WaveRep re-implementation (wr3): development of E3β€²

  • The official training code of WaveRep is not available. We re-implemented the recipe of the paper (Corvi et al., NeurIPS 2025, arXiv 2506.16802).
  • Model and loss: ViT and a linear head, trained end to end, with BCE on random frame crops and balanced batches.
  • WaveRep augmentation:
    • p = 0.1 on fake crops; 3-level Haar fully separable wavelet transform.
    • The baseband and the horizontal and vertical bands come from a real crop. The 9 diagonal bands stay from the fake.
  • Other augmentation:
    • JPEG and blur, p = 0.1 each.
    • Resize of both classes to the same short-side range.
  • Training settings: Adam, lr 1e-5 (cosine), weight decay 1e-6, batch 32, 20k iterations, crop 448. We select the checkpoint by the validation AUC.
  • Re-encode augmentation (--reenc): p = 0.3, 1–3 H.264 passes (CRF 23–35, scale 0.6–1.0) on a 16-frame window.

gen4 training fakes:

  • Input: the first frame of a real expert_train source video and a Qwen2.5-VL-7B caption. Each fake uses a different source video.
  • Generators:
    • Wan2.2-TI2V-5B (704Γ—1056, 40 steps).
    • HunyuanVideo-1.5 480p I2V step-distilled (12 steps).
    • LTX-2 19B I2V with the distilled LoRA (8 steps).
    • SkyReels-V3 R2V-14B (540P, 8 steps).
  • Encoding: H.264, CRF 18–28, 24 fps.
  • 2 s set: 49 frames, 300 for each generator. 5 s set: 121 frames, 300 for each generator.
  • Real videos: expert_train ActivityNet and InternVid (49 frames at 8 fps).
  • Data of E3β€² (v3_5s300): train 4,498 real and 1,086 fake; validation 456 real and 114 fake.
Run (G, Youden threshold on calib_select) ACC Macro F1 AUC TPR TNR TPR@1% GenVideo AUC
WaveRep released (DINOv2-B) 0.565 0.563 0.749 0.364 0.995 0.398 0.998
DINOv2-B, 2 s 0.747 0.687 0.811 0.868 0.486 0.284 0.715
DINOv3-L, 2 s 0.753 0.711 0.822 0.829 0.588 0.369 0.902
DINOv3-L, 2 s + re-encode 0.770 0.723 0.846 0.867 0.562 0.358 0.912
DINOv2-B, 5 s + re-encode (seed 0 / 1) 0.749 / 0.762 0.663 / 0.704 0.830 / 0.837 0.919 / 0.884 0.384 / 0.501 0.303 / 0.322 0.832 / 0.869
DINOv2-L, 5 s + re-encode 0.753 0.674 0.832 0.912 0.412 0.316 0.795
DINOv3-L, 5 s + re-encode (seed 0 / 1) = E3β€² 0.796 / 0.793 0.756 / 0.748 0.871 / 0.865 0.881 / 0.892 0.613 / 0.582 0.432 / 0.402 0.948 / 0.944
  • DINOv3 is better than DINOv2 also at the same model size (DINOv2-L 0.832, DINOv2-B 0.834). The gain comes from the pre-training, not from the size.
  • 2 s vs 5 s (100 fakes per generator, 2 seeds each): G AUC 0.802 vs 0.825; Macro F1 0.706 vs 0.712. The source videos are different in the two sets.
  • More fakes help more than longer clips (2 s: 100 per generator 0.802; 300 per generator 0.846).

8.1 Ablation of the WaveRep wavelet mix in E3β€² (added 2026-10-25; G, seed 0 for each variant)

E3β€² variant (DINOv3-L, 5 s, re-encode) Share of a mixed crop from the real crop G AUC / Macro F1 G 0 re-encodes G 20 re-encodes (AUC) GenVideo AUC / F1
3-level Haar, p = 0.1 (E3β€²) 23% 0.871 / 0.756 0.936 / 0.831 0.683 0.948 / 0.818
No wavelet mix (p = 0) 0 0.829 / 0.677 0.912 / 0.735 0.606 0.920 / 0.741
1-level Haar 75% 0.865 / 0.708 0.933 / 0.767 0.645 0.945 / 0.770
2-level Haar 44% 0.860 / 0.711 0.928 / 0.762 0.673 0.958 / 0.806
3-level Haar, p = 0.3 23% 0.845 / 0.666 0.926 / 0.724 0.658 0.953 / 0.735
DINOv2-B retrain (3-level, p = 0.1) 23% 0.830 / 0.663 0.904 / 0.719 0.600 0.832 / 0.611
WaveRep released (DINOv2-B) – 0.749 / 0.563 0.926 / 0.772 0.490 0.998
  • The wavelet mix is necessary: βˆ’0.042 AUC and βˆ’0.079 Macro F1 without it. This is about 7 times the seed difference of E3β€² (0.006).
  • The setting of the paper (3 levels, p = 0.1) is the best on G. More mixing (p = 0.3) is worse.
  • Without the mix, DINOv3-L (0.829) is at the level of the DINOv2-B retrain with the mix (0.830). The backbone and the mix both give a gain.
  • Limit: one seed for each variant. The 1- and 2-level variants are within 2 seed differences of E3β€².
  • Script: wr3_train.py --wr_p --wr_levels; data csv/wr3_eval_G_wrablation.csv.

9. Known problems and limits

  • D3 prediction bug in the benchmark CSVs:
    • tools/build_setting_{F,G}_per_video.py swaps the predictions when they flip the score sign. Thus pred_D3 is inverted on the rows with 5, 10 and 20 hops in F and G.
    • The correct rule: fake if score_D3 β‰₯ βˆ’5. Corrected D3 on G: 0.686 / 0.646 / 0.695 / 0.750 / 0.550 / 0.013.
    • All MoE scripts use the correct rule. The fix of the project CSVs needs approval.
  • Generator overlap:
    • gen4 uses the Wan2.2, HunyuanVideo, LTX and SkyReels families. G also has these families. Thus G is not an unseen-generator test for E3β€².
    • Evidence on unseen generators: Relight-Video (E3β€² weak, 0.54–0.58) and GenVideo.
  • Shortcut in G: Relight-Video clips are much shorter than real clips (clip length alone gives AUC 0.97). G should use clips of the same length, or the paper must report this limit.
  • WaveRep training generators (checked 2026-10-25): the released G4 checkpoint is trained on Pyramid Flow, CogVideoX-1.5, Allegro and OpenSora-Plan, with Panda70M real videos (Corvi et al., arXiv 2506.16802). Our GenVideo test set has "COG" (same family as CogVideoX) and "OpenSora" (HPC-AI, a different project from PKU OpenSora-Plan). The CogVideo family overlap partly explains the released WaveRep GenVideo AUC of 0.998.
  • Evidence limits: one test set; E3β€² results from 2 seeds; no confidence intervals yet for router vs mean of z vs stacker with E3β€².

10. Next steps (status 2026-10-26; target = mix_setting_v1)

# Experiment Status
1 Compression-invariant E3β€²: paired clean / re-encoded views, BCE on both views + consistency loss (wr3_train_pair.py, Ξ» = 1 and the Ξ» = 0 control) Done: negative (Β§0.3)
2 Leave-one-generator-out E3β€² (without SkyReels-V3, LTX-2, HunyuanVideo-1.5 or Wan2.2) Done: each generator gives a small part; no single one is necessary (Β§0.3)
3 Video extension: real extension clips from current generators (not SkyReels-V3), then the change-point transformer (Design 3) Deferred (user, 2026-10-26); the feature-splice pilot is negative (Β§0.2). The 61 pending SkyReels-V3 extension slots of mix are also deferred
4 Relit training videos with the same length and fps as the real videos Not run (user, 2026-10-26: stop here; MoE study ends with mean of z (E2, E3β€²))
5 Score E1 (released WaveRep) on mix_setting_v1 Done: no gain (Β§0.3)
6 Augmentation study (VAE-reconstruction fakes, 12-profile chains) Running in a separate session (claude_workspace/aug_study/STATUS.md)
7 Re-encode-aware gate / stacker on (E2, E3β€²) with crf_hat on mix Done: no gain (Β§0.3)
8 UCF-Net-style feature fusion (E2, E3β€²) on mix Done: equal to score fusion (Β§0.3)

11. Update 2026-11-02: LevelNet and the data flywheel for the pixel expert

The full record is in AUG_STUDY_LOG.md Sections 19–23. Short summary (mix_setting_v1, n = 9,939, fused with E2; AUC / Macro F1):

Study Result (E2, model) on mix
LevelNet (detail bands per SWT level; PREREG_LV) Proxy rule not met; full run LVBN is descriptive LVBN 0.829 / 0.727
FW4: one flywheel round, 70% base Round 1 passes (mixval deep +0.015 vs random selection) M_1β€² 0.833 / 0.729; C-randβ€² 0.828 / 0.728
FW5: multi-round, 30% base, new Kinetics + gen4 data on demand Round 1 fails (error-directed = random); loop stops M_0 0.808 / 0.709; C-all 0.829 / 0.729
FW6: depth-directed loop (probe at 1–20 re-encodes, deep training units) Preregistered round 1 fails narrowly; exploratory rounds 2–3: error-directed > uniform on mixval in every round M_3 0.819 / 0.719 = C-rand_3 0.820 / 0.721
  • The loop improves mixval strongly (deep AUC 0.750 β†’ 0.815), but the gain does not transfer to mix_setting_v1.
  • The best system on mix stays (E2, F3_s0) 0.849 / 0.739 (augmentation study, AUG_STUDY_LOG.md Section 13).

12. Update 2026-11-02 evening: FW7 and the model-size control

The full record is in AUG_STUDY_LOG.md Section 25. Short summary (fused with E2; AUC / Macro F1):

Study Result
FW6-X X4 (round 2, 2 seeds) mixval: pass (+0.019 vs C-rand); mix: M_2 0.813 / 0.711 = C-rand_2 0.812 / 0.710
FW7 pilots The mix normalize step is not the cause (Ξ”AUC ≀ 0.02); new high-quality Kinetics reals give many false positives (48% vs 27% for the probe reals)
FW7 round 4 (real-source-directed loop) PASS on srcval: S_src M_4 0.890 vs C-rand_4 0.884 (+0.0065, CI low +0.0031) vs M_3 0.846. Round 5 is running; mix scores follow
F3B (F3 recipe on ViT-B/16, 2 seeds) mix 0.833 / 0.728 and 0.828 / 0.718 vs F3_s0 (ViT-L/16) 0.849 / 0.739 and LVBN 0.829 / 0.727: the backbone size is the main factor; LevelConcat = full image
  • The best system on mix stays (E2, F3_s0) 0.849 / 0.739. A next loop must start from a ViT-L model.

13. Update 2026-11-03: FW7 and FW8 results on mix

Full record: AUG_STUDY_LOG.md Section 26 (fused with E2; AUC / Macro F1).

Study Validation decision mix_setting_v1
FW7 (ViT-B, real-source-directed, rounds 4–5) Both rounds pass on srcval (S_src 0.846 β†’ 0.908) M_5 0.823 / 0.723 vs start 0.819 / 0.719 and C-rand_5 0.826 / 0.721
FW8 (same loop on F3_s0, ViT-L) Round 1 fails narrowly (+0.0047 vs C-rand, need +0.005) M_1 0.850 / 0.735 vs F3_s0 0.849 / 0.739
  • No flywheel variant (FW4–FW8) transfers its validation gain to mix_setting_v1. The best system stays (E2, F3_s0) 0.849 / 0.739.

14. Update 2026-11-04: error-directed fakes, FW7 round 6

Full record: AUG_STUDY_LOG.md Section 27 (fused with E2; AUC / Macro F1). Since 2026-11-03, new training uses only LevelConcat and level-attention. F3_s0 and LVBN are reference rows.

Study Validation decision mix_setting_v1
FW7 round 6 (LevelConcat ViT-B; new SkyReels-V3 extensions and Relight fakes from Kinetics) Passes: S_hold 0.752 vs C-real 0.568 (CI low +0.158), no harm M_6 0.837 / 0.737 vs M_5 0.823 / 0.723 and C-real_6 0.825 / 0.721
  • This is the first flywheel round with a mix gain over its controls: +0.014 AUC [+0.011, +0.017] against C-real_6, for both seeds.
  • The gain comes from Relight: the mix miss rate goes from 0.56 to 0.18. The SkyReels-V3 miss rate goes from 0.54 to 0.49.
  • (E2, M6_s0) is still 0.012 AUC below (E2, F3_s0) 0.849 / 0.739, and the Macro F1 is equal. The DINOv2 rows are (E2, D2wr) 0.795 / 0.696 and (E2, E1) 0.759 / 0.583.
  • Round 7 (200 more SkyReels-V3 extensions) is pre-registered and paused at 114 / 200 clips, because of a GPU driver mismatch on the server.

15. Update 2026-11-13: QL data flywheel (rounds 1–7, controls, external sets)

QL = LevelNet --head xattn --rgb 4 on one DINOv3 ViT-B/16 (query = LL3, keys = Haar SWT bands). Test set = mix_setting_v1 (RPG, n = 9,939). Protocol: logistic calibration on router calib_fit, Youden J on router calib_select, 1,000 resamples grouped by source video, labels Β±0.005 with the 95% CI. Main value = QL alone; (E2, QL) next to it. AUC / Macro F1. Records: aug_study/levelnet/PREREG_ARCH_FW.md (rounds 1–5), PREREG_AFW2.md (rounds 6+), PREREG_AFW2X.md (external sets), PREREG_AFW2S.md (same-start selection test).

Chains. Mk = data selected by model errors (rounds 1–5: LevelConcat errors; rounds 6+: QL's own errors on the FW6 depth probe, 600 re-encoded deep units of 4 s per round). C-rand (random fakes) = same budget, random / uniform selection (from round 3). C-real (rounds 3–5) = new reals only. C-steps = same training steps, no new data.

mix_setting_v1, QL alone AUC / Macro F1 (E2, QL) AUC
Round 0 (AR2_QL_s0) 0.757 / 0.650 –
Mk round 5 0.815 / 0.713 0.849
Mk round 6 (stop round K = 6) 0.819 / 0.717 0.850
Mk round 7 (confirmation, report only) 0.821 / 0.717 –
C-rand (random fakes) round 6 0.773 / 0.672 0.822
C-real round 5 0.773 / 0.663 –
C-steps round 6 0.744 / 0.640 0.794
F3_s0 (ViT-L, reference) 0.821 / 0.709 –
dinov2_wr / released WaveRep (reference) 0.737 / 0.638, 0.637 / 0.495 –

Pre-registered reading (PREREG_AFW2): NOT convincing.

  • (a) Plateau: pass. Mk 5β†’6 +0.004 (one-third RPG +0.005 [0.001, 0.009]), 6β†’7 +0.003. Both Plateau.
  • (b) Mk better than both controls at K = 6: pass. +0.045 vs C-rand (random fakes), +0.075 vs C-steps.
  • (c) Mk 5β†’6 gain larger than C-rand 5β†’6 gain: fail. βˆ’0.002 [βˆ’0.005, 0.001], Equal.
  • Post hoc (not pre-registered): (Mk 3β†’6) βˆ’ (C-rand 3β†’6) = +0.034 [0.030, 0.039]. The gain over the control builds in rounds 3–5.

External sets (PREREG_AFW2X): the Mk advantage holds. QL alone, round 6.

Set Mk C-rand (random fakes) C-steps Mk βˆ’ C-rand Mk βˆ’ C-steps
AIGVDBench (2,000) 0.894 / 0.817 0.802 / 0.752 0.746 / 0.701 +0.091 Better +0.147 Better
r5–r30 pooled (5,000) 0.701 / 0.635 0.579 / 0.559 0.579 / 0.560 +0.122 Better +0.122 Better
GenVideo (report only) 0.953 / 0.891 0.904 / 0.829 0.834 / 0.810 +0.049 +0.120
  • r5–r30 use RPG source videos with new re-encodes; only AIGVDBench is fully outside RPG.

Same-start selection test (PREREG_AFW2S, 2 seeds). From Mk round 5: error-selected vs uniform round-6 units.

  • Sel βˆ’ Uni on mix: seed 0 +0.004, seed 1 βˆ’0.002. Both Equal; stable across seeds.
  • All four runs (0.815–0.819) equal the start model Mk round 5 (0.815).
  • External Sel βˆ’ Uni changes sign between seeds (about Β±0.015): training-seed noise.

Conclusion. Error-selected data gives a large gain over the control chains, on RPG and outside RPG. The gain builds in rounds 1–5 and then reaches a plateau at round 6. At the plateau, error selection equals uniform selection. One seed for the chains (two for the same-start test).

Running (report only, user request 2026-11-13): C-rand (random fakes) and C-steps rounds 7–10, Mk rounds 8–10, on GPUs 0–2. Figure: aug_study/levelnet/arch_fw/ql_flywheel.png (one-third RPG, all rounds).

16. Update 2026-11-15: MoE table, fusion and frame-selection tests, temporal keys, second-detector flywheels, last-checkpoint rule

Test set = mix_setting_v1 (RPG, n = 9,939) unless the text says "prop33" (the proportional one-third subset, n = 3,277, csv/rpg_prop33.csv; progress and stop decisions only). Protocol as in Section 15. AUC / Macro F1; Β± and [ ] = 95% CI from 1,000 resamples grouped by source video. Each detector is reported alone unless the row says "with D3".

16.1 Main MoE result with QL round 7 (misc/moe3/moe3_full.md)

QL = AFW_QL_Mk_r7 (Section 15). Pairs tested on prop33: (QL, D3) 0.851 / 0.749 is better than (QL, STALL) and (QL, SPLIT) (both 0.821, +0.030). STALL and SPLIT add nothing to QL.

Full RPG AUC Macro F1 TPR@1%FPR
Spatial-frequency expert (QL r7) 0.821 Β± 0.008 0.717 Β± 0.009 0.293 Β± 0.035
Temporal expert (D3) 0.693 Β± 0.011 0.554 Β± 0.010 0.013 Β± 0.005
MoE (QL, D3) 0.852 Β± 0.007 0.742 Β± 0.009 0.290 Β± 0.034
  • GenVideo test split: (QL, D3) 0.986 / 0.918, TPR@1%FPR 0.880. AIGVDBench closed-source sample (2,000 reals + 2,000 fakes of 11 generators, csv/aigvdb_cs_manifest.csv): 0.965 / 0.818, TPR@1%FPR 0.766.
  • Baselines run a second time on full RPG (misc/rpg2/rpg2_table.tex): 9 of 11 give the same scores (r β‰₯ 0.9999). Only SPLIT (0.568 β†’ 0.577) and STALL (0.517 β†’ 0.516) change (seeded sampling).

16.2 Round-0 ablation of QL (full RPG, seed 0; csv/arch_r0_tpr1.csv, PREREG_QLD0.md)

Variant Alone: AUC / Macro F1 / TPR@1%FPR With D3: AUC / Macro F1 / TPR@1%FPR
QL_s0 (full model) 0.757 / 0.650 / 0.192 0.803 / 0.710 / 0.191
w/o wavelet bands (R1_s0) 0.770 / 0.661 / 0.234 0.812 / 0.710 / 0.238
w/o content queries (X4_s0) 0.769 / 0.665 / 0.223 0.807 / 0.705 / 0.237
w/o fine-level dropout (AR3_QLD0_s0) running running
  • At round 0, both finished ablations are higher than the full model. The round-0 table thus does not support these two components. The main-table model is QL after the flywheel rounds, not round 0.
  • AR3_QLD0_s0 = AR2_QL_s0 with --p_drop 0 (pre-registered reading: "fine-level dropout helps" only if AUC with D3 is Worse). On the in-distribution validation set, AUC at 5 re-encodes is 0.005–0.028 lower than QL at all 16 points to iteration 16,000; AUC at 0 re-encodes is equal. The RPG result is not available yet.

16.3 Fusion tests: the mean of calibrated log-odds stays (PREREG_FUS.md, PREREG_RES.md; prop33, QL = AFW_QL_Mk_r6)

Reference: QL 0.819 / 0.721, D3 0.701 / 0.557, mean 0.849 / 0.754.

Idea Result Decision
Detail-survival gate (weight by an estimate of re-encodes) βˆ’0.004 Equal; even the best weight per re-encode count, fitted on the subset, gives βˆ’0.0001 STOP
Second-order dynamics of QL frame logits +0.000 over QL; βˆ’0.030 Worse as a D3 replacement STOP
Residual expert (train QL on BCE(logit + calibrated D3 log-odds)) 0.837 / 0.740 = βˆ’0.012 Worse STOP
UCF-Net uncertainty weight at the score level 0.842 / 0.757 = βˆ’0.007 Worse; the learned version returns to the plain mean STOP
  • Useful for the paper: mean βˆ’ QL alone = +0.008 / +0.051 / +0.056 / +0.046 at 1 / 5 / 10 / 20 re-encodes. D3 helps most after re-encodes.

16.4 D3 as a frame sampler (PREREG_D3SEL.md, PREREG_D3Q.md): STOP

  • LevelConcat M8, exact decode (src/exact_decode.py; decord gives wrong frames on 4–7% of RPG videos): Uniform-32 = Uniform-16 (0.815 vs 0.813). Uniform-32 βˆ’ random window-32 = +0.015 Better (coverage matters, not frame count). D3-frame-32 βˆ’ random-frame-32 = +0.001 Equal.
  • QL r6 with equal coverage (16 time strata; D3 frame vs random frame per stratum): +0.001 Equal; D3-weighted mean βˆ’0.003 Equal (SkyReels-V3 Worse).

16.5 Temporal keys (QLT; PREREG_QLT.md, PREREG_QLT_FB.md): negative

  • QLT adds temporal-difference keys to QL. Stage 1, prop33: QLT 0.640 / 0.572 vs QL 0.751 / 0.647 alone; 0.737 vs 0.800 with D3. All hypotheses Worse.
  • Stage 2, own-error feedback from a held-out pool of old flywheel videos (26,306 units; top 1,500 loss of a 6,000 sample per round), 5 rounds: Mk 0.640 β†’ 0.613 (Decrease); C-rand (random fakes) 0.640 β†’ 0.636; Mk_5 βˆ’ C-rand_5 = βˆ’0.023 Worse.

16.6 Flywheel on second detectors (5-round layout; full RPG, alone)

Detector (record) Mk C-rand (random fakes) C-steps Reading
FreqNet own loop, r7 (PREREG_FL.md) 0.596 / 0.529 0.603 / 0.515 0.575 / 0.549 Fails (Worse than C-rand βˆ’0.007)
VideoFACT own loop, r4 (PREREG_VFL.md) 0.622 / 0.569 0.628 / 0.578 – Worse than C-rand (βˆ’0.006)
VideoFACT round 9 (PREREG_VF9.md) 0.690 / 0.596 0.672 / 0.582 0.629 / 0.573 Gaps Better, but Mk 8β†’9 Plateau: fails
GenD r8, lr 1e-4 (PREREG_GD.md) 0.591 / 0.510 0.594 / 0.499 0.585 / 0.508 Mk = C-rand; all chains still rise
UnivFD fc-only r5 (PREREG_UFL.md) 0.679 / 0.602 0.690 / 0.620 0.666 / 0.599 Worse than C-rand (βˆ’0.011)
VideoFACT feedback from the QLT pool, K = 3 (PREREG_VFFB.md) 0.662 / 0.588 0.650 / 0.587 0.604 / 0.563 Helps: +0.012 vs C-rand, +0.058 vs C-steps (both Better)
  • VFFB is the first own-error loop on a second detector that passes. But selection adds only +0.012; coverage of the pool adds +0.055 (C-rand βˆ’ C-steps). Round 0 = FW7_VFL1_s0 0.595 / 0.530.

16.7 VideoFACT feedback extension (PREREG_VFFB2.md; running)

Rounds 6–12 from the round-5 models; stop after 2 extension rounds in a row without Improve on prop33. On 2026-11-15 the user replaced C-rand with C-match (Mk's type Γ— label counts per round, random units inside each cell). C-rand stopped at round 7 (report only).

prop33, alone Round 5 Round 7 Round 9 Round 10 Round 11
Mk (own errors) 0.672 / 0.585 0.690 / 0.609 Improve 0.702 / 0.586 Improve (K2 = 9) 0.698 / 0.611 Plateau 0.692 / 0.573 Inconclusive (βˆ’0.006)
C-match (Mk's mix) 0.679 / 0.597 running
C-steps (base set only) 0.589 / 0.544 0.594 / 0.548 0.617 / 0.569 0.603 / 0.539 running
  • Mk stopped at round 11 (N2 = 11, K2 = 9). The controls run to round 11; then full-RPG scoring and the pre-registered reading.
  • C-match equals or beats Mk at rounds 1–5 (round 4: 0.683 vs 0.675). This suggests that most of the gain comes from the type mix of the pool, not from the selection of single units. The reading at K2 will test this.
  • Figure: image/vffb_chains_prop33_live.png.

16.8 Last-checkpoint rule for chain rounds (user decision 2026-11-15)

  • Problem: VideoFACT chains move up and down from round to round (C-steps spread Β±0.012 prop33 AUC; QL Β±0.003). The best-validation checkpoint was picked on an in-distribution validation set (AUC about 0.90) that does not follow RPG. Mk kept non-final weights in 5 of 9 rounds (iterations 500, 1,000, 2,000, 2,000, 3,000), near the peak of the restarted learning-rate schedule. QL validated only at 2,500 and 5,000 and kept the final weights in 28 of 33 runs.
  • Rule: from round 10, all chains keep the final weights (train_vfact_fw6.py --keep last; the validation-best weights are kept as best_val.pt). Rounds 1–9 keep the old rule in all chains.
  • First data: Mk had no Decrease label in rounds 10–11 (Decrease at rounds 2 and 6 before). Two rounds are not sufficient to judge the change.
  • Mk_9 has a higher AUC but a lower Macro F1 than Mk_8. The Youden threshold from router calib_select is low on RPG: Mk_9 calls 79% of videos fake (TNR 0.322). At the best test threshold (report only), Macro F1 is 0.627 vs 0.613.

16.9 Other results

  • QL chains, rounds 7–9 (report only): Mk 0.822 (r7), 0.821 (r8) prop33, Plateau. C-steps 0.759 (r7), 0.749 (r8). The chains are paused before round 10 (user decision).
  • GenVideo re-encode test (200 GenVideo reals vs 800 fakes, 0/1/5/10/20 re-encodes): ours 0.962 β†’ 0.799, SPLIT 0.988 β†’ 0.979, D3 0.812 β†’ 0.778, WaveRep 0.824 β†’ 0.369. High AUCs at 0 re-encodes come partly from a format shortcut (reals 3 fps, 298Γ—224).
  • Trace by level (aug_study/fig2_trace_by_level/report.md): the absolute trace in b1–b3 rises with re-encodes after one re-encode (unsharp masks in the TikTok / RedNote profiles amplify high frequencies). The claim "the finest level loses its trace fastest" is not supported.

16.10 Open items

  • AR3_QLD0_s0 RPG row (w/o fine-level dropout).
  • PREREG_VFFB2: C-match rounds 6–11 and C-steps round 11, then full RPG and the reading at K2 = 9 and N2 = 11.
  • QL chains round 10: waits for user approval.