Commit History
-
Add CMS foundation: persistent outcome logging, video sample capture, DB + browsing UI
silvanus930- Compression service now writes a structured outcome record (codec, cq, vmaf, ratio, timing, searched-vs-fallback) per completed item to compression_outcomes.jsonl, following the same pattern as the existing cq_search_log.jsonl. This doesn't touch the CompressResponse contract with neurons/miner.py -- the CMS reads this log directly instead. - _search_cq_for_max_score now returns (cq, vmaf) instead of just cq, so the real measured VMAF for the chosen value can be logged instead of re-measuring or guessing. - neurons/miner.py captures a compressed-output sample alongside the existing reference-clip capture, using the same category-aware eviction policy (reserve=3 per codec/threshold, cap=80) -- generalized the reference-library eviction helpers to serve both libraries, since compressed outputs are normally deleted within seconds of upload and would otherwise never be available to preview. - New cms/ subsystem: SQLite-backed ingest of the outcome log, a FastAPI s…b5c3309 -
Add safety guards to secant CQ search; revert content-adaptive margin attempt
silvanus930- Guard against 1-cq baseline slope estimates producing wild extrapolated jumps (observed live: a noisy 1-cq gap slope extrapolated a probe 10 cq away, wasting the time budget) -- only trust the measured local slope with a >=2-cq baseline, fall back to the pooled prior otherwise. Cap jump2's extrapolated distance as defense in depth. - Attempted making the search margin content-adaptive (shrinking it on demonstrably gentle-slope clips, backed by paired subsampled-vs-full- precision VMAF measurements showing noise is content-dependent) but reverted: the first jump's direction is decided against the inflated margin-threshold, so on easy content it's already biased conservative before there's evidence the content is easy, undermining the adaptation before it can help. Needs the first-jump direction logic reworked too -- left as a scoped future project rather than iterating further live.8089b41 -
Session: AV1/HEVC CQ table fixes, VBR NVENC tuning, secant-based adaptive search, dashboard reset mechanism
silvanus930- Fix HEVC CRF Low (40->35) and Medium (35->30) tiers, both were landing at/below the hard-cutoff on real traffic, scoring near-zero - Add AV1 Low-tier size-based CQ bonus using reference file size as a free content-difficulty signal - Add NVENC advanced rate-control flags (multipass/lookahead/AQ) to VBR-mode encoding, A/B tested against real HEVC threshold-89 traffic - Replace blind +/-3 coordinate-ascent CQ search with a secant-method search that jumps directly toward the VMAF target using measured local slope - Add dynamic per-request deadline computation and download-source/speed instrumentation to neurons/miner.py - Fix cleanup-worker silently deleting calibration data via a protected _persistent/ subdirectory - Add dashboard batch-counter reset mechanism - Add real-validator-score-test harness (calls the literal validator scoring code locally) and CQ-curve calibration scripts for evidence-based tuningbc6f62c
-
Score against VMAF NEG model, switch AV1 to SVT-AV1, add probe timeouts
silvanus930 +1: Claude Sonnet 5- CQ search now measures against vmaf_v0.6.1neg (what the validator actually scores with) instead of the standard model, which read ~2.2-2.5 points optimistic and understated the hard-fail margin. - Default AV1 encoder to software SVT-AV1 over NVENC: avg score 0.2185 -> 0.2411 across a 22-video sweep (17 better/5 worse/0 tied), still well inside the search/validator time budgets. - Each ffmpeg/vmaf subprocess now has its own timeout so a single hung call can't block a request past CQ_SEARCH_MAX_SECONDS. - tracker_loop.sh now restarts watchdog.sh if it dies, since nothing else supervises the supervisor. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>16e3bfe -
Add adaptive VMAF-driven CQ search for compression miner, plus ops scripts
silvanus930 +1: Claude Sonnet 5Probe-encode a handful of CQ values against real libvmaf output and pick the one that maximizes the validator's compression score formula, instead of trusting the static per-band CQ table. Adds libvmaf to the compression Dockerfile and dashboard/tracker/watchdog helper scripts. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>eaba72e
-
ensure timeout of 3 minutes for comp batch of 5 videos less than 50 seconds length (#187)
Arpan TripathiUnverified signature8596d9d
-
compression competition mode (#179)
Arpan TripathiUnverified signature1f4a4dd
-
revise upscaling eligible codecs (#157)
Arpan TripathiUnverified signature* Update server.py * add sar 1:1 restriction to base compression miner97fab15
-
Add Modal serverless backend for base compression & upscaling miners (#156)
Arpan TripathiUnverified signature9ef67e5
-
replace base miner worker with docker-compose setup (#149)
Arpan TripathiUnverified signaturef0c392f