|
Download README.md from Eric-Yyvuw/yeepee-ctr: direct link, hf CLI and curl.
- Browser
- Download file 16.6 kB
-
https://huggingface.co/Eric-Yyvuw/yeepee-ctr/resolve/main/README.md
- Command line
-
hf download hf://Eric-Yyvuw/yeepee-ctr/README.md
-
curl -L -o README.md https://huggingface.co/Eric-Yyvuw/yeepee-ctr/resolve/main/README.md
16.6 kB
| library_name: lightgbm | |
| license: other | |
| metrics: | |
| - roc_auc | |
| pipeline_tag: tabular-classification | |
| tags: | |
| - ctr | |
| - lightgbm | |
| - tabular | |
| - recsys | |
| - click-through-rate | |
| - learning-to-rank | |
| - huawei-techarena | |
| model-index: | |
| - name: yeepee-ctr | |
| results: | |
| - task: | |
| type: tabular-classification | |
| name: CTR prediction (10 candidates per event) | |
| dataset: | |
| name: Huawei TechArena 2026 Phase 1 (private competition data, not included) | |
| type: huawei-techarena-2026-phase1 | |
| split: local-validation-cutoff-25jan | |
| metrics: | |
| - type: roc_auc | |
| value: 0.7775 | |
| name: ROC AUC, local validation (cutoff 25 Jan, first event per user after cutoff) | |
| verified: false | |
| - task: | |
| type: tabular-classification | |
| name: CTR prediction (10 candidates per event) | |
| dataset: | |
| name: Huawei TechArena 2026 Phase 1 (private competition data, not included) | |
| type: huawei-techarena-2026-phase1 | |
| split: public-leaderboard | |
| metrics: | |
| - type: roc_auc | |
| value: 0.788757 | |
| name: ROC AUC, public leaderboard (submission_v3, 23 Oct 2026) | |
| verified: false | |
| # Yeepee β Huawei TechArena 2026 CTR (Phase 1) | |
| **Private team repo.** LightGBM click-through-rate model for the Huawei TechArena 2026 Ireland CTR challenge. | |
| **No competition data is included** β no train/evaluation files, feature caches, parquet/duckdb, or row-level predictions/submissions. Only code, logs, metrics, charts and LightGBM model dumps. | |
| _Last sync: 2026-10-29 07:40 GMT Β· 2485 files._ | |
| ## Overview | |
| Team **Yeepee**, Huawei TechArena 2026 (Ireland), Phase 1 CTR challenge. Each event shows **10 candidate items**, | |
| with **at most one click**; each row carries up to 20 prior clicks of the user (including warm-up). Submissions are a | |
| click probability per row and are ranked by **global ROC AUC** over all rows (not per event). The solution has to run on | |
| a laptop CPU (no GPU). | |
| **Status:** best submission is `submission_v3` β **0.788757 public AUC** (23 Oct 2026), 0.7775 local validation. | |
| Phase 1 is still open; the code here is a work in progress. | |
| ## Model | |
| - **LightGBM binary** classifier (lr 0.05, 63 leaves, 400 rounds), **325 features** in v3 | |
| (the earlier `v3/` stand-alone pipeline used 164; see `v3/out/final_info.json`). | |
| - **3-seed average** of predicted probabilities (plain mean, no per-event normalisation). | |
| - Training rows: the first event per user in `[Ck, Ck+1d)` for six cutoff folds **Ck = 21β26 Jan** (about 2.1M rows), | |
| with every statistic frozen at Ck. Evaluation features are frozen at 27 Jan. | |
| - Runs on a laptop-class CPU; no GPU. | |
| ## Feature families | |
| - **SVD item embeddings** of the training co-click graph (k = 16 / 32 / 64, plus 128 for some centroid features). | |
| - **History-cosine similarity**: candidate vs the user's click-history centroid (plain, recency-weighted, contrastive, | |
| item-side variants). | |
| - **Within-event normalisation**: z-score and rank of key features across the 10 candidates of the same event. | |
| - **History-match counts**: how many of the user's prior clicks match / co-occur with the candidate. | |
| - **User CTR** and user-cluster CTR from training labels before the cutoff. | |
| - **Co-occurrence / transition** statistics between history items and candidates (training data only). | |
| - **Ridge taste vectors**: per-user ridge regression on frozen SVD-k64 item vectors (`v2work/extra2.py`). | |
| - **User-state** features (activity counts, history length, etc.). Time/gap features are dropped in v3 (`notime`). | |
| ## Validation protocol | |
| - Time-based split: fit on folds before the validation cutoff; validate on the **first event per user after the | |
| cutoff** (main cutoff 25 Jan, check cutoff 26 Jan). | |
| - **Label statistics frozen at the cutoff** (CTR, co-click graph, SVD/MF embeddings use only labels before it). | |
| - **Whole events kept together** (all 10 candidates in the same split). | |
| - Metric is **global ROC AUC**, plus segment AUCs (warm / history-only / empty-history; stale β₯ 1 day vs fresher) and | |
| event-level vs within-event AUC. | |
| - Leak checks in `v3/out/leakcheck.jsonl`. | |
| ## Data policy | |
| - **No aggregates over the unlabelled evaluation rows.** Every global / item-level statistic (popularity, | |
| co-occurrence, transitions, SVD factors, centroids, clusters) uses training data only. | |
| - Each evaluation row's own history is used only for that row's own user features. | |
| - `submission_v2` was withdrawn because it pooled evaluation-row histories into item statistics. | |
| ## Version history | |
| | version | local AUC (cutoff 25 Jan) | public AUC | notes | | |
| |---|---|---|---| | |
| | starter | 0.611 | β | competition starter baseline (`v3/out/baselines.json`) | | |
| | v2 | 0.762 | β | **superseded / not to be used** β item stats pooled evaluation-row histories | | |
| | v3 | **0.7775** | **0.788757** (23 Oct 2026) | clean: training data + each row's own history only; 325 features, 3 seeds | | |
| ## Limitations | |
| - **Empty-history users** are weak (AUC about 0.56); history-only users about 0.72 vs about 0.78 for warm users. | |
| - **Stale rows** (β₯ 1 day stale) score lower (about 0.758 vs 0.782 at cutoff 25 Jan) and make up | |
| a larger share of test than of local validation. | |
| - Event-level AUC (which events get any click) is low (about 0.61β0.65); within-event ranking is about 0.76β0.78. | |
| - **No competition data is in this repo**, so nothing here can be re-run without the organisers' files. | |
| ## Experiments that did not help (`v2work/screen_results.jsonl`) | |
| Screening runs on a 25% user sample, 1 seed; AUC = best point of the 100β400-round curve. Ξ is against the `base` run at the same cutoff. A new submission needs β₯ +0.002 locally. | |
| | cutoff | tag | what | n_feats | global AUC | Ξ vs base | event AUC | within-event AUC | empty-history | stale β₯1d | | |
| |---|---|---|---|---|---|---|---|---|---| | |
| | 25 | `base` | reference (v3 feature set) | 325 | 0.77286 | +0.00000 | 0.6118 | 0.7609 | 0.5499 | 0.7518 | | |
| | 25 | `ev` | + OOF event-level P(click) feature | 327 | 0.77315 | +0.00029 | 0.6129 | 0.7615 | 0.5416 | 0.7534 | | |
| | 26 | `base` | reference (v3 feature set) | 325 | 0.78965 | +0.00000 | 0.6183 | 0.7787 | 0.5300 | β | | |
| | 26 | `ev` | + OOF event-level P(click) feature | 327 | 0.78940 | -0.00025 | 0.6187 | 0.7786 | 0.5286 | β | | |
| | 25 | `lr` | lambdarank objective | 325 | 0.76349 | -0.00937 | 0.5756 | 0.7529 | 0.5078 | 0.7388 | | |
| | 25 | `dropcum` | drop cumulative-count features | 308 | 0.77183 | -0.00103 | 0.6106 | 0.7599 | 0.5476 | 0.7509 | | |
| | 26 | `dropcum` | drop cumulative-count features | 308 | 0.78938 | -0.00027 | 0.6177 | 0.7781 | 0.5295 | β | | |
| | 25 | `e_notime` | fold scheme e, no time features | 325 | 0.77296 | +0.00010 | 0.6118 | 0.7617 | 0.5321 | 0.7502 | | |
| | 25 | `e_stale` | fold scheme e + stale-like rows | 326 | 0.77356 | +0.00070 | 0.6104 | 0.7618 | 0.5333 | 0.7516 | | |
| | 25 | `fe_stale` | fold schemes f+e + stale-like rows | 326 | 0.77354 | +0.00068 | 0.6131 | 0.7622 | 0.5419 | 0.7531 | | |
| | 25 | `f_stale` | fold scheme f + stale-like rows | 326 | 0.77286 | +0.00000 | 0.6117 | 0.7610 | 0.5362 | 0.7519 | | |
| | 26 | `e_notime` | fold scheme e, no time features | 325 | 0.78938 | -0.00027 | 0.6164 | 0.7789 | 0.5502 | β | | |
| | 26 | `e_stale` | fold scheme e + stale-like rows | 326 | 0.78956 | -0.00009 | 0.6160 | 0.7786 | 0.5338 | β | | |
| | 26 | `fe_stale` | fold schemes f+e + stale-like rows | 326 | 0.78954 | -0.00011 | 0.6170 | 0.7785 | 0.5345 | β | | |
| | 26 | `f_stale` | fold scheme f + stale-like rows | 326 | 0.78934 | -0.00031 | 0.6168 | 0.7783 | 0.5476 | β | | |
| | 25 | `rc` | + repeat-click features | 352 | 0.77287 | +0.00001 | 0.6103 | 0.7615 | 0.5202 | 0.7530 | | |
| | 26 | `rc` | + repeat-click features | 352 | 0.78901 | -0.00064 | 0.6174 | 0.7784 | 0.5285 | β | | |
| | 25 | `fe_stale_rc` | stale folds + repeat-click | 353 | 0.77317 | +0.00031 | 0.6115 | 0.7627 | 0.5423 | 0.7530 | | |
| | 26 | `fe_stale_rc` | stale folds + repeat-click | 353 | 0.78935 | -0.00030 | 0.6171 | 0.7782 | 0.5286 | β | | |
| | 25 | `fe_stale_rc_cv` | stale folds + repeat-click + co-visitation | 372 | 0.77312 | +0.00026 | 0.6118 | 0.7616 | 0.5372 | 0.7520 | | |
| | 26 | `fe_stale_rc_cv` | stale folds + repeat-click + co-visitation | 372 | 0.78866 | -0.00099 | 0.6159 | 0.7781 | 0.5315 | β | | |
| | 25 | `gap_base` | | 325 | 0.76958 | -0.00328 | 0.6110 | 0.7576 | 0.5597 | 0.7494 | | |
| | 26 | `gap_base` | | 325 | 0.78377 | -0.00588 | 0.6144 | 0.7728 | 0.5366 | β | | |
| | 25 | `i_usfix` | | 325 | 0.77274 | -0.00012 | 0.6126 | 0.7613 | 0.5332 | 0.7520 | | |
| | 25 | `gap_usfix` | | 325 | 0.76979 | -0.00307 | 0.6095 | 0.7573 | 0.5479 | 0.7497 | | |
| | 26 | `i_usfix` | | 325 | 0.78970 | +0.00005 | 0.6180 | 0.7790 | 0.5367 | β | | |
| | 26 | `gap_usfix` | | 325 | 0.78350 | -0.00615 | 0.6131 | 0.7724 | 0.5214 | β | | |
| | 25 | `i_notr` | | 316 | 0.77267 | -0.00019 | 0.6127 | 0.7605 | 0.5271 | 0.7512 | | |
| | 25 | `gap_notr` | | 316 | 0.76939 | -0.00347 | 0.6111 | 0.7576 | 0.5462 | 0.7484 | | |
| | 26 | `i_notr` | | 316 | 0.79037 | +0.00072 | 0.6182 | 0.7796 | 0.5331 | β | | |
| | 26 | `gap_notr` | | 316 | 0.78382 | -0.00583 | 0.6111 | 0.7733 | 0.5344 | β | | |
| | 25 | `i_share` | | 328 | 0.77223 | -0.00063 | 0.6118 | 0.7604 | 0.5403 | 0.7508 | | |
| | 25 | `gap_share` | | 328 | 0.76926 | -0.00360 | 0.6103 | 0.7576 | 0.5348 | 0.7486 | | |
| | 26 | `i_share` | | 328 | 0.78921 | -0.00044 | 0.6160 | 0.7783 | 0.5251 | β | | |
| | 26 | `gap_share` | | 328 | 0.78432 | -0.00533 | 0.6147 | 0.7739 | 0.5103 | β | | |
| | 25 | `i_hs` | | 333 | 0.77267 | -0.00019 | 0.6120 | 0.7609 | 0.5577 | 0.7523 | | |
| | 25 | `gap_hs` | | 333 | 0.76932 | -0.00354 | 0.6098 | 0.7573 | 0.5601 | 0.7476 | | |
| | 26 | `i_hs` | | 333 | 0.78916 | -0.00049 | 0.6176 | 0.7781 | 0.5460 | β | | |
| | 26 | `gap_hs` | | 333 | 0.78336 | -0.00629 | 0.6123 | 0.7716 | 0.5059 | β | | |
| | 25 | `c_usfix_notr` | | 316 | 0.77260 | -0.00026 | 0.6119 | 0.7608 | 0.5360 | 0.7516 | | |
| | 25 | `gap_c_usfix_notr` | | 316 | 0.76986 | -0.00300 | 0.6109 | 0.7575 | 0.5391 | 0.7480 | | |
| | 26 | `c_usfix_notr` | | 316 | 0.78980 | +0.00015 | 0.6161 | 0.7793 | 0.5320 | β | | |
| | 26 | `gap_c_usfix_notr` | | 316 | 0.78352 | -0.00613 | 0.6145 | 0.7728 | 0.5283 | β | | |
| | 25 | `base_seed1` | | 325 | 0.77298 | +0.00012 | 0.6113 | 0.7613 | 0.5457 | 0.7512 | | |
| | 25 | `gap_base_seed1` | | 325 | 0.76972 | -0.00314 | 0.6103 | 0.7579 | 0.5502 | 0.7494 | | |
| | 26 | `base_seed1` | | 325 | 0.78937 | -0.00028 | 0.6165 | 0.7777 | 0.5167 | β | | |
| | 26 | `gap_base_seed1` | | 325 | 0.78349 | -0.00616 | 0.6115 | 0.7716 | 0.5409 | β | | |
| | 25 | `i_lr12` | | 328 | 0.79146 | +0.01860 | 0.6211 | 0.7803 | 0.5288 | 0.7736 | | |
| | 25 | `gap_lr12` | | 328 | 0.78908 | +0.01622 | 0.6149 | 0.7766 | 0.5232 | 0.7705 | | |
| | 26 | `i_lr12` | | 328 | 0.80298 | +0.01333 | 0.6235 | 0.7924 | 0.5316 | β | | |
| | 26 | `gap_lr12` | | 328 | 0.79821 | +0.00856 | 0.6168 | 0.7865 | 0.5097 | β | | |
| | 25 | `i_lrall` | | 334 | 0.79468 | +0.02182 | 0.6201 | 0.7837 | 0.5211 | 0.7765 | | |
| | 25 | `i_lr10` | | 328 | 0.79367 | +0.02081 | 0.6160 | 0.7809 | 0.5147 | 0.7733 | | |
| | 26 | `i_lrall` | | 334 | 0.80615 | +0.01650 | 0.6240 | 0.7961 | 0.5375 | β | | |
| | 26 | `i_lr10` | | 328 | 0.80616 | +0.01651 | 0.6238 | 0.7953 | 0.5225 | β | | |
| | 25 | `i_lr2all` | | 379 | 0.79734 | +0.02448 | 0.6204 | 0.7866 | 0.5267 | 0.7779 | | |
| | 25 | `i_lr2_1012c` | | 336 | 0.79656 | +0.02370 | 0.6199 | 0.7851 | 0.5296 | 0.7767 | | |
| | 25 | `i_lr2_81012` | | 308 | 0.79348 | +0.02062 | 0.6181 | 0.7821 | 0.5368 | 0.7749 | | |
| | 26 | `i_lr2all` | | 379 | 0.80834 | +0.01869 | 0.6256 | 0.7978 | 0.5148 | β | | |
| | 26 | `i_lr2_1012c` | | 336 | 0.80727 | +0.01762 | 0.6225 | 0.7968 | 0.5111 | β | | |
| | 26 | `i_lr2_81012` | | 308 | 0.80496 | +0.01531 | 0.6258 | 0.7949 | 0.5085 | β | | |
| | 25 | `gap_lr2all` | | 379 | 0.79579 | +0.02293 | 0.6190 | 0.7841 | 0.5449 | 0.7753 | | |
| | 25 | `gap_lrall` | | 334 | 0.79237 | +0.01951 | 0.6172 | 0.7808 | 0.5291 | 0.7737 | | |
| | 26 | `gap_lr2all` | | 379 | 0.80490 | +0.01525 | 0.6211 | 0.7928 | 0.4951 | β | | |
| | 26 | `gap_lrall` | | 334 | 0.80196 | +0.01231 | 0.6196 | 0.7904 | 0.4982 | β | | |
| | 25 | `v7_AB` | | 387 | 0.80842 | +0.03556 | 0.6193 | 0.7965 | 0.5390 | 0.7918 | | |
| | 25 | `v7_A` | | 382 | 0.80246 | +0.02960 | 0.6228 | 0.7911 | 0.5236 | 0.7844 | | |
| | 25 | `v7_B` | | 384 | 0.80842 | +0.03556 | 0.6160 | 0.7962 | 0.5163 | 0.7919 | | |
| | 26 | `v7_AB` | | 387 | 0.81854 | +0.02889 | 0.6230 | 0.8080 | 0.5078 | β | | |
| | 26 | `v7_A` | | 382 | 0.81220 | +0.02255 | 0.6282 | 0.8016 | 0.5098 | β | | |
| | 26 | `v7_B` | | 384 | 0.81814 | +0.02849 | 0.6229 | 0.8074 | 0.5197 | β | | |
| | 25 | `gap_v7_AB` | | 387 | 0.80754 | +0.03468 | 0.6190 | 0.7950 | 0.5169 | 0.7895 | | |
| | 25 | `gap_v7_A` | | 382 | 0.80059 | +0.02773 | 0.6197 | 0.7886 | 0.5411 | 0.7827 | | |
| | 25 | `gap_v7_B` | | 384 | 0.80738 | +0.03452 | 0.6152 | 0.7950 | 0.5496 | 0.7901 | | |
| | 26 | `gap_v7_AB` | | 387 | 0.81625 | +0.02660 | 0.6234 | 0.8042 | 0.5049 | β | | |
| | 26 | `gap_v7_A` | | 382 | 0.80897 | +0.01932 | 0.6236 | 0.7965 | 0.4926 | β | | |
| | 26 | `gap_v7_B` | | 384 | 0.81618 | +0.02653 | 0.6203 | 0.8047 | 0.5150 | β | | |
| | 25 | `xAn_lr2` | | 410 | 0.79933 | +0.02647 | 0.6213 | 0.7883 | 0.5091 | 0.7787 | | |
| | 25 | `xAn_v7` | | 418 | 0.80870 | +0.03584 | 0.6181 | 0.7965 | 0.5143 | 0.7915 | | |
| | 26 | `xAn_lr2` | | 410 | 0.81069 | +0.02104 | 0.6257 | 0.7991 | 0.5061 | β | | |
| | 26 | `xAn_v7` | | 418 | 0.81891 | +0.02926 | 0.6225 | 0.8078 | 0.5238 | β | | |
| | 25 | `gap_xAn_lr2` | | 410 | 0.79760 | +0.02474 | 0.6199 | 0.7859 | 0.4990 | 0.7776 | | |
| | 25 | `gap_xAn_v7` | | 418 | 0.80785 | +0.03499 | 0.6162 | 0.7955 | 0.5431 | 0.7898 | | |
| | 26 | `gap_xAn_lr2` | | 410 | 0.80726 | +0.01761 | 0.6216 | 0.7955 | 0.4764 | β | | |
| | 26 | `gap_xAn_v7` | | 418 | 0.81652 | +0.02687 | 0.6180 | 0.8037 | 0.5021 | β | | |
| ### Event Γ item decomposition (`v2work/posthoc_B3.json`) | |
| Separate event-level model p_event (does the event get any click), combined with the item model as `p_event Γ softmax(item)` or `logit(p_event) + Ξ±Β·item_logit` over Ξ± β {0.5β¦3}. Best combination vs the plain item model: | |
| | cutoff | item model | p_event only | best combination | its global AUC | | |
| |---|---|---|---|---| | |
| | 25 | 0.77174 | 0.55818 | `b_pev_x_softmax` | 0.77151 | | |
| | 26 | 0.78921 | 0.56130 | `b_pev_x_softmax` | 0.78913 | | |
| No combination beat the plain item model: its scores already carry the event-level signal. | |
| # Auto-generated results | |
| ## v2 combo runs (`v2work/combo_results.jsonl`) | |
| | cut | variant | n_feats | best iter | AUC all | AUC warm | AUC history-only | AUC empty-history | | |
| |---|---|---|---|---|---|---|---| | |
| | 25 | base | 276 | 300 | 0.7617 | 0.7636 | 0.7139 | 0.5505 | | |
| | 25 | notime | 268 | 450 | 0.7620 | 0.7638 | 0.7154 | 0.5608 | | |
| | 25 | notime_x | 306 | 450 | 0.7738 | 0.7758 | 0.7199 | 0.5757 | | |
| | 25 | notime | 268 | 400 | 0.7617 | 0.7635 | 0.7148 | 0.5527 | | |
| | 25 | notime_x2 | 325 | 350 | 0.7754 | 0.7774 | 0.7224 | 0.5538 | | |
| | 25 | notime | 325 | 400 | 0.7772 | 0.7793 | 0.7230 | 0.5602 | | |
| | 25 | notime_eft_ne | 325 | 300 | 0.7775 | 0.7795 | 0.7243 | 0.5662 | | |
| ## Repo layout | |
| - `v2work/` β main pipeline behind v3: `prep.py`, `v3fold.py` / `v3feat_ne.py` (v3 features), `build_folds.py` / | |
| `feats.py` (v2 features), `extra.py` (similarity extras), `extra2.py` (ridge taste vectors, user state), | |
| `combo.py` (train / validate / final fit), `write_sub.py`; results `combo_results.jsonl`, `screen_results.jsonl`, | |
| `event_results.jsonl`, `posthoc_B3.json`; seed models `final_model_s*.txt`; `out/solution_v3.py` (driver), | |
| `out/submission_log.csv`; `ens/` ensemble runs. | |
| - `v3/` β stand-alone pipeline (`prep.py`, `features.py`, `train.py`, `metric.py`, `segments.py`, | |
| `validate_submission.py`), results in `v3/out/`, LightGBM dumps `*.txt`. | |
| - `v4/` β staleness-aware protocol experiments. | |
| - `research/` β AUC improvement notes; `charts/` β training/profile charts. | |
| ## How to reproduce | |
| You need the competition files (`train.zip`, `evaluation.zip`), which are **not** included. | |
| ```bash | |
| # unpack competition data into /workspace/huawei/data, then: | |
| python v2work/out/solution_v3.py # features -> 3-seed final fit -> v2work/out/submission_v3.csv | |
| python v2work/out/solution_v3.py --validate # also runs the cutoff-25 validation (combo.py val 25 notime) | |
| python v3/validate_submission.py v2work/out/submission_v3.csv # format check | |
| ``` | |
| Paths in the scripts assume `/workspace/huawei`. The driver runs, in order: `prep.py`, `v3fold.py`, | |
| `build_folds.py c25/c26`, `extra.py`, `extra2.py`, `XF=1 XF2=1 combo.py final 400 3 notime`, `write_sub.py`. | |
| ## License | |
| `license: other`. The code and model files are the team's competition work. The competition data and its terms belong to | |
| the Huawei TechArena 2026 organisers, and **the competition rules apply** to any use. The data is not redistributed here. | |