Eric-Yyvuw/yeepee-ctr

Yeepee — Huawei TechArena 2026 CTR (Phase 1)

Private team repo. LightGBM click-through-rate model for the Huawei TechArena 2026 Ireland CTR challenge.

No competition data is included — no train/evaluation files, feature caches, parquet/duckdb, or row-level predictions/submissions. Only code, logs, metrics, charts and LightGBM model dumps.

Last sync: 2026-10-29 07:40 GMT · 2485 files.

Overview

Team Yeepee, Huawei TechArena 2026 (Ireland), Phase 1 CTR challenge. Each event shows 10 candidate items, with at most one click; each row carries up to 20 prior clicks of the user (including warm-up). Submissions are a click probability per row and are ranked by global ROC AUC over all rows (not per event). The solution has to run on a laptop CPU (no GPU).

Status: best submission is submission_v3 — 0.788757 public AUC (23 Oct 2026), 0.7775 local validation. Phase 1 is still open; the code here is a work in progress.

Model

• LightGBM binary classifier (lr 0.05, 63 leaves, 400 rounds), 325 features in v3 (the earlier v3/ stand-alone pipeline used 164; see v3/out/final_info.json).
• 3-seed average of predicted probabilities (plain mean, no per-event normalisation).
• Training rows: the first event per user in [Ck, Ck+1d) for six cutoff folds Ck = 21–26 Jan (about 2.1M rows), with every statistic frozen at Ck. Evaluation features are frozen at 27 Jan.
• Runs on a laptop-class CPU; no GPU.

Feature families

• SVD item embeddings of the training co-click graph (k = 16 / 32 / 64, plus 128 for some centroid features).
• History-cosine similarity: candidate vs the user's click-history centroid (plain, recency-weighted, contrastive, item-side variants).
• Within-event normalisation: z-score and rank of key features across the 10 candidates of the same event.
• History-match counts: how many of the user's prior clicks match / co-occur with the candidate.
• User CTR and user-cluster CTR from training labels before the cutoff.
• Co-occurrence / transition statistics between history items and candidates (training data only).
• Ridge taste vectors: per-user ridge regression on frozen SVD-k64 item vectors (v2work/extra2.py).
• User-state features (activity counts, history length, etc.). Time/gap features are dropped in v3 (notime).

Validation protocol

• Time-based split: fit on folds before the validation cutoff; validate on the first event per user after the cutoff (main cutoff 25 Jan, check cutoff 26 Jan).
• Label statistics frozen at the cutoff (CTR, co-click graph, SVD/MF embeddings use only labels before it).
• Whole events kept together (all 10 candidates in the same split).
• Metric is global ROC AUC, plus segment AUCs (warm / history-only / empty-history; stale ≥ 1 day vs fresher) and event-level vs within-event AUC.
• Leak checks in v3/out/leakcheck.jsonl.

Data policy

• No aggregates over the unlabelled evaluation rows. Every global / item-level statistic (popularity, co-occurrence, transitions, SVD factors, centroids, clusters) uses training data only.
• Each evaluation row's own history is used only for that row's own user features.
• submission_v2 was withdrawn because it pooled evaluation-row histories into item statistics.

Version history

version	local AUC (cutoff 25 Jan)	public AUC	notes
starter	0.611	—	competition starter baseline (v3/out/baselines.json)
v2	0.762	—	superseded / not to be used — item stats pooled evaluation-row histories
v3	0.7775	0.788757 (23 Oct 2026)	clean: training data + each row's own history only; 325 features, 3 seeds

Limitations

• Empty-history users are weak (AUC about 0.56); history-only users about 0.72 vs about 0.78 for warm users.
• Stale rows (≥ 1 day stale) score lower (about 0.758 vs 0.782 at cutoff 25 Jan) and make up a larger share of test than of local validation.
• Event-level AUC (which events get any click) is low (about 0.61–0.65); within-event ranking is about 0.76–0.78.
• No competition data is in this repo, so nothing here can be re-run without the organisers' files.

Experiments that did not help (v2work/screen_results.jsonl)

Screening runs on a 25% user sample, 1 seed; AUC = best point of the 100–400-round curve. Δ is against the base run at the same cutoff. A new submission needs ≥ +0.002 locally.

cutoff	tag	what	n_feats	global AUC	Δ vs base	event AUC	within-event AUC	empty-history	stale ≥1d
25	base	reference (v3 feature set)	325	0.77286	+0.00000	0.6118	0.7609	0.5499	0.7518
25	ev	+ OOF event-level P(click) feature	327	0.77315	+0.00029	0.6129	0.7615	0.5416	0.7534
26	base	reference (v3 feature set)	325	0.78965	+0.00000	0.6183	0.7787	0.5300	—
26	ev	+ OOF event-level P(click) feature	327	0.78940	-0.00025	0.6187	0.7786	0.5286	—
25	lr	lambdarank objective	325	0.76349	-0.00937	0.5756	0.7529	0.5078	0.7388
25	dropcum	drop cumulative-count features	308	0.77183	-0.00103	0.6106	0.7599	0.5476	0.7509
26	dropcum	drop cumulative-count features	308	0.78938	-0.00027	0.6177	0.7781	0.5295	—
25	e_notime	fold scheme e, no time features	325	0.77296	+0.00010	0.6118	0.7617	0.5321	0.7502
25	e_stale	fold scheme e + stale-like rows	326	0.77356	+0.00070	0.6104	0.7618	0.5333	0.7516
25	fe_stale	fold schemes f+e + stale-like rows	326	0.77354	+0.00068	0.6131	0.7622	0.5419	0.7531
25	f_stale	fold scheme f + stale-like rows	326	0.77286	+0.00000	0.6117	0.7610	0.5362	0.7519
26	e_notime	fold scheme e, no time features	325	0.78938	-0.00027	0.6164	0.7789	0.5502	—
26	e_stale	fold scheme e + stale-like rows	326	0.78956	-0.00009	0.6160	0.7786	0.5338	—
26	fe_stale	fold schemes f+e + stale-like rows	326	0.78954	-0.00011	0.6170	0.7785	0.5345	—
26	f_stale	fold scheme f + stale-like rows	326	0.78934	-0.00031	0.6168	0.7783	0.5476	—
25	rc	+ repeat-click features	352	0.77287	+0.00001	0.6103	0.7615	0.5202	0.7530
26	rc	+ repeat-click features	352	0.78901	-0.00064	0.6174	0.7784	0.5285	—
25	fe_stale_rc	stale folds + repeat-click	353	0.77317	+0.00031	0.6115	0.7627	0.5423	0.7530
26	fe_stale_rc	stale folds + repeat-click	353	0.78935	-0.00030	0.6171	0.7782	0.5286	—
25	fe_stale_rc_cv	stale folds + repeat-click + co-visitation	372	0.77312	+0.00026	0.6118	0.7616	0.5372	0.7520
26	fe_stale_rc_cv	stale folds + repeat-click + co-visitation	372	0.78866	-0.00099	0.6159	0.7781	0.5315	—
25	gap_base		325	0.76958	-0.00328	0.6110	0.7576	0.5597	0.7494
26	gap_base		325	0.78377	-0.00588	0.6144	0.7728	0.5366	—
25	i_usfix		325	0.77274	-0.00012	0.6126	0.7613	0.5332	0.7520
25	gap_usfix		325	0.76979	-0.00307	0.6095	0.7573	0.5479	0.7497
26	i_usfix		325	0.78970	+0.00005	0.6180	0.7790	0.5367	—
26	gap_usfix		325	0.78350	-0.00615	0.6131	0.7724	0.5214	—
25	i_notr		316	0.77267	-0.00019	0.6127	0.7605	0.5271	0.7512
25	gap_notr		316	0.76939	-0.00347	0.6111	0.7576	0.5462	0.7484
26	i_notr		316	0.79037	+0.00072	0.6182	0.7796	0.5331	—
26	gap_notr		316	0.78382	-0.00583	0.6111	0.7733	0.5344	—
25	i_share		328	0.77223	-0.00063	0.6118	0.7604	0.5403	0.7508
25	gap_share		328	0.76926	-0.00360	0.6103	0.7576	0.5348	0.7486
26	i_share		328	0.78921	-0.00044	0.6160	0.7783	0.5251	—
26	gap_share		328	0.78432	-0.00533	0.6147	0.7739	0.5103	—
25	i_hs		333	0.77267	-0.00019	0.6120	0.7609	0.5577	0.7523
25	gap_hs		333	0.76932	-0.00354	0.6098	0.7573	0.5601	0.7476
26	i_hs		333	0.78916	-0.00049	0.6176	0.7781	0.5460	—
26	gap_hs		333	0.78336	-0.00629	0.6123	0.7716	0.5059	—
25	c_usfix_notr		316	0.77260	-0.00026	0.6119	0.7608	0.5360	0.7516
25	gap_c_usfix_notr		316	0.76986	-0.00300	0.6109	0.7575	0.5391	0.7480
26	c_usfix_notr		316	0.78980	+0.00015	0.6161	0.7793	0.5320	—
26	gap_c_usfix_notr		316	0.78352	-0.00613	0.6145	0.7728	0.5283	—
25	base_seed1		325	0.77298	+0.00012	0.6113	0.7613	0.5457	0.7512
25	gap_base_seed1		325	0.76972	-0.00314	0.6103	0.7579	0.5502	0.7494
26	base_seed1		325	0.78937	-0.00028	0.6165	0.7777	0.5167	—
26	gap_base_seed1		325	0.78349	-0.00616	0.6115	0.7716	0.5409	—
25	i_lr12		328	0.79146	+0.01860	0.6211	0.7803	0.5288	0.7736
25	gap_lr12		328	0.78908	+0.01622	0.6149	0.7766	0.5232	0.7705
26	i_lr12		328	0.80298	+0.01333	0.6235	0.7924	0.5316	—
26	gap_lr12		328	0.79821	+0.00856	0.6168	0.7865	0.5097	—
25	i_lrall		334	0.79468	+0.02182	0.6201	0.7837	0.5211	0.7765
25	i_lr10		328	0.79367	+0.02081	0.6160	0.7809	0.5147	0.7733
26	i_lrall		334	0.80615	+0.01650	0.6240	0.7961	0.5375	—
26	i_lr10		328	0.80616	+0.01651	0.6238	0.7953	0.5225	—
25	i_lr2all		379	0.79734	+0.02448	0.6204	0.7866	0.5267	0.7779
25	i_lr2_1012c		336	0.79656	+0.02370	0.6199	0.7851	0.5296	0.7767
25	i_lr2_81012		308	0.79348	+0.02062	0.6181	0.7821	0.5368	0.7749
26	i_lr2all		379	0.80834	+0.01869	0.6256	0.7978	0.5148	—
26	i_lr2_1012c		336	0.80727	+0.01762	0.6225	0.7968	0.5111	—
26	i_lr2_81012		308	0.80496	+0.01531	0.6258	0.7949	0.5085	—
25	gap_lr2all		379	0.79579	+0.02293	0.6190	0.7841	0.5449	0.7753
25	gap_lrall		334	0.79237	+0.01951	0.6172	0.7808	0.5291	0.7737
26	gap_lr2all		379	0.80490	+0.01525	0.6211	0.7928	0.4951	—
26	gap_lrall		334	0.80196	+0.01231	0.6196	0.7904	0.4982	—
25	v7_AB		387	0.80842	+0.03556	0.6193	0.7965	0.5390	0.7918
25	v7_A		382	0.80246	+0.02960	0.6228	0.7911	0.5236	0.7844
25	v7_B		384	0.80842	+0.03556	0.6160	0.7962	0.5163	0.7919
26	v7_AB		387	0.81854	+0.02889	0.6230	0.8080	0.5078	—
26	v7_A		382	0.81220	+0.02255	0.6282	0.8016	0.5098	—
26	v7_B		384	0.81814	+0.02849	0.6229	0.8074	0.5197	—
25	gap_v7_AB		387	0.80754	+0.03468	0.6190	0.7950	0.5169	0.7895
25	gap_v7_A		382	0.80059	+0.02773	0.6197	0.7886	0.5411	0.7827
25	gap_v7_B		384	0.80738	+0.03452	0.6152	0.7950	0.5496	0.7901
26	gap_v7_AB		387	0.81625	+0.02660	0.6234	0.8042	0.5049	—
26	gap_v7_A		382	0.80897	+0.01932	0.6236	0.7965	0.4926	—
26	gap_v7_B		384	0.81618	+0.02653	0.6203	0.8047	0.5150	—
25	xAn_lr2		410	0.79933	+0.02647	0.6213	0.7883	0.5091	0.7787
25	xAn_v7		418	0.80870	+0.03584	0.6181	0.7965	0.5143	0.7915
26	xAn_lr2		410	0.81069	+0.02104	0.6257	0.7991	0.5061	—
26	xAn_v7		418	0.81891	+0.02926	0.6225	0.8078	0.5238	—
25	gap_xAn_lr2		410	0.79760	+0.02474	0.6199	0.7859	0.4990	0.7776
25	gap_xAn_v7		418	0.80785	+0.03499	0.6162	0.7955	0.5431	0.7898
26	gap_xAn_lr2		410	0.80726	+0.01761	0.6216	0.7955	0.4764	—
26	gap_xAn_v7		418	0.81652	+0.02687	0.6180	0.8037	0.5021	—

Event × item decomposition (v2work/posthoc_B3.json)

Separate event-level model p_event (does the event get any click), combined with the item model as p_event × softmax(item) or logit(p_event) + α·item_logit over α ∈ {0.5…3}. Best combination vs the plain item model:

cutoff	item model	p_event only	best combination	its global AUC
25	0.77174	0.55818	b_pev_x_softmax	0.77151
26	0.78921	0.56130	b_pev_x_softmax	0.78913

No combination beat the plain item model: its scores already carry the event-level signal.

Auto-generated results

v2 combo runs (v2work/combo_results.jsonl)

cut	variant	n_feats	best iter	AUC all	AUC warm	AUC history-only	AUC empty-history
25	base	276	300	0.7617	0.7636	0.7139	0.5505
25	notime	268	450	0.7620	0.7638	0.7154	0.5608
25	notime_x	306	450	0.7738	0.7758	0.7199	0.5757
25	notime	268	400	0.7617	0.7635	0.7148	0.5527
25	notime_x2	325	350	0.7754	0.7774	0.7224	0.5538
25	notime	325	400	0.7772	0.7793	0.7230	0.5602
25	notime_eft_ne	325	300	0.7775	0.7795	0.7243	0.5662

Repo layout

• v2work/ — main pipeline behind v3: prep.py, v3fold.py / v3feat_ne.py (v3 features), build_folds.py / feats.py (v2 features), extra.py (similarity extras), extra2.py (ridge taste vectors, user state), combo.py (train / validate / final fit), write_sub.py; results combo_results.jsonl, screen_results.jsonl, event_results.jsonl, posthoc_B3.json; seed models final_model_s*.txt; out/solution_v3.py (driver), out/submission_log.csv; ens/ ensemble runs.
• v3/ — stand-alone pipeline (prep.py, features.py, train.py, metric.py, segments.py, validate_submission.py), results in v3/out/, LightGBM dumps *.txt.
• v4/ — staleness-aware protocol experiments.
• research/ — AUC improvement notes; charts/ — training/profile charts.

How to reproduce

You need the competition files (train.zip, evaluation.zip), which are not included.

# unpack competition data into /workspace/huawei/data, then:
python v2work/out/solution_v3.py             # features -> 3-seed final fit -> v2work/out/submission_v3.csv
python v2work/out/solution_v3.py --validate  # also runs the cutoff-25 validation (combo.py val 25 notime)
python v3/validate_submission.py v2work/out/submission_v3.csv   # format check

Paths in the scripts assume /workspace/huawei. The driver runs, in order: prep.py, v3fold.py, build_folds.py c25/c26, extra.py, extra2.py, XF=1 XF2=1 combo.py final 400 3 notime, write_sub.py.

License

license: other. The code and model files are the team's competition work. The competition data and its terms belong to the Huawei TechArena 2026 organisers, and the competition rules apply to any use. The data is not redistributed here.