raw/source_audit.md

Source audit — arXiv 2603.00039 (CARE)

Field	Value
Title	CARE: Confounder-Aware Aggregation for Reliable LLM Evaluation
Authors	Jitian Zhao, Changho Shin, Tzu-Heng Huang, Satya Sai Srinath Namburi GNVV, Frederic Sala
OpenReview	https://openreview.net/forum?id=3WPDFjZ1UT
Source retrieved	https://ar5iv.labs.arxiv.org/html/2603.00039
Retrieved (UTC)	2026-08-01T04:20:00Z
User-Agent	Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120 Safari/537.36
SHA-256 of HTML	2e733a1609e1dd907dd839b9eed8d9fb7d88549f45d40fafbca7af94ee5e77ea
Bytes	865,686
Official code	https://github.com/SprocketLab/CARE @ 72f5b29a822d9934d31777c10a5c38369884c9dc

Reproduce the fetch and the hash:

curl -sL -A "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120 Safari/537.36" \
  "https://ar5iv.labs.arxiv.org/html/2603.00039" -o ar5iv.html
shasum -a 256 ar5iv.html

---

Anchors, assumptions and exact quantifiers

Table 1 — MAE, continuous scoring (Section 5.1)

Method	ASSET	FeedbackQA	Review-5K	Summarize	UltraFeedback	Yelp
MV	31.153 ± 0.000	0.822 ± 0.000	2.608 ± 0.000	1.417 ± 0.000	0.851 ± 0.000	0.923 ± 0.000
AVG	33.663 ± 0.000	0.830 ± 0.000	2.274 ± 0.000	1.394 ± 0.000	0.686 ± 0.000	1.037 ± 0.000
WS	29.073 ± 0.436	0.793 ± 0.009	2.593 ± 0.052	1.364 ± 0.007	0.829 ± 0.009	0.977 ± 0.008
UWS	33.928 ± 0.000	0.875 ± 0.000	2.602 ± 0.000	1.362 ± 0.000	0.680 ± 0.000	0.987 ± 0.000
CARE-SVD	27.629 ± 0.156	0.730 ± 0.002	1.957 ± 0.018	1.325 ± 0.004	0.623 ± 0.006	0.694 ± 0.004

Table 2 — Accuracy, classification / preference (Section 5.1)

Method	Chatbot-Arena	CivilComments	PKU-BETTER	PKU-SAFER	SHP	Summarize
MV	0.517	0.691	0.701	0.698	0.626	0.600
AVG	0.551	0.690	0.726	0.717	0.634	0.683
WS	0.543	0.739	0.575	0.570	0.619	0.705
UWS	0.507	0.713	0.703	0.701	0.629	0.713
Dawid–Skene	0.546	0.735	0.551	0.548	0.612	0.705
GLAD	0.510	0.695	0.697	0.671	0.644	0.718
MACE	0.550	0.732	0.734	0.735	0.580	0.706
CARE-SVD	0.580	0.778	0.691	0.690	0.543	0.695
CARE-Tensor	0.564	0.749	0.779	0.731	0.695	0.814

Prose quantifiers (Section 5.1)

"Specially, it reduces error by up to 26.8% compared to MV on UltraFeedback. Averaged across scoring datasets, CARE-SVD yields a 17.37% relative improvement over AVG and a 12.75% improvement over MV. On classification and preference benchmarks, CARE achieves the best accuracy on 5 of 6 datasets, with CARE-Tensor leading on three (PKU-BETTER, SHP, and Summarize), including a 13.4% relative improvement in accuracy on Summarize over the strongest baseline."

Proposition 4.1 (Section 4) — exact wording

"Assume K_HH = diag(d_1, …, d_h) with d_1 > ⋯ > d_h > 0 and the columns of K_JH are orthogonal. Then the columns of K_JH (equivalently, the latent directions encoded by L) are identifiable from L up to sign and permutation. Moreover, if K_JH is perturbed to K̃_JH = K_JH + E, letting δ_i denote the eigengap of L at eigenvector u_i, and û_i the eigenvector of the estimated low rank matrix L̂, then ‖û_i − u_i‖₂ ≲ ‖K_HH^{-1}‖₂ ‖E‖₂ / δ_i (i ∈ [h])."

Appendix form. Assumption D.2 states the columns of K_JH are orthonormal (K_JHᵀ K_JH = I_h); the proof of Theorem D.3 says explicitly "Under Assumption D.2 strengthened to orthonormal columns". Theorem D.4 gives the explicit constant

‖ũ_i − s_i u_i‖₂ ≤ 4 ‖K_HH^{-1}‖₂ ‖E‖₂ / δ_i + O(‖E‖₂²), δ_i := min{λ_i, min_{j≠i}|λ_i − λ_j|},

and its proof bounds ‖Δ‖₂ ≤ 2‖K_JH‖₂‖K_HH^{-1}‖₂‖E‖₂ + ‖K_HH^{-1}‖₂‖E‖₂² and then uses ‖K_JH‖₂ = 1, which holds only under orthonormality.

Theorem 4.2 (Section 4) = Theorem D.5

"… for any η > 0, with probability at least 1 − 2e^{−η}, `max_{i≤h} ‖û_i − u_i‖₂ = O( sqrt(η/n) · 1/(ξ(T) δ) )."

Proof chain (Appendix D.6): Chandrasekaran et al. (2012) Thm 4.1 gives ‖L̂_n − L*‖₂ ≤ C₁ sqrt(ε/n)/ξ(T); CARE then invokes Yu et al. (2015), Corollary 3, for ‖û_i − u_i‖₂ ≤ 2^{3/2}‖L̂_n − L*‖₂/δ_i; composing gives the stated rate, inverted to n ≥ 8C₁²η/(ξ(T)²δ²α²).

Literal discrepancy audited in Claim 5. D.5 defines δ=min_(1≤i<j≤h)|λ_i-λ_j|, using only the h positive eigenvalues, and writes the ordinary vector distance ‖û_i-u_i‖₂. D.4 immediately above instead defines the full-spectrum gap δ_i=min{λ_i,min_(j≠i)|λ_i-λ_j|} and includes sign alignment ‖ũ_i-s_i u_i‖₂. Yu Corollary 3 requires adjacent population gaps; for the last positive eigenvector of rank-h L*, this includes λ_h-λ_(h+1)=λ_h-0. D.5's proof first invokes that δ_i, then replaces it with the weaker positive-eigenvalue-only δ.

Frozen cited source	SHA-256	Audited anchor
Yu, Wang & Samworth, arXiv:1405.0680	6f4bc8b6a3cc768097102e54379ca894e92302ab5f9a9516fb55b71116ae0cd4	Corollary 3 uses adjacent population eigengaps, including the gap to zero
Chandrasekaran, Parrilo & Willsky, arXiv:1008.1290	3c7c5ae5941a5bee218bb5e38c06b70d2120f00ce0b1623a048fae09a5706716	Theorem 4.1 minimum-signal condition scales with the noise/regularization level

The exact sign and eigengap families and the two-model Gaussian Le Cam lower bound are recomputed by the claim module and independently by a direct 2x2 eigendecomposition and trace/log-determinant KL calculation.

Theorem 4.3 (Section 4) = Theorem D.9

"… there exist universal constants C₁, C₂ > 0 such that if n ≥ C₁ σ_max⁶/(δ² π_min²) · p log(p/ε), then with probability at least 1 − ε, max_{q,c} ‖μ̂_qc − μ_qc‖₂ ≤ C₁ (σ_max³/δ) sqrt(p log(p/ε)/n) and `max_{q,c} |π̂_qc − π_qc| ≤ C₂ sqrt(p log(p/ε)/n)."

Proof chain (Appendix D.6): eq. (8) ‖M̂ − M‖_op ≤ C σ_max³ sqrt(p log(p/ε)/n); eq. (10)+(12) mean error ≤ √3 C_dec ‖E‖_op/δ; eq. (11) weight error ≤ C_π ‖E‖_op.

Computing resources declared by the paper (Appendix E.2)

"We used a server equipped with an NVIDIA A100 (40GB). Generating LLM judge outputs took up to 3 hours per dataset."

This is the concrete blocking capability for the eight Table 1 / Table 2 columns whose judge-score matrices were not released.

---

What the official repository ships

SprocketLab/CARE @ 72f5b29 contains judge_outputs/ for exactly three datasets:

• judge_outputs/fully_gaussian/asset/ — 11 judge CSVs (Table 1, ASSET)
• judge_outputs/gaussian_mixture/civilcomments/ — 24 judge CSVs (Table 2, CivilComments)
• judge_outputs/gaussian_mixture/allenai_preference_test_sets_pku_better.tar.gz (Table 2, PKU-BETTER)

Its README.md documents two run entrypoints, scripts/fully_gaussian_main.py (Table 1) and scripts/gaussian_mixture_main.py (Table 2). Both are pure CPU: torch/vllm/transformers are needed only by scripts/save_judge_outputs.py, the judge-generation step.