# Reference data provenance

All files in this directory are derived from public CMS / CDC / NLM sources.
None contain PHI. All are redistributable under the terms of their upstream license.

## Files

| File | Upstream | License | Notes |
|---|---|---|---|
| `icd10_cm_descriptions.csv` | CDC/CMS ICD-10-CM annual release | Public domain (US government work) | Tabular list — code + title only |
| `hcc_v28_labels.csv` | CMS Risk Adjustment — HCC v28 model | Public | HCC id, label, hierarchy group, RAF weight by segment |
| `hcc_v28_crosswalk.csv` | CMS Risk Adjustment — 2024 Mapping | Public | ICD-10 → HCC v28 mapping |
| `hcc_v28_hierarchy.csv` | CMS Risk Adjustment — v28 Hierarchy File | Public | Trumping rules (disease-group-level) |

## Current state

**⚠ The seed files in this directory are hand-curated PLACEHOLDERS for the ~15 most common HCCs,
intended to let the pipeline + tests run end-to-end without a network call. Every RAF weight
marked `PLACEHOLDER` in the CSV comments must be replaced with the official CMS value before
publishing metrics.**

To hydrate the full v28 tables, first discover the current CMS zip URL on:
  https://www.cms.gov/medicare/payment/medicare-advantage-rates-statistics/risk-adjustment/2024-model-software/icd-10-mappings

Then run:

```bash
# (optional) inspect the zip contents first
uv run python scripts/fetch_cms_hcc_v28.py inspect \
    --zip-url "https://www.cms.gov/files/zip/<discovered>.zip"

uv run python scripts/fetch_cms_hcc_v28.py hydrate \
    --zip-url     "https://www.cms.gov/files/zip/<discovered>.zip" \
    --icd10-csv-url "https://www.cdc.gov/nchs/data/icd10cm/<year>/<file>.csv" \
    --out data/reference/
```

The script:
1. Downloads both archives, logs SHA-256.
2. Extracts the v28 zip and auto-classifies the crosswalk / labels / hierarchy
   tables by column-header sniffing (no filename dependency).
3. Writes a `PROVENANCE.lock` next to the CSVs with URLs + digests.

Normalizers for CMS's specific column names are TODO — the first run will stop
at the normalization step and print which classified files were found, so
Charles can inspect the headers and add column mappings in one shot.

## How to verify the seed data

Run `pytest tests/test_reference_data.py` — it asserts:
- Every HCC in `hcc_v28_labels.csv` appears in `hcc_v28_crosswalk.csv` at least once.
- Every ICD-10 in `hcc_v28_crosswalk.csv` has a description in `icd10_cm_descriptions.csv`.
- Every HCC hierarchy group in `hcc_v28_hierarchy.csv` has members in `hcc_v28_labels.csv`.

## License statement for downstream repos

The crosswalk tables are works of the U.S. federal government and not copyrightable under
17 U.S.C. § 105. Redistribution with attribution to CMS is permitted.