hcc-meat-coder / data /reference /PROVENANCE.md
chraos's picture
Claude Opus 4.7 (1M context)
Milestone 0.3.0: coder-voiced README, benchmark button, NER feature flag, CMS fetcher
e43bca6
|
Raw History Blame Contribute Delete
2.75 kB
# Reference data provenance
All files in this directory are derived from public CMS / CDC / NLM sources.
None contain PHI. All are redistributable under the terms of their upstream license.
## Files
| File | Upstream | License | Notes |
|---|---|---|---|
| `icd10_cm_descriptions.csv` | CDC/CMS ICD-10-CM annual release | Public domain (US government work) | Tabular list — code + title only |
| `hcc_v28_labels.csv` | CMS Risk Adjustment — HCC v28 model | Public | HCC id, label, hierarchy group, RAF weight by segment |
| `hcc_v28_crosswalk.csv` | CMS Risk Adjustment — 2024 Mapping | Public | ICD-10 → HCC v28 mapping |
| `hcc_v28_hierarchy.csv` | CMS Risk Adjustment — v28 Hierarchy File | Public | Trumping rules (disease-group-level) |
## Current state
**⚠ The seed files in this directory are hand-curated PLACEHOLDERS for the ~15 most common HCCs,
intended to let the pipeline + tests run end-to-end without a network call. Every RAF weight
marked `PLACEHOLDER` in the CSV comments must be replaced with the official CMS value before
publishing metrics.**
To hydrate the full v28 tables, first discover the current CMS zip URL on:
https://www.cms.gov/medicare/payment/medicare-advantage-rates-statistics/risk-adjustment/2024-model-software/icd-10-mappings
Then run:
```bash
# (optional) inspect the zip contents first
uv run python scripts/fetch_cms_hcc_v28.py inspect \
--zip-url "https://www.cms.gov/files/zip/<discovered>.zip"
uv run python scripts/fetch_cms_hcc_v28.py hydrate \
--zip-url "https://www.cms.gov/files/zip/<discovered>.zip" \
--icd10-csv-url "https://www.cdc.gov/nchs/data/icd10cm/<year>/<file>.csv" \
--out data/reference/
```
The script:
1. Downloads both archives, logs SHA-256.
2. Extracts the v28 zip and auto-classifies the crosswalk / labels / hierarchy
tables by column-header sniffing (no filename dependency).
3. Writes a `PROVENANCE.lock` next to the CSVs with URLs + digests.
Normalizers for CMS's specific column names are TODO — the first run will stop
at the normalization step and print which classified files were found, so
Charles can inspect the headers and add column mappings in one shot.
## How to verify the seed data
Run `pytest tests/test_reference_data.py` — it asserts:
- Every HCC in `hcc_v28_labels.csv` appears in `hcc_v28_crosswalk.csv` at least once.
- Every ICD-10 in `hcc_v28_crosswalk.csv` has a description in `icd10_cm_descriptions.csv`.
- Every HCC hierarchy group in `hcc_v28_hierarchy.csv` has members in `hcc_v28_labels.csv`.
## License statement for downstream repos
The crosswalk tables are works of the U.S. federal government and not copyrightable under
17 U.S.C. § 105. Redistribution with attribution to CMS is permitted.