medical_coding / README.md
shashanks's picture
Upload folder using huggingface_hub
8410397 verified
|
Raw History Blame Contribute Delete
14.2 kB
metadata
title: Medical Coding Auditor
emoji: πŸ₯
colorFrom: blue
colorTo: indigo
sdk: docker
app_port: 7860
pinned: false
tags:
  - openenv
base_path: /web

Medical Coding Auditor β€” OpenEnv Environment

A real-world reinforcement learning environment that simulates a hospital pre-bill compliance review. The AI agent acts as a Medical Coding Auditor, reviewing proposed ICD-10-CM and CPT billing codes against clinical notes and patient demographics to identify sequencing violations, demographic mismatches, and mutually exclusive code conflicts.

Motivation

Medical coding errors cost the US healthcare system billions of dollars annually in claim denials, penalties, and compliance audits. A skilled auditor must cross-reference ICD-10-CM guidelines, CMS NCCI (National Correct Coding Initiative) edit tables, and patient demographics β€” a task ideal for evaluating multi-step reasoning in language model agents.

This environment tests:

  • Demographic awareness β€” recognizing codes inapplicable to a patient's sex or age
  • Guideline reasoning β€” interpreting official ICD-10-CM Excludes1/2 notes
  • Regulatory knowledge β€” applying CMS NCCI PTP (Procedure-to-Procedure) bundling rules
  • 7th character specificity β€” validating encounter-type extensions for injury codes
  • Evidence grounding β€” extracting exact clinical note text to support coding decisions
  • Disambiguation β€” asking physician clarification questions when documentation is ambiguous
  • Hallucination resistance β€” penalizing references to codes outside the proposed set

Action Space

The agent selects one action per step from the following tools:

action_type Required Fields Description
query_guideline code Look up official ICD-10-CM or CPT coding guidelines, Excludes1/2 notes, and gender restrictions for a specific code.
check_ncci_edits code1, code2 Check the CMS NCCI PTP edit table to determine if two CPT codes have a bundling conflict (mutually exclusive).
flag_error code, error_type, justification Record a confirmed coding error in the draft audit report.
ask_clarifying_question question Ask the simulated physician for missing clinical information when the note is ambiguous. Returns a deterministic physician response.
extract_evidence evidence_text Highlight an exact text span from the clinical note as documentary evidence for a coding decision. Must be a verbatim substring of the note.
submit_audit (none) End the episode and submit the draft report for deterministic grading.

error_type values for flag_error:

Value Description
demographic_mismatch Code is inapplicable to the patient's sex or age (e.g., maternity code for a male patient).
excludes1_conflict Two mutually exclusive ICD-10-CM diagnosis codes are billed together (Excludes1 rule violation).
ncci_edit Two CPT codes have a CMS NCCI PTP bundling conflict; the component service is billed separately from its comprehensive code.
specificity_error Wrong 7th character extension (e.g., "initial encounter" used for a follow-up visit).
untraceable_code The code does not exist in any official ICD-10-CM or CPT code set.

Observation Space

Each step returns a MedicalCodingObservation with the following fields:

Field Type Description
task_id str Current task identifier.
difficulty str easy, medium, hard, or expert.
patient_demographics dict Age, sex, MRN, insurance carrier.
clinical_note str Unstructured clinical note documenting the encounter.
proposed_codes dict[code β†’ {description, code_type}] The billing codes to audit.
draft_report list[{code, error_type, justification, step}] Errors flagged so far.
tool_result str Text output of the last action (guideline text, NCCI result, physician response, etc.).
step_count int Steps taken this episode.
codes_queried list[str] Codes already queried via query_guideline (loop detection).
pairs_checked list[str] NCCI pairs already checked (formatted as code1|code2).
clarifications_asked list[str] Clarifying questions asked via ask_clarifying_question this episode.
extracted_evidence list[str] Text spans extracted from the clinical note via extract_evidence this episode.
last_action_error str | null Error message if the last action was invalid.
grader_score float | null Final grader score [0.0, 1.0] (set only after submit_audit).
episode_metrics dict | null Detailed trajectory stats (populated on done=True): trajectory length, tool failure rate, investigation coverage, flag precision/recall, avg reward per step, investigation-before-flag rate, clarification count.
reward float Reward for the last action.
done bool Whether the episode has ended.

Tasks

Five tasks spanning four difficulty levels:

Task 1 β€” Easy: Demographic Mismatch (easy_demographic)

Scenario: A 34-year-old male patient's annual physical exam chart contains a maternity-specific ICD-10 code (O80 β€” Encounter for full-term uncomplicated delivery) alongside valid codes E11.9 and Z00.00.

Objective: Query the guideline for O80, recognize the female-only restriction, and flag the demographic mismatch before submitting the audit.

Expected error: O80 β†’ demographic_mismatch

Max steps: 10


Task 2 β€” Medium: NCCI PTP Bundling Conflict (medium_ncci_conflict)

Scenario: A 67-year-old female's cardiology billing includes 93306 (complete transthoracic echocardiography) and 93307 (limited TTE), which are subject to a CMS NCCI Procedure-to-Procedure edit β€” a classic unbundling violation. Also includes a valid 99213 office visit.

Objective: Use check_ncci_edits to identify the bundling conflict, then flag the violation.

Expected error: 93306 β†’ ncci_edit

Max steps: 15


Task 3 β€” Medium: Excludes1 Conflict (medium_excludes1)

Scenario: A 58-year-old female presents with an acute COPD exacerbation. The proposed codes list both J44.1 (COPD with exacerbation) and J45.20 (mild intermittent asthma), which have an Excludes1 (mutually exclusive) relationship. A valid Z87.891 history code is present as a false-positive trap.

Objective: Query guidelines to identify the Excludes1 restriction, read the clinical note to confirm the asthma diagnosis was superseded, and flag the conflict.

Expected error: J44.1 β†’ excludes1_conflict

Max steps: 15


Task 4 β€” Hard: 7th Character Specificity + Untraceable Code (hard_specificity_untraceable)

Scenario: A 28-year-old male presents for a follow-up visit (6 weeks post-fracture). The proposed codes include:

  • S52.501A β€” uses 7th character A (initial encounter) for what the clinical note explicitly documents as a subsequent/follow-up encounter (should be S52.501D).
  • Z99.999 β€” a code that does not exist in any official ICD-10-CM code set.

Objective: Read the clinical note carefully, verify Z99.999 is untraceable, and flag both errors.

Expected errors: S52.501A β†’ specificity_error AND Z99.999 β†’ untraceable_code

Max steps: 20


Task 5 β€” Expert: Multi-Error Complex Encounter (expert_multi_error)

Scenario: A 65-year-old male has a comprehensive cardiology + endocrinology visit. The proposed 6 codes contain two distinct errors β€” an Excludes1 conflict between Type 1 and Type 2 diabetes (E10.9 + E11.9) and an NCCI PTP bundling violation between echocardiography codes (93306 + 93308). 99214 and M79.621 are valid and must NOT be flagged.

Objective: Identify both errors without false-positiving the valid codes.

Expected errors: E10.9 β†’ excludes1_conflict AND 93306 β†’ ncci_edit

Max steps: 25


Reward Function

The environment provides dense, multi-signal rewards throughout each episode:

Action Condition Reward
query_guideline Valid code, first query +0.10
query_guideline Code NOT in proposed set (hallucination) βˆ’0.50
query_guideline Code already queried (loop) βˆ’0.10
check_ncci_edits Valid pair, first check β€” no edit found +0.05
check_ncci_edits Valid pair, first check β€” edit found +0.10
check_ncci_edits Any code NOT in proposed set βˆ’0.50
ask_clarifying_question Relevant question (matches physician pool) +0.15
ask_clarifying_question Irrelevant or repeated question βˆ’0.10
extract_evidence Span is in note AND matches expected evidence +0.05
extract_evidence Span is in note but irrelevant Β±0.00
extract_evidence Span NOT in clinical note (hallucination) βˆ’0.05
flag_error Correct code + correct error_type (conflict type) +0.30 Γ— rarity
flag_error Correct code + correct error_type (other) +0.20 Γ— rarity
flag_error Correct code + prior evidence extracted additional +0.05
flag_error Correct code + justification contains key terms additional +0.05
flag_error Correct code + investigated before flagging additional +0.05
flag_error Correct code + no prior investigation βˆ’0.05
flag_error Correct code + wrong error_type (close miss) βˆ’0.05
flag_error Correct code + wrong error_type (far miss) βˆ’0.10
flag_error False positive (code has no expected error) βˆ’0.20
flag_error Code NOT in proposed set βˆ’0.50
submit_audit Terminal: grader score βˆ’ FP penalty + efficiency bonus [0.0 – 1.0]

Rarity multipliers (Asymmetric Loss β€” rare errors rewarded more): demographic_mismatch=1.0Γ—, ncci_edit/excludes1_conflict=1.2Γ—, untraceable_code=1.3Γ—, specificity_error=1.5Γ—

HERON-style hierarchical wrong-type penalty: close misclassifications (e.g. excludes1_conflict ↔ ncci_edit) penalized less (βˆ’0.05) than cross-category misclassifications (βˆ’0.10).

Grader (terminal)

The grader runs deterministically at submit_audit:

  • Full credit (1.0 Γ— rarity): correct code + correct error_type
  • HERON partial credit (0.4–0.55 Γ— rarity): correct code + wrong error_type (scaled by conceptual distance)
  • No credit (0.0): code not flagged at all
  • Score = sum(credits) / sum(rarity weights) β†’ range [0.0, 1.0]
  • Efficiency bonus: +0.1 Γ— (max_steps βˆ’ steps_used) / max_steps
  • False positive penalty: βˆ’0.15 Γ— false_positive_count
  • Thorough review bonus: +0.05 if all proposed codes were investigated before submit

Setup & Usage

Prerequisites

  • Python β‰₯ 3.10
  • Docker (for containerized deployment)
  • uv or pip for dependency management

Local Development

cd medical_coding_env

# Install dependencies
pip install "openenv-core[core]>=0.2.2" openai

# Start the server
uvicorn server.app:app --host 0.0.0.0 --port 7860
# or with uv:
uv run server

# Validate the environment
openenv validate .

Docker

cd medical_coding_env

# Build (using root Dockerfile β€” recommended)
docker build -t medical-coding-env:latest .

# Run
docker run -p 7860:7860 medical-coding-env:latest

Running Inference

export HF_TOKEN="your_hf_token"
export API_BASE_URL="https://router.huggingface.co/v1"
export MODEL_NAME="Qwen/Qwen2.5-72B-Instruct"
export ENV_BASE_URL="http://localhost:7860"

python inference.py

API Usage

import httpx

# Reset to a specific task
resp = httpx.post("http://localhost:7860/reset", json={"task_id": "easy_demographic"})
obs = resp.json()["observation"]
print(obs["clinical_note"])

# Execute an action
resp = httpx.post("http://localhost:7860/step", json={
    "action": {
        "action_type": "query_guideline",
        "code": "O80"
    }
})
print(resp.json()["observation"]["tool_result"])

Baseline Scores

Evaluated with Qwen/Qwen2.5-72B-Instruct via HuggingFace router at temperature 0.2:

Task Difficulty Expected Errors Score
easy_demographic Easy 1 ~0.75
medium_ncci_conflict Medium 1 ~0.65
medium_excludes1 Medium 1 ~0.65
hard_specificity_untraceable Hard 2 ~0.55
expert_multi_error Expert 2 ~0.40
Average β€” β€” ~0.60

(Scores are reproducible given the deterministic grader and fixed random seed.)


Project Structure

medical_coding_env/
β”œβ”€β”€ openenv.yaml               # OpenEnv spec manifest
β”œβ”€β”€ pyproject.toml             # Package config + uv/pip dependencies
β”œβ”€β”€ uv.lock                    # Reproducible dependency lockfile
β”œβ”€β”€ Dockerfile                 # Root Dockerfile (build context = project root)
β”œβ”€β”€ .dockerignore              # Excludes .git/, __pycache__, docs/ from image
β”œβ”€β”€ models.py                  # Action + Observation Pydantic models
β”œβ”€β”€ client.py                  # EnvClient (typed HTTP/WS client)
β”œβ”€β”€ inference.py               # Baseline inference script
β”œβ”€β”€ README.md                  # This file
β”œβ”€β”€ data/
β”‚   β”œβ”€β”€ ground_truth_cases.json  # ICD-10/CPT guidelines + 5 task scenarios
β”‚   └── case_generator.py        # Procedural random case generator (seeded)
└── server/
    β”œβ”€β”€ __init__.py
    β”œβ”€β”€ environment.py         # MedicalCodingEnvironment (reset/step/state)
    β”œβ”€β”€ app.py                 # FastAPI app (create_app wrapper)
    β”œβ”€β”€ requirements.txt       # Server runtime deps (pip fallback)
    └── Dockerfile             # Multi-stage Dockerfile (openenv-base pattern)

License

Open source under the MIT License. The ICD-10-CM codes and NCCI edit examples used in this environment are based on publicly available CMS government data.