Spaces:

TheRubberDuckDebuggers
/

cascade-containment

Sleeping

App Files Files Community

cascade-containment / README.md

RohitChandramouli6618

Update model references: Llama 3.3 70B → Llama 3.1 8B Instant, runtime 19.8min → ~10min

1e308ec 7 months ago

preview code

Raw History Blame Contribute Delete

14 kB

---
title: Cascade Containment
emoji: 🦠
colorFrom: red
colorTo: blue
sdk: docker
app_port: 7860
pinned: false
---
## 🦠 An RL Benchmark for Sequential Resource Allocation Under Spreading Cascade Dynamics

[![OpenEnv](https://img.shields.io/badge/OpenEnv-Compliant-blue?style=flat-square)](https://github.com/meta-pytorch/OpenEnv)
[![Python](https://img.shields.io/badge/Python-3.10%2B-blue?style=flat-square)](https://python.org)
[![Docker](https://img.shields.io/badge/Docker-Ready-blue?style=flat-square)](https://hub.docker.com)
[![HF Space](https://img.shields.io/badge/HF%20Space-Live-green?style=flat-square)](https://huggingface.co/spaces/TheRubberDuckDebuggers/cascade-containment)
[![License](https://img.shields.io/badge/License-MIT-green?style=flat-square)](LICENSE)

Meta PyTorch OpenEnv Hackathon × SST 2026 — [Live Demo](https://therubberduckdebuggers-cascade-containment.hf.space) · [GitHub](https://github.com/Rohitchandramouli/cascade-containment)

---

## The Problem

Sequential resource allocation under uncertainty is one of the most consequential decision problems in the real world. Whether containing an epidemic, deploying firefighting crews, isolating a cyberattack, or routing aid — the agent faces the same fundamental challenge:

- **Resources are scarce** — you cannot cover every district simultaneously
- **Data is delayed** — by the time a crisis is visible, it has already grown
- **Interventions cascade** — actions in one district affect adjacent ones
- **Acting too late is catastrophic** — hospital collapse ends the episode; proactive containment is rewarded exponentially more than reactive response

No existing OpenEnv benchmark formalises this problem class. Cascade Containment does.

---

## Environment Overview

A city health authority must allocate limited medical resources across districts to contain a spreading outbreak. Each step, the agent observes district infection rates (possibly lagged), hospital capacity levels, and growth signals — then decides where to deploy resources, impose restrictions, or gather data.

The environment is **not epidemic-specific**. The underlying mechanics — spreading cascade, delayed observation, resource scarcity, geographic spillover — are structurally identical across multiple real-world domains.

---

## Quick Start

```python
from client import CascadeContainmentEnv
from models import ContainmentAction
with CascadeContainmentEnv(
    base_url="https://therubberduckdebuggers-cascade-containment.hf.space"
).sync() as env:
    result = env.reset(task_name="medium")
    obs    = result.observation
    while not result.done:
        most_infected = max(obs.districts, key=lambda d: d.reported_infection_rate)
        action = ContainmentAction(
            action_type="allocate",
            district_id=most_infected.district_id
        )
        result = env.step(action)
        obs    = result.observation
        print(f"Step {obs.current_step}: reward={result.reward:.3f}")
```

## Running the Full Baseline Evaluation

```bash
export API_BASE_URL="https://router.huggingface.co/v1"
export MODEL_NAME="meta-llama/Llama-3.1-8B-Instruct"
export HF_TOKEN="hf_your_token_here"
export ENV_BASE_URL="https://therubberduckdebuggers-cascade-containment.hf.space"
# Full LLM+GRPO evaluation (~10 minutes, emits [START][STEP][END] logs)
python inference.py
# Local validation + greedy benchmark
python scripts/test_local.py
```

---

## Action Space

| Field | Type | Values |
| --- | --- | --- |
| `action_type` | `string` | `"test"` · `"restrict"` · `"allocate"` |
| `district_id` | `int` | 0-indexed district target |

| Action | Cost | Effect |
| --- | --- | --- |
| **test** | 1 resource | Reveals accurate current infection data for district |
| **restrict** | Free | Imposes movement restrictions; reduces spread rate; penalised if infection < 0.20 |
| **allocate** | 1 resource | Deploys medical resources; reduces existing infection by 5% and slows future spread |

Movement restrictions lift automatically once a district's infection drops below the safe threshold.

---

## Observation Space

The agent receives a filtered, potentially lagged view of the world — **never the full ground truth**:

```python
CityObservation:
  districts:            List[DistrictObservation]  # per-district visible state
  available_resources:  int                         # budget remaining this step
  current_step:         int
  max_steps:            int
  done:                 bool
  reward:               float | None
  message:              str | None
```

| Field | Description | Observability |
| --- | --- | --- |
| `reported_infection_rate` | Active infection fraction | Real-time (easy/medium); **3-day lagged** (hard) |
| `growth_rate_hint` | Noisy signal of true spread rate | Always real-time ± noise |
| `hospital_capacity_remaining` | ICU/ward capacity fraction | Always real-time |
| `population_density` | District's share of city population | Always real-time |
| `restriction_active` | Whether movement restrictions are in place | Always real-time |
| `tested_recently` | Tested within last 2 days | Always real-time |

---

## Epidemiological Model

```text
new_infection = current + (spread_rate − natural_recovery − intervention) + geographic_spillover
```

| Parameter | Value | Rationale |
| --- | --- | --- |
| Spread rate | 3–8% per day | Realistic for respiratory outbreaks (seasonal flu: 5–10%) |
| Natural recovery | 1% per day | Background case resolution without medical intervention |
| Treatment effect | −5% existing infection | Medical deployment (antivirals, PPE, rapid response) |
| Spread reduction | −10% per allocation | Resource-driven suppression of transmission |
| Geographic spillover | 1% to adjacent districts | Linear topology — no wrap-around (geographically realistic) |
| Hospital breach threshold | ≤10% capacity | Real ICU overflow and triage failure threshold |

---

## Three Tasks

| Task | Districts | Steps | Resources | Data Lag | Challenge |
| --- | --- | --- | --- | --- | --- |
| **easy** | 2 | 10 | 10 | None | Single outbreak; D1 starts infected, D0 is clean |
| **medium** | 4 | 15 | 8 | None | Two simultaneous outbreaks; forced triage between competing threats |
| **hard** | 6 | 15 | 7 | **3 days** | Six seeded infections (only D2 and D4 above safe threshold); 3-day data lag; scarce resources |

**Easy** — D1 starts at 0.50 infection, D0 is clean. The agent must observe and target the correct district. A fixed-target agent ignoring observations scores ~43% and breaches hospitals 60% of the time.

**Medium** — D0 starts above the infection threshold (0.42); D2 is in the warning zone (0.38, below the 0.40 threshold). D1 and D3 start low but grow into crisis within 4–6 steps via spillover. With 8 resources across 4 districts over 15 steps, genuine triage is required.

**Hard** — 3-day information lag means the agent sees infection rates from 3 days ago. The `growth_rate_hint` provides a noisy signal to estimate current state. Structural uncertainty — not testable around.

---

## Reward Function

| Term | Value | Fires When |
| --- | --- | --- |
| Infection penalty | `−0.50 × density` | District infection > 0.40 |
| Hospital breach | `−1.00` | Hospital capacity ≤ 10% |
| Early containment | `+0.50 × (1 − step/max_steps)` | District infection < 0.20 |
| Correct prioritisation | `+0.30` | Allocate to highest-infected district |
| Unnecessary restriction | `−0.20` | Restrict district below 0.20 |

Key design choices: early containment bonus decays over time (proactive action worth more); infection penalty scales with population density (realistic triage); hospital breach fires at 10% not 0% (real ICU thresholds); restrictions auto-lift when infection drops below safe threshold.

---

## Grader

Fully deterministic — no randomness, no LLM calls. Identical trajectories always produce identical scores in `[0.0, 1.0]`.

| Component | Weight | Measures |
| --- | --- | --- |
| **Hospital score** | 45% | Average capacity preserved; ×0.6 multiplier if any district collapsed |
| **Containment score** | 30% | Fraction of district-days below infection threshold (first 2 steps excluded) |
| **Efficiency score** | 15% | Fraction of resource actions targeting highest-infected district (grader uses pre-action state to avoid penalising successful treatments) |
| **Speed score** | 10% | `1 − (steps / max_steps)` if episode ends early; else 0 |

Hospital is weighted highest because system capacity preservation is the primary operational constraint in real outbreak response — a functioning healthcare system is the prerequisite for everything else.

---

## Baseline Agent — GRPO-Style Episodic Memory

No weight updates, no gradient computation. The prompt is the policy; memory updates are the policy improvement.

### Learning Loop

```text
Rollout 1: Base prompt, no prior knowledge
  compute advantage = R1 - mean([])
  store steps with reward > -0.3 into EpisodicMemory
Rollout 2: Memory-augmented prompt
  retrieve top-5 similar past decisions by L1 distance on infection profiles
  inject as concrete examples into prompt
  compute advantage = R2 - mean([R1])
  reinforce if advantage > -0.5
... repeat for N rollouts (easy=2, medium=3, hard=3)
Report best grader score across all rollouts
```

### Benchmark Scores

Results from `baseline/run.py` (Llama 3.1 8B Instant via Groq, runtime ~10 minutes):

| Task | Greedy (D0) | LLM+GRPO | Lift |
| --- | --- | --- | --- |
| Easy | 42.8% (breach 60%) | **90.8%** | +48pp |
| Medium | 39.6% (breach 100%) | **78.0%** | +38pp |
| Hard | 35.3% (breach 100%) | **61.1%** | +26pp |
| **Average** | **39.2%** | **75.9%** | **+37pp** |

The greedy baseline (always allocates to D0) scores 33–43% with 60–100% hospital breach rates — no trivial exploit path. LLM+GRPO reaches 66–91% with zero breaches, demonstrating that genuine triage reasoning is required and rewarded.

---

## Generalisation

| Domain | Spreading cascade | Delayed data | Resource scarcity |
| --- | --- | --- | --- |
| 🦠 **Epidemic containment** | Infection spreads between districts | Lagged case counts | Medical resources |
| 🔥 **Wildfire deployment** | Fire spreads across terrain | Satellite update delay | Firefighting crews |
| 🛡️ **Cyberattack isolation** | Lateral movement between systems | Detection lag | Security team hours |
| 📢 **Misinformation containment** | Narrative spread through networks | Viral detection lag | Correction budget |
| 🤝 **Poverty intervention** | Deprivation cascades through communities | Census data lag | Aid allocation |

---

## Project Structure

```text
cascade-containment/
├── inference.py              # Evaluation entry point (mandatory [START][STEP][END] logs)
├── models.py                 # Typed data contracts: Action, Observation, State
├── client.py                 # OpenEnv client interface
├── openenv.yaml              # Environment manifest for OpenEnv registry
│
├── server/
│   ├── app.py                # FastAPI server + judge dashboard + /grade /info /demo /validate endpoints
│   ├── environment.py        # Core RL loop (reset/step/state OpenEnv interface)
│   ├── grader.py             # Deterministic trajectory scorer — no LLM calls
│   ├── constants.py          # Single source of truth for all numeric configuration
│   ├── utils.py              # Spread computation, observation builder, helper functions
│   ├── Dockerfile            # Container definition
│   └── tasks/
│       ├── task_easy.py      # 2 districts, 10 steps, real-time data (D1 seeded)
│       ├── task_medium.py    # 4 districts, 15 steps, forced triage
│       └── task_hard.py      # 6 districts, 15 steps, 3-day data lag
│
├── baseline/
│   ├── policy.py             # LLM policy with chain-of-thought prompting
│   ├── evaluator.py          # GRPO episodic memory loop (easy=2, medium=3, hard=3 rollouts)
│   └── run.py                # CLI entry point
│
├── scripts/
│   └── test_local.py         # Phase 1 spec checks + greedy benchmark + variance analysis
│
└── core/
    ├── trajectory.py         # EpisodicMemory — L1 similarity retrieval, phase-weighted
    ├── reward.py             # Score normalisation utilities
    └── policy_update.py      # Advantage computation, memory gating (threshold -0.3)
```

---

## OpenEnv Compliance

| Requirement | Status |
| --- | --- |
| `reset()` returns `CityObservation` | ✅ |
| `step(action)` returns `CityObservation` | ✅ |
| `state` property returns `State` | ✅ |
| Typed `Action` subclass | ✅ `ContainmentAction(Action)` |
| Typed `Observation` subclass | ✅ `CityObservation(Observation)` |
| `openenv.yaml` manifest | ✅ |
| Dockerfile builds | ✅ |
| HF Space deploys | ✅ |
| `inference.py` at root | ✅ |
| `[START][STEP][END]` structured logs | ✅ |
| Runtime < 20 minutes | ✅ ~~10 minutes |
| `API_BASE_URL`, `MODEL_NAME`, `HF_TOKEN` env vars | ✅ |
| OpenAI client for all LLM calls | ✅ |
| Grader scores in `[0.0, 1.0]` | ✅ |
| 3+ tasks with difficulty progression | ✅ |
| Phase 1 automated validation | ✅ 10/10 checks pass |

---

## Setup and Local Development

### Local Server

```bash
pip install -r requirements.txt
uvicorn server.app:app --host 0.0.0.0 --port 7860
export ENV_BASE_URL=http://localhost:7860
python baseline/run.py
```

### Docker

```bash
docker build -t cascade-containment .
docker run -p 7860:7860 cascade-containment
```

---

## Tags

`reinforcement-learning` · `resource-allocation` · `sequential-decision-making` · `partial-observability` · `cascade-dynamics` · `epidemic-response` · `openenv` · `llm-agent` · `grpo` · `episodic-memory` · `triage` · `multi-district` · `docker` · `fastapi`

	---
	title: Cascade Containment
	emoji: 🦠
	colorFrom: red
	colorTo: blue
	sdk: docker
	app_port: 7860
	pinned: false
	---

	## 🦠 An RL Benchmark for Sequential Resource Allocation Under Spreading Cascade Dynamics

	[![OpenEnv](https://img.shields.io/badge/OpenEnv-Compliant-blue?style=flat-square)](https://github.com/meta-pytorch/OpenEnv)
	[![Python](https://img.shields.io/badge/Python-3.10%2B-blue?style=flat-square)](https://python.org)
	[![Docker](https://img.shields.io/badge/Docker-Ready-blue?style=flat-square)](https://hub.docker.com)
	[![HF Space](https://img.shields.io/badge/HF%20Space-Live-green?style=flat-square)](https://huggingface.co/spaces/TheRubberDuckDebuggers/cascade-containment)
	[![License](https://img.shields.io/badge/License-MIT-green?style=flat-square)](LICENSE)

	Meta PyTorch OpenEnv Hackathon × SST 2026 — [Live Demo](https://therubberduckdebuggers-cascade-containment.hf.space) · [GitHub](https://github.com/Rohitchandramouli/cascade-containment)

	---

	## The Problem

	Sequential resource allocation under uncertainty is one of the most consequential decision problems in the real world. Whether containing an epidemic, deploying firefighting crews, isolating a cyberattack, or routing aid — the agent faces the same fundamental challenge:

	- Resources are scarce — you cannot cover every district simultaneously
	- Data is delayed — by the time a crisis is visible, it has already grown
	- Interventions cascade — actions in one district affect adjacent ones
	- Acting too late is catastrophic — hospital collapse ends the episode; proactive containment is rewarded exponentially more than reactive response

	No existing OpenEnv benchmark formalises this problem class. Cascade Containment does.

	---

	## Environment Overview

	A city health authority must allocate limited medical resources across districts to contain a spreading outbreak. Each step, the agent observes district infection rates (possibly lagged), hospital capacity levels, and growth signals — then decides where to deploy resources, impose restrictions, or gather data.

	The environment is not epidemic-specific. The underlying mechanics — spreading cascade, delayed observation, resource scarcity, geographic spillover — are structurally identical across multiple real-world domains.

	---

	## Quick Start

	```python
	from client import CascadeContainmentEnv
	from models import ContainmentAction

	with CascadeContainmentEnv(
	base_url="https://therubberduckdebuggers-cascade-containment.hf.space"
	).sync() as env:
	result = env.reset(task_name="medium")
	obs = result.observation

	while not result.done:
	most_infected = max(obs.districts, key=lambda d: d.reported_infection_rate)
	action = ContainmentAction(
	action_type="allocate",
	district_id=most_infected.district_id
	)
	result = env.step(action)
	obs = result.observation
	print(f"Step {obs.current_step}: reward={result.reward:.3f}")
	```

	## Running the Full Baseline Evaluation

	```bash
	export API_BASE_URL="https://router.huggingface.co/v1"
	export MODEL_NAME="meta-llama/Llama-3.1-8B-Instruct"
	export HF_TOKEN="hf_your_token_here"
	export ENV_BASE_URL="https://therubberduckdebuggers-cascade-containment.hf.space"

	# Full LLM+GRPO evaluation (~10 minutes, emits [START][STEP][END] logs)
	python inference.py

	# Local validation + greedy benchmark
	python scripts/test_local.py
	```

	---

	## Action Space

	\| Field \| Type \| Values \|
	\| --- \| --- \| --- \|
	\| `action_type` \| `string` \| `"test"` · `"restrict"` · `"allocate"` \|
	\| `district_id` \| `int` \| 0-indexed district target \|

	\| Action \| Cost \| Effect \|
	\| --- \| --- \| --- \|
	\| test \| 1 resource \| Reveals accurate current infection data for district \|
	\| restrict \| Free \| Imposes movement restrictions; reduces spread rate; penalised if infection < 0.20 \|
	\| allocate \| 1 resource \| Deploys medical resources; reduces existing infection by 5% and slows future spread \|

	Movement restrictions lift automatically once a district's infection drops below the safe threshold.

	---

	## Observation Space

	The agent receives a filtered, potentially lagged view of the world — never the full ground truth:

	```python
	CityObservation:
	districts: List[DistrictObservation] # per-district visible state
	available_resources: int # budget remaining this step
	current_step: int
	max_steps: int
	done: bool
	reward: float \| None
	message: str \| None
	```

	\| Field \| Description \| Observability \|
	\| --- \| --- \| --- \|
	\| `reported_infection_rate` \| Active infection fraction \| Real-time (easy/medium); 3-day lagged (hard) \|
	\| `growth_rate_hint` \| Noisy signal of true spread rate \| Always real-time ± noise \|
	\| `hospital_capacity_remaining` \| ICU/ward capacity fraction \| Always real-time \|
	\| `population_density` \| District's share of city population \| Always real-time \|
	\| `restriction_active` \| Whether movement restrictions are in place \| Always real-time \|
	\| `tested_recently` \| Tested within last 2 days \| Always real-time \|

	---

	## Epidemiological Model

	```text
	new_infection = current + (spread_rate − natural_recovery − intervention) + geographic_spillover
	```

	\| Parameter \| Value \| Rationale \|
	\| --- \| --- \| --- \|
	\| Spread rate \| 3–8% per day \| Realistic for respiratory outbreaks (seasonal flu: 5–10%) \|
	\| Natural recovery \| 1% per day \| Background case resolution without medical intervention \|
	\| Treatment effect \| −5% existing infection \| Medical deployment (antivirals, PPE, rapid response) \|
	\| Spread reduction \| −10% per allocation \| Resource-driven suppression of transmission \|
	\| Geographic spillover \| 1% to adjacent districts \| Linear topology — no wrap-around (geographically realistic) \|
	\| Hospital breach threshold \| ≤10% capacity \| Real ICU overflow and triage failure threshold \|

	---

	## Three Tasks

	\| Task \| Districts \| Steps \| Resources \| Data Lag \| Challenge \|
	\| --- \| --- \| --- \| --- \| --- \| --- \|
	\| easy \| 2 \| 10 \| 10 \| None \| Single outbreak; D1 starts infected, D0 is clean \|
	\| medium \| 4 \| 15 \| 8 \| None \| Two simultaneous outbreaks; forced triage between competing threats \|
	\| hard \| 6 \| 15 \| 7 \| 3 days \| Six seeded infections (only D2 and D4 above safe threshold); 3-day data lag; scarce resources \|

	Easy — D1 starts at 0.50 infection, D0 is clean. The agent must observe and target the correct district. A fixed-target agent ignoring observations scores ~43% and breaches hospitals 60% of the time.

	Medium — D0 starts above the infection threshold (0.42); D2 is in the warning zone (0.38, below the 0.40 threshold). D1 and D3 start low but grow into crisis within 4–6 steps via spillover. With 8 resources across 4 districts over 15 steps, genuine triage is required.

	Hard — 3-day information lag means the agent sees infection rates from 3 days ago. The `growth_rate_hint` provides a noisy signal to estimate current state. Structural uncertainty — not testable around.

	---

	## Reward Function

	\| Term \| Value \| Fires When \|
	\| --- \| --- \| --- \|
	\| Infection penalty \| `−0.50 × density` \| District infection > 0.40 \|
	\| Hospital breach \| `−1.00` \| Hospital capacity ≤ 10% \|
	\| Early containment \| `+0.50 × (1 − step/max_steps)` \| District infection < 0.20 \|
	\| Correct prioritisation \| `+0.30` \| Allocate to highest-infected district \|
	\| Unnecessary restriction \| `−0.20` \| Restrict district below 0.20 \|

	Key design choices: early containment bonus decays over time (proactive action worth more); infection penalty scales with population density (realistic triage); hospital breach fires at 10% not 0% (real ICU thresholds); restrictions auto-lift when infection drops below safe threshold.

	---

	## Grader

	Fully deterministic — no randomness, no LLM calls. Identical trajectories always produce identical scores in `[0.0, 1.0]`.

	\| Component \| Weight \| Measures \|
	\| --- \| --- \| --- \|
	\| Hospital score \| 45% \| Average capacity preserved; ×0.6 multiplier if any district collapsed \|
	\| Containment score \| 30% \| Fraction of district-days below infection threshold (first 2 steps excluded) \|
	\| Efficiency score \| 15% \| Fraction of resource actions targeting highest-infected district (grader uses pre-action state to avoid penalising successful treatments) \|
	\| Speed score \| 10% \| `1 − (steps / max_steps)` if episode ends early; else 0 \|

	Hospital is weighted highest because system capacity preservation is the primary operational constraint in real outbreak response — a functioning healthcare system is the prerequisite for everything else.

	---

	## Baseline Agent — GRPO-Style Episodic Memory

	No weight updates, no gradient computation. The prompt is the policy; memory updates are the policy improvement.

	### Learning Loop

	```text
	Rollout 1: Base prompt, no prior knowledge
	compute advantage = R1 - mean([])
	store steps with reward > -0.3 into EpisodicMemory

	Rollout 2: Memory-augmented prompt
	retrieve top-5 similar past decisions by L1 distance on infection profiles
	inject as concrete examples into prompt
	compute advantage = R2 - mean([R1])
	reinforce if advantage > -0.5

	... repeat for N rollouts (easy=2, medium=3, hard=3)
	Report best grader score across all rollouts
	```

	### Benchmark Scores

	Results from `baseline/run.py` (Llama 3.1 8B Instant via Groq, runtime ~10 minutes):

	\| Task \| Greedy (D0) \| LLM+GRPO \| Lift \|
	\| --- \| --- \| --- \| --- \|
	\| Easy \| 42.8% (breach 60%) \| 90.8% \| +48pp \|
	\| Medium \| 39.6% (breach 100%) \| 78.0% \| +38pp \|
	\| Hard \| 35.3% (breach 100%) \| 61.1% \| +26pp \|
	\| Average \| 39.2% \| 75.9% \| +37pp \|

	The greedy baseline (always allocates to D0) scores 33–43% with 60–100% hospital breach rates — no trivial exploit path. LLM+GRPO reaches 66–91% with zero breaches, demonstrating that genuine triage reasoning is required and rewarded.

	---

	## Generalisation

	\| Domain \| Spreading cascade \| Delayed data \| Resource scarcity \|
	\| --- \| --- \| --- \| --- \|
	\| 🦠 Epidemic containment \| Infection spreads between districts \| Lagged case counts \| Medical resources \|
	\| 🔥 Wildfire deployment \| Fire spreads across terrain \| Satellite update delay \| Firefighting crews \|
	\| 🛡️ Cyberattack isolation \| Lateral movement between systems \| Detection lag \| Security team hours \|
	\| 📢 Misinformation containment \| Narrative spread through networks \| Viral detection lag \| Correction budget \|
	\| 🤝 Poverty intervention \| Deprivation cascades through communities \| Census data lag \| Aid allocation \|

	---

	## Project Structure

	```text
	cascade-containment/
	├── inference.py # Evaluation entry point (mandatory [START][STEP][END] logs)
	├── models.py # Typed data contracts: Action, Observation, State
	├── client.py # OpenEnv client interface
	├── openenv.yaml # Environment manifest for OpenEnv registry
	│
	├── server/
	│ ├── app.py # FastAPI server + judge dashboard + /grade /info /demo /validate endpoints
	│ ├── environment.py # Core RL loop (reset/step/state OpenEnv interface)
	│ ├── grader.py # Deterministic trajectory scorer — no LLM calls
	│ ├── constants.py # Single source of truth for all numeric configuration
	│ ├── utils.py # Spread computation, observation builder, helper functions
	│ ├── Dockerfile # Container definition
	│ └── tasks/
	│ ├── task_easy.py # 2 districts, 10 steps, real-time data (D1 seeded)
	│ ├── task_medium.py # 4 districts, 15 steps, forced triage
	│ └── task_hard.py # 6 districts, 15 steps, 3-day data lag
	│
	├── baseline/
	│ ├── policy.py # LLM policy with chain-of-thought prompting
	│ ├── evaluator.py # GRPO episodic memory loop (easy=2, medium=3, hard=3 rollouts)
	│ └── run.py # CLI entry point
	│
	├── scripts/
	│ └── test_local.py # Phase 1 spec checks + greedy benchmark + variance analysis
	│
	└── core/
	├── trajectory.py # EpisodicMemory — L1 similarity retrieval, phase-weighted
	├── reward.py # Score normalisation utilities
	└── policy_update.py # Advantage computation, memory gating (threshold -0.3)
	```

	---

	## OpenEnv Compliance

	\| Requirement \| Status \|
	\| --- \| --- \|
	\| `reset()` returns `CityObservation` \| ✅ \|
	\| `step(action)` returns `CityObservation` \| ✅ \|
	\| `state` property returns `State` \| ✅ \|
	\| Typed `Action` subclass \| ✅ `ContainmentAction(Action)` \|
	\| Typed `Observation` subclass \| ✅ `CityObservation(Observation)` \|
	\| `openenv.yaml` manifest \| ✅ \|
	\| Dockerfile builds \| ✅ \|
	\| HF Space deploys \| ✅ \|
	\| `inference.py` at root \| ✅ \|
	\| `[START][STEP][END]` structured logs \| ✅ \|
	\| Runtime < 20 minutes \| ✅ ~~10 minutes \|
	\| `API_BASE_URL`, `MODEL_NAME`, `HF_TOKEN` env vars \| ✅ \|
	\| OpenAI client for all LLM calls \| ✅ \|
	\| Grader scores in `[0.0, 1.0]` \| ✅ \|
	\| 3+ tasks with difficulty progression \| ✅ \|
	\| Phase 1 automated validation \| ✅ 10/10 checks pass \|

	---

	## Setup and Local Development

	### Local Server

	```bash
	pip install -r requirements.txt
	uvicorn server.app:app --host 0.0.0.0 --port 7860

	export ENV_BASE_URL=http://localhost:7860
	python baseline/run.py
	```

	### Docker

	```bash
	docker build -t cascade-containment .
	docker run -p 7860:7860 cascade-containment
	```

	---

	## Tags

	`reinforcement-learning` · `resource-allocation` · `sequential-decision-making` · `partial-observability` · `cascade-dynamics` · `epidemic-response` · `openenv` · `llm-agent` · `grpo` · `episodic-memory` · `triage` · `multi-district` · `docker` · `fastapi`