Upload folder using huggingface_hub

CLAUDE.md

@@ -0,0 +1,247 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+---
+
+## Hackathon Submission Requirements
+
+This project is being submitted to a hackathon. Every change must keep the following requirements satisfied — never regress them.
+
+### Mandatory env variables (inference.py must use exactly these names)
+
+| Variable | Purpose |
+|---|---|
+| `HF_TOKEN` | API key passed to OpenAI client (`api_key=`) |
+| `API_BASE_URL` | LLM base URL passed to OpenAI client (`base_url=`) |
+| `MODEL_NAME` | Model identifier used for all LLM calls |
+
+`STORE_ENV_URL` is a separate optional variable for the environment server URL (default: `http://localhost:8000`). Do not confuse it with `API_BASE_URL`.
+
+### Structured log format (exact field names required — do not rename)
+
+```
+[START] {"task": "...", "seed": N, "model": "...", "max_steps": N, "profit_target": N}
+[STEP]  {"task": "...", "step": N, "action": {...}, "reward": ..., "cumulative_profit": ...}
+[END]   {"task": "...", "cumulative_profit": ..., "score": ..., "profit_target": ...}
+```
+
+Extra fields in `[STEP]` (e.g. `holding_cost`, `placement_cost`, `restock_cost`) are allowed but the above fields must always be present with exact names.
+
+### Pre-submission checklist
+
+- [ ] `HF_TOKEN`, `API_BASE_URL`, `MODEL_NAME` read by inference.py ✅
+- [ ] `inference.py` in root directory ✅
+- [ ] `[START]`/`[STEP]`/`[END]` logs emitted to stdout ✅
+- [ ] 3 tasks (easy/medium/hard) with deterministic graders scoring 0.0–1.0 ✅
+- [ ] `openenv.yaml` present and valid ✅
+- [ ] `server/Dockerfile` builds cleanly ✅
+- [ ] `reset()`, `step()`, `state()` endpoints exposed via `create_app()` ✅
+- [ ] Typed Pydantic models for Action, Observation, State ✅
+- [ ] Meaningful reward function (non-sparse, multiple components) ✅
+- [ ] README has: env description, action/obs space, task descriptions, setup instructions, baseline scores ✅
+- [ ] Inference runtime < 20 min (3 tasks × ≤30 steps × fast model) ✅
+- [ ] Runs on vcpu=2, memory=8gb (pure Python, no GPU dependencies) ✅
+
+### Observation extraction in inference.py (do not regress)
+
+The WebSocket server sends: `{"type": "observation", "data": {"observation": {...}, "done": bool, "reward": float}}`.
+
+The `_unpack(msg)` helper in `inference.py` correctly extracts the inner observation and injects `done`/`reward` at the top level. **Never replace this with `msg.get("data", msg)` directly** — that returns the outer wrapper and `obs.get("inventory", [])` would always be `[]`, scoring every task 0.0.
+
+### Scoring weights (for prioritizing what to improve)
+
+| Criterion | Weight |
+|---|---|
+| Real-world utility | 30% |
+| Task & grader quality | 25% |
+| Environment design | 20% |
+| Code quality & spec compliance | 15% |
+| Creativity & novelty | 10% |
+
+---
+
+## Project Goal
+
+This is an **OpenEnv** reinforcement learning environment simulating a retail store manager. The RL task (see [docs/PROBLEM.md](docs/PROBLEM.md)) involves an LLM agent that manages store inventory to maximize profit. The agent can apply discounts, restock inventory, and move products between shelf zones. The environment is fully implemented in [server/StoreManager_environment.py](server/StoreManager_environment.py).
+
+## Common Commands
+
+```bash
+# Install dependencies
+uv sync
+
+# Run dev server (with auto-reload)
+uvicorn server.app:app --reload
+
+# Run server directly
+uv run --project . server
+python -m StoreManager.server.app --port 8000
+
+# Test environment logic directly (no HTTP server needed)
+python3 server/StoreManager_environment.py
+
+# Build Docker image
+docker build -t StoreManager-env:latest -f server/Dockerfile .
+
+# Deploy to Hugging Face Spaces
+openenv push
+openenv push --repo-id my-org/my-env --private
+
+# Run tests
+pytest
+pytest --cov
+```
+
+## Architecture
+
+The codebase follows the OpenEnv pattern: a FastAPI server wraps a Python environment class, and a typed client communicates via WebSocket.
+
+**Data flow:**
+1. Client (`client.py` → `StoremanagerEnv`) sends actions over a persistent WebSocket
+2. Server (`server/app.py`) receives them via `openenv.core.env_server.http_server.create_app`
+3. Environment (`server/StoreManager_environment.py` → `StoremanagerEnvironment`) processes actions and returns observations
+4. Models (`models.py`) define the Pydantic schemas for `StoremanagerAction` and `StoremanagerObservation`
+
+**Concurrency:** `SUPPORTS_CONCURRENT_SESSIONS = True` in the environment class; `max_concurrent_envs=1` in `server/app.py` (increase for parallel sessions).
+
+**Package layout:** The root directory is both the Python package root (`StoreManager/`) and the Docker build context. `PYTHONPATH=/app/env` is set in the container so imports resolve correctly.
+
+---
+
+## Implemented Mechanics
+
+### Core: Discount + Customer Simulation + Expiry
+
+The base mechanic. Each step the agent discounts one product; customers sample products weighted by `base_probability × zone_multiplier × (1 + discount_pct / 100)`. Products that reach their `expiry_step` are removed and trigger a penalty of `remaining_quantity × cost_price`.
+
+### Feature 1: Restocking / Procurement
+
+**What it models:** The agent can replenish stock by placing a purchase order. Stock doesn't arrive instantly — there is a configurable lead time. Holding inventory costs money every step regardless of whether it sells.
+
+**Action format:**
+```json
+{ "action_type": "restock", "product_id": 1, "restock_quantity": 15 }
+```
+
+**How it works in the environment (`server/StoreManager_environment.py`):**
+
+- `_pending_orders` is a list of dicts `{product_id, quantity, arrive_at_step}` maintained on the environment instance.
+- At the **start** of every `step()` call, pending orders whose `arrive_at_step <= _current_step` are delivered: `product.quantity += order["quantity"]` and the pending fields are cleared.
+- When a `"restock"` action is dispatched, a new entry is appended to `_pending_orders` with `arrive_at_step = _current_step + product.restock_lead_time`. The cost `restock_quantity × restock_cost_per_unit` is charged immediately from this step's profit.
+- `holding_cost` is computed every step for all active products: `sum(p.quantity × p.cost_price × p.holding_cost_rate for all active p)`. This is always charged, regardless of action type.
+
+**Per-product restock parameters (set randomly in `reset()`):**
+
+| Field | Range | Description |
+|---|---|---|
+| `restock_lead_time` | 1–3 steps | Steps until a placed order is delivered |
+| `restock_cost_per_unit` | `cost_price × 1.00–1.25` | Cost to order one additional unit |
+| `holding_cost_rate` | 0.01–0.04 | Fraction of `cost_price` charged per unit per step |
+
+**Observation fields added:**
+
+| Field | Description |
+|---|---|
+| `restock_cost_this_step` | Ordering cost charged this step (0 if no restock) |
+| `holding_cost_this_step` | Total holding cost charged this step |
+| `pending_restock_quantity` (on ProductState) | Units currently on order |
+| `pending_restock_arrives_at_step` (on ProductState) | Step when pending order delivers |
+
+**Error cases:** Returns an error observation (step does not advance) if:
+- `restock_quantity` is None or ≤ 0
+- The product already has a pending order (`pending_restock_quantity > 0`)
+
+**Strategic tension:** Over-restocking inflates holding costs and risks expiry penalties if the new stock can't sell through. Under-restocking leads to stockouts on high-demand products.
+
+---
+
+### Feature 2: Zone / Shelf Placement
+
+**What it models:** The physical location of a product on the shelf affects how often customers pick it. Premium zones (entrance, eye-level) get more foot traffic. Moving products costs staff labor.
+
+**Action format:**
+```json
+{ "action_type": "place", "product_id": 3, "target_zone": 1 }
+```
+
+**How it works in the environment:**
+
+- Each `ProductState` carries a `zone: int` field (1, 2, or 3).
+- When a `"place"` action is dispatched, the product's zone is updated and `PLACEMENT_COST = $3.00` is deducted from this step's profit.
+- The pick-probability formula uses the zone multiplier: `effective_weight[i] = base_probability[i] × ZONE_MULTIPLIERS[zone[i]] × (1 + discount_pct[i] / 100)`.
+
+**Zone constants (defined at the top of `StoreManager_environment.py`):**
+
+```python
+ZONE_MULTIPLIERS = {1: 1.5, 2: 1.0, 3: 0.6}
+ZONE_CAPACITY    = {1: 2, 2: 4, 3: 99}
+PLACEMENT_COST   = 3.0
+```
+
+**Initial zone assignment in `reset()`:** Products are assigned zones greedily in order — first 2 products go to zone 1, next 4 to zone 2, remainder to zone 3. This means zone 1 is always full at episode start.
+
+**Observation fields added:**
+
+| Field | Description |
+|---|---|
+| `zone_capacity` | `{1: 2, 2: 4, 3: 99}` — max products per zone |
+| `zone_occupancy` | `{1: N, 2: N, 3: N}` — active products currently in each zone |
+| `zone_multipliers` | `{1: 1.5, 2: 1.0, 3: 0.6}` — pick-prob multipliers |
+| `placement_cost_this_step` | $3.00 if a placement was made, else $0.00 |
+| `zone` (on ProductState) | Current zone of the product |
+
+**Error cases:** Returns an error observation (step does not advance) if:
+- `target_zone` is None or not in {1, 2, 3}
+- The product is already in `target_zone`
+- `target_zone` is already at capacity (`zone_occupancy[target_zone] >= ZONE_CAPACITY[target_zone]`)
+
+**Strategic tension:** Zone 1 has only 2 slots but gives a 1.5× boost — the agent must decide which products deserve premium placement at any given moment. Moving a product out to make room for a more urgent one costs $3.00 each way.
+
+---
+
+## Reward Formula (full)
+
+```
+step_reward = step_revenue
+            − cost_of_goods_sold
+            − expiry_penalty         # remaining_qty × cost_price for expired products
+            − holding_cost           # Σ(qty × cost_price × holding_cost_rate) all active products
+            − placement_cost         # $3.00 flat, only if action_type == "place"
+            − restock_cost           # restock_qty × restock_cost_per_unit, only if action_type == "restock"
+```
+
+All components are available individually in the `metadata` dict of each observation.
+
+---
+
+## Key Files
+
+| File | Role |
+|---|---|
+| [models.py](models.py) | Pydantic schemas: `ProductState`, `StoremanagerAction`, `StoremanagerObservation`, `StoremanagerState` |
+| [server/StoreManager_environment.py](server/StoreManager_environment.py) | Full environment logic: `reset()`, `step()`, zone constants, pending orders |
+| [client.py](client.py) | WebSocket client: `_step_payload()`, `_parse_result()`, `_parse_state()` |
+| [inference.py](inference.py) | LLM agent runner: system prompt, inventory table, action parsing for all 3 types |
+| [tasks.py](tasks.py) | Task configs (easy/medium/hard) + `grade()` function |
+
+---
+
+## OpenEnv Framework
+
+- Environments inherit from `openenv.core.env_server.interfaces.Environment`
+- Actions/Observations inherit from `openenv.core.env_server.types.Action` / `Observation`
+- `openenv.yaml` declares the runtime config; `create_app()` auto-generates `/reset`, `/step`, `/state`, `/schema` HTTP endpoints and a `/ws` WebSocket endpoint
+- The client base class `EnvClient` (from `openenv.core`) requires implementing `_step_payload()`, `_parse_result()`, and `_parse_state()`
+
+## Reference Implementation: Wordle
+
+[Examples/wordle/](Examples/wordle/) is a complete, working OpenEnv environment. Key things to learn from it:
+
+- **Richer models** ([Examples/wordle/models.py](Examples/wordle/models.py)): Shows how to add a `State` class with custom fields and nested sub-models inside an `Observation`.
+- **Non-trivial environment logic** ([Examples/wordle/server/environment.py](Examples/wordle/server/environment.py)): Shows how to manage external state, clear accumulated state on `reset()`, compute rewards, and snapshot internal state for the `state` property.
+- **Reward providers** ([Examples/wordle/rewards.py](Examples/wordle/rewards.py)): A clean pattern for separating reward computation into pluggable `RewardProvider` classes.
+- **Factory mode for concurrency** ([Examples/wordle/server/app.py](Examples/wordle/server/app.py)): Passes a factory function instead of a class to `create_app()` when environment construction needs arguments.
+- **OpenEnv core source** ([Examples/wordle/src/core/](Examples/wordle/src/core/)): The full source of `openenv-core` is vendored here.
+
+Also see [.claude/docs/PATTERNS.md](.claude/docs/PATTERNS.md) for the canonical OpenEnv coding conventions.

Dockerfile

@@ -0,0 +1,81 @@
+# Copyright (c) Meta Platforms, Inc. and affiliates.
+# All rights reserved.
+#
+# This source code is licensed under the BSD-style license found in the
+# LICENSE file in the root directory of this source tree.
+
+# Multi-stage build using openenv-base
+# This Dockerfile is flexible and works for both:
+# - In-repo environments (with local OpenEnv sources)
+# - Standalone environments (with openenv from PyPI/Git)
+# The build script (openenv build) handles context detection and sets appropriate build args.
+
+ARG BASE_IMAGE=ghcr.io/meta-pytorch/openenv-base:latest
+FROM ${BASE_IMAGE} AS builder
+
+WORKDIR /app
+
+# Ensure git is available (required for installing dependencies from VCS)
+RUN apt-get update && \
+    apt-get install -y --no-install-recommends git && \
+    rm -rf /var/lib/apt/lists/*
+
+# Build argument to control whether we're building standalone or in-repo
+ARG BUILD_MODE=in-repo
+ARG ENV_NAME=StoreManager
+
+# Copy environment code (always at root of build context)
+COPY . /app/env
+
+# For in-repo builds, openenv is already vendored in the build context
+# For standalone builds, openenv will be installed via pyproject.toml
+WORKDIR /app/env
+
+# Ensure uv is available (for local builds where base image lacks it)
+RUN if ! command -v uv >/dev/null 2>&1; then \
+        curl -LsSf https://astral.sh/uv/install.sh | sh && \
+        mv /root/.local/bin/uv /usr/local/bin/uv && \
+        mv /root/.local/bin/uvx /usr/local/bin/uvx; \
+    fi
+    
+# Install dependencies using uv sync
+# If uv.lock exists, use it; otherwise resolve on the fly
+RUN --mount=type=cache,target=/root/.cache/uv \
+    if [ -f uv.lock ]; then \
+        uv sync --frozen --no-install-project --no-editable; \
+    else \
+        uv sync --no-install-project --no-editable; \
+    fi
+
+RUN --mount=type=cache,target=/root/.cache/uv \
+    if [ -f uv.lock ]; then \
+        uv sync --frozen --no-editable; \
+    else \
+        uv sync --no-editable; \
+    fi
+
+# Final runtime stage
+FROM ${BASE_IMAGE}
+
+WORKDIR /app
+
+# Copy the virtual environment from builder
+COPY --from=builder /app/env/.venv /app/.venv
+
+# Copy the environment code
+COPY --from=builder /app/env /app/env
+
+# Set PATH to use the virtual environment
+ENV PATH="/app/.venv/bin:$PATH"
+
+# Set PYTHONPATH so imports work correctly
+ENV PYTHONPATH="/app/env:$PYTHONPATH"
+
+# Health check
+HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
+    CMD curl -f http://localhost:8000/health || exit 1
+
+# Run the FastAPI server
+# The module path is constructed to work with the /app/env structure
+ENV ENABLE_WEB_INTERFACE=true
+CMD ["sh", "-c", "cd /app/env && uvicorn server.app:app --host 0.0.0.0 --port 8000"]

README.md

@@ -1,10 +1,443 @@
----
-title: StoreManager
-emoji: 📈
-colorFrom: yellow
-colorTo: pink
-sdk: docker
-pinned: false
----
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+---
+title: Store Manager Environment
+emoji: 🛒
+colorFrom: blue
+colorTo: yellow
+sdk: docker
+pinned: false
+app_port: 8000
+base_path: /web
+tags:
+  - openenv
+---
+
+# Store Manager Environment
+
+A retail store simulator where an LLM agent acts as the store manager. The agent observes full inventory state each step and chooses one action — discount a product, restock inventory, or move a product to a better shelf zone — to influence customer demand and maximize cumulative profit over a fixed episode horizon.
+
+---
+
+## What Makes This Environment Interesting
+
+This environment models three interconnected real-world retail mechanics that require the agent to reason about **compounding consequences**:
+
+- **Discounts are permanent within an episode.** Every discount cuts the current (already-discounted) price further. A 20% discount applied twice yields 36% off the original price. The agent must decide how aggressively to discount — deep cuts clear stock fast but permanently erode margin.
+- **Restocking has a time lag.** Stock doesn't arrive instantly. Each product has a `restock_lead_time` (1–3 steps). The agent must plan ahead — ordering too late leaves the shelf empty; ordering too much risks expiry penalties and inflated holding costs.
+- **Zone placement directly multiplies pick probability.** Zone 1 (premium) boosts visibility 1.5×, but only holds 2 products. The agent must continuously rotate which products deserve premium placement.
+
+These three mechanics combine into a rich multi-step decision problem where each action affects future state: a discount today changes tomorrow's revenue; a restock today affects holding costs for the next several steps; a zone move today costs $3 but can double a product's sell-through rate.
+
+---
+
+## Task Overview
+
+| Component | Description |
+|---|---|
+| **Products** | N products, each with cost price, selling price, quantity, expiry step, shelf zone, and a base pick probability |
+| **Customers** | Fixed number of customers per step pick products according to a normalized probability distribution |
+| **Agent Action** | Exactly one of three action types per step: `discount`, `restock`, or `place` |
+| **Reward** | Normalized sigmoid reward in `[0.0, 1.0]` based on step profit vs. per-step target |
+| **Expiry penalty** | `remaining_quantity × cost_price` charged when a product reaches its expiry step |
+| **Episode end** | After `max_steps` steps, or when all products are inactive |
+
+### Pick Probability Formula
+
+Customer pick probability combines zone placement and discount multiplier:
+
+```
+effective_weight[i] = base_probability[i] × zone_multiplier[zone[i]] × (1 + discount_pct[i] / 100)
+pick_prob[i]        = effective_weight[i] / sum(effective_weight for all active products)
+```
+
+---
+
+## Actions
+
+The agent submits exactly one action per step.
+
+### 1. Discount
+
+Reduce the selling price of a product. Discounts **compound across steps within an episode** — each discount cuts the current price further, not the original base price.
+
+```json
+{ "action_type": "discount", "product_id": 2, "discount_pct": 20 }
+```
+
+| Field | Type | Values |
+|---|---|---|
+| `action_type` | string | `"discount"` |
+| `product_id` | int | any valid product ID |
+| `discount_pct` | int | `0`, `10`, `20`, or `50` |
+
+**Compounding example** (base price $10.00):
+
+| Step | Discount | Selling Price | Total off base |
+|------|----------|--------------|----------------|
+| 1 | 20% | $8.00 | −20% |
+| 2 | 20% | $6.40 | −36% |
+| 3 | 20% | $5.12 | −49% |
+| 4 | 50% | $2.56 | −74% |
+
+- `base_selling_price` is never modified — it serves as a reference for how far prices have been cut.
+- `current_discount_pct` reflects only the discount tier applied **this step** (used in pick-probability formula).
+- Prices reset to `base_selling_price` at the start of each new episode (`reset()`).
+- A `0%` discount leaves the price unchanged but still advances the step.
+
+**When to use:** Apply deep discounts to near-expiry stock to clear inventory before the expiry penalty hits. Use light discounts on high-margin products to drive volume without destroying margin.
+
+---
+
+### 2. Restock
+
+Place a purchase order for more units of a product. Stock arrives after `restock_lead_time` steps. The ordering cost is charged **immediately** this step.
+
+```json
+{ "action_type": "restock", "product_id": 1, "restock_quantity": 15 }
+```
+
+| Field | Type | Values |
+|---|---|---|
+| `action_type` | string | `"restock"` |
+| `product_id` | int | any valid product ID |
+| `restock_quantity` | int | any positive integer |
+
+**How it works:**
+- On restock: `pending_restock_quantity = restock_quantity`, `pending_restock_arrives_at_step = current_step + restock_lead_time`
+- When delivery step is reached: `quantity += restock_quantity`, pending fields cleared
+- Only **one pending order per product** at a time — wait for delivery before reordering
+
+**Costs:**
+- `restock_cost = restock_quantity × restock_cost_per_unit` charged upfront this step
+- `holding_cost = quantity × cost_price × holding_cost_rate` charged every step on all stock regardless of action
+
+**Per-product restock parameters (randomised at episode start):**
+
+| Field | Range | Description |
+|---|---|---|
+| `restock_lead_time` | 1–3 steps | Steps until delivery |
+| `restock_cost_per_unit` | `cost_price × 1.00–1.25` | Ordering cost per unit |
+| `holding_cost_rate` | 0.01–0.04 | Fraction of `cost_price` charged per unit per step |
+
+**Error cases:** Invalid if `restock_quantity ≤ 0` or product already has a pending order.
+
+**When to use:** Replenish a fast-selling product before it stocks out, only when enough steps remain to sell through the new inventory before expiry.
+
+---
+
+### 3. Place (Zone Placement)
+
+Move a product to a different shelf zone. Takes effect immediately. Charges a flat **labor fee of $3.00** per move.
+
+```json
+{ "action_type": "place", "product_id": 3, "target_zone": 1 }
+```
+
+| Field | Type | Values |
+|---|---|---|
+| `action_type` | string | `"place"` |
+| `product_id` | int | any valid product ID |
+| `target_zone` | int | `1`, `2`, or `3` |
+
+**Zone multipliers and capacity:**
+
+| Zone | Location | Pick Probability Multiplier | Max Products |
+|------|----------|-----------------------------|--------------|
+| 1 | Premium / entrance / eye-level | **×1.5** | 2 |
+| 2 | Standard shelves | **×1.0** | 4 |
+| 3 | Back / low-traffic | **×0.6** | unlimited |
+
+**Initial zone assignment at episode start:** products fill zone 1 first (up to capacity 2), then zone 2, remainder in zone 3.
+
+**Error cases:** Invalid if `target_zone` is not in {1, 2, 3}, product is already in `target_zone`, or target zone is at capacity.
+
+**When to use:** Rotate zone 1 to whichever product most needs visibility — near-expiry items benefit most since clearance speed directly avoids penalty. Demote slow-movers to zone 3 to free up premium space.
+
+---
+
+## Reward Function
+
+Each step returns a **normalized reward in `[0.0, 1.0]`** using a sigmoid centred at break-even:
+
+```
+step_profit = step_revenue − cost_of_goods_sold − expiry_penalty
+            − holding_cost − placement_cost − restock_cost
+
+x      = step_profit / per_step_target
+reward = sigmoid(2x) = 1 / (1 + exp(−2x))
+```
+
+| Step profit vs. target | reward |
+|---|---|
+| Catastrophic loss (−∞) | → 0.0 |
+| Break-even (0) | = 0.50 |
+| On target | ≈ 0.88 |
+| 2× target | ≈ 0.98 |
+
+The raw per-step cost breakdown is available in `metadata`:
+
+```json
+{
+  "step_revenue": 45.20,
+  "step_cost": 18.40,
+  "expiry_penalty": 0.0,
+  "holding_cost": 12.35,
+  "placement_cost": 3.0,
+  "restock_cost": 0.0,
+  "step_profit": 11.45
+}
+```
+
+---
+
+## Difficulty Tasks
+
+Four pre-configured tasks with increasing complexity. Each task has a fixed seed for deterministic, reproducible grading.
+
+| Task | Products | Steps | Customers/step | Expiry range | Profit target | Dominant mechanic |
+|------|----------|-------|----------------|--------------|--------------|-------------------|
+| **easy** | 4 | 20 | 15 | 18–30 steps | $120 | Discount |
+| **medium** | 8 | 28 | 12 | 12–22 steps | $200 | Discount + zone placement |
+| **hard** | 10 | 30 | 10 | 8–18 steps | $240 | Discount + zone + expiry timing |
+| **expert** | 12 | 25 | 8 | 4–10 steps | $190 | All three actions coordinated |
+
+### Task Descriptions
+
+**easy** — Products expire well after the episode ends; no expiry pressure. The agent learns to identify high-margin products and apply targeted discounts to boost their pick probability.
+
+**medium** — Several products expire mid-episode. Zone 1 has only 2 slots for 8 competing products — the agent must combine discounts with smart zone rotation to clear stock before expiry.
+
+**hard** — Most products expire in 8–18 steps. Expiry management is critical: rotate zone 1 frequently to accelerate clearance, and consider restocking fast-sellers that run out early.
+
+**expert** — Aggressive expiry (4–10 steps) with limited initial stock. All three actions must be coordinated: discount near-expiry items heavily, restock high-margin products before they stock out, and rotate zone 1 continuously.
+
+### Grading
+
+```
+score = clamp(cumulative_profit / profit_target, 0.0, 1.0)
+```
+
+---
+
+## Baseline Scores
+
+Expected performance using `gpt-4o-mini` (deterministic, fixed seeds):
+
+| Task | Profit Target | Expected Profit | Expected Score |
+|------|--------------|-----------------|----------------|
+| easy | $120 | ~$140–180 | ~0.90–1.0 |
+| medium | $200 | ~$160–220 | ~0.80–1.0 |
+| hard | $240 | ~$180–260 | ~0.75���1.0 |
+| expert | $190 | ~$100–170 | ~0.50–0.90 |
+
+Scores are fully reproducible. Re-run `inference.py` with the same model and seeds to verify.
+
+---
+
+## Quick Start
+
+```python
+from StoreManager import StoremanagerAction, StoremanagerEnv
+
+with StoremanagerEnv(base_url="http://localhost:8000") as env:
+    result = env.reset()
+    print([p.name for p in result.observation.inventory])
+
+    # Discount: compounds on current price each time applied
+    result = env.step(StoremanagerAction(action_type="discount", product_id=0, discount_pct=20))
+    print(f"Reward: {result.reward:.2f}")
+
+    # Restock: stock arrives after restock_lead_time steps
+    result = env.step(StoremanagerAction(action_type="restock", product_id=1, restock_quantity=10))
+    print(f"Restock cost: {result.observation.restock_cost_this_step:.2f}")
+
+    # Place: move product to premium zone (costs $3.00 labor)
+    result = env.step(StoremanagerAction(action_type="place", product_id=2, target_zone=1))
+    print(f"Placement cost: {result.observation.placement_cost_this_step:.2f}")
+    print(f"Zone occupancy: {result.observation.zone_occupancy}")
+```
+
+---
+
+## Running Locally
+
+```bash
+# Install dependencies
+uv sync
+
+# Run dev server (auto-reload, default config)
+uvicorn server.app:app --reload
+
+# Run with a specific difficulty task baked in at startup
+STORE_TASK=hard uvicorn server.app:app --reload
+
+# Test environment logic directly (no server needed)
+python server/StoreManager_environment.py
+
+# Run the WebSocket test suite (server must be running)
+python test_ws.py
+```
+
+---
+
+## Running the Inference Script
+
+The `inference.py` script runs an LLM agent through all four tasks sequentially and reports a score for each.
+
+### Required Environment Variables
+
+| Variable | Description |
+|---|---|
+| `HF_TOKEN` | API key — your Hugging Face token or OpenAI API key |
+| `API_BASE_URL` | LLM API base URL (e.g. `https://api-inference.huggingface.co/v1/`) |
+| `MODEL_NAME` | Model identifier (e.g. `meta-llama/Llama-3.1-8B-Instruct` or `gpt-4o-mini`) |
+
+```bash
+# Server must be running first
+uvicorn server.app:app &
+
+# Run with Hugging Face Inference API
+HF_TOKEN=hf_... \
+API_BASE_URL=https://api-inference.huggingface.co/v1/ \
+MODEL_NAME=meta-llama/Llama-3.1-8B-Instruct \
+python inference.py
+
+# Run with OpenAI (API_BASE_URL defaults to OpenAI when unset)
+HF_TOKEN=sk-... MODEL_NAME=gpt-4o-mini python inference.py
+
+# Override the environment server URL
+HF_TOKEN=sk-... MODEL_NAME=gpt-4o-mini STORE_ENV_URL=http://localhost:8000 python inference.py
+```
+
+### Structured Log Format
+
+```
+[START] {"task": "easy", "seed": 42, "model": "gpt-4o-mini", "max_steps": 20, "profit_target": 120.0}
+[STEP]  {"task": "easy", "step": 1, "action": {"action_type": "discount", "product_id": 2, "discount_pct": 20}, "reward": 0.88, "cumulative_profit": 12.5, "holding_cost": 8.2, "placement_cost": 0, "restock_cost": 0}
+...
+[END]   {"task": "easy", "cumulative_profit": 145.3, "score": 1.0, "profit_target": 120.0}
+```
+
+A single server instance handles all four tasks — the task config is passed in the `reset` message, so no server restart is needed between tasks.
+
+---
+
+## Action & Observation Reference
+
+### Action: `StoremanagerAction`
+
+| Field | Type | Default | Description |
+|---|---|---|---|
+| `action_type` | `"discount"` \| `"restock"` \| `"place"` | `"discount"` | Type of action |
+| `product_id` | int | required | ID of the product to act on |
+| `discount_pct` | `0` \| `10` \| `20` \| `50` | `0` | Discount tier (for `discount` actions) |
+| `restock_quantity` | int | `None` | Units to order (for `restock` actions) |
+| `target_zone` | `1` \| `2` \| `3` | `None` | Destination zone (for `place` actions) |
+
+### Observation: `StoremanagerObservation`
+
+| Field | Type | Description |
+|---|---|---|
+| `inventory` | `List[ProductState]` | Full state of all products (including inactive) |
+| `current_step` | int | Current step number (0 at reset, increments after each valid step) |
+| `max_steps` | int | Total steps in this episode |
+| `steps_remaining` | int | Steps left in the episode |
+| `cumulative_profit` | float | Total profit accumulated so far |
+| `last_step_profit` | float | Net profit from the most recent step |
+| `last_expired_products` | `List[str]` | Names of products that expired this step |
+| `zone_capacity` | `Dict[int, int]` | Max products per zone: `{1: 2, 2: 4, 3: 99}` |
+| `zone_occupancy` | `Dict[int, int]` | Active products currently in each zone |
+| `zone_multipliers` | `Dict[int, float]` | Pick-prob multiplier per zone: `{1: 1.5, 2: 1.0, 3: 0.6}` |
+| `holding_cost_this_step` | float | Total holding cost charged this step |
+| `placement_cost_this_step` | float | `3.0` if a `place` action was taken, else `0.0` |
+| `restock_cost_this_step` | float | Ordering cost if a `restock` was placed, else `0.0` |
+| `task_name` | str | Active task: `easy` / `medium` / `hard` / `expert` / `custom` |
+| `error` | str \| None | Error message if the action was invalid (step does **not** advance) |
+| `metadata` | dict | Per-step breakdown: `step_revenue`, `step_cost`, `expiry_penalty`, `holding_cost`, `placement_cost`, `restock_cost`, `step_profit` |
+
+### ProductState Fields (inside `inventory`)
+
+| Field | Description |
+|---|---|
+| `product_id` | Unique integer ID |
+| `name` | Product name (e.g. `"Milk"`) |
+| `cost_price` | Per-unit cost to the store |
+| `selling_price` | Current selling price (decreases each time a discount is applied) |
+| `base_selling_price` | Original price — never modified, reference for total discount depth |
+| `quantity` | Units currently in stock |
+| `expiry_step` | Episode step at which this product expires |
+| `zone` | Current shelf zone: `1` (premium), `2` (standard), `3` (back) |
+| `effective_pick_prob` | Normalized pick probability this step (sums to 1.0 across all active products) |
+| `current_discount_pct` | Discount tier applied **this step only** (0 if no discount action taken) |
+| `holding_cost_rate` | Holding cost per unit per step as a fraction of `cost_price` |
+| `restock_cost_per_unit` | Cost to order one additional unit |
+| `restock_lead_time` | Steps until a placed order is delivered |
+| `pending_restock_quantity` | Units currently on order (not yet delivered) |
+| `pending_restock_arrives_at_step` | Step number when the pending order will arrive |
+| `is_active` | `False` if the product is sold out or expired |
+
+---
+
+## Error Handling
+
+When an action is invalid, the step does **not** advance. The returned observation contains an `error` string and reflects the pre-action state. Common error cases:
+
+| Scenario | Error |
+|---|---|
+| Discount on inactive / sold-out product | `"Product '...' is no longer active"` |
+| Restock with `quantity ≤ 0` | `"restock_quantity must be a positive integer"` |
+| Restock when pending order exists | `"already has a pending restock order"` |
+| Place to same zone product is already in | `"is already in zone N"` |
+| Place to zone at capacity | `"Zone N is at capacity"` |
+| Invalid `product_id` | `"Invalid product_id"` |
+
+Always check `obs.get("error")` before processing the observation.
+
+---
+
+## Configuration
+
+Control the server via environment variables:
+
+| Variable | Default | Description |
+|---|---|---|
+| `STORE_TASK` | _(none)_ | Pre-load a task config at startup: `easy`, `medium`, `hard`, or `expert` |
+| `STORE_NUM_PRODUCTS` | 8 | Number of products (overridden by `STORE_TASK`) |
+| `STORE_MAX_STEPS` | 30 | Episode length in steps (overridden by `STORE_TASK`) |
+| `STORE_NUM_CUSTOMERS` | 10 | Customers simulated per step (overridden by `STORE_TASK`) |
+| `STORE_MAX_CONCURRENT_ENVS` | 4 | Max parallel WebSocket sessions |
+
+**Task switching at runtime:** A single server instance can serve all four tasks. Pass `"task": "<name>"` in the WebSocket `reset` message and the environment reconfigures immediately without a server restart.
+
+---
+
+## Deploying to Hugging Face Spaces
+
+```bash
+openenv push
+openenv push --repo-id my-org/store-manager --private
+```
+
+After deployment, your space includes:
+- **Web Interface** at `/web`
+- **API Documentation** at `/docs`
+- **WebSocket** at `/ws` — persistent session for low-latency agent interactions
+
+---
+
+## Project Structure
+
+```
+StoreManager/
+├── __init__.py                       # Module exports
+├── models.py                         # Pydantic schemas: Action, Observation, State
+├── client.py                         # StoremanagerEnv WebSocket client
+├── tasks.py                          # Task configs (easy/medium/hard/expert) + grader
+├── inference.py                      # LLM agent runner with structured logs
+├── test_ws.py                        # WebSocket integration test suite (7 test groups)
+├── openenv.yaml                      # OpenEnv manifest
+├── pyproject.toml                    # Dependencies
+└── server/
+    ├── StoreManager_environment.py   # Core RL environment logic
+    ├── app.py                        # FastAPI server (HTTP + WebSocket)
+    └── Dockerfile                    # Container image
+```

__init__.py

@@ -0,0 +1,23 @@
+# Copyright (c) Meta Platforms, Inc. and affiliates.
+# All rights reserved.
+#
+# This source code is licensed under the BSD-style license found in the
+# LICENSE file in the root directory of this source tree.
+
+"""Store Manager Environment."""
+
+from .client import StoremanagerEnv
+from .models import (
+    ProductState,
+    StoremanagerAction,
+    StoremanagerObservation,
+    StoremanagerState,
+)
+
+__all__ = [
+    "ProductState",
+    "StoremanagerAction",
+    "StoremanagerObservation",
+    "StoremanagerState",
+    "StoremanagerEnv",
+]

client.py

@@ -0,0 +1,109 @@
+# Copyright (c) Meta Platforms, Inc. and affiliates.
+# All rights reserved.
+#
+# This source code is licensed under the BSD-style license found in the
+# LICENSE file in the root directory of this source tree.
+
+"""Store Manager Environment Client."""
+
+from typing import Any, Dict
+
+from openenv.core import EnvClient
+from openenv.core.client_types import StepResult
+
+from .models import (
+    ProductState,
+    StoremanagerAction,
+    StoremanagerObservation,
+    StoremanagerState,
+)
+
+
+class StoremanagerEnv(
+    EnvClient[StoremanagerAction, StoremanagerObservation, StoremanagerState]
+):
+    """
+    Client for the Store Manager Environment.
+
+    Maintains a persistent WebSocket connection to the environment server,
+    enabling efficient multi-step interactions with lower latency.
+
+    Example:
+        >>> with StoremanagerEnv(base_url="http://localhost:8000") as env:
+        ...     result = env.reset()
+        ...     print([p.name for p in result.observation.inventory])
+        ...
+        ...     action = StoremanagerAction(product_id=0, discount_pct=20)
+        ...     result = env.step(action)
+        ...     print(f"Reward: {result.reward:.2f}")
+
+    Example with Docker:
+        >>> client = StoremanagerEnv.from_docker_image("StoreManager-env:latest")
+        >>> try:
+        ...     result = client.reset()
+        ...     result = client.step(StoremanagerAction(product_id=0, discount_pct=10))
+        ... finally:
+        ...     client.close()
+    """
+
+    def _step_payload(self, action: StoremanagerAction) -> Dict:
+        """Convert StoremanagerAction to JSON payload for the step message."""
+        payload: Dict = {
+            "action_type": action.action_type,
+            "product_id": action.product_id,
+            "discount_pct": action.discount_pct,
+        }
+        if action.restock_quantity is not None:
+            payload["restock_quantity"] = action.restock_quantity
+        if action.target_zone is not None:
+            payload["target_zone"] = action.target_zone
+        return payload
+
+    def _parse_result(self, payload: Dict) -> StepResult[StoremanagerObservation]:
+        """Parse server response into StepResult[StoremanagerObservation]."""
+        obs_data = payload.get("observation", {})
+
+        raw_inventory = obs_data.get("inventory", [])
+        inventory = [ProductState(**p) for p in raw_inventory if isinstance(p, dict)]
+
+        observation = StoremanagerObservation(
+            inventory=inventory,
+            current_step=obs_data.get("current_step", 0),
+            max_steps=obs_data.get("max_steps", 30),
+            steps_remaining=obs_data.get("steps_remaining", 0),
+            cumulative_profit=obs_data.get("cumulative_profit", 0.0),
+            last_step_profit=obs_data.get("last_step_profit", 0.0),
+            last_step_units_sold=obs_data.get("last_step_units_sold", 0),
+            last_expired_products=obs_data.get("last_expired_products", []),
+            num_customers_per_step=obs_data.get("num_customers_per_step", 10),
+            error=obs_data.get("error"),
+            zone_capacity=obs_data.get("zone_capacity", {1: 2, 2: 4, 3: 99}),
+            zone_occupancy=obs_data.get("zone_occupancy", {1: 0, 2: 0, 3: 0}),
+            zone_multipliers=obs_data.get("zone_multipliers", {1: 1.5, 2: 1.0, 3: 0.6}),
+            holding_cost_this_step=obs_data.get("holding_cost_this_step", 0.0),
+            placement_cost_this_step=obs_data.get("placement_cost_this_step", 0.0),
+            restock_cost_this_step=obs_data.get("restock_cost_this_step", 0.0),
+            unjustified_discount_penalty=obs_data.get("unjustified_discount_penalty", 0.0),
+            done=payload.get("done", False),
+            reward=payload.get("reward"),
+            metadata=obs_data.get("metadata", {}),
+        )
+
+        return StepResult(
+            observation=observation,
+            reward=payload.get("reward"),
+            done=payload.get("done", False),
+        )
+
+    def _parse_state(self, payload: Dict[str, Any]) -> StoremanagerState:
+        """Parse state endpoint response into StoremanagerState."""
+        return StoremanagerState(
+            episode_id=payload.get("episode_id"),
+            step_count=payload.get("step_count", 0),
+            num_products=payload.get("num_products", 8),
+            max_steps=payload.get("max_steps", 30),
+            cumulative_profit=payload.get("cumulative_profit", 0.0),
+            last_reward=payload.get("last_reward", 0.0),
+            active_product_count=payload.get("active_product_count", 0),
+            inventory_snapshot=payload.get("inventory_snapshot", []),
+        )

inference.py

@@ -0,0 +1,383 @@
+"""
+Inference script for the Store Manager environment.
+
+Runs an LLM agent through all three difficulty tasks (easy → medium → hard)
+and reports a deterministic grade (0.0–1.0) per task.
+
+Usage:
+    python inference.py
+    python inference.py --url http://localhost:8000
+
+Required environment variables:
+    HF_TOKEN       API key for the LLM provider (Hugging Face token or OpenAI key)
+    API_BASE_URL   Base URL of the LLM API  (e.g. https://api-inference.huggingface.co/v1/)
+    MODEL_NAME     Model identifier          (e.g. meta-llama/Llama-3.1-8B-Instruct)
+
+Optional environment variables:
+    STORE_ENV_URL  Base URL of the running Store Manager server (default: http://localhost:8000)
+
+Structured log format emitted to stdout:
+    [START] {"task": "...", "seed": N, "model": "...", "max_steps": N, "profit_target": N}
+    [STEP]  {"task": "...", "step": N, "action": {...}, "reward": ..., "cumulative_profit": ...}
+    [END]   {"task": "...", "cumulative_profit": ..., "score": ..., "profit_target": ...}
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import re
+import sys
+import time
+
+try:
+    import websocket  # websocket-client library
+except ImportError:
+    print(
+        "ERROR: websocket-client not installed. Run: pip install websocket-client",
+        file=sys.stderr,
+    )
+    sys.exit(1)
+
+try:
+    from openai import OpenAI
+except ImportError:
+    print(
+        "ERROR: openai not installed. Run: pip install openai",
+        file=sys.stderr,
+    )
+    sys.exit(1)
+
+# Add project root to path so tasks.py is importable
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+from tasks import TASKS, grade
+
+
+# ── Prompt helpers ─────────────────────────────────────────────────────────────
+
+_SYSTEM_PROMPT = """You are an expert retail store manager optimizing profit.
+
+Each step you choose EXACTLY ONE action. There are three action types:
+
+── 1. DISCOUNT ──────────────────────────────────────────────────────────────────
+Apply a price discount to a product for this step only (ephemeral).
+Discounts boost customer pick probability and clear near-expiry stock.
+  {"action_type": "discount", "product_id": <int>, "discount_pct": <0|10|20|50>}
+
+Discount tiers and effect:
+  0%  -> no change   | 10% -> +10% relative boost | 20% -> +20% boost | 50% -> +50% boost
+
+PRICE FLOOR RULES (selling below cost is penalized unless near expiry):
+  In(steps) >= 4  -> floor = cost_price   (no below-cost selling allowed)
+  In(steps) == 3  -> floor = 70% of cost  (max 30% loss/unit)
+  In(steps) == 2  -> floor = 40% of cost  (max 60% loss/unit)
+  In(steps) == 1  -> floor = 10% of cost  (max 90% loss/unit — clear it!)
+
+  Selling below cost when In(steps) >= 4 incurs an EXTRA penalty equal to the
+  per-unit loss × units sold. Avoid below-cost discounts unless near expiry.
+
+When to discount: near-expiry items (avoid expiry penalty = remaining_qty x cost),
+high-margin products (Margin > $0) to drive volume, or oversupplied items.
+Near expiry (In <= 3): aggressive discounts are worthwhile to avoid the full expiry penalty.
+
+── 2. RESTOCK ───────────────────────────────────────────────────────────────────
+Place a reorder for a product. Stock arrives after restock_lead_time steps.
+Costs restock_cost_per_unit per unit (upfront, this step). One order per product
+at a time -- wait for delivery before reordering the same product.
+  {"action_type": "restock", "product_id": <int>, "restock_quantity": <int>}
+
+When to restock: low stock on a high-demand product, enough steps left before expiry
+to sell through the restocked units, and available budget.
+
+── 3. PLACE ─────────────────────────────────────────────────────────────────────
+Move a product to a different shelf zone. Costs a flat placement fee ($3.00).
+  {"action_type": "place", "product_id": <int>, "target_zone": <1|2|3>}
+
+Zone multipliers on pick probability:
+  Zone 1 (premium/entrance) -> x1.5   [capacity: 2 products max]
+  Zone 2 (standard)         -> x1.0   [capacity: 4 products max]
+  Zone 3 (back/low-traffic) -> x0.6   [unlimited]
+
+When to place: move high-value or near-expiry products to zone 1 to increase
+their pick probability. Move slow-movers to zone 3.
+
+── COSTS CHARGED EVERY STEP ─────────────────────────────────────────────────────
+- Holding cost: each unit in stock costs holding_cost_rate x cost_price per step.
+  Minimize idle inventory by balancing restocks with sales velocity.
+- Expiry penalty: remaining_qty x cost_price for each expired product.
+
+Goal: maximize cumulative profit = revenue - COGS - expiry_penalty - holding_cost
+      - placement_cost - restock_cost.
+
+Respond ONLY with a single JSON object (one of the three formats above).
+No explanation, no extra text -- just the JSON.
+"""
+
+
+def _format_inventory(obs: dict) -> str:
+    """Format inventory state as a readable table for the LLM prompt."""
+    # JSON serializes int dict keys as strings; coerce back to int for lookup
+    zm_raw = obs.get("zone_multipliers", {})
+    zc_raw = obs.get("zone_capacity", {})
+    zo_raw = obs.get("zone_occupancy", {})
+    zm = {int(k): v for k, v in zm_raw.items()} if zm_raw else {1: 1.5, 2: 1.0, 3: 0.6}
+    zc = {int(k): v for k, v in zc_raw.items()} if zc_raw else {1: 2, 2: 4, 3: 99}
+    zo = {int(k): v for k, v in zo_raw.items()} if zo_raw else {1: 0, 2: 0, 3: 0}
+
+    lines = [
+        f"Step {obs['current_step'] + 1}/{obs['max_steps']} "
+        f"| Steps remaining: {obs['steps_remaining']} "
+        f"| Cumulative profit: ${obs['cumulative_profit']:.2f}",
+        f"Task difficulty: {obs.get('task_name', 'unknown')}",
+        f"Holding cost this step: ${obs.get('holding_cost_this_step', 0.0):.4f}",
+        "",
+        "Zone info: "
+        + "  ".join(
+            f"Zone {z}: x{zm.get(z, 1.0)} ({zo.get(z, 0)}/{zc.get(z, '?')} slots)"
+            for z in [1, 2, 3]
+        ),
+        "",
+        f"{'ID':<4} {'Name':<10} {'Cost':>6} {'Price':>7} {'Margin':>7} {'Qty':>5} "
+        f"{'In(steps)':>9} {'Zone':>5} {'PickProb':>9} {'PopMult':>8} {'HoldRate':>9} "
+        f"{'RestockCost':>11} {'Lead':>5} {'PendingQty':>10} {'Active':>7}",
+        "-" * 122,
+    ]
+    for p in obs["inventory"]:
+        pending_str = (
+            f"{p['pending_restock_quantity']}@s{p['pending_restock_arrives_at_step']}"
+            if p.get("pending_restock_quantity", 0) > 0
+            else "-"
+        )
+        margin = p.get("margin_per_unit", round(p["selling_price"] - p["cost_price"], 4))
+        steps_left = p.get("steps_until_expiry", p.get("expiry_step", 0) - obs.get("current_step", 0))
+        lines.append(
+            f"{p['product_id']:<4} {p['name']:<10} "
+            f"${p['cost_price']:>5.2f} ${p['selling_price']:>6.2f} "
+            f"${margin:>+6.2f} "
+            f"{p['quantity']:>5} "
+            f"{steps_left:>9} "
+            f"{p.get('zone', 2):>5} "
+            f"{p['effective_pick_prob']:>9.4f} "
+            f"{p.get('popularity_multiplier', 1.0):>8.3f} "
+            f"{p.get('holding_cost_rate', 0.0):>9.4f} "
+            f"${p.get('restock_cost_per_unit', 0.0):>10.2f} "
+            f"{p.get('restock_lead_time', 2):>5} "
+            f"{pending_str:>10} "
+            f"{'YES' if p['is_active'] else 'no':>7}"
+        )
+
+    if obs.get("last_expired_products"):
+        lines.append(f"\nJust expired: {', '.join(obs['last_expired_products'])}")
+
+    return "\n".join(lines)
+
+
+def _parse_action(text: str, active_ids: list[int]) -> dict:
+    """Extract a valid action from LLM output; fall back to safe default."""
+    def _validate(data: dict) -> dict | None:
+        action_type = data.get("action_type", "discount")
+        pid = int(data.get("product_id", active_ids[0]))
+        if pid not in active_ids:
+            return None
+
+        if action_type == "discount":
+            disc = int(data.get("discount_pct", 0))
+            if disc in (0, 10, 20, 50):
+                return {"action_type": "discount", "product_id": pid, "discount_pct": disc}
+
+        elif action_type == "restock":
+            qty = data.get("restock_quantity")
+            if qty is not None and int(qty) > 0:
+                return {"action_type": "restock", "product_id": pid, "restock_quantity": int(qty)}
+
+        elif action_type == "place":
+            zone = data.get("target_zone")
+            if zone is not None and int(zone) in (1, 2, 3):
+                return {"action_type": "place", "product_id": pid, "target_zone": int(zone)}
+
+        return None
+
+    # Try direct JSON parse
+    try:
+        result = _validate(json.loads(text.strip()))
+        if result:
+            return result
+    except (json.JSONDecodeError, ValueError, KeyError):
+        pass
+
+    # Try extracting JSON from surrounding text
+    match = re.search(r'\{[^{}]+\}', text)
+    if match:
+        try:
+            result = _validate(json.loads(match.group(0)))
+            if result:
+                return result
+        except (json.JSONDecodeError, ValueError, KeyError):
+            pass
+
+    # Safe fallback
+    return {"action_type": "discount", "product_id": active_ids[0], "discount_pct": 0}
+
+
+# ── WebSocket helpers ─────────��────────────────────────────────────────────────
+
+def _ws_url(http_url: str) -> str:
+    """Convert http(s) URL to ws(s)."""
+    return http_url.replace("https://", "wss://").replace("http://", "ws://") + "/ws"
+
+
+def _recv_json(ws: websocket.WebSocket) -> dict:
+    """Receive one WebSocket message and parse as JSON."""
+    return json.loads(ws.recv())
+
+
+def _unpack(msg: dict) -> dict:
+    """
+    Unpack a WebSocket message into a flat observation dict.
+
+    The server sends: {"type": "observation", "data": {"observation": {...}, "done": bool, "reward": float}}
+    This extracts the inner observation and injects done/reward at the top level.
+    """
+    data = msg.get("data", msg)
+    obs = data.get("observation", data)
+    obs["done"] = data.get("done", obs.get("done", False))
+    obs["reward"] = data.get("reward", obs.get("reward", 0.0))
+    return obs
+
+
+# ── Episode runner ─────────────────────────────────────────────────────────────
+
+def run_episode(
+    ws_url: str,
+    task_name: str,
+    client: OpenAI,
+    model: str,
+) -> tuple[float, float]:
+    """
+    Run a full episode for `task_name` using the LLM agent.
+
+    Returns:
+        (cumulative_profit, score)
+    """
+    task = TASKS[task_name]
+
+    ws = websocket.create_connection(ws_url, timeout=60)
+    cumulative = 0.0
+    score = 0.0
+    try:
+        # ── Reset ──────────────────────────────────────────────────────────────
+        ws.send(json.dumps({"type": "reset", "data": {"task": task_name, "seed": task.seed}}))
+        obs = _unpack(_recv_json(ws))
+
+        print(
+            f"[START] {json.dumps({'task': task_name, 'seed': task.seed, 'model': model, 'max_steps': task.max_steps, 'profit_target': task.profit_target})}",
+            flush=True,
+        )
+
+        # ── Episode loop ───────────────────────────────────────────────────────
+        step_num = 0
+        while True:
+            active_ids = [
+                p["product_id"]
+                for p in obs.get("inventory", [])
+                if p.get("is_active") and p.get("quantity", 0) > 0
+            ]
+            if not active_ids or obs.get("done", False):
+                break
+
+            # Build LLM prompt
+            user_msg = _format_inventory(obs)
+
+            # Call LLM
+            response = client.chat.completions.create(
+                model=model,
+                messages=[
+                    {"role": "system", "content": _SYSTEM_PROMPT},
+                    {"role": "user", "content": user_msg},
+                ],
+                temperature=0.2,
+                max_tokens=64,
+            )
+            llm_text = response.choices[0].message.content or ""
+            action = _parse_action(llm_text, active_ids)
+
+            # Send action to environment
+            ws.send(json.dumps({"type": "step", "data": action}))
+            obs = _unpack(_recv_json(ws))
+
+            step_num += 1
+            print(
+                f"[STEP] {json.dumps({'task': task_name, 'step': step_num, 'action': action, 'reward': round(obs.get('reward', 0.0) or 0.0, 4), 'cumulative_profit': round(obs.get('cumulative_profit', 0.0), 4), 'holding_cost': round(obs.get('holding_cost_this_step', 0.0), 4), 'placement_cost': obs.get('placement_cost_this_step', 0.0), 'restock_cost': round(obs.get('restock_cost_this_step', 0.0), 4), 'unjustified_penalty': round(obs.get('unjustified_discount_penalty', 0.0), 4)})}",
+                flush=True,
+            )
+
+            if obs.get("done", False):
+                break
+
+        cumulative = obs.get("cumulative_profit", 0.0)
+        score = grade(task_name, cumulative)
+
+        return cumulative, score
+
+    finally:
+        ws.close()
+        print(
+            f"[END] {json.dumps({'task': task_name, 'cumulative_profit': round(cumulative, 4), 'score': round(score, 4), 'profit_target': task.profit_target})}",
+            flush=True,
+        )
+
+
+# ── Main ───────────────────────────────────────────────────────────────────────
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description="Run LLM agent on Store Manager tasks")
+    parser.add_argument(
+        "--url",
+        default=os.getenv("STORE_ENV_URL", "http://localhost:8000"),
+        help="Base URL of the running Store Manager server (env: STORE_ENV_URL)",
+    )
+    args = parser.parse_args()
+
+    # ── Mandatory env vars (per hackathon spec) ────────────────────────────────
+    hf_token = os.getenv("HF_TOKEN") or os.getenv("OPENAI_API_KEY")
+    if not hf_token:
+        print(
+            "ERROR: HF_TOKEN environment variable not set. "
+            "Set it to your Hugging Face token or OpenAI API key.",
+            file=sys.stderr,
+        )
+        sys.exit(1)
+
+    api_base_url = os.getenv("API_BASE_URL")
+    model = os.getenv("MODEL_NAME") or os.getenv("OPENAI_MODEL", "gpt-4o-mini")
+
+    # Build OpenAI-compatible client
+    client_kwargs: dict = {"api_key": hf_token}
+    if api_base_url:
+        client_kwargs["base_url"] = api_base_url
+
+    client = OpenAI(**client_kwargs)
+    ws_url = _ws_url(args.url)
+
+    results = {}
+    for task_name in ["easy", "medium", "hard", "expert"]:
+        try:
+            profit, score = run_episode(ws_url, task_name, client, model)
+            results[task_name] = {"profit": round(profit, 4), "score": round(score, 4)}
+        except Exception as exc:
+            print(f"[ERROR] task={task_name} error={exc}", file=sys.stderr, flush=True)
+            results[task_name] = {"profit": 0.0, "score": 0.0}
+        # Brief pause between tasks to avoid overwhelming the server
+        time.sleep(1)
+
+    # Final summary
+    print("\n=== Summary ===")
+    for task_name, r in results.items():
+        print(f"  {task_name:<8} profit={r['profit']:>10.2f}  score={r['score']:.4f}")
+    overall = sum(r["score"] for r in results.values()) / len(results)
+    print(f"  {'AVERAGE':<8}                   score={overall:.4f}")
+
+
+if __name__ == "__main__":
+    main()

models.py

@@ -0,0 +1,201 @@
+# Copyright (c) Meta Platforms, Inc. and affiliates.
+# All rights reserved.
+#
+# This source code is licensed under the BSD-style license found in the
+# LICENSE file in the root directory of this source tree.
+
+"""
+Data models for the Store Manager Environment.
+
+The Store Manager environment simulates a retail store where an LLM agent
+acts as the store manager, applying discounts to products to maximize profit
+while managing inventory and handling product expiry.
+"""
+
+from __future__ import annotations
+
+from typing import Dict, List, Literal, Optional
+
+from openenv.core.env_server.types import Action, Observation, State
+from pydantic import BaseModel, Field
+
+
+class ProductState(BaseModel):
+    """State of a single product in the store inventory."""
+
+    product_id: int = Field(..., description="Unique integer identifier for the product")
+    name: str = Field(..., description="Human-readable product name")
+    cost_price: float = Field(..., description="Cost to the store per unit")
+    selling_price: float = Field(..., description="Current selling price (after discount)")
+    base_selling_price: float = Field(..., description="Original selling price before any discount")
+    quantity: int = Field(..., description="Units currently in stock")
+    expiry_step: int = Field(
+        ..., description="Episode step number at which this product expires"
+    )
+    base_probability: float = Field(
+        ..., description="Base pick-probability weight before discount"
+    )
+    current_discount_pct: float = Field(
+        default=0.0, description="Active discount percentage this step (0/10/20/50)"
+    )
+    effective_pick_prob: float = Field(
+        default=0.0,
+        description="Normalised pick probability this step after applying discount (sums to 1 across all active products)",
+    )
+    is_active: bool = Field(
+        default=True, description="False if the product is sold out or expired"
+    )
+    sales_history: List[Dict] = Field(
+        default_factory=list,
+        description="List of {step, discount_pct, units_sold} records",
+    )
+    # Zone / Shelf Placement fields
+    zone: int = Field(
+        default=2, description="Current shelf zone (1=premium/high-traffic, 2=standard, 3=back/low-traffic)"
+    )
+    # Restocking / Procurement fields
+    holding_cost_rate: float = Field(
+        default=0.02, description="Holding cost per unit per step as a fraction of cost_price"
+    )
+    restock_cost_per_unit: float = Field(
+        default=0.0, description="Cost per unit when reordering (may exceed cost_price due to rush ordering)"
+    )
+    restock_lead_time: int = Field(
+        default=2, description="Number of steps until a restock order is delivered"
+    )
+    pending_restock_quantity: int = Field(
+        default=0, description="Units currently on order (not yet delivered)"
+    )
+    pending_restock_arrives_at_step: Optional[int] = Field(
+        default=None, description="Step number when the pending restock order will arrive"
+    )
+    # Compounding popularity effect
+    popularity_multiplier: float = Field(
+        default=1.0,
+        description=(
+            "Cumulative pick-probability multiplier earned through repeated discounting. "
+            "Starts at 1.0 and compounds up each time a discount action is applied: "
+            "popularity_multiplier *= (1 + discount_pct / 100). "
+            "Persists across steps within an episode; resets to 1.0 on reset()."
+        ),
+    )
+    # Derived signal fields (recomputed each step — do not set manually)
+    steps_until_expiry: int = Field(
+        default=0,
+        description=(
+            "Steps remaining before this product expires (expiry_step - current_step). "
+            "Near-expiry threshold: <= 3 steps. Below-cost discounting is permitted "
+            "when steps_until_expiry <= 3; the price floor loosens progressively."
+        ),
+    )
+    margin_per_unit: float = Field(
+        default=0.0,
+        description=(
+            "Current per-unit profit margin: selling_price - cost_price. "
+            "Positive = profitable. Zero = break-even. "
+            "Negative = selling at a loss (only possible when steps_until_expiry <= 3)."
+        ),
+    )
+
+
+class StoremanagerAction(Action):
+    """Agent action: discount a product, restock inventory, or move a product to a different zone."""
+
+    action_type: Literal["discount", "restock", "place"] = Field(
+        default="discount",
+        description="Type of action: 'discount' to apply a price discount, 'restock' to order more inventory, 'place' to move product to a different shelf zone",
+    )
+    product_id: int = Field(..., description="ID of the product to act on")
+    discount_pct: Literal[0, 10, 20, 50] = Field(
+        default=0, description="Discount percentage (used when action_type='discount'): 0, 10, 20, or 50"
+    )
+    restock_quantity: Optional[int] = Field(
+        default=None, description="Number of units to order (used when action_type='restock'); must be > 0"
+    )
+    target_zone: Optional[Literal[1, 2, 3]] = Field(
+        default=None, description="Destination zone (used when action_type='place'): 1=premium, 2=standard, 3=back"
+    )
+
+
+class StoremanagerObservation(Observation):
+    """Full observation returned to the agent after each step."""
+
+    inventory: List[ProductState] = Field(
+        default_factory=list,
+        description="Current state of all products (including inactive ones)",
+    )
+    current_step: int = Field(default=0, description="Current step number (0-indexed)")
+    max_steps: int = Field(default=30, description="Total steps in this episode")
+    steps_remaining: int = Field(
+        default=30, description="Steps remaining in the episode"
+    )
+    cumulative_profit: float = Field(
+        default=0.0, description="Total profit accumulated so far in the episode"
+    )
+    last_step_profit: float = Field(
+        default=0.0, description="Net profit from the most recent step"
+    )
+    last_step_units_sold: int = Field(
+        default=0, description="Total units sold across all products in the last step"
+    )
+    last_expired_products: List[str] = Field(
+        default_factory=list,
+        description="Names of products that expired at the end of the last step",
+    )
+    num_customers_per_step: int = Field(
+        default=10, description="Fixed number of customers simulated per step"
+    )
+    task_name: str = Field(
+        default="custom",
+        description="Active difficulty task: easy / medium / hard / expert / custom",
+    )
+    error: Optional[str] = Field(
+        default=None, description="Error message if the last action was invalid"
+    )
+    # Zone information
+    zone_capacity: Dict[int, int] = Field(
+        default_factory=lambda: {1: 2, 2: 4, 3: 99},
+        description="Maximum number of products allowed per zone",
+    )
+    zone_occupancy: Dict[int, int] = Field(
+        default_factory=lambda: {1: 0, 2: 0, 3: 0},
+        description="Current number of active products in each zone",
+    )
+    zone_multipliers: Dict[int, float] = Field(
+        default_factory=lambda: {1: 1.5, 2: 1.0, 3: 0.6},
+        description="Pick-probability multiplier per zone",
+    )
+    # Step cost breakdown
+    holding_cost_this_step: float = Field(
+        default=0.0, description="Total holding cost charged this step across all products"
+    )
+    placement_cost_this_step: float = Field(
+        default=0.0, description="Labor cost charged this step if a zone placement was made"
+    )
+    restock_cost_this_step: float = Field(
+        default=0.0, description="Ordering cost charged this step if a restock order was placed"
+    )
+    unjustified_discount_penalty: float = Field(
+        default=0.0,
+        description=(
+            "Extra penalty this step for selling below cost_price when steps_until_expiry >= 4. "
+            "Equals the total per-unit loss on unjustified below-cost units sold. "
+            "Zero when below-cost selling is justified by near-expiry urgency."
+        ),
+    )
+
+
+class StoremanagerState(State):
+    """Internal state snapshot for debugging and orchestration."""
+
+    num_products: int = Field(default=8, description="Number of products in this episode")
+    max_steps: int = Field(default=30, description="Episode length in steps")
+    cumulative_profit: float = Field(default=0.0, description="Total episode profit so far")
+    last_reward: float = Field(default=0.0, description="Reward from the last step")
+    active_product_count: int = Field(
+        default=0, description="Number of currently active (non-expired, in-stock) products"
+    )
+    inventory_snapshot: List[Dict] = Field(
+        default_factory=list,
+        description="Serialised inventory for state introspection",
+    )

openenv.yaml

@@ -0,0 +1,7 @@
+spec_version: 1
+name: StoreManager
+type: space
+runtime: fastapi
+app: server.app:app
+port: 8000
+

pyproject.toml

@@ -0,0 +1,47 @@
+# Copyright (c) Meta Platforms, Inc. and affiliates.
+# All rights reserved.
+#
+# This source code is licensed under the BSD-style license found in the
+# LICENSE file in the root directory of this source tree.
+
+[build-system]
+requires = ["setuptools>=45", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "openenv-StoreManager"
+version = "0.1.0"
+description = "Storemanager environment for OpenEnv"
+requires-python = ">=3.10"
+dependencies = [
+    # Core OpenEnv runtime (provides FastAPI server + HTTP client types)
+    # install from github
+    # "openenv-core[core] @ git+https://github.com/meta-pytorch/OpenEnv.git",
+    "openenv-core[core]>=0.2.1",
+    # Environment-specific dependencies
+    "numpy>=1.24.0",
+    # Inference script dependencies
+    "websocket-client>=1.6.0",
+    "openai>=1.0.0",
+    # Examples:
+    # "torch>=2.0.0",
+    # "gymnasium>=0.29.0",
+    # "openspiel>=1.0.0",
+    # "smolagents>=1.22.0,<2",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=8.0.0",
+    "pytest-cov>=4.0.0",
+]
+
+[project.scripts]
+# Server entry point - enables running via: uv run --project . server
+# or: python -m StoreManager.server.app
+server = "StoreManager.server.app:main"
+
+[tool.setuptools]
+include-package-data = true
+packages = ["StoreManager", "StoreManager.server"]
+package-dir = { "StoreManager" = ".", "StoreManager.server" = "server" }

server/StoreManager_environment.py

@@ -0,0 +1,681 @@
+# Copyright (c) Meta Platforms, Inc. and affiliates.
+# All rights reserved.
+#
+# This source code is licensed under the BSD-style license found in the
+# LICENSE file in the root directory of this source tree.
+
+"""
+Store Manager Environment Implementation.
+
+A retail store simulator where an LLM agent acts as the store manager.
+The agent can apply discounts, restock inventory, or move products between
+shelf zones each step to maximise cumulative profit over a fixed episode horizon.
+
+Episode dynamics:
+  - N products, each with a cost price, selling price, stock quantity,
+    expiry step, a base pick-probability weight, a shelf zone, and restock params.
+  - Each step the agent chooses one action:
+      - "discount": apply a discount (0/10/20/50%) to one product
+      - "restock":  place a reorder for a product (delivered after lead_time steps)
+      - "place":    move a product to a different shelf zone (1/2/3)
+  - Discounts are ephemeral — they apply only for the current step.
+  - Zone multiplier scales pick probability: zone 1 = 1.5x, 2 = 1.0x, 3 = 0.6x.
+  - 10 customers sample products per step; probabilities incorporate zone + discount.
+  - Holding cost is charged every step based on quantity on hand.
+  - Restock cost is charged upfront; stock arrives after lead_time steps.
+  - Placement cost is a flat labor fee per move.
+  - Reward = step_revenue - step_cost - expiry_penalty - holding_cost
+             - placement_cost - restock_cost.
+  - Expiry penalty = remaining_quantity * cost_price for each expired product.
+  - Episode ends after max_steps steps, or when all products are inactive.
+"""
+
+from __future__ import annotations
+
+import sys
+import os
+
+# Ensure the project root (parent of server/) is on sys.path so that
+# models.py and tasks.py are importable whether this file is run directly
+# or imported as part of the package.
+_ROOT = os.path.join(os.path.dirname(__file__), "..")
+if _ROOT not in sys.path:
+    sys.path.insert(0, _ROOT)
+
+import math
+from typing import Any, Dict, List, Optional
+from uuid import uuid4
+
+import numpy as np
+from openenv.core.env_server.interfaces import Environment
+
+from models import (
+    ProductState,
+    StoremanagerAction,
+    StoremanagerObservation,
+    StoremanagerState,
+)
+
+try:
+    from tasks import TaskConfig
+except ImportError:
+    TaskConfig = None  # type: ignore[assignment,misc]
+
+# ── Curated product catalogue ──────────────────────────────────────────────────
+_PRODUCT_NAMES: List[str] = [
+    "Milk", "Bread", "Cheese", "Yogurt", "Juice", "Butter",
+    "Eggs", "Cream", "Honey", "Jam", "Soda", "Water",
+    "Tea", "Coffee", "Flour", "Rice",
+]
+
+# ── Zone constants ─────────────────────────────────────────────────────────────
+ZONE_MULTIPLIERS: Dict[int, float] = {1: 1.5, 2: 1.0, 3: 0.6}
+ZONE_CAPACITY: Dict[int, int] = {1: 2, 2: 4, 3: 99}
+PLACEMENT_COST: float = 3.0  # flat labor fee per zone move
+
+
+class StoremanagerEnvironment(Environment):
+    """
+    Retail store manager RL environment.
+
+    The agent observes full inventory state and chooses one action per step:
+    apply a discount, place a restock order, or move a product to a better zone.
+
+    Args:
+        num_products: Number of distinct products in the store (default 8).
+        max_steps: Episode length in steps (default 30).
+        num_customers: Customers simulated per step (default 10).
+        seed: Optional base random seed for reproducible episodes.
+    """
+
+    SUPPORTS_CONCURRENT_SESSIONS: bool = True
+
+    def __init__(
+        self,
+        task_config: Optional["TaskConfig"] = None,  # type: ignore[name-defined]
+        num_products: int = 8,
+        max_steps: int = 30,
+        num_customers: int = 10,
+        seed: Optional[int] = None,
+    ) -> None:
+        super().__init__()
+        if task_config is not None:
+            num_products = task_config.num_products
+            max_steps = task_config.max_steps
+            num_customers = task_config.num_customers
+            self._expiry_offset_min = task_config.expiry_offset_min
+            self._expiry_offset_max = task_config.expiry_offset_max
+            self._task_name = task_config.name
+            self._profit_target = task_config.profit_target
+        else:
+            self._expiry_offset_min = 8
+            self._expiry_offset_max = 25
+            self._task_name = "custom"
+            # Default: assume ~$8 average profit per step for custom configs
+            self._profit_target = 8.0 * max_steps
+
+        self._num_products = min(num_products, len(_PRODUCT_NAMES))
+        self._max_steps = max_steps
+        self._num_customers = num_customers
+        self._base_seed = seed
+
+        self._rng: np.random.Generator = np.random.default_rng(seed)
+        self._inventory: List[ProductState] = []
+        self._pending_orders: List[Dict[str, Any]] = []
+        self._current_step: int = 0
+        self._cumulative_profit: float = 0.0
+
+        self._state = StoremanagerState(
+            num_products=self._num_products,
+            max_steps=self._max_steps,
+        )
+
+    # ── OpenEnv API ───────────────────────────────────────────────────────────
+
+    def reset(
+        self,
+        seed: Optional[int] = None,
+        episode_id: Optional[str] = None,
+        **kwargs: Any,
+    ) -> StoremanagerObservation:
+        """Reset the store and generate a fresh inventory.
+
+        Accepts an optional ``task`` keyword argument to dynamically switch the
+        task configuration without restarting the server.  This allows a single
+        server instance to serve all difficulty levels sequentially.
+        """
+        # ── Dynamic task reconfiguration ────────────────────────────────────
+        task_name = kwargs.get("task")
+        if task_name:
+            try:
+                from tasks import TASKS  # local import to avoid circular deps
+                task_config = TASKS.get(task_name)
+                if task_config is not None:
+                    self._num_products = min(task_config.num_products, len(_PRODUCT_NAMES))
+                    self._max_steps = task_config.max_steps
+                    self._num_customers = task_config.num_customers
+                    self._expiry_offset_min = task_config.expiry_offset_min
+                    self._expiry_offset_max = task_config.expiry_offset_max
+                    self._task_name = task_config.name
+                    self._profit_target = task_config.profit_target
+            except ImportError:
+                pass  # tasks module unavailable; keep current config
+
+        self._reset_rubric()
+
+        effective_seed = seed if seed is not None else self._base_seed
+        self._rng = np.random.default_rng(effective_seed)
+
+        self._current_step = 0
+        self._cumulative_profit = 0.0
+        self._pending_orders = []
+        self._inventory = self._generate_inventory()
+        self._recompute_pick_probs()
+
+        self._update_state(episode_id=episode_id or str(uuid4()), last_reward=0.0)
+        self._refresh_product_derived_fields()
+
+        return StoremanagerObservation(
+            inventory=list(self._inventory),
+            current_step=0,
+            max_steps=self._max_steps,
+            steps_remaining=self._max_steps,
+            cumulative_profit=0.0,
+            last_step_profit=0.0,
+            last_step_units_sold=0,
+            last_expired_products=[],
+            num_customers_per_step=self._num_customers,
+            task_name=self._task_name,
+            zone_capacity=dict(ZONE_CAPACITY),
+            zone_occupancy=self._zone_occupancy(),
+            zone_multipliers=dict(ZONE_MULTIPLIERS),
+            holding_cost_this_step=0.0,
+            placement_cost_this_step=0.0,
+            restock_cost_this_step=0.0,
+            unjustified_discount_penalty=0.0,
+            done=False,
+            reward=0.0,
+        )
+
+    def step(
+        self,
+        action: StoremanagerAction,
+        timeout_s: Optional[float] = None,
+        **kwargs: Any,
+    ) -> StoremanagerObservation:
+        """Execute one step: process deliveries, apply action, simulate customers, handle expiry."""
+        # ── 1. Validate product_id ──────────────────────────────────────────
+        valid_ids = {p.product_id for p in self._inventory}
+        if action.product_id not in valid_ids:
+            return self._error_observation(
+                f"Invalid product_id {action.product_id}. "
+                f"Valid IDs: {sorted(valid_ids)}"
+            )
+
+        # ── 2. Process pending deliveries ───────────────────────────────────
+        delivered = [o for o in self._pending_orders if o["arrive_at_step"] <= self._current_step]
+        for order in delivered:
+            product = next((p for p in self._inventory if p.product_id == order["product_id"]), None)
+            if product is not None:
+                product.quantity += order["quantity"]
+                product.is_active = True
+                product.pending_restock_quantity = 0
+                product.pending_restock_arrives_at_step = None
+        self._pending_orders = [o for o in self._pending_orders if o["arrive_at_step"] > self._current_step]
+
+        # ── 3. Compute holding cost (charged before action) ──────────���──────
+        holding_cost = sum(
+            p.quantity * p.cost_price * p.holding_cost_rate
+            for p in self._inventory
+            if p.is_active and p.quantity > 0
+        )
+
+        # ── 4. Reset per-step discount tracker (selling_price persists across steps) ──
+        # current_discount_pct tracks only this step's discount for the pick-prob formula.
+        # selling_price is NOT reset — discounts compound: each action cuts the current price.
+        for product in self._inventory:
+            product.current_discount_pct = 0.0
+
+        # ── 5. Dispatch action ──────────────────────────────────────────────
+        placement_cost = 0.0
+        restock_cost = 0.0
+        target = next(p for p in self._inventory if p.product_id == action.product_id)
+
+        if action.action_type == "discount":
+            if not target.is_active:
+                return self._error_observation(
+                    f"Product '{target.name}' (id={target.product_id}) is no longer active "
+                    f"(sold out or expired). Choose an active product to discount."
+                )
+            if target.quantity <= 0:
+                return self._error_observation(
+                    f"Product '{target.name}' (id={target.product_id}) has no stock remaining."
+                )
+            target.current_discount_pct = float(action.discount_pct)
+            # Phase 1: dynamic expiry-aware price floor.
+            # Far from expiry: protect margin — floor at cost_price.
+            # Near expiry: allow progressive below-cost markdown to clear stock
+            # before the far-worse full expiry penalty hits.
+            steps_left = target.expiry_step - self._current_step
+            if steps_left >= 4:
+                price_floor = target.cost_price           # full protection
+            elif steps_left == 3:
+                price_floor = round(target.cost_price * 0.70, 2)  # 30% loss/unit max
+            elif steps_left == 2:
+                price_floor = round(target.cost_price * 0.40, 2)  # 60% loss/unit max
+            else:                                                    # 1 step left
+                price_floor = round(target.cost_price * 0.10, 2)  # 90% loss/unit max
+            target.selling_price = round(
+                max(price_floor, target.selling_price * (1.0 - action.discount_pct / 100.0)),
+                2,
+            )
+            # popularity_multiplier compounds up: repeated discounting makes the
+            # product progressively more visible/attractive to customers
+            if action.discount_pct > 0:
+                target.popularity_multiplier = round(
+                    target.popularity_multiplier * (1.0 + action.discount_pct / 100.0), 6
+                )
+
+        elif action.action_type == "restock":
+            qty = action.restock_quantity
+            if qty is None or qty <= 0:
+                return self._error_observation(
+                    f"restock_quantity must be a positive integer, got {qty!r}."
+                )
+            if target.expiry_step <= self._current_step:
+                return self._error_observation(
+                    f"Product '{target.name}' (id={target.product_id}) has expired "
+                    f"(expiry_step={target.expiry_step}, current_step={self._current_step}). "
+                    f"Restocking an expired product wastes money — the stock would "
+                    f"immediately re-expire and incur an expiry penalty."
+                )
+            if target.pending_restock_quantity > 0:
+                return self._error_observation(
+                    f"Product '{target.name}' already has a pending restock order "
+                    f"for {target.pending_restock_quantity} units arriving at step "
+                    f"{target.pending_restock_arrives_at_step}. Wait for delivery first."
+                )
+            arrive_at = self._current_step + target.restock_lead_time
+            if arrive_at >= target.expiry_step:
+                return self._error_observation(
+                    f"Restock order for '{target.name}' would arrive at step {arrive_at} "
+                    f"but the product expires at step {target.expiry_step}. "
+                    f"The delivered units would immediately expire and be penalized. "
+                    f"Order cancelled to prevent wasted spend."
+                )
+            self._pending_orders.append({
+                "product_id": target.product_id,
+                "quantity": qty,
+                "arrive_at_step": arrive_at,
+            })
+            target.pending_restock_quantity = qty
+            target.pending_restock_arrives_at_step = arrive_at
+            restock_cost = qty * target.restock_cost_per_unit
+
+        elif action.action_type == "place":
+            zone = action.target_zone
+            if zone is None:
+                return self._error_observation("target_zone must be specified for 'place' actions.")
+            if zone == target.zone:
+                return self._error_observation(
+                    f"Product '{target.name}' is already in zone {zone}."
+                )
+            # Count active products currently in the target zone (excluding this product)
+            occupants = sum(
+                1 for p in self._inventory
+                if p.is_active and p.zone == zone and p.product_id != target.product_id
+            )
+            if occupants >= ZONE_CAPACITY[zone]:
+                return self._error_observation(
+                    f"Zone {zone} is at capacity ({ZONE_CAPACITY[zone]} products). "
+                    f"Move another product out first."
+                )
+            target.zone = zone
+            placement_cost = PLACEMENT_COST
+
+        # ── 6. Gather sellable products ─────────────────────────────────────
+        active = [p for p in self._inventory if p.is_active and p.quantity > 0]
+        if not active:
+            done = True
+            self._update_state(last_reward=0.0)
+            self._refresh_product_derived_fields()
+            return StoremanagerObservation(
+                inventory=list(self._inventory),
+                current_step=self._current_step,
+                max_steps=self._max_steps,
+                steps_remaining=max(0, self._max_steps - self._current_step),
+                cumulative_profit=self._cumulative_profit,
+                last_step_profit=0.0,
+                last_step_units_sold=0,
+                last_expired_products=[],
+                num_customers_per_step=self._num_customers,
+                task_name=self._task_name,
+                zone_capacity=dict(ZONE_CAPACITY),
+                zone_occupancy=self._zone_occupancy(),
+                zone_multipliers=dict(ZONE_MULTIPLIERS),
+                holding_cost_this_step=round(holding_cost, 4),
+                placement_cost_this_step=placement_cost,
+                restock_cost_this_step=round(restock_cost, 4),
+                unjustified_discount_penalty=0.0,
+                done=done,
+                reward=self._normalize_reward(0.0),
+            )
+
+        # ── 7. Compute pick probabilities (zone + discount + popularity) ────
+        effective_weights = np.array([
+            p.base_probability
+            * ZONE_MULTIPLIERS[p.zone]
+            * (1.0 + p.current_discount_pct / 100.0)
+            * p.popularity_multiplier
+            for p in active
+        ])
+        pick_probs = effective_weights / effective_weights.sum()
+
+        for product, prob in zip(active, pick_probs):
+            product.effective_pick_prob = round(float(prob), 6)
+
+        # ── 8. Simulate customers ───────────────────────────────────────────
+        chosen_ids = self._rng.choice(
+            a=[p.product_id for p in active],
+            size=self._num_customers,
+            p=pick_probs,
+            replace=True,
+        )
+        sales_counts: dict[int, int] = {}
+        for pid in chosen_ids:
+            sales_counts[pid] = sales_counts.get(int(pid), 0) + 1
+
+        # ── 9. Apply sales (clamp by available quantity) ────────────────────
+        step_revenue = 0.0
+        step_cost = 0.0
+        units_sold_total = 0
+        unjustified_discount_penalty = 0.0
+
+        for product in active:
+            wanted = sales_counts.get(product.product_id, 0)
+            sold = min(wanted, product.quantity)
+            if sold > 0:
+                step_revenue += sold * product.selling_price
+                step_cost += sold * product.cost_price
+                product.quantity -= sold
+                units_sold_total += sold
+                product.sales_history.append(
+                    {
+                        "step": self._current_step,
+                        "discount_pct": product.current_discount_pct,
+                        "units_sold": sold,
+                    }
+                )
+                # Phase 3: if selling below cost with no urgency, double the per-unit
+                # loss as an extra penalty — gives a sharp "this is bad" signal.
+                if product.selling_price < product.cost_price:
+                    steps_left = product.expiry_step - self._current_step
+                    if steps_left >= 4:
+                        unjustified_discount_penalty += sold * (product.cost_price - product.selling_price)
+            if product.quantity == 0:
+                product.is_active = False
+                product.effective_pick_prob = 0.0
+
+        # ── 10. Advance step counter ────────────────────────────────────────
+        self._current_step += 1
+
+        # ── 11. Check expiry ────────────────────────────────────────────────
+        expiry_penalty = 0.0
+        expired_names: List[str] = []
+        for product in self._inventory:
+            if product.is_active and product.expiry_step <= self._current_step:
+                expiry_penalty += product.quantity * product.cost_price
+                expired_names.append(product.name)
+                product.is_active = False
+                product.quantity = 0
+                product.effective_pick_prob = 0.0
+
+        # ── 12. Reward and cumulative profit ────────────────────────────────
+        step_profit = (
+            step_revenue - step_cost - expiry_penalty
+            - holding_cost - placement_cost - restock_cost
+            - unjustified_discount_penalty
+        )
+        self._cumulative_profit += step_profit
+
+        # ── 13. Terminal condition ───────────────────────────────────────────
+        done = self._current_step >= self._max_steps or all(
+            not p.is_active for p in self._inventory
+        )
+
+        # ── 14. State update and return ─────────────────────────────────────
+        normalized_reward = self._normalize_reward(step_profit)
+        self._update_state(last_reward=normalized_reward)
+        self._refresh_product_derived_fields()
+
+        return StoremanagerObservation(
+            inventory=list(self._inventory),
+            current_step=self._current_step,
+            max_steps=self._max_steps,
+            steps_remaining=max(0, self._max_steps - self._current_step),
+            cumulative_profit=self._cumulative_profit,
+            last_step_profit=step_profit,
+            last_step_units_sold=units_sold_total,
+            last_expired_products=expired_names,
+            num_customers_per_step=self._num_customers,
+            task_name=self._task_name,
+            zone_capacity=dict(ZONE_CAPACITY),
+            zone_occupancy=self._zone_occupancy(),
+            zone_multipliers=dict(ZONE_MULTIPLIERS),
+            holding_cost_this_step=round(holding_cost, 4),
+            placement_cost_this_step=placement_cost,
+            restock_cost_this_step=round(restock_cost, 4),
+            unjustified_discount_penalty=round(unjustified_discount_penalty, 4),
+            done=done,
+            reward=normalized_reward,
+            metadata={
+                "step_revenue": round(step_revenue, 4),
+                "step_cost": round(step_cost, 4),
+                "expiry_penalty": round(expiry_penalty, 4),
+                "holding_cost": round(holding_cost, 4),
+                "placement_cost": placement_cost,
+                "restock_cost": round(restock_cost, 4),
+                "unjustified_discount_penalty": round(unjustified_discount_penalty, 4),
+                "step_profit": round(step_profit, 4),
+            },
+        )
+
+    @property
+    def state(self) -> StoremanagerState:
+        return self._state
+
+    # ── Private helpers ───────────────────────────────────────────────────────
+
+    def _normalize_reward(self, step_profit: float) -> float:
+        """
+        Map raw step profit to a meaningful reward in (0.0, 1.0) using a
+        sigmoid centered at break-even (step_profit = 0).
+
+        Formula:
+            x = step_profit / per_step_target   (normalise by average needed per step)
+            reward = sigmoid(k * x) = 1 / (1 + exp(-k * x))
+
+        Where k=2 gives a useful spread:
+            step_profit =  0            -> 0.50  (neutral, broke even)
+            step_profit =  per_step_tgt -> 0.88  (on-track for target)
+            step_profit = 2*per_step_tgt -> 0.98 (excellent step)
+            step_profit = -per_step_tgt -> 0.12  (below break-even)
+            step_profit = -2*per_step_tgt -> 0.02 (large penalty step)
+
+        This gives the model a continuous, differentiable gradient signal
+        across the full [0, 1] range — much more informative than a hard clamp.
+        Raw dollar profit is preserved in last_step_profit and metadata.
+        """
+        per_step_target = self._profit_target / self._max_steps
+        x = step_profit / per_step_target if per_step_target != 0 else 0.0
+        return round(1.0 / (1.0 + math.exp(-2.0 * x)), 6)
+
+    def _zone_occupancy(self) -> Dict[int, int]:
+        """Count active products in each zone."""
+        occupancy: Dict[int, int] = {1: 0, 2: 0, 3: 0}
+        for p in self._inventory:
+            if p.is_active and p.zone in occupancy:
+                occupancy[p.zone] += 1
+        return occupancy
+
+    def _refresh_product_derived_fields(self) -> None:
+        """Recompute ephemeral derived fields on every product before returning an observation."""
+        for p in self._inventory:
+            p.steps_until_expiry = max(0, p.expiry_step - self._current_step)
+            p.margin_per_unit = round(p.selling_price - p.cost_price, 4)
+
+    def _recompute_pick_probs(self) -> None:
+        """Recompute normalised pick probabilities for all active products (zone + popularity)."""
+        active = [p for p in self._inventory if p.is_active and p.quantity > 0]
+        if not active:
+            return
+        effs = np.array([
+            p.base_probability
+            * ZONE_MULTIPLIERS[p.zone]
+            * (1.0 + p.current_discount_pct / 100.0)
+            * p.popularity_multiplier
+            for p in active
+        ])
+        probs = effs / effs.sum()
+        for product, prob in zip(active, probs):
+            product.effective_pick_prob = round(float(prob), 6)
+
+    def _generate_inventory(self) -> List[ProductState]:
+        """Generate a randomised product inventory for a new episode."""
+        chosen_names = self._rng.choice(
+            _PRODUCT_NAMES, size=self._num_products, replace=False
+        )
+
+        # Draw base probability weights via Dirichlet, then rescale to [0.05, 0.4]
+        raw_probs = self._rng.dirichlet(alpha=[1.0] * self._num_products)
+        max_raw = raw_probs.max()
+        base_probs = 0.05 + raw_probs * (0.40 - 0.05) / max_raw
+
+        # Assign initial zones: fill zone 1 first (up to capacity), then zone 2, rest to zone 3
+        zone_slots = {z: ZONE_CAPACITY[z] for z in [1, 2, 3]}
+        product_zones: List[int] = []
+        for _ in range(self._num_products):
+            for zone in [1, 2, 3]:
+                if zone_slots[zone] > 0:
+                    product_zones.append(zone)
+                    zone_slots[zone] -= 1
+                    break
+
+        inventory: List[ProductState] = []
+        for i in range(self._num_products):
+            cost = round(float(self._rng.uniform(1.0, 8.0)), 2)
+            margin = float(self._rng.uniform(1.3, 2.5))
+            selling = round(cost * margin, 2)
+            qty = int(self._rng.integers(15, 41))
+            expiry_offset = int(self._rng.integers(self._expiry_offset_min, self._expiry_offset_max + 1))
+            holding_rate = round(float(self._rng.uniform(0.01, 0.04)), 4)
+            restock_cost_per_unit = round(cost * float(self._rng.uniform(1.0, 1.25)), 2)
+            lead_time = int(self._rng.integers(1, 4))  # 1–3 steps
+
+            inventory.append(
+                ProductState(
+                    product_id=i,
+                    name=str(chosen_names[i]),
+                    cost_price=cost,
+                    selling_price=selling,
+                    base_selling_price=selling,
+                    quantity=qty,
+                    expiry_step=self._current_step + expiry_offset,
+                    base_probability=float(base_probs[i]),
+                    current_discount_pct=0.0,
+                    is_active=True,
+                    sales_history=[],
+                    zone=product_zones[i],
+                    holding_cost_rate=holding_rate,
+                    restock_cost_per_unit=restock_cost_per_unit,
+                    restock_lead_time=lead_time,
+                    pending_restock_quantity=0,
+                    pending_restock_arrives_at_step=None,
+                    popularity_multiplier=1.0,
+                )
+            )
+        return inventory
+
+    def _update_state(
+        self,
+        *,
+        episode_id: Optional[str] = None,
+        last_reward: float = 0.0,
+    ) -> None:
+        """Sync the internal StoremanagerState with current environment data."""
+        if episode_id is not None:
+            self._state.episode_id = episode_id
+        self._state.step_count = self._current_step
+        self._state.num_products = self._num_products
+        self._state.max_steps = self._max_steps
+        self._state.cumulative_profit = self._cumulative_profit
+        self._state.last_reward = last_reward
+        self._state.active_product_count = sum(
+            1 for p in self._inventory if p.is_active
+        )
+        self._state.inventory_snapshot = [p.model_dump() for p in self._inventory]
+
+    def _error_observation(self, message: str) -> StoremanagerObservation:
+        """Return an observation flagging an invalid action (step does NOT advance)."""
+        self._refresh_product_derived_fields()
+        return StoremanagerObservation(
+            inventory=list(self._inventory),
+            current_step=self._current_step,
+            max_steps=self._max_steps,
+            steps_remaining=max(0, self._max_steps - self._current_step),
+            cumulative_profit=self._cumulative_profit,
+            last_step_profit=0.0,
+            last_step_units_sold=0,
+            last_expired_products=[],
+            num_customers_per_step=self._num_customers,
+            task_name=self._task_name,
+            zone_capacity=dict(ZONE_CAPACITY),
+            zone_occupancy=self._zone_occupancy(),
+            zone_multipliers=dict(ZONE_MULTIPLIERS),
+            holding_cost_this_step=0.0,
+            placement_cost_this_step=0.0,
+            restock_cost_this_step=0.0,
+            unjustified_discount_penalty=0.0,
+            done=False,
+            reward=0.0,
+            error=message,
+        )
+
+
+# ── Standalone smoke test ─────────────────────────────────────────────────────
+
+if __name__ == "__main__":
+    env = StoremanagerEnvironment(num_products=4, max_steps=6, seed=42)
+    obs = env.reset()
+    print("=== RESET ===")
+    for p in obs.inventory:
+        print(f"  [{p.product_id}] {p.name:10s}  zone={p.zone}  qty={p.quantity}  "
+              f"lead={p.restock_lead_time}  hold_rate={p.holding_cost_rate:.3f}")
+    print(f"  Zone occupancy: {obs.zone_occupancy}")
+
+    actions = [
+        StoremanagerAction(action_type="discount", product_id=0, discount_pct=20),
+        StoremanagerAction(action_type="restock", product_id=1, restock_quantity=10),
+        StoremanagerAction(action_type="place", product_id=2, target_zone=1),
+        StoremanagerAction(action_type="place", product_id=3, target_zone=1),  # should fail — zone 1 at capacity
+        StoremanagerAction(action_type="discount", product_id=0, discount_pct=10),
+        StoremanagerAction(action_type="discount", product_id=1, discount_pct=0),
+    ]
+
+    for step_num, action in enumerate(actions):
+        obs = env.step(action)
+        print(f"\n=== STEP {step_num + 1} ({action.action_type.upper()}) ===")
+        if obs.error:
+            print(f"  ERROR: {obs.error}")
+        else:
+            print(f"  Reward:          {obs.reward:.2f}")
+            print(f"  Holding cost:    {obs.holding_cost_this_step:.4f}")
+            print(f"  Placement cost:  {obs.placement_cost_this_step:.2f}")
+            print(f"  Restock cost:    {obs.restock_cost_this_step:.4f}")
+            print(f"  Cumulative:      {obs.cumulative_profit:.2f}")
+            print(f"  Zone occupancy:  {obs.zone_occupancy}")
+            if obs.last_expired_products:
+                print(f"  Expired:         {obs.last_expired_products}")
+        if obs.done:
+            print("  Episode done.")
+            break

server/__init__.py

@@ -0,0 +1,11 @@
+# Copyright (c) Meta Platforms, Inc. and affiliates.
+# All rights reserved.
+#
+# This source code is licensed under the BSD-style license found in the
+# LICENSE file in the root directory of this source tree.
+
+"""Storemanager environment server components."""
+
+from .StoreManager_environment import StoremanagerEnvironment
+
+__all__ = ["StoremanagerEnvironment"]

server/app.py

@@ -0,0 +1,104 @@
+# Copyright (c) Meta Platforms, Inc. and affiliates.
+# All rights reserved.
+#
+# This source code is licensed under the BSD-style license found in the
+# LICENSE file in the root directory of this source tree.
+
+"""
+FastAPI application for the Store Manager Environment.
+
+This module creates an HTTP + WebSocket server that exposes the
+StoremanagerEnvironment over the OpenEnv API.
+
+Endpoints:
+    POST /reset  — Reset the environment, return initial observation
+    POST /step   — Execute an action, return next observation + reward
+    GET  /state  — Get current environment state
+    GET  /schema — Action / Observation / State JSON schemas
+    WS   /ws     — Persistent WebSocket session (lower latency)
+
+Configuration (via environment variables):
+    STORE_NUM_PRODUCTS       Number of products per episode  (default 8)
+    STORE_MAX_STEPS          Episode length in steps         (default 30)
+    STORE_NUM_CUSTOMERS      Customers simulated per step    (default 10)
+    STORE_MAX_CONCURRENT_ENVS Max parallel WebSocket sessions (default 4)
+
+Usage:
+    # Development (with auto-reload):
+    uvicorn server.app:app --reload --host 0.0.0.0 --port 8000
+
+    # Production:
+    uvicorn server.app:app --host 0.0.0.0 --port 8000 --workers 4
+
+    # Or run directly:
+    python -m server.app
+"""
+
+import os
+
+try:
+    from openenv.core.env_server.http_server import create_app
+except Exception as e:  # pragma: no cover
+    raise ImportError(
+        "openenv is required. Install dependencies with:\n    uv sync\n"
+    ) from e
+
+try:
+    from models import StoremanagerAction, StoremanagerObservation
+    from StoreManager_environment import StoremanagerEnvironment
+    from tasks import TASKS
+except ModuleNotFoundError:
+    from models import StoremanagerAction, StoremanagerObservation
+    from server.StoreManager_environment import StoremanagerEnvironment
+    from tasks import TASKS
+
+
+def create_store_environment() -> StoremanagerEnvironment:
+    """Factory: create a fresh StoremanagerEnvironment from env-var config.
+
+    If STORE_TASK is set to a valid task name (easy/medium/hard), the task
+    config takes precedence over individual STORE_* vars.
+    """
+    task_name = os.getenv("STORE_TASK", "")
+    task_config = TASKS.get(task_name) if task_name else None
+    return StoremanagerEnvironment(
+        task_config=task_config,
+        num_products=int(os.getenv("STORE_NUM_PRODUCTS", "8")),
+        max_steps=int(os.getenv("STORE_MAX_STEPS", "30")),
+        num_customers=int(os.getenv("STORE_NUM_CUSTOMERS", "10")),
+    )
+
+
+app = create_app(
+    create_store_environment,
+    StoremanagerAction,
+    StoremanagerObservation,
+    env_name="StoreManager",
+    max_concurrent_envs=int(os.getenv("STORE_MAX_CONCURRENT_ENVS", "4")),
+)
+
+
+def main(host: str = "0.0.0.0", port: int = 8000) -> None:
+    """
+    Entry point for direct execution via uv run or python -m.
+
+        uv run --project . server
+        uv run --project . server --port 8001
+        python -m StoreManager.server.app
+
+    Args:
+        host: Host address to bind to (default: "0.0.0.0")
+        port: Port number to listen on (default: 8000)
+    """
+    import uvicorn
+
+    uvicorn.run(app, host=host, port=port)
+
+
+if __name__ == "__main__":
+    import argparse
+
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--port", type=int, default=8000)
+    args = parser.parse_args()
+    main(port=args.port)

server/requirements.txt

@@ -0,0 +1,9 @@
+openenv-core[core]>=0.2.1
+fastapi>=0.115.0
+uvicorn>=0.24.0
+numpy>=1.24.0
+websocket-client>=1.6.0
+openai>=1.0.0
+
+
+

tasks.py

@@ -0,0 +1,139 @@
+# Copyright (c) Meta Platforms, Inc. and affiliates.
+# All rights reserved.
+#
+# This source code is licensed under the BSD-style license found in the
+# LICENSE file in the root directory of this source tree.
+
+"""
+Task configurations and graders for the Store Manager environment.
+
+Four difficulty levels are defined, in strict increasing order of complexity:
+
+  - easy   : 4 products, 20 steps — learn discount mechanics, no expiry pressure
+  - medium : 8 products, 28 steps — discount + zone placement (zone 1 is contested)
+  - hard   : 10 products, 30 steps — discount + zone + expiry timing (most expire mid-episode)
+  - expert : 12 products, 25 steps — all three actions required (restock essential, aggressive expiry)
+
+Each task has a fixed seed so grading is deterministic: the same agent always
+produces the same episode given the same seed, and the score is computed as:
+
+    score = clamp(cumulative_profit / profit_target, 0.0, 1.0)
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import Dict
+
+
+@dataclass
+class TaskConfig:
+    """Configuration for one difficulty level of the Store Manager task."""
+
+    name: str
+    description: str
+    num_products: int
+    max_steps: int
+    num_customers: int
+    expiry_offset_min: int  # min steps before a product expires
+    expiry_offset_max: int  # max steps before a product expires
+    seed: int               # fixed seed for deterministic episode generation
+    profit_target: float    # cumulative profit at which grade == 1.0
+
+
+# ── Canonical task registry ────────────────────────────────────────────────────
+
+TASKS: Dict[str, TaskConfig] = {
+    "easy": TaskConfig(
+        name="easy",
+        description=(
+            "4 products, 20 steps, 15 customers/step. "
+            "Products expire well after the episode ends — no expiry pressure. "
+            "Focus on learning the discount mechanic to boost high-margin products. "
+            "Zone placement is available but low-stakes; restocking is unnecessary."
+        ),
+        num_products=4,
+        max_steps=20,
+        num_customers=15,
+        expiry_offset_min=18,
+        expiry_offset_max=30,
+        seed=42,
+        profit_target=120.0,
+    ),
+    "medium": TaskConfig(
+        name="medium",
+        description=(
+            "8 products, 28 steps, 12 customers/step. "
+            "Several products expire mid-episode (12-22 steps). "
+            "Zone 1 has only 2 slots for 8 competing products — "
+            "combine targeted discounts with zone placement to maximize clearance "
+            "before expiry. Restocking is rarely necessary."
+        ),
+        num_products=8,
+        max_steps=28,
+        num_customers=12,
+        expiry_offset_min=12,
+        expiry_offset_max=22,
+        seed=200,
+        profit_target=200.0,
+    ),
+    "hard": TaskConfig(
+        name="hard",
+        description=(
+            "10 products, 30 steps, 10 customers/step. "
+            "Most products expire in 8-18 steps — expiry management is critical. "
+            "Zone strategy (rotate zone 1 for near-expiry items) combined with "
+            "discounts drives clearance. Restock fast-sellers that run out early "
+            "to maintain revenue through the full episode."
+        ),
+        num_products=10,
+        max_steps=30,
+        num_customers=10,
+        expiry_offset_min=8,
+        expiry_offset_max=18,
+        seed=456,
+        profit_target=240.0,
+    ),
+    "expert": TaskConfig(
+        name="expert",
+        description=(
+            "12 products, 25 steps, 8 customers/step. "
+            "Aggressive expiry (4-10 steps) with limited initial stock — "
+            "all three actions must be coordinated: discount near-expiry items to "
+            "clear them fast, restock high-margin products before they stock out, "
+            "and continuously rotate zone 1 to prioritise the most urgent products."
+        ),
+        num_products=12,
+        max_steps=25,
+        num_customers=8,
+        expiry_offset_min=4,
+        expiry_offset_max=10,
+        seed=789,
+        profit_target=190.0,
+    ),
+}
+
+
+# ── Grader ─────────────────────────────────────────────────────────────────────
+
+def grade(task_name: str, cumulative_profit: float) -> float:
+    """
+    Return a deterministic score in [0.0, 1.0] for a completed episode.
+
+    Formula:
+        score = clamp(cumulative_profit / profit_target, 0.0, 1.0)
+
+    Args:
+        task_name: One of "easy", "medium", "hard", "expert" (or a custom string).
+        cumulative_profit: Total profit accumulated over the episode.
+
+    Returns:
+        Score between 0.0 (no profit) and 1.0 (hit or exceeded target).
+    """
+    config = TASKS.get(task_name)
+    if config is None or config.profit_target <= 0:
+        return 1.0
+    return min(1.0, max(0.0, cumulative_profit / config.profit_target))
+
+
+__all__ = ["TaskConfig", "TASKS", "grade"]