---
title: Smart Study Session Optimizer
emoji: 📚
colorFrom: blue
colorTo: green
sdk: docker
pinned: false
---
# Smart Study Session Optimizer

## Why I Built This

Okay so honestly this project started from a very relatable place — I used to study for 4-5 hours straight
during exam prep, and by the third hour I was basically just staring at my notes without retaining anything.
I knew I was supposed to take breaks but it felt like "wasting time". Turns out I was doing it completely wrong.

I came across some research on the Pomodoro Technique and ultradian rhythms, and I got curious: could you
actually *learn* the optimal study break pattern using reinforcement learning? The environment models a simplified
version of mental energy and focus, and lets an RL agent figure out when to grind and when to rest.

It's not a perfect simulation of human cognition (not even close), but it's a fun approximation and it
passed all the hackathon criteria, so I'm calling it a win.

---

## Project Structure

```
study-optimizer/
├── src/
│   ├── __init__.py
│   ├── models.py        # Pydantic models (Observation, Action, StepResult)
│   ├── environment.py   # Core RL environment logic
│   └── client.py        # HTTP client for the environment API
├── server/
│   └── app.py           # FastAPI server
├── inference.py         # Main script: LLM agent runs an episode
├── Dockerfile
├── requirements.txt
└── openenv.yaml         # OpenEnv spec compliance
```

---

## How It Works

The environment simulates a study session. At each step, the agent sees:
- **Energy** (0–100): how much mental stamina is left
- **Focus** (0–100): current concentration level  
- **Time studied**: cumulative minutes of studying
- **Session elapsed**: total minutes including breaks
- **Consecutive study minutes**: streak without a break (focus drops fast here)
- **Break count**: how many breaks taken
- **Productive minutes**: minutes where focus was above 60

The agent picks one of 3 actions:
- `0` → **Study** (5 min, drains energy & focus)
- `1` → **Short break** (10 min, moderate recovery)
- `2` → **Long break** (20 min, full recovery but costs time)

---

## The 3 Tasks

| Task ID | Goal | Score Formula |
|---------|------|---------------|
| `survive_8_hours` | Keep energy > 0 for 480 min | `clamp(energy_remaining / 100, 0.01, 0.99)` |
| `maximize_productivity` | Max productive minutes in 240 min | `clamp(productive_minutes / 240, 0.01, 0.99)` |
| `smart_break_taker` | Only break when energy < 40% | `clamp(good_breaks / total_breaks, 0.01, 0.99)` |

---

## Quick Start

### 1. Install dependencies

```bash
pip install -r requirements.txt
```

### 2. Run the server

```bash
uvicorn server.app:app --host 0.0.0.0 --port 7860 --reload
```

### 3. Run inference (LLM agent plays the game)

```bash
# Set env vars first
export API_BASE_URL="https://api.openai.com/v1"
export MODEL_NAME="gpt-4o-mini"
export HF_TOKEN="your_hf_token"

python inference.py --task maximize_productivity
```

### 4. Try it manually with curl

```bash
# Start an episode
curl -X POST http://localhost:7860/reset \
  -H "Content-Type: application/json" \
  -d '{"task_id": "maximize_productivity"}'

# Study for 5 minutes
curl -X POST http://localhost:7860/step \
  -H "Content-Type: application/json" \
  -d '{"action": 0}'

# Health check
curl http://localhost:7860/health
```

---

## Docker

```bash
# Build
docker build -t study-optimizer .

# Run
docker run -p 7860:7860 study-optimizer

# With env vars for inference
docker run -p 7860:7860 \
  -e API_BASE_URL="https://api.openai.com/v1" \
  -e MODEL_NAME="gpt-4o-mini" \
  -e HF_TOKEN="your_token" \
  study-optimizer
```

---

## Environment Variables

| Variable | Description | Required |
|----------|-------------|----------|
| `API_BASE_URL` | Base URL for OpenAI-compatible LLM API | Yes (for inference) |
| `MODEL_NAME` | Model identifier (e.g. `gpt-4o-mini`) | Yes (for inference) |
| `HF_TOKEN` | HuggingFace API token | Yes (for HF Spaces) |
| `PORT` | Server port (default: 7860) | No |

---

## OpenEnv Compliance

This environment follows the OpenEnv spec:

- `POST /reset` → returns initial observation
- `POST /step` → takes action, returns `{observation, reward, done, info}`
- `GET /health` → returns `{"status": "ok"}`
- `GET /state` → returns full env state
- `openenv.yaml` → spec definition file

---

## Reward Design

Reward at each step:
- Studying with high energy + focus: **+1.0**
- Studying with just energy (focus low): **+0.3**
- Studying while exhausted: **-0.5**
- Smart break (energy < 40): **+0.5 to +1.0**  
- Unnecessary break: **-0.3 to -0.6**
- Terminal reward at end varies by task

The reward shaping is intentionally conservative — it's meant to guide the agent
without being so prescriptive that it removes the need to learn anything interesting.

---

## Known Limitations / TODOs

- The energy drain model is simplified (doesn't account for circadian rhythms, caffeine, etc.)
- No persistence — session state resets on server restart
- Haven't tried training a proper PPO agent yet, only tested with LLM inference
- Should add a WebSocket endpoint for real-time streaming

---

*Built for the Meta PyTorch OpenEnv Hackathon by Scaler School of Technology*