Blame for docs/LABELLING.md

# Labelling frames for the Birdie accuracy test

Thank you for helping! You will look at frames from short clips of tables and say, for each seat, whether
someone is sitting there, whether only their things are there, or whether it is empty. Birdie's detector
is then scored against your answers. Budget about **20 minutes per clip**.

**Privacy:** the tool never saves the original video frames. It saves copies where people are pixelated.
Don't share the source videos (the `eval_clips` folder) with anyone.

## 1. One-time setup (10 minutes)

**Install uv** (it installs Python and everything else for you):

- **Mac:** open the Terminal app and paste:
  ```
  curl -LsSf https://astral.sh/uv/install.sh | sh
  ```
  Then close the Terminal window and open a new one.
- **Windows:** open PowerShell (Start menu → type "PowerShell") and paste:
  ```
  powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
  ```
  Then close PowerShell and open a new one.

**Get the code.** If you have git:
```
git clone https://github.com/Dzyreee/birdie.git
cd birdie
```
No git? Open https://github.com/Dzyreee/birdie, click **Code → Download ZIP**, unzip it, then in the
Terminal/PowerShell type `cd ` (with a space), drag the unzipped `birdie` folder into the window, and press Enter.

**Add the clips.** Copy the phone videos into the `eval_clips` folder inside `birdie` (create it if it isn't there).
Any of .mp4, .mov, .m4v, .avi, .mkv or .webm works. If a video doesn't show up or won't open, re-export it as
H.264 ("Most Compatible" on iPhone).

## 2. Start the labelling tool

From the `birdie` folder:
```
uv run tools/label_app.py
```
The first start downloads about 1 GB (Python, PyTorch, the detector) and can take a few minutes. Later starts take
seconds. Your browser opens **http://127.0.0.1:7861** (open it yourself if it doesn't). Leave the Terminal window
open while you work; press `Ctrl+C` in it to stop the tool.

Type your name at the top, then pick a clip in the **Clip** list (press **↻ Refresh clips** if you just added one).

## 3. Step 1 · Draw the seats (once per clip)

1. Click **🖼 Show first frame**. The first frame appears with people pixelated.
2. For each seat you want to label, click its **4 corners**. Draw the area of **the chair plus the table area directly
   in front of it** (where that person's laptop, cup or bag would be). Don't overlap neighbouring seats.
3. Click **💾 Save seat**. The seat gets a number on the picture and the next id (S01, S02, ...) fills in by itself.
   Made a mistake? **↺ Redo corners** before saving, or pick the seat in **Seat to delete** after saving.
4. When every seat is drawn, click **▶ Extract frames** (keep 2 seconds between frames unless told otherwise).
   Wait for "✅ ... frames ready".

Only draw seats that stay visible in most of the clip. Drawing them is quick, but don't change them after you start
labelling.

## 4. Step 2 · Label the frames

Open the **Step 2 · Label frames** tab. You see one frame with numbered seats, and one row of buttons per seat.
For every seat pick one answer, then **Save & next ▶**. Your work is saved after every frame, so you can stop and
come back any time (**⏭ First unlabelled frame** takes you back to where you were).

Labels from the previous frame are pre-filled to save time (the line above the picture says "pre-filled"). **Check
every seat before saving**: change anything that is different in this frame.

### Keyboard shortcuts

| Key | Does |
|---|---|
| `1` or `P` | Person |
| `2` or `I` | Items only |
| `3` or `E` | Empty |
| `4` or `U` | Unclear |
| `↓` / `↑` | Move to the next / previous seat (the highlighted one is set by the keys above) |
| `Enter` or `→` | Save & next |
| `←` | Back |

After a label key, the highlight moves to the next seat on its own, so a frame is usually: `P` `E` `I` `Enter`.

## 5. The rules

Label **only what you see in this frame**, even if you know what happened before or after. (A person who has just
walked away and left their bag is "Items only" now. Birdie works out the timing itself.)

| Answer | When |
|---|---|
| **Person** | Someone is **sitting** on the seat, or clearly **leaning on it or kneeling on it**, even if partly hidden. A pixelated blob in the seat counts. |
| **Items only** | No person, but **belongings** are on the seat or on the **table area in front of it**: bag, backpack, laptop, phone, cup, bottle, book, jacket on the chair, and so on. |
| **Empty** | No person and no belongings. Things that belong to the venue (menus, napkins, sugar, table numbers, chairs pushed in) don't count. |
| **Unclear** | You honestly can't tell: the seat is blocked by someone standing in front, too dark, blurred by motion, or out of the picture. |

Tricky cases:

- **A person standing next to the seat, or walking past it, is Empty** (or Items only if their things are on it),
  unless they are sitting or leaning on it.
- **A bag on the table in front of a seat counts as Items only** for that seat, even if the chair is pushed in.
- **Something between two seats** belongs to the seat it is closer to. If it is exactly in the middle, choose the
  seat whose drawn area covers more of it.
- **Someone sitting with their things** is Person (Person always wins over Items only).
- **A jacket hanging on the chair** is Items only.
- **Two people sharing one seat area** is Person.
- When in doubt between two answers, pick **Unclear**. Unclear answers are left out of the score, so they never hurt it.

### If you can see a person who is NOT pixelated

Click **🗑 Remove frame (a person is not pixelated)**. This deletes the picture from the disk and skips the frame.
Don't take screenshots of frames.

## 6. When you are done

Tell Yacine, and send him the **`eval_data`** folder (zip it). **Don't send `eval_clips`**: those are the original
videos and stay on your computer. The `eval_data` folder only has pixelated frames, numbers and your answers.

## For Yacine: from labels to the score

```bash
# 1. Merge the eval_data folders you received into eval_data/ (one sub-folder per clip), then check locally:
uv run jobs/evaluate.py                                  # writes eval_data/results/results.{json,md}

# 2. Upload the labelled data to a PRIVATE dataset (pixelated frames, detections, seats, labels only):
scripts/push_eval_dataset.sh <owner>/birdie-eval

# 3. Run the evaluation as an HF Job on cpu-basic (20 min timeout); results land in the dataset's results/:
scripts/run_eval_job.sh <owner>/birdie-eval
```

The job only scores stored detections (the detector already ran on the original frames, in memory, during
extraction), so it needs no GPU and finishes in seconds.