Rewrite the model card in plainer language

README.md

@@ -11,122 +11,135 @@ tags:
 
 # clockface
 
-Reads an analog clock face and returns the time. Two stages: find the dial,
-then read it. 11.5 MB of weights in total.
+Reads an analog clock and returns the time. Finds the dial first, then reads it.
+11.5 MB of weights.
 
-> **This model does not work well on real photographs.** On 1,728 real photos it
-> scores **155.9 minutes** mean error, against **180 minutes for guessing at
-> random**. It scores 4.8 minutes on the synthetic data it was trained on. It is
-> published because the gap between those two numbers is the useful part.
+> **It does not work on real photographs.** On 1,728 real photos it is off by
+> 155.9 minutes on average. Guessing at random is off by 180. On the synthetic
+> renders it trained on, it is off by 4.8 minutes. I am publishing it because
+> that gap is the only interesting thing here.
 
-Version 263701. Trained on synth v2, revision `main` of the dataset above.
+Version 263701, trained on synth v2.
 
 ## The metric
 
-A clock face is a ring of 720 minutes. Error is the shorter way round it, so it
-lies in [0, 360] and 11:58 against 12:02 is 4 minutes, not 716. AM/PM does not
-exist: a face cannot show it, so 3:47 and 15:47 are the same reading.
+A clock face is a ring of 720 minutes, and error is the shorter way round it. So
+11:58 against 12:02 is 4 minutes, not 716. AM and PM do not exist, because a face
+cannot show them: 3:47 and 15:47 are the same reading.
 
-Guessing uniformly at random scores 180 minutes. That is the number to beat, and
-it is the number this model is measured against.
+Random guessing scores 180 minutes. That is the bar.
 
-## Results
+## What it scores on real photographs
 
-### Real photographs (1,728 images, third-party labels)
+1,728 images with third-party labels.
 
 | | MAE | median | within 5 min | worse than 30 min |
 |---|---|---|---|---|
-| two-stage | 155.9 min | 139.9 | 11.4% | 82.2% |
+| two stages | 155.9 min | 139.9 | 11.4% | 82.2% |
 | whole image, no locator | 151.8 min | 133.7 | 9.2% | 82.5% |
-| **random guessing** | **180 min** | — | — | — |
+| random guessing | 180 min | | | |
 
-Sources: 1,092 film stills from Marclay's *The Clock*, 506 photographs from
-`moondream/analog-clock-benchmark`, 131 cropped faces from
-`oliverj990/clock-faces-v1-times`. Their labels are third party and mostly
-unverified; restricted to the 106 labels checked by hand against the image, the
-result is MAE 148.5, so label noise does not explain it.
+Dropping the locator makes it slightly better, which tells you how much the
+locator is contributing.
 
-By source, the pattern is a domain gap and not much else:
+The images are 1,092 film stills from Marclay's *The Clock*, 506 photos from
+`moondream/analog-clock-benchmark`, and 131 cropped faces from
+`oliverj990/clock-faces-v1-times`. Most of those labels I have not checked. On
+the 106 I did check by hand against the image, the error is 148.5 minutes, so bad
+labels are not the excuse.
+
+Split by source, the story is obvious:
 
 | source | MAE | within 5 min |
 |---|---|---|
-| cropped clock faces (dial fills frame) | 87.8 min | 45.0% |
-| photographs of scenes | 163.3 min | 9.3% |
+| cropped faces, dial fills the frame | 87.8 min | 45.0% |
+| photographs of rooms and streets | 163.3 min | 9.3% |
+
+The closer a photo looks to my renders, the better it does. That is a domain gap
+and nothing subtler.
 
-The closer an image is to the synthetic framing, the better it does.
+## What it scores on synthetic data
 
-### Synthetic validation (1,200 held-out renders) — training telemetry only
+1,200 held-out renders.
 
 | | MAE | median | within 1 min | within 5 min |
 |---|---|---|---|---|
 | ground-truth crops | 4.82 min | 0.42 | 79.0% | 92.5% |
 | end to end | 5.07 min | 0.43 | 79.0% | 92.2% |
 
-**These numbers are not a result.** The model is being scored by the same
-renderer that produced its labels. They are here to show the size of the
-sim-to-real gap, which is the point: 4.8 minutes on renders, 156 on photographs.
+Do not quote these. The renderer that made the labels also made the pixels, so
+the model is being marked by its own teacher. They are here for one reason: 4.8
+minutes on renders against 156 on photographs is the whole story.
+
+## Where it goes wrong
 
-## What goes wrong
+Of the 1,421 predictions off by more than half an hour, 720 would improve if you
+swapped the hour and minute hands. My generator drew hands from a narrow band of
+lengths and widths, and the model learned that band instead of learning which
+hand is longer. Real clocks do not agree with my band.
 
-- **Hand confusion.** 720 of the 1,421 gross failures would improve if the hour
-  and minute hands were swapped. The generator used a narrow band of hand
-  lengths and widths, and the model learned that convention rather than learning
-  which hand is longer.
-- **The confidence signal does not survive.** The two hands' disagreement is a
-  good confidence signal on synthetic data (MAE 4.82 falls to 1.64 when keeping
-  the most confident 75%) and is inert on real photographs (155.9 to 153.5). The
-  model is confidently wrong, which is worse than being uncertain.
-- **Small clocks.** The locator finds a dial to 1.5% of image width on renders
-  and fails on wide scenes where the clock is a small part of the frame.
+The confidence signal does not survive the trip to real data. On renders it works
+well: keep the most confident 75% and error drops from 4.82 to 1.64 minutes. On
+photographs it does nothing, 155.9 to 153.5. The model is confidently wrong,
+which is worse than being uncertain.
 
-## Architecture
+The locator finds a dial to within 1.5% of image width on renders. On a photo of
+a room where the clock is a smudge on a far wall, it does not.
 
-    stage 1  DialLocator      554K params, 2.2 MB   image -> dial centre, radius
-    stage 2  PretrainedReader 2.3M params, 9.2 MB   crop -> two angle distributions
+## How it is built
 
-Stage 2 is a MobileNetV3-Small backbone pretrained on ImageNet, with two heads
-predicting a distribution over 180 angular bins for each hand. Targets are von
-Mises bumps rather than one-hot, and decoding takes a circular soft-argmax.
+    stage 1   DialLocator        554K params, 2.2 MB    image -> centre, radius
+    stage 2   PretrainedReader   2.3M params, 9.2 MB    crop -> two angle distributions
 
-An identical from-scratch network never leaves chance on this task: it sits at
-2*ln(180), a uniform distribution, however long it trains, while overfitting 400
-images perfectly. ImageNet initialisation is what makes it learn at all.
+Stage 2 is MobileNetV3-Small pretrained on ImageNet, with two heads over 180
+angular bins each. Targets are von Mises bumps rather than one-hot, so being one
+bin out costs less than being ten out, and decoding uses a circular soft-argmax
+to get back below bin resolution.
 
-Both hands are predicted, never the time directly, so their disagreement gives a
-confidence signal for free. Decoding is a vernier: the minute hand supplies
-precision, the hour hand picks which hour. Measured against reading the hour hand
-alone, that is about 11x more accurate under noise.
+The same network without ImageNet weights never learns this at all. It sits at
+2·ln(180), a flat distribution, for as long as you care to train it, while
+happily memorising 400 images. The pretrained initialisation is the difference
+between a model and a plateau.
 
-## Usage
+It predicts both hands and never the time directly. That costs nothing and buys
+two things. The hands disagreeing is a confidence signal. And decoding can work
+like a vernier scale, taking precision from the minute hand and only the hour
+count from the hour hand, which under noise is about 11 times more accurate than
+reading the hour hand alone.
+
+## Using it
 
     import torch
-    from twostage import DialLocator, PretrainedReader, crop_dial
+    from twostage import PretrainedReader
     from model import decode_cls
 
     reader = PretrainedReader(bins=180, arch="mobilenet_v3_small")
     reader.load_state_dict(torch.load("stage2_best.pt")["model"])
-    hour_logits, minute_logits = reader(crop)          # crop: 1x3x256x256, ImageNet norm
+    hour_logits, minute_logits = reader(crop)      # 1x3x256x256, ImageNet normalised
     minutes, hour_only, disagreement, sharpness = decode_cls(hour_logits, minute_logits)
 
-`minutes` is the position on the 720-minute ring; `disagreement` is how far apart
+`minutes` is the position on the 720-minute ring. `disagreement` is how far apart
 the two hands' readings are, in minutes.
 
-## Intended use
+## What it is for
 
-Reading round analog clock faces. Out of scope: digital-analog hybrids, subdials,
-chronographs, 24-hour faces.
+Round analog clock faces. Not digital-analog hybrids, subdials, chronographs or
+24-hour faces.
 
-**Not suitable for any use where a wrong time carries a cost.** It is wrong by
-more than half an hour on 82% of real photographs.
+Do not use it for anything where a wrong time costs something. It is off by more
+than half an hour on 82% of real photographs.
 
 ## Reproducing
 
     git clone https://github.com/lyte-codes/clockface
-    ./eval.py --self-test                              # 32/32, checks the metric
+    ./eval.py --self-test
     python3 predict.py --labels <labels> --root <dir> \
         --stage2 stage2_best.pt --stage1 stage1_best.pt --out preds.jsonl
     ./eval.py --labels <labels> --pred preds.jsonl
 
+`--self-test` runs 32 checks on the metric itself, including both directions
+across the 12 o'clock seam.
+
 ## Licence
 
 Apache 2.0. The synthetic dataset is CC BY 4.0.