Upload folder using huggingface_hub
Browse files- .gitattributes +1 -0
- MD5SUMS +3 -0
- README.md +65 -1
- datasets.json +22 -0
- figures/before_after_S4_S1.png +3 -0
- reader_v3_depth_head.safetensors +3 -0
- reader_v3_depth_head_last.pt +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
figures/before_after_S4_S1.png filter=lfs diff=lfs merge=lfs -text
|
MD5SUMS
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
6a52f35d09b732cb73a7cf6cf52ada4b reader_v3_depth_head_last.pt
|
| 2 |
+
57098479d8b0bda9dca21d76f8d37fef reader_v3_depth_head.safetensors
|
| 3 |
+
3e3e4c6a93757ba9bf2d47a1808227f1 datasets.json
|
README.md
CHANGED
|
@@ -1,3 +1,67 @@
|
|
| 1 |
---
|
| 2 |
-
license:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
license: other
|
| 3 |
+
license_name: same-as-upstream-reader-v3
|
| 4 |
+
library_name: pytorch
|
| 5 |
+
pipeline_tag: image-segmentation
|
| 6 |
+
tags:
|
| 7 |
+
- vesuvius-challenge
|
| 8 |
+
- ink-detection
|
| 9 |
+
- herculaneum
|
| 10 |
+
- reader-v3
|
| 11 |
+
- depth-head
|
| 12 |
---
|
| 13 |
+
|
| 14 |
+
# 3D Reader V3: reader-v3 with a sheet-localised depth head
|
| 15 |
+
|
| 16 |
+
**In one sentence:** the stock reader-v3 ink model plus a 16-plane depth head that says on which plane (sheet) each stroke sits, fine-tuned without any 3-D human labels, at about the same cost as reader-v3 (+1.3 % time, measured).
|
| 17 |
+
|
| 18 |
+
This is an experimental fine-tune of the public reader-v3 checkpoint (34.55 M parameters, 3-D depth-fusion stem + 2-D U-Net, 17 input planes at ~9.6 um). The extra head is a 1x1 convolution (528 parameters) on the finest decoder stage that outputs 16 logits per pixel, one per plane of the central 16 input planes. Code, tests and the full write-up: https://github.com/StewartSethA/villa/tree/progress-2026-11/sheet-localized-3d-ink-head
|
| 19 |
+
|
| 20 |
+

|
| 21 |
+
*Columns: CT middle plane | refined label | stock reader-v3 | this model | this model's depth hue (blue = top plane, red = mesh plane, green = bottom plane) | Hecate depth hue. Rows: Scroll 4 20251212185248 and Scroll 1 20230929220926 windows in the held-out right 10 % band of segments the model was fine-tuned on (so the sharper map partly reflects fine-tuning to the refined labels; the transfer evidence is the table below).*
|
| 22 |
+
|
| 23 |
+
## Files
|
| 24 |
+
| file | content |
|
| 25 |
+
|---|---|
|
| 26 |
+
| `reader_v3_depth_head.safetensors` | net + head (net keys keep the `model.` prefix; alias keys `.all_modules.` are omitted because they are the same tensors as `.conv.` / `.norm.`; head keys start with `head.`). Loading it into a freshly built network reproduces the original checkpoint's logits exactly (max abs difference 0.0). |
|
| 27 |
+
| `reader_v3_depth_head_last.pt` | the same weights as a plain tensor `.pt` (`state_dict`, `head`, `step`; loads with `weights_only=True`) |
|
| 28 |
+
| `datasets.json` | the data record of the run (see below) |
|
| 29 |
+
| `MD5SUMS` | checksums |
|
| 30 |
+
|
| 31 |
+
## Training data (all of it)
|
| 32 |
+
- **Initialisation:** the stock reader-v3 checkpoint.
|
| 33 |
+
- **Additional data:** 13 open-data segments, 8 of Scroll 1 (PHercParis4: 20230702185753, 20230929220926, 20231007101619, 20231012184424, 20231016151002, 20231031143852, 20231106155351, 20231210121321) and 5 of Scroll 4 (PHerc1667: 20240304144031, 20240304161941, 20251208130119, 20251212185248, 20251223230000), as 17-plane 9.6 um surface renders.
|
| 34 |
+
Labels: "refined core" = human label AND the registered upstream 2.4 um prediction >= 128; "maybe" pixels are never trained on; the right 10 % of every segment is held out.
|
| 35 |
+
Depth supervision: Hecate 9.6 um 3-D probability volumes (https://huggingface.co/scrollprize/hecate) on those segments; the 2-D label is snapped onto the ink-bearing sheet (the teacher's depth profile restricted to +-1 plane around its peak). Hecate's own training segment list is not published, so overlap of Hecate with the test segments cannot be ruled out.
|
| 36 |
+
- **Store mix:** every batch also contains ~9 windows of the reader-v3 training store (the data the stock model was trained on), x in [0, 0.88) of each representation. Without it the net specialises to Scroll 1/4 and loses transfer (0.75-0.79 upstream-test AUC in our first attempts).
|
| 37 |
+
- **Recipe:** SGD momentum 0.99 nesterov, lr 2e-4, 200-step warm-up, per-window robust-MAD normalisation, flips + rot90 on the GPU, masked BCE (2-D) + masked BCE (16 depth planes), batch 6 refined windows + ~9 store windows at 320 px, seed 0, stopped by a held-out plateau rule (step 7500).
|
| 38 |
+
- No data from any other scroll was used.
|
| 39 |
+
|
| 40 |
+
## Results (upstream test: 8 segments, human-label AUC, equal weights, same harness for every row, one seed each)
|
| 41 |
+
| model | P384 | P448 |
|
| 42 |
+
|---|---|---|
|
| 43 |
+
| stock reader-v3 (positive control; README reference 0.892) | 0.8893 | 0.892 |
|
| 44 |
+
| single-task control (same recipe, depth head not trained) | 0.8911 | 0.8941 |
|
| 45 |
+
| **this model (film +-1)** | **0.8968** | **0.8994** |
|
| 46 |
+
| film +-1, other seeds / 13-segment run | 0.8947 / 0.8968 | 0.8957 / 0.8991 |
|
| 47 |
+
| film +-3 | 0.8933 | 0.8953 |
|
| 48 |
+
| any sheet (no film restriction) | 0.8841 | 0.8871 |
|
| 49 |
+
|
| 50 |
+
The gain over stock (+0.0054 to +0.0075 at P384) is small and single-seed; replicate seeds are still training. The 3 of the 8 test segments that also appear in the reader-v3 training list (see the leakage-audit branch) are in-sample for stock and fine-tuned models alike.
|
| 51 |
+
|
| 52 |
+
## Use
|
| 53 |
+
```python
|
| 54 |
+
import torch
|
| 55 |
+
ck = torch.load("reader_v3_depth_head_last.pt", weights_only=True) # keys: state_dict, head, step
|
| 56 |
+
net = build_reader_v3() # the reader-v3 network of villa's vesuvius.ink_detection
|
| 57 |
+
net.load_state_dict(ck["state_dict"]) # strip the "model." prefix if your build uses bare keys
|
| 58 |
+
head = torch.nn.Conv2d(32, 16, 1); head.load_state_dict({k.removeprefix("head."): v for k, v in ck["head"].items()})
|
| 59 |
+
# hook the finest decoder stage (net.model.network.shared_decoder.stages[-1]) and apply `head` to its output:
|
| 60 |
+
# sigmoid(logits) per plane = probability that the ink sits on that plane (planes 0-15 of the 17-plane window).
|
| 61 |
+
# Normalise each window with the reader-v3 robust-MAD recipe; plane order must be the one the checkpoint expects
|
| 62 |
+
# (see the render-direction-and-intensity-check branch).
|
| 63 |
+
```
|
| 64 |
+
Reference implementation: `attach_depth_head` in the GitHub branch above.
|
| 65 |
+
|
| 66 |
+
## Limitations
|
| 67 |
+
Single training seed per arm; no confidence intervals; the upstream-test gain is within what seed noise could produce until replicates finish; not run through the `vesuvius` package loaders (adapter point documented in the branch); depth probabilities are soft targets, not calibrated.
|
datasets.json
ADDED
|
@@ -0,0 +1,22 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"train_segments": [
|
| 3 |
+
"PHerc1667__20240304144031",
|
| 4 |
+
"PHerc1667__20240304161941",
|
| 5 |
+
"PHerc1667__20251208130119",
|
| 6 |
+
"PHerc1667__20251212185248",
|
| 7 |
+
"PHerc1667__20251223230000",
|
| 8 |
+
"PHercParis4__20230702185753",
|
| 9 |
+
"PHercParis4__20230929220926",
|
| 10 |
+
"PHercParis4__20231007101619",
|
| 11 |
+
"PHercParis4__20231012184424",
|
| 12 |
+
"PHercParis4__20231016151002",
|
| 13 |
+
"PHercParis4__20231031143852",
|
| 14 |
+
"PHercParis4__20231106155351",
|
| 15 |
+
"PHercParis4__20231210121321"
|
| 16 |
+
],
|
| 17 |
+
"held_out": "right 10 % x-band of the same segments (NOT a transfer set)",
|
| 18 |
+
"store": "V3 unified store, per-scroll counts {'0139': 2, '1667': 1, 'Paris4': 1, '0814': 1, '0009B': 1, '0343P': 1, '0500P2': 1, '0172': 1} (9 windows per batch), x in [0, 0.88)",
|
| 19 |
+
"label_source": "refined core = human label AND model >= 128 (registered upstream 2.4 um predictions), maybe pixels held out; Hecate 9.6 um teacher volumes",
|
| 20 |
+
"init": "stock reader-v3 checkpoint",
|
| 21 |
+
"reconstructed_from": "train.log after the run (the trainer did not write datasets.json before 2026-11-01)"
|
| 22 |
+
}
|
figures/before_after_S4_S1.png
ADDED
|
Git LFS Details
|
reader_v3_depth_head.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7e30e29b2512ac2d1e84ed88e6b097008bf378e4446a4956684f1a55a673e8d0
|
| 3 |
+
size 138222952
|
reader_v3_depth_head_last.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c3b969fbb09fa3a4c66a772b0c9845324e9de51d05032f0e5f324fb0d6288277
|
| 3 |
+
size 138364314
|