---
license: other
license_name: sam-license
license_link: https://huggingface.co/Sonam5/FarmSAM/blob/main/LICENSE_SAM.txt
library_name: pytorch
pipeline_tag: mask-generation
tags:
- knowledge-distillation
- feature-distillation
- sam3
- video-object-tracking
- multi-object-tracking
- tinyvit
- livestock
- pigs
- dairy-calves
- rf-detr
- object-detection
- arxiv:2604.27128
datasets:
- Sonam5/Calf-Play-Behavior-Dataset
base_model:
- timm/tiny_vit_21m_512.dist_in22k_ft_in1k
---
# FarmSAM student checkpoints
Compact TinyViT students for SAM 3's video tracker. A student replaces SAM 3's image backbone and learns from unlabelled frames of one farm, while the rest of the tracker stays the same. On dairy calves it keeps identity correct 99.23 % of the time (SAM 3: 99.80 %), with a backbone that is 91 % smaller and 3.6× faster.
You also need SAM 3's own weights from Meta, under the gate of [`facebook/sam3`](https://huggingface.co/facebook/sam3). Code and full results: [github.com/Sonam525/FarmSAM](https://github.com/Sonam525/FarmSAM). Preprint: [arXiv 2604.27128](https://arxiv.org/abs/2604.27128).
## Checkpoints
Each student: TinyViT-21M-512 trunk plus feature-pyramid head, 40.66 M backbone parameters, 60.20 M in total with the SAM 3 tracker. Input 1,024 × 1,024. Output map `[1, 1024, 72, 72]`.
| Student | File | Distilled on | Frames |
|:---|:---|:---|---:|
| Calf student (night-mined) | `calf/calf_student_night_mined/best_backbone_student_v2.pt` | Dairy calves, day and night | 10,514 |
| PigTrack student | `pigtrack/pigtrack_student/best_backbone_student_pigtrack.pt` | PigTrack | 3,000 |
| Edinburgh student | `edinburgh_noncommercial/edinburgh_student/best_backbone_student.pt` | Edinburgh | 3,000 |
There is one student per dataset. Pick the one closest to your setting, and distil your own if your pens look different. Hashes are in `CHECKSUMS.sha256`.
## Detectors
The RF-DETR detectors behind the detector-started results. Each folder holds `checkpoint_best_total.pth` and a `detector.json` with its resolution, threshold, training data and test AP.
| Detector | Folder | Trained on | Test AP50 / AP50:95 |
|:---|:---|:---|:---|
| Pig, RF-DETR Small, 512 | `detectors/pigtrack_rfdetr/` | PigTrack | 0.979 / 0.793 on PigTrack |
| Pig, RF-DETR Small, 512 | `edinburgh_noncommercial/edinburgh_rfdetr/` | Edinburgh, starting from the PigTrack detector | 0.990 / 0.628 on Edinburgh |
| Calf, RF-DETR-Seg Small, 624 | `detectors/calf_rfdetr_seg/` | FarmA_R1, 18 and 19 April | see the calf identity project |
```python
from huggingface_hub import hf_hub_download
from rfdetr import from_checkpoint # pip install rfdetr==1.10.1
path = hf_hub_download("Sonam5/FarmSAM", "detectors/pigtrack_rfdetr/checkpoint_best_total.pth")
detector = from_checkpoint(path, resolution=512) # pass the resolution: the file does not store it
```
The results used a threshold of 0.70 for both pig detectors. The calf detector runs at 0.35, with the post-processing given in its `detector.json`. The calf detector and the calf student are also in [Sonam5/Calf-Identity-Tracking-Weights](https://huggingface.co/Sonam5/Calf-Identity-Tracking-Weights), next to the calf identity work.
## How to load
```bash
git clone https://github.com/Sonam525/FarmSAM && cd FarmSAM
pip install -r requirements.txt
hf auth login # with facebook/sam3 terms accepted
```
```python
import sys, torch
from huggingface_hub import hf_hub_download
from PIL import Image
sys.path.insert(0, "benchmark/core")
import bench_core as core
path = hf_hub_download("Sonam5/FarmSAM", "calf/calf_student_night_mined/best_backbone_student_v2.pt")
student, meta = core.load_student(path, device="cuda") # prefers the EMA weights, casts to bf16
x = core.make_student_transform()(Image.open("frame.jpg").convert("RGB"))
with torch.no_grad():
features = student(x[None].to("cuda", torch.bfloat16)) # [1, 1024, 72, 72]
```
`core.load_arm` swaps the student into `Sam3TrackerVideoModel`. The tracker's neck, mask decoder, memory attention and memory encoder stay unchanged.
`.pt` is pickle, so check the SHA-256 before you load a file. Use the EMA weights; `load_student` picks them for you.
## Results
Each row uses a student distilled on that dataset. Tracks start from reference boxes or masks at the first frame.
| Dataset, metric | Student | SAM 3 | SAM 2.1-small | SAM 2.1-tiny | EdgeTAM |
|:---|---:|---:|---:|---:|---:|
| Dairy calves, identity-correct %, 6 windows | **99.23** | 99.80 | 98.77 | 98.96 | 91.29 |
| Edinburgh pigs, HOTA, 4 clips | **74.05** | 75.54 | 75.18 | 74.52 | 71.21 |
| PigTrack, HOTA, 25 sequences | **73.25** | 85.87 | 80.90 | 79.38 | 71.16 |
| Model | Backbone parameters | Total parameters | Backbone, ms per frame | End to end, fps |
|:---|---:|---:|---:|---:|
| Student | 40.66 M | 60.20 M | 41.56–42.61 | 4.83–5.01 |
| SAM 3 | 446.24 M | 465.78 M | 152.46 | 3.25 |
| SAM 2.1-small | 33.95 M | 46.06 M | 15.04 | 7.83 |
| SAM 2.1-tiny | 26.85 M | 38.96 M | 12.78 | 8.00 |
| EdgeTAM | 4.71 M | 13.90 M | 10.03 | 13.05 |
One A10 GPU, 4 objects, frame decoding included. Calf identity is scored against expert-verified identity chains on masks from an earlier run of the calf student, so it measures how well a track stays on its animal rather than mask quality. The speed gain shrinks as more objects are tracked. Full tables: [RESULTS.md](https://github.com/Sonam525/FarmSAM/blob/main/docs/RESULTS.md).
## Intended use
Research on feature distillation and on tracking animals from fixed farm cameras. Evaluate a student on your own pen before you rely on it. Tracks need initial boxes, masks or a detector.
Not intended for decisions about individual animals without review, for tracking people, or for any use the SAM License excludes. The Edinburgh student is for non-commercial use only.
## Terms
The code on GitHub is MIT, except parts under other terms. **These weights are not.** All students and the calf detector are released under the SAM License. The two pig detectors are released under Apache-2.0, the licence of RF-DETR. Everything in `edinburgh_noncommercial/` carries a non-commercial limit on top.
- The students were trained to match SAM 3 backbone features and run inside SAM 3's tracker, so we apply the [SAM License](LICENSE_SAM.txt) to be safe. If you pass a student on, you must do so under those terms and include a copy of `LICENSE_SAM.txt`. Its trade control limits apply.
- The pig detectors are fine-tuned RF-DETR Small models under [Apache-2.0](LICENSE_APACHE-2.0.txt). The calf detector was trained on masks from SAM 3, so it falls under the SAM License like the students.
- **Non-commercial limit.** `edinburgh_noncommercial/` holds a student and a detector that may be used for non-commercial purposes only. Both were trained on frames of the [Edinburgh pig dataset](https://homepages.inf.ed.ac.uk/rbf/PIGDATA/) (CC BY-NC; Bergamini et al. 2021). This limit comes from the data licence and applies on top of the SAM License (student) and Apache-2.0 (detector). Please cite that dataset.
- The PigTrack student and detector were trained on [PigTrack](https://doi.org/10.25625/P7VQTP) (CC BY 4.0). Please credit it.
- The calf student and detector were trained on [Calf-Play-Behavior-Dataset, FarmA_R1](https://huggingface.co/datasets/Sonam5/Calf-Play-Behavior-Dataset) (CC0 1.0), days 18 and 19 April only.
- The trunk starts from `timm/tiny_vit_21m_512.dist_in22k_ft_in1k`, changed through training.
See `NOTICE.md`. This note is not legal advice.
## Citation
```bibtex
@misc{yang2026distillation,
title = {Lightweight Distillation of {SAM}~3 and {DINOv3} for Edge-Deployable
Individual-Level Livestock Monitoring and Longitudinal Visual Analytics},
author = {Yang, Haiyu and Hostens, Miel},
year = {2026},
eprint = {2604.27128},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2604.27128}
}
```
If you publish results from a student, you must acknowledge SAM 3: clause 1.b.ii of the SAM License requires it.