CLERC Épée v0.4

The first AI-grade sign language data layer.

CLERC builds the data infrastructure layer for sign language in multimodal AI. 🌐 clerc.io · 💻 Toolkit on GitHub · ✉️ florian@clerc.io Commercial licensing and full corpus access on request.

The corpus is built around one idea: the same phrase, signed by several Deaf signers, in a controlled recording protocol. This is what that looks like, straight from the release keypoints:

Every clip in the parallel grid has this property. Inter-signer variation - style, rhythm, signing space, handshape choices - stops being noise you fight and becomes the thing you can measure.

v0.4 doubles the corpus to 2,400 clips and the signer pool from 6 to 8, widens the parallel grid from 201 to 302 phrases, adds two new signers on their own registers, and fixes the hand slots of every earlier release (see the correction note).

CLERC builds the data layer underneath sign language AI - not a translation tool, not an accessibility app. Infrastructure.

Quick start

import json, numpy as np, pandas as pd
from pathlib import Path
from huggingface_hub import snapshot_download

ROOT = Path(snapshot_download(repo_id="CLERC-DATA/epee", repo_type="dataset"))
meta = pd.read_csv(ROOT / "metadata.csv")

# five renderings of the same phrase, ready to compare
grid = meta[meta.parallel == "yes"]
phrase = grid[grid.phrase_id == 2]
clips = {r.signer_id: np.load(ROOT / "keypoints" / f"{r.clip_id}.npy")
         for r in phrase.itertuples()}          # each (n_frames, 128, 3)

Or in five minutes with the open Python toolkit: epee-asl-toolkit - loaders, keypoint visualization, inter-signer variability analysis.

---

Dataset Summary

• 2,400 ASL clips, 179.93 minutes of continuous signing - 8 Deaf signers, 300 clips each
• Parallel grid - 302 phrases signed by 5 of the 8 signers (1,500 clips), for direct inter-signer variability analysis
• Three more registers - FOXTROT, GOLF and HOTEL each contribute 300 sentences from their own prompt sets, disjoint from the grid and from each other, so the corpus is not only short conversational phrases
• Hand slots verified per clip - 0-20 is the hand at the signer's left wrist, 21-41 at the right wrist, checked against the pose on every clip of the built release
• Explicit phrase alignment - a phrase_id column pairs renderings of the same phrase across signers, no id arithmetic required
• Multimodal keypoints - hands, body, eyes, mouth, head silhouette (MediaPipe-extracted)
• Linguistically validated - ASL gloss annotations with temporal segmentation

This release ships extracted keypoints and annotations only - no raw video. Source clips remain proprietary; access is reserved for commercial licensing (contact florian@clerc.io).

This is v0.4, a pilot release representing a portion of the full CLERC catalog. Full corpus access available via commercial license.

---

Benchmark - why multi-signer data matters

A small BiLSTM trained on the release signers and tested on signers held entirely outside the training set shows the core result: one signer does not generalize to a stranger, more signers do.

• Train on 1 → … → 8 signers, tested on a brand-new signer: 42% → 64% → 67% → 73% → 75% → 77% → 79% → 81% accuracy (macro-F1 0.28 → 0.74).
• More data still lifts it: 42% → 80% as training examples grow (2,877 segments in the v0.4 pool, 1,393 in v0.3).
• Second stranger, same protocol: 69% on the full v0.4 pool.
• Tested on two signers with no clips in this release, same 24-gloss subset, same test segments and same 8 seeds as the v0.2 and v0.3 measurements.

These numbers are not comparable to the v0.3 card as published, because v0.3 was trained and tested with crossed hand slots. To keep the comparison honest the run separates the two effects on the stranger signer: the v0.3 recordings scored 69% as published, 76% once their hand slots are fixed (correction alone, same clips), and the full v0.4 pool reaches 81%.

Full method, numbers, and honest caveats: BENCHMARK.md.

---

Dataset Statistics

Top 10 glosses (cumulative coverage of corpus):

---

Languages

• American Sign Language (ASL) - ISO 639-3: ase
• Written translations in English

---

Dataset Structure

epee/
├── keypoints/           # 2,400 .npy arrays, shape (n_frames, 128, 3)
├── annotations/         # 2,400 .json files
└── metadata.csv         # master index (1 row per clip)

Keypoint layout (128 landmarks per frame)

The 36 head-silhouette landmarks come from MediaPipe FaceMesh FACE_OVAL indices 10, 338, 297, 332, 284, 251, 389, 356, 454, 323, 361, 288, 397, 365, 379, 378, 400, 377, 152, 148, 176, 149, 150, 136, 172, 58, 132, 93, 234, 127, 162, 21, 54, 103, 67, 109 (in that traversal order). The points form a closed polygon outlining the head - no internal facial features are included, so the privacy stance is preserved.

Coordinate space

Coordinates are MediaPipe's image-normalized space, NOT clipped to [0, 1]:

• x is in [0, 1] (frame width); a landmark extrapolated just off-frame can fall slightly outside [0, 1]
• y is in [0, 1] for points visible in frame, but can exceed 1.0 for body landmarks extrapolated below the visible frame
• z is depth relative to the hips, roughly in MediaPipe Pose's world-scale units

Source clips are framed waist-up. Lower-body landmarks (dataset indices 54–63) come from MediaPipe Pose's full-body prediction. For hand/face-only SLR pipelines, they can be dropped:

kp_slr = np.concatenate([kp[:, :54], kp[:, 64:]], axis=1)  # → (n_frames, 118, 3)

Zero values (0, 0, 0) indicate a landmark was not detected for that frame (e.g. an off-screen hand).

Correction note: hand slots

v0.1, v0.2 and v0.3 shipped with crossed hand slots. In those releases the 0-20 / 21-41 blocks were not consistently the same hand: measured on the published v0.3 files, 61.7% of the clips whose convention can be decided had the two hands swapped, and the rate differed by signer. The swap correction existed in the build but a bug kept it from firing, and nothing checked the built files.

In v0.4 every clip is re-anchored at build time (hand root compared to the Pose wrists, majority over all frames where a hand is visible), and the built files are re-checked independently before release: 0 crossed out of 2,395 decidable clips (0.00%). The hand_slots column says verified for every clip whose convention was decided, unverified for the 5 clips with too little hand evidence to decide; those are left as extracted.

If you trained on v0.1 to v0.3, re-download: a model that learned on mixed slots learns signer identity as a shortcut and under-performs on new signers. Earlier versions stay reachable under their tags for reproducibility, with the defect.

Gloss conventions

Glosses (uppercase ASL labels) follow a few conventions worth knowing before training:

Base gloss - WHAT, YOU, BATHROOM. The standard form of a sign.

Variants - BASE_N (e.g. SIGN_2, WHAT_3). Alternative ways to sign the same English concept (different handshape, location, or movement). The number N is an internal disambiguator, not an intensity marker. Treat WHAT, WHAT_2, WHAT_3 as siblings sharing the same English target.

Directional / movement suffixes - POINTER_RIGHT, GO_LEFT, HOW_RIGHT_MOVE. These mark spatial/movement components inherent to the sign and should not be collapsed with their base form.

Phrase repetitions - Some clips contain the target phrase signed more than once (emphasis, demonstration, self-correction). Each occurrence is a separate gloss segment. This is natural signer behavior, not a labeling error.

Recommended preprocessing

import re
def base_gloss(g):
    return re.sub(r"_\d+$", "", g)   # SIGN_2 → SIGN

from collections import Counter
def has_repeat(segments):
    return any(c >= 2 for c in Counter(s["gloss"] for s in segments).values())

Annotation schema (per clip)

{
  "clip_id": "clerc_v04_0001",
  "signer_id": "ALPHA",
  "sign_language": "ASL",
  "text_en": "What's up?",
  "fps": 30.0,
  "n_frames": 139,
  "phrase_id": 1,
  "parallel": true,
  "hand_slots": "verified",
  "segments": [
    { "gloss": "WHAT'S UP", "start": 0.9, "end": 1.4 },
    { "gloss": "QUESTION",  "start": 2.0, "end": 2.8 }
  ]
}

phrase_id identifies the phrase, not the clip: two clips sharing a phrase_id are two signers' renderings of the same phrase, and their text_en strings are identical by construction. parallel is true for the 5-signer grid and false for the three own-register signers: FOXTROT (phrase_id 1001-1300), GOLF (2001-2300) and HOTEL (3001-3300). hand_slots is verified or unverified (see the correction note).

metadata.csv columns

---

Signers

Demographic distribution: 4 female / 4 male, aged 18–45. Signer identities are pseudonymized. ALPHA–FOXTROT are the same six signers as v0.3, under the same pseudonyms and with their v0.3 recordings kept for every phrase already published.

Signers participated under written informed consent. The signing space, framing, lighting, and recording protocol were standardized across signers.

Parallel structure: five signers (ALPHA–ECHO) cover a grid of 302 phrases. Pair renderings through the phrase_id column rather than by arithmetic on clip_id: 5 signers on 298 phrases, 4 signers on 2 phrases, 1 signer on 2 phrases. The phrases at four signers are ECHO's gaps (one v0.2 phrase she did not record, one rendering withdrawn for privacy in v0.4); the phrases at one signer are extras recorded by ECHO alone to bring her block to 300.

FOXTROT, GOLF and HOTEL are deliberately not parallel. Each was recorded on a prompt set of full declarative sentences rather than the short conversational phrases and number expressions of the grid, and the three sets do not overlap. Aligning them would have meant dropping them. They are included as own registers, 300 phrases each, unique to that signer. Filter on parallel == "no" to isolate them, or on parallel == "yes" for strictly comparable inter-signer work.

Stylistic note: Phrase repetition rates vary by signer - natural inter-signer stylistic variation, annotated as separate gloss segments. See gloss conventions for filtering.

---

Intended Use

Designed for

• Inter-signer variability analysis (style, rhythm, signing space)
• Research on sign language linguistics, gesture recognition, multimodal AI
• Educational use in academic settings
• Prototyping sign language recognition (SLR) pipelines on a parallel multi-signer corpus

Not designed for

• Speaker identification or biometric applications
• Surveillance or evaluation of individual signers

For production-grade systems or sign language generation models trained at scale, see commercial licensing for access to the full multi-signer corpus.

---

Loading the Dataset

This release ships as plain .npy + .json files for transparency and zero-dependency loading.

import json
import numpy as np
import pandas as pd
from pathlib import Path
from huggingface_hub import snapshot_download

ROOT = Path(snapshot_download(repo_id="CLERC-DATA/epee", repo_type="dataset"))

metadata = pd.read_csv(ROOT / "metadata.csv")

clip_id = "clerc_v04_0001"
with open(ROOT / "annotations" / f"{clip_id}.json") as f:
    annotation = json.load(f)
keypoints = np.load(ROOT / "keypoints" / f"{clip_id}.npy")

hands       = keypoints[:, :42]
upper_body  = keypoints[:, 42:54]
face_inner  = keypoints[:, 64:92]
head_oval   = keypoints[:, 92:128]

Comparing signers on the same phrase - group by phrase_id inside the parallel grid:

grid = metadata[metadata.parallel == "yes"]
for phrase_id, group in grid.groupby("phrase_id"):
    # one row per signer, same text_en, directly comparable
    print(phrase_id, group.text_en.iloc[0], list(group.signer_id))

---

License

CC BY-NC-SA 4.0 - creativecommons.org/licenses/by-nc-sa/4.0

Commercial licensing: for enterprise use, training of commercial models, or integration into commercial products, contact florian@clerc.io.

---

Ethical Considerations

CLERC is Deaf-led infrastructure. This release adheres to:

• Informed consent - all signers have provided written consent for public release under this license
• Privacy protection - no video, no pixels; face landmarks restricted to non-identifying features (eyes + mouth + head outline), no face mesh, no blendshapes
• Community benefit - released to advance sign language technology research; commercial revenue supports continued Deaf-led data infrastructure
• No surveillance use - must not be used for individual identification, behavioral profiling, or signer surveillance

Skeletons are body data

Removing the video removes the face, not the body. Body proportions in keypoint data act as a soft biometric: on this release, a plain logistic regression separates the six v0.3 signers with 94% accuracy from nine limb-ratio features alone (chance is 17%), using no hand and no face landmarks at all. We measured this rather than assume it, and we state it rather than let "anonymized skeletons" imply more than it delivers.

What this means for you as a user of this data:

• Pseudonymization here protects names, not bodies. ALPHA is a stable identity across clips and across CLERC releases, by design, because parallel analysis requires it.
• Linking these skeletons to any other recording of the same person, in this dataset or outside it, is identification, and it is prohibited under the no-surveillance term above regardless of technical feasibility.
• If you redistribute derivatives under the share-alike term, this property travels with them. Carry this note forward.

We are a Deaf-led organization publishing data about Deaf bodies. Naming this limitation is part of the job, not an admission against interest.

---

Limitations

• Pilot release - 2,400 clips is a baseline pilot, not a production-scale corpus
• 8 signers - limited inter-signer diversity; full catalog includes a broader signer pool
• Three signers outside the grid - FOXTROT, GOLF and HOTEL are not phrase-matched to the others, so strictly parallel analyses run on 1,500 clips, not 2,400
• A few clips with unverified hand slots - 5 clips had too little hand evidence to check, see hand_slots
• Uneven grid coverage - 5 signers on 298 phrases, 4 signers on 2 phrases, 1 signer on 2 phrases inside the grid
• Phrase domain - conversational phrases, number expressions, and (FOXTROT, GOLF, HOTEL) general declarative sentences; not domain-specific (medical, legal, technical)
• Reduced face landmarks - full facial grammar (brow, cheeks, head tilt) not included
• Gloss only - no morphological, prosodic, or spatial annotation layers in v0.4
• Skeletons are a soft biometric - signer identity is recoverable from body proportions (see Ethical Considerations); treat pseudonyms as stable identities, not as anonymity

---

Versioning & Roadmap

Superseded versions stay reachable as git tags on this repo (v0.1, v0.2, v0.3), so earlier clip ids remain resolvable for anyone who cited them:

snapshot_download(repo_id="CLERC-DATA/epee", repo_type="dataset", revision="v0.3")

---

How to Cite

@dataset{clerc_epee_v04_2026,
  author       = {M{\'e}loux, Florian and {CLERC}},
  title        = {{CLERC} {\'E}p{\'e}e v0.4: Sign Language Data Layer},
  year         = {2026},
  publisher    = {Hugging Face},
  version      = {0.4},
  doi          = {10.5281/zenodo.23587708},
  url          = {https://huggingface.co/datasets/CLERC-DATA/epee}
}

---

About CLERC

CLERC builds the data infrastructure that lets AI understand sign language as a first-class language - not an accessibility afterthought.

Sign language is not to be translated. It is to be inscribed.

Website: clerc.io · Contact: florian@clerc.io · LinkedIn: clerc-io

---

Changelog

v0.4 - October 2026

• Hand-slot correction. v0.1 to v0.3 shipped with crossed hand slots (61.7% of decidable v0.3 clips). Every clip is now re-anchored to the Pose wrists and the built files are re-checked before release. New hand_slots column. See the correction note
• Signer pool 6 → 8: added GOLF and HOTEL, each on an own prompt set. Corpus 1,200 → 2,400 clips, 300 per signer
• Parallel grid widened 201 → 302 phrases across 5 signers. Phrase 201, ECHO-only in v0.3, is now signed by all five. 5 signers on 298 phrases, 4 signers on 2 phrases, 1 signer on 2 phrases
• FOXTROT 200 → 300 clips on his own register (phrase_id 1001-1300)
• Every v0.3 recording is kept for its phrase except one, withdrawn for privacy (a fill-in prompt answered with personal information); clips renumbered clerc_v04_0001–clerc_v04_2400, v0.3 ids stay resolvable under tag v0.3
• Benchmark re-run 1 → 8 signers with hand slots fixed in train and test; the effect of the correction alone is reported separately
• No clip and no new phrase of the sequestered Épée 200 exam is published

v0.3 - August 2026

• Signer pool 4 → 6: added ECHO and FOXTROT. Corpus 600 → 1,200 clips, 200 per signer
• Parallel grid widened 150 → 201 phrases across 5 signers. 5 signers on 199 phrases, 4 signers on 1 phrase, 1 signer on 1 phrase
• FOXTROT added as a second register: 200 declarative sentences on a prompt set disjoint from the grid, flagged parallel = no
• New phrase_id and parallel fields in metadata.csv and in each annotation, so cross-signer pairing no longer depends on clip-id arithmetic
• New oval_detection column reporting per-clip head-silhouette coverage
• Clips renumbered clerc_v03_0001–clerc_v03_1200; v0.2 ids stay resolvable under tag v0.2
• Head silhouette now extracted on a pose-guided head crop rather than the full frame. MediaPipe's face detector drops faces below roughly 8% of image width; on the clips where this happened the old path lost up to 21% of frames, the new one loses none. The 92 body/hand/face-subset keypoints are produced by the unchanged pipeline
• The four v0.2 signers keep their pseudonyms and their original recordings for the 150 phrases already published
• Signer-robustness benchmark re-run on the six v0.3 signers, same protocol, vocabulary and held-out signers as v0.2 - the two curves are directly comparable (v0.2 overlaid in the figure)
• Added a measured statement on skeletons as a soft biometric, replacing the v0.2 card's "full biometric data excluded" wording, which overstated what removing video achieves

v0.2 - June 2026

• Added 4th signer (DELTA) and expanded to 600 clips
• Restructured to 150 fully-parallel phrases × 4 signers (phrase-aligned clip blocks)
• Same 128 multimodal keypoints/frame and gloss schema as v0.1

v0.1 - May 2026

• Initial public release - 300 clips, 3 signers, parallel structure