tesda-e5-small-v2

The matching model for TESDA-TRACK. It is a fine-tune of intfloat/multilingual-e5-small that matches learner goals, written in English, Filipino or Taglish, to TESDA qualifications.

The repo contains the ONNX model only, with no PyTorch weights. The TESDA-TRACK API runs it on CPU through fastembed.

Set up in TESDA-TRACK

From the repository root, download the model into backend/models/:

pip install -U huggingface_hub
hf download Nyanta19/tesda-e5-small-v2 --local-dir backend/models/tesda-e5-small-v2

Add both lines to the root .env:

EMBEDDING_MODEL=tesda-track/tesda-e5-small-v2
EMBEDDING_MODEL_PATH=backend/models/tesda-e5-small-v2

Then embed the catalog with the new model, and restart the API:

cd backend
python -m tesda_track.embeddings sync

• Keep the name as given. EMBEDDING_MODEL must stay tesda-track/tesda-e5-small-v2, because stored embeddings are keyed by that name. Never set it to the base model's name.
• Docker: download into the same folder before docker compose up -d --build. The build copies backend/models/ to /app/finetuned/, so set EMBEDDING_MODEL_PATH=/app/finetuned/tesda-e5-small-v2.
• Fallback: if neither variable is set, the API uses the base model. A missing or broken folder leaves the API on keyword matching.

calibration.json holds the score parameters tuned for this model: z_floor 3.0, z_ceiling 6.0, keyword_weight 0.4, min_score 25. The API reads them from the folder and uses them in place of the settings' values.

Files

File	Purpose
onnx/model.onnx	The model (470 MB, fp32)
tokenizer.json, tokenizer_config.json, config.json	Tokenizer and model config
modules.json, 1_Pooling/, 2_Normalize/, sentence_bert_config.json, config_sentence_transformers.json	Pooling (mean) and L2 normalisation
calibration.json	Score parameters for the TESDA-TRACK matcher
training.json	Training run record

As with every E5 model, prefix goals with query: and qualification text with passage: . The API adds the prefixes itself. Embeddings have 384 dimensions, mean-pooled and L2-normalised; compare them with cosine similarity.

Training

• Data: TESDA catalog text only (qualification names, skills, jobs, keywords and core competencies), plus 4,682 synthetic learner-style queries for all 319 qualifications. 15% of the queries also got one typo or text-speak copy (688 pairs). No learner, worker, T2MIS or TSP records were used. Any query that equals an evaluation goal was dropped.
• Pairs: 24,631 in total: 23,399 for training and 1,232 held out. Held-out loss fell from 2.69 to 0.147.
• Settings: 3 epochs, learning rate 2e-5, batch 64 (mini-batch 16), CachedMultipleNegativesRankingLoss, seed 42, no mined hard negatives.
• Run: 1,098 steps, 17 minutes on a Colab GPU.
• ONNX check: the export reproduces the PyTorch embeddings with a cosine of 0.999999.

Evaluation

The dev set has 1,454 hand-written goals: 1,263 with a TVET match and 191 without one. Top-1 and top-3 count goals whose correct qualification is first, or in the first three. Rejection is the share of no-match goals that correctly get no result.

Model	Score parameters	Top-1	Top-3	MRR	Rejection
tesda-e5-small-v1	4.0 / 5.5 / 0.4 / 25 (its own)	0.578	0.633	0.605	0.927
tesda-e5-small-v1	4.0 / 6.0 / 0.3 / 20 (recalibrated on dev)	0.591	0.639	0.614	0.921
tesda-e5-small-v2	3.0 / 6.0 / 0.4 / 25 (calibrated on dev)	0.719	0.783	0.750	0.901

Score parameters are z_floor / z_ceiling / keyword_weight / min_score. Both calibrations required at least 90% rejection.

Top-1 by language (each model with its dev calibration):

Model	English	Taglish	Filipino
v1	0.743	0.571	0.436
v2	0.846	0.740	0.551

Limitations:

• Optimistic scores: the score parameters were calibrated on this same dev set. The sealed test set (479 goals) has not been run yet.
• Weak spots:
  • Goals next to TVET but outside it: top-1 0.46.
  • Non-goal text (greetings, questions): rejection 0.86.
  • Typo'd no-match goals: rejection 0.75, down from 1.00 with v1.
  • Filipino-only goals: top-1 0.55.
• Scope: made for ranking TESDA qualifications in TESDA-TRACK; it is not a general-purpose embedding model.