tesda-e5-small-v3

The matching model for TESDA-TRACK. It is a fine-tune of intfloat/multilingual-e5-small that matches learner goals, written in English, Filipino or Taglish, to TESDA qualifications. Its training data also has synthetic examples in Cebuano, Ilocano and Hiligaynon (see the limitations).

It replaces tesda-e5-small-v2, which stays published for rollback. The repo contains the ONNX model only, with no PyTorch weights. The TESDA-TRACK API runs it on CPU through fastembed.

Set up in TESDA-TRACK

From the repository root, download the model into backend/models/:

pip install -U huggingface_hub
hf download Nyanta19/tesda-e5-small-v3 --local-dir backend/models/tesda-e5-small-v3

Add both lines to the root .env, replacing any earlier EMBEDDING_MODEL* lines:

EMBEDDING_MODEL=tesda-track/tesda-e5-small-v3
EMBEDDING_MODEL_PATH=backend/models/tesda-e5-small-v3

Then embed the catalog with the new model, and restart the API:

cd backend
python -m tesda_track.embeddings sync

• Keep the name as given. EMBEDDING_MODEL must stay tesda-track/tesda-e5-small-v3, because stored embeddings are keyed by that name. Never set it to the base model's name.
• Rollback: keep the v2 folder. Point both lines back at tesda-e5-small-v2, run sync, and restart the API.
• Docker: download into the same folder before docker compose up -d --build. The build copies backend/models/ to /app/finetuned/, so set EMBEDDING_MODEL_PATH=/app/finetuned/tesda-e5-small-v3.
• Fallback: if neither variable is set, the API uses the base model. A missing or broken folder leaves the API on keyword matching.

calibration.json holds the score parameters tuned for this model: z_floor 3.5, z_ceiling 5.5, keyword_weight 0.3, min_score 20. The API reads them from the folder and uses them in place of the settings' values.

Files

File	Purpose
onnx/model.onnx	The model (470 MB, fp32)
tokenizer.json, tokenizer_config.json, config.json	Tokenizer and model config
modules.json, 1_Pooling/, 2_Normalize/, sentence_bert_config.json, config_sentence_transformers.json	Pooling (mean) and L2 normalisation
calibration.json	Score parameters for the TESDA-TRACK matcher
training.json, run_config.json	Training run record and settings
coverage.json	Training-data counts per language, example kind and qualification
checksums.json	SHA-256 of the files produced by the training run

As with every E5 model, prefix goals with query: and qualification text with passage: . The API adds the prefixes itself. Embeddings have 384 dimensions, mean-pooled and L2-normalised; compare them with cosine similarity.

Training

• Data: TESDA catalog text only (qualification names, skills, jobs, keywords and core competencies), plus 12,185 synthetic learner-style queries for all 319 qualifications. New in v3:

  • English requests built from each qualification's Core units: single tasks (2,798), the exact NC level (319) and two skills together (310).
  • 1,179 examples each in Cebuano/Bisaya, Ilocano and Hiligaynon/Ilonggo: code-switched job requests plus 15 hand-written task requests per language.

  15% of the queries also got one typo or text-speak copy (1,778 pairs). No learner, worker, T2MIS or TSP records were used. Every evaluation goal was excluded before training and augmentation.

• Pairs: 40,356 in total: 38,335 for training and 2,021 held out. The same normalised query never appears on both sides. Held-out loss fell from 2.77 to 0.118.

• Settings: 3 epochs, learning rate 2e-5, batch 64 (mini-batch 16), CachedMultipleNegativesRankingLoss, seed 42, no mined hard negatives.

• Run: 1,797 steps, 41 minutes on a Colab GPU.

• ONNX check: the export reproduces the PyTorch embeddings with a cosine of 0.999999.

Evaluation

Top-1 and top-3 count goals whose correct qualification is first, or in the first three. Rejection is the share of goals with no TVET match that correctly get no result. Score parameters are z_floor / z_ceiling / keyword_weight / min_score.

All numbers come from the current TESDA-TRACK scoring code. That code now applies plain-language goal rules before the model, which were added after v2's model card was written. This is why v2's numbers here differ from its card.

Sealed test set: 494 goals (406 with a TVET match, 88 without). It was never used for training or tuning, and was run once per model, after calibration.

Model	Score parameters	Top-1	Top-3	MRR	Rejection
tesda-e5-small-v2	3.0 / 6.0 / 0.4 / 25 (its own)	0.746	0.825	0.784	0.932
tesda-e5-small-v3	3.5 / 5.5 / 0.3 / 20 (its own)	0.805	0.877	0.840	0.932

Top-1 by language on the sealed set:

Model	English	Taglish	Filipino
v2	0.809	0.857	0.569
v3	0.840	0.879	0.691

Dev set: 1,475 goals (1,278 with a TVET match, 197 without). v3's score parameters were calibrated on this set (at least 90% rejection required), so its dev numbers are optimistic.

Model	Score parameters	Top-1	Top-3	MRR	Rejection
tesda-e5-small-v2	3.0 / 6.0 / 0.4 / 25 (its own)	0.736	0.783	0.758	1.000
tesda-e5-small-v2	3.5 / 5.5 / 0.3 / 20 (recalibrated on dev)	0.760	0.808	0.782	0.985
tesda-e5-small-v3	3.5 / 5.5 / 0.3 / 20 (calibrated on dev)	0.780	0.840	0.808	0.954

v1 test set: 120 goals (99 with a TVET match, 21 without).

Model	Top-1	Top-3	MRR	Rejection
v2	0.687	0.717	0.699	1.000
v3	0.788	0.818	0.800	0.905

The TESDA-TRACK real-model test (8 goals with a known best match, 5 with none) passes with v3.

Limitations:

• Regional languages are not established. The regional training examples are synthetic and have not been reviewed by native speakers. The evaluation sets hold only 5 matchable and 2 no-match goals per language in dev, and 4 and 1 in the sealed set. On the sealed set v3 ranked the right qualification first for 3 of 4 Cebuano, 2 of 4 Ilocano and 4 of 4 Hiligaynon goals. Do not rely on Cebuano, Ilocano or Hiligaynon results yet.
• More results for goals with no TVET match. Rejection is equal on the sealed set, but lower than v2's on dev (0.954 against 0.985 at the same parameters) and on the v1 test set (19 of 21 against 21 of 21).
• Weak spots (sealed set):
  • Filipino-only goals: top-1 0.69.
  • Indirect goals: top-1 0.70.
  • Goals next to TVET but outside it: top-1 0.72 (0.54 on dev).
  • No-TVET and vague goals: rejection 0.92.
• Scope: made for ranking TESDA qualifications in TESDA-TRACK; it is not a general-purpose embedding model.