Trust evidence classifier

Labels how strong a piece of evidence is for an AI deployment review, on Trust's evidence ladder. Trust (https://huggingface.co/spaces/abdulaziz01/team-103-trust-app) uses it on the Evidence screen to suggest the strength of each item a project team submits. Fixed rules then cap what is counted: an uploaded document counts at most as document-inspected, and test results only count when Trust runs the tests itself.

Label Meaning
declared A claim or promise with nothing behind it ("access is restricted")
document-inspected An excerpt from a formal document: policy, procedure, DPIA, access matrix, supplier terms
configuration-checked Live settings or configuration: YAML, JSON, key=value, admin exports
test-observed Recorded test results: cases, pass/fail counts, traces
not-evidence Text that supports no check: logistics, marketing, questions

Training

  • Base model: jhu-clsp/mmBERT-small (multilingual ModernBERT, Arabic and English)
  • Data: abdulaziz01/team-103-trust-evidence-data, 1530 training and 170 validation examples, synthetic and bilingual
  • 4 epochs, learning rate 5e-5, batch size 32, max length 256, trained on Hugging Face Jobs (T4)

Results on the held-out test set

The 150 test examples come from separate templates the model never saw during training, written with different phrasing, so the score reflects generalisation to new wording.

Accuracy Macro F1
This model 0.920 0.923
Rules baseline (Trust's previous heuristic) 0.360 0.317

Accuracy by language: English 0.885, Arabic 0.968.

Label Precision Recall F1 Support
declared 1.00 0.87 0.93 30
document-inspected 1.00 0.97 0.98 30
configuration-checked 1.00 0.77 0.87 30
test-observed 1.00 1.00 1.00 30
not-evidence 0.71 1.00 0.83 30

Limitations

  • All training and test data are synthetic. Real project documents are longer, messier and mix languages; expect lower accuracy on them until a human-written test set is added.
  • The model suggests a strength; it never decides a finding. Trust's caps and its own tests decide what counts.
  • It reads up to 256 tokens, so a long document is judged from its opening.
  • Built at Qai ร— Hugging Face Hackathon 01. Not for use as the sole basis of any approval.
Downloads last month
42
Safetensors
Model size
0.1B params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for abdulaziz01/team-103-trust

Finetuned
(76)
this model

Dataset used to train abdulaziz01/team-103-trust

Space using abdulaziz01/team-103-trust 1