tylermaister/flan-t5-small-triggering

FLAN-T5 Triggering Classifier

This model classifies whether conversational text should trigger one of two independent behaviors:

• action_triggering
• memory_triggering

For each task, the model returns exactly one of:

• TRIGGER
• NO_TRIGGER

The model only classifies intent. It does not authorize or execute actions and should be used together with a separate authorization and safety layer.

Intended input format

Provide the task and text in the following format:

task: action_triggering
text: Take a seat please

The expected output is:

TRIGGER

Usage

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

model_id = "tylermaister/flan-t5-small-triggering"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSeq2SeqLM.from_pretrained(model_id)

prompt = "task: memory_triggering\ntext: What did I ask you to remember?"
inputs = tokenizer(prompt, return_tensors="pt", truncation=True, max_length=128)
outputs = model.generate(**inputs, max_new_tokens=8)
prediction = tokenizer.decode(outputs[0], skip_special_tokens=True).strip()
print(prediction)

Applications should accept only TRIGGER and NO_TRIGGER. Any other generated text should be treated as invalid rather than interpreted as a trigger decision.

Training data and evaluation

The model was fine-tuned from google/flan-t5-small on 476 deduplicated examples covering both tasks and both labels. The data includes synthetic examples and manually collected negative examples from game testing.

Evaluation was performed on an 80-example synthetic test set that was kept separate from training:

Metric	Result
Overall accuracy	98.75%
Overall macro F1	98.75%
Invalid outputs	0

These results should not be treated as an estimate of production performance. The evaluation set is synthetic, and real-world language may contain ambiguity, typos, indirect phrasing, or examples outside the training distribution. Build and evaluate a manually reviewed real-world test set before deployment.

Limitations and safety

• The model is not an authorization or policy engine.
• Classification errors can result in false triggers or missed triggers.
• Synthetic-data performance may not generalize to real user conversations.
• Outputs must be normalized and validated against the two allowed labels.
• Keep action execution outside the model and behind explicit authorization and safety checks.

Citation

This model is based on FLAN-T5 and uses the Apache-2.0 license inherited from the base model.