sammmmmmm25/alfred-ner-bert

Alfred NER (fine-tuned dslim/bert-base-NER)

This is the entity-extraction subcomponent of Alfred AI Investigations, a Graph RAG assistant for reasoning across criminal case documents. It is not intended to be used standalone -- see the main project repository linked above for full context, architecture, and evaluation results.

Model Description

A fine-tuned version of dslim/bert-base-NER, retrained to recognize 8 entity types relevant to investigative case documents rather than the base model's original 4 general-purpose types:

Label	Description
PER	Person names
ORG	Organizations
LOC	Locations
DATE	Dates and times
VEH	Vehicle descriptions
WEAPON	Weapons
CASE	Case numbers
ALIAS	Aliases / nicknames / saved contact names

At inference time, extracted entities are used to build a chunk-level co-occurrence graph: document chunks sharing an entity (by exact match or fuzzy word-overlap match for multi-word spans) are linked, enabling retrieval to traverse cross-document connections that similarity search alone would miss.

Training Data

Fine-tuned on a small hand-authored, BIO-tagged dataset (~16-20 examples) covering all 8 entity types, written to match the vocabulary of Alfred's synthetic case corpus (witness names, vehicle descriptions, call log entries, case numbers).

Training Procedure

• Base model: dslim/bert-base-NER
• 10 epochs, learning rate 3e-5, batch size 4
• Best checkpoint selected by validation F1

Evaluation

Metric	Value
Precision	0.778
Recall	0.778
F1	0.778
Accuracy	0.923

These numbers are modest and expected given the small training set -- see Limitations below and the main project's Limitations section for the practical consequences this has on downstream retrieval quality.

Limitations

• Small training set. ~16-20 examples is enough to produce a working entity graph for a small demonstration corpus, but far below what a production NER model would use. F1 (0.778) is well below the base model's original performance on its general-purpose entity types.
• Non-deterministic across training runs. Even with fixed random seeds, retraining this model has produced graphs with noticeably different edge counts across separate runs (as few as 2, as many as 11, on the identical downstream corpus) -- likely due to GPU/cuDNN non-determinism not addressed by seeding alone.
• Not validated on real case data. Trained and evaluated entirely on a synthetic, hand-authored corpus. Performance on real investigative documents, which vary far more in structure and phrasing, is untested.

Intended Use

Component of the Alfred AI Investigations pipeline only. Not intended as a general-purpose NER model, and not validated for use in real investigations without substantial additional training data and evaluation.