Shreyas94/needle-event-classifier

Sentinel Needle Event Classifier

A 13-class financial-news event classifier: a StandardScaler + LogisticRegression(C=0.01) head trained on top of frozen Cactus Compute Needle 3 text embeddings.

This repo is the classifier head only. Needle 3 is a third-party, frozen embedding model from Cactus Compute — we did not train it and make no claim to it. What was trained here is the logistic-regression classification head fit on Needle 3's 3072-dimensional embeddings. Credit for the base embedder belongs to Cactus Compute.

What it does

Given a financial news headline (embedded by Needle 3), predicts one of 13 event categories used to tag and route headlines in a financial-news monitoring pipeline (originally deployed as needle_shadow.py in the Sentinel News Tracker project, where it is the authoritative source for event_category).

Taxonomy (13 classes)

capital_markets, default, downgrade, earnings, liquidity, ma (mergers & acquisitions), management_change, other, portfolio_company_event, rating_action, refinancing, regulatory, upgrade

Methodology

An earlier nearest-centroid classifier on the same embeddings was found to be the accuracy bottleneck (not the embedding or dataset): on a frozen, human-reviewed 678-row test set, centroid classification scored ~58–65% accuracy, while a plain logistic regression on the identical embeddings scored ~90–91%. A separate, independent issue with the prior deployed artifact (fit on stale, purely-templated training data) was also found and fixed by refitting on real, human-reviewed data.

Preprocessing: StandardScaler on the raw Needle embedding only — no L2-normalization, no mean-centering, no PCA.

Model: LogisticRegression(C=0.01, max_iter=2000), multiclass, 13 classes.

Training data: 2,360 rows — 2,188 real, human-reviewed ("GOLD") examples (13-class taxonomy, with a fundraising category merged into capital_markets) plus 172 Gemini-augmented rows targeting underrepresented classes (refinancing, liquidity, ma).

Embedding dimension: 3072 (Needle 3 output).

Evaluation

Validated on a frozen, human-reviewed 678-row real-world test set, held out from training and never used for fitting or hyperparameter tuning:

Metric	Score
Accuracy	91.45%
Macro F1	0.9057

Usage

This repo ships the fitted classifier head (classifier_artifact.json) and a standalone inference wrapper (inference.py). You need the cactus-needle package (or equivalent access to Needle 3 embeddings) to produce the input embedding — this repo does not include or reimplement the embedder itself.

from inference import NeedleEventClassifier

clf = NeedleEventClassifier()

# Option 1: you already have a Needle 3 embedding (3072-dim list/array)
result = clf.predict(embedding)

# Option 2: classify raw text directly (requires `pip install cactus-needle`)
result = clf.predict_text("Acme Corp downgraded to B- by Fitch on liquidity concerns")

print(result["prediction"], result["confidence"])
# -> "downgrade" 0.87

predict() / predict_text() return:

{
    "prediction": "downgrade",
    "confidence": 0.87,
    "second_best": "rating_action",
    "second_best_confidence": 0.06,
    "probabilities": {...},   # full 13-class distribution
    "model_version": "logreg-v1-2360row-13class",
    "taxonomy_version": "13-class-v2",
}

Files

• classifier_artifact.json — fitted StandardScaler mean/scale and LogisticRegression coefficients/intercept, plus taxonomy and metadata.
• inference.py — standalone scoring wrapper (numpy-only for the head; cactus-needle optional, only needed for predict_text).

Credits

• Base embedder: Cactus Compute's Needle 3 (frozen, not trained or owned by this repo).
• Classifier head, training data curation, and evaluation: Shreyas Desai, as part of the Sentinel News Tracker project.

License

MIT for the classifier head and code in this repo. Use of Needle 3 itself is governed by Cactus Compute's own terms.