Text Classification
PEFT
Safetensors
English
Japanese
emotion-recognition
affective-computing
multitask-learning
transfer-learning
lora
Instructions to use Affectosphere-Group/Affeline-1-8B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use Affectosphere-Group/Affeline-1-8B with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
Affectra-1-8B: initial release (adapter+heads+read_delta, inference package, model card)
Browse files- LICENSE +12 -0
- README.md +127 -0
- adapter/adapter_config.json +23 -0
- adapter/adapter_model.safetensors +3 -0
- affectra/__init__.py +16 -0
- affectra/format.py +47 -0
- affectra/metrics.py +146 -0
- affectra/model.py +194 -0
- affectra/output.py +82 -0
- affectra/task_metadata.py +311 -0
- heads.pt +3 -0
- read_delta.pt +3 -0
- setup.py +12 -0
LICENSE
ADDED
|
@@ -0,0 +1,12 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Affectra-1-8B model weights
|
| 2 |
+
Copyright (c) 2026 the Affectra authors.
|
| 3 |
+
|
| 4 |
+
Licensed under the Creative Commons Attribution-NonCommercial 4.0 International
|
| 5 |
+
License (CC BY-NC 4.0). You may share and adapt the weights for non-commercial
|
| 6 |
+
purposes with attribution. Full text: https://creativecommons.org/licenses/by-nc/4.0/legalcode
|
| 7 |
+
|
| 8 |
+
The most restrictive training-data license (WRIME v2, non-commercial) is inherited.
|
| 9 |
+
No training text is distributed with these weights. Tasks derived from X/Twitter ship
|
| 10 |
+
no text; WRIME and MELD text are not redistributed.
|
| 11 |
+
|
| 12 |
+
NOT FOR CLINICAL OR DIAGNOSTIC USE. Provided "as is", without warranty of any kind.
|
README.md
ADDED
|
@@ -0,0 +1,127 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: cc-by-nc-4.0
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
- ja
|
| 6 |
+
- multilingual
|
| 7 |
+
library_name: peft
|
| 8 |
+
base_model: Qwen/Qwen3-8B-Base
|
| 9 |
+
tags:
|
| 10 |
+
- emotion-recognition
|
| 11 |
+
- affective-computing
|
| 12 |
+
- text-classification
|
| 13 |
+
- multitask-learning
|
| 14 |
+
- transfer-learning
|
| 15 |
+
- lora
|
| 16 |
+
pipeline_tag: text-classification
|
| 17 |
+
---
|
| 18 |
+
|
| 19 |
+
# Affectra-1-8B
|
| 20 |
+
|
| 21 |
+
**Affectra** is a text-emotion **foundation model**: a single set of weights that solves
|
| 22 |
+
15 heterogeneous affective tasks — discrete-emotion classification, dimensional (VAD)
|
| 23 |
+
regression, intensity regression, interpersonal judgments (irony, hate speech, stance),
|
| 24 |
+
an aspect-level applied task, and multilingual emotion detection — in English, Japanese,
|
| 25 |
+
and multilingual data.
|
| 26 |
+
|
| 27 |
+
This is `Affectra-1-8B`, the first release. The series is versioned (`Affectra-1`,
|
| 28 |
+
`Affectra-2`, …) and sized (`-8B`).
|
| 29 |
+
|
| 30 |
+
> **Weights only.** No training text is distributed with this model (see *License & data*).
|
| 31 |
+
> **Research / non-commercial** use only (CC BY-NC 4.0).
|
| 32 |
+
> **Not a diagnostic tool.** Do not use for clinical, psychiatric, or any
|
| 33 |
+
> individual-level consequential decision.
|
| 34 |
+
|
| 35 |
+
## Architecture
|
| 36 |
+
|
| 37 |
+
- **Backbone:** `Qwen/Qwen3-8B-Base`, **frozen**.
|
| 38 |
+
- **Adapter:** LoRA (`r=32`, `alpha=64`) on all seven linear projections
|
| 39 |
+
(`q,k,v,o,gate,up,down`); **88.7 M** trainable parameters (~334 MB).
|
| 40 |
+
- **Late-Read input format:** `[txt: <body>] <ch> labels: <names> </ch> [READ]`. The
|
| 41 |
+
channel (label schema) is placed *after* the body, so under causal masking the body's
|
| 42 |
+
hidden states are provably independent of the channel; the final `[READ]` state attends
|
| 43 |
+
to both and is task-schema conditioned.
|
| 44 |
+
- **Multi-depth readout:** the `[READ]` hidden state is concatenated from layers
|
| 45 |
+
`{-1, -9, -17}` (dim 12288), with a trained `[READ]` embedding delta.
|
| 46 |
+
- **Heads:** one linear head per task on top of the shared representation.
|
| 47 |
+
|
| 48 |
+
## How to use
|
| 49 |
+
|
| 50 |
+
```python
|
| 51 |
+
# pip install git+https://huggingface.co/<namespace>/Affectra-1-8B (package ships with the repo)
|
| 52 |
+
from affectra import Affectra
|
| 53 |
+
|
| 54 |
+
model = Affectra.from_pretrained("<namespace>/Affectra-1-8B", device="cuda")
|
| 55 |
+
|
| 56 |
+
out = model.predict("I'm so grateful for today!", task="M01_GoEmotions")
|
| 57 |
+
print(out.top_labels) # e.g. ['gratitude', 'joy']
|
| 58 |
+
|
| 59 |
+
out = model.predict("What a terrible, infuriating day.", task="M06_TweetEval_Emotion")
|
| 60 |
+
print(out.top_labels) # e.g. ['anger']
|
| 61 |
+
```
|
| 62 |
+
|
| 63 |
+
Provide `task=<task_id>` (or its public name). Open-label zero-shot for an arbitrary
|
| 64 |
+
unseen label set is not a capability of this checkpoint (see the paper's future work).
|
| 65 |
+
|
| 66 |
+
## Results (test set, mean ± SD over 3 seeds)
|
| 67 |
+
|
| 68 |
+
| Task | Metric | Score |
|
| 69 |
+
|---|---|---|
|
| 70 |
+
| GoEmotions | macro-F1 | 0.511 ± 0.006 |
|
| 71 |
+
| WRIME-reader | Pearson r | 0.660 ± 0.007 |
|
| 72 |
+
| WRIME-writer | Pearson r | 0.513 ± 0.006 |
|
| 73 |
+
| SemEval-2025 Track A | macro-F1 | 0.784 ± 0.005 |
|
| 74 |
+
| SemEval-2025 Track B (intensity) | Pearson r | 0.781 ± 0.004 |
|
| 75 |
+
| MELD | weighted-F1 | 0.660 ± 0.003 |
|
| 76 |
+
| TweetEval-Emotion | macro-F1 | 0.823 ± 0.004 |
|
| 77 |
+
| EI-reg | Pearson r | 0.741 ± 0.004 |
|
| 78 |
+
| EmoBank (VAD) | MAE (lower better) | 0.168 ± 0.003 |
|
| 79 |
+
| ABSA Restaurant | accuracy | 0.897 ± 0.012 |
|
| 80 |
+
| Irony | macro-F1 | 0.833 ± 0.011 |
|
| 81 |
+
| HateXplain | macro-F1 | 0.683 ± 0.007 |
|
| 82 |
+
| EmpatheticDialogues | accuracy | 0.350 ± 0.006 |
|
| 83 |
+
| FinancialPhraseBank | macro-F1 | 0.983 ± 0.008 |
|
| 84 |
+
| Stance (5 topics) | F (favor/against) | 0.802 ± 0.008 |
|
| 85 |
+
|
| 86 |
+
The single model is, overall, on par with same-capacity task-specific specialists; the
|
| 87 |
+
benefit of sharing weights is **resource-dependent** (larger on low-resource tasks). The
|
| 88 |
+
largest per-task seed standard deviation is 0.012.
|
| 89 |
+
|
| 90 |
+
**Transfer to unseen tasks.** With the model retrained on 12 tasks and three held out, a
|
| 91 |
+
linear head on the *frozen* Affectra representation far exceeds the same probe on the
|
| 92 |
+
plain Qwen3-8B-Base representation and approaches a task-specific specialist — e.g.
|
| 93 |
+
TweetEval-Emotion macro-F1 0.815 vs. 0.463 (base) at 100% of the labels, and already
|
| 94 |
+
0.683 vs. 0.209 with 1% — which is the property that makes the representation reusable.
|
| 95 |
+
|
| 96 |
+
## Limitations
|
| 97 |
+
|
| 98 |
+
- **Not for clinical / diagnostic use** or any individual-level consequential decision.
|
| 99 |
+
- **Hate-speech / toxicity heads over-flag.** HateXplain macro-F1 is 0.68; benign text can
|
| 100 |
+
be misclassified. Do not deploy as a moderation oracle.
|
| 101 |
+
- GoEmotions (28-class multi-label), MELD (conversational), and EmpatheticDialogues
|
| 102 |
+
(32-way) are the hardest tasks and are below specialist/published bests.
|
| 103 |
+
- English-centric overall; Japanese is covered only by the two WRIME tasks.
|
| 104 |
+
- Emotion labels encode a perspective rather than a single ground truth; error rises on
|
| 105 |
+
sentences where annotators disagree.
|
| 106 |
+
|
| 107 |
+
## License & data
|
| 108 |
+
|
| 109 |
+
- **Model weights:** CC BY-NC 4.0 (research / non-commercial). The most restrictive
|
| 110 |
+
training-data license (WRIME v2, non-commercial) is inherited.
|
| 111 |
+
- **No training text is included.** Tasks derived from X/Twitter (TweetEval, EI-reg,
|
| 112 |
+
irony, stance) ship no text; WRIME and MELD text are not redistributed. Only adapter
|
| 113 |
+
weights, task heads, the `[READ]` delta, and inference code are released.
|
| 114 |
+
- Reconstruct any dataset from its original source; recorded body-text hashes allow a
|
| 115 |
+
later check of identity.
|
| 116 |
+
|
| 117 |
+
## Citation
|
| 118 |
+
|
| 119 |
+
A paper describing Affectra is under review; citation is provisional.
|
| 120 |
+
|
| 121 |
+
```bibtex
|
| 122 |
+
@misc{affectra2026,
|
| 123 |
+
title = {Affectra: A Late-Read Foundation Model for Transferable Text Emotion Understanding},
|
| 124 |
+
note = {Under review; model at Hugging Face},
|
| 125 |
+
year = {2026}
|
| 126 |
+
}
|
| 127 |
+
```
|
adapter/adapter_config.json
ADDED
|
@@ -0,0 +1,23 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"base_model_name_or_path": "Qwen/Qwen3-8B-Base",
|
| 3 |
+
"bias": "none",
|
| 4 |
+
"fan_in_fan_out": false,
|
| 5 |
+
"inference_mode": true,
|
| 6 |
+
"init_lora_weights": true,
|
| 7 |
+
"lora_alpha": 64,
|
| 8 |
+
"lora_dropout": 0.05,
|
| 9 |
+
"modules_to_save": null,
|
| 10 |
+
"peft_type": "LORA",
|
| 11 |
+
"r": 32,
|
| 12 |
+
"revision": null,
|
| 13 |
+
"target_modules": [
|
| 14 |
+
"q_proj",
|
| 15 |
+
"k_proj",
|
| 16 |
+
"v_proj",
|
| 17 |
+
"o_proj",
|
| 18 |
+
"gate_proj",
|
| 19 |
+
"up_proj",
|
| 20 |
+
"down_proj"
|
| 21 |
+
],
|
| 22 |
+
"task_type": "CAUSAL_LM"
|
| 23 |
+
}
|
adapter/adapter_model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3ef4d65e58e57a8c45a519d13c5d50d26122338369f8fbc39fafbbc5c505ec8f
|
| 3 |
+
size 349243720
|
affectra/__init__.py
ADDED
|
@@ -0,0 +1,16 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Affectra: a Late-Read foundation model for text emotion understanding.
|
| 2 |
+
|
| 3 |
+
from affectra import Affectra
|
| 4 |
+
model = Affectra.from_pretrained("affectosphere/Affectra-1-8B")
|
| 5 |
+
model.predict("great movie!", task="tweet_emotion")
|
| 6 |
+
"""
|
| 7 |
+
from .model import Affectra, READOUT_LAYERS, HEAD_IN_DIM, BASE_MODEL
|
| 8 |
+
from .format import build_late_read, build_early_channel, READ_TOKEN
|
| 9 |
+
from .output import EmotionOutput
|
| 10 |
+
from .task_metadata import TASKS, TaskType, TaskMeta
|
| 11 |
+
from . import metrics
|
| 12 |
+
|
| 13 |
+
__all__ = ["Affectra", "EmotionOutput", "TASKS", "TaskType", "TaskMeta",
|
| 14 |
+
"build_late_read", "build_early_channel", "READ_TOKEN",
|
| 15 |
+
"READOUT_LAYERS", "HEAD_IN_DIM", "BASE_MODEL", "metrics"]
|
| 16 |
+
__version__ = "0.1.0-review"
|
affectra/format.py
ADDED
|
@@ -0,0 +1,47 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Late-Read input formatting for Affectra.
|
| 2 |
+
|
| 3 |
+
The channel (the list of candidate label names, plus optional ERC context / ABSA target)
|
| 4 |
+
is placed *after* the body, followed by a [READ] token. Placing the channel after the
|
| 5 |
+
body makes the hidden states at the body positions independent of the channel, which is
|
| 6 |
+
the property that lets one model serve any task without a task identifier.
|
| 7 |
+
|
| 8 |
+
This module is pure string logic and mirrors the training-time format exactly, so it can
|
| 9 |
+
be unit-tested without the model weights.
|
| 10 |
+
"""
|
| 11 |
+
from __future__ import annotations
|
| 12 |
+
|
| 13 |
+
from typing import List, Optional
|
| 14 |
+
|
| 15 |
+
READ_TOKEN = "[READ]"
|
| 16 |
+
|
| 17 |
+
|
| 18 |
+
def build_late_read(
|
| 19 |
+
text: str,
|
| 20 |
+
labels: List[str],
|
| 21 |
+
erc_context: Optional[str] = None,
|
| 22 |
+
absa_target: Optional[str] = None,
|
| 23 |
+
) -> str:
|
| 24 |
+
"""Return the Late-Read string: [ctx] [tgt] [txt: body] <ch> labels </ch> [READ].
|
| 25 |
+
|
| 26 |
+
Parameters
|
| 27 |
+
----------
|
| 28 |
+
text : the body text to analyze.
|
| 29 |
+
labels : the candidate label names for the task (the channel content).
|
| 30 |
+
erc_context : prior dialogue turns, for emotion-recognition-in-conversation only.
|
| 31 |
+
absa_target : the aspect term, for aspect-based sentiment analysis only.
|
| 32 |
+
"""
|
| 33 |
+
parts = []
|
| 34 |
+
if erc_context:
|
| 35 |
+
parts.append(f"[ctx: {erc_context}]")
|
| 36 |
+
if absa_target:
|
| 37 |
+
parts.append(f"[tgt: {absa_target}]")
|
| 38 |
+
parts.append(f"[txt: {text}]")
|
| 39 |
+
channel = ", ".join(labels)
|
| 40 |
+
parts.append(f"<ch> {channel} </ch> {READ_TOKEN}")
|
| 41 |
+
return " ".join(parts)
|
| 42 |
+
|
| 43 |
+
|
| 44 |
+
def build_early_channel(text: str, labels: List[str]) -> str:
|
| 45 |
+
"""The conventional early-channel format, for ablation/comparison only."""
|
| 46 |
+
channel = ", ".join(labels)
|
| 47 |
+
return f"<ch> {channel} </ch> [txt: {text}] {READ_TOKEN}"
|
affectra/metrics.py
ADDED
|
@@ -0,0 +1,146 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Evaluation-safe metrics for Affectra.
|
| 2 |
+
|
| 3 |
+
This module is the practical form of the paper's methodological contribution. Cross-task
|
| 4 |
+
emotion evaluation breaks silently in three ways, and each has a guard here:
|
| 5 |
+
|
| 6 |
+
1. A metric silently falls back to a default when the registered one is not computed.
|
| 7 |
+
-> `score()` raises if asked for a metric it did not actually compute.
|
| 8 |
+
2. A zero-inflated continuous target rewards predicting zero under MAE.
|
| 9 |
+
-> `constant_floor()` reports the all-zero / majority predictor so a model that has
|
| 10 |
+
not learned cannot hide behind a plausible-looking number.
|
| 11 |
+
3. A published value is compared on a mismatched split or scored class set.
|
| 12 |
+
-> `assert_split()` records the body-text hash and the scored label set.
|
| 13 |
+
|
| 14 |
+
The official primary metric per task is named explicitly; an unlisted task raises rather
|
| 15 |
+
than falling back, exactly as in the training code.
|
| 16 |
+
"""
|
| 17 |
+
from __future__ import annotations
|
| 18 |
+
|
| 19 |
+
import hashlib
|
| 20 |
+
from typing import Dict, List, Sequence
|
| 21 |
+
|
| 22 |
+
import numpy as np
|
| 23 |
+
|
| 24 |
+
# registered primary metric per task family; never fall back to "whatever was computed"
|
| 25 |
+
PRIMARY_METRIC = {
|
| 26 |
+
"classification_macro_f1": "macro_f1",
|
| 27 |
+
"classification_weighted_f1": "weighted_f1",
|
| 28 |
+
"classification_accuracy": "accuracy",
|
| 29 |
+
"stance_favor_against": "macro_f1_favor_against",
|
| 30 |
+
"regression_pearson": "pearson_r_mean",
|
| 31 |
+
"regression_mae": "mae_mean",
|
| 32 |
+
}
|
| 33 |
+
LOWER_BETTER = {"mae_mean"}
|
| 34 |
+
|
| 35 |
+
|
| 36 |
+
class MetricMismatch(Exception):
|
| 37 |
+
pass
|
| 38 |
+
|
| 39 |
+
|
| 40 |
+
def _macro_f1(y_true, y_pred, n_classes) -> float:
|
| 41 |
+
from sklearn.metrics import f1_score
|
| 42 |
+
return float(f1_score(y_true, y_pred, labels=list(range(n_classes)),
|
| 43 |
+
average="macro", zero_division=0))
|
| 44 |
+
|
| 45 |
+
|
| 46 |
+
def _weighted_f1(y_true, y_pred) -> float:
|
| 47 |
+
from sklearn.metrics import f1_score
|
| 48 |
+
return float(f1_score(y_true, y_pred, average="weighted", zero_division=0))
|
| 49 |
+
|
| 50 |
+
|
| 51 |
+
def _accuracy(y_true, y_pred) -> float:
|
| 52 |
+
return float(np.mean(np.asarray(y_true) == np.asarray(y_pred)))
|
| 53 |
+
|
| 54 |
+
|
| 55 |
+
def _favor_against_f1(y_true, y_pred) -> float:
|
| 56 |
+
"""SemEval-2016 Task 6A: mean of the F1 of FAVOR(=2) and AGAINST(=1); NONE(=0) is
|
| 57 |
+
predicted but excluded. This is NOT the 3-class macro-F1."""
|
| 58 |
+
from sklearn.metrics import f1_score
|
| 59 |
+
f = f1_score(y_true, y_pred, labels=[2, 1], average=None, zero_division=0)
|
| 60 |
+
return float(np.mean(f))
|
| 61 |
+
|
| 62 |
+
|
| 63 |
+
def _pearson_mean(y_true, y_pred) -> float:
|
| 64 |
+
yt, yp = np.asarray(y_true, float), np.asarray(y_pred, float)
|
| 65 |
+
if yt.ndim == 1:
|
| 66 |
+
yt, yp = yt[:, None], yp[:, None]
|
| 67 |
+
rs = []
|
| 68 |
+
for j in range(yt.shape[1]):
|
| 69 |
+
if np.std(yt[:, j]) < 1e-9 or np.std(yp[:, j]) < 1e-9:
|
| 70 |
+
rs.append(0.0)
|
| 71 |
+
else:
|
| 72 |
+
rs.append(float(np.corrcoef(yt[:, j], yp[:, j])[0, 1]))
|
| 73 |
+
return float(np.mean(rs))
|
| 74 |
+
|
| 75 |
+
|
| 76 |
+
def _mae_mean(y_true, y_pred) -> float:
|
| 77 |
+
return float(np.mean(np.abs(np.asarray(y_true, float) - np.asarray(y_pred, float))))
|
| 78 |
+
|
| 79 |
+
|
| 80 |
+
def score(metric: str, y_true, y_pred, n_classes: int = None) -> float:
|
| 81 |
+
"""Compute exactly the requested metric. Raise on an unknown metric rather than
|
| 82 |
+
silently returning a default, so a configuration error surfaces immediately."""
|
| 83 |
+
if metric == "macro_f1":
|
| 84 |
+
return _macro_f1(y_true, y_pred, n_classes)
|
| 85 |
+
if metric == "weighted_f1":
|
| 86 |
+
return _weighted_f1(y_true, y_pred)
|
| 87 |
+
if metric == "accuracy":
|
| 88 |
+
return _accuracy(y_true, y_pred)
|
| 89 |
+
if metric == "macro_f1_favor_against":
|
| 90 |
+
return _favor_against_f1(y_true, y_pred)
|
| 91 |
+
if metric == "pearson_r_mean":
|
| 92 |
+
return _pearson_mean(y_true, y_pred)
|
| 93 |
+
if metric == "mae_mean":
|
| 94 |
+
return _mae_mean(y_true, y_pred)
|
| 95 |
+
raise MetricMismatch(
|
| 96 |
+
f"unknown metric {metric!r}; refusing to fall back to a default. "
|
| 97 |
+
f"Known: {sorted(set(PRIMARY_METRIC.values()))}"
|
| 98 |
+
)
|
| 99 |
+
|
| 100 |
+
|
| 101 |
+
def constant_floor(metric: str, y_true, n_classes: int = None) -> float:
|
| 102 |
+
"""Score of a constant predictor under the same metric, so a model that has not
|
| 103 |
+
learned cannot be mistaken for one that has."""
|
| 104 |
+
yt = np.asarray(y_true)
|
| 105 |
+
if metric in ("macro_f1", "weighted_f1", "accuracy", "macro_f1_favor_against"):
|
| 106 |
+
vals, counts = np.unique(yt, return_counts=True)
|
| 107 |
+
majority = vals[int(np.argmax(counts))]
|
| 108 |
+
y_pred = np.full(len(yt), majority)
|
| 109 |
+
return score(metric, yt, y_pred, n_classes)
|
| 110 |
+
if metric == "pearson_r_mean":
|
| 111 |
+
return 0.0 # a constant prediction has zero correlation
|
| 112 |
+
if metric == "mae_mean":
|
| 113 |
+
ytf = np.asarray(y_true, float)
|
| 114 |
+
zero = _mae_mean(ytf, np.zeros_like(ytf))
|
| 115 |
+
med = _mae_mean(ytf, np.full_like(ytf, np.median(ytf)))
|
| 116 |
+
return float(min(zero, med))
|
| 117 |
+
raise MetricMismatch(f"unknown metric {metric!r}")
|
| 118 |
+
|
| 119 |
+
|
| 120 |
+
def assert_not_below_floor(metric: str, value: float, floor: float,
|
| 121 |
+
task: str = "") -> None:
|
| 122 |
+
"""Record an anomaly if a trained model does not beat the constant floor."""
|
| 123 |
+
worse = (value > floor) if metric in LOWER_BETTER else (value < floor)
|
| 124 |
+
if worse:
|
| 125 |
+
raise MetricMismatch(
|
| 126 |
+
f"{task}: {metric}={value:.4f} is not better than the constant floor "
|
| 127 |
+
f"{floor:.4f}; the model may not have learned this task."
|
| 128 |
+
)
|
| 129 |
+
|
| 130 |
+
|
| 131 |
+
def body_hash(texts: Sequence[str]) -> str:
|
| 132 |
+
"""Hash of the evaluation texts, so a later run can prove it used identical data."""
|
| 133 |
+
joined = "\n".join(str(t) for t in texts).encode("utf-8")
|
| 134 |
+
return hashlib.md5(joined).hexdigest()
|
| 135 |
+
|
| 136 |
+
|
| 137 |
+
def assert_split(texts: Sequence[str], expected_hash: str, scored_labels: List) -> Dict:
|
| 138 |
+
"""Confirm the evaluation split and the scored label set before any comparison with a
|
| 139 |
+
published value."""
|
| 140 |
+
h = body_hash(texts)
|
| 141 |
+
return {
|
| 142 |
+
"hash_matches": h == expected_hash,
|
| 143 |
+
"body_hash": h,
|
| 144 |
+
"n_examples": len(texts),
|
| 145 |
+
"scored_labels": list(scored_labels),
|
| 146 |
+
}
|
affectra/model.py
ADDED
|
@@ -0,0 +1,194 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Affectra: inference API for the Late-Read multi-task emotion foundation model.
|
| 2 |
+
|
| 3 |
+
Architecture (the adopted configuration; this mirrors the training code
|
| 4 |
+
`python/p3_bc_full.py::Qwen3LateReadModel` exactly, which is authoritative):
|
| 5 |
+
|
| 6 |
+
Base : Qwen/Qwen3-8B-Base (causal LM, frozen)
|
| 7 |
+
Adapter : LoRA r=32, alpha=64 on ALL seven linear projections
|
| 8 |
+
(q,k,v,o,gate,up,down); ~88.7M trainable in total
|
| 9 |
+
Readout : the [READ] hidden state is taken from THREE depths, layers
|
| 10 |
+
{-1, -9, -17}, and concatenated -> head input dim 3*4096 = 12288
|
| 11 |
+
[READ] : an added special token whose embedding carries a trained delta
|
| 12 |
+
Heads : one Linear(12288, n_labels) per task
|
| 13 |
+
|
| 14 |
+
Loaded artifacts (produced by scripts/convert_to_peft.py from the training checkpoint):
|
| 15 |
+
adapter/ PEFT LoRA adapter (adapter_config.json carries the 7 targets)
|
| 16 |
+
heads.pt {head_key: {"weight":..., "bias":...}} for each task
|
| 17 |
+
read_delta.pt a (hidden,) tensor added at the [READ] position
|
| 18 |
+
|
| 19 |
+
This file is desk-reviewed against the training code; it is validated end-to-end once
|
| 20 |
+
the released checkpoint is available (requires the weights and a GPU).
|
| 21 |
+
"""
|
| 22 |
+
from __future__ import annotations
|
| 23 |
+
|
| 24 |
+
from pathlib import Path
|
| 25 |
+
from typing import Dict, List, Optional, Union
|
| 26 |
+
|
| 27 |
+
from .format import build_late_read
|
| 28 |
+
from .output import EmotionOutput
|
| 29 |
+
from .task_metadata import TASKS, TaskType
|
| 30 |
+
|
| 31 |
+
READOUT_LAYERS = (-1, -9, -17)
|
| 32 |
+
HIDDEN_SIZE = 4096
|
| 33 |
+
HEAD_IN_DIM = HIDDEN_SIZE * len(READOUT_LAYERS) # 12288
|
| 34 |
+
READ_TOKEN = "[READ]"
|
| 35 |
+
BASE_MODEL = "Qwen/Qwen3-8B-Base"
|
| 36 |
+
|
| 37 |
+
|
| 38 |
+
def _head_key(task_id: str) -> str:
|
| 39 |
+
"""Head key used in the checkpoint: task_id with '-' and '.' replaced by '_'
|
| 40 |
+
(identical to training)."""
|
| 41 |
+
return task_id.replace("-", "_").replace(".", "_")
|
| 42 |
+
|
| 43 |
+
|
| 44 |
+
class Affectra:
|
| 45 |
+
"""One shared base + LoRA + per-task heads, read in the Late-Read format."""
|
| 46 |
+
|
| 47 |
+
def __init__(self, backbone, tokenizer, heads, read_delta, read_token_id,
|
| 48 |
+
device="cuda"):
|
| 49 |
+
self.backbone = backbone
|
| 50 |
+
self.tokenizer = tokenizer
|
| 51 |
+
self.heads = heads # {head_key: torch.nn.Linear}
|
| 52 |
+
self.read_delta = read_delta # torch.Tensor (hidden,) or None
|
| 53 |
+
self.read_token_id = read_token_id
|
| 54 |
+
self.device = device
|
| 55 |
+
self._public = {m.public_name: tid for tid, m in TASKS.items()}
|
| 56 |
+
|
| 57 |
+
# ------------------------------------------------------------------ load
|
| 58 |
+
@classmethod
|
| 59 |
+
def from_pretrained(cls, repo_or_path: str, device: str = "cuda"):
|
| 60 |
+
"""Assemble the model from a local dir or a Hugging Face repo.
|
| 61 |
+
|
| 62 |
+
Expects `adapter/` (PEFT), `heads.pt`, and optionally `read_delta.pt` under
|
| 63 |
+
`repo_or_path`. Downloads via huggingface_hub when the path is a repo id.
|
| 64 |
+
"""
|
| 65 |
+
import torch
|
| 66 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 67 |
+
from peft import PeftModel
|
| 68 |
+
|
| 69 |
+
root = Path(repo_or_path)
|
| 70 |
+
if not root.exists(): # treat as a hub repo id
|
| 71 |
+
from huggingface_hub import snapshot_download
|
| 72 |
+
root = Path(snapshot_download(repo_or_path))
|
| 73 |
+
|
| 74 |
+
tok = AutoTokenizer.from_pretrained(BASE_MODEL)
|
| 75 |
+
if READ_TOKEN not in tok.get_vocab():
|
| 76 |
+
tok.add_special_tokens({"additional_special_tokens": [READ_TOKEN]})
|
| 77 |
+
read_token_id = tok.convert_tokens_to_ids(READ_TOKEN)
|
| 78 |
+
|
| 79 |
+
base = AutoModelForCausalLM.from_pretrained(
|
| 80 |
+
BASE_MODEL, torch_dtype=torch.bfloat16, output_hidden_states=True)
|
| 81 |
+
base.resize_token_embeddings(len(tok))
|
| 82 |
+
backbone = PeftModel.from_pretrained(base, str(root / "adapter"),
|
| 83 |
+
is_trainable=False)
|
| 84 |
+
backbone.eval().to(device)
|
| 85 |
+
|
| 86 |
+
heads = {}
|
| 87 |
+
head_sd = torch.load(root / "heads.pt", map_location=device)
|
| 88 |
+
for hk, sd in head_sd.items():
|
| 89 |
+
lin = torch.nn.Linear(HEAD_IN_DIM, sd["weight"].shape[0])
|
| 90 |
+
lin.load_state_dict(sd)
|
| 91 |
+
heads[hk] = lin.to(device).float().eval()
|
| 92 |
+
|
| 93 |
+
read_delta = None
|
| 94 |
+
rd = root / "read_delta.pt"
|
| 95 |
+
if rd.exists():
|
| 96 |
+
read_delta = torch.load(rd, map_location=device).to(device)
|
| 97 |
+
|
| 98 |
+
return cls(backbone, tok, heads, read_delta, read_token_id, device)
|
| 99 |
+
|
| 100 |
+
# --------------------------------------------------------------- readout
|
| 101 |
+
def _readout(self, input_ids, attention_mask, read_pos):
|
| 102 |
+
"""Concatenate the [READ] hidden state across READOUT_LAYERS; add the trained
|
| 103 |
+
[READ] delta when present. Mirrors Qwen3LateReadModel.forward."""
|
| 104 |
+
import torch
|
| 105 |
+
idx = torch.arange(input_ids.shape[0], device=input_ids.device)
|
| 106 |
+
if self.read_delta is not None:
|
| 107 |
+
emb = self.backbone.get_input_embeddings()(input_ids)
|
| 108 |
+
delta = torch.zeros_like(emb)
|
| 109 |
+
delta[idx, read_pos] = self.read_delta.to(emb.dtype)
|
| 110 |
+
out = self.backbone(inputs_embeds=emb + delta,
|
| 111 |
+
attention_mask=attention_mask,
|
| 112 |
+
output_hidden_states=True)
|
| 113 |
+
else:
|
| 114 |
+
out = self.backbone(input_ids=input_ids, attention_mask=attention_mask,
|
| 115 |
+
output_hidden_states=True)
|
| 116 |
+
return torch.cat(
|
| 117 |
+
[out.hidden_states[l][idx, read_pos] for l in READOUT_LAYERS], dim=-1)
|
| 118 |
+
|
| 119 |
+
# --------------------------------------------------------------- predict
|
| 120 |
+
def predict(
|
| 121 |
+
self,
|
| 122 |
+
text: str,
|
| 123 |
+
task: Optional[str] = None,
|
| 124 |
+
labels: Optional[List[str]] = None,
|
| 125 |
+
erc_context: Optional[str] = None,
|
| 126 |
+
absa_target: Optional[str] = None,
|
| 127 |
+
) -> EmotionOutput:
|
| 128 |
+
"""Predict for one trained task.
|
| 129 |
+
|
| 130 |
+
Provide `task` (a task_id or its public_name) or `labels` (matched to a trained
|
| 131 |
+
task's label set). Open-label zero-shot for an arbitrary unseen label set is not
|
| 132 |
+
a capability of this checkpoint (future work; see the paper's RQ4).
|
| 133 |
+
"""
|
| 134 |
+
import torch
|
| 135 |
+
meta = self._resolve(task, labels)
|
| 136 |
+
hk = _head_key(meta.task_id)
|
| 137 |
+
lr = build_late_read(text, meta.label_names, erc_context, absa_target)
|
| 138 |
+
enc = self.tokenizer(lr, return_tensors="pt").to(self.device)
|
| 139 |
+
ids = enc["input_ids"]
|
| 140 |
+
read_pos = (ids[0] == self.read_token_id).nonzero(as_tuple=False)
|
| 141 |
+
if len(read_pos) == 0:
|
| 142 |
+
raise RuntimeError("the [READ] token is missing from the tokenized input")
|
| 143 |
+
rp = read_pos[-1, 0].view(1)
|
| 144 |
+
with torch.no_grad():
|
| 145 |
+
z = self._readout(ids, enc["attention_mask"], rp).float()
|
| 146 |
+
logits = self.heads[hk](z)[0]
|
| 147 |
+
return self._decode(meta, logits)
|
| 148 |
+
|
| 149 |
+
def _resolve(self, task, labels):
|
| 150 |
+
if task is not None:
|
| 151 |
+
if task in TASKS:
|
| 152 |
+
return TASKS[task]
|
| 153 |
+
if task in self._public:
|
| 154 |
+
return TASKS[self._public[task]]
|
| 155 |
+
raise KeyError(f"unknown task {task!r}; known task_ids={list(TASKS)} "
|
| 156 |
+
f"or public names={list(self._public)}")
|
| 157 |
+
if labels is not None:
|
| 158 |
+
want = {l.lower() for l in labels}
|
| 159 |
+
for m in TASKS.values():
|
| 160 |
+
if {l.lower() for l in m.label_names} == want:
|
| 161 |
+
return m
|
| 162 |
+
raise ValueError(
|
| 163 |
+
"the given label set matches no trained task. Open-label zero-shot is "
|
| 164 |
+
"future work (paper RQ4); pass task=<task_id or public_name> instead.")
|
| 165 |
+
raise ValueError("provide task=... or labels=[...]")
|
| 166 |
+
|
| 167 |
+
def _decode(self, meta, logits) -> EmotionOutput:
|
| 168 |
+
import torch
|
| 169 |
+
raw = logits.detach().cpu().tolist()
|
| 170 |
+
md = {"label_names": meta.label_names, "n_labels": len(meta.label_names),
|
| 171 |
+
"public_name": meta.public_name, "primary_metric": meta.primary_metric}
|
| 172 |
+
if meta.task_type == TaskType.MULTI_LABEL:
|
| 173 |
+
p = torch.sigmoid(logits).cpu().tolist()
|
| 174 |
+
probs = dict(zip(meta.label_names, p))
|
| 175 |
+
top = [l for l, v in probs.items() if v >= 0.5]
|
| 176 |
+
return EmotionOutput(task=meta.public_name, raw=raw, probs=probs,
|
| 177 |
+
top_labels=top, uncertainty=float(1 - max(p)),
|
| 178 |
+
metadata=md)
|
| 179 |
+
if meta.task_type == TaskType.SINGLE_LABEL:
|
| 180 |
+
p = torch.softmax(logits, dim=-1).cpu().tolist()
|
| 181 |
+
probs = dict(zip(meta.label_names, p))
|
| 182 |
+
top = [meta.label_names[int(torch.argmax(logits))]]
|
| 183 |
+
return EmotionOutput(task=meta.public_name, raw=raw, probs=probs,
|
| 184 |
+
top_labels=top, uncertainty=float(1 - max(p)),
|
| 185 |
+
metadata=md)
|
| 186 |
+
# regression (VAD / WRIME intensities / scalar intensity)
|
| 187 |
+
vals = dict(zip(meta.label_names, raw))
|
| 188 |
+
if meta.public_name == "vad":
|
| 189 |
+
return EmotionOutput(task=meta.public_name, raw=raw, vad=vals, metadata=md)
|
| 190 |
+
if len(raw) == 1:
|
| 191 |
+
return EmotionOutput(task=meta.public_name, raw=raw, scalar=raw[0],
|
| 192 |
+
metadata=md)
|
| 193 |
+
return EmotionOutput(task=meta.public_name, raw=raw, intensities=vals,
|
| 194 |
+
metadata=md)
|
affectra/output.py
ADDED
|
@@ -0,0 +1,82 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Output data structures for Affectra predictions."""
|
| 2 |
+
from __future__ import annotations
|
| 3 |
+
|
| 4 |
+
from dataclasses import dataclass, field
|
| 5 |
+
from typing import Dict, List, Optional
|
| 6 |
+
|
| 7 |
+
|
| 8 |
+
@dataclass
|
| 9 |
+
class EmotionOutput:
|
| 10 |
+
"""Standardized output from Affectra.predict().
|
| 11 |
+
|
| 12 |
+
Attributes
|
| 13 |
+
----------
|
| 14 |
+
task : str
|
| 15 |
+
Task identifier (public name or task_id).
|
| 16 |
+
raw : list of float
|
| 17 |
+
Raw model outputs before post-processing (logits for classification,
|
| 18 |
+
direct values for regression).
|
| 19 |
+
probs : dict of str -> float, optional
|
| 20 |
+
For classification tasks: sigmoid or softmax probabilities per label.
|
| 21 |
+
For regression tasks: None.
|
| 22 |
+
vad : dict of str -> float, optional
|
| 23 |
+
For the 'vad' (EmoBank) task: {"valence": ..., "arousal": ..., "dominance": ...}.
|
| 24 |
+
Values are normalized to [0, 1]. None for other tasks.
|
| 25 |
+
intensities : dict of str -> float, optional
|
| 26 |
+
For multi-dimensional regression tasks (WRIME): emotion intensity per dimension.
|
| 27 |
+
None for classification tasks.
|
| 28 |
+
scalar : float, optional
|
| 29 |
+
For single-output regression tasks (emotion_intensity): the scalar prediction.
|
| 30 |
+
None for other tasks.
|
| 31 |
+
top_labels : list of str
|
| 32 |
+
For classification tasks: labels whose probability exceeds threshold (0.5 for
|
| 33 |
+
multi-label; argmax for single-label). Empty for regression.
|
| 34 |
+
uncertainty : float
|
| 35 |
+
Calibrated uncertainty estimate in [0, 1].
|
| 36 |
+
- For multi-label tasks: 1 - max(prob), clipped to [0, 1].
|
| 37 |
+
- For single-label tasks: 1 - max(softmax_prob).
|
| 38 |
+
- For regression tasks: not applicable; set to -1.0.
|
| 39 |
+
Higher values indicate less confident predictions.
|
| 40 |
+
indeterminate : str or None
|
| 41 |
+
Indeterminate-output flag with three distinct cases:
|
| 42 |
+
- None: model has a clear, reliable prediction.
|
| 43 |
+
- "neutral_dominant": for emotion tasks, the "neutral" class has the
|
| 44 |
+
highest probability and all other emotion probabilities are below 0.3.
|
| 45 |
+
The model detects no particular emotional signal.
|
| 46 |
+
- "ambiguous": multiple labels are nearly tied (max_prob < 0.4 and
|
| 47 |
+
second_prob > 0.3 for single-label; more than half of labels exceed 0.3
|
| 48 |
+
for multi-label). Reflects genuine human disagreement per EmotionConfusers
|
| 49 |
+
findings (Inoshita et al., 2026).
|
| 50 |
+
- "out_of_scope": input text is extremely short (< 3 tokens), or the task
|
| 51 |
+
was designed for a specific domain/language and the input appears
|
| 52 |
+
out-of-distribution (heuristic; not a trained classifier).
|
| 53 |
+
metadata : dict
|
| 54 |
+
Additional structured information (label_names, n_labels, etc.).
|
| 55 |
+
"""
|
| 56 |
+
|
| 57 |
+
task: str
|
| 58 |
+
raw: List[float]
|
| 59 |
+
probs: Optional[Dict[str, float]] = None
|
| 60 |
+
vad: Optional[Dict[str, float]] = None
|
| 61 |
+
intensities: Optional[Dict[str, float]] = None
|
| 62 |
+
scalar: Optional[float] = None
|
| 63 |
+
top_labels: List[str] = field(default_factory=list)
|
| 64 |
+
uncertainty: float = -1.0
|
| 65 |
+
indeterminate: Optional[str] = None
|
| 66 |
+
metadata: Dict = field(default_factory=dict)
|
| 67 |
+
|
| 68 |
+
def __repr__(self) -> str:
|
| 69 |
+
parts = [f"task={self.task!r}"]
|
| 70 |
+
if self.probs is not None:
|
| 71 |
+
top = sorted(self.probs.items(), key=lambda x: -x[1])[:3]
|
| 72 |
+
top_str = ", ".join(f"{k}: {v:.3f}" for k, v in top)
|
| 73 |
+
parts.append(f"top_probs=[{top_str}]")
|
| 74 |
+
if self.vad is not None:
|
| 75 |
+
vad_str = ", ".join(f"{k}: {v:.3f}" for k, v in self.vad.items())
|
| 76 |
+
parts.append(f"vad={{{vad_str}}}")
|
| 77 |
+
if self.scalar is not None:
|
| 78 |
+
parts.append(f"scalar={self.scalar:.4f}")
|
| 79 |
+
parts.append(f"uncertainty={self.uncertainty:.3f}")
|
| 80 |
+
if self.indeterminate is not None:
|
| 81 |
+
parts.append(f"indeterminate={self.indeterminate!r}")
|
| 82 |
+
return f"EmotionOutput({', '.join(parts)})"
|
affectra/task_metadata.py
ADDED
|
@@ -0,0 +1,311 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Task metadata: label names, task types, output post-processing specs.
|
| 2 |
+
|
| 3 |
+
This file is the single source of truth for how each task head is interpreted
|
| 4 |
+
during inference. It is derived from python/task_configs.yaml in the training
|
| 5 |
+
repository.
|
| 6 |
+
"""
|
| 7 |
+
from __future__ import annotations
|
| 8 |
+
|
| 9 |
+
from dataclasses import dataclass, field
|
| 10 |
+
from enum import Enum
|
| 11 |
+
from typing import Dict, List, Optional
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
class TaskType(str, Enum):
|
| 15 |
+
SINGLE_LABEL = "single_label" # softmax over N classes
|
| 16 |
+
MULTI_LABEL = "multi_label" # sigmoid per class (threshold=0.5)
|
| 17 |
+
REGRESSION = "regression" # scalar output (e.g., intensity 0-1)
|
| 18 |
+
REGRESSION_MULTI = "regression_multi" # vector regression (e.g., VAD or 8-dim Plutchik)
|
| 19 |
+
STRUCTURED = "structured" # sequence generation (not exposed via this API)
|
| 20 |
+
|
| 21 |
+
|
| 22 |
+
@dataclass
|
| 23 |
+
class TaskMeta:
|
| 24 |
+
task_id: str
|
| 25 |
+
public_name: str # human-readable alias usable as `task=` argument
|
| 26 |
+
task_type: TaskType
|
| 27 |
+
label_names: List[str] # ordered list of output dimensions
|
| 28 |
+
language: str # "en", "ja", "multilingual"
|
| 29 |
+
primary_metric: str # metric used during training (for documentation)
|
| 30 |
+
notes: str = ""
|
| 31 |
+
|
| 32 |
+
|
| 33 |
+
# Canonical task registry. Key = task_id (must match checkpoint head name).
|
| 34 |
+
TASKS: Dict[str, TaskMeta] = {
|
| 35 |
+
# ------------------------------------------------------------------
|
| 36 |
+
# M01 — GoEmotions (28-class multi-label discrete emotion)
|
| 37 |
+
# ------------------------------------------------------------------
|
| 38 |
+
"M01_GoEmotions": TaskMeta(
|
| 39 |
+
task_id="M01_GoEmotions",
|
| 40 |
+
public_name="emotion28",
|
| 41 |
+
task_type=TaskType.MULTI_LABEL,
|
| 42 |
+
label_names=[
|
| 43 |
+
"admiration", "amusement", "anger", "annoyance", "approval",
|
| 44 |
+
"caring", "confusion", "curiosity", "desire", "disappointment",
|
| 45 |
+
"disapproval", "disgust", "embarrassment", "excitement", "fear",
|
| 46 |
+
"gratitude", "grief", "joy", "love", "nervousness", "optimism",
|
| 47 |
+
"pride", "realization", "relief", "remorse", "sadness", "surprise",
|
| 48 |
+
"neutral",
|
| 49 |
+
],
|
| 50 |
+
language="en",
|
| 51 |
+
primary_metric="macro_f1",
|
| 52 |
+
notes=(
|
| 53 |
+
"GoEmotions 28-class multi-label classification. "
|
| 54 |
+
"Probabilities via sigmoid; multiple labels can be active simultaneously. "
|
| 55 |
+
"Reddit-domain data; may underperform on formal text."
|
| 56 |
+
),
|
| 57 |
+
),
|
| 58 |
+
|
| 59 |
+
# ------------------------------------------------------------------
|
| 60 |
+
# M02 — WRIME v2 reader-perspective intensity (8-dim regression)
|
| 61 |
+
# ------------------------------------------------------------------
|
| 62 |
+
"M02_WRIME_v2": TaskMeta(
|
| 63 |
+
task_id="M02_WRIME_v2",
|
| 64 |
+
public_name="wrime_reader",
|
| 65 |
+
task_type=TaskType.REGRESSION_MULTI,
|
| 66 |
+
label_names=[
|
| 67 |
+
"joy", "sadness", "anticipation", "surprise",
|
| 68 |
+
"anger", "fear", "disgust", "trust",
|
| 69 |
+
],
|
| 70 |
+
language="ja",
|
| 71 |
+
primary_metric="mae",
|
| 72 |
+
notes=(
|
| 73 |
+
"WRIME v2 reader-perspective Plutchik intensity regression (0-4 scale). "
|
| 74 |
+
"Japanese SNS text domain. "
|
| 75 |
+
"Non-commercial research use only (WRIME license restriction)."
|
| 76 |
+
),
|
| 77 |
+
),
|
| 78 |
+
|
| 79 |
+
# ------------------------------------------------------------------
|
| 80 |
+
# M02_writer — WRIME v2 writer-perspective intensity
|
| 81 |
+
# ------------------------------------------------------------------
|
| 82 |
+
"M02_WRIME_v2_writer": TaskMeta(
|
| 83 |
+
task_id="M02_WRIME_v2_writer",
|
| 84 |
+
public_name="wrime_writer",
|
| 85 |
+
task_type=TaskType.REGRESSION_MULTI,
|
| 86 |
+
label_names=[
|
| 87 |
+
"joy", "sadness", "anticipation", "surprise",
|
| 88 |
+
"anger", "fear", "disgust", "trust",
|
| 89 |
+
],
|
| 90 |
+
language="ja",
|
| 91 |
+
primary_metric="mae",
|
| 92 |
+
notes=(
|
| 93 |
+
"WRIME v2 writer-perspective Plutchik intensity regression (0-4 scale). "
|
| 94 |
+
"Japanese SNS text domain. "
|
| 95 |
+
"Non-commercial research use only (WRIME license restriction)."
|
| 96 |
+
),
|
| 97 |
+
),
|
| 98 |
+
|
| 99 |
+
# ------------------------------------------------------------------
|
| 100 |
+
# M03 — SemEval-2025 Task 11 Track A (6-class multi-label)
|
| 101 |
+
# ------------------------------------------------------------------
|
| 102 |
+
"M03_SemEval2025_T11_TrackA": TaskMeta(
|
| 103 |
+
task_id="M03_SemEval2025_T11_TrackA",
|
| 104 |
+
public_name="brighter_emotion",
|
| 105 |
+
task_type=TaskType.MULTI_LABEL,
|
| 106 |
+
label_names=["anger", "disgust", "fear", "joy", "sadness", "surprise"],
|
| 107 |
+
language="multilingual",
|
| 108 |
+
primary_metric="macro_f1",
|
| 109 |
+
notes=(
|
| 110 |
+
"SemEval-2025 Task 11 Track A (BRIGHTER). 28-language multilingual "
|
| 111 |
+
"6-class emotion classification. Standard Ekman categories."
|
| 112 |
+
),
|
| 113 |
+
),
|
| 114 |
+
|
| 115 |
+
# ------------------------------------------------------------------
|
| 116 |
+
# M04 — SemEval-2025 Task 11 Track B
|
| 117 |
+
# ------------------------------------------------------------------
|
| 118 |
+
"M04_SemEval2025_T11_TrackB": TaskMeta(
|
| 119 |
+
task_id="M04_SemEval2025_T11_TrackB",
|
| 120 |
+
public_name="brighter_emotion_b",
|
| 121 |
+
task_type=TaskType.MULTI_LABEL,
|
| 122 |
+
label_names=["anger", "disgust", "fear", "joy", "sadness", "surprise"],
|
| 123 |
+
language="multilingual",
|
| 124 |
+
primary_metric="macro_f1",
|
| 125 |
+
notes=(
|
| 126 |
+
"SemEval-2025 Task 11 Track B (BRIGHTER). 11-language multilingual. "
|
| 127 |
+
"Note: Track B uses a proprietary split not matching public benchmarks."
|
| 128 |
+
),
|
| 129 |
+
),
|
| 130 |
+
|
| 131 |
+
# ------------------------------------------------------------------
|
| 132 |
+
# M05 — MELD (7-class dialogue emotion)
|
| 133 |
+
# ------------------------------------------------------------------
|
| 134 |
+
"M05_MELD": TaskMeta(
|
| 135 |
+
task_id="M05_MELD",
|
| 136 |
+
public_name="dialogue_emotion",
|
| 137 |
+
task_type=TaskType.SINGLE_LABEL,
|
| 138 |
+
label_names=["neutral", "surprise", "fear", "sadness", "joy", "disgust", "anger"],
|
| 139 |
+
language="en",
|
| 140 |
+
primary_metric="weighted_f1",
|
| 141 |
+
notes=(
|
| 142 |
+
"MELD 7-class dialogue emotion recognition. Trained on 3-utterance "
|
| 143 |
+
"context (speaker: utterance format). Single-utterance input loses context."
|
| 144 |
+
),
|
| 145 |
+
),
|
| 146 |
+
|
| 147 |
+
# ------------------------------------------------------------------
|
| 148 |
+
# M06 — TweetEval Emotion (4-class)
|
| 149 |
+
# ------------------------------------------------------------------
|
| 150 |
+
"M06_TweetEval_Emotion": TaskMeta(
|
| 151 |
+
task_id="M06_TweetEval_Emotion",
|
| 152 |
+
public_name="tweet_emotion",
|
| 153 |
+
task_type=TaskType.SINGLE_LABEL,
|
| 154 |
+
label_names=["anger", "joy", "optimism", "sadness"],
|
| 155 |
+
language="en",
|
| 156 |
+
primary_metric="macro_f1",
|
| 157 |
+
notes="TweetEval 4-class emotion. Twitter/X-domain text. Non-commercial use only.",
|
| 158 |
+
),
|
| 159 |
+
|
| 160 |
+
# ------------------------------------------------------------------
|
| 161 |
+
# M07 — SemEval-2018 Task 1 EI-reg (emotion intensity regression)
|
| 162 |
+
# ------------------------------------------------------------------
|
| 163 |
+
"M07_SemEval2018_T1_EIreg_EN": TaskMeta(
|
| 164 |
+
task_id="M07_SemEval2018_T1_EIreg_EN",
|
| 165 |
+
public_name="emotion_intensity",
|
| 166 |
+
task_type=TaskType.REGRESSION,
|
| 167 |
+
label_names=["intensity"],
|
| 168 |
+
language="en",
|
| 169 |
+
primary_metric="pearson_r",
|
| 170 |
+
notes=(
|
| 171 |
+
"SemEval-2018 Task 1 EI-reg. Scalar emotion intensity in [0, 1]. "
|
| 172 |
+
"Input should be prefixed with the emotion being measured: "
|
| 173 |
+
"e.g., 'anger: <tweet text>'. Twitter-domain."
|
| 174 |
+
),
|
| 175 |
+
),
|
| 176 |
+
|
| 177 |
+
# ------------------------------------------------------------------
|
| 178 |
+
# M08 — EmoBank VAD reader-perspective
|
| 179 |
+
# ------------------------------------------------------------------
|
| 180 |
+
"M08_EmoBank": TaskMeta(
|
| 181 |
+
task_id="M08_EmoBank",
|
| 182 |
+
public_name="vad",
|
| 183 |
+
task_type=TaskType.REGRESSION_MULTI,
|
| 184 |
+
label_names=["valence", "arousal", "dominance"],
|
| 185 |
+
language="en",
|
| 186 |
+
primary_metric="mae",
|
| 187 |
+
notes=(
|
| 188 |
+
"EmoBank VAD (reader-perspective). Predicts valence, arousal, dominance "
|
| 189 |
+
"in [1, 5] scale (raw) or normalized to [0, 1]. "
|
| 190 |
+
"CC BY-SA 4.0 licensed dataset."
|
| 191 |
+
),
|
| 192 |
+
),
|
| 193 |
+
|
| 194 |
+
# ------------------------------------------------------------------
|
| 195 |
+
# M09 — SemEval-2014 ABSA Restaurant (aspect-level sentiment)
|
| 196 |
+
# ------------------------------------------------------------------
|
| 197 |
+
"M09_SemEval2014_T4_Restaurant": TaskMeta(
|
| 198 |
+
task_id="M09_SemEval2014_T4_Restaurant",
|
| 199 |
+
public_name="absa_sentiment",
|
| 200 |
+
task_type=TaskType.SINGLE_LABEL,
|
| 201 |
+
label_names=["positive", "negative", "neutral"],
|
| 202 |
+
language="en",
|
| 203 |
+
primary_metric="macro_f1",
|
| 204 |
+
notes=(
|
| 205 |
+
"Aspect-level sentiment analysis. Input format: "
|
| 206 |
+
"'[sentence] [SEP] [aspect_term]'. Restaurant domain."
|
| 207 |
+
),
|
| 208 |
+
),
|
| 209 |
+
|
| 210 |
+
# ------------------------------------------------------------------
|
| 211 |
+
# M10 — SemEval-2018 Task 3 (irony detection)
|
| 212 |
+
# ------------------------------------------------------------------
|
| 213 |
+
"M10_SemEval2018_T3_SubtaskA": TaskMeta(
|
| 214 |
+
task_id="M10_SemEval2018_T3_SubtaskA",
|
| 215 |
+
public_name="irony",
|
| 216 |
+
task_type=TaskType.SINGLE_LABEL,
|
| 217 |
+
label_names=["non_ironic", "ironic"],
|
| 218 |
+
language="en",
|
| 219 |
+
primary_metric="macro_f1",
|
| 220 |
+
notes="Twitter irony detection (SemEval-2018 Task 3A). Twitter-domain.",
|
| 221 |
+
),
|
| 222 |
+
|
| 223 |
+
# ------------------------------------------------------------------
|
| 224 |
+
# M11 — HateXplain (hate speech detection)
|
| 225 |
+
# ------------------------------------------------------------------
|
| 226 |
+
"M11_HateXplain": TaskMeta(
|
| 227 |
+
task_id="M11_HateXplain",
|
| 228 |
+
public_name="hate_speech",
|
| 229 |
+
task_type=TaskType.SINGLE_LABEL,
|
| 230 |
+
label_names=["normal", "hatespeech", "offensive"],
|
| 231 |
+
language="en",
|
| 232 |
+
primary_metric="macro_f1",
|
| 233 |
+
notes="HateXplain 3-class hate speech detection. CC BY 4.0.",
|
| 234 |
+
),
|
| 235 |
+
|
| 236 |
+
# ------------------------------------------------------------------
|
| 237 |
+
# M12 — EmpatheticDialogues (32-class emotion context)
|
| 238 |
+
# ------------------------------------------------------------------
|
| 239 |
+
"M12_EmpatheticDialogues": TaskMeta(
|
| 240 |
+
task_id="M12_EmpatheticDialogues",
|
| 241 |
+
public_name="empathetic_emotion",
|
| 242 |
+
task_type=TaskType.SINGLE_LABEL,
|
| 243 |
+
label_names=[
|
| 244 |
+
"sentimental", "afraid", "proud", "faithful", "terrified",
|
| 245 |
+
"joyful", "angry", "sad", "jealous", "grateful", "prepared",
|
| 246 |
+
"embarrassed", "excited", "annoyed", "lonely", "surprised",
|
| 247 |
+
"furious", "disappointed", "caring", "trusting", "disgusted",
|
| 248 |
+
"anticipating", "anxious", "hopeful", "content", "impressed",
|
| 249 |
+
"apprehensive", "devastated", "guilty", "nostalgic", "confident",
|
| 250 |
+
"wistful",
|
| 251 |
+
],
|
| 252 |
+
language="en",
|
| 253 |
+
primary_metric="accuracy",
|
| 254 |
+
notes=(
|
| 255 |
+
"EmpatheticDialogues 32-class emotion-grounded utterance classification. "
|
| 256 |
+
"Performance (accuracy ~0.35-0.36) is substantially below 32-class human ceiling."
|
| 257 |
+
),
|
| 258 |
+
),
|
| 259 |
+
|
| 260 |
+
# ------------------------------------------------------------------
|
| 261 |
+
# M13 — Financial PhraseBank (financial sentiment)
|
| 262 |
+
# ------------------------------------------------------------------
|
| 263 |
+
"M13_FinancialPhraseBank": TaskMeta(
|
| 264 |
+
task_id="M13_FinancialPhraseBank",
|
| 265 |
+
public_name="financial_sentiment",
|
| 266 |
+
task_type=TaskType.SINGLE_LABEL,
|
| 267 |
+
label_names=["negative", "neutral", "positive"],
|
| 268 |
+
language="en",
|
| 269 |
+
primary_metric="macro_f1",
|
| 270 |
+
notes="Financial PhraseBank all-agree split. Financial news domain. CC BY-SA 3.0.",
|
| 271 |
+
),
|
| 272 |
+
|
| 273 |
+
# ------------------------------------------------------------------
|
| 274 |
+
# M17 — SemEval-2016 Task 6 Stance (atheism topic only)
|
| 275 |
+
# ------------------------------------------------------------------
|
| 276 |
+
"M17_SemEval2016_T6_Stance": TaskMeta(
|
| 277 |
+
task_id="M17_SemEval2016_T6_Stance",
|
| 278 |
+
public_name="stance",
|
| 279 |
+
task_type=TaskType.SINGLE_LABEL,
|
| 280 |
+
label_names=["FAVOR", "AGAINST", "NONE"],
|
| 281 |
+
language="en",
|
| 282 |
+
primary_metric="macro_f1_favor_against",
|
| 283 |
+
notes=(
|
| 284 |
+
"Stance detection. IMPORTANT: trained and evaluated on atheism topic only. "
|
| 285 |
+
"Generalization to other topics is unvalidated."
|
| 286 |
+
),
|
| 287 |
+
),
|
| 288 |
+
}
|
| 289 |
+
|
| 290 |
+
# Public-name to task_id mapping (for user convenience)
|
| 291 |
+
PUBLIC_NAME_TO_TASK_ID: Dict[str, str] = {
|
| 292 |
+
meta.public_name: tid for tid, meta in TASKS.items()
|
| 293 |
+
}
|
| 294 |
+
|
| 295 |
+
# All supported task aliases (task_id or public_name)
|
| 296 |
+
SUPPORTED_TASKS = set(TASKS.keys()) | set(PUBLIC_NAME_TO_TASK_ID.keys())
|
| 297 |
+
|
| 298 |
+
|
| 299 |
+
def resolve_task(task: str) -> TaskMeta:
|
| 300 |
+
"""Resolve a task string (task_id or public_name) to TaskMeta.
|
| 301 |
+
|
| 302 |
+
Raises ValueError for unknown tasks.
|
| 303 |
+
"""
|
| 304 |
+
if task in TASKS:
|
| 305 |
+
return TASKS[task]
|
| 306 |
+
if task in PUBLIC_NAME_TO_TASK_ID:
|
| 307 |
+
return TASKS[PUBLIC_NAME_TO_TASK_ID[task]]
|
| 308 |
+
raise ValueError(
|
| 309 |
+
f"Unknown task: {task!r}. "
|
| 310 |
+
f"Supported tasks: {sorted(SUPPORTED_TASKS)}"
|
| 311 |
+
)
|
heads.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:120fb180cd7c51c946bb3863870c1d7bc77289d2c5e15f52e27aea9ef8db55ac
|
| 3 |
+
size 5660728
|
read_delta.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:399e9ee7209975a8317423c93f0f8c97bcba81a2563fcd15be838ef331b39d4c
|
| 3 |
+
size 17579
|
setup.py
ADDED
|
@@ -0,0 +1,12 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Setup for the affectra inference package."""
|
| 2 |
+
from setuptools import setup, find_packages
|
| 3 |
+
|
| 4 |
+
setup(
|
| 5 |
+
name="affectra",
|
| 6 |
+
version="0.1.0.dev0",
|
| 7 |
+
description="Affectra: a Late-Read multi-task emotion foundation model (Qwen3-8B-Base + LoRA)",
|
| 8 |
+
packages=find_packages(),
|
| 9 |
+
python_requires=">=3.9",
|
| 10 |
+
install_requires=["torch>=2.2", "transformers>=4.51", "peft>=0.11",
|
| 11 |
+
"numpy>=1.26", "huggingface_hub>=0.23", "scikit-learn>=1.2"],
|
| 12 |
+
)
|