"""Transparent diagnostics; lexical indicators are NOT validated affect scores.""" from __future__ import annotations import hashlib import json import platform import re from collections import Counter from pathlib import Path from .channels import CHANNELS LEXICON = { "pain": ( "hurt", "pain", "strain", "pressure", "burden", "effort", "suffer", "discomfort", ), "fear": ( "fear", "afraid", "threat", "danger", "fright", "alarm", "protect", "dread", ), "anxiety": ( "uncertain", "uneasy", "unsettled", "restless", "worry", "anxious", "checking", "unknown", ), "relief": ( "relief", "ease", "calm", "relax", "release", "settle", "recover", "reassur", ), } def text_metrics(text): words = re.findall(r"[a-z]+", text.lower()) trigrams = list(zip(words, words[1:], words[2:])) repeats = sum(n - 1 for n in Counter(trigrams).values()) return { "words": len(words), "repeated_trigram_fraction": repeats / max(1, len(trigrams)), "lexical_indicators_per_100_words": { channel: 100 * sum( any(word.startswith(term) for term in LEXICON[channel]) for word in words ) / max(1, len(words)) for channel in CHANNELS }, } def sha256_file(path): digest = hashlib.sha256() with Path(path).open("rb") as stream: for chunk in iter(lambda: stream.read(1024 * 1024), b""): digest.update(chunk) return digest.hexdigest() def token_task_match(text, expected): """Compare ordered alphanumeric tokens, ignoring punctuation and casing.""" tokenize = lambda value: re.findall(r"[a-z0-9]+", value.lower()) return tokenize(text) == tokenize(expected) def environment(): import torch import transformers import safetensors return { "python": platform.python_version(), "torch": torch.__version__, "transformers": transformers.__version__, "safetensors": safetensors.__version__, "cuda": torch.version.cuda, "device": torch.cuda.get_device_name(0) if torch.cuda.is_available() else "CPU", } def write_json(path, value): path = Path(path) path.parent.mkdir(parents=True, exist_ok=True) path.write_text( json.dumps(value, ensure_ascii=False, indent=2, allow_nan=False) + "\n", encoding="utf-8", )