leloss/LayaCrime-v0

LayaCrime.v0

LayaCrime.v0 is an open-weight, non-autoregressive model for Criminal Sentiment Analysis (CSA): deciding whether a supplied article associates a specified entity with criminal behavior or intent. It specializes the English typed-decisions checkpoint from Laya 0.3.10, built on ModernBERT-large.

The model evaluates the named entity, not the article's overall tone or every person mentioned in it. CSA is distinct from generic sentiment analysis, crime-topic detection, identity resolution, and legal judgment.

Model Details

Property	Value
Developer	Leandro A. Loss
Base checkpoint	convaiinnovations/laya, English typed decisions
Backbone	ModernBERT-large, 28 layers, hidden size 1,024
Decision head	Two-layer Transformer head and shared two-layer scorer
Parameters	421,293,827, approximately 421.3 million
Language	English
Input	Entity name, article text, question, and option criteria
Output	Binary decision and temperature-scaled option probabilities
Maximum sequence length	1,024 tokens, including the prompt
Maximum question-head budget	256 tokens
Runtime	Laya 0.3.10; this is not a standard Transformers sequence-classification export
License	Apache-2.0

Task and Labels

• negative / adverse: the article credibly associates the named entity with alleged, investigated, charged, convicted, sanctioned, or admitted criminal behavior or intent.
• positive / non-adverse: the article does not make that association, or identifies the entity only as a victim, witness, investigator, authority, or unrelated party.

Dataset labels are 1 = positive and 2 = negative. In the Laya choice request below, A = negative and B = positive; these option identifiers are not the dataset's numeric label IDs.

The exact question and criteria are stored in rl_agent_config.json under prompt. Preserve that instruction and label mapping when reproducing the reported evaluation.

Usage

Install the pinned Laya runtime and the Hugging Face download client. Install a PyTorch build appropriate for your hardware when provisioning the environment.

python -m pip install "laya @ git+https://github.com/NandhaKishorM/laya.git@v0.3.10" huggingface_hub

The example pins the verified weights revision. It uses CUDA; use device="cpu" when running without a compatible GPU, with different performance expectations.

import json
from pathlib import Path

from huggingface_hub import snapshot_download
from laya import Agent

model_dir = snapshot_download(
    repo_id="leloss/LayaCrime-v0",
    revision="c372ed8e40d1f5f8360e9bf0c05e0a5dc93366dd",
)
config = json.loads(
    (Path(model_dir) / "rl_agent_config.json").read_text(encoding="utf-8")
)
prompt = config["prompt"]
criteria_by_decision = {
    criterion["decision"]: criterion["text"]
    for criterion in prompt["criteria"]
}

entity_name = "Acme Corp"
state = {
    "entity_name": entity_name,
    "article": (
        "Authorities charged Acme Corp with falsifying invoices. "
        "The company denies the allegations."
    ),
}
questions = {
    "criminal_association": {
        "type": "choice",
        "instructions": prompt["question"].replace(
            "{entity_name}", repr(entity_name)
        ),
        "criteria": {
            "A": criteria_by_decision["negative"],
            "B": criteria_by_decision["positive"],
        },
    }
}

agent = Agent(model_dir, device="cuda")
result = agent.system_one(state, questions)
answer = result["answers"]["criminal_association"]
decision = {"A": "negative", "B": "positive"}[answer["choice"]]
probabilities = {
    "negative": answer["probabilities"]["A"],
    "positive": answer["probabilities"]["B"],
}
print({"decision": decision, "probabilities": probabilities})

Use the Laya loader so that the choice temperature in rl_agent_config.json is applied. The separate confidence returned by Laya is normalized entropy certainty, not the winning class probability. Neither quantity establishes the truth of an allegation or guarantees correctness on new data.

Inference can run offline once the model files and runtime dependencies are installed. The model performs a bounded decision in one bidirectional encoder pass rather than generating an autoregressive response.

Evaluation

The public benchmark contains 1,000 human-annotated entity/article pairs: 557 positive and 443 negative examples, covering 1,000 unique articles and 479 unique entities. Its articles are disjoint from the public tuning set.

Metric	Published checkpoint	Original Laya router
Accuracy	91.10%	63.60%
Macro precision	90.92%	69.80%
Macro recall	91.36%	66.29%
Macro F1	91.04%	62.64%
Mean end-to-end latency	63.09 ms	46.45 ms

Precision, recall, and F1 are unweighted macro averages across both classes. Macro F1 averages the two class-specific F1 scores; it is not the harmonic mean of macro precision and macro recall. Machine-readable metric values above use the 0-to-1 scale.

Latency was measured in a sequential run on an NVIDIA Tesla T4, including local dispatch through prediction extraction, with float16 autocast. These measurements describe the tested configuration, not hardware-independent speed or concurrent production throughput.

Selection caveat: the published checkpoint was selected using benchmark performance. The benchmark is therefore not an untouched blind test for this selected model. These results are descriptive and selection-conditioned; use a separate blind corpus to estimate generalization after freezing the model, instruction, and operating thresholds.

Training and Checkpoint Provenance

Fine-tuning used the 2,000-example public tuning set, balanced between 1,000 positive and 1,000 negative labels, covering 2,000 unique articles and 610 unique entities. The labels were recovered from historical human annotations rather than collected in a new annotation exercise for this model.

Setting	Published checkpoint
Strategy	adaptation_encoder_lr_2e5
Run ID	group_k_fold-5-3__adaptation_encoder_lr_2e5__config-e4d82f70cd__seed-20260923
Seed	20260923
Partition	Grouped five-fold partition; calibration fold ID 3, with fold IDs 0-4
Training / calibration examples	1,549 / 451
Training / calibration entities	474 / 136
Encoder / head learning rate	2e-5 / 5e-5
Weight decay	0.01
Effective batch size	32, from micro-batch 4 and accumulation 8 on one GPU
Objective	Inverse-frequency-weighted cross-entropy over option logits
Schedule	AdamW, cosine decay, no linear warm-up, gradient clipping at norm 1
Completed epochs / selected epoch	4 / 2
Selected validation NLL	0.24818324531013447
Selected validation accuracy	91.1308%
Choice temperature	1.3232303857803345
Total training time	2,085.7 seconds, approximately 34.8 minutes on one Tesla T4

Within the run, minimum validation loss selected epoch 2. Choice-temperature scaling was fitted on the same held-out calibration fold. Training used deterministic option-order reversal for half of the examples, with remapped targets; validation and inference retained the fixed negative/positive option order.

The export includes training_report.json, data_manifest.json, and rl_agent_config.json for inspection of the training history, split provenance, instruction, and calibration. The inherited architecture is unchanged.

Intended Use and Limitations

The intended use is human-reviewed adverse-media triage after candidate retrieval and entity resolution. A qualified reviewer should verify the entity match, inspect the source and full article, distinguish allegations from adjudicated facts, consider recency and corrections, and apply the organization's policy before action.

The model does not retrieve evidence, verify identity, establish guilt or intent, determine whether an offense occurred, or verify the truth of the reporting. Its output must not be used as an autonomous basis for denying services, employment decisions, or other consequential actions. It should not replace corroboration, due process, or qualified human judgment.

• The evidence covers curated English-language examples and one binary screening instruction, not multilingual or representative live-stream performance.
• Historical labels can reflect annotator inconsistency, source-workflow bias, and uneven coverage. The dataset does not establish inter-annotator agreement or population representativeness.
• False adverse associations can wrongly stigmatize entities and increase review burden; missed adverse associations can suppress relevant evidence from review.
• The 1,024-token limit includes the prompt, so evidence late in a long article may be truncated.
• Ambiguous names, difficult aliases, coreference, outdated reporting, domain shift, and rare entity roles require additional evaluation.
• Operators should validate on independent data, calibrate review thresholds for their workflow, monitor drift and error rates, and maintain access, retention, audit, correction, and escalation controls.

Reproducibility and License

The pinned weights revision used in the example is c372ed8e40d1f5f8360e9bf0c05e0a5dc93366dd.

model.safetensors has 842,609,220 bytes and SHA-256:

2a6b81b6407ef2671a22f60a83d93e07ef06d9f4f5aab8e3ab4efe6993634152

The model is distributed under Apache-2.0. Users remain responsible for applicable legal, privacy, security, and governance requirements in their deployment context.