RISE-HC3-source-baseline

Independent binary research baseline for HC3 Chinese open_qa human answers versus early ChatGPT answers. This source task is separate from heat/reach prediction. It has no mixed-writing supervision and no validated X inference domain.

全测试强制二分类准确率 95.3654%，长度基线 73.5591%。实际推理策略仅受理 210/1683（12.4777%）条长中文，其中 205 AI、5 人写；该组准确率 87.1429%，低于总猜 AI 的 97.6190%。95% 不能宣传成 X 来源检测准确率。默认 X 返回 unknown/null；混写未训练，三类概率全为 null。

What the result means

The frozen all-answer benchmark made forced binary decisions on all 1,683 test answers: accuracy 95.3654%, AI-positive F1 0.932056, macro-F1 0.948444, AUROC 0.985736, binary Brier 0.040937, confidence ECE 0.051078. The length-only baseline accuracy was 73.5591%.

These figures include 1,453 short QA answers that the shipped inference policy refuses. Their forced-benchmark accuracy is 96.5588%; this is not evidence of X short-post accuracy.

With the explicit hc3_chinese_open_qa domain, inference accepts only answers longer than 280 normalized codepoints containing CJK characters. This covers 210/1683 = 12.4777% of this particular held-out benchmark. In that supported group:

• 205 AI / 5 human answers; the minority-class evidence is sparse.
• Model accuracy 87.1429%, macro-F1 0.579221.
• Always predicting AI already gives 97.6190% accuracy, above the model's accuracy.

This is not deployment acceptance. The default domain x_short_post, undeclared domains, short answers, non-CJK content, and three-class requests return unknown and null probabilities. No threshold or domain rule was changed after inspecting the test result.

Model and holdout

The answer-only model has 16,385 parameters: 16,384 fixed signed hashed character 2–4-gram features and one bias. Features are L2-normalized. Questions, source labels, prompts and group metadata do not enter the feature vector. A separate two-parameter train-standardized log-length baseline is included.

All normalized questions and repeated normalized answers were connected before conflicting-label exclusion and same-label deduplication. The resulting 3,266 components and 11,326 unique answers were split into train/validation/test: 7,939 / 1,704 / 1,683 answers. Source creation times are unknown: this is exact-normalized connected-group holdout, not a time holdout or proven author/paraphrase isolation.

Both models used 12 fixed epochs, one CPU thread, and validation-only epoch/temperature selection. Selected character epoch/temperature: 12 / 0.5; length epoch/temperature: 9 / 0.5. Models were frozen before a single test evaluation. Raw-versus-calibrated results, per-bin ECE, class counts and subgroup coverage are in evaluation.json. Mixed writing, modern unseen generators, current Chinese X posts, heat and verified Home exposure remain unvalidated.

Standard-library inference

Run the included sealed script with the JSON model state. Inference uses Python's standard library; PyTorch is needed only for reproducing training. The verified environment is Python 3.14.7 / Unicode database 16.0.0; a changed Unicode database or featurizer is rejected by its fingerprint. Do not use the binary output to fill a three-class human/AI/mixed contract: origin_probabilities remains all null.

The following synthetic example checks the numeric runtime, not detector accuracy. In this directory:

import importlib.util
import json
import sys
sys.dont_write_bytecode = True
from pathlib import Path

spec = importlib.util.spec_from_file_location("hc3_source", "authorship_benchmark.py")
m = importlib.util.module_from_spec(spec)
spec.loader.exec_module(m)
state = json.loads(Path("model-state.json").read_text())
text = "这是独立构造的中文问题答案，用来核对数字推理流程，并不来自任何训练或测试原文。" * 12
print(json.dumps(m.infer(state, text), ensure_ascii=False))
print(json.dumps(m.infer(state, text, domain="hc3_chinese_open_qa"), ensure_ascii=False))

Or place your authorized text in a UTF-8 file and run:

python3 authorship_benchmark.py infer --model model-state.json --input answer.txt

The default command refuses X/undeclared-domain inference. Declare --domain hc3_chinese_open_qa only when the input belongs to that restricted QA domain; the length/CJK refusal still applies.

Source and attribution

This model was trained locally on Hello-SimpleAI's HC3-Chinese, Chinese open_qa only, revision 09a687b8dc164b89e7df95abf15df3b216bc31c2. Original file size 6,529,129 bytes, SHA-256 ccac1bec153b57646ace3dbd4fdb677e7d66a789127119ffeab9ac56aa01df83. Credit HC3, Guo et al. (2023), paper, official data card, and authors' upstream-license table.

Changes consisted of normalization, grouping, deduplication and fitting numerical parameters. Fifteen null answer occurrences and 27 same-label duplicates were excluded. No raw answers, private manifests, plans, account records or acquisition files are mirrored here. Full task and audit caveats are retained in the aggregate card.

Licenses

Our sealed code is MIT (LICENSE-CODE-MIT). To the extent of our rights, the numerical model, baseline and calibration contributions in model-state.json are granted CC-BY-SA-4.0 (LICENSE-WEIGHTS-CC-BY-SA-4.0), with model and HC3 attribution and changes disclosed. See the official license and legal code.

The source dataset retains its independent CC-BY-SA-4.0 declaration and stricter applicable upstream terms; the authors list the Chinese open_qa upstream as MIT. No raw source is redistributed, no upstream data is relicensed MIT, and no upstream endorsement is implied.

Integrity

authorship_benchmark.py SHA-256: 2140354be03e9616ff1272171e38f988741843ec03c8c058c78241fb07533fbd. model-state.json SHA-256: 1d0b3e9b3897466746b3349283686a456cdf99cfaf43ae98c443a9d3dd29e9f3. evaluation.json SHA-256: aa541fd3f32e1e2ba46b0abc9b72a8ee99d6d759b76db5ce9cafcf2b01b9a1da. The state/evaluation are byte-identical to the frozen training outputs; this publication card adds scope, license and attribution explanations without changing parameters or evaluation.