Instructions to use Shuakami/douban-sentiment-weights with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Shuakami/douban-sentiment-weights with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="Shuakami/douban-sentiment-weights")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Shuakami/douban-sentiment-weights", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Douban Sentiment Classification at Scale — Deployed Weights
中文短评三分类(正面 / 中性 / 负面)情感分析的部署权重:三个 MacBERT-base 学生模型 + 三个基于冻结 MegatronBERT-1.3B 特征的 MLP 头,组成论文中的混合集成系统。
This repository hosts the deployed v3 ensemble weights from the project douban-sentiment. The system performs three-class sentiment classification (positive / neutral / negative) and binary classification on terse Chinese Douban movie reviews, and reaches 89.5% accuracy / 85.9 macro-F1 (3-class) and 95.9% / 95.6 (binary) on a 600-row human-verified gold set under stratified 5-fold out-of-fold evaluation.
Model description
The deployed system is a hybrid ensemble with three channels:
| Component | Path | Role |
|---|---|---|
| 3-class student | macbert_3c/ |
chinese-macbert-base, distilled from a 7-channel teacher ensemble (hard-label CE + soft-target KL, α=0.5, τ=2, 8 epochs) |
| Binary student | macbert_bin/ |
same recipe, binary task |
| Neutral specialist | macbert_neu/ |
MacBERT binary detector trained on 115K corpus-scale pseudo-labels (zero gold labels) — separates "genuinely neutral" from polarised reviews |
| 3-class judge head | heads/mlp_3c_mb13b_m256.pt |
MLP on frozen Erlangshen-MegatronBERT-1.3B features |
| Binary judge head | heads/mlp_bin_mb13b_m256.pt |
same backbone, binary head |
| Neutral judge head | heads/mlp_neu_mb13b_m256_h256.pt |
same backbone, neutral-detection head (hidden=256) |
The judge heads consume 4096-dimensional features: the
concatenation of the CLS token and the attention-masked mean of
last_hidden_state from
IDEA-CCNL/Erlangshen-MegatronBert-1.3B-Sentiment
(external dependency, not included in this repo — download it
separately). Each .pt file stores state_dict, in_dim,
n_classes, hidden, scaler_mean, scaler_scale; standardise
features with the stored scaler before the MLP.
Ensemble recipe (as deployed)
dep_3c = (p_student_3c + 2 · p_judge_3c) / 3 # student:judge = 1:2
dep_bin = (p_student_bin + 2 · p_judge_bin) / 3
p_n = 0.4 · P_neu(macbert_neu) + 0.6 · P_neu(mlp_neu head)
r_± = d_± / (d_+ + d_-) from dep_3c # renormalised pos/neg
spec = [(1−p_n)·r_+, p_n, (1−p_n)·r_-]
final_3c = (dep_3c + 0.5 · spec) / 1.5 # then temperature T=0.37
Intended uses & limitations
Intended for: sentiment analysis of short Chinese movie-review text (Douban-style, typically < 200 characters), research and demonstration. The neutral class targets mixed-polarity reviews (e.g. “特效无敌,但是剧情简直像一坨”).
Limitations:
- Trained and evaluated on Douban movie reviews; performance on other domains, platforms, or review lengths is unverified.
- The neutral/mixed class remains the hardest: neutral recall is 70.7% on the gold set and degrades on very terse corpus-style comments (honest OOD study in the paper reports ~60% three-class accuracy on fresh wild reviews vs. LLM reference labels).
- Not a substitute for human judgement on high-stakes decisions.
How to use
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification
REPO = "Shuakami/douban-sentiment-weights"
# --- MacBERT students (standard sequence-classification checkpoints) ---
stu3_tok = AutoTokenizer.from_pretrained(f"{REPO}/macbert_3c")
stu3 = AutoModelForSequenceClassification.from_pretrained(f"{REPO}/macbert_3c")
# --- MLP judge heads ---
# 1) load backbone (external):
# bb_id = "IDEA-CCNL/Erlangshen-MegatronBert-1.3B-Sentiment"
# 2) features = cat([last_hidden_state[:,0], masked_mean(last_hidden_state)]) # 4096-d
# 3) blob = torch.load("heads/mlp_3c_mb13b_m256.pt")
# x = (feats - blob["scaler_mean"]) / blob["scaler_scale"]
# net = MLP(in_dim=4096, hidden=blob["hidden"], n_classes=blob["n_classes"])
# net.load_state_dict(blob["state_dict"])
A runnable reference implementation (FastAPI demo backend) is in
web/app.py of the
project repository, and a live demo is hosted at
https://huggingface.co/spaces/Shuakami/douban-sentiment.
Training data
- Gold set: 600 Douban reviews with human-verified semantic
labels (
label_semantic), single-film-page sample; class distribution 316 positive / 92 neutral / 192 negative. - Corpus: Moviedata-10M (csuldw/AntSpider), 910,998 cleaned
Douban comments with star ratings only. Pseudo-labels were produced
by the deployed ensemble; the confident pool (max p ≥ 0.7) contains
271,905 rows used to train
macbert_neu. - lab2000: a 2,000-row cross-movie benchmark with two-pass LLM blind annotation and 479 human-adjudicated rows, released in the project repository for OOD/active-learning evaluation.
Training procedure
- Students: knowledge distillation from a 7-channel teacher ensemble (TF-IDF LR + frozen-backbone MLP heads), hard-label CE + soft-target KL (α=0.5, τ=2), 8 epochs, batch 32, lr 2e-5, bf16, max_len 256, stratified 5-fold OOF for evaluation; production weights trained on 100% of the gold set.
macbert_neu: binary MacBERT classifier on 115,000 pseudo-labelled corpus rows (per-class cap 60K/side), 2 epochs — no gold labels.- Judge heads: MLP (4096→512→256→C, GELU, dropout 0.2) on frozen MegatronBERT-1.3B features, 300 epochs, trained on gold-600.
Evaluation results
Gold-600, stratified 5-fold out-of-fold (every prediction on unseen rows):
| System | 3-class acc / macro-F1 | Binary acc / macro-F1 |
|---|---|---|
| TF-IDF + LR baseline | 0.723 / 0.619 | 0.841 / 0.823 |
| Qwen3-4B zero-shot | 0.783 | — |
| Corpus student (zero gold training) | 0.875 / 0.846 | 0.949 / 0.946 |
| Deployed v3 ensemble (this repo) | 0.895 / 0.859 | 0.959 / 0.956 |
Out-of-distribution (lab2000, cross-movie wild reviews): deployed v3 reaches 65.0% three-class / 85.0% binary; adding 1,000 actively-selected boundary examples lifts the SFT student to 71.4% (+8.8pp, McNemar p = 4.0×10⁻⁸); scaling to ~17K frontier-LLM pseudo-labels reaches 77.1% three-class / 92.3% binary (details in the paper).
Limitations and bias
- Neutral-class recall is the weakest point (70.7% gold OOF); models tend to over-predict polarised classes on ambiguous text.
- Star ratings correlate imperfectly with semantic sentiment (measured corpus-wide agreement ≈48% three-class); do not use ratings as labels.
- LLM-generated pseudo-labels carry teacher biases; the paper documents where they help (scale) and where they cannot (beyond-teacher knowledge).
Citation
@misc{shuakami2026douban,
author = {Shuakami},
title = {Weakly-Supervised Chinese Review Sentiment
Classification at Scale: Pseudo-Labeling,
Specialist Decomposition, and Honest
Out-of-Distribution Evaluation},
year = {2026},
howpublished = {\url{https://github.com/shuakami/douban-sentiment}},
note = {Undergraduate thesis; bilingual PDF in papers/}
}
Please cite the project repository and the papers in papers/ if you
use these weights.
License & data note
Code and weights are provided for research / coursework use only (no formal OSS license granted). The external Douban corpus (Moviedata-10M) is research-use only with attribution required and is not redistributed in this repository; the 600 gold labels are hand-annotated for this project. See the project repository for full data and provenance documentation.
Model tree for Shuakami/douban-sentiment-weights
Base model
hfl/chinese-macbert-base