ishwarbb23/hw1-hc3-detector

HC3 human / ChatGPT answer classifier

Fine-tuned from sentence-transformers/all-MiniLM-L6-v2 on the English HC3 dataset, revision 4d0ff18143b5a7e1b1e79beb540c04549d1e59d3. Labels: 0 = human, 1 = ChatGPT.

Question-disjoint, balanced 80/10/10 train/validation/test splits, seed 42. Only answer text is used. Training uses AdamW, learning rate 2e-05, 5 epochs, batch size 32, and maximum sequence length 256.

Measured results

• Frozen SentenceTransformer + logistic regression test accuracy: 0.844901
• Fine-tuned classifier test accuracy: 0.991859
• Baseline confusion matrix (rows=true, columns=predicted): [[1946, 388], [336, 1998]]
• Fine-tuned confusion matrix: [[2297, 37], [1, 2333]]

Limitations

This is a historical benchmark experiment. Results may depend on domain, answer style, and the generator used to build HC3. This model has not been validated for current student work or newer generators and should not be used as sole evidence of authorship or misconduct.

Dataset reference: Guo et al. (2023), How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection, arXiv:2301.07597.