Commit History
-
Add measured limitations from an independent MEDLINE-ground-truth evaluation
Verified signatureField evaluation against NCBI MEDLINE publication types (1,857 sampled + 4,479 whole-population records, biomedical corpus). Adds what a user filtering a training set needs to know: * PER-CLASS COMPOSITION: only literature_review identifies reviews. survey_study is 98.4% primary research, tool_paper 95.4%, status_report 77.9%. status_report is a clinical case-report/trial class (Case Reports 132 vs Review 96 in its MEDLINE composition), not a review class. * OPERATING POINT: p(literature_review) alone gives 87.5% precision and loses 0.14 research papers per review caught; p_lit+p_status gives 48.5% and loses 1.13:1. Higher AUC, worse at threshold. * ERA BIAS: pre-1990 papers flagged at ~8.7x their true non-research rate with an 89.4% false positive rate, driven by abstract length. Apply a year guard. * VENUE BIAS: CMAJ 62.5% flagged vs 6.2% true, BMJ 42.1% vs 0.0%; specialist molecular journals are calibrated. * INPUT: classify abstracts, not full text (paired design: 179 pa…0a3ee40 -
Fix SetFitModel.from_pretrained: add _name_or_path to config.json
Verified signaturesetfit/model_card.py::infer_st_id() reads config['_name_or_path'] and calls Path() on it. The key was absent, so it received None and raised: TypeError: argument should be a str or an os.PathLike object ... not 'NoneType' which made from_pretrained() unusable on every snapshot of this repo. Value set to BAAI/bge-base-en-v1.5, the base already declared in the model card and confirmed by fingerprint (vocab.txt byte-identical to bge, 10/10 architecture config fields identical). Verified the change is behaviourally inert: embeddings are bit-identical to the previous manual load path (max abs diff 0.000000) and predictions are unchanged. README: added a Usage section covering the direct two-part load for anyone pinned to an older revision (the head needs joblib, not pickle -- plain pickle.load fails with 'invalid load key'), the sklearn 1.6.1 pickle-version warning, and the finding that single-sentence inputs are unreliable and collapse to status_report, so callers should pass a full ab…97e0474 -
Fix SetFitModel.from_pretrained: add _name_or_path to config.json
Verified signaturesetfit/model_card.py::infer_st_id() reads config['_name_or_path'] and calls Path() on it. The key was absent, so it received None and raised: TypeError: argument should be a str or an os.PathLike object ... not 'NoneType' which made from_pretrained() unusable on every snapshot of this repo. Value set to BAAI/bge-base-en-v1.5, the base already declared in the model card and confirmed by fingerprint (vocab.txt byte-identical to bge, 10/10 architecture config fields identical). Verified the change is behaviourally inert: embeddings are bit-identical to the previous manual load path (max abs diff 0.000000) and predictions are unchanged. README: added a Usage section covering the direct two-part load for anyone pinned to an older revision (the head needs joblib, not pickle -- plain pickle.load fails with 'invalid load key'), the sklearn 1.6.1 pickle-version warning, and the finding that single-sentence inputs are unreliable and collapse to status_report, so callers should pass a full ab…9ee10e9
-
V2: 5-class classifier (research_paper, literature_review, survey_study, tool_paper, status_report). 78.9% accuracy on 9K test set. Trained on 50K samples.
Verified signature2aa5993
-
Initial release: SetFit review classifier (87.3% accuracy)
Verified signaturea164f7a -
initial commit
Verified signature6cef521