Joe-Davis commited on
Commit
957cff8
·
verified ·
1 Parent(s): 1fd0c66

Refresh card: clean two-panel benchmark figure, worked example, multi-domain framing

Browse files
Files changed (1) hide show
  1. README.md +51 -5
README.md CHANGED
@@ -18,11 +18,13 @@ tags:
18
  # CFM-Methods-3B · MorphMind
19
 
20
  **A tiny control model that reads a methods section and tells you exactly where the methodology is
21
- unsound.** Give it a description of experimental or statistical methods and it returns a structured
22
- verdict --- **support** or **refute** --- pinpoints the offending statement, and explains why. It is a
23
- **high-recall screen**: it surfaces methodological red flags (data leakage, p-hacking, uncorrected
24
- multiple comparisons, train/test contamination, optional stopping, correlation-as-causation, post-hoc
25
- outlier removal, $\dots$) so a human misses almost nothing.
 
 
26
 
27
  At just **3B parameters**, CFM-Methods-3B delivers **frontier-level methodology screening** that runs
28
  on a single GPU, on-premise, at a tiny fraction of the cost of a frontier API. It is the compact member
@@ -53,6 +55,50 @@ over-flag clean methods heavily (Opus 28%, GPT-4o 47% false-positive rate). So i
53
  **frontier-grade methodology screening with the precision of a careful expert --- on-prem, in a 3B
54
  model, at a tiny fraction of the cost.**
55
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
56
  ## When & how to use it
57
  Use it as a **fast, private, first-pass methodology screen** --- a pre-submission self-check for
58
  researchers, triage for journals / reviewers / grant panels, QA over a stack of submissions, or a check
 
18
  # CFM-Methods-3B · MorphMind
19
 
20
  **A tiny control model that reads a methods section and tells you exactly where the methodology is
21
+ unsound.** Give it a methods or experimental-design block from any empirical-science paper ---
22
+ **statistics, machine learning, quantitative biology, econometrics, materials science, or chemical
23
+ physics** --- and it returns a structured verdict, **support** or **refute**, pinpoints the offending
24
+ statement, and explains why. It is a **high-recall screen**: it surfaces methodological red flags ---
25
+ data leakage, p-hacking, uncorrected multiple comparisons, train/test contamination, optional stopping,
26
+ correlation-as-causation, post-hoc outlier removal, unblinded scoring, and more --- so a human misses
27
+ almost nothing.
28
 
29
  At just **3B parameters**, CFM-Methods-3B delivers **frontier-level methodology screening** that runs
30
  on a single GPU, on-premise, at a tiny fraction of the cost of a frontier API. It is the compact member
 
55
  **frontier-grade methodology screening with the precision of a careful expert --- on-prem, in a 3B
56
  model, at a tiny fraction of the cost.**
57
 
58
+ ## Worked example
59
+
60
+ **1 — it catches an uncorrected multiple-comparisons flaw.** Given this methods block:
61
+
62
+ > *"We screened 60 candidate protein markers for association with disease status. Each marker was
63
+ > tested individually with a univariate logistic regression at the 0.05 level. The 14 markers reaching
64
+ > p < 0.05 in univariate tests are reported as significant and carried forward as the disease signature."*
65
+
66
+ CFM-Methods-3B returns (verbatim model output):
67
+
68
+ ```json
69
+ {
70
+ "analysis": "There is a methodological flaw: Many tests without correction inflate the false-positive rate.",
71
+ "verdict": "refute",
72
+ "error_spans": [
73
+ {
74
+ "text": "The 14 markers reaching p < 0.05 in univariate tests are reported as significant",
75
+ "why": "Many tests without correction inflate the false-positive rate."
76
+ }
77
+ ],
78
+ "action": "suggest_edit"
79
+ }
80
+ ```
81
+
82
+ It pinpoints the exact offending sentence and names the failure mode — 60 simultaneous tests at α = 0.05
83
+ with no correction.
84
+
85
+ **2 — and it passes clean methods without crying wolf.** Given a sound instrumental-variables design:
86
+
87
+ > *"We estimate the causal effect of schooling on wages with two-stage least squares, instrumenting
88
+ > years of education with quarter-of-birth. We report first-stage F-statistics to confirm instrument
89
+ > strength and cluster standard errors at the state level."*
90
+
91
+ ```json
92
+ {
93
+ "analysis": "The methodology is sound: instrument strength is verified and standard errors are clustered appropriately.",
94
+ "verdict": "support",
95
+ "error_spans": [],
96
+ "action": "accept"
97
+ }
98
+ ```
99
+
100
+ No false flag — the near-zero false-positive rate in the benchmark above is what this looks like in practice.
101
+
102
  ## When & how to use it
103
  Use it as a **fast, private, first-pass methodology screen** --- a pre-submission self-check for
104
  researchers, triage for journals / reviewers / grant panels, QA over a stack of submissions, or a check