scholarflow-research-studio / METHODOLOGY.md
Hugdrigo's picture
Upload 10 files
03fa372 verified
|
Raw History Blame Contribute Delete
3.12 kB
# ScholarFlow Project #19: Methodology, constraints, and reproducibility
## Purpose
ScholarFlow supports **organizing** a research review, not replacing scholarly judgment. A user can catalog records, document screening decisions, inspect the included set and export a transparent review log.
## Inputs and provenance
Initial records are synthetic teaching examples, labeled `synthetic: true`; they are not articles, studies, results or verified references. Real bibliographic metadata and evidence must be collected and verified outside this tool. A user-added record has `synthetic: false`, which means only that the user created it, **not that a claim has been validated**.
## Screening decisions
The available states are `unscreened`, `include`, `maybe`, `exclude`. Their meaning should be set within the user's review protocol and recorded in free-text notes. This release intentionally does not infer screening decisions from titles or abstracts.
## Evidence matrix
Only records explicitly marked `include` appear in the evidence matrix. A five-field **metadata completeness** count tests whether the source, method, findings, limitations and participants fields have values. The synthetic source placeholder is not counted as a valid source field. This count is never a quality grade, bias rating or proof of valid findings.
## Coverage analysis
The gap explorer reports frequencies for the `region` and `language` metadata fields among included records and raises flags when fewer than three distinct methodological, regional or linguistic categories appear. A record set of fewer than ten included items is flagged as small. The threshold is an explicit heuristic for prompting review, not a measure of statistical significance or research neglect. All interpretations are restricted to the imported library.
## Storage and reproducibility
The application uses browser-local `localStorage` under `scholarflow-project-19-v1`. It makes no API calls or uploads. JSON exports contain all records and reviewer notes, an export timestamp, schema version, and a warning that demo data are synthetic. CSV exports are useful for tabular inspection. Markdown exports are useful for human-readable audit trails. The same dataset and decisions will produce the same summary counts on the same app version.
## Responsible academic use
- Verify each paper's existence, DOI, metadata and original claims against the primary source.
- Predefine question, eligibility rules and search strategy before conclusions.
- Preserve reasons for exclusion and unresolved screening disagreements.
- Assess study quality with a **suitable domain-specific validated rubric**, not the completeness count.
- Do not convert descriptive coverage counts into claims about the whole field.
- Cite publications themselves, never synthetic demo records.
- Export backups before browser resets or data imports.
## Non-goals
This version is not a PRISMA-compliant search workflow, automated systematic review, external literature search, model-powered summarizer, statistical meta-analysis, or a publication-verification service.