scholarflow-research-studio / METHODOLOGY.md
Hugdrigo's picture
Upload 10 files
03fa372 verified
|
Raw History Blame Contribute Delete
3.12 kB

ScholarFlow Project #19: Methodology, constraints, and reproducibility

Purpose

ScholarFlow supports organizing a research review, not replacing scholarly judgment. A user can catalog records, document screening decisions, inspect the included set and export a transparent review log.

Inputs and provenance

Initial records are synthetic teaching examples, labeled synthetic: true; they are not articles, studies, results or verified references. Real bibliographic metadata and evidence must be collected and verified outside this tool. A user-added record has synthetic: false, which means only that the user created it, not that a claim has been validated.

Screening decisions

The available states are unscreened, include, maybe, exclude. Their meaning should be set within the user's review protocol and recorded in free-text notes. This release intentionally does not infer screening decisions from titles or abstracts.

Evidence matrix

Only records explicitly marked include appear in the evidence matrix. A five-field metadata completeness count tests whether the source, method, findings, limitations and participants fields have values. The synthetic source placeholder is not counted as a valid source field. This count is never a quality grade, bias rating or proof of valid findings.

Coverage analysis

The gap explorer reports frequencies for the region and language metadata fields among included records and raises flags when fewer than three distinct methodological, regional or linguistic categories appear. A record set of fewer than ten included items is flagged as small. The threshold is an explicit heuristic for prompting review, not a measure of statistical significance or research neglect. All interpretations are restricted to the imported library.

Storage and reproducibility

The application uses browser-local localStorage under scholarflow-project-19-v1. It makes no API calls or uploads. JSON exports contain all records and reviewer notes, an export timestamp, schema version, and a warning that demo data are synthetic. CSV exports are useful for tabular inspection. Markdown exports are useful for human-readable audit trails. The same dataset and decisions will produce the same summary counts on the same app version.

Responsible academic use

  • Verify each paper's existence, DOI, metadata and original claims against the primary source.
  • Predefine question, eligibility rules and search strategy before conclusions.
  • Preserve reasons for exclusion and unresolved screening disagreements.
  • Assess study quality with a suitable domain-specific validated rubric, not the completeness count.
  • Do not convert descriptive coverage counts into claims about the whole field.
  • Cite publications themselves, never synthetic demo records.
  • Export backups before browser resets or data imports.

Non-goals

This version is not a PRISMA-compliant search workflow, automated systematic review, external literature search, model-powered summarizer, statistical meta-analysis, or a publication-verification service.