Rely on a version-controlled repository to organize and trace review data (Curate)
The principle in action
As an illustrative example, consider this version-controlled literature review repository:
What to explore:
- Commit history: Navigate to the Commits view to see how bibliographic records, screening decisions, and extracted evidence evolve over time.
- File-level diffs: Open individual commits to inspect precise changes to metadata (e.g., corrections, deduplication decisions, or extraction of evidence).
- Reproducibility: Possibility to fork or clone the repository locally and rerun processing steps to reproduce intermediate and final outcomes.
- Auditability: Commit messages and pull requests trace who changed what, when, and why of each change in the data.
Why it matters
- One shared place for the evidence base: A single repository helps teams coordinate on the same (meta)data, protocols, and outputs.
- Transparent change history: Every modification is recorded (who changed what, when, and why), enabling traceability of the evolving evidence base.
- Stronger control over data quality: Version control supports internal validation (e.g., through pull requests), training of reviewers based on standardized protocols, and rollback of problematic changes.
Practical implementation
- Maintain a remote repository (e.g., GitHub) for collaboration and backup, and local clones for day-to-day work and computation.
- Use distributed version control to leverage local computing resources (often cheaper and faster for researchers) for tasks like API querying, deduplication, and compute-intensive methods (ML/LLM) on full texts—while keeping results reproducible and auditable.
- Keep repositories private by default during active research, but make them publicly accessible when appropriate. When making a repository public, consider sharing metadata (e.g., primary records and extracted evidence) rather than raw search results or full-text PDFs, which may be subject to licensing restrictions. In private repositories, both metadata and raw materials (including PDFs) can be maintained; large binary files such as PDFs can be managed using Git Large File Storage (Git LFS) to keep the repository lightweight while preserving reproducibility and auditability.
Resources