Python 3.10+ GPL-3.0 NFDI JupyterHub Field-Agnostic

Score Any GitHub Repository for
Notebook Reproducibility

An automated pipeline that clones any GitHub repository, executes its Jupyter notebooks in isolated environments, and scores reproducibility using the ReproScore framework (RRS, ROS, RCS).

View Dashboard View on GitHub
โ€”
Repositories Processed
โ€”
Notebooks Executed
โ€”
Successful Runs
โ€”
Avg RRS Score

How the Pipeline Works

Automated workflow from repository input to reproducibility scoring

STEP 01

Input Repository

Provide any public GitHub repository URL โ€” from any research field.

STEP 02

Clone Repo

Clone the repository with shallow depth for efficiency.

STEP 03

Score (RRS)

Static analysis across 5 categories: Environment, Data, Documentation, Portability, Signals.

STEP 04

Detect Python

Detect required Python version from runtime.txt, .python-version, or setup files.

STEP 05

Isolate Environment

Create a per-repo pyenv + venv with all declared dependencies installed.

STEP 06

Execute Notebooks

Run all notebooks via nbconvert, capturing outputs and errors per cell.

STEP 07

Compare Outputs

Diff original vs re-executed notebook outputs at the cell level.

STEP 08

Score (ROS/RCS)

Compute Reproducibility Outcome Score and Composite Score from execution evidence.


Results Dashboard

Live data from the latest pipeline runs โ€” updated on each push

Loading scores...
Loading executions...
Loading metrics...

RRS / ROS / RCS by Repository

RRS Category Breakdown

Reproducibility Score per Notebook

Execution Status Distribution


Running the Pipeline

Works on any machine with Python 3.10+ and pyenv

# 1. Clone the repo
git clone https://github.com/VasundharaShaw/ReproScore-Any
cd ReproScore-Any

# 2. (Optional) Set GitHub token for higher API rate limits
export GITHUB_API_TOKEN=your_token_here

# 3. Check environment
python3 -m pipeline setup

# 4. Score a repository interactively
python3 -m pipeline run --interactive

# 5. Or score multiple repos from the DB
python3 -m pipeline run --count 5

Try It Live

Score any GitHub repository directly in your browser


Analysis Notebook

Explore reproducibility results across all scored repositories

▶ Open in Google Colab 📓 View on GitHub

The notebook covers score distributions, execution results, reproducibility metrics, and error analysis. To run with your own data, point it at your local output/db/db.sqlite.