Contributing guide

Mosaic grows as people add solvers, tune the ones it has, and extend it to new physics. This page explains how to set up a development environment, which path fits your contribution, and what happens after you open a pull request. We ask everyone who takes part to follow our code of conduct.

Setup

The requirements (Linux, Docker, and an NVIDIA GPU for full coverage) are listed in Run the benchmark, along with the install steps. Mosaic needs Python 3.11 or later. For development, install the dev extras and the pre-commit hooks:

git clone git@github.com:pasteurlabs/mosaic.git
cd mosaic
uv sync --extra dev
uv run pre-commit install

With pip, install the shared schema package first, since mosaic-shared is not on PyPI:

python -m venv .venv
. .venv/bin/activate
pip install -e mosaic/mosaic_shared
pip install -e ".[dev]"
pre-commit install

The hooks run on every commit. They lint and format Python with ruff, format JSON, YAML, Markdown and TOML with prettier, and run a few file checks. Run the tests with uv run pytest (or pytest in the pip environment).

Contribution paths

Add a solver

A solver wraps one library as a Tesseract for an existing domain. It lives in its own directory under mosaic/tesseracts/<domain>/, implements the domain’s canonical schema, and describes itself in the metadata.mosaic block of its tesseract_config.yaml, including the intervention record. Add a solver walks through every step, from the directory layout to the first benchmark run.

Add a problem

A problem is the benchmark package for one kind of physics, selected with mosaic run -p. Adding one means writing the canonical schema its solvers share (mosaic/mosaic_shared/problems/<domain>/schemas.py), a problem package under mosaic/benchmarks/problems/ that registers the experiments, the setups that results are reported for, and at least one solver. mosaic new-domain <name> --from-template <template> scaffolds the files from a built-in template (mosaic templates lists them), and mosaic validate-domain <problem> checks the problem package. Add a problem is the full walkthrough.

Tune an existing solver

Tuning is a normal part of using a solver, and Mosaic records it so that readers can see what each result took. If you think a solver can do better with different settings, open a pull request with the change and record it in the solver’s intervention record.

You can change a solver’s behavior in three places:

Solver-internal parameters in its tesseract_api.py, such as sub-stepping counts, relaxation factors, convergence tolerances or the time integration order.
Input overrides in the problem package (mosaic/benchmarks/problems/<package>/config.py, via _SOLVERS["<solver>"].input_overrides), which set solver-specific values for input fields, such as inner_steps for JAX-CFD on ns-3d-grid.
Schema defaults in the solver’s InputSchema subclass in tesseract_api.py.

Record each change in the interventions list under metadata.mosaic. Set tuned: true when you picked a value by measuring alternatives, such as a tolerance, linear solver, preconditioner or device, and link the measurement under evidence. If a change makes the solver run a substitute for the prescribed setup, record it under deviations instead. The intervention record points to the format, and the test suite checks each record.

Before you ask for CI time, check that the solver still runs with a quick forward pass at reduced problem size:

mosaic run -p <problem> --suites forward -s "<Display Name>" --debug

-s takes the solver’s display name (metadata.mosaic.name), matched case-insensitively. CI then runs the full comparison. Once a maintainer applies the benchmark:solver label, it re-runs the changed solver, and the PR benchmark comment shows how its results moved against the main baseline. Reviewers look for improvements without regressions on the other setups the solver runs, and for an intervention record that matches the change.

Change the harness

The harness is the code that runs experiments and reports results, mainly the CLI in mosaic/benchmarks/cli/ and the runner and status pipeline in mosaic/benchmarks/core/. Add or update tests under tests/ alongside the change. Since a harness change can alter results for every solver, the maintainer reviewing it picks the benchmark label accordingly, usually benchmark:all, or benchmark:none if the change can’t affect results.

Pull request workflow

Open the pull request against main and fill in the template. CI starts right away, while benchmarks wait until a maintainer applies a benchmark label, which also works for pull requests from forks.

Before we can merge your first pull request, we ask you to sign our Contributor License Agreement. A bot explains how in a comment on the pull request, and you only need to do it once.

CI checks

Every pull request runs these checks:

Workflow	Check
`ci.yml`	Lint with the pre-commit hooks, tests on Python 3.12 and 3.14 (including the format check of every intervention record and a validation of every problem config), and a validation of every `tesseract_config.yaml`. The merge gate is “All checks passed”.
`conventional-pr-linter.yml`	The pull request title follows the conventions.
`cla.yml`	Every contributor has signed the CLA.
`benchmark.yml`	Runs benchmarks according to the label and renders a docs preview. The merge gate is “Benchmark checks passed”.

Benchmark labels

Benchmarks need GPU runners and take a long time, so they run on demand. A pull request that changes anything under mosaic/ can’t merge until a maintainer applies exactly one of these labels:

Label	What runs
`benchmark:none`	No benchmarks. For changes that can’t alter results.
`benchmark:solver`	The solvers whose directories under `mosaic/tesseracts/` changed, on the problems that contain them. Any number of solvers may change. Other changes in the same pull request, including harness changes, don’t widen the run.
`benchmark:domain`	Every solver in the domain whose problem definition changed, on every problem of that domain. The domain is derived from changed files under `mosaic/benchmarks/problems/<package>/`, so changes to solver code don’t trigger it. Use it when a change to a problem alters the reference that the whole domain is measured against. It covers a single domain, and a pull request that changes problems in more than one domain needs `benchmark:all`.
`benchmark:all`	Every suite on every problem and solver.

More than one benchmark label fails the run. Release pull requests always run the full benchmark and need no label. The exact rules are in the plan job of benchmark.yml.

With benchmark:none, CI still renders the docs, and a failed render blocks the merge. Every pull request gets a docs preview on Read the Docs, populated with the pull request’s results overlaid on the baseline, so you can check how your change looks on the results pages.

The PR benchmark comment

CI posts a single comment on the pull request and updates it on every run. It links to the docs preview and summarizes the run against the published main baseline (a release pull request compares against the previous release instead). The summary lists, in order:

Regressions and improvements, meaning cells whose status changed, for example a solver that went from ok to fail or back.
Metric changes, meaning solvers that stayed ok while a metric such as gradient error, converged loss or forward error moved past a relative threshold. The status alone would hide these, so a change that improves accuracy without changing the status still shows up.
Timing, in a separate section marked as indicative. The benchmark jobs run on GitHub-hosted runners whose hardware and load can differ between runs, so wall-clock times vary. A large move in a consistent direction is more meaningful than the exact percentage.
Resource-frontier shifts, meaning changes in the sweep size at which a solver first runs out of memory or time. A ceiling that comes at a smaller size than in the baseline is a scaling regression, and one that moves further out is an improvement. In limit sweeps such as horizon_sweep_limits, which exist to find where a solver stops working, a failure after a successful point marks the frontier, and only a failure at the first point fails the cell.

A coverage banner at the top states what ran. Under benchmark:solver, only the changed solvers re-run on the problems that contain them, so “no regressions” means no regressions among what was measured, and the other cells come from the baseline unchanged. A pull request that runs no benchmarks still gets a one-line verdict, so that an empty comment isn’t mistaken for a clean pass.

Note

Frontier tracking depends on the solver container reporting a failure_type of OOM or timeout. A container that dies from memory pressure but reports a generic error counts as a crash, which is a hard fail. A mislabeled out-of-memory error therefore shows up as a louder regression, which errs on the safe side, but it also means that the frontier signal is best-effort and may miss some out-of-memory failures. When a container dies, the failure record and the report show the exit code that Docker recorded and the last lines of the container’s log, which are the place to look for the cause.

Conventions

Code style

Python code targets 3.11 and later and is formatted and linted with ruff (ruff check --fix && ruff format). The pre-commit hooks enforce this on every commit, and the configuration lives in pyproject.toml. If you change dependencies in pyproject.toml, the hooks also update production.uv.lock and requirements.txt.

Commit messages and pull request titles

Pull requests are squash-merged, and the commit on main takes its title and message from the pull request. Titles therefore follow Conventional Commits, in the form type(scope): description with an optional scope. The PR linter checks the title, and the changelog is generated from these commits.

The types are:

chore for the build system, external dependencies and general housekeeping.
ci for changes to CI.
doc for documentation-only changes.
feat for a new feature.
fix for a bug fix.
perf for a change that improves performance.
refactor for a change that neither adds a feature nor fixes a bug.
security for a fix to a security issue.
test for adding or fixing tests.

The linter accepts the scopes harness (the benchmark harness and CLI), solver (solvers), domain (domains and problems) and deps (dependencies), as listed in conventional-pr-linter.yml. A pull request that needs a new scope adds it there.

Mark a breaking change with an exclamation mark after the type or scope (feat(harness)!: ...), or with a BREAKING CHANGE: trailer in the message.

Building the docs

The site is built with Quarto from the .qmd files under docs/, configured in docs/_quarto.yml. Quarto is a standalone binary, not a Python package. CI uses Quarto 1.7.31, and using the same version locally avoids rendering differences. Download it from the Quarto releases, for example on Linux x86-64:

wget https://github.com/quarto-dev/quarto-cli/releases/download/v1.7.31/quarto-1.7.31-linux-amd64.deb
sudo dpkg -i quarto-1.7.31-linux-amd64.deb
quarto --version

docs/Makefile wraps the commands, running them through uv:

cd docs
make html      # full site in docs/_site/
make preview   # live-reloading preview
make clean     # remove docs/_build/ and docs/_site/

Quarto writes its cache and each page’s intermediate files next to the sources, so the Makefile copies docs/ into docs/_build/ and renders that copy. During make preview, it copies your edits there every second. Both directories are gitignored.

The results pages and the solver reference include generated files from _generated/. Before each render or preview, the Makefile runs two scripts that write them into docs/_build/_generated/. docs/scripts/generate_results.py writes the results overview’s tables, the body of each setup’s page and a copy of the plots. It fills them from mosaic-results/ if that directory exists and writes placeholders otherwise. docs/scripts/generate.py then writes the solver reference. Both scripts import mosaic, which make takes care of by running them through uv.

To preview the results pages with real data, put a result tree in mosaic-results/, either from a local run or from the published baseline. make data fetches the baseline, which is several hundred megabytes and needs the GitHub CLI for a token, and then runs mosaic run --plots-only. That command regenerates the plots from the result.json files without Docker or a GPU, which is needed because the baseline stores data only.

make figures redraws the setup schematics.

Maintainers

This section describes the CI/CD internals, the artifacts they produce, and the release process. Contributors don’t need it to open a pull request.

Workflows

On pull requests:

benchmark.yml plans the run from the label and the changed files and generates a matrix of (suite, problem, hardware) jobs. It builds the changed solvers through build-tesseracts.yml, runs the matrix through benchmark-run.yml, and then reports. The report job merges the results, overlays them on the baseline, writes the status report, renders the docs and uploads the rendered site. It uses no repository secrets, so it behaves the same on fork pull requests.
build-tesseracts.yml, when called from benchmark.yml, builds each changed solver and uploads its image as an artifact instead of pushing it, since a fork’s token can’t push to GHCR.
benchmark-run.yml generates the initial conditions and runs each matrix job on its own runner (mosaic-bench-gpu or mosaic-bench-cpu). It loads the images built for the pull request and pulls the unchanged solvers from GHCR at the base commit.
docs-publish.yml runs after a successful benchmark.yml run on a pull request, in the context of the base repository, where secrets are available. It triggers the Read the Docs preview build, resolves the preview URL and posts the PR benchmark comment. It never checks out code from the pull request.
ci.yml, conventional-pr-linter.yml and cla.yml run the checks listed above.

On every push to main:

build-tesseracts.yml builds every solver and pushes it to GHCR, tagged with the commit SHA and latest. Pull request runs pull unchanged solvers by the SHA tag and use latest as a build cache.
publish-results.yml finds the benchmark run of the merged pull request, overlays its results on the baseline artifact and uploads the new baseline. It then renders the site from that result tree, uploads it and triggers a Read the Docs build. Runs are serialized, so two merges in quick succession can’t overwrite each other’s results.

On a schedule, bump_lockfile.yml and pre-commit-cron-updater.yml refresh the lockfile and the pre-commit hooks weekly, and cleanup-ghcr.yml deletes old SHA-tagged images.

Read the Docs

Read the Docs serves the site but doesn’t build it, because its builders run out of memory on import mosaic. The site is rendered on GitHub Actions, and .readthedocs.yaml only runs .github/scripts/rtd-serve-site.sh, which downloads the rendered site artifact matching the build and copies it into the output directory. Pull request builds fetch docs-site-pr-<number>, branch builds fetch docs-site-latest, and tag builds fetch docs-site-release-<version> with docs-site-latest as a fallback. If the artifact doesn’t exist yet, for example on the build that the GitHub webhook starts when a pull request is pushed, the script writes a placeholder page. docs-publish.yml and publish-results.yml trigger a fresh build once the artifact is uploaded. The download needs a GITHUB_TOKEN with actions:read, set as an environment variable on Read the Docs.

Artifacts

Benchmark results and rendered sites are stored as GitHub Actions artifacts:

Artifact	Contents
`baseline`	The accumulated results on `main`, replaced by `publish-results.yml` after each merge that produced results. Plots are stripped and regenerated at render time.
`results-<suite>-<problem>-<hardware>`, `results-ics`	The results of one benchmark job.
`solver-image-<name>`	A solver image built for a pull request, kept for a few days.
`docs-site-pr-<number>`, `docs-site-latest`, `docs-site-release-<version>`	Rendered sites that Read the Docs serves.
`docs-publish-meta-<number>`	The pull request number and status report handed from `benchmark.yml` to `docs-publish.yml`.

The results of each release are also attached to its GitHub release as mosaic-results-<version>.zip. Release pull requests compare against that asset of the previous release.

Releases

Releases are run by code owners through GitHub Actions. Mosaic follows semantic versioning.

Make sure main is in a good state, with tests passing and the docs up to date.
Trigger release.yml from the GitHub UI. It generates the changelog with git-cliff and opens a pull request from a bot/release/<version> branch. The full benchmark runs on that pull request.
Add release notes to the pull request description below the --- line.
Merge the pull request. release.yml then creates the GitHub release and tag, publish.yml builds the package and publishes it to PyPI as mosaic-bench, and publish-results.yml updates the baseline, attaches the results to the release, uploads docs-site-release-<version> and triggers the stable and latest docs builds.
Check that the release, the PyPI package and the docs were published.