Two-way GitHub materialization

Weave a GitHub repository's full history into a monorepo folder

Materialization transplants a source Git repository's entire commit history into a Vex monorepo subfolder as native, content-addressed history — then keeps that folder and the source GitHub branch in sync in both directions.

Fastest way to get started

Most teams don't call the materialize API directly — sign in, open /onboarding/github under your organization, install the GitHub App, and pick all or some of your repositories. The onboarding wizard runs the exact flow documented below for every repository you select, one at a time, and shows live progress per repository.

What materialization does

A virtual repository can declare a root_path and a GitHub (or generic Git) remote. Materializing it clones the remote's full history, transplants every commit into native Vex objects, grafts the source's root tree at root_path/, and writes a two-parent merge onto the trunk — additive only, so nothing outside root_path/ is touched.

The result is staged on a bookmark, never landed automatically — landing the merge onto the trunk bookmark is a deliberate, separate review step. During the transplant, Vex persists a durable git ↔ native commit identity map, so the same history round-trips with byte-identical git commit ids on every later sync.

Trigger a materialization

On the repository's sync page, use the Materialize history button. It opens a confirmation dialog explaining that the run is staged for review and can take a while for large histories, then dispatches the run. Triggering it manually requires organization admin access.

vex materialize <tenant>/<repo> \
  --source-url https://github.com/<owner>/<repo>.git \
  --root-path <root-path> \
  --source-branch master \
  --trunk-bookmark master

Re-running the same command is incremental: it weaves only commits newer than the last woven source tip, and no-ops when the source is already fully woven — it does not re-transplant the full history each time.

Two sync directions, one identity map

Pull: GitHub → Vex

A materialization run fetches the remote branch and transplants only the commits newer than the last-woven tip, grafts them onto the current trunk tip, and stages a new merge on the staging bookmark. Idempotent: nothing changed upstream means no new commits, no ref moves, and a no-op result.

Push: Vex → GitHub

A push-projection run walks native trunk commits under root_path since the last projection, rebuilds each as a real git commit with the original author, committer, and message preserved, and pushes with a compare-and-swap on the remote head — never a force push.

If the remote branch moved outside Vex since the last projection, the push fails closed with a Diverged status instead of guessing: “Remote branch moved outside Vex. Review the remote changes; Vex will not force-push.”

Sync modes

The repository's sync mode decides which direction the generic Run sync button dispatches:

pullGitHub to Vex (or Git to Vex for generic remotes)Pull from remote

pushVex to GitHub (or Vex to Git)Push to remote

bidirectionalBoth waysRun sync

Automatic sync is event-driven for repositories with auto sync enabled: a push that lands on the Vex trunk dispatches a push projection within seconds when it changed the materialized folder, and a push to the synced remote branch dispatches an incremental materialization as soon as the provider webhook arrives. A scheduled sweep (five minutes by default) backstops both directions, so a dropped webhook or event only delays a sync — it never loses one.

Watching a run

Every direction — materialize, push, or pull — is a sync run shown on the repository's sync page with a kind badge, live log lines, and (for materialize runs) the staging bookmark it wrote to:

MaterializeA pull-materialization run: fetches the source and weaves new history under root_path.

PushA push-projection run: replays new Vex-side commits under root_path back to the remote branch.

PullA generic pull sync run (non-materialize) for repositories still on the legacy import path.

The run history table also reports commits read, written, and skipped per run, the before/after remote SHA, and any error message. Active runs poll and stream their log panel every couple of seconds.

Operator notes

--backfill-map recomputes and persists the git ↔ native identity map for a subtree that was already materialized before the identity map existed. It transplants nothing and never moves a bookmark — it only writes the mapping so push projection and future incremental pulls have a boundary to walk from.

vex materialize <tenant>/<repo> \
  --source-url https://github.com/<owner>/<repo>.git \
  --root-path <root-path> \
  --backfill-map

Landing a staged materialization merge onto the trunk bookmark is intentionally manual: materialization only ever advances the staging bookmark (imports/<root-path>/materialize by default), so review the woven history before advancing the trunk bookmark to it.

Live example

The Vex monorepo's own jj/ folder is a two-way materialization of github.com/VEXscm/jj. A commit landed on the Vex trunk under jj/ (f435c449) was projected to GitHub as 93daaabc4 with the original author and message preserved, and a later upstream push flowed back in as an incremental merge that wove only the new commits rather than the full history.

API surface

POST /api/v1/virtual_repositories/:id/materializePOST /api/v1/virtual_repositories/:virtual_repository_id/sync_runsGET /api/v1/virtual_repositories/:virtual_repository_id/sync_runsGET /api/v1/virtual_repositories/:virtual_repository_id/sync_runs/:id

All four routes require organization access; creating a run requires organization admin and returns 422 when another run is already active for the virtual repository.

Remote Git sync configuration