All demos/ Flyway adoption

DEMO-02-FLYAUG 2026

Retrofit generates the migration scripts. Flyway runs them.

From a schema dump imported into its operation log, retrofit writes the migrations that add and then validate a foreign key between existing tables, then backs the change out and reapplies it. Every script is generated, none is written by hand.

Stack: Flyway + PostgreSQL
Change: AddForeignKey, NOT VALID
Reverse: generated + classified

Chapters

01 Adopt and deploy46 secAdd retrofit to a live Flyway project and ship one change. 02 Validate the existing rows41 secProve every existing row points at a real run, then turn the constraint fully on. 03 Reverse and reapply50 secRoll back the foreign key, then redeploy it. No SQL written by hand. 04 Prove the entire lifecycle4 min 01 secSix Flyway migrations, V75 to V80, with provenance and byte checks.

Summary

Your production migrations run through Flyway, and you want them generated and checked, not written by hand.

Marquez builds its schema with 74 Flyway migrations. We take a schema-only dump of that schema, and retrofit import reads it into the operation log as 469 operations, with nothing replayed and nothing rewritten (Fig. 1A). Then we add the foreign key the schema is missing: jobs.current_run_uuid already points at runs.uuid and is indexed, and nothing enforces it. From here on retrofit renders every migration, and Flyway runs them.

We create the foreign key with PostgreSQL’s NOT VALID option: the constraint checks new writes immediately and defers the full-table scan of existing rows to a later validation step (Fig. 1B). We check those rows first, so the second migration’s VALIDATE CONSTRAINT passes when Flyway applies it.

Then we back the change out and put it back. This is a deliberate reversal after V75 and V76 succeeded, not a failed migration rolling back. Flyway Community is append-only, so the generated reverses deploy as V77 and V78, and the same forward bodies reapply as V79 and V80. Deleting a run leaves the job in place with a null current_run_uuid (Fig. 1C), and Flyway’s history keeps every step, V74 to V80.

Each generated migration leaves transaction framing to Flyway. That is the one advisory the demo attests to, and it fails unless the rest of the stream is empty.

Import Existing Schema: a stack of 74 Flyway migrations becomes one card of 469 Retrofit schema operations. — Fig. 1AOne schema-only dump imports as 469 schema operations; the 74-migration Flyway history is not replayed.

Stage Changes in Retrofit: the jobs table and runs table converge into a new foreign key box marked ON DELETE SET NULL, NOT VALID. — Fig. 1BThe foreign key is staged as `NOT VALID`: new writes are protected immediately, and the scan waits for validation.

Confirm with Application: deleting a run leaves the job in place; current_run_uuid becomes NULL; migration history untouched; new writes protected by PostgreSQL. — Fig. 1CThe application confirms the result: deleting a run leaves the job in place with a null `current_run_uuid`.

Chapter 01Adopt and deploy

Can retrofit fit into the Flyway workflow we already have?

Import Marquez’s current PostgreSQL schema after 74 existing Flyway migrations, add a foreign key from jobs.current_run_uuid to runs.uuid, and hand transaction framing and deployment back to Flyway.

46 sec space to pause f for fullscreen

Read transcript

flyway-adopt-retrofit-demo: adopt existing project

Generated from the demo script. Every command below is run verbatim by the recording.

1/4 start with an existing Flyway project

Marquez already has 74 Flyway versions in production. Keep every applied migration:

psql "$DB" -X -Atq -c "SELECT 'Latest Flyway version: V' || version FROM flyway_schema_history ORDER BY installed_rank DESC LIMIT 1"

capture the schema as it exists now, without replaying its migration history:

pg_dump "$DB" --schema-only --no-owner --no-privileges --exclude-table=public.flyway_schema_history > project/baseline.sql

2/4 add Retrofit

import that dump once, and seal it: the point Retrofit’s log starts from:

retrofit import project/baseline.sql --force --allow-capability-gaps --seal='Flyway V74 baseline' --summary-file project/import-summary.json --skip-manifest project/import-skips.jsonl -q > project/import.log

a mature schema becomes 469 schema operations, including 30 tables, 199 columns, 181 constraints, and 46 indexes; nothing skipped:

cat project/import-summary.json

3/4 define the schema change

469 operations describe the schema you already have. The change you want is one: a foreign key from jobs.current_run_uuid to runs.uuid with ON DELETE SET NULL. NOT VALID protects new writes without scanning existing jobs rows in this deployment:

retrofit op add AddForeignKey jobs referenced_table=runs on_delete=set_null name=jobs_current_run_uuid_fkey not_valid=true --column=jobs.current_run_uuid --referenced_column=runs.uuid -q

retrofit op seal --reason 'Flyway V75: protect jobs.current_run_uuid' -q

4/4 generate migration SQL for Flyway

envelope=caller hands transaction framing to Flyway; the flag next to it attests that we know Flyway opens it. every other diagnostic still prints:

retrofit migrate --from seal~1 --to seal --envelope caller --allow-external-atomicity -q > project/src/main/resources/db/migration/V75__add_jobs_current_run_fk.sql

retrofit migrate --from seal~1 --to seal --down --strict --envelope caller --allow-external-atomicity -q > project/U75__add_jobs_current_run_fk.sql

ordinary reviewable SQL in both directions:

sed -n '/^ALTER TABLE/p' project/src/main/resources/db/migration/V75__add_jobs_current_run_fk.sql project/U75__add_jobs_current_run_fk.sql

Flyway owns the transaction and deploys V75 through the same pipeline:

docker compose -p '$COMPOSE_PROJECT' -f compose.yaml run --rm flyway migrate

psql "$DB" -X -Atq -c "SELECT 'Flyway applied: V' || version FROM flyway_schema_history ORDER BY installed_rank DESC LIMIT 1"

the database now clears the pointer when a current run is deleted. the job survives:

./consumer/after/test-current-run.sh "$DB"

Retrofit writes the migration. Flyway remains the deployer.

Dump the schema 74 Flyway migrations built. Every applied migration stays in place; nothing is replayed.
import reads the DDL in the dump and writes it to the log as operations: 469, covering 30 tables, 199 columns, 181 constraints, and 46 indexes. --seal seals them as the V74 baseline, the point retrofit’s log starts from.
op add appends one operation to the log. A foreign key from jobs.current_run_uuid to runs.uuid, ON DELETE SET NULL, authored NOT VALID: new writes are checked at once and existing rows in a later migration.
migrate renders the SQL from one seal to the next; --down --strict renders the reverse from the same operations. --envelope caller leaves the transaction to Flyway. Flyway deploys V75, and the consumer test passes: a deleted run leaves its job with a null pointer.

Chapter 02Validate the existing rows

Can retrofit model a production rollout, not just emit valid DDL?

Confirm every stored current-run value points to a row in runs, validate the foreign key in a second migration, and show its reverse is a no-op: there is nothing to un-prove.

41 sec space to pause f for fullscreen

Read transcript

flyway-adopt-retrofit-demo: staged validation

Generated from the demo script. Every command below is run verbatim by the recording.

1/3 V75 enforces the FK for new writes

Flyway deployed the foreign key without forcing an old-row validation scan:

psql "$DB" -X -Atq -c "SELECT 'Flyway V' || max(version) || '  FK active  validation=' || CASE WHEN bool_and(convalidated) THEN 'complete' ELSE 'pending' END FROM flyway_schema_history, pg_constraint WHERE conname='jobs_current_run_uuid_fkey'"

check that every jobs.current_run_uuid value matches an existing runs.uuid:

psql "$DB" -X -Atq -c "SELECT 'existing orphan rows: ' || count(*) FROM jobs j LEFT JOIN runs r ON r.uuid=j.current_run_uuid WHERE j.current_run_uuid IS NOT NULL AND r.uuid IS NULL"

2/3 validate existing rows as Flyway V76

validate jobs_current_run_uuid_fkey as a migration of its own, sealed as V76:

retrofit op add ValidateConstraint jobs_current_run_uuid_fkey -q

retrofit op seal --reason 'Flyway V76: validate jobs.current_run_uuid' -q

retrofit migrate --from seal~1 --to seal --envelope caller --allow-external-atomicity -q > project/src/main/resources/db/migration/V76__validate_jobs_current_run_fk.sql

retrofit migrate --from seal~1 --to seal --down --strict --envelope caller --allow-external-atomicity -q > project/U76__validate_jobs_current_run_fk.sql

the forward proves the rows. there is nothing to un-prove, so the strict reverse is an explicit no-op:

sed -n '/^ALTER TABLE/p;/^-- reverse-noop/p' project/src/main/resources/db/migration/V76__validate_jobs_current_run_fk.sql project/U76__validate_jobs_current_run_fk.sql

3/3 Flyway V76 completes validation

docker compose -p '$COMPOSE_PROJECT' -f compose.yaml run --rm flyway migrate

psql "$DB" -X -Atq -c "SELECT 'Flyway V' || max(version) || '  constraint validated=' || bool_and(convalidated) FROM flyway_schema_history, pg_constraint WHERE conname='jobs_current_run_uuid_fkey'"

Retrofit models the rollout boundary. the deployment’s undo lands through U75’s DROP.

Every non-null jobs.current_run_uuid matches a runs.uuid. Zero orphans, counted before we validate.
One operation, sealed as V76. The full-table scan NOT VALID deferred is paid here, in a migration of its own.
--down --strict renders the reverse, and this one is an explicit no-op: PostgreSQL cannot unvalidate a constraint. The three reverse-noop lines say so. Undoing the deployment is U75’s DROP.

Chapter 03Reverse and reapply

Can the generated reverse be used in a release process?

With V75 and V76 applied, deploy their generated reverses as append-only migrations V77 and V78, then reapply the same two steps as V79 and V80.

50 sec space to pause f for fullscreen

Read transcript

flyway-adopt-retrofit-demo: community recovery

Generated from the demo script. Every command below is run verbatim by the recording.

1/3 deploy generated reverse SQL as V77 and V78

V75 and V76 succeeded. This is not failed-migration recovery; PostgreSQL handles that transactionally. Community Flyway is append-only, so render Retrofit’s reverse as the next versioned releases:

retrofit migrate --from seal~1 --to seal --down --strict --envelope caller --allow-external-atomicity -q > project/src/main/resources/db/migration/V77__revert_validation.sql

retrofit migrate --from seal~2 --to seal~1 --down --strict --envelope caller --allow-external-atomicity -q > project/src/main/resources/db/migration/V78__revert_jobs_current_run_fk.sql

V77 preserves proven validation; V78 drops jobs_current_run_uuid_fkey:

sed -n '/^ALTER TABLE/p;/^-- reverse-noop/p' project/src/main/resources/db/migration/V77__revert_validation.sql project/src/main/resources/db/migration/V78__revert_jobs_current_run_fk.sql

docker compose -p '$COMPOSE_PROJECT' -f compose.yaml run --rm flyway migrate

psql "$DB" -X -Atq -c "SELECT 'after V' || max(version) || ': foreign key count=' || count(c.oid) FROM flyway_schema_history h LEFT JOIN pg_constraint c ON c.conname='jobs_current_run_uuid_fkey'"

2/3 reapply the original changes as V79 and V80

render the same two forwards again, from the same seals. no copied or hand-edited SQL:

retrofit migrate --from seal~2 --to seal~1 --envelope caller --allow-external-atomicity -q > project/src/main/resources/db/migration/V79__reapply_jobs_current_run_fk.sql

retrofit migrate --from seal~1 --to seal --envelope caller --allow-external-atomicity -q > project/src/main/resources/db/migration/V80__revalidate_jobs_current_run_fk.sql

docker compose -p '$COMPOSE_PROJECT' -f compose.yaml run --rm flyway migrate

jobs_current_run_uuid_fkey is validated again. deleting a run still keeps the jobs row while clearing jobs.current_run_uuid:

./consumer/after/test-current-run.sh "$DB"

3/3 Flyway records the rollback and reapply

psql "$DB" -X -Atq -c "SELECT version || '  ' || description || '  success=' || success FROM flyway_schema_history WHERE version BETWEEN '75' AND '80' ORDER BY installed_rank"

Forward → reverse → forward. Retrofit derives every body; Flyway records every deployment.

Flyway Community is append-only, so each reverse is rendered as a compensating migration, the next version up: V77 is the validation’s no-op, V78 drops the key. Flyway runs both; the foreign key count reads 0.
The same two forward migrations, rendered again from the same seals as V79 and V80: the key NOT VALID, then its validation. No SQL is copied or edited by hand. The consumer test passes again.
Six rows, V75 to V80, all success=true: forward, reverse, forward. Flyway recorded every deployment and retrofit rendered every body.

Chapter 04Prove the entire lifecycle

The uninterrupted run includes authentic provenance, deterministic rerenders and byte checks, database assertions, append-only reversal, reapply, and the final application proof.

4 min 01 sec space to pause f for fullscreen

Read transcript

flyway-adopt-retrofit-demo: complete walkthrough

Generated from the demo script. Every command below is run verbatim by the recording.

KEEP=1: leaving the database and project/ in place

a current application, one weak invariant

this is an existing Flyway project at V74, frozen by source commit and checksum:

sed -n '1,10p' inputs/MANIFEST.md

Retrofit is source-identifiable too:

retrofit --version

we will add a foreign key from jobs.current_run_uuid to runs.uuid; deleting a referenced runs row will clear that column without deleting the jobs row. the policy is illustrative, not an upstream proposal. start a disposable PostgreSQL 17 database:

docker compose -p '$COMPOSE_PROJECT' -f compose.yaml up -d --wait db

add Retrofit to the existing project

the checked-in config matches the dump’s explicit empty search_path:

cp inputs/retrofit.toml project/retrofit.toml

retrofit schema create public project/ --is-default -q

import the frozen V74 schema; save the 469-line operation report off-screen:

retrofit import inputs/marquez-v74.schema.sql --force --allow-capability-gaps --seal='Marquez V74 baseline' --summary-file project/import-summary.json --skip-manifest project/import-skips.jsonl -q > project/import.log

the mature schema becomes 469 schema operations: 30 tables, 199 columns, 181 constraints, 46 indexes, and supporting database objects:

cat project/import-summary.json

the last operation is a seal with a reason, not a SQL filename:

retrofit op show --include-sealed -q | tail -2

snapshot is a build artifact. render it twice and prove byte identity:

retrofit snapshot -q > project/marquez-v74.snapshot.sql

retrofit snapshot -q > project/marquez-v74.snapshot.again.sql

cmp project/marquez-v74.snapshot.sql project/marquez-v74.snapshot.again.sql

byte-identical. now restore that generated snapshot, not a hand-maintained migration stack:

psql "$DB" -X -v ON_ERROR_STOP=1 -q -f project/marquez-v74.snapshot.sql

the authentic shape is here: column, index, and no foreign key:

psql "$DB" -X -v ON_ERROR_STOP=1 -P pager=off -f assertions/baseline-proof.sql

keep representative Marquez job/run rows through the whole rollout:

psql "$DB" -X -v ON_ERROR_STOP=1 -q -f assertions/setup-fixture.sql

the consumer exposes today’s failure: cleanup leaves a dangling pointer:

./consumer/before/test-current-run.sh "$DB"

Flyway still owns deployed history; mark this non-empty database as V74:

docker compose -p '$COMPOSE_PROJECT' -f compose.yaml run --rm flyway -url=jdbc:postgresql://db:5432/marquez -user=postgres -password=pw -locations=filesystem:/flyway/sql -baselineVersion=74 '-baselineDescription=Marquez V74 baseline' baseline

V75 protect new writes

add jobs.current_run_uuid -> runs.uuid with ON DELETE SET NULL and NOT VALID:

retrofit op add AddForeignKey jobs referenced_table=runs on_delete=set_null name=jobs_current_run_uuid_fkey not_valid=true --column=jobs.current_run_uuid --referenced_column=runs.uuid -q

retrofit op seal --reason 'protect jobs.current_run_uuid' -q

NOT VALID protects new writes immediately while old rows are checked separately. Flyway owns the transaction, so render this migration with envelope=caller and attest it. the attestation answers one named fact; it silences no other diagnostic:

retrofit migrate --from seal~1 --to seal --envelope caller --allow-external-atomicity -q > project/src/main/resources/db/migration/V75__add_jobs_current_run_fk.sql

retrofit migrate --from seal~1 --to seal --down --strict --envelope caller --allow-external-atomicity -q > project/U75__add_jobs_current_run_fk.sql

the generated forward and strict reverse are ordinary reviewable SQL:

sed -n 1,12p project/src/main/resources/db/migration/V75__add_jobs_current_run_fk.sql

sed -n 1,12p project/U75__add_jobs_current_run_fk.sql

Flyway deploys the generated file unchanged:

docker compose -p '$COMPOSE_PROJECT' -f compose.yaml run --rm flyway -url=jdbc:postgresql://db:5432/marquez -user=postgres -password=pw -locations=filesystem:/flyway/sql migrate

the catalog proves jobs_current_run_uuid_fkey exists and awaits old-row validation:

psql "$DB" -X -v ON_ERROR_STOP=1 -P pager=off -f assertions/catalog-unvalidated.sql

now the revised consumer test proves the payoff through the same cleanup path:

./consumer/after/test-current-run.sh "$DB"

V76 validate existing data

pressure-test the old rows explicitly before taking the validation lock:

psql "$DB" -X -v ON_ERROR_STOP=1 -P pager=off -f assertions/orphan-check.sql

zero orphans. author the validation, and seal it as a second migration:

retrofit op add ValidateConstraint jobs_current_run_uuid_fkey -q

retrofit op seal --reason 'validate jobs.current_run_uuid' -q

retrofit migrate --from seal~1 --to seal --envelope caller --allow-external-atomicity -q > project/src/main/resources/db/migration/V76__validate_jobs_current_run_fk.sql

retrofit migrate --from seal~1 --to seal --down --strict --envelope caller --allow-external-atomicity -q > project/U76__validate_jobs_current_run_fk.sql

forward proves the rows. there is nothing to un-prove, so the strict reverse is an explicit no-op:

sed -n 1,12p project/src/main/resources/db/migration/V76__validate_jobs_current_run_fk.sql

sed -n 1,14p project/U76__validate_jobs_current_run_fk.sql

docker compose -p '$COMPOSE_PROJECT' -f compose.yaml run --rm flyway -url=jdbc:postgresql://db:5432/marquez -user=postgres -password=pw -locations=filesystem:/flyway/sql migrate

the same constraint is now validated:

psql "$DB" -X -v ON_ERROR_STOP=1 -P pager=off -f assertions/catalog-validated.sql

change, deployment, behavior

Retrofit records the schema operations and the two seals:

retrofit op show --include-sealed -q | grep -E 'AddForeignKey|ValidateConstraint|Seal' | tail -5

Flyway records what reached the database:

psql "$DB" -X -Atc "SELECT version || '  ' || description FROM flyway_schema_history WHERE version IN ('75','76') ORDER BY installed_rank"

and the consumer test records why this migration exists.

reverse, then reapply

Community Flyway is append-only, so render the V76 reverse directly as V77:

retrofit migrate --from seal~1 --to seal --down --strict --envelope caller --allow-external-atomicity -q > project/src/main/resources/db/migration/V77__revert_validate_jobs_current_run_fk.sql

cmp project/U76__validate_jobs_current_run_fk.sql project/src/main/resources/db/migration/V77__revert_validate_jobs_current_run_fk.sql

docker compose -p '$COMPOSE_PROJECT' -f compose.yaml run --rm flyway -url=jdbc:postgresql://db:5432/marquez -user=postgres -password=pw -locations=filesystem:/flyway/sql migrate

the proof-preserving no-op leaves the stronger validated state in place:

psql "$DB" -X -v ON_ERROR_STOP=1 -P pager=off -f assertions/catalog-validated.sql

then render the V75 reverse directly as V78; this drops jobs_current_run_uuid_fkey:

retrofit migrate --from seal~2 --to seal~1 --down --strict --envelope caller --allow-external-atomicity -q > project/src/main/resources/db/migration/V78__revert_add_jobs_current_run_fk.sql

cmp project/U75__add_jobs_current_run_fk.sql project/src/main/resources/db/migration/V78__revert_add_jobs_current_run_fk.sql

docker compose -p '$COMPOSE_PROJECT' -f compose.yaml run --rm flyway -url=jdbc:postgresql://db:5432/marquez -user=postgres -password=pw -locations=filesystem:/flyway/sql migrate

psql "$DB" -X -v ON_ERROR_STOP=1 -P pager=off -f assertions/catalog-absent.sql

render the same two forwards again, from the same seals, as V79 and V80. no copied or edited SQL:

retrofit migrate --from seal~2 --to seal~1 --envelope caller --allow-external-atomicity -q > project/src/main/resources/db/migration/V79__reapply_jobs_current_run_fk.sql

retrofit migrate --from seal~1 --to seal --envelope caller --allow-external-atomicity -q > project/src/main/resources/db/migration/V80__revalidate_jobs_current_run_fk.sql

cmp project/src/main/resources/db/migration/V75__add_jobs_current_run_fk.sql project/src/main/resources/db/migration/V79__reapply_jobs_current_run_fk.sql

cmp project/src/main/resources/db/migration/V76__validate_jobs_current_run_fk.sql project/src/main/resources/db/migration/V80__revalidate_jobs_current_run_fk.sql

docker compose -p '$COMPOSE_PROJECT' -f compose.yaml run --rm flyway -url=jdbc:postgresql://db:5432/marquez -user=postgres -password=pw -locations=filesystem:/flyway/sql migrate

psql "$DB" -X -v ON_ERROR_STOP=1 -P pager=off -f assertions/catalog-validated.sql

and the application proof still passes after the full operational cycle:

./consumer/after/test-current-run.sh "$DB"

six append-only Flyway rows; two seals in the log; one behavior engineers care about:

psql "$DB" -X -Atc "SELECT version || '  ' || description || '  success=' || success FROM flyway_schema_history WHERE version BETWEEN '75' AND '80' ORDER BY installed_rank"

Retrofit writes the migrations. Flyway runs them. The application proves the result.