This sequence revisits the methodolgy of Evolutionary Database Design, twenty years later. A key constraint to database-changes-as-code has at all times been round shared database assets. With copy-on-write branching in Databricks Lakebase, a one-second, zero-storage-at-creation department of a terabyte-scale manufacturing database is now an O(1) operation, and the constraint that stored Apply #4 (all people will get their very own database occasion) aspirational has lifted. On this sequence, the authors describes what adjustments when the constraint lifts: not the methodology, that holds, however the practices that emerge for the primary time, the team-scale governance that turns into computerized, the position evolution for the DBA, and the brand new functionality that brokers share with their human counterparts.
Jen is the developer character from Evolutionary Database Design. In that essay she applied a database refactoring, splitting an inventory_code area into location_code, batch_number, and serial_number, as a routine person story, illustrating that DBAs and builders can collaborate, schemas can evolve in small increments, and migrations carry the change ahead safely.
The sequence picks up with Jen twenty years later. The methodology she follows is similar one she adopted in 2003. What’s new is the technical functionality beneath her workflow, enabled by the lakebase structure: copy-on-write database branching, which makes the practices she has been studying about operationally actual at manufacturing scale. Throughout the three components of this sequence she is similar Jen at three scopes, her day (Half 1), her new playbook (Half 2), and her crew (Half 3).
Half 1 walked Jen by way of one function. The practices she adopted have been described within the 2003 Evolutionary Database Design essay, expanded within the 2006 Refactoring Databases e book , and introduced into the CI/CD pipeline within the 2010 Steady Supply e book (Chapter 12).
Authentic Seven Practices
The 2003 essay named seven practices. 5 of the seven had limitations of their utility till 2026.
DBAs collaborate carefully with builders.All database artifacts are model managed with utility code.All database adjustments are migrations.All people will get their very own database occasion.Builders constantly combine database adjustments.All database adjustments are database refactorings.Builders can replace their databases on demand.
Limitations of their utility
Apply #1 (DBA collaboration). Each schema change had production-scale penalties if it acquired unfastened, so DBA overview remained synchronous and gating. Collaboration was constrained by the DBA’s calendar.Apply #4 (All people will get their very own database occasion). Licensing prices, infrastructure prices, DBA time. Aspirational on most groups. Most groups fell again to shared improvement databases and accepted the rivalry.Apply #5 (Steady integration of database adjustments). The 2010 Steady Supply wave introduced migrations into the pipeline, however the pipeline ran migrations towards shared goal databases. Per-pipeline isolation was lacking.Apply #6 (All database adjustments are refactorings). Making use of every refactoring required observe areas (check databases) that almost all groups didn’t have at PR granularity.Apply #7 (Builders replace on demand). Builders may run migrations towards shared environments on demand, however couldn’t safely experiment, as a result of their experiments would have an effect on others.
What Modified
The expertise launched by Databricks Lakebase removes the roadblocks to the implementation of the above practices. Databricks Lakebase is a managed Postgres database that makes use of the identical object storage layer (the info lake) that the remainder of the Databricks lakehouse runs on. The database’s knowledge lives in shared, sturdy storage – successfully S3 buckets; the Postgres engine runs as a separate compute layer above it. Compute and storage scale independently. The engine can scale up below load, down when visitors drops, and to zero when idle. Lakebase is built-in with the unity catalog permitting for unified governance throughout a number of environments.
Copy-on-write database branching is what the decoupled structure makes sensible at scale. A department creates a brand new pointer into the identical shared storage with a divergence marker. Till the department writes, it shares all pages with its guardian. When the department writes, solely the modified pages diverge; the guardian stays untouched. Branching is a metadata operation, no knowledge copy required, finishing in roughly one second no matter guardian dimension. The branches keep knowledge within the modified pages solely.
When the technical value of a department is decoupled from the scale of the info inside it, the constraint behind Apply #4 (all people will get their very own database occasion) is unblocked. Per-developer, per-PR, per-experiment branches turn into routine. The compensating layer above can come out. Mocks come out of the check loop, changed by actual Postgres on a per-test department. Shared staging stops being the one place to check schema adjustments. In-memory database substitutes (H2, SQLite) come out of the unit-test layer. The devops tax to create docker based mostly containers to run native databases is just not vital, DBA ticket queues for provisioning shrink as a result of branches are self-service.
The expertise is what allows methodology optimizations and completes the objective of the practices behind the unique 2003 publish.
Rising practices for 2026
Lakebase copy-on-write database branching lifts the earlier limitations on the unique practices and allows 4 extra ones for elaboration.
DBAs collaborate carefully with builders. With the schema diff posted on each PR, the DBA critiques async, like some other code reviewer. For the reason that provisioning tax is negligible the DBA’s now have the bandwidth to overview and perhaps even work with the builders to create the options within the first place as an alternative of reviewing publish implementation.All database artifacts are model managed with utility code. The schema diff, the database migrations and migration check outcomes now be part of the artifact set.All database adjustments are migrations. Plus a brand new authorship rule: idempotency. Permitting all merged migrations to be deployed to downstream environments like QA, staging and Manufacturing mechanically.All people will get their very own database occasion. Operational at per-developer, per-PR, and per-experiment granularity. Permitting builders to experiment on a number of options and never simply accept the primary answer that works. This freedom will lead to optimum options being developed for manufacturing scale.Builders constantly combine database adjustments. Operational at PR granularity. Each PR runs by way of CI by itself department.All database adjustments are database refactorings. The 2006 catalog nonetheless applies; branches provide you with an affordable rehearsal house to attempt the refactorings on varied sized environments, migrations that labored in improvement databases virtually at all times didn’t carry out in manufacturing, now you may attempt migrations in bigger dimension databases to make sure they’re performant.Builders can replace their databases on demand. “On demand” now means one second, remoted, towards production-shaped knowledge.Damaging testing as a default possibility. Blast radius is zero; reset prices one second. Don’t have to fret about, “Will my check pollute knowledge for others?”, since now after each check run I can get a brand new department.A/B variant prototyping on the database degree. Construct two designs on parallel branches; work with the bigger crew and the DBA’s to debate the answer and do a present and inform, then hold the winner.With the built-in unity catalog, governance is designed as soon as, inherited by all of the branches. Insurance policies observe every department mechanically, we are going to talk about this intimately in Half 3.Agent-as-practitioner with the identical branching functionality. Brokers get branches, not manufacturing, we are going to talk about this intimately in Half 3.
How the workflow runs in CI
The 2 practices that get improved probably the most due to the aptitude supplied by Lakebase are #4 (all people will get their very own database occasion) and #5 (builders constantly combine database adjustments). Not simply everybody will get a database, each PR or each department or a number of databases per department might be had with negligible value and time. The mechanism might be automated utilizing two GitHub Actions workflow templates that may be scaffolded into each mission.
Per-PR department creation. pr.yml triggers on pull_request: [opened, synchronize] and creates ci-pr- forked from the PR’s base department:
The identical job applies migrations towards the CI department and runs the appliance’s check suite towards actual Postgres. Full instance of the workflow might be discovered right here: pr.yml
Schema diff posted on the PR. The identical pr.yml job dumps the schema of each branches, codecs the diff, and posts it as a PR remark. That is what lets the DBA overview async like all code reviewer (Apply #1, re-cast):
The DBA, the code reviewer, the crew and any agent writer of the migration see the identical artifact on the PR.
Department cleanup on merge. merge.yml destroys the CI Pull Request Department ci-pr- and the linked function department’s Lakebase department the second the PR merges:
Full instance of the merge workflow might be discovered right here: merge.yml With so many branches floating round it could be price having a weekly cleanup script that removes orphaned and unused branches, an instance workflow might be discovered right here: cleanup-orphans.yml. Lastly, if the merge is right into a ‘tiered’ department, e.g. used for staging or fundamental/manufacturing (this idea will likely be additional elaborated partly 3), the workflow could also be orchestrated to chop a recent department from the tier to check migrations towards it earlier than making use of the ultimate migration to any setting the place customers are discovered stay and always including knowledge.
Collectively these workflows implement each PR will get its personal database and branches are ephemeral as properties of the pipeline, not developer disciplines.
The brand new playbook for Evolutionary Database Growth
Apply #1: DBAs collaborate carefully with builders
Rule. The DBA collaborates with builders all through the function, not simply at gate-review time. The collaboration is asynchronous, inline with the PR, the way in which different code reviewers take part.
Why is that this a sturdy behavior now? The schema diff and migration check outcomes land on each PR mechanically (see How the workflow runs in CI above). The DBA critiques a concrete artifact on their very own schedule. The migration has already run towards a real-data CI department, so the DBA doesn’t have to mentally simulate the change.
Mechanics:
The DBA is a CODEOWNER on schema-affecting paths (migrations/, db/, schema check directories).The DBA critiques on the PR like some other reviewer, async.Overview focus shifts from will this break the database to is that this the precise design? Has it been applied the precise method? Matters: knowledge integrity guidelines, indexing technique, design cohesion, future extensibility, long-term maintainability.
Anti-pattern. Not together with the DBA within the PR circulation with there being legit artifacts to overview.
The DBA position additionally features new duties in coverage administration and governance at crew scale. Half 3 covers these.
Apply #2: All database artifacts are model managed with utility code
Rule. Each SQL file, migration script, and schema check lives in the identical repository as the appliance code. The schema diff and migration check outcomes be part of the artifact set as PR-time outputs.
Why is that this a sturdy behavior now? Branching provides two new artifacts to the set: the per-PR schema diff and the per-PR schema migration check outcomes. Each are generated by CI from the precise department state. Each land within the PR as concrete proof about what was modified and the way the change migration script carried out.
Mechanics:
Migration recordsdata in a versioned listing (migrations/, db/migration/, alembic/variations/, framework-dependent).Schema-affecting checks within the check tree alongside utility checks.Schema diff generated per PR by CI, posted as a PR remark.Migration check ends in the CI run abstract and PR remark.
Anti-pattern. Producing the schema diff outdoors the PR circulation (a separate dashboard the reviewer has to open). The artifact has to stay the place the overview occurs. For the reason that schema adjustments are tied to the code adjustments and breaking this dependency creates downstream issues for deployment.
Apply #3: All database adjustments are migrations
Rule. No guide ALTER TABLE towards any setting. Each schema change is a versioned, checked-in migration script. Migrations are idempotent.
Why is that this a sturdy behavior now? The migration-as-artifact rule is unchanged from 2003. What’s new is the authorship self-discipline of idempotency. The identical migration runs towards many branches over the lifetime of a transition, so it has to behave the identical method each time. A migration that fails on re-apply is a bug.
Mechanics:
Use migration frameworks like flyway, liquibase, Knex, Alembic or others, these frameworks hold monitor of which migrations have been run and which haven’t been, this permits the crew to use a command like flyway migrate which simply applies the adjustments that haven’t been utilized (by holding monitor of the adjustments in a metadata desk)It’s higher to separate irreversible work throughout migrations. For instance a migration that provides new columns and drops the supply column in a single shot makes rollback tougher than they have to be, so utilizing the develop first and contract later technique, offers many extra choices after a deploy cycle confirms no stay readers reference it.The 2006 database refactoring catalog names which refactorings are reversible and which aren’t. Use it.
Anti-pattern. A migration that depends upon the schema being in a particular intermediate state due to native adjustments made within the department. The migration should apply appropriately towards any guardian department that features prior migrations.
Apply #4: All people will get their very own database occasion
Rule. Each developer, each PR, each experiment, each check run will get its personal Lakebase department.
Why is that this a sturdy behavior now? The additional effort to create docker containers, to put in native database servers, purchase licenses, hydrate empty databases with present schema and check knowledge is not required. Only a easy create-branch Lakebase command branches a 1TB database in the identical one second as a 1MB database. No knowledge is copied at creation; solely modified pages eat storage. Per-developer, per-PR, and per-experiment cases are routine.
Mechanics:
Per-developer branches: created on demand through databricks postgres create-branch or the SCM extension’s branch-create circulation.Per-PR branches: created mechanically by CI on PR open (pr.yml), destroyed on merge or shut (merge.yml). See How the workflow runs in CI above for the PR and merge snippets.Per-experiment branches: forked off staging or manufacturing for design exploration; discarded after the experiment.
Anti-pattern. Sharing a improvement database throughout the crew “for comfort.” The contention-driven serialization Half 1 named comes again the second the branches collapse.
The place Jen’s instance extends. Her per-developer department was forked off staging at function begin. The CI department was forked off staging on PR open. Her A/B exploration branches (Apply #9) have been forked off staging in parallel. 4 branches throughout one function, all in seconds, all remoted.
Apply #5: Builders constantly combine database adjustments
Rule. Each PR runs by way of CI towards a recent Lakebase department, with migrations utilized and checks run towards actual Postgres.
Why is that this a sturdy behavior now? The CI pipeline has had migration self-discipline since 2010. What’s new is per-pipeline isolation: every PR will get its personal department, so integration runs towards real-shaped knowledge with out rivalry.
Mechanics:
CI creates a department on PR open. See How the workflow runs in CI above for the PR snippet.Migrations utilized towards the department through lakebase-migrate apply.Utility checks run towards the migrated department by way of the ORM, no mocks.Schema diff posted on the PR.Department destroyed on merge or shut.
Anti-pattern. Operating PR validation towards shared staging. The serialization comes again; the per-PR isolation property is misplaced.
Apply #6: All database adjustments are database refactorings
Rule. Schema adjustments observe named refactoring patterns: Break up Column, Rename Column, Transfer Column, Substitute Sort, and so forth. Every has specific transition mechanics (hold outdated plus new in parallel, populate from outdated, swap readers, drop outdated).
Why is that this a sturdy behavior now? The 2006 catalog at databaserefactoring.com names 70+ refactorings with labored examples. What’s new in 2026 is an affordable place to rehearse the transition mechanics: a developer department absorbs the rehearsal; the CI department verifies; manufacturing sees solely the verified end result.
Mechanics:
Determine the refactoring by identify. Look it up within the catalog.Apply the named transition mechanics on a per-developer department. Validate towards production-shaped knowledge.Open the PR. CI runs the migration by itself department and posts the diff.Merge as soon as the diff and the check outcomes land approval.
Anti-pattern. A one-off schema change that doesn’t map to a named refactoring. The 70+ catalog covers the frequent circumstances; if you end up outdoors it, you might be probably combining a number of refactorings into one migration and will break up.
The place Jen’s instance extends. Her V87 migration is the Break up Column refactoring: splitting inventory_code into location_code, batch_number, and serial_number. The catalog web page at databaserefactoring.com/SplitColumns.html names the transition mechanics. Her department was the rehearsal house; the PR’s CI run was the verification.
Apply #7: Builders can replace their databases on demand
Rule. A developer can refresh their department’s database state on demand: reset to the guardian’s present state, fork a recent department off manufacturing, discard an experimental department, share a department with a teammate. All in seconds.
Why is that this a sturdy behavior now? “On demand” in 2026 means one second, remoted, towards production-shaped knowledge. None of those operations seek the advice of ops calendars or DBA queues.
Mechanics:
Reset: delete and recreate the department from its guardian.Fork off manufacturing: databricks postgres create-branch –source manufacturing.Discard: databricks postgres delete-branch.Share: hand the department endpoint to a teammate for a pairing session.
Anti-pattern. Treating any department as sturdy past its objective. The migration is the sturdy artifact; the department is the workspace.
Apply #8: Damaging testing as a default possibility
Rule. When the blast radius of a harmful motion is zero, harmful testing turns into a each day possibility quite than a quarterly train.
Why is that this a sturdy behavior now? A department resets in a single second. Something you do to a department might be undone by making a recent one off the identical guardian. Damaging checks cease needing ops calendars and approval gates.
Issues that now match inside a standard function cycle:
“What occurs if my migration fails midway by way of the UPDATE assertion?” Run it. Kill the method at 50%. Confirm the rollback works. Reset.“What occurs if a backup-restore is mid-way by way of when a failover triggers?” Simulate the partial state. Confirm the appliance’s conduct. Reset.“What’s the precise time-to-recover for our DR runbook?” Run the runbook. Measure. Reset.“Does this migration run efficiently towards manufacturing form knowledge or dimension” confirm it earlier than c
Cultural impact. When reset prices nothing, groups cease treating the check database as a valuable useful resource. Assessments might be aggressive. Cleanup might be skipped, as a result of the following department begins recent.
The place Jen’s instance extends. Earlier than opening her PR, she took the production-shaped knowledge on her department and intentionally corrupted round one % of the inventory_code values to seem like edge circumstances: lacking digits, embedded areas, trailing whitespace. The sorts of artifacts that historic knowledge accumulates. She ran her migration. Two rows failed her substring math. She mounted the script and re-ran. The department absorbed the harmful check. Manufacturing by no means noticed it.
Apply #9: A/B variant prototyping on the database degree
Rule. When two designs are in rivalry, construct them on parallel branches, examine towards production-shaped knowledge, and hold the higher answer.
Why is that this a sturdy behavior now? Per-branch value is close to zero. Exploring two schema designs not requires choosing one prematurely, and it not requires a provisioning course of most groups won’t undertake for an exploratory query.
Mechanics:
Create two branches off the identical guardian.Apply Design A’s migration to at least one department, Design B’s migration to the opposite.Run the appliance towards every. Measure what issues: question latency on the frequent learn path, migration time at manufacturing quantity, index footprint, lock rivalry below simulated load.Choose the higher answer. Discard the sub optimum answer. Doc the choice within the PR description so the following one who has to increase the schema is aware of what was thought of and why this design was picked.
Anti-pattern. Operating A/B prototypes with out writing down the choice and the reasoning. The branches are gone in a second; the design resolution needs to be everlasting.
The place Jen’s instance extends. She thought of two designs for the placement/batch/serial function: three new columns on the present stock desk, or a separate inventory_attributes lookup desk keyed by inventory_id, anticipating that extra attributes could be added later. She constructed each on parallel branches off staging. She ran the appliance’s learn path towards every, measured question efficiency towards production-shaped knowledge, and checked out how every migration would scale to manufacturing volumes. The lookup-table model carried out worse on the frequent learn path as a result of each stock show required a be part of. She shipped the columns model, threw away the lookup-table department, and left a word within the PR description: Thought of lookup-table model; rejected as a result of the frequent learn path turns into a be part of. Revisit if greater than 5 attributes accumulate.
What Jen’s New Playbook Reveals
We’ve named the seven practices from 2003 with the restrictions that stored 5 of them aspirational, and re-cast them for 2026 as soon as branching landed, and added 4 new practices that branching allows. Eleven practices whole within the new playbook for Evolutionary Database Growth, 9 of that are defined above.
In Half 1 – Jen’s story: one function, one database we noticed Jen work by way of one function: she paired a code department with a Lakebase department, ran an actual migration towards production-shaped knowledge in seconds, examined with out mocks, opened a PR with the schema diff posted inline, and merged with the migration utilized and the ephemeral branches cleaned up. Database change grew to become a part of regular improvement.
In Half 3 – Jen’s Workforce at Scale, we have a look at the playbook at fifty builders, the DBA’s advanced duties in coverage administration and governance, and the brokers creating branches alongside Jen. Practices #10 and #11 get their full therapy there.
The Companion: Plugin Walkthrough covers the Lakebase SCM Extension for VS Code and Cursor.
A Lakebase App Dev Package for brokers, with a companion e-book for human practitioners, will likely be launched as a follow-on.
