Conversation
Whether an index has to be built without blocking writes is not a property of the index. It is whether the table is being created -- nothing to scan, and the statement belongs inline in the CREATE TABLE script -- or altered, where there are rows to scan and ACCESS EXCLUSIVE for the whole build is a write outage. Today that decision is made where the index is declared, through IsConcurrent, so every consumer has to anticipate it per index: Marten carries BuildHStoreTagIndexConcurrently for one index on mt_events and is about to add a second flag for another. TableDelta already knows which of the two it is. PostgresqlMigrator gets BuildIndexesConcurrentlyOnAlter, and an index added to (or changed on) a table that exists is then built concurrently without being marked. The create path is untouched: WriteCreateStatement still writes what the index itself declares, so an empty table is not paid for with a concurrent build it cannot benefit from -- and, on a partitioned parent, one PostgreSQL would refuse outright. Off by default, because it changes what a generated patch script is and not just how fast it runs: a concurrent build cannot run inside a transaction, so the script stops being runnable as one block, and a failed build leaves an invalid index that has to be dropped before a retry. Applying through PostgresqlMigrator is unaffected -- it already splits on the index-creation markers and each statement auto-commits. ToCreateSql(Table, bool) is the seam, with the caller's answer authoritative in both directions; ToCreateSql(Table) keeps taking it from IsConcurrent.
erdtsieck
added a commit
to erdtsieck/marten
that referenced
this pull request
Sep 15, 2026
EnableEventTypeIndex is recommended for large event stores -- its own docs list "projection rebuilds time out" and "millions of events" as the symptoms -- but it is added as a plain CREATE INDEX, which holds ACCESS EXCLUSIVE on mt_events for the whole build. So the stores the option exists for are exactly the ones that cannot afford to create it, and under UseTenantPartitionedEvents there is no CREATE INDEX CONCURRENTLY to fall back on by hand either: PostgreSQL refuses CONCURRENTLY on a partitioned parent. The optimizing page now says so, and the index name is a constant beside HStoreTagIndexName, so the way out it documents -- ignore it and own it -- does not need a string literal that has to be kept in step with Marten. Deliberately no third thing here. The first draft added a BuildEventTypeIndexConcurrently flag mirroring BuildHStoreTagIndexConcurrently, which would have been the second per-index flag on one table; whether an index needs a concurrent build is a property of the moment rather than of the index, and the delta is where that is known. JasperFx/weasel#594 proposes it there instead.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
Whether an index has to be built without blocking writes is not a property of the index. It is whether the table is being created — nothing to scan, and the statement belongs inline in the
CREATE TABLEscript — or altered, where there are rows to scan andACCESS EXCLUSIVEfor the whole build is a write outage rather than a migration.Today that decision is made where the index is declared, through
IsConcurrent, so every consumer has to anticipate it per index. Marten already carriesBuildHStoreTagIndexConcurrentlyfor one index onmt_events(JasperFx/marten#5268) and I was about to add a second flag for another (marten#5453) before it became obvious that a third index would want a third flag.TableDeltaalready knows which of the two cases it is in.What this does
PostgresqlMigrator.BuildIndexesConcurrentlyOnAlter. With it on, an index added to — or changed on — a table that already exists is built concurrently without anyone marking it:The create path is deliberately untouched:
WriteCreateStatementstill writes what the index itself declares, so an empty table is not paid for with a concurrent build it cannot benefit from — and, on a partitioned parent, one PostgreSQL would refuse outright.ToCreateSql(Table, bool)is the seam, with the caller's answer authoritative in both directions;ToCreateSql(Table)keeps taking it fromIsConcurrent, so nothing changes for existing callers.Why it is off by default
It changes what a generated patch script is, not just how fast it runs:
PostgresqlMigratoris unaffected — that already splits on the index-creation markers and each statement auto-commits);I would argue for on-by-default eventually — a blocking build against a live table is never what anyone wanted — but that is your call to make, not something to slip in. Say the word and I will flip it.
Tests
adding_an_index_to_an_existing_table— concurrent with the option on, blocking with it off, and the create path left blocking even with the option on.IndexDefinitionTestspinning that theconcurrentlyargument decides in both directions.Weasel.Postgresql.Testssuite green locally: 987 passed, 3 skipped, against PostgreSQL 17.Knock-on, if you take it
BuildHStoreTagIndexConcurrentlybecomes redundant, and the docs that tell people to set it (Marten'sdocs/events/dcb.md) would want a pass. I have left both alone here — happy to follow up in Marten once you have decided what you want this to look like.