Skip to content

Fix: needs_schema_definitions_from_json triggers full rebuild on every incremental build#1711

Closed
vanhci wants to merge 1 commit into
useblocks:masterfrom
vanhci:fix/1711-schema-definitions-deepcopy
Closed

Fix: needs_schema_definitions_from_json triggers full rebuild on every incremental build#1711
vanhci wants to merge 1 commit into
useblocks:masterfrom
vanhci:fix/1711-schema-definitions-deepcopy

Conversation

@vanhci
Copy link
Copy Markdown

@vanhci vanhci commented May 20, 2026

Summary

Bug: When needs_schema_definitions_from_json is set, every incremental sphinx-build rebuilds ALL documents because Sphinx detects a spurious [config changed ('needs_schema_definitions')].

Root Cause: resolve_schemas_config is connected to env-before-read-docs, and via populate_field_type mutates needs_config.schema_definitions['schemas'] in place — injecting "type": "string" / "object" etc. into every nested schema. This event fires AFTER Sphinx's config-inited config-change checkpoint:

  1. config-initedschemas.json loaded → {"const": "role"}
  2. Sphinx compares pickled config vs in-memory config ← still sees {"const": "role"}
  3. env-before-read-docspopulate_field_type mutates in place → {"const": "role", "type": "string"}
  4. End of build → pickle persists the mutated config

Next build: step 2 compares fresh JSON ({"const": "role"}) vs pickled mutated config ({"const": "role", "type": "string"}) → mismatch → full rebuild → permanent loop.

Fix: In resolve_schemas_config, deep-copy each schema with copy.deepcopy() before passing it to populate_field_type(), so the original config remains unchanged and the pickled config matches the fresh load.

Changes

  • File: sphinx_needs/schema/config_utils.py
  • Lines: Added import copy at line 5, modified resolve_schemas_config() (around line 103) to pass copy.deepcopy(schema) instead of schema directly to populate_field_type()

Testing

The fix was verified by understanding the mutation flow: populate_field_type injects type fields into schemas, and by deep-copying before mutation, the original pickled config remains identical across builds.

Fixes #1711

…utation

In resolve_schemas_config(), schemas from needs_config.schema_definitions['schemas']
were passed directly to populate_field_type(), which mutates them in place by injecting
'type' fields. Since resolve_schemas_config is connected to 'env-before-read-docs'
(which fires after Sphinx's config-inited checkpoint), the mutations persisted in the
pickled config. On subsequent builds, Sphinx detected a difference between the fresh
JSON-loaded config and the pickled mutated config, triggering a full rebuild every time.

Fix: Use copy.deepcopy() on each schema before passing to populate_field_type(), so
the original config remains unchanged and the pickled config matches the fresh load.
@ubmarco
Copy link
Copy Markdown
Member

ubmarco commented May 20, 2026

Thanks for the PR.
Closed in favor of #1712
Will release this today.

@ubmarco ubmarco closed this May 20, 2026
@ubmarco
Copy link
Copy Markdown
Member

ubmarco commented May 20, 2026

Release is out!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants