[15.0][OU-ADD] ptplus_accounting + ptplus_stock: migration scripts (15.0.5.0.0)#12
Open
eantones wants to merge 7 commits into
Open
[15.0][OU-ADD] ptplus_accounting + ptplus_stock: migration scripts (15.0.5.0.0)#12eantones wants to merge 7 commits into
eantones wants to merge 7 commits into
Conversation
The 15.0.5.0.0 vendor upgrade adds this stored computed field on
account.move.line. On a large database the ORM's column-creation mass
compute ("Storing computed values") loads millions of rows into the
cache on top of ~300 modules' worth of upgrade state and the process is
OOM-killed. Pre-creating the column in a pre-migration makes _auto_init
skip the mass init (the openupgradelib.add_fields mechanism); the field
is recomputed in controlled batches at the target version.
The 15.0.5.0.0 vendor upgrade adds this stored computed field on stock.picking. Same mass-compute OOM as ptplus_accounting's l10n_pt_account_expected_balance: pre-creating the column makes the ORM skip the whole-table init; the field is recomputed in controlled batches at the target version.
…te before rename Databases that already carry l10n_pt_vat_adjustment_norm_id alongside the old vat_adjustment_norm_id duplicate crash the module's own pre-migration: its openupgrade.rename_fields() finds the target column already present. Drop the old duplicate first, guarded to abort if it holds any data. The pre-00- basename makes it run before the module's own pre-migration.py (same-version stage scripts execute in basename order).
…balance Fill the field pre-created empty by pre-migration.py, in memory-safe chunks using the model's own compute via the ORM (add_to_compute + flush), so the vendor's real compute logic runs without ever reading the packaged code. Runs in post-migration at 15.0.5.0.0, so the field is created AND filled within this same hop — no cross-version dependency, each hop self-contained.
Fill the field pre-created empty by pre-migration.py, in memory-safe chunks using the model's own compute via the ORM (add_to_compute + flush), so the vendor's real compute logic runs without ever reading the packaged code. Runs in post-migration at 15.0.5.0.0, so the field is created AND filled within this same hop — no cross-version dependency, each hop self-contained.
@openupgrade.migrate() wraps the script in a savepoint: a cr.commit() per chunk destroys it (RELEASE SAVEPOINT crashes on script exit, aborting the whole upgrade at the finish line) and breaks per-module transactionality. Memory stays bounded by the per-chunk cache invalidation alone; the recompute measured ~3 minutes for 2.28M rows.
Same savepoint/transactionality fix as ptplus_accounting: the decorator wraps the script in a savepoint that a mid-loop commit destroys. Cache invalidation per chunk alone keeps memory bounded.
eantones
added
the
local-only
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Split of #3 (fork-only part). Migration scripts for the Oxigen PT Plus custom modules: pre-create + batched recompute of
l10n_pt_account_expected_balance/l10n_pt_is_national, VAT-adjustment-norm dedup, no-commit fix inside the batched recomputes. Never intended for OCA — these modules are customer-specific.Files (disjoint from the sibling split PRs):
openupgrade_scripts/scripts/ptplus_accounting/15.0.5.0.0/*+openupgrade_scripts/scripts/ptplus_stock/15.0.5.0.0/*.