Repository navigation
Lexicon: incremental update and rollback (mirror of #36, crash-safe swap) - #43
Merged
Merged
Conversation
…-rollback) magiah lexicon --update recounts only the books added, changed or removed since the last update (seforim.db: contentHash + line count; files: size + mtime), keeping per-book counts in a side file, lexicon_books.pkl. The result equals a full count. --rollback swaps in the version the last update replaced; --status reports what an update would do. Plain magiah lexicon is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
40% smaller side file on the Otzaria library than a compressed pickle. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # magiah/cli.py
The swap moved files one by one, so a failure midway could leave lexicon.pkl missing or mismatched with names.pkl and lexicon_prev half filled. Now one transaction (_exchange) serves --update and --rollback: current files go to lexicon_old/, new ones into place, a journal (lexicon_txn.json) is removed as the commit, then lexicon_old/ becomes lexicon_prev/. Any failure before the commit puts everything back; a killed process is repaired by recover() on the next update, rollback or plain lexicon run. OSError during an update or rollback is now a Hebrew StageError (exit 1); a leftover lexicon_new is cleaned up, or reported clearly when it cannot be removed. Tests inject OSError at every os.replace of update and rollback (single failure and killed process), leftover lexicon_new, write errors, and the CLI output. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Mirror of #36 (fork PR; CodeQL does not run on forks) plus QA fixes: crash-safe transactional swap with a journal and recovery, file errors reported as StageError (Hebrew message, exit 1), leftover lexicon_new cleaned up. Merged with main; the cli.py conflict kept both the lexicon-update and the extra/guard options.
🤖 Generated with Claude Code