Skip to content

Add memoization cache to get_id() with signal-based invalidation#266

Merged
cmutel merged 2 commits into
mainfrom
feat/get-id-cache
May 13, 2026
Merged

Add memoization cache to get_id() with signal-based invalidation#266
cmutel merged 2 commits into
mainfrom
feat/get-id-cache

Conversation

@cmutel
Copy link
Copy Markdown
Member

@cmutel cmutel commented May 13, 2026

Summary

  • Adds a module-level dict cache to get_id() for (database, code) → id lookups, avoiding repeated SQLite queries during large imports (reported to save ~1 minute on ecoinvent imports in Adding memoization (cache) to 'get_id()', saves 1 minute per ecoinvent import. #254)
  • Cache is invalidated via blinker signal handlers on all relevant events: project switch, database delete/reset/write, individual activity deletion, and activity code or database changes
  • Chose a plain dict over functools.lru_cache because selective per-database and per-activity invalidation requires __delitem__, which lru_cache does not expose

Test plan

  • test_get_id_cache_populates — cache is written on first call
  • test_get_id_cache_returns_same_value — cache hit returns correct value
  • test_get_id_cache_project_changedproject_changed clears entire cache
  • test_get_id_cache_database_deleteon_database_delete removes only that database's entries
  • test_get_id_cache_database_reseton_database_reset removes only that database's entries
  • test_get_id_cache_database_writeon_database_write removes only that database's entries
  • test_get_id_cache_activity_deletesignaleddataset_on_delete removes only that activity's entry
  • test_get_id_cache_activity_code_changeon_activity_code_change removes old key, leaves others
  • test_get_id_cache_activity_database_changeon_activity_database_change removes old key, leaves others

Closes #254

raphaeljolivet and others added 2 commits May 13, 2026 10:45
Caches (database, code) → id lookups in a module-level dict to avoid
repeated SQLite queries during large imports (e.g. ecoinvent).

Cache is invalidated on project switch, database delete/reset/write,
individual activity deletion, and activity code/database changes via
blinker signal handlers. Tests cover all invalidation paths.

Closes #254
@cmutel cmutel force-pushed the feat/get-id-cache branch from d75a1a4 to d39c449 Compare May 13, 2026 08:52
@cmutel cmutel merged commit 89f9b75 into main May 13, 2026
9 checks passed
@cmutel cmutel deleted the feat/get-id-cache branch May 13, 2026 09:23
This was referenced May 13, 2026
@cmutel cmutel self-assigned this May 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants