Skip to content

Conversation

@nabinchha
Copy link
Contributor

@nabinchha nabinchha commented Dec 18, 2025

Strategy:
Create two facades as proxy for modules that are heavy

  • src/data_designer/engine/__init__.py is a facade for all engine related public resources.
  • src/data_designer/lazy_imports.py is a facade for external deps like pandas, pyarrow, etc.

Use resources through the facade:

from data_designer import lazy_imports

lazy_imports.pd.read_parquet(...)
from data_designer import engine

engine.ArtifactStorage.mkdir_if_needed(self._artifact_path)

Before after show roughly 92% load time improvement down to 1.6s from 18.3s.

may be we can name lazy_imports > lazy_third_party_imports but that's long.

We can change the usage above to be like the current pattern, but it has a slight perf hit (~2.8s):

Use resources through the facade:

from data_designer.lazy_imports import pd

pd.read_parquet(...)
from data_designer.engine import ArtifactStorage

ArtifactStorage.mkdir_if_needed(self._artifact_path)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants