feat: implement self-healing Python deployment patterns and runtime modules#13
feat: implement self-healing Python deployment patterns and runtime modules#13badMade wants to merge 7 commits into
Conversation
…odules Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request implements a self-healing module for Python applications, providing environment validation, automated configuration repair, resilience patterns (circuit breakers and retries), and health check endpoints. The review identifies several critical issues, including a dangerous side effect of installing dependencies during module import, thread-safety vulnerabilities in the circuit breaker, and the risk of configuration corruption due to non-atomic file writes. Additionally, the feedback suggests optimizing the configuration repair logic and making disk space check paths configurable to avoid issues with hardcoded root directory paths.
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
|
@copilot code review |
|
@claude code review |
There was a problem hiding this comment.
Pull request overview
Adds a new “self-healing” Python subsystem (runtime modules + packaging + CI workflows) intended to improve deployment reliability and runtime robustness, and updates the README to document these features.
Changes:
- Introduces
src/selfheal/modules for environment validation, config healing, health endpoints, and resilience helpers (retry/circuit breaker). - Adds GitHub Actions workflows for CI (ruff/mypy/pytest + deploy health check rollback) and a “self-heal” workflow_run automation.
- Adds Python packaging/config via
pyproject.tomlplus new unit tests for the selfheal components.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 15 comments.
Show a summary per file
| File | Description |
|---|---|
src/selfheal/__init__.py |
Public exports + auto-install behavior for dependencies |
src/selfheal/config_healer.py |
Pydantic-based config load/repair/regen logic |
src/selfheal/env_validator.py |
Startup environment validation checks |
src/selfheal/health.py |
Flask/FastAPI health endpoints and readiness checks |
src/selfheal/resilience.py |
Tenacity retry + circuit breaker implementation |
tests/test_selfheal.py |
Pytest coverage for validator/config/resilience |
.github/workflows/ci.yml |
CI + deploy + health-check rollback |
.github/workflows/self_heal.yml |
Workflow_run “self heal” automation (branch/PR flow) |
pyproject.toml |
Project deps + ruff/mypy config + hatchling packaging |
README.md |
Documents “Self-Healing Features” |
src/query_engine.py |
Tweaks structured output typing (Mapping) |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Agent-Logs-Url: https://github.com/badMade/claw-code/sessions/992def0f-49d7-4011-a01c-1376ba4d91c5 Co-authored-by: badMade <106821302+badMade@users.noreply.github.com>
Agent-Logs-Url: https://github.com/badMade/claw-code/sessions/992def0f-49d7-4011-a01c-1376ba4d91c5 Co-authored-by: badMade <106821302+badMade@users.noreply.github.com>
Agent-Logs-Url: https://github.com/badMade/claw-code/sessions/992def0f-49d7-4011-a01c-1376ba4d91c5 Co-authored-by: badMade <106821302+badMade@users.noreply.github.com>
|
@codex Code Review |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 11 out of 12 changed files in this pull request and generated 7 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if "structlog" not in sys.modules: | ||
| structlog = ModuleType("structlog") | ||
|
|
||
| class _NoopLogger: | ||
| def info(self, *args: object, **kwargs: object) -> None: | ||
| pass | ||
|
|
||
| def warning(self, *args: object, **kwargs: object) -> None: | ||
| pass | ||
|
|
||
| def error(self, *args: object, **kwargs: object) -> None: | ||
| pass | ||
|
|
||
| def critical(self, *args: object, **kwargs: object) -> None: | ||
| pass | ||
|
|
||
| def _get_logger(*args: object, **kwargs: object) -> _NoopLogger: | ||
| return _NoopLogger() | ||
|
|
||
| structlog.get_logger = _get_logger # type: ignore[attr-defined] | ||
| sys.modules["structlog"] = structlog | ||
|
|
||
| if "tenacity" not in sys.modules: |
There was a problem hiding this comment.
These dependency stubs check sys.modules to decide whether to create fake structlog/tenacity/pydantic_settings modules. sys.modules only reflects what has been imported, so this will shadow the real installed libraries in most test runs and can hide integration issues. Prefer try: import ... / except ImportError: (or importlib.util.find_spec) before injecting stubs.
| class SelfHealingConfig(BaseSettings): | ||
| """ | ||
| Pydantic BaseSettings subclass that self-heals its configuration file. | ||
| It will auto-regenerate missing files, backup+repair corrupt ones, | ||
| and raise explicitly for missing secrets. | ||
| """ | ||
|
|
||
| @classmethod | ||
| def load_or_heal( | ||
| cls: Type[T], | ||
| config_path: str, | ||
| sensitive_fields: Optional[list[str]] = None | ||
| ) -> T: | ||
| sensitive_fields = sensitive_fields or [] | ||
| path = Path(config_path) | ||
|
|
||
| # 1. Regenerate missing config | ||
| if not path.exists(): | ||
| logger.warning("Config file missing, generating defaults", path=config_path) | ||
| return cls._generate_and_save(path, sensitive_fields) | ||
|
|
There was a problem hiding this comment.
sensitive_fields is accepted and threaded through helpers, and the class docstring claims it will "raise explicitly for missing secrets", but the parameter is never used to enforce/validate anything. Either implement secret-field enforcement (e.g., require non-default/non-empty values and avoid writing them to disk) or remove the parameter and update the docstring to match behavior.
| gh api \ | ||
| --method POST \ | ||
| -H "Accept: application/vnd.github+json" \ | ||
| /repos/${{ github.repository }}/git/refs/heads/main \ | ||
| -f sha=${{ github.event.before }} \ | ||
| -F force=true |
There was a problem hiding this comment.
The rollback gh api call uses POST /git/refs/heads/main, which creates refs; updating main typically requires PATCH /git/refs/heads/main (or /git/refs/{ref}) with sha/force. As written this will likely fail and leave deployments un-rolled back.
| ignore_missing_imports = true | ||
|
|
||
| [tool.hatch.build.targets.wheel] | ||
| packages = ["src/selfheal"] |
There was a problem hiding this comment.
Packaging config looks inconsistent with the repo's import paths: the code imports src.selfheal, and src/ is itself a Python package (has src/__init__.py). Limiting wheel packages to src/selfheal may produce an install that doesn't provide src.selfheal correctly. Consider packaging the src package (or adjusting package/module layout) so installed imports match runtime imports.
| packages = ["src/selfheal"] | |
| packages = ["src"] |
| This project includes robust self-healing patterns for both the deployment pipeline and the Python runtime. | ||
|
|
||
| ### CI/CD Self-Healing (.github/workflows) | ||
| - **Automatic rollback:** Health checks ping `/health` with exponential backoff post-deploy. If they fail after 10 attempts, the deploy is rolled back automatically. |
There was a problem hiding this comment.
This section claims the deploy health-check uses exponential backoff, but the workflow uses a fixed retry-delay: 5s via jtalk/url-health-check-action. Either update the docs to match the actual behavior or adjust the workflow to implement exponential backoff.
| - **Automatic rollback:** Health checks ping `/health` with exponential backoff post-deploy. If they fail after 10 attempts, the deploy is rolled back automatically. | |
| - **Automatic rollback:** Health checks ping `/health` at a fixed retry interval post-deploy. If they fail after 10 attempts, the deploy is rolled back automatically. |
| # Loop prevention: Only trigger if the failing branch isn't already a selfheal branch | ||
| if: > | ||
| github.event.workflow_run.conclusion == 'failure' && | ||
| !startsWith(github.event.workflow_run.head_branch, 'selfheal-') | ||
| runs-on: ubuntu-latest |
There was a problem hiding this comment.
The self-heal workflow is described as running when "tests fail on main CI", but the current workflow_run trigger/condition will run for failures on any branch (including PR branches) as long as it isn't selfheal-*. If it's intended to only self-heal main, add a head_branch == 'main' check (or otherwise scope it).
| - name: Health Check | ||
| id: health_check | ||
| uses: jtalk/url-health-check-action@v5 | ||
| with: | ||
| url: "https://your-production-url.com/health" |
There was a problem hiding this comment.
The deploy health check targets the placeholder URL https://your-production-url.com/health, which will fail in real runs and make the deploy job (and rollback logic) trigger on every push to main. This should be parameterized (repo/environment variable) or replaced with the actual deployment URL before enabling this workflow on main.
| - name: Health Check | |
| id: health_check | |
| uses: jtalk/url-health-check-action@v5 | |
| with: | |
| url: "https://your-production-url.com/health" | |
| - name: Validate health check URL | |
| run: | | |
| if [ -z "${{ vars.PRODUCTION_HEALTH_URL }}" ]; then | |
| echo "::error::Repository or environment variable PRODUCTION_HEALTH_URL must be set to the deployed application's health endpoint." | |
| exit 1 | |
| fi | |
| - name: Health Check | |
| id: health_check | |
| uses: jtalk/url-health-check-action@v5 | |
| with: | |
| url: "${{ vars.PRODUCTION_HEALTH_URL }}" |
|
@codex code review |
|
@copilot, resolve the merge conflicts in this pull request. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7482e0dbea
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| import os | ||
| import sys | ||
| import subprocess | ||
| import structlog |
There was a problem hiding this comment.
Importing structlog at module load time defeats the advertised self-healing path: in any environment where structlog is missing, import src.selfheal fails before the SELFHEAL_AUTO_INSTALL logic can run, so dependencies are never installed and the package cannot bootstrap itself. This is user-facing in fresh CI/runtime images and should be fixed by moving or guarding that import until after dependency detection/install.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
@copilot, make changes based on the above suggestion.
There was a problem hiding this comment.
@copilot, make changes based on the above suggestion.
| --method POST \ | ||
| -H "Accept: application/vnd.github+json" \ | ||
| /repos/${{ github.repository }}/git/refs/heads/main \ |
There was a problem hiding this comment.
The rollback step calls gh api with --method POST against /git/refs/heads/main, but this path is for updating an existing ref, which GitHub expects as a PATCH request; POST is for creating refs at /git/refs. In a failed health check scenario this request will error instead of resetting main to github.event.before, so the workflow reports rollback but leaves the bad deploy in place.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
@copilot, make changes based on the above suggestion.
There was a problem hiding this comment.
@copilot, make changes based on the above suggestion.
Co-authored-by: badMade <106821302+badMade@users.noreply.github.com>
Resolved merge conflicts by merging |
|
@copilot, make changes based on:
See: #13 (comment) |
|
@copilot, make changes based on:
See: #13 (comment) |
Implements self-healing patterns for Python project, including:
PR created automatically by Jules for task 15047887301401954275 started by @badMade