Rewrite placeholder as AI Alpha Research Lab - #4
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
This PR rewrites the repository into a small, auditable “formula alpha research lab” with a clean-room expression evaluator, a deterministic synthetic demo, a strict promotion gate (OOS IC / turnover / net return), and CI + regression tests to enforce evidence boundaries.
Changes:
- Adds an allow-listed AST-based formula evaluator and a strict candidate evaluation/promotion gate.
- Introduces a deterministic synthetic panel demo and unittest-based regression/safety tests.
- Adds repository documentation/evidence records, packaging metadata, and a GitHub Actions test workflow.
Reviewed changes
Copilot reviewed 11 out of 12 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_research.py | Adds regression/safety tests for formula evaluation and the promotion gate. |
| src/ai_alpha_lab/research.py | Implements candidate diagnostics and promotion gating logic. |
| src/ai_alpha_lab/expressions.py | Implements the allow-listed AST formula evaluator and contract enforcement. |
| src/ai_alpha_lab/demo.py | Provides deterministic synthetic data and a demo run. |
| src/ai_alpha_lab/init.py | Exposes the intended public API surface. |
| research_state.json | Records the repo’s evidence boundary and current “closed” status. |
| README.md | Replaces placeholder with bilingual overview, quickstart, and evidence boundary. |
| pyproject.toml | Defines packaging, dependencies, and src-layout configuration. |
| LICENSE | Adds MIT license. |
| evidence/validation-summary.md | Adds a compact public evidence summary for the initial record. |
| .gitignore | Adds standard Python ignores and artifacts directory. |
| .github/workflows/test.yml | Adds CI running editable install + unittest discovery. |
Comments suppressed due to low confidence (3)
src/ai_alpha_lab/research.py:36
- Same pandas-compatibility issue here:
include_groups=Falsewill break when running under the declared minimumpandas>=2.0.
returns = panel.assign(position=positions).groupby("date", sort=False).apply(
lambda group: group.loc[group["position"] > 0, "forward_return"].mean(),
include_groups=False,
)
src/ai_alpha_lab/expressions.py:55
ast.Constanttreats booleans as ints (isinstance(True, int) == True), so expressions likemean(returns, True)are currently accepted and silently treated as a 1-day window. If the intent is “integer window only”, explicitly rejectbool.
if isinstance(node, ast.Constant) and isinstance(node.value, (int, float)):
return node.value
src/ai_alpha_lab/expressions.py:64
- Scalar division by zero can still raise a Python
ZeroDivisionError(e.g.1 / 0), which escapes the evaluator as an unhandled exception rather than aFormulaError. This also makes behavior inconsistent with the Series division path (which is sanitized to NaN).
return left / right.replace(0, float("nan")) if isinstance(right, pd.Series) else left / right
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+22
to
+24
| return data.groupby("date", sort=False).apply( | ||
| lambda group: group["signal"].rank().corr(group["forward_return"].rank()), include_groups=False | ||
| ) |
Comment on lines
+49
to
+53
| if node.id not in contract.allowed_fields or node.id not in frame.columns: | ||
| raise FormulaError(f"field is not allowed: {node.id}") | ||
| if "future" in node.id.lower() or "label" in node.id.lower(): | ||
| raise FormulaError(f"future-looking field is forbidden: {node.id}") | ||
| return frame[node.id].astype(float) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes
Verification
python -m ai_alpha_lab.demoreports positive gross diagnostic return but negative cost-aware net return, so the candidate is rejected.python -m unittest discover -s tests -vpasses 5 local tests.testpassed on the current branch head.Boundary
No employer code, private data, credentials, raw experiment data, or claim of tradable performance is included.