Autoresearch

[ludovicc]

Add an autoresearch harness based on https://github.com/karpathy/autoresearch and https://github.com/hwchase17/autoresearch-agents

[ludovicc]

@jjang-ai what do you think about that autoresearch process? We can use autoresearch to select the best parameters for a model then store them as default parameters for that model.

[jjang-ai]

This is a great idea, I would totally use something like this. Sorry for late reply.

[jjang-ai]

Thanks for this — the autoresearch concept is really interesting and I'd love to use something like this for automated parameter/prompt optimization.

However, can't merge as-is for a few reasons:

1. vmlx_engine/simple.py import change breaks our codebase — changing from ..api to from .api is specific to your fork's directory layout. Our simple.py lives inside vmlx_engine/engine/simple.py so the ..api relative import is correct.

2. Hardcoded paths — /Users/ludovicclaude/Exploration/vmlx appears in the README and notebook.

3. Binary artifacts — the .png images and .ipynb notebook outputs shouldn't be in the main repo.

4. Dev dependencies in pyproject.toml — ipykernel and matplotlib are not needed for the core engine.

Suggestion: This would work better as either:

• A standalone repo (vmlx-autoresearch) that imports vmlx as a dependency
• Or a contribution to docs/examples/autoresearch/ with just the Python scripts and README (no binaries/notebooks/hardcoded paths), and without modifying core engine files

Happy to revisit if you want to rework it along those lines.