Skip to content

🤖 Fix #51: docs: write 'First Benchmark Run' step-by-step walkthrough#63

Open
JacobPEvans-personal wants to merge 2 commits into
mainfrom
fix/issue-51-first-benchmark-run-walkthrough
Open

🤖 Fix #51: docs: write 'First Benchmark Run' step-by-step walkthrough#63
JacobPEvans-personal wants to merge 2 commits into
mainfrom
fix/issue-51-first-benchmark-run-walkthrough

Conversation

@JacobPEvans-personal
Copy link
Copy Markdown
Member

Closes #51

Problem

README.md shows a 3-step CLI block (lm-eval → dry-run → publish) but jumps over real-world
setup questions: how to bring up vllm-mlx + llama-swap, what max_gen_toks=4096 means, how to
handle slow models or timeouts, and the difference between dry-run and a real publish.

Approach

New docs/first-benchmark-run.md walks through a complete first run end-to-end in 7 steps
(model setup → inference check → lm-eval dry run → envelope inspection → dry-run publish →
real publish → HF Space viewer) with a Common Pitfalls section covering the most frequent
failure modes. Cross-link added in README's Usage section.

Files changed

  • docs/first-benchmark-run.md — new 7-step walkthrough doc with Common Pitfalls section
  • README.md — add getting-started callout linking to the walkthrough; also de-quotes HF_TOKEN placeholder for clarity

CI status

none — no CI workflows triggered on docs-only change

Self-review

This PR was drafted by Issue Solver and is opened as a DRAFT for human review before merge.
The Hard Rules in the prompt enforce: signed commits via Contents API, no dependency changes,
no infra/workflow edits, secret-pattern pre-flight scan.


Generated by Issue Solver — prompt source: https://github.com/JacobPEvans/claude-code-routines/blob/main/routines/issue-solver.prompt.md

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a comprehensive step-by-step walkthrough guide (docs/first-benchmark-run.md) for running a first MLX benchmark, from setting up the repository and verifying the inference stack to publishing and viewing results on Hugging Face. It also updates the README.md to link to this new guide and updates the placeholder for the Hugging Face token. There are no review comments to address, and I have no additional feedback to provide.

@JacobPEvans-personal JacobPEvans-personal marked this pull request as ready for review May 26, 2026 00:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs: write 'First Benchmark Run' step-by-step walkthrough

1 participant