Skip to content

fix(prefill): isolate cache budget policy from MLX runtime - #172

Merged
FluffyAIcode merged 1 commit into
mainfrom
AgentMemory/tiered-kv-offload-0716
Jul 16, 2026
Merged

fix(prefill): isolate cache budget policy from MLX runtime#172
FluffyAIcode merged 1 commit into
mainfrom
AgentMemory/tiered-kv-offload-0716

Conversation

@FluffyAIcode

Copy link
Copy Markdown
Owner

Summary

  • move adaptive cache-budget arithmetic into a platform-neutral distributed module
  • keep the MLX worker entrypoint responsible only for runtime memory measurement and applying the policy
  • allow Linux CI to collect and test budget logic without importing mlx.core

Why

PR #171 auto-merged despite a failed Linux collection gate because its test imported the MLX-only worker script. The production implementation and Mac integration passed; this follow-up restores the platform-neutral test contract.

Tests

  • focused budget/cache tests — 10 passed
  • full platform-neutral selection — 837 passed, 1 skipped
  • Python compile and diff checks pass

Made with Cursor

Keep adaptive memory arithmetic importable on Linux without loading the MLX worker entrypoint, preserving the platform-neutral CI contract.

Co-authored-by: Cursor <cursoragent@cursor.com>
@cursor

cursor Bot commented Jul 16, 2026

Copy link
Copy Markdown

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

@FluffyAIcode
FluffyAIcode merged commit 9212de1 into main Jul 16, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant