Skip to content

docs: correct batch endpoint, feature store, and traffic splitting claims - #1833

Open
apurvapatkeshwar wants to merge 1 commit into
michelangelo-ai:mainfrom
apurvapatkeshwar:docs/correct-batch-feature-traffic-claims
Open

docs: correct batch endpoint, feature store, and traffic splitting claims#1833
apurvapatkeshwar wants to merge 1 commit into
michelangelo-ai:mainfrom
apurvapatkeshwar:docs/correct-batch-feature-traffic-claims

Conversation

@apurvapatkeshwar

Copy link
Copy Markdown
Contributor

Summary

Follow-up in the same spirit as #1788 (Ray Tune / autoscaling corrections): three more places where the getting-started docs describe capabilities that are on the roadmap but not in the code yet.

  1. overview.md - the ML workflow mapping table says Batch Inference maps to "Deployment to batch endpoint". There is no batch endpoint resource anywhere in proto/ or go/ (grep for it is empty), and the roadmap doesn't list one either. Offline inference today is a hand-written Uniflow task using Ray - that's what python/examples/llm_prediction/ does. The row now says Uniflow tasks with Ray for offline inference.

  2. faq.md - "Yes, Michelangelo AI integrates with feature stores" contradicts the roadmap, which lists the entire Feature Store section (online + offline store, drift monitors, lineage) under Planned. No feature-store integration exists in code; the only trace is a forward-looking feature_store_features schema field in model.proto that references a Feature resource type that doesn't exist. The answer now says not yet, it's on the roadmap, and describes what you can do today (data prep pipelines, data catalog).

  3. glossary.md - the Deployment entry claims "traffic splitting and canary rollout" and the Endpoint entry claims "traffic splitting across model revisions (e.g., for A/B testing)". In code, strategies/strategy.go has exactly one strategy case (rolling), and TrafficRoutingActor adds a path-based route per deployment (/cluster/{inferenceServer}/{deployment}) - there is no weighted split or canary anywhere. The roadmap itself lists "Endpoint traffic splitting and shadow routing" and "Deployment rollout strategies (Blast, Zonal, Shadow/A-B)" as Planned. Both entries now describe rolling rollout / name-based routing and defer splitting to the roadmap.

Why it matters

Same rationale as #1788: users evaluating a migration (from SageMaker/Vertex especially, where batch transform and canary endpoints are table stakes) will make decisions off these three sentences. The roadmap already tells the honest story; this aligns the other docs with it.

Testing

  • 3 files, +4/-4 lines, prose only - no code changes.
  • Verified each claim against code at current main before rewording: no batch endpoint type in proto/go, no feature-store implementation, single rolling strategy case, path-based (not weighted) traffic routing.
  • Relative link style (./roadmap.md) matches the existing links in faq.md.

…aims

Three claims in the getting-started docs are ahead of the code:

- overview.md maps Batch Inference to "Deployment to batch endpoint", but
  no batch endpoint resource exists anywhere in proto/ or go/. Offline
  inference today is a Uniflow task using Ray (see
  python/examples/llm_prediction/).
- faq.md says the platform "integrates with feature stores", while the
  roadmap lists the entire Feature Store surface as Planned and no
  feature-store integration exists in code.
- glossary.md says Deployments support "traffic splitting and canary
  rollout" and Endpoints support "traffic splitting across model
  revisions". The only implemented strategy is rolling
  (strategies/strategy.go has a single case), and TrafficRoutingActor
  does path-based routing per deployment, not weighted splits. The
  roadmap lists traffic splitting/shadow routing and additional rollout
  strategies as Planned.

Reword all three to describe what ships today and point at the roadmap,
same as the Ray Tune correction in michelangelo-ai#1788.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant