docs: correct batch endpoint, feature store, and traffic splitting claims - #1833
Open
apurvapatkeshwar wants to merge 1 commit into
Open
Conversation
…aims Three claims in the getting-started docs are ahead of the code: - overview.md maps Batch Inference to "Deployment to batch endpoint", but no batch endpoint resource exists anywhere in proto/ or go/. Offline inference today is a Uniflow task using Ray (see python/examples/llm_prediction/). - faq.md says the platform "integrates with feature stores", while the roadmap lists the entire Feature Store surface as Planned and no feature-store integration exists in code. - glossary.md says Deployments support "traffic splitting and canary rollout" and Endpoints support "traffic splitting across model revisions". The only implemented strategy is rolling (strategies/strategy.go has a single case), and TrafficRoutingActor does path-based routing per deployment, not weighted splits. The roadmap lists traffic splitting/shadow routing and additional rollout strategies as Planned. Reword all three to describe what ships today and point at the roadmap, same as the Ray Tune correction in michelangelo-ai#1788.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up in the same spirit as #1788 (Ray Tune / autoscaling corrections): three more places where the getting-started docs describe capabilities that are on the roadmap but not in the code yet.
overview.md - the ML workflow mapping table says Batch Inference maps to "Deployment to batch endpoint". There is no batch endpoint resource anywhere in
proto/orgo/(grep for it is empty), and the roadmap doesn't list one either. Offline inference today is a hand-written Uniflow task using Ray - that's whatpython/examples/llm_prediction/does. The row now says Uniflow tasks with Ray for offline inference.faq.md - "Yes, Michelangelo AI integrates with feature stores" contradicts the roadmap, which lists the entire Feature Store section (online + offline store, drift monitors, lineage) under Planned. No feature-store integration exists in code; the only trace is a forward-looking
feature_store_featuresschema field inmodel.protothat references aFeatureresource type that doesn't exist. The answer now says not yet, it's on the roadmap, and describes what you can do today (data prep pipelines, data catalog).glossary.md - the Deployment entry claims "traffic splitting and canary rollout" and the Endpoint entry claims "traffic splitting across model revisions (e.g., for A/B testing)". In code,
strategies/strategy.gohas exactly one strategy case (rolling), andTrafficRoutingActoradds a path-based route per deployment (/cluster/{inferenceServer}/{deployment}) - there is no weighted split or canary anywhere. The roadmap itself lists "Endpoint traffic splitting and shadow routing" and "Deployment rollout strategies (Blast, Zonal, Shadow/A-B)" as Planned. Both entries now describe rolling rollout / name-based routing and defer splitting to the roadmap.Why it matters
Same rationale as #1788: users evaluating a migration (from SageMaker/Vertex especially, where batch transform and canary endpoints are table stakes) will make decisions off these three sentences. The roadmap already tells the honest story; this aligns the other docs with it.
Testing
./roadmap.md) matches the existing links in faq.md.