[RFC] Pod lifecycle subscription through data layer

[vMaroon]

Motivation

For Active-Active Multi-Replica HA enablement in llm-d's precise prefix-cache indexing, a new sub-system is introduced to handle discovery and subscription to vLLM KVEvent publishers.

For the discovery part, informer-like functionality is required. For standalone deployments, I appended a controller example, but for proper integration with the llm-d inference-scheduler, the logic should be provided through the data-layer.

Proposal

After discussions with @elevran, we propose adding a Pod lifecycle source into the data-layer, for the precise-prefix-cache scorer to subscribe to.

cc @elevran @kfswain

[danehans]

@vMaroon will you be creating a design spec for these proposed changes?

[vMaroon]

@danehans I was hoping someone can pick this up as it is important for the llm-d v0.5 roadmap. I know Etai and Kellen had previously thought about this use-case - so perhaps someone can followup?

@elevran @kfswain

[elevran]

plan to follow up after the holiday.
Happy to collaborate/pair on this if there's interest.