Wire model-config-datasource into the runner and helm chart.

[Mohammad-nassar10]

What type of PR is this?
/kind feature

What this PR does / why we need it:
Register model-config-datasource plugin in the runner and expose a way to ship the JSON file through the chart, so users use it to seed candidate models.

Which issue(s) this PR fixes:

Fixes #

Release note (write NONE if no user-facing change):

NONE

[nirrozenbaum]

it's not clear what is listModels from this code.
can you please add the relevant documentation to the readme.MD inside the helm chart dir?
the optional flags should be documentation so whoever uses the helm can configure IPP easily.

[Mohammad-nassar10]

Good point, added, thanks!

[nirrozenbaum]

@Mohammad-nassar10 thanks for adding the doc.
now that I understand what was the intention in the PR - this looks somehow very limiting - the list of models is static, there is no way to add a model dynamically (nor remove). it seems like an easy way for a PoC but not the proper way moving forward.

the payload processing config is intended to be used for plugins configuration.
seems like this PR is making the config define the source of truth of available models (this is not a plugin, although one could argue data collector is).

If we take it to EPP's world to compare just to make the point -
think how it would look like if one would have to configure the available endpoints of EPP through the plugins config file. this sounds incorrect to me.

alternative direction that could work:
add a configmap that stores models. configmap could also add metadata for every model entry (e.g., like its cost per 1M tokens). add a controller that watches the "modelsList" configmap(s) and fills that data into the datastore.

[shmuelk]

@nirrozenbaum wrote:

the list of models is static, there is no way to add a model dynamically (nor remove). it seems like an easy way for a PoC but not the proper way moving forward.

The updates to the README.md state:

payloadProcessor.listModels is a list of model names that the IPP should be aware of. When set, the chart renders a models.json file into the ConfigMap, which is mounted at /config/models.json inside the pod. The model-config-datasource plugin reads this file to populate the IPP datastore and watches it for live updates on ConfigMap remounts.

That is not part of the IPP's configuration ConfigMap. It is a separate ConfigMap. If and how that ConfigMap is checked for changes is a different thing, which should be handled separately from this PR.

[ronenkat]

@Mohammad-nassar10 thanks for adding the doc. now that I understand what was the intention in the PR - this looks somehow very limiting - the list of models is static, there is no way to add a model dynamically (nor remove). it seems like an easy way for a PoC but not the proper way moving forward.

the payload processing config is intended to be used for plugins configuration. seems like this PR is making the config define the source of truth of available models (this is not a plugin, although one could argue data collector is).

If we take it to EPP's world to compare just to make the point - think how it would look like if one would have to configure the available endpoints of EPP through the plugins config file. this sounds incorrect to me.

alternative direction that could work: add a configmap that stores models. configmap could also add metadata for every model entry (e.g., like its cost per 1M tokens). add a controller that watches the "modelsList" configmap(s) and fills that data into the datastore.

We already have a modelconfigcollector data source that reads a models-config.json file when ever it changes.
This PR enables IPP to run with a pre-defined list of models.
There could be multiple ways to configure IPP with models, this is just the first simple option.