Management API for profiler agent in POD/Container

[IldarMinaev]

Is your feature request related to a problem? Please describe

It is required to manage profiler agent (enable/disable and change configuration) without POD/Container restart.

Requirements:

• Hot enable/disable — no POD/Container restart to enable/disable profiling.
• Hot reloading — no POD/Container restart to update configuration.
• User-friendly way to manage enabling/disabling, configuration changes and configuration apply.

Describe the solution you'd like

My suggestion to implement centralized configuration management for profiler agent and use CDT to store and manage the configuration.

During startup the application with agent, use default configuration (stored as files with the agent) for instrumentation and install agent by entrypoint script based on PROFILER_ENABLED env variable as it already done.

During runtime javaagent to watch changes on config files (via inotify) to enable hot reload.

After javaagent started, it should connect to profiler configuration management service to get updated configuration (including enabling/disabling) and do hot apply of configuration.

It is expected what profiler configuration management service able to store/provide special configuration per namespace and/or microservice name.

Open questions:

• Check possibility to use OpAMP protocol
• is it good idea to start the agent in a kind of "standby state" with deactivated features and activate these at a later point in time?

Describe alternatives you've considered

• Store configuration in mounted configmap.
• Use special sidecar container for hot reload of configuration changes to main container with java application and agent

Additional information

No response