Centralized configuration management

[poxet]

It is a bit messy to keep track of all servers and how Influx-Capacitor is configured on them.

nathanwebb suggested a way of configuring them from one central point.

My idea here would be a simple service exposing a WebAPI that Influx-Capacitor can talk to. What you need to do then is just pointing Influx-Capacitor to the http address and the configurations will be read from there.

[nathanwebb]

Just thinking out loud here, but perhaps keeping it simple by only having the clients configured with the influxdb host and database as they currently are, and then doing a read of the influxdb to get the URL to download the configuration. This is an extension of the metadata concept (but reads instead of writes), and using influxdb for slow changing data (not just performance metrics).

So at a high level, the steps would be:

1. Influxdb is setup with a database (e.g. capacitor) and some 'configuration' data is loaded into a 'config' measurement, with each option differentiated by a tagkey of "option". This setup could be scripted initially, or even manually, and eventually done through an admin app.

e.g. "create database capacitor"

1. As part of the initial configuration, a new series is created where the value is the URL to get the XML config. The 'config' series will effectively contain the history of the URL, and is updated everytime the configuration is updated. For example, this could be called "config,option='config.url'"

e.g. "insert config,option='config.url' value='http://capacitor.domain/configuration/v1/config.xml'"

1. On the client, configure database.xml to point to influxdb, as per existing. No change here... (Agento - agento.org - uses an interesting approach in that it comes with a default configuration where the server's hostname defaults to "agento.{domain}". So if you've setup the server with a dns entry of agento, then there is zero configuration.)
2. When the Influx-capacitor service starts, it begins by doing a query against the 'config' series,
   e.g. "select * from config where option='config.url' order by time desc limit 1"
3. The influx-capacitor agent then retrieves the config and restarts.
4. At the same time, it can also update the metadata measurements as per usual. But in addition to storing the current version of the agent, it could also store the current version of the configuration.

So, some of the possibilities here:

• Central webpage to manage all agents.
• keep track of each agent, to see if they've downloaded the latest version of the config.
• easily update the configuration for all hosts, e.g. to add another set of metrics. Just add a new point to the config,option='config.url' series and the clients will download it automatically.
• provide different configuration for a subset of agents. I haven't gone into any of that detail here, but when the agent does a query for the config, it might be able to fetch a config that is specific for itself. So apart from the 'option' tagkey, there could also be a 'hostname' tagkey.
  E.g. most agents might poll every 60 seconds, but you could turn on "fast poll" for an agent, and it could drop down to 10 second polling for an hour or two, to assist with troubleshooting.
• Some agents could be configured with extra metrics
• Extend this concept to also download a new version of the software itself.

[poxet]

Never thought about storing the configuration in influxDB it self. Cool idea! :)

[Confusingboat]

Please pardon my potential ramblings...

As it currently sits, I've centralized the config management by deploying the XML config files, and even Influx-Capacitor itself, through Group Policy. Seems to work just fine.

I've actually built an entire service framework at my day job designed around the concept of central management that dynamically loads/unloads configs to determine which services should be running. The whole thing is quite complex in its detail, but very robust and kinda cool to watch when run in CLI mode.

The way I do it is by loading an XML config file on start, but then I poll the server for the latest version, pull it down, overwrite the XML config file I loaded on start, and then reload the new config from the same file location. Since the central server location is also in this file, you can update that value remotely as well. To avoid dependency between the functions, there are actually two separate logical loops to achieve this (I say logical because it all gets pumped through one central message/action pipeline loop): one for reloading the config every 30 seconds (or whatever the configured duration is), and one for constantly polling for the latest version from the central server. This might be the easiest way to implement the functionality.

It also seems like this would be best implemented (aka most easily implemented client-side) by offering a hosted Influx-Capacitor management service on the web, with the option of hosting one on-premise. Log in to your portal and manage your sites, server groups and whatnot with their various configurations, then just deploy your agents to do your bidding.

All that having been said, has there been any work done on this?