Testing plan for live instance of statscache

[rtnpro]

Problem statement

statscache is already live. However, we need to test it thoroughly for possible bugs and issues before other apps start using it.

Possible solutions

• add functional tests to check data in datagrepper and compare stats in statscache.

[ralphbean]

For some history/exposition:

• We started here.
• @Yazman worked on the statscache codebase over the summer of 2015 and it went through a number of refactors. We were pretty confident at the end of his internship that the design was strong and that it could handle a number of edge cases we had come up with.
• I deployed it in November of 2015, but I never followed up to start using it and making sure it was working properly.
• This means it is probably not working properly (since software almost never works as you expect the first time).

Hence, the motivation for this ticket.. we need to validate that its doing the right thing and then start using it for stuff.

As an example of what not doing this costs us, in December 2015 @mattdm and @decause needed stats on various fedmsg-related things for a presentation at devconf.cz. Instead of programming what they needed into statscache-plugins and using the output of that (write once, and re-use) I wrote some one-off python scripts and produced csvs like we used to. We got the graphs, but multiple days were wasted on scripts that eventually get lost and thrown away. If I had done the work in statscache, we would have data free and ready for us in the statscache db when we need it again for Flock this year (and devconf again next year).

[sayanchowdhury]

I am planning to build a fedimg status dashboard to know the status of the AWS uploads (I am having a tough time to know which regions AMIs are uploaded and which are failing). So, I was planning to scrap off the releng code(except the AMI on), and build the stats too that is needed by @mattdm.

[pwnfoo]

Hi,

After discussing with @bexelbie, we added a CommOps SoC idea this time to generate the metrics required for the community as well as those required by @mattdm.

Please take a look at the ticket on Pagure[1] and let us know what you think @sayanchowdhury.

[1] https://pagure.io/fedora-commops/issue/105