Nightscout compat?

[bewest]

Would love to chat about integrating with Nightscout and open source dosing systems.
This is a compelling project, it would be great to garner more support for this.

[scaubrey]

Hi Ben, yes, happy to chat. I agree that integrating with Nightscout would be a great addition. It would expand reach with (I think) relatively low effort.

[scaubrey]

I see there is Nightscout API documentation. Is this doc up to date? Would integrating against the API described there work for some/most/all users?

[scaubrey]

@bewest I began an implementation for this by pulling treament data from the Nightscout API, but ran into an issue.

After some digging in the Nighscout github code, it seems that Tidepool stores basal data differently than Nightscout. Tidepool stores every basal event regardless of whether it is scheduled by the profile or it is a temp basal, but Nightscout seems to store just the basal profile and temp deviations from the profile. So computing total basal insulin from the NS treatment data requires analyzing profile(s) in a given day and then adjusting the total from the aggregate temp events. Given that analysis of implicit insulin delivery via basal profiles has inherent error-prone complexity and it is already core functionality in NS, I am hoping to avoid re-implementing the same logic in this package.

So my question: does Nightscout expose its Javascript functions for computing total basal (or total bolus or any other metrics for that matter) through the API that I can call directly? I scanned a couple of the yaml files I could find for the API (e.g. this v1 version) but I didn't see anything for computing statistics on treatments.

[bewest]

After some digging in the Nighscout github code, it seems that Tidepool stores basal data differently than Nightscout. Tidepool stores every basal event regardless of whether it is scheduled by the profile or it is a temp basal, but Nightscout seems to store just the basal profile and temp deviations from the profile. So computing total basal insulin from the NS treatment data requires analyzing profile(s) in a given day and then adjusting the total from the aggregate temp events. Given that analysis of implicit insulin delivery via basal profiles has inherent error-prone complexity and it is already core functionality in NS, I am hoping to avoid re-implementing the same logic in this package.

Yes, there is a legacy decision made here. Older pumps such as the Medtronic 515 (the one I still use!) do not record every delivered basal tick. After the Medtronic 523, pumps started more commonly explicitly recording every basal tick delivered, but prior to this, only the schedule and the departures from it were available.
At one point early in Tidepool's formation I had some code that could unspool a schedule and some temporary events into a correctly contiguous expected ticks delivered and departure events in a way that filled the timeline consistently, but Tidepool decided not to support these pumps in the meantime. Are you wanting a linear sequence of events you can sum, or a precalculated sum?

The v2 api has a way to get commonly used properties (/api/v2/properties/, but the calculation for tdd is done in the report rendering ~ https://github.com/nightscout/cgm-remote-monitor/blob/master/lib/report_plugins/daytoday.js#L686

It would be nice to have some of the stats/reporting done in the /api/v2/properties or a new /api/v2/stats type of area, that was the intent.

[scaubrey]

@bewest Makes sense, thanks for the context! As a first step, a pre-calculated sum for each day would work just fine. Yes, I was looking in the daytoday.js file at the report computation, because what is already implemented in there is what I would need.

Edit: I'll add that having the raw sequence of events would be ideal since the summation would benefit from flexibility in the time boundaries. But I understanding that exposing this may be more work.

[bewest]

Some notes to tie some things together:
Sulka's done some nice work cleaning up the v2 api in this pull request: nightscout/cgm-remote-monitor#8029
While there's nothing specific to TDD here, somewhere in this area is where TDD code would likely go, likely with some way to fetch it from client.

I did find some old code https://github.com/bewest/mmcsv#generator, which allows unrolling a profile schedule description into a discrete set of events. Afterwards, set operations can be performed with treatment events, and then it's possible to sum a TDD.

[scaubrey]

Thanks to letting me know about the v2 api work. What would you say is the timeline for insulin stats available in the v2 api? And do existing NS users have to update their code in order to have the v2 api work?

Thanks for forwarding your old code. That would be useful for verifying an eventual Python implementation in InsuLearner.

[scaubrey]

Hey @bewest, I've released a new version that includes Nightscout integration. Would you be willing to try it?

Also, I created a test suite that does a parity test between the results from both Nightscout and Tidepool. For my son's data, this test passes, but it'd be great to have more people check it.

[scaubrey]

Also, @bewest I've updated the free website version of this code to support Nightscout along with my website with accompanying notes:

Insulearner Web
https://www.cameronsummers.com/

[jbfeutz]

@scaubrey tried it just now but am justing getting the spinning gear. Should work without an access token but tried with and without and just getting spinning gear for a few minutes.

[scaubrey]

@jbfeutz It looks like the server is running out of memory. I'd have to look at the logs but either 1) too many people at once are running it or 2) sometimes people have significantly more data in the api download window for unexpected reasons.

If 1, maybe try again later and select just 30 days. If 2, try to download this code base and run locally. Your machine probably has more memory than the server I'm running - since I'm funding it myself, I'm trying to economize. If it persists, open a new issue here and I can try to troubleshoot.

[jbfeutz]

@scaubrey I wouldn't be surprised if it was #2. We do a lot of profile changes which I believe creates a lot of duplicated data. I'll work on running local when I get back to my workstation.