Ghost Pours

[jlopp]

We've been experiencing a number of invalid pours being reported by Kegbot. Some of them are small volumes of less than 1 ounce, but some of them are quite significant (several pints)

We're quite confident that this is not due to unauthorized people using our kegerator because we have locks installed on the tap handles and we have video surveillance on the kegerator itself. We have reviewed the footage which shows that no one was present at the times the pours were registered.

[mik3y]

Please see our knowledge base article on this subject. It's a work in progress, and links to a forum discussion where you can help us narrow down the issue or get further help (GitHub for the tablet app probably isn't the best place for this).

[jlopp]

Gotcha. This makes sense, as the majority of our ghost pours are from the tap closest to the kegerator's compressor. Guess we'll look into adding shielding to the flow meter and data cable. I created it as a tablet app issue in the hopes that it might be possible to detect invalid data in the software. Wouldn't an EMI-triggered event likely cause the data to arrive in a burst rather than over time like a normal pour?

[zylantha]

Could be the motor of the fridge running - which would be a longer
interference.

Can you try adding an RF choke to the cable that runs from the flow meter
to the Kegboard? That would be very simple to try out and may be effective
(you may have some RF chokes lying around, or steal one from an old VGA
monitor cable, you can also find them on some USB cables). Run the cable
twice through the RF choke (through the centre, back around the outside,
then back through the middle of the choke again.

On 30 July 2014 23:07, jlopp notifications@github.com wrote:

Gotcha. This makes sense, as the majority of our ghost pours are from the
tap closest to the kegerator's compressor. Guess we'll look into adding
shielding to the flow meter and data cable. I created it as a tablet app
issue in the hopes that it might be possible to detect invalid data in the
software. Wouldn't an EMI-triggered event likely cause the data to arrive
in a burst rather than over time like a normal pour?

—
Reply to this email directly or view it on GitHub
#97 (comment)
.

[mik3y]

Wouldn't an EMI-triggered event likely cause the data to arrive in a burst rather than over time like a normal pour?

Unfortunately no, not as far as we can tell. We do have low-pass filters on the KBPM, which help to isolate high-frequency bursts -- but noise being what it is, some transients nonetheless escape them.

FWIW we do keep a rough time series of meter reports for each drink. It's not exposed in the UI (yet), but take a look at /api/drinks/<id> and look for tick_time_series. Each space-delimited entry is a pair of "ms_since_start:total_ticks". You could use this information to determine whether a pour happened at a realistic/smooth rate.

[mik3y]

Can you try adding an RF choke to the cable that runs from the flow meter
to the Kegboard? That would be very simple to try out and may be effective

I'm definitely interested to hear any results -- it's something we list in the knowledge base article (under "ferrite bead"), and I believe someone reported success, but I can't find that post..

[jlopp]

Here are two of the time series samples from obviously invalid pours; I'm not sure how to interpret the data but it seems like it's over an absurdly long period of time. Perhaps some sanity thresholds are in order?

http://pastebin.com/DqZFsKrP

We have a few engineers here with electrical engineering degrees and after discussing the issue and showing them the specs of the hardware, this was the consensus:

We think the compressor is interfering with the flow sensor and the analog side of things (not the digital signal transmission) - in other words we don't think we need to wrap an RC filter on the RJ-45 - that's after the signal is higher frequency digital and not likely the EMF from compressor is messing that up. This is because the flow sensor is a hall effect sensor which is quite susceptible to EMF.
http://www.gemssensors.com/~/media/GemsNA/NEW%20CATALOG%20FILES%20-%202013/FT-330.pdf

http://en.wikipedia.org/wiki/Hall_effect

So, we're first going to try shielding the flow sensors with aluminum foil; if that doesn't work we may try shielding them with copper mesh.

[mik3y]

Perhaps some sanity thresholds are in order?

If by that you mean limits or heuristics based on pour volume/time, I'm open to any which do not compromise legitimate pours (ie, false positives are not acceptable).

[jlopp]

Sure; if I'm correct then the second example was a 13 minute pour? That seems... obviously invalid.

[mik3y]

Sure; can you tell me what the length of time that those example drinks was for? I'm assuming the first number of the pairs is elapsed milliseconds,

Correct (see above).

... the first example was a 13 minute pour? That seems... obviously invalid.

Definitely, 13 minutes is much longer than the typical pour! Nonetheless:

• Many "ghost" pours are shorter (one of your entries is about 3 minutes; still long but more realistic).
• Longer "open tap" pours may be commonplace on some systems (especially those which do not use accounts). I've personally witnessed this at a party / beer pong tournament.
• Noise-induced meter ticks are still indistinguishable from true activity; what happens when a "real" pour occurs in the middle of that 13 minute noise?

It's easy to imagine a simple max-duration check, though IMO it would just be papering over a deeper problem -- and only partially at that.. Moving/shielding the meter sounds promising.

[jlopp]

Time-only thresholds would be prone to failure in cases like you described, but once you take the volume being poured into account, it should be clearer. To throw out a ballpark figure, it seems to me that anything under 1 milliliter per second is definitely not a pour.

[mik3y]

min-rate may work in this particular case, but (a) we don't have enough data to say that's true of all cases (few people experience this to begin with), and (b) it doesn't resolve bullet 3 (noise masked by a true pour).

Really it isn't worth treating the symptoms until you've run out of options to treat the cause.