Repository navigation
Take an audience's counts off a response the caller already asked for - #88
Conversation
The registry has carried list_member_count through lib_hot, lib_domain
and lib_domain_daily since it was built, and it has never held a real
value: the library only ever observed the account root, so the legacy
estate could report an account-wide figure and nothing per audience. On
an account with several store views that is where the account number
stops answering the question.
observeList() mirrors observeRoot(): passive, on a response the caller
already has. The extension's statistics job reads lists/{id} every twelve
hours, so nothing new is requested, no quota is spent and no latency is
added.
getLists() gains include_total_contacts, appended last so every existing
positional caller is unaffected and off by default. total_contacts is the
billable figure and it is absent unless asked for; absent is left as null
rather than derived, because member_count plus unsubscribe_count omits
non-subscribed contacts entirely -- on an ecommerce account that is 96 to
98 percent of the total, and a number that nearly matches is worse than
one that is missing.
total_contacts INCLUDES cleaned, measured on an audience with 631 of
them: 1,767 against 1,752 for the sum that includes cleaned and 1,136 for
the sum that does not. So the docblock says a consumer deriving the
non-subscribed residual must subtract cleaned explicitly -- 15 rather
than 646 on that audience -- and that even then the residual is an upper
bound, since pending and archived land in it and archived are not billed.
The counts are tied to the audience the bucket already latched, so a
process reading several audiences cannot report one audience's identity
with another's numbers.
dipola
left a comment
There was a problem hiding this comment.
Approving. The measurement work behind it is the part I would not have got right on my own — you tested the parameter on both endpoints, and you settled the cleaned question on real data rather than reasoning about what total_contacts ought to mean.
The latch guard is the detail that matters most and it is the right way round: first audience wins, a different one is ignored rather than overwriting, so we never publish one audience's identity with another's counts. And leaving total_contacts null when the caller did not ask, instead of deriving something from the other three, is the whole argument of this thread honoured in code.
One finding: the single-audience test is the response shape, not the path
family() returns lists for every lists/... path, and harvest() returns a list_id for any path whose second segment exists — so lists/{id}/members/{hash} harvests {id} just as lists/{id} does. The only thing separating them is isset($result['stats']).
The collection case is genuinely covered twice (no second path segment, so $listId is null and it returns before the stats check). Sub-resources are covered once, by a property of Mailchimp's responses rather than by anything we asked for.
And there is a real collision underneath it. A member object has its own stats:
GET lists/{id}/members/{hash} -> { "id": …, "email_address": …,
"stats": { "avg_open_rate": …, "avg_click_rate": … } }
So observeList() does fire on every member fetch during a subscriber sync — a hot path. It writes nothing today, because a member's stats carries none of the four keys, so every isset() misses. Harmless, and it even latches the list id usefully. But the correctness rests on Mailchimp never adding a key named member_count, unsubscribe_count, cleaned_count or total_contacts to a member's stats object — and if that ever happened it would write one subscriber's numbers into the audience fields, silently, with the right-looking shape.
One line closes it, and makes the guard say what you actually mean:
$parts = explode('/', trim($path, '/'));
if (count($parts) !== 2) { return; } // lists/{id} only, never a sub-resourceNot a blocker — your call whether it rides this PR or the next. I would take it, because everything else here is guarded on what we asked for rather than on what came back, and this is the one place that is the other way round.
Field names: keep them
list_member_count is not just consistent with the column, it is already on the receiver's whitelist — so that one starts landing the moment you tag, with no edge change at all. The other three are dropped today and I am adding them along with their columns; that is my side and it does not gate you.
The half that is not here
Agreed, and worth saying plainly for whoever reads this later: until the extension passes the parameter, this ships the three counts that are not the billable figure and omits the one that is. member_count + unsubscribe_count is 2–4% of the truth on an ecommerce account. So the extension one-liner is not a nice-to-have that can drift — it is the half that makes the other half mean anything. As long as it is queued against the next extension release rather than remembered, we are fine.
tools/no-deprecated-calls.php clean, lean envelope empty, independent instances per case — all checked.
Raised in review. Everything else here is guarded on what we asked for;
this was guarded on what came back.
family() answers `lists` for every path under lists/, and harvest()
yields an id for any of them, so lists/{id}/members/{hash} reached
observeList() too -- on the subscriber sync, which is a hot path. It
wrote nothing only because a member's own stats object carries
avg_open_rate, avg_click_rate and ecommerce_data rather than any of the
four keys read here. Confirmed against a real member rather than assumed.
That is a property of Mailchimp's responses and not of anything we
control. The day a member's stats gained a key named member_count, this
would have written one subscriber's numbers into the audience fields --
silently, with the right-looking shape, and on the path that runs most.
Checked with a member whose stats carries member_count=999: before, the
audience field took it; now the path is refused before the response is
looked at. lists/{id} still observes, and the collection, the members
collection and interest-categories are all refused.
Nothing is lost by refusing them: record() already latches list_id from
harvest() on every call, so a member fetch still contributes the audience
id the way it always did.
|
Taken — I checked the collision against a real member before adding the guard, rather than reasoning about what So it was firing on every member fetch and writing nothing by luck rather than by design. Then the case that matters, with a member whose
And nothing is lost by refusing them, which I checked because your note said the member path latched On the field namesGood to hear On the half that is not hereAgreed, and your phrasing is the one I would keep: until the extension passes the parameter this ships |
dipola
left a comment
There was a problem hiding this comment.
Re-approving on b623f6c — my earlier approval was recorded against 03cc764, so it was stale against the new head rather than absent. This one is on the current commit.
Checked the guard rather than taking it: 17 lines, four of them code, the rest the reason; the file still parses; and the path set behaves —
lists/abc parts=2 OBSERVE
lists/abc?include_total_contacts=true parts=2 OBSERVE
/lists/abc parts=2 OBSERVE
lists/abc/members/hash parts=4 refused
lists parts=1 refused
Poisoning a real member with member_count = 999 is better than what I asked for. I raised it as a property of Mailchimp's responses we do not control; you turned it into a case that fails before and passes after. That is the difference between believing a guard works and knowing it does.
And checking that record() still latches list_id from harvest() on every call closes the one thing my note left open — I said the member path latched the id usefully, and you confirmed refusing it costs nothing because the latch happens elsewhere. I had not verified that; you did.
One nit, not a blocker and not reachable today. harvest() strips the query string before splitting; the new guard does not. They agree on every path the library actually builds, because getLists() produces 'lists/'.$id with no trailing slash — but lists/{id}/?x=1 splits into three and would be refused. A false negative, so the safe direction, and unreachable from any current caller. Worth one line only if you are in the file anyway.
On PR 2365
Good — queued rather than remembered, which was the whole of my concern. Your last sentence is the one I would put on both: it does not need this tagged to merge; it needs this tagged to mean anything. Send it over when you want it looked at.
Tag whenever you like. The receiver takes list_member_count today, and the other three plus their columns are mine and are not a gate on you.
|
@gonzaloebiz the harvest works — Two things for whenever you pick this up, neither urgent:
No rush. |
|
Both checked against the live API rather than the docs. 1. It is five call sites, not two — and the consequence is on your side
All five are admin, web lane — and the web lane is unsampled, so each one emits a beacon carrying So the question this raises is not really about scope, it is about your merge rule: does a later I would rather ask than widen the extension's scope on a guess. If overwriting is a risk, the cheap fix 2.
|
|
Your concern is real, not hypothetical — an incomplete reading can displace a complete one on our side. That is ours to fix, not a reason to constrain the scope of this PR. Please do pass the argument at all five. It is the better fix on its own terms regardless of what we change: it makes every reading complete, which is what the handling was designed around. I would rather that than special-case an absent value and end up holding figures read at different moments as if they were one. We will harden our side too, so an incomplete reading cannot displace a complete one even from a call site nobody anticipated. Your five are the ones we know about; the rule should not depend on knowing. Agreed on the bridge, and your reasoning is better than my question was. The deprecation is in the schema rather than in behaviour — the trigger to revisit is the parameter failing, not the notice existing. And Good catch on the call sites. I had two of five. |
|
Done — mc-magento2, PR 2366, all five passing the argument. Verified by the shortest of the five — constructing the admin audience dropdown:
One sequencing point, since last-wins makes it matter. This has to ship in the same extension release |
|
Checked the five files on 2366 — all there, and Your sequencing point is the important part of this comment, and you are right. The cron change alone would be worse than shipping neither: complete readings twice a day against incomplete ones on every admin page load, which is exactly the case that goes wrong. Same tag for both, or neither. Worth saying so it does not sit as an unspoken dependency: we are hardening our side so an incomplete reading cannot displace a complete one regardless of what any caller does or what order anything ships in. That work does not change your conclusion — the same-tag rule is right on its own terms, and I would rather it held even if our side were perfect, because the next call site nobody anticipates should not be able to cause this either. Two of five was my reading of the code; five of five was yours from the live API. Second time today. Noted. |
list_member_counthas been plumbed throughlib_hot,lib_domainandlib_domain_dailysince it was built and has never held a real value — the library only ever observed the account root, so the legacy estate could report an account-wide figure and nothing per audience.What it does
observeList()mirrorsobserveRoot(): passive, on a response the caller already has. The extension's statistics job readslists/{id}every twelve hours, so no call, no quota, no latency. It ignores anything that is not a single audience — the collection endpoint answers withlistsand nostats.getLists()gainsinclude_total_contacts, appended last so every existing positional caller is unaffected, and off by default.Why
total_contactsand not the summember_count + unsubscribe_countomits non-subscribed contacts, and on an ecommerce account that is 96–98% of the total — 413 against 20,971 on one account in the estate. So the sum is not an approximation of the billable figure, it is a different number.total_contactsis absent unless asked for, and absent is left asnullrather than derived. A fabricated total that nearly matches is worse than one that is missing, because nobody checks a number that looks right.total_contactsincludes cleaned — measured, not assumedOn an audience with 631 cleaned:
total_contacts1,767 sits above the sum that includes cleaned, not near the one that excludes it.
So the docblock tells a consumer deriving the non-subscribed residual to subtract cleaned explicitly —
15on that audience rather than646— and that even then the residual is an upper bound on transactional, since pending and archived land in it and archived are never billed.Verified against the live API
member=3 unsub=0 cleaned=0,total_contactsnulltotal_contacts=5lists?count=1rootEnvelope, with independent instances so no bucket leaks between cases:
And the guard that keeps the numbers attached to the right audience: a response for
lists/AAAarriving while the bucket has already latchedBBBis ignored, rather than reporting one audience's identity with another's counts.tools/no-deprecated-calls.php: 64 files, 0 findings.Field names, and the half that is not here
The names mirror the existing
list_member_countcolumn rather than inventing a second vocabulary — say the word if the receiver would rather have different ones and I will rename before this is tagged.This delivers nothing on its own:
total_contactsonly arrives if a caller asks, and the extension'sGenerateStatisticscurrently callsgetLists($listId)bare. That is one query-string argument on the extension side, queued separately so it rides the next extension release rather than being remembered.