B2B Enrollment email deliverability indications via webhooks - #3775
B2B Enrollment email deliverability indications via webhooks#3775dsubak wants to merge 42 commits into
Conversation
OpenAPI ChangesShow/hide changesUnexpected changes? Ensure your branch is up-to-date with |
…et from within the task now
c78581d to
e0f8f9c
Compare
|
| GitGuardian id | GitGuardian status | Secret | Commit | Filename | |
|---|---|---|---|---|---|
| 9430286 | Triggered | Generic Password | 81caefd | .github/workflows/ci.yml | View secret |
🛠 Guidelines to remediate hardcoded secrets
- Understand the implications of revoking this secret by investigating where it is used in your code.
- Replace and store your secret safely. Learn here the best practices.
- Revoke and rotate this secret.
- If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.
To avoid such incidents in the future consider
- following these best practices for managing and storing secrets including API keys and other credentials
- install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.
🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.
| # Just send the email reminder and update the last sent timestamp | ||
| queue_send_enrollment_code_assignment_email.delay([assignment_record.id]) | ||
| assignment_record.last_reminder_sent_on = now_in_utc() | ||
| clear_assignment_email_deliverability_fields(assignment_record) |
There was a problem hiding this comment.
This used to set last_reminder_sent_on = now_in_utc() here, but now it just clears the deliverability fields and that timestamp only gets set later when the send task runs. So the response we return right below still has the old last_sent value (or null if it was never sent), and the comment above about updating the last sent timestamp isn't true anymore.
If the dashboard shows a "reminder sent" time after this call, it'll be stale until the task finishes and the manager refetches. Could we set last_reminder_sent_on = now_in_utc() here too so the response reflects the click right away?
There was a problem hiding this comment.
Hmmm, my rationale for moving it to the task is that this as populated field is currently a bit misleading - it's entirely possible that we kick out the task to send and the mailgun API call outright fails, or the task is super super delayed or something. If that occurs, right now it looks like you sent a reminder to someone but in reality the learner didn't get any email. It's "more accurate" to have it set only once the task runs to completion.
That said, the staleness thing is a straight up oversight on my part - I definitely meant for that helper to also clear last_reminder_sent_on. That'd result in the the response here showing None until the task runs and actually sends it.
Maybe what we could do is leave last_reminder_sent_on set only at the API layer, but indicate to the user that a task is "Pending" or something until we get an accepted webhook from mailgun? Even that's not quite accurate in the event of an outright API call failure, but it's more accurate in the case of a task processing delay. I wasn't planning on configuring accepted webhooks since it's typically a short lived/uninteresting state for users, but it's easy enough to do if we think this is a decent solve. Thoughts?
…heck timestamps to ensure we dont overwrite a status with an older event
What are the relevant tickets?
https://github.com/mitodl/hq/issues/12114
Description (What does it do?)
At a high level, this PR has:
Screenshots (if appropriate):
After populating a synthetic delivered event via the local webhook endpoint w/ signature validation disabled.

How can this be tested?
Webhook status persistence
Included in this PR is a small test script to enable rudimentary testing without the need to set up a mailgun account or webhook. This is probably the most reasonable way to approach testing this functionality.
./manage.py b2b_send_mailgun_webhook <record_id> <email_event_status> --host='http://<local_mitxonline_host>:<port>'EMAIL_STATUSESas the status. We don't make any meaningful differentiation between them at the moment, though we may in the future.Status reset on reassignment/resending
Additional Context
I did some initial testing with a mailgun RC domain webhook by exposing the newly created endpoint via ngrok. This, frankly, sucks - exposing the endpoint alone is annoying, but even then you have to go through the rigamarole of spoofing your local data to have DCARs with matching message IDs before the webhook comes in since you're sending from RC.
To wit, I am providing a small script which sends synthetic webhooks against the endpoint for use in testing. Obviously this is a bit less representative, but the payload structure should be consistent and stable, so it should be sufficient for local testing (for now at least).
I am not a fan of how large this PR has become, but it's more or less the minimum amount for anything like an end-to-end test. If folks would prefer to review it in parts, I'm happy to break this down into more independently shippable parts - it just won't be meaningfully testable until several have landed.
Checklist: