fix: dirty-check edX sync and ProductPage saves to cut Fastly purges - #3806
Open
Anas12091101 wants to merge 2 commits into
Open
fix: dirty-check edX sync and ProductPage saves to cut Fastly purges#3806Anas12091101 wants to merge 2 commits into
Anas12091101 wants to merge 2 commits into
Conversation
OpenAPI ChangesShow/hide changesUnexpected changes? Ensure your branch is up-to-date with |
Contributor
There was a problem hiding this comment.
Pull request overview
Reduces unnecessary Fastly surrogate-key purges in MITx Online by avoiding unconditional model saves in two high-frequency paths: the hourly Open edX course run sync and Wagtail ProductPage saves that mirror titles onto Course/Program records.
Changes:
- Added a “dirty check” path for edX
CourseRunsync sosave()only happens when incoming values differ. - Updated
ProductPage.save()to only save the linkedCourse/Programwhen the title actually changed. - Added regression tests to ensure unchanged data does not trigger additional saves/purges.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
courses/api.py |
Introduces _sync_course_run_from_edx and skips CourseRun.save() when edX data is unchanged. |
courses/api_test.py |
Adds a test to verify the sync does not enqueue additional purges on a second identical pass. |
cms/models.py |
Avoids redundant Course/Program saves from ProductPage.save() when title is unchanged. |
cms/models_test.py |
Adds a test ensuring courseware save() is not called when a ProductPage title hasn’t changed. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+1262
to
+1269
| mock_course_list = mocker.patch("courses.api.get_edx_api_course_list_client") | ||
| mock_course_list.return_value.get_courses.return_value = [course_detail] | ||
|
|
||
| # First pass writes the edX values and enqueues one purge. | ||
| success_count, failure_count = sync_course_runs([course_run]) | ||
| assert (success_count, failure_count) == (1, 0) | ||
| purge_calls_after_first = mock_purge_delay.call_count | ||
| assert purge_calls_after_first >= 1 |
Comment on lines
716
to
718
| def sync_course_runs(runs): | ||
| """ | ||
| Sync course run dates and title from Open edX using course list API |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What are the relevant tickets?
https://github.com/mitodl/hq/issues/12614
Description (What does it do?)
Fastly surrogate-key purges were firing hundreds of times a day for changes that never happened. Root cause: two code paths call
.save()unconditionally, and everyCourseRun/Coursesave triggers apost_savepurge signal.This PR adds dirty checks so those saves (and their purges) only happen when something actually changed:
sync_course_runs(hourly edX sync): compares incoming edX values against the stored run and skipssave()when nothing differs. This eliminates the bulk of the purges (a course with 8 live runs was emitting ~192 identical purges/day, none reflecting a change) and a pointless full-columnUPDATE.ProductPage.save(): only pushes the title down to the linkedCourse/Program(and saves it) when the title actually changed, removing the redundant purges emitted on every page save.Screenshots (if appropriate):
How can this be tested?
The goal is to confirm that a purge is only scheduled when data actually changes. We do this in a Django shell by watching the purge task and running the sync twice — the second run should schedule zero purges.
purges=1for a change that didn't happen.New regression tests covering both fixes are included and run automatically in CI, so no extra steps are needed for those.
Additional Context
.save()in the sync path (didn't switch to.update()) soCourseRun.clean()still runs — theexpiration_datelogic depends on it.CourseRun.updated_on(auto_now). Nothing reads it (no adminlist_display/serializer), so this is safe to lose.on_rollback).