From aca2cd0eb31ef1ead710039bf9b6992332990658 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 14 Sep 2026 17:27:27 +0000 Subject: [PATCH 1/5] feat(quiz): give a hint and an explanation when an answer is wrong A learner who picks the wrong option is told only "Your answer is incorrect". Nothing says what they should have been thinking about or where to go and read it, and because the quiz can be restarted with the options reshuffled, they can retake it having learned nothing from the attempt that failed. All 45 questions now carry a hint (what to think about, and which theme to revisit) and an explanation (why the right answer is right, and where it helps, why the tempting distractor is not), shown only when the answer was wrong. The text lives in each question's `feedback` attribute -- the quiz wizard's per-question "General Feedback" field, so the XOT editor exposes it as a normal text area and round-trips it. No hint gives the answer away, and neither field refers to an option by letter or position, because `answerOrder="random"` shuffles them per attempt. models_html5/quiz.html renders that attribute on every submitted answer, right or wrong: after grading it fills three fixed slots in the order the question's `feedbackPos` gives (G the question's own feedback, A the selected option's, C the right/wrong line), defaulting to GAC. Part 6 of source/scorm_progress.js wraps quiz.showFeedBackandTrackResults -- re-wrapped from x_pageLoaded, because the model object is rebuilt for each quiz page -- and empties that slot again when quiz.myProgress[quiz.currentQ] says the answer was right. That is deliberately the failure-safe direction: if upstream renames the hook, the help shows on correct answers too rather than disappearing. The per-option `feedback` field was the alternative and is not used: it says nothing to a learner who fails a Multiple Answer question by under-selecting, and it would duplicate the same paragraph across every wrong option. Option feedback stays empty, so the SCORM interaction records (XTExitInteraction is passed the option feedback) are unchanged. Tests: test_wrong_answer_gets_a_hint_and_an_explanation and test_every_question_carries_wrong_answer_help, plus answer_question() and next_question() helpers -- answer_quiz(correctly=False) ticked the first option, which with shuffled options is not reliably a wrong one. tools/render_preview.py renders the text under each question so reviewers can read all 45 in the PR preview. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_0149Y1yzjVXVWwXfsKKeDASd --- README.md | 7 ++- docs/COURSE_SPEC.md | 9 ++++ docs/COURSE_VERIFICATION.md | 36 +++++++++++++++ docs/PROJECT_CONTEXT.md | 56 +++++++++++++++++++++++ source/data.xml | 18 ++++---- source/preview.xml | 18 ++++---- source/scorm_progress.js | 81 ++++++++++++++++++++++++++++++++ tests/test_scorm_tracking.py | 89 ++++++++++++++++++++++++++++++++++++ tools/render_preview.py | 11 ++++- 9 files changed, 304 insertions(+), 21 deletions(-) diff --git a/README.md b/README.md index 01933cd..3d83436 100644 --- a/README.md +++ b/README.md @@ -194,7 +194,8 @@ These are binding on this derivative work — see time-of-use)`. - **Quizzes**: full descriptive names in question stems (never "What causes A01" — OWASP renumbers between releases); no free-text questions; no empty - answer options; questions nested under their `` node. + answer options; questions nested under their `` node; every question + carries a hint and an explanation, shown when a learner answers it wrong. - **No auto-playing text**: content pages show all content at once (Plain Text, or Bullets with `delaySecs="0"`). - **Incident callouts** must be self-contained for learners who haven't read the @@ -202,6 +203,10 @@ These are binding on this derivative work — see ## Progress and completion: what learners and the LMS see +- **In a quiz**: a wrong answer is met with a hint (what to think about, and + which theme to revisit) and an explanation of the right answer; a correct + answer is not. See + [`docs/PROJECT_CONTEXT.md`](docs/PROJECT_CONTEXT.md#wrong-answer-help-hint--explanation-on-every-question). - **In the course**: a header progress bar (percentage of pages opened, with a marker per quiz), contents-page ticks that mean *opened* for content pages and *submitted* for quizzes (an opened-but-unsubmitted quiz shows a diff --git a/docs/COURSE_SPEC.md b/docs/COURSE_SPEC.md index 2872ac0..977d775 100644 --- a/docs/COURSE_SPEC.md +++ b/docs/COURSE_SPEC.md @@ -29,6 +29,12 @@ Model, version 1.2)** package with **LMS (Learning Management System)-reported s releases. 3. **No free-text questions**: all quiz items are selection-based (single choice, multiple response, true/false, matching, categorise). +4. **Every question teaches on failure**: each question carries a **hint** (what to think about + and which theme to revisit) and an **explanation** (why the right answer is right, and where + it helps, why the tempting distractor is not), shown **only when the learner answered it + wrong**. The hint must not give the answer away; neither may refer to an option by letter or + position, because the options shuffle per attempt. Both are held in the question's + `feedback` attribute — see `PROJECT_CONTEXT.md` § "Wrong-answer help". ## Template / technical setup @@ -44,6 +50,9 @@ Model, version 1.2)** package with **LMS (Learning Management System)-reported s pages done/still to visit, quiz results, weighted score and the status reported to the LMS. - **Option order**: every quiz question sets `answerOrder="random"` so the answer options shuffle on each attempt (Nottingham per-question "Answer Order" property). +- **Wrong-answer help**: every quiz question sets `feedback` (the per-question "General + Feedback" property) to a hint plus an explanation; the root script clears it again when the + answer was right, so only a learner who got it wrong is shown it. - **Question types used** (all selection-based, no free text): Multiple Choice (single), Multiple Response (select all), True/False, Matching, Categorise. - Editor opened directly at `/edit.php?template_id=`; **Publish** before export. diff --git a/docs/COURSE_VERIFICATION.md b/docs/COURSE_VERIFICATION.md index 207d6a3..d83524c 100644 --- a/docs/COURSE_VERIFICATION.md +++ b/docs/COURSE_VERIFICATION.md @@ -278,3 +278,39 @@ other tracking attrs), export a SCORM package, and grep the exported `template.xml` for the checklist values in `PROJECT_CONTEXT.md`. Then walk a quiz end-to-end (answer → Check → Next → complete → Restart) and confirm options reorder between attempts. + +## Wrong-answer help: hint + explanation on every question (post-0.0.6) + +Every `` gained a `feedback` attribute (the per-question *General +Feedback* field) holding a hint and an explanation, and part 6 of +`source/scorm_progress.js` clears that feedback slot again when the answer was +right, so only a learner who got the question wrong is shown it. See +`PROJECT_CONTEXT.md` § "Wrong-answer help" for the mechanism and why the +per-option `feedback` field was not used. + +### Local checks (run against `source/data.xml`; XOT re-export still required per convention 9) + +- 45 `` elements, 45 with a `feedback` attribute; each unescapes to + `

Hint: …

Why: …

`, the same + double-escaped shape as the existing `prompt`/`text` values. +- No hint or explanation names an option by letter or position (options + shuffle per attempt), and no question-level `feedback` gives the answer away + in its hint. +- Option `feedback` attributes are still empty ×161, so the SCORM interaction + records (`XTExitInteraction` is passed the *option* feedback) are unchanged. +- `trackingWeight`, `trackingMode`, `trackingPassed`, `judge="true"` ×8, + `answerOrder="random"` ×45, `delaySecs="0"` ×27 unchanged. +- `python3 tools/sync_root_script.py --check` passes; the compiled root script + extracted from `data.xml` passes `node --check`. +- `tools/render_preview.py` renders 44 pages without error, with the help text + under each question for review. +- `source/data.xml` and `source/preview.xml` are byte-identical. + +**Pending (convention 9 — MANDATORY before merge)**: the PR's CI job builds the +package in a real XOT container and runs `tests/test_scorm_tracking.py` +(including `test_wrong_answer_gets_a_hint_and_an_explanation` and +`test_every_question_carries_wrong_answer_help`) — that is the import/export +gate. Still to do by hand: open the HTML5 editor and confirm a Publish +round-trip preserves the new `feedback` values, and walk a quiz in `play.php` +answering one question wrong and one right, confirming the help appears only +on the wrong one and that the layout holds on a narrow screen. diff --git a/docs/PROJECT_CONTEXT.md b/docs/PROJECT_CONTEXT.md index 9925bc7..754a522 100644 --- a/docs/PROJECT_CONTEXT.md +++ b/docs/PROJECT_CONTEXT.md @@ -148,6 +148,12 @@ These were agreed with the course owner and must be preserved: computed from the shuffled array, so SCORM tracking is unaffected. When adding a new question, set `answerOrder="random"` on it. Keep `data.xml` and `preview.xml` byte-identical. +11. **Every question explains itself when it is failed**: each `` + carries a `feedback` attribute holding a hint and an explanation, shown + only to a learner who answered it wrong (see "Wrong-answer help" below). + A new question needs one; the hint must not give the answer away, and + neither field may refer to an option by letter or position, because + `answerOrder="random"` shuffles them. ## Known deviations (acknowledged, not to "fix" without asking) @@ -480,6 +486,55 @@ Tests: `test_status_panel_tells_the_learner_what_is_left`, `test_header_progress_bar_counts_viewed_pages`, and the panel assertions at the end of `test_full_walkthrough_reports_passed`. +## Wrong-answer help (hint + explanation on every question) + +A learner who picked the wrong option used to see nothing but *"Your answer is +incorrect"*, so there was nothing to learn from and nothing to revise before +the restart. Every one of the 45 questions now carries a **hint** (what to +think about, and which theme to revisit) and an **explanation** (why the right +answer is right, and where useful why the tempting distractor is not), shown +**only when the answer was wrong**. + +**Where the text lives.** In the question's own `feedback` attribute — the +quiz wizard's *General Feedback* field (`quiz.xwd`, so the XOT editor exposes +it as a normal per-question text area and round-trips it). The value is the +same double-escaped HTML as `prompt`/`text`: + +```xml + +``` + +**Why it needs the root script.** `models_html5/quiz.html` renders that +attribute on **every** submitted answer, right or wrong: after grading it +fills three fixed slots (`#topFeedback`, `#middleFeedback`, `#bottomFeedback`) +in the order the question's `feedbackPos` gives — `G` the question's own +feedback, `A` the selected option's, `C` the right/wrong line — defaulting to +`GAC`, so ours lands in `#topFeedback`. Part 6 of `source/scorm_progress.js` +wraps `quiz.showFeedBackandTrackResults` (re-wrapped from `x_pageLoaded`, +because the model object is rebuilt for every quiz page) and, when +`quiz.myProgress[quiz.currentQ] === true`, empties that slot again. + +This is deliberately the failure-safe direction: if upstream ever renames the +hook, the help shows on correct answers as well instead of disappearing. The +per-option `feedback` field was the other candidate and was not used — it says +nothing to a learner who gets a *Multiple Answer* question wrong by +under-selecting (every option they ticked was a correct one), and it would +duplicate the same paragraph across each wrong option. + +**Nothing about tracking changes**: `XTExitInteraction` is handed +`trackData.feedback`, which is built from the *option* `feedback` values (all +still empty), so SCORM interaction records are untouched. + +Tests: `test_wrong_answer_gets_a_hint_and_an_explanation` (a wrong answer is +given both, a correct one is not) and +`test_every_question_carries_wrong_answer_help` (all 45 questions have it, so +a question added later cannot silently ship without one). Reviewers can read +the text in the PR preview — `tools/render_preview.py` renders it under each +question. + +**Adding a question**: give it a `feedback` value in the same shape, or +`test_every_question_carries_wrong_answer_help` fails. + ## Build version stamping (`{{BUILD_VERSION}}` placeholder) So an operator can tell **which version is running** in an LMS, the SCORM @@ -628,6 +683,7 @@ unzip -p Secure_code_development_scorm.zip template.xml > /tmp/c.xml grep -c "Trial MCQ" /tmp/c.xml # 0 grep -oE ' -Welcome to Secure code development +Welcome to Secure code development

This course is for engineers building software with agents and for engineers building agents. Whether you are designing agentic systems, integrating LLMs, or writing the services that agents call, secure code development matters.

@@ -211,7 +211,7 @@
  • Detect delegated and transitive permissions — model the full delegation chain. If agent A delegates to agent B, the effective permissions should be the intersection of what A and B are each allowed to do.
  • Managed NHI (Non-Human Identity) — place agent identities in a dedicated identity platform such as Entra, AWS Bedrock, or Agentforce. Use lifecycle management, rotation, and audit for non-human accounts just as you would for human employees.
    • -]]>    This theme traces how untrusted input becomes unintended action. We start with classic injection, then follow the chain into agentic territory: agents whose goals can be hijacked, whose tools can be misused, and whose generated code can execute unexpectedly.

    +]]>    This theme traces how untrusted input becomes unintended action. We start with classic injection, then follow the chain into agentic territory: agents whose goals can be hijacked, whose tools can be misused, and whose generated code can execute unexpectedly.

    For engineers building agents, the lesson is that input validation, intent checks, and execution boundaries matter at every layer.

    ]]>    Injection @@ -333,7 +333,7 @@
  • Static scans before execution. Run security scanners and dependency checks on generated code before it is executed.
  • Runtime monitoring. Observe file, network, and process activity during code execution and terminate anomalous behaviour.
    • -]]>    Theme 3: Supply Chain & Integrity looks at trust in code, data, and context. In classic software, the supply chain is mostly static: libraries, packages, build tools, and the CI/CD (Continuous Integration / Continuous Deployment) pipeline that produces your artefact. If any of those ingredients are tampered with, the final application carries the flaw. The OWASP Top 10:2025 tracks this as Software Supply Chain Failures (A03:2025) and Software or Data Integrity Failures (A08:2025).

    +]]>    Theme 3: Supply Chain & Integrity looks at trust in code, data, and context. In classic software, the supply chain is mostly static: libraries, packages, build tools, and the CI/CD (Continuous Integration / Continuous Deployment) pipeline that produces your artefact. If any of those ingredients are tampered with, the final application carries the flaw. The OWASP Top 10:2025 tracks this as Software Supply Chain Failures (A03:2025) and Software or Data Integrity Failures (A08:2025).

    Agentic systems extend that same idea into a dynamic, runtime supply chain. An agent does not just import a dependency at build time; it can discover and compose tools, models, prompts, and other agents at runtime. This composition happens through protocols such as the Model Context Protocol (MCP) and Agent-to-Agent (A2A) frameworks, and through registries that advertise tool or agent capabilities. Each extra capability is a dependency that can be malicious, misconfigured, or compromised.

    @@ -496,7 +496,7 @@
  • Decay / expire unverified memory. Reduce the trust weight of unverified entries over time and eventually remove them.
  • Trust-weighted retrieval. Rank search results by source trust and confidence, not just embedding similarity.
    • -]]>    Theme 4: Cryptographic Failures & Secrets +]]>Theme 4: Cryptographic Failures & Secrets

    Theme 4 explores Cryptographic Failures (A04:2025) from the OWASP Top 10:2025, plus what those failures mean for agents and Non-Human Identities (NHI).

    @@ -539,7 +539,7 @@
  • Private keys should never be directly accessible to agents. Use an orchestrator or trusted signer to perform operations on behalf of the agent.
  • Treat Non-Human Identity (NHI) credentials as a full lifecycle asset: issue with least privilege, monitor use, rotate frequently, and revoke immediately when an agent is decommissioned or suspected of compromise.
    • -]]>    Theme 5: Configuration, Design & Resilience +]]>Theme 5: Configuration, Design & Resilience

    Theme 5 brings together three overlapping risks from the OWASP Top 10:2025 — Security Misconfiguration (A02:2025), Insecure Design (A06:2025), and Mishandling of Exceptional Conditions (A10:2025) — and adds the agentic risk of Cascading Failures (ASI08).

    @@ -653,7 +653,7 @@
  • Use digital-twin replay and policy gating to test risky decisions before live execution.
  • Keep tamper-evident logging and non-repudiation so every propagated action can be traced and audited.
    • -]]>    Theme 6 covers the risks that are unique to agentic systems, where software does not just respond to a single user but can act on its own, talk to other agents, and influence the people around it.

    +]]>    Theme 6 covers the risks that are unique to agentic systems, where software does not just respond to a single user but can act on its own, talk to other agents, and influence the people around it.

    Where earlier themes mapped classic application risks to the agent world, this theme is different: it focuses on three risks that only appear when multiple agents, or a human and an agent, share responsibility.

    @@ -755,7 +755,7 @@
  • Require periodic behavioural attestation and per-run ephemeral credentials, so compromise is time-bound.
  • Tie recovery or reintegration to fresh attestation and explicit human approval.
    • -]]>    Theme 7 brings the focus onto observability, logging and detection. These controls are cross-cutting: they support almost every other theme in this course.

    +]]>    Theme 7 brings the focus onto observability, logging and detection. These controls are cross-cutting: they support almost every other theme in this course.

    For traditional applications, good logging and alerting helps you detect breaches, investigate incidents and prove compliance. For agentic systems, the need is even stronger. Agents make sequences of decisions, call tools, talk to other agents and influence humans. Without a clear, tamper-evident record of what happened, you cannot attribute actions, detect drift or recover from failure.

    @@ -789,7 +789,7 @@

    For agents, the same principles apply, but the events are different. You should log tool calls, inter-agent messages, goal state and action sequences against a baseline. Track a stable goal identifier across the lifetime of a task so that every action can be traced back to the original request. Alert on goal drift and anomalous tool chains, and ensure non-repudiation and lineage for every propagated action.

    Strong observability turns an opaque agent into an accountable one: it is not enough to know what the agent said; you must be able to reconstruct what it did.

    -]]>    Putting it together: secure SDLC and threat modelling +]]>Putting it together: secure SDLC and threat modelling

    Secure code development is not a single step; it is threaded through the whole SDLC (Software Development Life Cycle). The two OWASP lists in this course — the OWASP Top 10:2025 for classic applications and the OWASP Top 10 for Agentic Applications 2026 — give you concrete risks to address at each phase.

    @@ -990,7 +990,7 @@

    License of this course

    This course is licensed under CC BY-SA 4.0 (Creative Commons Attribution-ShareAlike 4.0). It is an adapted summary/derivative; changes were made; it is not endorsed by OWASP. Generated with the assistance of an AI coding agent.

    -]]>    Course complete +]]>Course complete
    diff --git a/source/preview.xml b/source/preview.xml index 8c87b74..bcee21e 100644 --- a/source/preview.xml +++ b/source/preview.xml @@ -1,5 +1,5 @@ -Welcome to Secure code development +Welcome to Secure code development

    This course is for engineers building software with agents and for engineers building agents. Whether you are designing agentic systems, integrating LLMs, or writing the services that agents call, secure code development matters.

    @@ -211,7 +211,7 @@
  • Detect delegated and transitive permissions — model the full delegation chain. If agent A delegates to agent B, the effective permissions should be the intersection of what A and B are each allowed to do.
  • Managed NHI (Non-Human Identity) — place agent identities in a dedicated identity platform such as Entra, AWS Bedrock, or Agentforce. Use lifecycle management, rotation, and audit for non-human accounts just as you would for human employees.
    • -]]>    This theme traces how untrusted input becomes unintended action. We start with classic injection, then follow the chain into agentic territory: agents whose goals can be hijacked, whose tools can be misused, and whose generated code can execute unexpectedly.

    +]]>    This theme traces how untrusted input becomes unintended action. We start with classic injection, then follow the chain into agentic territory: agents whose goals can be hijacked, whose tools can be misused, and whose generated code can execute unexpectedly.

    For engineers building agents, the lesson is that input validation, intent checks, and execution boundaries matter at every layer.

    ]]>    Injection @@ -333,7 +333,7 @@
  • Static scans before execution. Run security scanners and dependency checks on generated code before it is executed.
  • Runtime monitoring. Observe file, network, and process activity during code execution and terminate anomalous behaviour.
    • -]]>    Theme 3: Supply Chain & Integrity looks at trust in code, data, and context. In classic software, the supply chain is mostly static: libraries, packages, build tools, and the CI/CD (Continuous Integration / Continuous Deployment) pipeline that produces your artefact. If any of those ingredients are tampered with, the final application carries the flaw. The OWASP Top 10:2025 tracks this as Software Supply Chain Failures (A03:2025) and Software or Data Integrity Failures (A08:2025).

    +]]>    Theme 3: Supply Chain & Integrity looks at trust in code, data, and context. In classic software, the supply chain is mostly static: libraries, packages, build tools, and the CI/CD (Continuous Integration / Continuous Deployment) pipeline that produces your artefact. If any of those ingredients are tampered with, the final application carries the flaw. The OWASP Top 10:2025 tracks this as Software Supply Chain Failures (A03:2025) and Software or Data Integrity Failures (A08:2025).

    Agentic systems extend that same idea into a dynamic, runtime supply chain. An agent does not just import a dependency at build time; it can discover and compose tools, models, prompts, and other agents at runtime. This composition happens through protocols such as the Model Context Protocol (MCP) and Agent-to-Agent (A2A) frameworks, and through registries that advertise tool or agent capabilities. Each extra capability is a dependency that can be malicious, misconfigured, or compromised.

    @@ -496,7 +496,7 @@
  • Decay / expire unverified memory. Reduce the trust weight of unverified entries over time and eventually remove them.
  • Trust-weighted retrieval. Rank search results by source trust and confidence, not just embedding similarity.
    • -]]>    Theme 4: Cryptographic Failures & Secrets +]]>Theme 4: Cryptographic Failures & Secrets

    Theme 4 explores Cryptographic Failures (A04:2025) from the OWASP Top 10:2025, plus what those failures mean for agents and Non-Human Identities (NHI).

    @@ -539,7 +539,7 @@
  • Private keys should never be directly accessible to agents. Use an orchestrator or trusted signer to perform operations on behalf of the agent.
  • Treat Non-Human Identity (NHI) credentials as a full lifecycle asset: issue with least privilege, monitor use, rotate frequently, and revoke immediately when an agent is decommissioned or suspected of compromise.
    • -]]>    Theme 5: Configuration, Design & Resilience +]]>Theme 5: Configuration, Design & Resilience

    Theme 5 brings together three overlapping risks from the OWASP Top 10:2025 — Security Misconfiguration (A02:2025), Insecure Design (A06:2025), and Mishandling of Exceptional Conditions (A10:2025) — and adds the agentic risk of Cascading Failures (ASI08).

    @@ -653,7 +653,7 @@
  • Use digital-twin replay and policy gating to test risky decisions before live execution.
  • Keep tamper-evident logging and non-repudiation so every propagated action can be traced and audited.
    • -]]>    Theme 6 covers the risks that are unique to agentic systems, where software does not just respond to a single user but can act on its own, talk to other agents, and influence the people around it.

    +]]>    Theme 6 covers the risks that are unique to agentic systems, where software does not just respond to a single user but can act on its own, talk to other agents, and influence the people around it.

    Where earlier themes mapped classic application risks to the agent world, this theme is different: it focuses on three risks that only appear when multiple agents, or a human and an agent, share responsibility.

    @@ -755,7 +755,7 @@
  • Require periodic behavioural attestation and per-run ephemeral credentials, so compromise is time-bound.
  • Tie recovery or reintegration to fresh attestation and explicit human approval.
    • -]]>    Theme 7 brings the focus onto observability, logging and detection. These controls are cross-cutting: they support almost every other theme in this course.

    +]]>    Theme 7 brings the focus onto observability, logging and detection. These controls are cross-cutting: they support almost every other theme in this course.

    For traditional applications, good logging and alerting helps you detect breaches, investigate incidents and prove compliance. For agentic systems, the need is even stronger. Agents make sequences of decisions, call tools, talk to other agents and influence humans. Without a clear, tamper-evident record of what happened, you cannot attribute actions, detect drift or recover from failure.

    @@ -789,7 +789,7 @@

    For agents, the same principles apply, but the events are different. You should log tool calls, inter-agent messages, goal state and action sequences against a baseline. Track a stable goal identifier across the lifetime of a task so that every action can be traced back to the original request. Alert on goal drift and anomalous tool chains, and ensure non-repudiation and lineage for every propagated action.

    Strong observability turns an opaque agent into an accountable one: it is not enough to know what the agent said; you must be able to reconstruct what it did.

    -]]>    Putting it together: secure SDLC and threat modelling +]]>Putting it together: secure SDLC and threat modelling

    Secure code development is not a single step; it is threaded through the whole SDLC (Software Development Life Cycle). The two OWASP lists in this course — the OWASP Top 10:2025 for classic applications and the OWASP Top 10 for Agentic Applications 2026 — give you concrete risks to address at each phase.

    @@ -990,7 +990,7 @@

    License of this course

    This course is licensed under CC BY-SA 4.0 (Creative Commons Attribution-ShareAlike 4.0). It is an adapted summary/derivative; changes were made; it is not endorsed by OWASP. Generated with the assistance of an AI coding agent.

    -]]>    Course complete +]]>Course complete
    diff --git a/source/scorm_progress.js b/source/scorm_progress.js index 952a4b5..e52326e 100644 --- a/source/scorm_progress.js +++ b/source/scorm_progress.js @@ -74,6 +74,16 @@ * page) with the required-page count, the pages still to do, each quiz * result, the weighted score and the exact status the LMS holds. * + * 6. shows the hint and explanation authored on a question only to a learner + * who got that question wrong. The text lives in the question's own + * feedback attribute (the quiz wizard's "General Feedback" field), which + * models_html5/quiz.html renders into a feedback slot on every submitted + * answer, right or wrong. This script clears that slot again when the + * answer was correct, so the learner who needs the help sees it and the + * learner who does not is left with the plain "correct" message. If this + * hook ever stops matching the engine, the feedback simply shows on + * correct answers too: nothing is lost. + * * CONSTRAINT: the compiled one-liner is stored raw in an XML attribute, so it * must not contain < > & or " characters (tools/sync_root_script.py enforces * this). Use === / !== / indexOf() instead of ordering comparisons, nested ifs @@ -649,11 +659,82 @@ var orig = window.x_pageLoaded; window.x_pageLoaded = function () { var result = orig.apply(this, arguments); + try { wrapQuizFeedback(); } catch (e) { } refreshUi(); return result; }; } + /* ---- wrong-answer help: the hint and explanation on a question ---- */ + + var FEEDBACK_SLOTS = ['topFeedback', 'middleFeedback', 'bottomFeedback']; + + /* quiz.currentQ indexes quiz.questions, which holds the position of each + question within the quiz page, in the order this attempt asks them. */ + function questionNode() { + if (typeof quiz === 'undefined') { return null; } + if (!quiz) { return null; } + if (!isArray(quiz.questions)) { return null; } + var pos = quiz.questions[quiz.currentQ]; + if (typeof pos !== 'number') { return null; } + if (typeof jQuery !== 'function') { return null; } + if (typeof x_currentPageXML === 'undefined') { return null; } + if (!x_currentPageXML) { return null; } + var node = jQuery(x_currentPageXML).children()[pos]; + if (!node) { return null; } + return node; + } + + /* The model fills three fixed slots in the order the question's + feedbackPos gives, one letter per slot: G is the question's own + feedback attribute, A the selected option's, C the right/wrong line. + No question here sets feedbackPos, so G lands in the first slot. */ + function helpSlot(node) { + var order = 'GAC'; + var attr = node.getAttribute('feedbackPos'); + if (typeof attr === 'string') { + if (attr !== '') { order = attr; } + } + var pos = order.indexOf('G'); + var id = FEEDBACK_SLOTS[pos]; + if (typeof id !== 'string') { return null; } + return document.getElementById(id); + } + + /* myProgress holds each question's graded result, and currentQ still + names the question just submitted until Next is pressed. */ + function answerWasRight() { + if (typeof quiz === 'undefined') { return false; } + if (!quiz) { return false; } + if (!isArray(quiz.myProgress)) { return false; } + return quiz.myProgress[quiz.currentQ] === true; + } + + function hideHelpWhenRight() { + if (!answerWasRight()) { return; } + var node = questionNode(); + if (!node) { return; } + var slot = helpSlot(node); + if (!slot) { return; } + slot.textContent = ''; + } + + /* The model object is built afresh for every quiz page, so this runs from + x_pageLoaded rather than once at startup. */ + function wrapQuizFeedback() { + if (typeof quiz === 'undefined') { return; } + if (!quiz) { return; } + if (quiz.__xtHelp) { return; } + if (typeof quiz.showFeedBackandTrackResults !== 'function') { return; } + quiz.__xtHelp = true; + var orig = quiz.showFeedBackandTrackResults; + quiz.showFeedBackandTrackResults = function () { + var result = orig.apply(this, arguments); + try { hideHelpWhenRight(); } catch (e) { } + return result; + }; + } + /* the quiz results screen is the only caller of XTSetPageScore */ function wrapPageScore() { if (typeof window.XTSetPageScore !== 'function') { return; } diff --git a/tests/test_scorm_tracking.py b/tests/test_scorm_tracking.py index 56d0e55..4093672 100644 --- a/tests/test_scorm_tracking.py +++ b/tests/test_scorm_tracking.py @@ -131,6 +131,33 @@ } """ +# Options are shuffled per attempt, so a wrong answer has to be chosen by its +# correct flag rather than by position. +SELECT_A_WRONG_OPTION = """ +() => { + const inputs = window.jQuery('#optionHolder input'); + let picked = 0; + inputs.each(function () { + if (picked) { return; } + const correct = window.jQuery(this).data('correct'); + if (correct !== 'true' && correct !== true) { + window.jQuery(this).prop('checked', true).trigger('change'); + picked++; + } + }); + return {options: inputs.length, picked: picked}; +} +""" + +# What the learner is left looking at once a question has been marked. +FEEDBACK_SHOWN = """ +() => ({ + help: window.jQuery('#topFeedback').text().trim(), + verdict: window.jQuery('#bottomFeedback').text().trim(), + marking: window.jQuery('#feedbackGroup').attr('class') || '' +}) +""" + # The SCO always runs inside a frame in an LMS; reproduce that. LMS_SHELL = """ @@ -282,6 +309,21 @@ def answer_quiz(self, correctly: bool = True) -> int: ) return questions + def answer_question(self, correctly: bool) -> dict: + """Answer the question now on screen and return the feedback shown.""" + self.sco.wait_for_selector("#optionHolder input", timeout=15000) + picked = self.sco.evaluate( + SELECT_CORRECT_OPTIONS if correctly else SELECT_A_WRONG_OPTION + ) + assert picked["picked"], f"no option to pick: {picked}" + self.sco.evaluate("() => window.jQuery('#checkBtn').trigger('click')") + self.page.wait_for_timeout(300) + return self.sco.evaluate(FEEDBACK_SHOWN) + + def next_question(self) -> None: + self.sco.evaluate("() => window.jQuery('#nextBtn').trigger('click')") + self.page.wait_for_timeout(200) + def terminate(self) -> None: self.sco.evaluate("() => XTTerminate()") self.page.wait_for_timeout(200) @@ -952,6 +994,53 @@ def test_status_panel_tells_the_learner_what_is_left(h: Harness): assert not s.page_errors, f"page errors: {s.page_errors}" +@test +def test_wrong_answer_gets_a_hint_and_an_explanation(h: Harness): + """Getting a question wrong tells the learner why, and what to revisit. + + The help is authored on the question itself, which the quiz model renders + on every submitted answer. Only a learner who got it wrong needs it, so + the block is cleared again when the answer was right. + """ + with h.session() as s: + s.open() + s.goto(THEME1_QUIZ) + before = s.sco.evaluate(FEEDBACK_SHOWN) + assert before["help"] == "", f"help shown before answering: {before}" + wrong = s.answer_question(correctly=False) + assert "incorrectFeedback" in wrong["marking"], wrong + assert wrong["help"].startswith("Hint:"), f"no hint on a wrong answer: {wrong}" + assert "Why:" in wrong["help"], f"no explanation on a wrong answer: {wrong}" + s.next_question() + right = s.answer_question(correctly=True) + assert "correctFeedback" in right["marking"], right + assert right["help"] == "", f"a correct answer should not be given help: {right}" + assert not s.page_errors, f"page errors: {s.page_errors}" + + +@test +def test_every_question_carries_wrong_answer_help(h: Harness): + """No question is left without a hint and an explanation. + + The text lives in the course XML, so a question added later would silently + have none: walk the parsed project and check all 45 of them. + """ + with h.session() as s: + s.open() + missing = s.sco.evaluate( + "() => { const out = [];" + " window.jQuery(x_pages).each(function () {" + " if (this.nodeName !== 'quiz') { return; }" + " window.jQuery(this).children('question').each(function () {" + " const fb = this.getAttribute('feedback') || '';" + " if (fb.indexOf('Hint:') === -1 || fb.indexOf('Why:') === -1) {" + " out.push(this.getAttribute('name')); } }); });" + " return out; }" + ) + assert missing == [], f"questions without a hint and explanation: {missing}" + assert not s.page_errors, f"page errors: {s.page_errors}" + + @test def test_quiz_tick_means_submitted_not_opened(h: Harness): """A quiz that has been opened but not submitted gets a distinct marker. diff --git a/tools/render_preview.py b/tools/render_preview.py index a882b68..ad08370 100644 --- a/tools/render_preview.py +++ b/tools/render_preview.py @@ -44,8 +44,13 @@ def render_question(q): out = [f'
    ', f'
    Q: {html.escape(prompt)}
    ', f'
    {html.escape(qtype)} ({len(opts)} options)
    ', - render_options(opts), - f'
    '] + render_options(opts)] + # shown to a learner who gets this question wrong + help_html = unesc(q.get("feedback", "")).strip() + if help_html: + out.append(f'
    shown on a wrong answer
    ' + f'{help_html}
    ') + out.append('') return "\n".join(out) def render_page(node, idx): @@ -117,6 +122,8 @@ def main(): .opt.correct {{ font-weight:bold; }} .opt .lbl {{ color:var(--muted); }} .opt .mark {{ color:var(--accent); font-size:.8rem; }} +.q-help {{ margin-top:8px; padding:6px 12px; border-left:3px solid #c9a227; background:#fffdf3; }} +.q-help p {{ margin:.3em 0; }} footer {{ color:var(--muted); font-size:.8rem; text-align:center; padding:20px; }}

    {html.escape(title)}

    From 31c95cd69c0596eb6694c556894c6989793c2c7d Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 15 Sep 2026 09:34:51 +0000 Subject: [PATCH 2/5] fix(quiz): correct the wrong-answer help against the current OWASP text Review of the first green build, across runtime correctness, content and fidelity to the two OWASP sources, found real problems in all three. Mitigations, checked against the live top10.owasp.org/2025 pages: - Password storage recommended "Argon2 or bcrypt". A04:2025 names Argon2, yescrypt, scrypt and PBKDF2-HMAC-SHA-512, mentions bcrypt only for legacy systems, and leads with the work factor, which we omitted entirely. - Mishandling of exceptional conditions taught "keep sensitive detail out of errors", which is Security Misconfiguration's advice; A10:2025 says to inform the user understandably, and leads with catching errors where they occur, a global handler, full rollback (its term is "failing closed", not "failing secure") and rate limits. We had none of those. - Supply chain recommended "reproducible builds", a word that does not appear on the A03:2025 page, and omitted official sources, deliberately chosen versions and hardened CI/CD. - Insecure design kept "usage limits", a 2021 bullet dropped from A06:2025, and omitted the secure development lifecycle and secure design patterns that page now leads with. Taxonomy and question quality: - Broken access control was described as "a person exceeding permissions". OWASP says users, never human-only, which is the distinction this course exists to teach for agents and non-human identities. - Two adjacent final-quiz hints stated opposite classification rules, by harm and by root cause, so a learner applying either consistently failed the other. Both now point at the disambiguator in the question's own stem. - One hint asked whether provenance is any use "once it is already running", which argues for False on a question keyed True. - One hint said the agent "did exactly what its target rewarded", leaking the correct option "Rogue Agents (reward hacking)" by word stem. - One sentence reproduced OWASP's own ASI01 phrasing closely enough to read as copied, which the course's own-words rule does not allow; plus a lower-case acronym expansion and an imprecise "direct object reference". Runtime, traced against the engine in models_html5/quiz.html: - helpSlot treated an empty feedbackPos as GAC, but the engine spreads the value it finds, so an empty one leaves it no slot letters and puts the right/wrong line in all three. On a correct answer we would have deleted the verdict. It now follows the engine's rule: only an absent attribute means GAC. - Where the panel is centred vertically the model slides feedback in, and jQuery captures the target height while the text is still present, so emptying afterwards animated a blank gap open and snapped it shut. The slot is now cleared through jQuery, ending the animation first. Latent today (no page sets verticalAlign) but it would surface the moment one did. - The header comment and PROJECT_CONTEXT claimed the hook was failure-safe in general. That holds for a renamed call, not for a change of slot order. Both now say which. Tests, two of which could pass while the feature was broken: - The every-question walk asserted only that its "missing" list was empty, so it would have passed if the walk matched no nodes at all. It now asserts it inspected all 45. - "correctFeedback" is a Python substring of "incorrectFeedback", so that assertion held whatever the grading outcome. It is now an exact match, and the correct-answer case also asserts the verdict line survived the clear. - The wrong-answer test now continues onto a second quiz page, which is the case the re-wrap from x_pageLoaded exists for and nothing covered. Also corrects a pre-existing stale count in PROJECT_CONTEXT (the 45 questions are 27 theme + 18 final, not "23 theme + 18 final + 4"), in both the prose and the verification checklist, and records the carve-out the new text needs from the per-page acronym convention: wrong-answer help is read on its own, so each acronym is expanded within the hint and within the explanation. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_0149Y1yzjVXVWwXfsKKeDASd --- docs/COURSE_SPEC.md | 4 +++- docs/COURSE_VERIFICATION.md | 31 +++++++++++++++++++++++++++++++ docs/PROJECT_CONTEXT.md | 19 +++++++++++++------ source/data.xml | 12 ++++++------ source/preview.xml | 12 ++++++------ source/scorm_progress.js | 31 ++++++++++++++++++++++++------- tests/test_scorm_tracking.py | 24 +++++++++++++++++++----- 7 files changed, 102 insertions(+), 31 deletions(-) diff --git a/docs/COURSE_SPEC.md b/docs/COURSE_SPEC.md index 977d775..cd19bdc 100644 --- a/docs/COURSE_SPEC.md +++ b/docs/COURSE_SPEC.md @@ -34,7 +34,9 @@ Model, version 1.2)** package with **LMS (Learning Management System)-reported s it helps, why the tempting distractor is not), shown **only when the learner answered it wrong**. The hint must not give the answer away; neither may refer to an option by letter or position, because the options shuffle per attempt. Both are held in the question's - `feedback` attribute — see `PROJECT_CONTEXT.md` § "Wrong-answer help". + `feedback` attribute — see `PROJECT_CONTEXT.md` § "Wrong-answer help". This text is read + on its own, so convention 1 does not carry across it: expand each acronym on first use + within the hint and within the explanation, even where it is already expanded on the page. ## Template / technical setup diff --git a/docs/COURSE_VERIFICATION.md b/docs/COURSE_VERIFICATION.md index d83524c..8b346ce 100644 --- a/docs/COURSE_VERIFICATION.md +++ b/docs/COURSE_VERIFICATION.md @@ -306,6 +306,37 @@ per-option `feedback` field was not used. under each question for review. - `source/data.xml` and `source/preview.xml` are byte-identical. +### Review pass (three reviewers, post-CI) + +The first green build was reviewed for runtime correctness, content/XML +integrity and fidelity to the OWASP sources. Corrections applied: + +- **Mitigations brought up to the 2025 text** (verified against the live + `top10.owasp.org/2025` pages): password storage now names an adaptive salted + hash with a work factor (Argon2, yescrypt, scrypt, PBKDF2-HMAC-SHA-512) with + bcrypt as legacy-only, per A04; mishandling of exceptional conditions now + teaches catching at source, a global handler, full rollback ("failing + closed") and rate limits, per A10, instead of Security Misconfiguration's + stack-trace advice; supply chain drops "reproducible builds" (absent from + A03) for official sources, deliberate versions and hardened CI/CD; insecure + design drops the retired "usage limits" bullet for the secure development + lifecycle and secure design patterns A06 leads with. +- **Taxonomy**: broken access control is no longer described as human-only + (OWASP says *users*, which is the point for agent and non-human identities). +- **Two adjacent final-quiz hints** stated opposite classification rules + (by harm / by root cause); both now point at the stem's own disambiguator. +- **One hint resolved against its own keyed answer**, one leaked the answer by + word stem, one expansion was not title case — all corrected. +- **Runtime**: `helpSlot` now follows the engine's `feedbackPos` rule exactly + (an empty value leaves no G slot, so nothing is cleared — previously it + would have deleted the verdict line); the slot is cleared through jQuery so + the model's slide-in animation cannot animate an emptied block. +- **Tests**: the every-question walk now asserts it inspected 45 questions + (it would previously have passed if the walk found none); + `"correctFeedback" in "incorrectFeedback"` is true as a Python substring, so + the marking assertion is now exact; the wrong-answer test covers a second + quiz page, which is what the re-wrap on `x_pageLoaded` exists for. + **Pending (convention 9 — MANDATORY before merge)**: the PR's CI job builds the package in a real XOT container and runs `tests/test_scorm_tracking.py` (including `test_wrong_answer_gets_a_hint_and_an_explanation` and diff --git a/docs/PROJECT_CONTEXT.md b/docs/PROJECT_CONTEXT.md index 754a522..3ffb484 100644 --- a/docs/PROJECT_CONTEXT.md +++ b/docs/PROJECT_CONTEXT.md @@ -64,8 +64,8 @@ with agents and building agents. - **All 20 OWASP items** present: A01–A10 (classic) + ASI01–ASI10 (agentic), each with its own page, interleaved by theme. - **Quizzes**: 7 theme quizzes + 1 final quiz, all real `` nodes with - nested `` children. **45 scored questions total** (23 theme + - 18 final + 4 … see `COURSE_VERIFICATION.md`). Every question has ≥2 options + nested `` children. **45 scored questions total** (27 theme + + 18 final; see `COURSE_VERIFICATION.md`). Every question has ≥2 options and ≥1 `correct="true"`; **no empty option nodes**. - **Option order is randomised per attempt**: every `` carries `answerOrder="random"` (the Nottingham wizard's per-question "Answer Order" @@ -153,7 +153,10 @@ These were agreed with the course owner and must be preserved: only to a learner who answered it wrong (see "Wrong-answer help" below). A new question needs one; the hint must not give the answer away, and neither field may refer to an option by letter or position, because - `answerOrder="random"` shuffles them. + `answerOrder="random"` shuffles them. Wrong-answer help is read on its + own, so convention 2 does not carry across it: expand each acronym on + first use **within the hint and within the explanation**, even where the + same acronym is already expanded elsewhere on that page. ## Known deviations (acknowledged, not to "fix" without asking) @@ -514,8 +517,12 @@ wraps `quiz.showFeedBackandTrackResults` (re-wrapped from `x_pageLoaded`, because the model object is rebuilt for every quiz page) and, when `quiz.myProgress[quiz.currentQ] === true`, empties that slot again. -This is deliberately the failure-safe direction: if upstream ever renames the -hook, the help shows on correct answers as well instead of disappearing. The +If upstream ever renames that call the wrapper does nothing and the help shows +on correct answers as well, which loses nothing. A change to the slot order +would be worse — the wrapper would clear whatever else landed in that slot — +so `helpSlot` follows the engine's own `feedbackPos` rule (an *absent* +attribute means `GAC`; an empty one leaves the engine no slot letters at all, +so there is no G slot to clear) rather than assuming the first slot. The per-option `feedback` field was the other candidate and was not used — it says nothing to a learner who gets a *Multiple Answer* question wrong by under-selecting (every option they ticked was a correct one), and it would @@ -681,7 +688,7 @@ unzip -p Secure_code_development_scorm.zip template.xml | grep -oE 'trackingPass ```bash unzip -p Secure_code_development_scorm.zip template.xml > /tmp/c.xml grep -c "Trial MCQ" /tmp/c.xml # 0 -grep -oE ' -Welcome to Secure code development +Welcome to Secure code development

    This course is for engineers building software with agents and for engineers building agents. Whether you are designing agentic systems, integrating LLMs, or writing the services that agents call, secure code development matters.

    @@ -211,7 +211,7 @@
  • Detect delegated and transitive permissions — model the full delegation chain. If agent A delegates to agent B, the effective permissions should be the intersection of what A and B are each allowed to do.
  • Managed NHI (Non-Human Identity) — place agent identities in a dedicated identity platform such as Entra, AWS Bedrock, or Agentforce. Use lifecycle management, rotation, and audit for non-human accounts just as you would for human employees.
    • -]]>    This theme traces how untrusted input becomes unintended action. We start with classic injection, then follow the chain into agentic territory: agents whose goals can be hijacked, whose tools can be misused, and whose generated code can execute unexpectedly.

    +]]>    This theme traces how untrusted input becomes unintended action. We start with classic injection, then follow the chain into agentic territory: agents whose goals can be hijacked, whose tools can be misused, and whose generated code can execute unexpectedly.

    For engineers building agents, the lesson is that input validation, intent checks, and execution boundaries matter at every layer.

    ]]>    Injection @@ -333,7 +333,7 @@
  • Static scans before execution. Run security scanners and dependency checks on generated code before it is executed.
  • Runtime monitoring. Observe file, network, and process activity during code execution and terminate anomalous behaviour.
    • -]]>    Theme 3: Supply Chain & Integrity looks at trust in code, data, and context. In classic software, the supply chain is mostly static: libraries, packages, build tools, and the CI/CD (Continuous Integration / Continuous Deployment) pipeline that produces your artefact. If any of those ingredients are tampered with, the final application carries the flaw. The OWASP Top 10:2025 tracks this as Software Supply Chain Failures (A03:2025) and Software or Data Integrity Failures (A08:2025).

    +]]>    Theme 3: Supply Chain & Integrity looks at trust in code, data, and context. In classic software, the supply chain is mostly static: libraries, packages, build tools, and the CI/CD (Continuous Integration / Continuous Deployment) pipeline that produces your artefact. If any of those ingredients are tampered with, the final application carries the flaw. The OWASP Top 10:2025 tracks this as Software Supply Chain Failures (A03:2025) and Software or Data Integrity Failures (A08:2025).

    Agentic systems extend that same idea into a dynamic, runtime supply chain. An agent does not just import a dependency at build time; it can discover and compose tools, models, prompts, and other agents at runtime. This composition happens through protocols such as the Model Context Protocol (MCP) and Agent-to-Agent (A2A) frameworks, and through registries that advertise tool or agent capabilities. Each extra capability is a dependency that can be malicious, misconfigured, or compromised.

    @@ -496,7 +496,7 @@
  • Decay / expire unverified memory. Reduce the trust weight of unverified entries over time and eventually remove them.
  • Trust-weighted retrieval. Rank search results by source trust and confidence, not just embedding similarity.
    • -]]>    Theme 4: Cryptographic Failures & Secrets +]]>Theme 4: Cryptographic Failures & Secrets

    Theme 4 explores Cryptographic Failures (A04:2025) from the OWASP Top 10:2025, plus what those failures mean for agents and Non-Human Identities (NHI).

    @@ -653,7 +653,7 @@
  • Use digital-twin replay and policy gating to test risky decisions before live execution.
  • Keep tamper-evident logging and non-repudiation so every propagated action can be traced and audited.
    • -]]>    Theme 6 covers the risks that are unique to agentic systems, where software does not just respond to a single user but can act on its own, talk to other agents, and influence the people around it.

    +]]>    Theme 6 covers the risks that are unique to agentic systems, where software does not just respond to a single user but can act on its own, talk to other agents, and influence the people around it.

    Where earlier themes mapped classic application risks to the agent world, this theme is different: it focuses on three risks that only appear when multiple agents, or a human and an agent, share responsibility.

    @@ -990,7 +990,7 @@

    License of this course

    This course is licensed under CC BY-SA 4.0 (Creative Commons Attribution-ShareAlike 4.0). It is an adapted summary/derivative; changes were made; it is not endorsed by OWASP. Generated with the assistance of an AI coding agent.

    -]]>    Course complete +]]>Course complete
    diff --git a/source/preview.xml b/source/preview.xml index bcee21e..4a4f438 100644 --- a/source/preview.xml +++ b/source/preview.xml @@ -1,5 +1,5 @@ -Welcome to Secure code development +Welcome to Secure code development

    This course is for engineers building software with agents and for engineers building agents. Whether you are designing agentic systems, integrating LLMs, or writing the services that agents call, secure code development matters.

    @@ -211,7 +211,7 @@
  • Detect delegated and transitive permissions — model the full delegation chain. If agent A delegates to agent B, the effective permissions should be the intersection of what A and B are each allowed to do.
  • Managed NHI (Non-Human Identity) — place agent identities in a dedicated identity platform such as Entra, AWS Bedrock, or Agentforce. Use lifecycle management, rotation, and audit for non-human accounts just as you would for human employees.
    • -]]>    This theme traces how untrusted input becomes unintended action. We start with classic injection, then follow the chain into agentic territory: agents whose goals can be hijacked, whose tools can be misused, and whose generated code can execute unexpectedly.

    +]]>    This theme traces how untrusted input becomes unintended action. We start with classic injection, then follow the chain into agentic territory: agents whose goals can be hijacked, whose tools can be misused, and whose generated code can execute unexpectedly.

    For engineers building agents, the lesson is that input validation, intent checks, and execution boundaries matter at every layer.

    ]]>    Injection @@ -333,7 +333,7 @@
  • Static scans before execution. Run security scanners and dependency checks on generated code before it is executed.
  • Runtime monitoring. Observe file, network, and process activity during code execution and terminate anomalous behaviour.
    • -]]>    Theme 3: Supply Chain & Integrity looks at trust in code, data, and context. In classic software, the supply chain is mostly static: libraries, packages, build tools, and the CI/CD (Continuous Integration / Continuous Deployment) pipeline that produces your artefact. If any of those ingredients are tampered with, the final application carries the flaw. The OWASP Top 10:2025 tracks this as Software Supply Chain Failures (A03:2025) and Software or Data Integrity Failures (A08:2025).

    +]]>    Theme 3: Supply Chain & Integrity looks at trust in code, data, and context. In classic software, the supply chain is mostly static: libraries, packages, build tools, and the CI/CD (Continuous Integration / Continuous Deployment) pipeline that produces your artefact. If any of those ingredients are tampered with, the final application carries the flaw. The OWASP Top 10:2025 tracks this as Software Supply Chain Failures (A03:2025) and Software or Data Integrity Failures (A08:2025).

    Agentic systems extend that same idea into a dynamic, runtime supply chain. An agent does not just import a dependency at build time; it can discover and compose tools, models, prompts, and other agents at runtime. This composition happens through protocols such as the Model Context Protocol (MCP) and Agent-to-Agent (A2A) frameworks, and through registries that advertise tool or agent capabilities. Each extra capability is a dependency that can be malicious, misconfigured, or compromised.

    @@ -496,7 +496,7 @@
  • Decay / expire unverified memory. Reduce the trust weight of unverified entries over time and eventually remove them.
  • Trust-weighted retrieval. Rank search results by source trust and confidence, not just embedding similarity.
    • -]]>    Theme 4: Cryptographic Failures & Secrets +]]>Theme 4: Cryptographic Failures & Secrets

    Theme 4 explores Cryptographic Failures (A04:2025) from the OWASP Top 10:2025, plus what those failures mean for agents and Non-Human Identities (NHI).

    @@ -653,7 +653,7 @@
  • Use digital-twin replay and policy gating to test risky decisions before live execution.
  • Keep tamper-evident logging and non-repudiation so every propagated action can be traced and audited.
    • -]]>    Theme 6 covers the risks that are unique to agentic systems, where software does not just respond to a single user but can act on its own, talk to other agents, and influence the people around it.

    +]]>    Theme 6 covers the risks that are unique to agentic systems, where software does not just respond to a single user but can act on its own, talk to other agents, and influence the people around it.

    Where earlier themes mapped classic application risks to the agent world, this theme is different: it focuses on three risks that only appear when multiple agents, or a human and an agent, share responsibility.

    @@ -990,7 +990,7 @@

    License of this course

    This course is licensed under CC BY-SA 4.0 (Creative Commons Attribution-ShareAlike 4.0). It is an adapted summary/derivative; changes were made; it is not endorsed by OWASP. Generated with the assistance of an AI coding agent.

    -]]>    Course complete +]]>Course complete
    diff --git a/source/scorm_progress.js b/source/scorm_progress.js index e52326e..d09ed18 100644 --- a/source/scorm_progress.js +++ b/source/scorm_progress.js @@ -80,9 +80,12 @@ * models_html5/quiz.html renders into a feedback slot on every submitted * answer, right or wrong. This script clears that slot again when the * answer was correct, so the learner who needs the help sees it and the - * learner who does not is left with the plain "correct" message. If this - * hook ever stops matching the engine, the feedback simply shows on - * correct answers too: nothing is lost. + * learner who does not is left with the plain "correct" message. If the + * engine renames that call the wrapper does nothing and the feedback + * simply shows on correct answers too, which loses nothing; a change to + * the slot order would be worse, because the wrapper would clear whatever + * else landed there, so helpSlot follows the engine's own rule for + * feedbackPos rather than assuming the first slot. * * CONSTRAINT: the compiled one-liner is stored raw in an XML attribute, so it * must not contain < > & or " characters (tools/sync_root_script.py enforces @@ -692,9 +695,11 @@ function helpSlot(node) { var order = 'GAC'; var attr = node.getAttribute('feedbackPos'); - if (typeof attr === 'string') { - if (attr !== '') { order = attr; } - } + /* Only an absent attribute means GAC. The engine spreads the value it + finds, so an empty one leaves it no slot letters at all and puts the + right/wrong line in all three: G is then nowhere, and clearing the + first slot would delete the verdict instead. */ + if (typeof attr === 'string') { order = attr; } var pos = order.indexOf('G'); var id = FEEDBACK_SLOTS[pos]; if (typeof id !== 'string') { return null; } @@ -710,13 +715,25 @@ return quiz.myProgress[quiz.currentQ] === true; } + /* Where the panel is centred vertically the model slides the feedback in, + and jQuery measures the height it is animating to while the text is + still there. Emptying the element after that would animate a blank gap + open and snap it shut, so finish the animation and hide it instead. */ + function clearSlot(slot) { + if (typeof jQuery === 'function') { + jQuery(slot).stop(true, true).empty().hide(); + return; + } + slot.textContent = ''; + } + function hideHelpWhenRight() { if (!answerWasRight()) { return; } var node = questionNode(); if (!node) { return; } var slot = helpSlot(node); if (!slot) { return; } - slot.textContent = ''; + clearSlot(slot); } /* The model object is built afresh for every quiz page, so this runs from diff --git a/tests/test_scorm_tracking.py b/tests/test_scorm_tracking.py index 4093672..bb6854c 100644 --- a/tests/test_scorm_tracking.py +++ b/tests/test_scorm_tracking.py @@ -44,6 +44,7 @@ NR_PAGES = 45 # 44 content pages + the table of contents THEME1_QUIZ = 9 QUIZ_PAGES = [9, 15, 21, 24, 30, 35, 38, 43] +QUESTIONS = 45 # nodes across the 8 quizzes SUSPEND_LIMIT = 4096 # SCORM 1.2 cmi.suspend_data # A minimal but honest SCORM 1.2 API. Records everything so tests can assert on @@ -1013,8 +1014,18 @@ def test_wrong_answer_gets_a_hint_and_an_explanation(h: Harness): assert "Why:" in wrong["help"], f"no explanation on a wrong answer: {wrong}" s.next_question() right = s.answer_question(correctly=True) - assert "correctFeedback" in right["marking"], right + # "correctFeedback" is a substring of "incorrectFeedback", so match exactly + assert right["marking"].split() == ["correctFeedback"], right assert right["help"] == "", f"a correct answer should not be given help: {right}" + # clearing the help must not take the correct/incorrect line with it + assert right["verdict"], f"the verdict was cleared too: {right}" + # a second quiz page rebuilds the model object, which has to be re-wrapped + s.goto(QUIZ_PAGES[1]) + again = s.answer_question(correctly=False) + assert again["help"].startswith("Hint:"), f"not re-wrapped on a later quiz: {again}" + s.next_question() + still = s.answer_question(correctly=True) + assert still["help"] == "", f"help not cleared on a later quiz: {still}" assert not s.page_errors, f"page errors: {s.page_errors}" @@ -1027,17 +1038,20 @@ def test_every_question_carries_wrong_answer_help(h: Harness): """ with h.session() as s: s.open() - missing = s.sco.evaluate( - "() => { const out = [];" + walk = s.sco.evaluate( + "() => { const out = []; let checked = 0;" " window.jQuery(x_pages).each(function () {" " if (this.nodeName !== 'quiz') { return; }" " window.jQuery(this).children('question').each(function () {" + " checked++;" " const fb = this.getAttribute('feedback') || '';" " if (fb.indexOf('Hint:') === -1 || fb.indexOf('Why:') === -1) {" " out.push(this.getAttribute('name')); } }); });" - " return out; }" + " return {checked: checked, missing: out}; }" ) - assert missing == [], f"questions without a hint and explanation: {missing}" + # without this the walk finding nothing at all would pass silently + assert walk["checked"] == QUESTIONS, f"walked {walk['checked']} questions, expected {QUESTIONS}" + assert walk["missing"] == [], f"questions without a hint and explanation: {walk['missing']}" assert not s.page_errors, f"page errors: {s.page_errors}" From d736c8303b26b23ccb445a23aa43e5c23c842a4e Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 15 Sep 2026 09:44:27 +0000 Subject: [PATCH 3/5] docs: plainer prose in the wrong-answer help documentation The new documentation had the tells of machine-written text: bolded mini headings chopping short prose into labelled chunks, em-dash asides stacked two to a sentence, a rule-of-three summary line, and one bullet carrying four separate corrections in a single semicolon-joined sentence. The conventions were also headed with slogans ("Every question teaches on failure") where a plain name does the job. Rewritten as prose. No technical claim, count, file path, attribute name or test name changes; the README bullet now says what a correct answer shows instead of leaving it as an implied contrast. The 45 learner-facing strings were checked against the same criteria and left alone: they carry no stock contrast constructions, no inflated vocabulary and no em-dashes, and their three-item lists enumerate real OWASP controls rather than padding a cadence. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_0149Y1yzjVXVWwXfsKKeDASd --- README.md | 6 +-- docs/COURSE_SPEC.md | 8 ++-- docs/COURSE_VERIFICATION.md | 65 ++++++++++++++++++-------------- docs/PROJECT_CONTEXT.md | 75 ++++++++++++++++++------------------- 4 files changed, 81 insertions(+), 73 deletions(-) diff --git a/README.md b/README.md index 3d83436..1bb6b67 100644 --- a/README.md +++ b/README.md @@ -203,9 +203,9 @@ These are binding on this derivative work — see ## Progress and completion: what learners and the LMS see -- **In a quiz**: a wrong answer is met with a hint (what to think about, and - which theme to revisit) and an explanation of the right answer; a correct - answer is not. See +- **In a quiz**: answering a question wrong brings up a hint (what to think + about, and which theme to revisit) and an explanation of the right answer. + Getting it right shows the usual confirmation and nothing more. See [`docs/PROJECT_CONTEXT.md`](docs/PROJECT_CONTEXT.md#wrong-answer-help-hint--explanation-on-every-question). - **In the course**: a header progress bar (percentage of pages opened, with a marker per quiz), contents-page ticks that mean *opened* for content pages diff --git a/docs/COURSE_SPEC.md b/docs/COURSE_SPEC.md index cd19bdc..f16b3a3 100644 --- a/docs/COURSE_SPEC.md +++ b/docs/COURSE_SPEC.md @@ -29,10 +29,10 @@ Model, version 1.2)** package with **LMS (Learning Management System)-reported s releases. 3. **No free-text questions**: all quiz items are selection-based (single choice, multiple response, true/false, matching, categorise). -4. **Every question teaches on failure**: each question carries a **hint** (what to think about - and which theme to revisit) and an **explanation** (why the right answer is right, and where - it helps, why the tempting distractor is not), shown **only when the learner answered it - wrong**. The hint must not give the answer away; neither may refer to an option by letter or +4. **Wrong-answer help on every question**: each question carries a hint (what to think about + and which theme to revisit) and an explanation (why the right answer is right, and where + it helps, why the tempting distractor is not), shown only when the learner answered it + wrong. The hint must not give the answer away; neither may refer to an option by letter or position, because the options shuffle per attempt. Both are held in the question's `feedback` attribute — see `PROJECT_CONTEXT.md` § "Wrong-answer help". This text is read on its own, so convention 1 does not carry across it: expand each acronym on first use diff --git a/docs/COURSE_VERIFICATION.md b/docs/COURSE_VERIFICATION.md index 8b346ce..c66ad33 100644 --- a/docs/COURSE_VERIFICATION.md +++ b/docs/COURSE_VERIFICATION.md @@ -308,34 +308,43 @@ per-option `feedback` field was not used. ### Review pass (three reviewers, post-CI) -The first green build was reviewed for runtime correctness, content/XML -integrity and fidelity to the OWASP sources. Corrections applied: - -- **Mitigations brought up to the 2025 text** (verified against the live - `top10.owasp.org/2025` pages): password storage now names an adaptive salted - hash with a work factor (Argon2, yescrypt, scrypt, PBKDF2-HMAC-SHA-512) with - bcrypt as legacy-only, per A04; mishandling of exceptional conditions now - teaches catching at source, a global handler, full rollback ("failing - closed") and rate limits, per A10, instead of Security Misconfiguration's - stack-trace advice; supply chain drops "reproducible builds" (absent from - A03) for official sources, deliberate versions and hardened CI/CD; insecure - design drops the retired "usage limits" bullet for the secure development - lifecycle and secure design patterns A06 leads with. -- **Taxonomy**: broken access control is no longer described as human-only - (OWASP says *users*, which is the point for agent and non-human identities). -- **Two adjacent final-quiz hints** stated opposite classification rules - (by harm / by root cause); both now point at the stem's own disambiguator. -- **One hint resolved against its own keyed answer**, one leaked the answer by - word stem, one expansion was not title case — all corrected. -- **Runtime**: `helpSlot` now follows the engine's `feedbackPos` rule exactly - (an empty value leaves no G slot, so nothing is cleared — previously it - would have deleted the verdict line); the slot is cleared through jQuery so - the model's slide-in animation cannot animate an emptied block. -- **Tests**: the every-question walk now asserts it inspected 45 questions - (it would previously have passed if the walk found none); - `"correctFeedback" in "incorrectFeedback"` is true as a Python substring, so - the marking assertion is now exact; the wrong-answer test covers a second - quiz page, which is what the re-wrap on `x_pageLoaded` exists for. +The first green build was reviewed for runtime correctness, content and XML +integrity, and fidelity to the OWASP sources. The mitigations were the weakest +part and four were out of date against the live `top10.owasp.org/2025` pages: + +- Password storage named bcrypt, which A04 now mentions only for legacy + systems. It now names an adaptive salted hash with a work factor (Argon2, + yescrypt, scrypt, PBKDF2-HMAC-SHA-512). +- Mishandling of exceptional conditions carried Security Misconfiguration's + advice about keeping detail out of errors. It now teaches what A10 leads + with: catching at source, a global handler, full rollback (its own term is + "failing closed") and rate limits. +- Supply chain recommended reproducible builds, a phrase absent from A03. It + now names official sources, deliberately chosen versions and hardened CI/CD. +- Insecure design kept "usage limits", a bullet dropped from A06, in place of + the secure development lifecycle and secure design patterns it now leads + with. + +Four content problems were fixed as well: broken access control was described +as human-only, where OWASP says users, which is the whole point for agent and +non-human identities; two adjacent final-quiz hints stated opposite +classification rules, and both now point at the stem's own disambiguator; one +hint argued against its own keyed answer; and one leaked its answer by word +stem. + +Two latent runtime defects were corrected. `helpSlot` now follows the engine's +`feedbackPos` rule exactly, because an empty value leaves no G slot and the +previous code would have deleted the verdict line. The slot is also cleared +through jQuery, so the model's slide-in animation cannot animate a block that +has already been emptied. + +Two tests could pass while the feature was broken. The every-question walk +asserted only that its list of failures was empty, so it would have passed if +the walk had matched nothing; it now asserts it inspected all 45. +`"correctFeedback"` is a Python substring of `"incorrectFeedback"`, so that +assertion held whatever the grading outcome; it is now an exact match. The +wrong-answer test also continues onto a second quiz page, which is the case +the re-wrap on `x_pageLoaded` exists for. **Pending (convention 9 — MANDATORY before merge)**: the PR's CI job builds the package in a real XOT container and runs `tests/test_scorm_tracking.py` diff --git a/docs/PROJECT_CONTEXT.md b/docs/PROJECT_CONTEXT.md index 3ffb484..67efd3c 100644 --- a/docs/PROJECT_CONTEXT.md +++ b/docs/PROJECT_CONTEXT.md @@ -148,8 +148,8 @@ These were agreed with the course owner and must be preserved: computed from the shuffled array, so SCORM tracking is unaffected. When adding a new question, set `answerOrder="random"` on it. Keep `data.xml` and `preview.xml` byte-identical. -11. **Every question explains itself when it is failed**: each `` - carries a `feedback` attribute holding a hint and an explanation, shown +11. **Wrong-answer help on every question**: each `` carries a + `feedback` attribute holding a hint and an explanation, shown only to a learner who answered it wrong (see "Wrong-answer help" below). A new question needs one; the hint must not give the answer away, and neither field may refer to an option by letter or position, because @@ -493,54 +493,53 @@ end of `test_full_walkthrough_reports_passed`. A learner who picked the wrong option used to see nothing but *"Your answer is incorrect"*, so there was nothing to learn from and nothing to revise before -the restart. Every one of the 45 questions now carries a **hint** (what to -think about, and which theme to revisit) and an **explanation** (why the right -answer is right, and where useful why the tempting distractor is not), shown -**only when the answer was wrong**. +the restart. All 45 questions now carry a hint (what to think about, and which +theme to revisit) and an explanation (why the right answer is right, and where +useful why the tempting distractor is not), shown only when the answer was +wrong. -**Where the text lives.** In the question's own `feedback` attribute — the -quiz wizard's *General Feedback* field (`quiz.xwd`, so the XOT editor exposes -it as a normal per-question text area and round-trips it). The value is the -same double-escaped HTML as `prompt`/`text`: +The text lives in the question's own `feedback` attribute, the quiz wizard's +*General Feedback* field (`quiz.xwd`), so the XOT editor exposes it as a normal +per-question text area and round-trips it. The value carries the same +double-escaped HTML as `prompt` and `text`: ```xml ``` -**Why it needs the root script.** `models_html5/quiz.html` renders that -attribute on **every** submitted answer, right or wrong: after grading it +It needs the root script because `models_html5/quiz.html` renders that +attribute on every submitted answer, right or wrong. After grading, the model fills three fixed slots (`#topFeedback`, `#middleFeedback`, `#bottomFeedback`) -in the order the question's `feedbackPos` gives — `G` the question's own -feedback, `A` the selected option's, `C` the right/wrong line — defaulting to -`GAC`, so ours lands in `#topFeedback`. Part 6 of `source/scorm_progress.js` -wraps `quiz.showFeedBackandTrackResults` (re-wrapped from `x_pageLoaded`, -because the model object is rebuilt for every quiz page) and, when -`quiz.myProgress[quiz.currentQ] === true`, empties that slot again. +in the order the question's `feedbackPos` gives, one letter per slot: `G` is +the question's own feedback, `A` the selected option's, `C` the right/wrong +line. That defaults to `GAC`, so ours lands in `#topFeedback`. Part 6 of +`source/scorm_progress.js` wraps `quiz.showFeedBackandTrackResults`, re-wrapped +from `x_pageLoaded` because the model object is rebuilt for every quiz page, +and empties that slot again when `quiz.myProgress[quiz.currentQ] === true`. If upstream ever renames that call the wrapper does nothing and the help shows on correct answers as well, which loses nothing. A change to the slot order -would be worse — the wrapper would clear whatever else landed in that slot — -so `helpSlot` follows the engine's own `feedbackPos` rule (an *absent* -attribute means `GAC`; an empty one leaves the engine no slot letters at all, -so there is no G slot to clear) rather than assuming the first slot. The -per-option `feedback` field was the other candidate and was not used — it says -nothing to a learner who gets a *Multiple Answer* question wrong by -under-selecting (every option they ticked was a correct one), and it would -duplicate the same paragraph across each wrong option. - -**Nothing about tracking changes**: `XTExitInteraction` is handed -`trackData.feedback`, which is built from the *option* `feedback` values (all -still empty), so SCORM interaction records are untouched. +would be worse, because the wrapper would clear whatever else landed in that +slot. So `helpSlot` follows the engine's own `feedbackPos` rule rather than +assuming the first slot: an *absent* attribute means `GAC`, while an empty one +leaves the engine no slot letters at all, so there is no G slot to clear. + +The per-option `feedback` field was the other candidate and was not used. It +says nothing to a learner who gets a *Multiple Answer* question wrong by +under-selecting, because every option they ticked was a correct one, and it +would repeat the same paragraph on each wrong option. + +Tracking is unaffected. `XTExitInteraction` is handed `trackData.feedback`, +which is built from the *option* `feedback` values, and those are all still +empty, so the SCORM interaction records are untouched. Tests: `test_wrong_answer_gets_a_hint_and_an_explanation` (a wrong answer is -given both, a correct one is not) and -`test_every_question_carries_wrong_answer_help` (all 45 questions have it, so -a question added later cannot silently ship without one). Reviewers can read -the text in the PR preview — `tools/render_preview.py` renders it under each -question. - -**Adding a question**: give it a `feedback` value in the same shape, or -`test_every_question_carries_wrong_answer_help` fails. +given both, a correct one is not, across two quiz pages) and +`test_every_question_carries_wrong_answer_help` (all 45 questions have it, so a +question added later cannot silently ship without one). Reviewers can read the +text in the PR preview, which `tools/render_preview.py` renders under each +question. A question added later needs a `feedback` value in the same shape, or +that second test fails. ## Build version stamping (`{{BUILD_VERSION}}` placeholder) From f12c05e1007eab186a5b8d63f85c8f6fd381ded3 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 15 Sep 2026 10:22:32 +0000 Subject: [PATCH 4/5] test(progress): cover viewed state across a save and resume Reported: save the activity, leave, come back, and the header progress bar reads 0% with the contents-page ticks gone. Both read x_pageInfo[i].viewed, which is session state. It survives a resume only through the engine's pagesViewed field: x_pagesViewed() -> state.pagesViewed -> cmi.suspend_data -> setVars -> x_restorePagesViewed(). Nothing covered that round trip. test_header_progress_bar_counts_viewed_pages never resumes, and test_resume_returns_to_the_saved_page only checks which page comes back, so a break anywhere along the chain shipped unnoticed. This test walks three pages, terminates, and asserts pagesViewed is in the saved record with those pages in it, then resumes and asserts the viewed flags, the progress bar and the ticks all come back. The halves are separate so a failure says whether the record was written wrong or read back wrong. Reading the code did not find the fault: persist() sets state.pagesViewed before saving, repairOutgoing keeps it, repairIncoming filters it against the live page list, and x_pageInfo is built in x_projectDataLoaded well before setVars runs. Two candidates remain that the test will discriminate. fitBudget sheds pagesViewed second, before quiz scores, so a record over the 4000 character budget loses the ticks first. And persist() sets cmi.core.exit to '' whenever the status is not incomplete, which some LMSs treat as attempt-over and answer with empty suspend data on the next launch. This environment has no Docker, so CI is the only executor: the test is pushed to reproduce, not because it is known to pass. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_0149Y1yzjVXVWwXfsKKeDASd --- tests/test_scorm_tracking.py | 71 ++++++++++++++++++++++++++++++++++++ 1 file changed, 71 insertions(+) diff --git a/tests/test_scorm_tracking.py b/tests/test_scorm_tracking.py index bb6854c..1a8bd8d 100644 --- a/tests/test_scorm_tracking.py +++ b/tests/test_scorm_tracking.py @@ -408,6 +408,27 @@ def lms_status_shown(s: Session, page_nr: int) -> str: return s.sco.get_attribute(f"#x_page{page_nr} .xtLmsStatus", "data-status") +UNTICKED_PAGES = """ +() => { + const out = []; + const offset = XENITH.PAGEMENU.menuPage ? 1 : 0; + window.jQuery('#x_page0 .menuItem').each(function () { + if (window.jQuery(this).find('i.notvisited').length) { + out.push(x_normalPages[window.jQuery(this).data('pageIndex') + offset]); + } + }); + return out; +} +""" + +VIEWED_PAGES = """ +() => { + const out = []; + x_pageInfo.forEach(function (p, i) { if (p.viewed === true) { out.push(i); } }); + return out; +} +""" + PENDING_QUIZ_TICKS = """ () => { const out = []; @@ -427,6 +448,16 @@ def pending_quiz_ticks(s: Session) -> list[int]: return s.sco.evaluate(PENDING_QUIZ_TICKS) +def unticked_pages(s: Session) -> list[int]: + """Page indexes the contents page still shows as never visited.""" + return s.sco.evaluate(UNTICKED_PAGES) + + +def viewed_pages(s: Session) -> list[int]: + """Page indexes the engine currently considers viewed.""" + return s.sco.evaluate(VIEWED_PAGES) + + def assert_resumable(record: dict, where: str) -> None: interactions = record.get("interactions") assert isinstance(interactions, list), f"{where}: interactions is not a list" @@ -1106,6 +1137,46 @@ def test_older_records_infer_submission_from_the_score(h: Harness): assert "Theme 1 Quiz: 75%" in panel_text(s, MENU_PAGE), panel_text(s, MENU_PAGE) +@test +def test_viewed_pages_survive_a_save_and_resume(h: Harness): + """Ticks and the progress bar must still mean something after a resume. + + Both read `x_pageInfo[i].viewed`, which is session state: it survives only + through the engine's `pagesViewed` field in the saved record + (`x_pagesViewed()` -> `state.pagesViewed` -> `cmi.suspend_data` -> + `setVars` -> `x_restorePagesViewed()`). A learner who saves, leaves and + comes back to an empty progress bar and no ticks has lost that round trip. + The two halves are asserted separately so a failure names which one broke. + """ + seen = [FIRST_CONTENT_PAGE, FIRST_CONTENT_PAGE + 1, FIRST_CONTENT_PAGE + 2] + with h.session() as s: + s.open() + for page_nr in seen: + s.goto(page_nr) + s.terminate() + saved = s.suspends()[-1] + record = json.loads(saved) + # save side + assert "pagesViewed" in record, ( + f"the saved record has no pagesViewed, so nothing can be restored: {sorted(record)}" + ) + missing = [p for p in seen if p not in record["pagesViewed"]] + assert not missing, f"visited pages missing from pagesViewed: {missing}" + assert not s.page_errors, f"page errors: {s.page_errors}" + # restore side + with h.resumed(saved) as s: + s.open() + restored = viewed_pages(s) + lost = [p for p in seen if p not in restored] + assert not lost, f"pages lost their viewed flag on resume: {lost} (restored {restored})" + s.goto(MENU_PAGE) + label = s.sco.inner_text("#x_headerProgress .pbTxt") + assert not label.startswith("0%"), f"progress bar empty after a resume: {label}" + still_unticked = [p for p in seen if p in unticked_pages(s)] + assert not still_unticked, f"contents-page ticks lost on resume: {still_unticked}" + assert not s.page_errors, f"page errors: {s.page_errors}" + + @test def test_header_progress_bar_counts_viewed_pages(h: Harness): """The header progress bar is on, labelled honestly, with a marker per quiz.""" From 4a5b9d690b5239a6fe7c18f737329cdd38b3fc44 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 15 Sep 2026 10:33:11 +0000 Subject: [PATCH 5/5] test(progress): reproduce the lost ticks with a record that has grown The first resume test passes: after three pages, pagesViewed is written and restored, and the ticks and bar come back. So the round trip is sound and the reproduction was too gentle. fitBudget sheds pagesViewed second when the record will not fit the 4000 character suspend_data budget, ahead of the quiz scores, which makes the ticks and the progress bar the first learner-visible thing to go as a record grows. A learner saves after a real session, not after three pages. This walks pages 1 to 35, answering the quizzes on the way, then saves and resumes. It asserts the record still fits, that pagesViewed survived, and that the flags, ticks and bar come back, with the record size in the failure message so a red run says how far over budget it went. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_0149Y1yzjVXVWwXfsKKeDASd --- tests/test_scorm_tracking.py | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/tests/test_scorm_tracking.py b/tests/test_scorm_tracking.py index 1a8bd8d..b3870c6 100644 --- a/tests/test_scorm_tracking.py +++ b/tests/test_scorm_tracking.py @@ -1177,6 +1177,41 @@ def test_viewed_pages_survive_a_save_and_resume(h: Harness): assert not s.page_errors, f"page errors: {s.page_errors}" +@test +def test_viewed_pages_survive_a_resume_late_in_the_course(h: Harness): + """The same round trip for a learner who has actually done the work. + + `pagesViewed` is the second thing `fitBudget` sheds when the record will + not fit `cmi.suspend_data`, ahead of the quiz scores, so the ticks and the + progress bar are the first learner-visible casualty of a record that has + grown. A learner saves after a real session, not after three pages. + """ + walked = list(range(FIRST_CONTENT_PAGE, 36)) + with h.session() as s: + s.open() + for page_nr in walked: + s.goto(page_nr, dwell=120) + if page_nr in QUIZ_PAGES: + s.answer_quiz() + s.terminate() + saved = s.suspends()[-1] + record = json.loads(saved) + assert len(saved) <= SUSPEND_LIMIT, f"record is {len(saved)} characters" + assert "pagesViewed" in record, ( + f"pagesViewed was shed from a {len(saved)} character record: kept {sorted(record)}" + ) + assert not s.page_errors, f"page errors: {s.page_errors}" + with h.resumed(saved) as s: + s.open() + restored = viewed_pages(s) + lost = [p for p in walked if p not in restored] + assert not lost, f"pages lost their viewed flag on resume: {lost}" + s.goto(MENU_PAGE) + label = s.sco.inner_text("#x_headerProgress .pbTxt") + assert not label.startswith("0%"), f"progress bar empty after a resume: {label}" + assert not s.page_errors, f"page errors: {s.page_errors}" + + @test def test_header_progress_bar_counts_viewed_pages(h: Harness): """The header progress bar is on, labelled honestly, with a marker per quiz."""