diff --git a/README.md b/README.md index 01933cd..1bb6b67 100644 --- a/README.md +++ b/README.md @@ -194,7 +194,8 @@ These are binding on this derivative work — see time-of-use)`. - **Quizzes**: full descriptive names in question stems (never "What causes A01" — OWASP renumbers between releases); no free-text questions; no empty - answer options; questions nested under their `` node. + answer options; questions nested under their `` node; every question + carries a hint and an explanation, shown when a learner answers it wrong. - **No auto-playing text**: content pages show all content at once (Plain Text, or Bullets with `delaySecs="0"`). - **Incident callouts** must be self-contained for learners who haven't read the @@ -202,6 +203,10 @@ These are binding on this derivative work — see ## Progress and completion: what learners and the LMS see +- **In a quiz**: answering a question wrong brings up a hint (what to think + about, and which theme to revisit) and an explanation of the right answer. + Getting it right shows the usual confirmation and nothing more. See + [`docs/PROJECT_CONTEXT.md`](docs/PROJECT_CONTEXT.md#wrong-answer-help-hint--explanation-on-every-question). - **In the course**: a header progress bar (percentage of pages opened, with a marker per quiz), contents-page ticks that mean *opened* for content pages and *submitted* for quizzes (an opened-but-unsubmitted quiz shows a diff --git a/docs/COURSE_SPEC.md b/docs/COURSE_SPEC.md index 2872ac0..f16b3a3 100644 --- a/docs/COURSE_SPEC.md +++ b/docs/COURSE_SPEC.md @@ -29,6 +29,14 @@ Model, version 1.2)** package with **LMS (Learning Management System)-reported s releases. 3. **No free-text questions**: all quiz items are selection-based (single choice, multiple response, true/false, matching, categorise). +4. **Wrong-answer help on every question**: each question carries a hint (what to think about + and which theme to revisit) and an explanation (why the right answer is right, and where + it helps, why the tempting distractor is not), shown only when the learner answered it + wrong. The hint must not give the answer away; neither may refer to an option by letter or + position, because the options shuffle per attempt. Both are held in the question's + `feedback` attribute — see `PROJECT_CONTEXT.md` § "Wrong-answer help". This text is read + on its own, so convention 1 does not carry across it: expand each acronym on first use + within the hint and within the explanation, even where it is already expanded on the page. ## Template / technical setup @@ -44,6 +52,9 @@ Model, version 1.2)** package with **LMS (Learning Management System)-reported s pages done/still to visit, quiz results, weighted score and the status reported to the LMS. - **Option order**: every quiz question sets `answerOrder="random"` so the answer options shuffle on each attempt (Nottingham per-question "Answer Order" property). +- **Wrong-answer help**: every quiz question sets `feedback` (the per-question "General + Feedback" property) to a hint plus an explanation; the root script clears it again when the + answer was right, so only a learner who got it wrong is shown it. - **Question types used** (all selection-based, no free text): Multiple Choice (single), Multiple Response (select all), True/False, Matching, Categorise. - Editor opened directly at `/edit.php?template_id=`; **Publish** before export. diff --git a/docs/COURSE_VERIFICATION.md b/docs/COURSE_VERIFICATION.md index 207d6a3..c66ad33 100644 --- a/docs/COURSE_VERIFICATION.md +++ b/docs/COURSE_VERIFICATION.md @@ -278,3 +278,79 @@ other tracking attrs), export a SCORM package, and grep the exported `template.xml` for the checklist values in `PROJECT_CONTEXT.md`. Then walk a quiz end-to-end (answer → Check → Next → complete → Restart) and confirm options reorder between attempts. + +## Wrong-answer help: hint + explanation on every question (post-0.0.6) + +Every `` gained a `feedback` attribute (the per-question *General +Feedback* field) holding a hint and an explanation, and part 6 of +`source/scorm_progress.js` clears that feedback slot again when the answer was +right, so only a learner who got the question wrong is shown it. See +`PROJECT_CONTEXT.md` § "Wrong-answer help" for the mechanism and why the +per-option `feedback` field was not used. + +### Local checks (run against `source/data.xml`; XOT re-export still required per convention 9) + +- 45 `` elements, 45 with a `feedback` attribute; each unescapes to + `

Hint: …

Why: …

`, the same + double-escaped shape as the existing `prompt`/`text` values. +- No hint or explanation names an option by letter or position (options + shuffle per attempt), and no question-level `feedback` gives the answer away + in its hint. +- Option `feedback` attributes are still empty ×161, so the SCORM interaction + records (`XTExitInteraction` is passed the *option* feedback) are unchanged. +- `trackingWeight`, `trackingMode`, `trackingPassed`, `judge="true"` ×8, + `answerOrder="random"` ×45, `delaySecs="0"` ×27 unchanged. +- `python3 tools/sync_root_script.py --check` passes; the compiled root script + extracted from `data.xml` passes `node --check`. +- `tools/render_preview.py` renders 44 pages without error, with the help text + under each question for review. +- `source/data.xml` and `source/preview.xml` are byte-identical. + +### Review pass (three reviewers, post-CI) + +The first green build was reviewed for runtime correctness, content and XML +integrity, and fidelity to the OWASP sources. The mitigations were the weakest +part and four were out of date against the live `top10.owasp.org/2025` pages: + +- Password storage named bcrypt, which A04 now mentions only for legacy + systems. It now names an adaptive salted hash with a work factor (Argon2, + yescrypt, scrypt, PBKDF2-HMAC-SHA-512). +- Mishandling of exceptional conditions carried Security Misconfiguration's + advice about keeping detail out of errors. It now teaches what A10 leads + with: catching at source, a global handler, full rollback (its own term is + "failing closed") and rate limits. +- Supply chain recommended reproducible builds, a phrase absent from A03. It + now names official sources, deliberately chosen versions and hardened CI/CD. +- Insecure design kept "usage limits", a bullet dropped from A06, in place of + the secure development lifecycle and secure design patterns it now leads + with. + +Four content problems were fixed as well: broken access control was described +as human-only, where OWASP says users, which is the whole point for agent and +non-human identities; two adjacent final-quiz hints stated opposite +classification rules, and both now point at the stem's own disambiguator; one +hint argued against its own keyed answer; and one leaked its answer by word +stem. + +Two latent runtime defects were corrected. `helpSlot` now follows the engine's +`feedbackPos` rule exactly, because an empty value leaves no G slot and the +previous code would have deleted the verdict line. The slot is also cleared +through jQuery, so the model's slide-in animation cannot animate a block that +has already been emptied. + +Two tests could pass while the feature was broken. The every-question walk +asserted only that its list of failures was empty, so it would have passed if +the walk had matched nothing; it now asserts it inspected all 45. +`"correctFeedback"` is a Python substring of `"incorrectFeedback"`, so that +assertion held whatever the grading outcome; it is now an exact match. The +wrong-answer test also continues onto a second quiz page, which is the case +the re-wrap on `x_pageLoaded` exists for. + +**Pending (convention 9 — MANDATORY before merge)**: the PR's CI job builds the +package in a real XOT container and runs `tests/test_scorm_tracking.py` +(including `test_wrong_answer_gets_a_hint_and_an_explanation` and +`test_every_question_carries_wrong_answer_help`) — that is the import/export +gate. Still to do by hand: open the HTML5 editor and confirm a Publish +round-trip preserves the new `feedback` values, and walk a quiz in `play.php` +answering one question wrong and one right, confirming the help appears only +on the wrong one and that the layout holds on a narrow screen. diff --git a/docs/PROJECT_CONTEXT.md b/docs/PROJECT_CONTEXT.md index 9925bc7..67efd3c 100644 --- a/docs/PROJECT_CONTEXT.md +++ b/docs/PROJECT_CONTEXT.md @@ -64,8 +64,8 @@ with agents and building agents. - **All 20 OWASP items** present: A01–A10 (classic) + ASI01–ASI10 (agentic), each with its own page, interleaved by theme. - **Quizzes**: 7 theme quizzes + 1 final quiz, all real `` nodes with - nested `` children. **45 scored questions total** (23 theme + - 18 final + 4 … see `COURSE_VERIFICATION.md`). Every question has ≥2 options + nested `` children. **45 scored questions total** (27 theme + + 18 final; see `COURSE_VERIFICATION.md`). Every question has ≥2 options and ≥1 `correct="true"`; **no empty option nodes**. - **Option order is randomised per attempt**: every `` carries `answerOrder="random"` (the Nottingham wizard's per-question "Answer Order" @@ -148,6 +148,15 @@ These were agreed with the course owner and must be preserved: computed from the shuffled array, so SCORM tracking is unaffected. When adding a new question, set `answerOrder="random"` on it. Keep `data.xml` and `preview.xml` byte-identical. +11. **Wrong-answer help on every question**: each `` carries a + `feedback` attribute holding a hint and an explanation, shown + only to a learner who answered it wrong (see "Wrong-answer help" below). + A new question needs one; the hint must not give the answer away, and + neither field may refer to an option by letter or position, because + `answerOrder="random"` shuffles them. Wrong-answer help is read on its + own, so convention 2 does not carry across it: expand each acronym on + first use **within the hint and within the explanation**, even where the + same acronym is already expanded elsewhere on that page. ## Known deviations (acknowledged, not to "fix" without asking) @@ -480,6 +489,58 @@ Tests: `test_status_panel_tells_the_learner_what_is_left`, `test_header_progress_bar_counts_viewed_pages`, and the panel assertions at the end of `test_full_walkthrough_reports_passed`. +## Wrong-answer help (hint + explanation on every question) + +A learner who picked the wrong option used to see nothing but *"Your answer is +incorrect"*, so there was nothing to learn from and nothing to revise before +the restart. All 45 questions now carry a hint (what to think about, and which +theme to revisit) and an explanation (why the right answer is right, and where +useful why the tempting distractor is not), shown only when the answer was +wrong. + +The text lives in the question's own `feedback` attribute, the quiz wizard's +*General Feedback* field (`quiz.xwd`), so the XOT editor exposes it as a normal +per-question text area and round-trips it. The value carries the same +double-escaped HTML as `prompt` and `text`: + +```xml + +``` + +It needs the root script because `models_html5/quiz.html` renders that +attribute on every submitted answer, right or wrong. After grading, the model +fills three fixed slots (`#topFeedback`, `#middleFeedback`, `#bottomFeedback`) +in the order the question's `feedbackPos` gives, one letter per slot: `G` is +the question's own feedback, `A` the selected option's, `C` the right/wrong +line. That defaults to `GAC`, so ours lands in `#topFeedback`. Part 6 of +`source/scorm_progress.js` wraps `quiz.showFeedBackandTrackResults`, re-wrapped +from `x_pageLoaded` because the model object is rebuilt for every quiz page, +and empties that slot again when `quiz.myProgress[quiz.currentQ] === true`. + +If upstream ever renames that call the wrapper does nothing and the help shows +on correct answers as well, which loses nothing. A change to the slot order +would be worse, because the wrapper would clear whatever else landed in that +slot. So `helpSlot` follows the engine's own `feedbackPos` rule rather than +assuming the first slot: an *absent* attribute means `GAC`, while an empty one +leaves the engine no slot letters at all, so there is no G slot to clear. + +The per-option `feedback` field was the other candidate and was not used. It +says nothing to a learner who gets a *Multiple Answer* question wrong by +under-selecting, because every option they ticked was a correct one, and it +would repeat the same paragraph on each wrong option. + +Tracking is unaffected. `XTExitInteraction` is handed `trackData.feedback`, +which is built from the *option* `feedback` values, and those are all still +empty, so the SCORM interaction records are untouched. + +Tests: `test_wrong_answer_gets_a_hint_and_an_explanation` (a wrong answer is +given both, a correct one is not, across two quiz pages) and +`test_every_question_carries_wrong_answer_help` (all 45 questions have it, so a +question added later cannot silently ship without one). Reviewers can read the +text in the PR preview, which `tools/render_preview.py` renders under each +question. A question added later needs a `feedback` value in the same shape, or +that second test fails. + ## Build version stamping (`{{BUILD_VERSION}}` placeholder) So an operator can tell **which version is running** in an LMS, the SCORM @@ -626,8 +687,9 @@ unzip -p Secure_code_development_scorm.zip template.xml | grep -oE 'trackingPass ```bash unzip -p Secure_code_development_scorm.zip template.xml > /tmp/c.xml grep -c "Trial MCQ" /tmp/c.xml # 0 -grep -oE ' -Welcome to Secure code development +Welcome to Secure code development

This course is for engineers building software with agents and for engineers building agents. Whether you are designing agentic systems, integrating LLMs, or writing the services that agents call, secure code development matters.

@@ -211,7 +211,7 @@
  • Detect delegated and transitive permissions — model the full delegation chain. If agent A delegates to agent B, the effective permissions should be the intersection of what A and B are each allowed to do.
  • Managed NHI (Non-Human Identity) — place agent identities in a dedicated identity platform such as Entra, AWS Bedrock, or Agentforce. Use lifecycle management, rotation, and audit for non-human accounts just as you would for human employees.
    • -]]>    This theme traces how untrusted input becomes unintended action. We start with classic injection, then follow the chain into agentic territory: agents whose goals can be hijacked, whose tools can be misused, and whose generated code can execute unexpectedly.

    +]]>    This theme traces how untrusted input becomes unintended action. We start with classic injection, then follow the chain into agentic territory: agents whose goals can be hijacked, whose tools can be misused, and whose generated code can execute unexpectedly.

    For engineers building agents, the lesson is that input validation, intent checks, and execution boundaries matter at every layer.

    ]]>    Injection @@ -333,7 +333,7 @@
  • Static scans before execution. Run security scanners and dependency checks on generated code before it is executed.
  • Runtime monitoring. Observe file, network, and process activity during code execution and terminate anomalous behaviour.
    • -]]>    Theme 3: Supply Chain & Integrity looks at trust in code, data, and context. In classic software, the supply chain is mostly static: libraries, packages, build tools, and the CI/CD (Continuous Integration / Continuous Deployment) pipeline that produces your artefact. If any of those ingredients are tampered with, the final application carries the flaw. The OWASP Top 10:2025 tracks this as Software Supply Chain Failures (A03:2025) and Software or Data Integrity Failures (A08:2025).

    +]]>    Theme 3: Supply Chain & Integrity looks at trust in code, data, and context. In classic software, the supply chain is mostly static: libraries, packages, build tools, and the CI/CD (Continuous Integration / Continuous Deployment) pipeline that produces your artefact. If any of those ingredients are tampered with, the final application carries the flaw. The OWASP Top 10:2025 tracks this as Software Supply Chain Failures (A03:2025) and Software or Data Integrity Failures (A08:2025).

    Agentic systems extend that same idea into a dynamic, runtime supply chain. An agent does not just import a dependency at build time; it can discover and compose tools, models, prompts, and other agents at runtime. This composition happens through protocols such as the Model Context Protocol (MCP) and Agent-to-Agent (A2A) frameworks, and through registries that advertise tool or agent capabilities. Each extra capability is a dependency that can be malicious, misconfigured, or compromised.

    @@ -496,7 +496,7 @@
  • Decay / expire unverified memory. Reduce the trust weight of unverified entries over time and eventually remove them.
  • Trust-weighted retrieval. Rank search results by source trust and confidence, not just embedding similarity.
    • -]]>    Theme 4: Cryptographic Failures & Secrets +]]>Theme 4: Cryptographic Failures & Secrets

    Theme 4 explores Cryptographic Failures (A04:2025) from the OWASP Top 10:2025, plus what those failures mean for agents and Non-Human Identities (NHI).

    @@ -539,7 +539,7 @@
  • Private keys should never be directly accessible to agents. Use an orchestrator or trusted signer to perform operations on behalf of the agent.
  • Treat Non-Human Identity (NHI) credentials as a full lifecycle asset: issue with least privilege, monitor use, rotate frequently, and revoke immediately when an agent is decommissioned or suspected of compromise.
    • -]]>    Theme 5: Configuration, Design & Resilience +]]>Theme 5: Configuration, Design & Resilience

    Theme 5 brings together three overlapping risks from the OWASP Top 10:2025 — Security Misconfiguration (A02:2025), Insecure Design (A06:2025), and Mishandling of Exceptional Conditions (A10:2025) — and adds the agentic risk of Cascading Failures (ASI08).

    @@ -653,7 +653,7 @@
  • Use digital-twin replay and policy gating to test risky decisions before live execution.
  • Keep tamper-evident logging and non-repudiation so every propagated action can be traced and audited.
    • -]]>    Theme 6 covers the risks that are unique to agentic systems, where software does not just respond to a single user but can act on its own, talk to other agents, and influence the people around it.

    +]]>    Theme 6 covers the risks that are unique to agentic systems, where software does not just respond to a single user but can act on its own, talk to other agents, and influence the people around it.

    Where earlier themes mapped classic application risks to the agent world, this theme is different: it focuses on three risks that only appear when multiple agents, or a human and an agent, share responsibility.

    @@ -755,7 +755,7 @@
  • Require periodic behavioural attestation and per-run ephemeral credentials, so compromise is time-bound.
  • Tie recovery or reintegration to fresh attestation and explicit human approval.
    • -]]>    Theme 7 brings the focus onto observability, logging and detection. These controls are cross-cutting: they support almost every other theme in this course.

    +]]>    Theme 7 brings the focus onto observability, logging and detection. These controls are cross-cutting: they support almost every other theme in this course.

    For traditional applications, good logging and alerting helps you detect breaches, investigate incidents and prove compliance. For agentic systems, the need is even stronger. Agents make sequences of decisions, call tools, talk to other agents and influence humans. Without a clear, tamper-evident record of what happened, you cannot attribute actions, detect drift or recover from failure.

    @@ -789,7 +789,7 @@

    For agents, the same principles apply, but the events are different. You should log tool calls, inter-agent messages, goal state and action sequences against a baseline. Track a stable goal identifier across the lifetime of a task so that every action can be traced back to the original request. Alert on goal drift and anomalous tool chains, and ensure non-repudiation and lineage for every propagated action.

    Strong observability turns an opaque agent into an accountable one: it is not enough to know what the agent said; you must be able to reconstruct what it did.

    -]]>    Putting it together: secure SDLC and threat modelling +]]>Putting it together: secure SDLC and threat modelling

    Secure code development is not a single step; it is threaded through the whole SDLC (Software Development Life Cycle). The two OWASP lists in this course — the OWASP Top 10:2025 for classic applications and the OWASP Top 10 for Agentic Applications 2026 — give you concrete risks to address at each phase.

    @@ -990,7 +990,7 @@

    License of this course

    This course is licensed under CC BY-SA 4.0 (Creative Commons Attribution-ShareAlike 4.0). It is an adapted summary/derivative; changes were made; it is not endorsed by OWASP. Generated with the assistance of an AI coding agent.

    -]]>    Course complete +]]>Course complete
    diff --git a/source/preview.xml b/source/preview.xml index 8c87b74..4a4f438 100644 --- a/source/preview.xml +++ b/source/preview.xml @@ -1,5 +1,5 @@ -Welcome to Secure code development +Welcome to Secure code development

    This course is for engineers building software with agents and for engineers building agents. Whether you are designing agentic systems, integrating LLMs, or writing the services that agents call, secure code development matters.

    @@ -211,7 +211,7 @@
  • Detect delegated and transitive permissions — model the full delegation chain. If agent A delegates to agent B, the effective permissions should be the intersection of what A and B are each allowed to do.
  • Managed NHI (Non-Human Identity) — place agent identities in a dedicated identity platform such as Entra, AWS Bedrock, or Agentforce. Use lifecycle management, rotation, and audit for non-human accounts just as you would for human employees.
    • -]]>    This theme traces how untrusted input becomes unintended action. We start with classic injection, then follow the chain into agentic territory: agents whose goals can be hijacked, whose tools can be misused, and whose generated code can execute unexpectedly.

    +]]>    This theme traces how untrusted input becomes unintended action. We start with classic injection, then follow the chain into agentic territory: agents whose goals can be hijacked, whose tools can be misused, and whose generated code can execute unexpectedly.

    For engineers building agents, the lesson is that input validation, intent checks, and execution boundaries matter at every layer.

    ]]>    Injection @@ -333,7 +333,7 @@
  • Static scans before execution. Run security scanners and dependency checks on generated code before it is executed.
  • Runtime monitoring. Observe file, network, and process activity during code execution and terminate anomalous behaviour.
    • -]]>    Theme 3: Supply Chain & Integrity looks at trust in code, data, and context. In classic software, the supply chain is mostly static: libraries, packages, build tools, and the CI/CD (Continuous Integration / Continuous Deployment) pipeline that produces your artefact. If any of those ingredients are tampered with, the final application carries the flaw. The OWASP Top 10:2025 tracks this as Software Supply Chain Failures (A03:2025) and Software or Data Integrity Failures (A08:2025).

    +]]>    Theme 3: Supply Chain & Integrity looks at trust in code, data, and context. In classic software, the supply chain is mostly static: libraries, packages, build tools, and the CI/CD (Continuous Integration / Continuous Deployment) pipeline that produces your artefact. If any of those ingredients are tampered with, the final application carries the flaw. The OWASP Top 10:2025 tracks this as Software Supply Chain Failures (A03:2025) and Software or Data Integrity Failures (A08:2025).

    Agentic systems extend that same idea into a dynamic, runtime supply chain. An agent does not just import a dependency at build time; it can discover and compose tools, models, prompts, and other agents at runtime. This composition happens through protocols such as the Model Context Protocol (MCP) and Agent-to-Agent (A2A) frameworks, and through registries that advertise tool or agent capabilities. Each extra capability is a dependency that can be malicious, misconfigured, or compromised.

    @@ -496,7 +496,7 @@
  • Decay / expire unverified memory. Reduce the trust weight of unverified entries over time and eventually remove them.
  • Trust-weighted retrieval. Rank search results by source trust and confidence, not just embedding similarity.
    • -]]>    Theme 4: Cryptographic Failures & Secrets +]]>Theme 4: Cryptographic Failures & Secrets

    Theme 4 explores Cryptographic Failures (A04:2025) from the OWASP Top 10:2025, plus what those failures mean for agents and Non-Human Identities (NHI).

    @@ -539,7 +539,7 @@
  • Private keys should never be directly accessible to agents. Use an orchestrator or trusted signer to perform operations on behalf of the agent.
  • Treat Non-Human Identity (NHI) credentials as a full lifecycle asset: issue with least privilege, monitor use, rotate frequently, and revoke immediately when an agent is decommissioned or suspected of compromise.
    • -]]>    Theme 5: Configuration, Design & Resilience +]]>Theme 5: Configuration, Design & Resilience

    Theme 5 brings together three overlapping risks from the OWASP Top 10:2025 — Security Misconfiguration (A02:2025), Insecure Design (A06:2025), and Mishandling of Exceptional Conditions (A10:2025) — and adds the agentic risk of Cascading Failures (ASI08).

    @@ -653,7 +653,7 @@
  • Use digital-twin replay and policy gating to test risky decisions before live execution.
  • Keep tamper-evident logging and non-repudiation so every propagated action can be traced and audited.
    • -]]>    Theme 6 covers the risks that are unique to agentic systems, where software does not just respond to a single user but can act on its own, talk to other agents, and influence the people around it.

    +]]>    Theme 6 covers the risks that are unique to agentic systems, where software does not just respond to a single user but can act on its own, talk to other agents, and influence the people around it.

    Where earlier themes mapped classic application risks to the agent world, this theme is different: it focuses on three risks that only appear when multiple agents, or a human and an agent, share responsibility.

    @@ -755,7 +755,7 @@
  • Require periodic behavioural attestation and per-run ephemeral credentials, so compromise is time-bound.
  • Tie recovery or reintegration to fresh attestation and explicit human approval.
    • -]]>    Theme 7 brings the focus onto observability, logging and detection. These controls are cross-cutting: they support almost every other theme in this course.

    +]]>    Theme 7 brings the focus onto observability, logging and detection. These controls are cross-cutting: they support almost every other theme in this course.

    For traditional applications, good logging and alerting helps you detect breaches, investigate incidents and prove compliance. For agentic systems, the need is even stronger. Agents make sequences of decisions, call tools, talk to other agents and influence humans. Without a clear, tamper-evident record of what happened, you cannot attribute actions, detect drift or recover from failure.

    @@ -789,7 +789,7 @@

    For agents, the same principles apply, but the events are different. You should log tool calls, inter-agent messages, goal state and action sequences against a baseline. Track a stable goal identifier across the lifetime of a task so that every action can be traced back to the original request. Alert on goal drift and anomalous tool chains, and ensure non-repudiation and lineage for every propagated action.

    Strong observability turns an opaque agent into an accountable one: it is not enough to know what the agent said; you must be able to reconstruct what it did.

    -]]>    Putting it together: secure SDLC and threat modelling +]]>Putting it together: secure SDLC and threat modelling

    Secure code development is not a single step; it is threaded through the whole SDLC (Software Development Life Cycle). The two OWASP lists in this course — the OWASP Top 10:2025 for classic applications and the OWASP Top 10 for Agentic Applications 2026 — give you concrete risks to address at each phase.

    @@ -990,7 +990,7 @@

    License of this course

    This course is licensed under CC BY-SA 4.0 (Creative Commons Attribution-ShareAlike 4.0). It is an adapted summary/derivative; changes were made; it is not endorsed by OWASP. Generated with the assistance of an AI coding agent.

    -]]>    Course complete +]]>Course complete
    diff --git a/source/scorm_progress.js b/source/scorm_progress.js index 952a4b5..d09ed18 100644 --- a/source/scorm_progress.js +++ b/source/scorm_progress.js @@ -74,6 +74,19 @@ * page) with the required-page count, the pages still to do, each quiz * result, the weighted score and the exact status the LMS holds. * + * 6. shows the hint and explanation authored on a question only to a learner + * who got that question wrong. The text lives in the question's own + * feedback attribute (the quiz wizard's "General Feedback" field), which + * models_html5/quiz.html renders into a feedback slot on every submitted + * answer, right or wrong. This script clears that slot again when the + * answer was correct, so the learner who needs the help sees it and the + * learner who does not is left with the plain "correct" message. If the + * engine renames that call the wrapper does nothing and the feedback + * simply shows on correct answers too, which loses nothing; a change to + * the slot order would be worse, because the wrapper would clear whatever + * else landed there, so helpSlot follows the engine's own rule for + * feedbackPos rather than assuming the first slot. + * * CONSTRAINT: the compiled one-liner is stored raw in an XML attribute, so it * must not contain < > & or " characters (tools/sync_root_script.py enforces * this). Use === / !== / indexOf() instead of ordering comparisons, nested ifs @@ -649,11 +662,96 @@ var orig = window.x_pageLoaded; window.x_pageLoaded = function () { var result = orig.apply(this, arguments); + try { wrapQuizFeedback(); } catch (e) { } refreshUi(); return result; }; } + /* ---- wrong-answer help: the hint and explanation on a question ---- */ + + var FEEDBACK_SLOTS = ['topFeedback', 'middleFeedback', 'bottomFeedback']; + + /* quiz.currentQ indexes quiz.questions, which holds the position of each + question within the quiz page, in the order this attempt asks them. */ + function questionNode() { + if (typeof quiz === 'undefined') { return null; } + if (!quiz) { return null; } + if (!isArray(quiz.questions)) { return null; } + var pos = quiz.questions[quiz.currentQ]; + if (typeof pos !== 'number') { return null; } + if (typeof jQuery !== 'function') { return null; } + if (typeof x_currentPageXML === 'undefined') { return null; } + if (!x_currentPageXML) { return null; } + var node = jQuery(x_currentPageXML).children()[pos]; + if (!node) { return null; } + return node; + } + + /* The model fills three fixed slots in the order the question's + feedbackPos gives, one letter per slot: G is the question's own + feedback attribute, A the selected option's, C the right/wrong line. + No question here sets feedbackPos, so G lands in the first slot. */ + function helpSlot(node) { + var order = 'GAC'; + var attr = node.getAttribute('feedbackPos'); + /* Only an absent attribute means GAC. The engine spreads the value it + finds, so an empty one leaves it no slot letters at all and puts the + right/wrong line in all three: G is then nowhere, and clearing the + first slot would delete the verdict instead. */ + if (typeof attr === 'string') { order = attr; } + var pos = order.indexOf('G'); + var id = FEEDBACK_SLOTS[pos]; + if (typeof id !== 'string') { return null; } + return document.getElementById(id); + } + + /* myProgress holds each question's graded result, and currentQ still + names the question just submitted until Next is pressed. */ + function answerWasRight() { + if (typeof quiz === 'undefined') { return false; } + if (!quiz) { return false; } + if (!isArray(quiz.myProgress)) { return false; } + return quiz.myProgress[quiz.currentQ] === true; + } + + /* Where the panel is centred vertically the model slides the feedback in, + and jQuery measures the height it is animating to while the text is + still there. Emptying the element after that would animate a blank gap + open and snap it shut, so finish the animation and hide it instead. */ + function clearSlot(slot) { + if (typeof jQuery === 'function') { + jQuery(slot).stop(true, true).empty().hide(); + return; + } + slot.textContent = ''; + } + + function hideHelpWhenRight() { + if (!answerWasRight()) { return; } + var node = questionNode(); + if (!node) { return; } + var slot = helpSlot(node); + if (!slot) { return; } + clearSlot(slot); + } + + /* The model object is built afresh for every quiz page, so this runs from + x_pageLoaded rather than once at startup. */ + function wrapQuizFeedback() { + if (typeof quiz === 'undefined') { return; } + if (!quiz) { return; } + if (quiz.__xtHelp) { return; } + if (typeof quiz.showFeedBackandTrackResults !== 'function') { return; } + quiz.__xtHelp = true; + var orig = quiz.showFeedBackandTrackResults; + quiz.showFeedBackandTrackResults = function () { + var result = orig.apply(this, arguments); + try { hideHelpWhenRight(); } catch (e) { } + return result; + }; + } + /* the quiz results screen is the only caller of XTSetPageScore */ function wrapPageScore() { if (typeof window.XTSetPageScore !== 'function') { return; } diff --git a/tests/test_scorm_tracking.py b/tests/test_scorm_tracking.py index 56d0e55..b3870c6 100644 --- a/tests/test_scorm_tracking.py +++ b/tests/test_scorm_tracking.py @@ -44,6 +44,7 @@ NR_PAGES = 45 # 44 content pages + the table of contents THEME1_QUIZ = 9 QUIZ_PAGES = [9, 15, 21, 24, 30, 35, 38, 43] +QUESTIONS = 45 # nodes across the 8 quizzes SUSPEND_LIMIT = 4096 # SCORM 1.2 cmi.suspend_data # A minimal but honest SCORM 1.2 API. Records everything so tests can assert on @@ -131,6 +132,33 @@ } """ +# Options are shuffled per attempt, so a wrong answer has to be chosen by its +# correct flag rather than by position. +SELECT_A_WRONG_OPTION = """ +() => { + const inputs = window.jQuery('#optionHolder input'); + let picked = 0; + inputs.each(function () { + if (picked) { return; } + const correct = window.jQuery(this).data('correct'); + if (correct !== 'true' && correct !== true) { + window.jQuery(this).prop('checked', true).trigger('change'); + picked++; + } + }); + return {options: inputs.length, picked: picked}; +} +""" + +# What the learner is left looking at once a question has been marked. +FEEDBACK_SHOWN = """ +() => ({ + help: window.jQuery('#topFeedback').text().trim(), + verdict: window.jQuery('#bottomFeedback').text().trim(), + marking: window.jQuery('#feedbackGroup').attr('class') || '' +}) +""" + # The SCO always runs inside a frame in an LMS; reproduce that. LMS_SHELL = """ @@ -282,6 +310,21 @@ def answer_quiz(self, correctly: bool = True) -> int: ) return questions + def answer_question(self, correctly: bool) -> dict: + """Answer the question now on screen and return the feedback shown.""" + self.sco.wait_for_selector("#optionHolder input", timeout=15000) + picked = self.sco.evaluate( + SELECT_CORRECT_OPTIONS if correctly else SELECT_A_WRONG_OPTION + ) + assert picked["picked"], f"no option to pick: {picked}" + self.sco.evaluate("() => window.jQuery('#checkBtn').trigger('click')") + self.page.wait_for_timeout(300) + return self.sco.evaluate(FEEDBACK_SHOWN) + + def next_question(self) -> None: + self.sco.evaluate("() => window.jQuery('#nextBtn').trigger('click')") + self.page.wait_for_timeout(200) + def terminate(self) -> None: self.sco.evaluate("() => XTTerminate()") self.page.wait_for_timeout(200) @@ -365,6 +408,27 @@ def lms_status_shown(s: Session, page_nr: int) -> str: return s.sco.get_attribute(f"#x_page{page_nr} .xtLmsStatus", "data-status") +UNTICKED_PAGES = """ +() => { + const out = []; + const offset = XENITH.PAGEMENU.menuPage ? 1 : 0; + window.jQuery('#x_page0 .menuItem').each(function () { + if (window.jQuery(this).find('i.notvisited').length) { + out.push(x_normalPages[window.jQuery(this).data('pageIndex') + offset]); + } + }); + return out; +} +""" + +VIEWED_PAGES = """ +() => { + const out = []; + x_pageInfo.forEach(function (p, i) { if (p.viewed === true) { out.push(i); } }); + return out; +} +""" + PENDING_QUIZ_TICKS = """ () => { const out = []; @@ -384,6 +448,16 @@ def pending_quiz_ticks(s: Session) -> list[int]: return s.sco.evaluate(PENDING_QUIZ_TICKS) +def unticked_pages(s: Session) -> list[int]: + """Page indexes the contents page still shows as never visited.""" + return s.sco.evaluate(UNTICKED_PAGES) + + +def viewed_pages(s: Session) -> list[int]: + """Page indexes the engine currently considers viewed.""" + return s.sco.evaluate(VIEWED_PAGES) + + def assert_resumable(record: dict, where: str) -> None: interactions = record.get("interactions") assert isinstance(interactions, list), f"{where}: interactions is not a list" @@ -952,6 +1026,66 @@ def test_status_panel_tells_the_learner_what_is_left(h: Harness): assert not s.page_errors, f"page errors: {s.page_errors}" +@test +def test_wrong_answer_gets_a_hint_and_an_explanation(h: Harness): + """Getting a question wrong tells the learner why, and what to revisit. + + The help is authored on the question itself, which the quiz model renders + on every submitted answer. Only a learner who got it wrong needs it, so + the block is cleared again when the answer was right. + """ + with h.session() as s: + s.open() + s.goto(THEME1_QUIZ) + before = s.sco.evaluate(FEEDBACK_SHOWN) + assert before["help"] == "", f"help shown before answering: {before}" + wrong = s.answer_question(correctly=False) + assert "incorrectFeedback" in wrong["marking"], wrong + assert wrong["help"].startswith("Hint:"), f"no hint on a wrong answer: {wrong}" + assert "Why:" in wrong["help"], f"no explanation on a wrong answer: {wrong}" + s.next_question() + right = s.answer_question(correctly=True) + # "correctFeedback" is a substring of "incorrectFeedback", so match exactly + assert right["marking"].split() == ["correctFeedback"], right + assert right["help"] == "", f"a correct answer should not be given help: {right}" + # clearing the help must not take the correct/incorrect line with it + assert right["verdict"], f"the verdict was cleared too: {right}" + # a second quiz page rebuilds the model object, which has to be re-wrapped + s.goto(QUIZ_PAGES[1]) + again = s.answer_question(correctly=False) + assert again["help"].startswith("Hint:"), f"not re-wrapped on a later quiz: {again}" + s.next_question() + still = s.answer_question(correctly=True) + assert still["help"] == "", f"help not cleared on a later quiz: {still}" + assert not s.page_errors, f"page errors: {s.page_errors}" + + +@test +def test_every_question_carries_wrong_answer_help(h: Harness): + """No question is left without a hint and an explanation. + + The text lives in the course XML, so a question added later would silently + have none: walk the parsed project and check all 45 of them. + """ + with h.session() as s: + s.open() + walk = s.sco.evaluate( + "() => { const out = []; let checked = 0;" + " window.jQuery(x_pages).each(function () {" + " if (this.nodeName !== 'quiz') { return; }" + " window.jQuery(this).children('question').each(function () {" + " checked++;" + " const fb = this.getAttribute('feedback') || '';" + " if (fb.indexOf('Hint:') === -1 || fb.indexOf('Why:') === -1) {" + " out.push(this.getAttribute('name')); } }); });" + " return {checked: checked, missing: out}; }" + ) + # without this the walk finding nothing at all would pass silently + assert walk["checked"] == QUESTIONS, f"walked {walk['checked']} questions, expected {QUESTIONS}" + assert walk["missing"] == [], f"questions without a hint and explanation: {walk['missing']}" + assert not s.page_errors, f"page errors: {s.page_errors}" + + @test def test_quiz_tick_means_submitted_not_opened(h: Harness): """A quiz that has been opened but not submitted gets a distinct marker. @@ -1003,6 +1137,81 @@ def test_older_records_infer_submission_from_the_score(h: Harness): assert "Theme 1 Quiz: 75%" in panel_text(s, MENU_PAGE), panel_text(s, MENU_PAGE) +@test +def test_viewed_pages_survive_a_save_and_resume(h: Harness): + """Ticks and the progress bar must still mean something after a resume. + + Both read `x_pageInfo[i].viewed`, which is session state: it survives only + through the engine's `pagesViewed` field in the saved record + (`x_pagesViewed()` -> `state.pagesViewed` -> `cmi.suspend_data` -> + `setVars` -> `x_restorePagesViewed()`). A learner who saves, leaves and + comes back to an empty progress bar and no ticks has lost that round trip. + The two halves are asserted separately so a failure names which one broke. + """ + seen = [FIRST_CONTENT_PAGE, FIRST_CONTENT_PAGE + 1, FIRST_CONTENT_PAGE + 2] + with h.session() as s: + s.open() + for page_nr in seen: + s.goto(page_nr) + s.terminate() + saved = s.suspends()[-1] + record = json.loads(saved) + # save side + assert "pagesViewed" in record, ( + f"the saved record has no pagesViewed, so nothing can be restored: {sorted(record)}" + ) + missing = [p for p in seen if p not in record["pagesViewed"]] + assert not missing, f"visited pages missing from pagesViewed: {missing}" + assert not s.page_errors, f"page errors: {s.page_errors}" + # restore side + with h.resumed(saved) as s: + s.open() + restored = viewed_pages(s) + lost = [p for p in seen if p not in restored] + assert not lost, f"pages lost their viewed flag on resume: {lost} (restored {restored})" + s.goto(MENU_PAGE) + label = s.sco.inner_text("#x_headerProgress .pbTxt") + assert not label.startswith("0%"), f"progress bar empty after a resume: {label}" + still_unticked = [p for p in seen if p in unticked_pages(s)] + assert not still_unticked, f"contents-page ticks lost on resume: {still_unticked}" + assert not s.page_errors, f"page errors: {s.page_errors}" + + +@test +def test_viewed_pages_survive_a_resume_late_in_the_course(h: Harness): + """The same round trip for a learner who has actually done the work. + + `pagesViewed` is the second thing `fitBudget` sheds when the record will + not fit `cmi.suspend_data`, ahead of the quiz scores, so the ticks and the + progress bar are the first learner-visible casualty of a record that has + grown. A learner saves after a real session, not after three pages. + """ + walked = list(range(FIRST_CONTENT_PAGE, 36)) + with h.session() as s: + s.open() + for page_nr in walked: + s.goto(page_nr, dwell=120) + if page_nr in QUIZ_PAGES: + s.answer_quiz() + s.terminate() + saved = s.suspends()[-1] + record = json.loads(saved) + assert len(saved) <= SUSPEND_LIMIT, f"record is {len(saved)} characters" + assert "pagesViewed" in record, ( + f"pagesViewed was shed from a {len(saved)} character record: kept {sorted(record)}" + ) + assert not s.page_errors, f"page errors: {s.page_errors}" + with h.resumed(saved) as s: + s.open() + restored = viewed_pages(s) + lost = [p for p in walked if p not in restored] + assert not lost, f"pages lost their viewed flag on resume: {lost}" + s.goto(MENU_PAGE) + label = s.sco.inner_text("#x_headerProgress .pbTxt") + assert not label.startswith("0%"), f"progress bar empty after a resume: {label}" + assert not s.page_errors, f"page errors: {s.page_errors}" + + @test def test_header_progress_bar_counts_viewed_pages(h: Harness): """The header progress bar is on, labelled honestly, with a marker per quiz.""" diff --git a/tools/render_preview.py b/tools/render_preview.py index a882b68..ad08370 100644 --- a/tools/render_preview.py +++ b/tools/render_preview.py @@ -44,8 +44,13 @@ def render_question(q): out = [f'
    ', f'
    Q: {html.escape(prompt)}
    ', f'
    {html.escape(qtype)} ({len(opts)} options)
    ', - render_options(opts), - f'
    '] + render_options(opts)] + # shown to a learner who gets this question wrong + help_html = unesc(q.get("feedback", "")).strip() + if help_html: + out.append(f'
    shown on a wrong answer
    ' + f'{help_html}
    ') + out.append('') return "\n".join(out) def render_page(node, idx): @@ -117,6 +122,8 @@ def main(): .opt.correct {{ font-weight:bold; }} .opt .lbl {{ color:var(--muted); }} .opt .mark {{ color:var(--accent); font-size:.8rem; }} +.q-help {{ margin-top:8px; padding:6px 12px; border-left:3px solid #c9a227; background:#fffdf3; }} +.q-help p {{ margin:.3em 0; }} footer {{ color:var(--muted); font-size:.8rem; text-align:center; padding:20px; }}

    {html.escape(title)}