A capstone that inverts the whole course: take the findings you produced as an attacker, and build — then test — the detections that would have caught you, measuring honestly which attacks leave evidence and which leave none at all.
[!warning] Authorized use only The attack replays in this project are real attacks. Run them only against the disposable targets you built for the labs, on an isolated host, bound to
127.0.0.1. "I was only generating training data" is not an authorisation, and a detection-engineering exercise against somebody else's production logging is still an attack against their production system.
Type: Capstone · Targets: the lab targets from this course, plus a log pipeline you assemble · Difficulty: Advanced · Time: ~12–14 h · OWASP: A09:2021
Every vulnerability note in this course carries a ## Detection section, and the course makes an unusual claim in them: it says, repeatedly and specifically, what a defender cannot see. A DOM-based XSS payload in a URL fragment never reaches the server. An XS-Leak read is a property access inside the attacker's own page. A cache-deception exfiltration is a cache HIT the origin never hears about. A successful mass assignment returns 200 OK with a perfectly ordinary body.
Those statements are only worth anything if somebody has checked them. This capstone is that check. You will replay your own attacks against instrumented targets, measure what actually appears in the logs, write detections for the attacks that leave evidence, and — just as importantly — produce a defensible list of the attacks that leave none, so that nobody spends a quarter building a rule that cannot work.
The output is not a rule pack. It is a coverage statement: for each attack technique, what the evidence is, what the rule is, what its false-positive rate was on benign traffic, and what the residual blind spot is.
Three reasons, in increasing order of importance.
- It makes your reports better. A finding whose
Detectionsection says "your WAF will not see this, here is what would" is worth several that say "high severity". - It calibrates severity. A vulnerability that is undetectable is materially worse than an identical one that trips an alert on the first attempt, and almost no severity model captures that.
- It is the only honest way to know whether the
Detectionsections are true. Writing "a defender would see X" without having looked is exactly the kind of unverified claim this course refuses to make elsewhere.
| Source | Sees | Systematically blind to |
|---|---|---|
| Reverse proxy / web server access log | Method, path, query string, status, size, timing, and whatever headers you configured it to log | Request bodies, URL fragments, anything served from a cache in front of it |
| Application log and audit trail | Business events, authentication, authorisation decisions, data writes — if instrumented | Everything nobody thought to instrument |
| Database / data-layer audit | The actual state change, whichever code path caused it | Intent, and reads that were never authorised in the first place |
| Client-side telemetry (RUM, CSP reports, error tracking) | Script errors, CSP violations, some browser-side anomalies | Anything the attacker's page does in its own context |
Two rules follow, and both are counter-intuitive enough to be worth stating explicitly:
- The best signal is usually a ratio or a sequence, not a string. Payload matching fails against encoding; a 99% error rate on one endpoint does not.
- The most valuable detection is often not at the point of attack. Mass assignment is invisible in the request and unmistakable in the data-layer audit.
Reuse the targets you already built. That is the point — you have working attacks against them and you know exactly what "success" looks like.
- The padding-oracle target from [[Practical-Labs/Lab-Padding-Oracle-Decryption|Lab: Padding Oracle Decryption]], which produced a real, quantified log signature on this host.
- The cache-deception chain from [[Practical-Labs/Lab-Cache-Deception|Lab: Cache Deception]], which has a two-layer log story.
- The mass-assignment API from [[Practical-Labs/Lab-Mass-Assignment-Privilege-Escalation|Lab: Mass Assignment Privilege Escalation]], the "silent success" case.
- DVWA for the classic injection classes.
- The XS-Leaks and WebSocket targets, for the two cases where the honest answer is "almost nothing".
You need somewhere to put the logs and something to query them with. Keep it simple; the engineering is in the rules, not the pipeline.
mkdir -p detect/{raw,rules,evidence}
# Minimal, dependency-free: structured JSON lines you can query with python3 or jq.
# Point each target's log at detect/raw/<target>.log, then normalise:
python3 - <<'PY'
import re, json, sys
LINE = re.compile(r'(?P<ip>\S+) \S+ \S+ \[(?P<ts>[^\]]+)\] "(?P<method>\S+) (?P<path>\S+) [^"]*" (?P<status>\d+)')
for line in open('detect/raw/padding.log'):
m = LINE.search(line)
if m: print(json.dumps(m.groupdict()))
PYIf you have Elasticsearch, Loki, Splunk or a SIEM available, use it — but write every rule so that it is also expressible as a query over the JSON lines above, because a rule you can only express in one product is a rule you cannot review.
| Item | Value for this capstone |
|---|---|
| In scope | The loopback lab targets and the log pipeline you build around them. |
| Out of scope | Any production logging system. Any log containing real user data. |
| Permitted | Replaying your own attacks, generating synthetic benign traffic, python3, jq, curl. |
| Prohibited | Testing rules against real traffic captures. Copying a client's logs to your own machine. |
| Data discipline | Generate the benign baseline yourself. A rule with no measured false-positive rate is not a finished rule. |
| Time box | 12–14 hours. |
| Reset point | Truncate and regenerate logs between techniques so that one attack's noise does not contaminate another's measurement. |
Turn on the logging you wish a client had, and note what it costs. At minimum: method, path, full query string, status, byte count, duration, User-Agent, Referer, Origin, Sec-Fetch-Site, Sec-Fetch-Dest, and a session or user identifier. Origin and the Fetch Metadata headers are absent from almost every default log format and are the two highest-value additions you can make — they are what makes the WebSocket and XS-Leak rules possible at all.
Then generate benign traffic: browse the application normally, log in and out, use every feature, produce genuine errors by mistyping things. Aim for several thousand requests. This baseline is what every false-positive number in your report is measured against, and without it your rules are guesses.
for i in $(seq 1 500); do
curl -s -o /dev/null --cookie "session=$SESSION" \
"http://127.0.0.1:5000/profile"
doneFor each technique, run the attack you already know works, then answer four questions in writing:
- What appeared in the access log?
- What appeared in the application log?
- What appeared in the data layer?
- What did not appear anywhere?
The padding oracle gives the clearest possible example, and the numbers below were captured on this host during that lab's run:
total requests to /profile: 12864
500s: 12758
200s: 21
400s: 85
Individual lines look entirely ordinary:
127.0.0.1 - - [20/Aug/2026 20:54:40] "GET /profile HTTP/1.1" 500 -
127.0.0.1 - - [20/Aug/2026 20:54:40] "GET /profile HTTP/1.1" 500 -
127.0.0.1 - - [20/Aug/2026 20:54:40] "GET /profile HTTP/1.1" 500 -
127.0.0.1 - - [20/Aug/2026 20:54:40] "GET /profile HTTP/1.1" 200 -
There is no payload to match — the "payload" is a cookie value that changes every request and means nothing on its own. The signal is entirely in the aggregate: a 99.2% error rate on one endpoint. And note the trap that this measurement exposes: the 21 successful requests include the ones that actually broke the cryptography. A rule that alerts only on anomalous successes catches 0.16% of the attack and misses the break.
Do the same for each technique and build the table. Expect the results to be uncomfortable in places — the XS-Leak replay will show you an access log that is indistinguishable from a user clicking a link.
Write each rule in three parts: the signal, the query, and the reason it is not a payload match. Six worked examples, one per family.
A. Padding oracle / crypto probing — a ratio rule.
# Alert when one endpoint's error ratio for one client exceeds a threshold
# over a short window. Benign clients produce ~0 padding failures.
from collections import Counter
err, tot = Counter(), Counter()
for e in events(window="5m"):
key = (e["ip"], e["path"])
tot[key] += 1
if e["status"] >= 500: err[key] += 1
for key in tot:
if tot[key] >= 50 and err[key] / tot[key] > 0.5:
alert("crypto oracle probing", key, err[key], tot[key])B. Cache deception — a sequence rule. The strongest rule in this whole project, because it describes the attack rather than the payload:
ALERT WHEN a URL is first requested WITH credentials
AND subsequently served FROM CACHE to a request WITHOUT credentials
within the cache TTL.
Cheaper approximations, useful if the edge cannot express the above: alert on any cache key whose path is not in the deployed static-asset manifest, and alert on application routes requested with a static extension.
C. Mass assignment — a data-layer rule. There is nothing in the request worth matching, so do not try:
ALERT WHEN a privileged column (role, credit, is_verified, tenant_id)
is written BY a self-service endpoint
WHERE the actor is the record owner
AND no corresponding admin action exists.
Better still, make the application reject unknown fields with a 400 and log them — as the lab's remediation section does — which converts a silent probe into an explicit event. That is detection engineering feeding back into design, and it is the most valuable move available in this domain.
D. Cross-site WebSocket hijacking — a header rule. Trivial once Origin is logged:
ALERT WHEN a WebSocket upgrade authenticates successfully
AND Origin is not in the application's own origin set,
OR Origin is absent.
E. Cross-site navigation to a sensitive route — a Fetch Metadata rule.
ALERT WHEN Sec-Fetch-Site: cross-site
AND Sec-Fetch-Dest: document
AND path is in the sensitive-route set.
Note this is simultaneously a detection and a prevention: the same condition can return 403. Say so in the report, because a control that also alerts is worth two that only do one.
F. Server-side prototype pollution — a key rule with a stated weakness.
ALERT WHEN a request body or query string contains a key named
__proto__, constructor or prototype.
Write the weakness down next to it: this is a literal match, and literal matches lose to encoding, to nested bracket notation from a query-string parser, and to constructor.prototype chains. It is a tripwire, not a control. A rule shipped without its own limitations documented is how organisations end up believing they are covered.
An untested rule is an opinion. For each rule, measure three numbers against the Pass 1 baseline plus your attack replays:
- True positives — did it fire on the attack? On which request number? A rule that fires on request 12,000 of 12,864 technically works and practically does not.
- False positives — how many benign requests matched? Express it as a rate per thousand, because a 0.1% false-positive rate on ten million requests a day is ten thousand alerts.
- Time to detect — how far into the attack did the first alert fire? For the padding oracle, a threshold of 50 requests detects at roughly 0.4% of the attack; a threshold of 5,000 detects after the key material is already gone.
Then tune, and record what you tuned and why. The tuning log is a deliverable: it is the evidence that the threshold was chosen rather than guessed.
This is the pass that distinguishes this capstone from a rules-writing exercise, and it is the part a client will value most.
For each technique, state plainly what cannot be detected and why, in terms of where the evidence physically is:
| Technique | Blind spot | Physical reason |
|---|---|---|
| DOM XSS via URL fragment | The payload, entirely | The fragment is never sent to the server |
| Client-side prototype pollution | The payload and the effect | Both occur in the browser's JavaScript realm |
| XS-Leaks | The extraction | It is a property read inside the attacker's own page |
| Web cache deception (the read) | The exfiltration request | Served from cache; the origin is never contacted |
| Mass assignment | The request | A well-formed body with one extra valid key |
| Business-logic abuse | The individual requests | Every request is legitimate; only the sequence is not |
For each blind spot, propose the compensating control — because "you cannot detect this" is only half an answer. CSP violation reporting and client-side error telemetry for the browser-side rows; edge-side correlation for the cache row; data-layer auditing for mass assignment; invariant monitoring for logic abuse. That mapping from undetectable to preventable is the report's most useful page.
Deliver rules that somebody can actually operate: each with a name, the query, the threshold and its justification, the measured false-positive rate, a triage procedure, and an explicit statement of what it does not cover.
Include the replay harness — the scripts that generate both the attack and the benign traffic — so the client can re-verify every rule after any change to the application. A detection that is never re-tested silently stops working the day a route is renamed.
- Instrumentation gap analysis — what the target logs today, what it must log for these rules, and the cost of each addition.
- Evidence table — per technique, what appeared in each of the four sources, with real captured log extracts.
- Rule pack — each rule with signal, query, threshold, justification and coverage statement.
- Test results — true positives, false-positive rate against the benign baseline, and time-to-detect for every rule.
- Blind-spot register — the undetectable techniques, the physical reason, and the compensating control for each.
- Replay harness — attack and benign traffic generators, handed over.
- Coverage summary — one page mapping the techniques you attacked to detected / partially detected / undetectable, which is the artefact a security lead will actually read.
| Criterion | Developing | Competent | Strong |
|---|---|---|---|
| Instrumentation | Used default logs. | Added the headers the rules need. | Gap analysis with costs, and a rationale per field. |
| Evidence gathering | Assumed what logs would show. | Replayed attacks and read the logs. | Measured all four sources, including the negative results. |
| Rule quality | Payload string matches. | Ratio and sequence rules where appropriate. | Every rule states why a payload match would fail. |
| Testing | Rules written, not tested. | Fired on the attack. | False-positive rate and time-to-detect measured and tuned, with the log kept. |
| Blind spots | Not addressed. | Listed. | Physical reason given and a compensating control proposed for each. |
| Feedback to design | None. | Noted that a fix would help. | Concrete design changes that create evidence where none existed. |
| Operability | A list of ideas. | Queries a defender can run. | Full runbook plus a replay harness for re-verification. |
- Run it as a purple-team exercise. One person attacks on an unannounced schedule; the other operates the rules. Measure what was caught, what was missed, and how long triage took. Nothing else exposes a bad threshold this quickly.
- Instrument the client side. Stand up a CSP
report-uricollector and an error-telemetry endpoint, then re-run the browser-side attacks. Some of the "undetectable" rows move — and finding out exactly which ones is genuinely new information. - Detect the recon, not just the attack. Content discovery and parameter mining have very clean signatures. Feed the route inventory from [[Mini-Projects/Recon-and-Attack-Surface-Mapping|Recon and Attack Surface Mapping]] into a rule that alerts on requests to routes that do not exist in it.
- Detection as a deliverable. Add a
Detectionappendix to a real report from [[Mini-Projects/Web-Pentest-Report-Writing|Web Pentest Report Writing]] and see how the client responds. In practice it changes the conversation from "when will you fix this" to "what do we do on Monday".
- OWASP Top 10 2021 — A09: Security Logging and Monitoring Failures
- OWASP Logging Cheat Sheet
- OWASP Application Logging Vocabulary Cheat Sheet
- MITRE ATT&CK — Initial Access and the Enterprise matrix
- Sigma — generic signature format for SIEM systems
- MDN — Fetch metadata request headers
- web.dev — Protect your resources from web attacks with Fetch Metadata
- NIST SP 800-92 — Guide to Computer Security Log Management
- [[Practical-Labs/Lab-Padding-Oracle-Decryption|Lab: Padding Oracle Decryption]] — the source of the captured error-ratio numbers used in Pass 2.
- [[Practical-Labs/Lab-Cache-Deception|Lab: Cache Deception]] — the source of the two-layer cache detection story.
- [[Practical-Labs/Lab-Mass-Assignment-Privilege-Escalation|Lab: Mass Assignment Privilege Escalation]] — the "silent success" case that forces a data-layer rule.
- [[Practical-Labs/Lab-XS-Leaks-Frame-Counting|Lab: XS-Leaks Frame Counting]] — the hardest blind spot in the register.
- [[Practical-Labs/Lab-WebSocket-Hijacking|Lab: WebSocket Hijacking]] — the
Originrule, and why logging one header changes everything. - [[Crypto-and-Configuration/HTTP-Security-Response-Headers/Fetch-Metadata-Headers|Fetch Metadata Headers]] — the header set behind rules D and E.
- [[Crypto-and-Configuration/Debug-and-Error-Leakage|Debug and Error Leakage]] — the flip side: what the application tells the attacker through its errors.
- [[Mini-Projects/Remediation-Verification-Project|Remediation Verification Project]] — the sibling blue-team capstone; run them as a pair.
- [[Mini-Projects/Injection-Hunt-Project|Injection Hunt Project]] — the attack findings this project builds detections for.
- [[Mini-Projects/Recon-and-Attack-Surface-Mapping|Recon and Attack Surface Mapping]] — the route inventory that makes an unknown-route rule possible.
- [[Mini-Projects/Web-Pentest-Report-Writing|Web Pentest Report Writing]] — where the coverage summary lands in a client deliverable.
- [[Mini-Projects/Readme|Mini Projects]] — capstone index and shared engagement guidance.
- Home: [[Readme|Course Home]]