feat: v0.4.0 — where your site contradicts itself

.claude-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "omnirank",
-  "version": "0.3.0",
+  "version": "0.4.0",
   "description": "Audit any website across SEO, answer-engine and generative-engine criteria, and generate llms.txt, llms-full.txt and facts.json so AI engines can ingest it as ground truth.",
   "author": { "name": "S M Moshiur Rahman", "email": "moshiur@publicpulse.com.bd" },
   "homepage": "https://github.com/bemoshiur/OmniRank",

.github/workflows/ci.yml

@@ -16,7 +16,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python-version: ["3.11", "3.12", "3.13"]
+        python-version: ["3.11", "3.12", "3.13", "3.14"]
     steps:
       - uses: actions/checkout@v7
 

CHANGELOG.md

@@ -6,6 +6,82 @@ All notable changes to this project are documented here. The format follows
 
 ## [Unreleased]
 
+## [0.4.0] - 2026-08-04
+
+**"OmniRank finds where your site contradicts itself."** 18 new finding ids, a new
+`security` layer, and the scoring fix that had to land before any of them.
+
+This release writes nothing. `omnirank fix` still previews diffs and modifies no
+file — v0.3.0's strings promising `--write` "in v0.4.0" were corrected rather than
+honoured, because auditing better is zero-risk and a richer audit is what earns
+the right to edit files later.
+
+### Changed — scoring (behavioural)
+
+- Each layer's 100-point budget is now divided by the number of gates that layer
+  actually has, instead of being a flat constant. One maximally-broken gate costs
+  its layer exactly `1/N`, and a layer reaches 0 only when **every** one of its
+  gates is maxed. Under the old model, seven broken gates zeroed a layer whether
+  it had seven gates or thirty — so each of this release's eight new `seo` gates
+  would have made saturation cheaper (44% of the layer to floor it at 16 gates,
+  29% at 24). `GATE_CAP` is unchanged and still caps any single gate.
+  **Scores from 0.3.0 and 0.4.0 are not comparable.**
+
+### Added — `security` layer
+
+`security.hsts.missing`, `security.hsts.short-max-age`, `security.nosniff.missing`,
+`security.csp.absent`, `security.referrer-policy.missing` — all `info`, all costing
+zero points. The gap analysis puts header grading out of scope by identity, so
+these are reported as inventory facts and never graded. `HSTS_MIN_MAX_AGE` is
+OmniRank's own floor and the finding says so; no claim is made about any browser
+preload list. CSP **absence** is reported; CSP **contents** deliberately are not.
+
+`security.mixed-content.subresource` (error) and `security.https-redirect.missing`
+(error) are the two that break something observable. Mixed content is suppressed by
+`upgrade-insecure-requests` and ignores `<link>` rels that are not subresources.
+The layer enters `layersRun` only when it actually ran.
+
+### Added — indexability contradictions
+
+`seo.robots-sitemap.disallowed`, `seo.canonical-target.noindexed`,
+`seo.canonical-target.not-found`, `seo.canonical-target.redirects`,
+`seo.hreflang-noindex.alternate`. Every one is provable from the site's own
+declarations. Canonical targets outside the crawled set are probed once each,
+deduplicated, capped at 25 per audit; anything past the cap is `budget-exceeded`
+in `notEvaluated`, never skipped silently.
+
+### Added — structured data
+
+`seo.schema-required.missing-property` validates Google's required rich-result
+properties for Article/NewsArticle/BlogPosting, Product, FAQPage, BreadcrumbList,
+Organization and LocalBusiness. `warning`, not `error`: the table is a transcription
+on a fixed date with no freshness test, and a stale required-property table
+fabricates errors. Every finding states that the markup is valid schema.org and
+that what is unmet is Google's requirement — two different claims.
+
+### Added — on-page
+
+`seo.image-alt.missing` (never fires on `alt=""`, which is the spec's decorative
+marker), `seo.heading-order.skipped`, `seo.link-text.empty`, `seo.link-text.generic`
+(info, English-only word list), `seo.lang.missing` (error — a missing `lang` makes
+`bands.resolve_band()` measure a non-Latin page against the Latin word band, so
+`aeo.answer-block.length` reports a problem that does not exist).
+
+### Added — report schema (additive)
+
+`"security"` in the layer enum; `"matcher-unsupported"` and `"budget-exceeded"` in
+the `notEvaluated` reason enum. Reports written before 0.4.0 still validate.
+
+### Notes on Python version
+
+`urllib.robotparser` became RFC 9309 compliant in **Python 3.14**. On 3.11–3.13 it
+ignores path wildcards and matches rules in file order, so it can report a URL as
+disallowed that RFC 9309 says is allowed. `seo.robots-sitemap.disallowed` probes
+the interpreter's actual behaviour and, when the site's published rules need a
+capability this interpreter lacks, records `matcher-unsupported` in `notEvaluated`
+rather than answering. Coverage of that one gate is therefore broader on Python
+3.14+ than on 3.11–3.13; correctness is identical on both.
+
 ## [0.3.0] - 2026-08-04
 
 "The locator." OmniRank can now say which file is wrong and show the diff that would
@@ -233,7 +309,8 @@ against a real 57-URL site.
   path and a Node in-repo path producing identical output.
 - `omnirank` CLI with a zero-config URL mode and CI-usable exit codes.
 
-[Unreleased]: https://github.com/bemoshiur/OmniRank/compare/v0.3.0...HEAD
+[Unreleased]: https://github.com/bemoshiur/OmniRank/compare/v0.4.0...HEAD
+[0.4.0]: https://github.com/bemoshiur/OmniRank/compare/v0.3.0...v0.4.0
 [0.3.0]: https://github.com/bemoshiur/OmniRank/compare/v0.2.1...v0.3.0
 [0.2.1]: https://github.com/bemoshiur/OmniRank/compare/v0.2.0...v0.2.1
 [0.2.0]: https://github.com/bemoshiur/OmniRank/compare/v0.1.1...v0.2.0

CITATION.cff

@@ -22,5 +22,5 @@ keywords:
   - structured-data
   - llms-txt
 license: MIT
-version: 0.3.0
+version: 0.4.0
 date-released: "2026-08-04"

README.md

@@ -110,18 +110,27 @@ Exit codes: `0` clean · `1` a `--fail-on` gate failed · `2` usage or config er
 non-zero exit is what makes it a CI gate, not just a report — see
 [Use OmniRank as a CI gate](#use-omnirank-as-a-ci-gate) below.
 
-## What ships in v0.3.0
+## What ships in v0.4.0
 
 | Skill | Status | What it does |
 |---|---|---|
-| `audit` | **Shipped** | Scores SEO, AEO, GEO, perf and structured-data gates against a site's real HTML — including a cross-URL pass over the whole crawled set — and reports `observed` / `expected` / `fix` for every gap |
+| `audit` | **Shipped** | Scores SEO, AEO, GEO, perf, security and structured-data gates against a site's real HTML — including a cross-URL pass over the whole crawled set — and reports `observed` / `expected` / `fix` for every gap |
 | `geo-artifacts` | **Shipped** | Generates `llms.txt`, `llms-full.txt` and `facts.json`, each with an explicit citation licence |
-| `fix` *(preview)* | **Shipped** | Resolves a finding's URL to the source file that owns it and prints the unified diff it would apply for the four `mechanical` findings. Writes nothing — there is no `--write` flag in 0.3.0 |
-
-Still exactly two skills. 0.3.0 added the locator — URL to source file — plus a
-`fixTier` on every finding and the `omnirank fix` diff preview. **This release writes
-nothing to your project.** File modification arrives in v0.4.0, deliberately after the
-locator has been proven. Everything under Roadmap below is not present in v0.3.0.
+| `fix` *(preview)* | **Shipped** | Resolves a finding's URL to the source file that owns it and prints the unified diff it would apply for the four `mechanical` findings. Writes nothing — there is no `--write` flag |
+
+Still exactly two skills. v0.4.0 adds 18 new finding ids across four groups: a new
+`security` layer (HSTS, `nosniff`, CSP absence, `Referrer-Policy`, mixed content, and a
+missing HTTP→HTTPS redirect); indexability contradictions provable from a site's own
+declarations (a sitemap URL its own `robots.txt` disallows, a canonical pointing at a
+noindexed/redirecting/missing page, a `hreflang` alternate that is itself noindexed);
+Google's required rich-result properties for eight structured-data types; and on-page
+accessibility overlap (missing `alt`, skipped heading levels, empty or generic link
+text, missing `lang`). It also normalises the score model so a layer's budget scales
+with its own gate count instead of a flat constant — **scores from 0.3.0 and 0.4.0 are
+not comparable.** `omnirank fix` still writes nothing to your project; file
+modification ships once the locator is proven against real repositories and the write
+guarantees in `docs/research/2026-08-04-automation-architecture.md` §2.5 are
+implemented and tested. Everything under Roadmap below is not present in v0.4.0.
 
 ## Roadmap
 
@@ -144,13 +153,15 @@ Google" will not trigger anything, because `aeo-onpage` and `indexing` are not b
 
 ## Every gate OmniRank checks, grouped by layer
 
-28 gate names exist in the config schema; 15 can fail a build and 13 are warning-only by
-design. (As of v0.2.1, `crawl-hygiene` — a gate name that could never actually fire,
-since its only source was never called from `audit_site()` — was removed from the schema
-rather than shipped as a config option that silently did nothing; `hygiene.check_sitemap()`
-was wired into `sitemap-health` instead, so a redirecting or dead sitemap URL now gets its
-own dedicated finding.) Full detail, including which gates can never trip `--fail-on`, is in
-[audit-guide.md](docs/audit-guide.md#gate-reference) and
+42 gate names exist in the config schema; 21 can fail a build (error-capable) and 21
+cannot — 16 are warning-only, 4 (the security header gates) are info-only, and one mixes
+warning and info. (As of v0.2.1, `crawl-hygiene` — a gate name that could never actually
+fire, since its only source was never called from `audit_site()` — was removed from the
+schema rather than shipped as a config option that silently did nothing;
+`hygiene.check_sitemap()` was wired into `sitemap-health` instead, so a redirecting or
+dead sitemap URL now gets its own dedicated finding.) Full detail, including which gates
+can never trip `--fail-on`, is in [audit-guide.md](docs/audit-guide.md#gate-reference)
+and
 [ci-integration.md](docs/ci-integration.md#choosing---fail-on-gates--and-why-gate-on-everything-is-a-trap).
 
 **SEO**
@@ -189,6 +200,19 @@ own dedicated finding.) Full detail, including which gates can never trip `--fai
 |---|---|---|
 | `schema` | At least one valid `application/ld+json` block with `@type` | error |
 | `schema-fabrication` | `AggregateRating` has a real `ratingCount`; every `Review` has an `author` | error |
+| `schema-required` | Google's rich-result properties (headline/image/datePublished, offers, mainEntity, …) are present for the types that need them | warning |
+
+**Security** — new in v0.4.0; four header gates are info-only inventory, never graded
+
+| Gate | Rule | Severity |
+|---|---|---|
+| `hsts` | `Strict-Transport-Security` present, `max-age` ≥ 180 days | info |
+| `nosniff` | `X-Content-Type-Options: nosniff` present | info |
+| `csp` | A `Content-Security-Policy` exists, by header or meta | info |
+| `referrer-policy` | A `Referrer-Policy` header exists | info |
+| `mixed-content` | An `https://` page requests a blockable subresource (`script`, `iframe`, stylesheet/preload) over `http://` | error |
+| `mixed-content` | An `https://` page requests a passive subresource (`img`, favicon) over `http://` — browsers auto-upgrade these | warning |
+| `https-redirect` | The site's `http://` origin 3xx-redirects to `https://` | error |
 
 **Site-level (cross-URL)** — new in 0.2.0; needs the whole crawled set, not one page
 
@@ -200,6 +224,15 @@ own dedicated finding.) Full detail, including which gates can never trip `--fai
 | `canonical-cluster` | A canonical points at a page that itself canonicalises elsewhere | warning |
 | `hreflang-reciprocity` | An `hreflang` alternate does not link back | warning |
 
+**Indexability contradictions** — new in v0.4.0; 100% precision, both halves of every
+contradiction come from the site's own declarations
+
+| Gate | Rule | Severity |
+|---|---|---|
+| `robots-sitemap` | A sitemap URL is `Disallow`-ed by the site's own `robots.txt` | error |
+| `canonical-target` | A canonical points at a page that is noindexed, 404s/410s, or redirects | error / warning |
+| `hreflang-noindex` | An `hreflang` alternate points at a page that is noindexed | error |
+
 **Performance** — new in 0.2.0; derived from one HTTP response, no browser involved
 
 | Gate | Rule | Severity |
@@ -215,6 +248,15 @@ Next Paint, or a Lighthouse score — OmniRank has no browser. See
 constants and [faq.md](docs/faq.md#does-omnirank-measure-core-web-vitals) for the full
 answer.
 
+**On-page** — new in v0.4.0; body markup, not the `<head>` indexing signals above
+
+| Gate | Rule | Severity |
+|---|---|---|
+| `image-alt` | Every `<img>` has an `alt` attribute (an empty `alt=""` is never flagged) | warning |
+| `heading-order` | The heading outline never skips more than one level | warning |
+| `link-text` | Every link has an accessible name, and it isn't a generic phrase like "click here" | warning / info |
+| `lang` | `<html>` declares a non-empty `lang` | error |
+
 ## Download and install OmniRank
 
 **Option 1 — packaged skill for Claude Code, no clone required:**
@@ -282,7 +324,8 @@ python3 -m omnirank.cli fix --config omnirank.config.json --root .
 ```
 
 Picking `--fail-on h1 canonical schema` (structural baseline) is a better starting point
-than listing all 28 gate names — 13 of them are warning-only and can never fail a build.
+than listing all 42 gate names — 21 of them are warning- or info-only and can never fail
+a build.
 The full reasoning, plus a GitLab CI job and a generic shell script, is in
 [ci-integration.md](docs/ci-integration.md).
 
@@ -403,14 +446,18 @@ will eventually read from `secrets`.
 <details>
 <summary>Why does my site score 0 on one layer?</summary>
 
-As of v0.2.1, each GATE's contribution to its layer is capped at `GATE_CAP = 15`
+Each GATE's contribution to its layer is capped at `GATE_CAP = 15`
 (`min(15, 10*errors + 3*warnings)`), so one gate failing on every page of a large site
 can no longer alone drag a layer to 0 — that would conflate issue COUNT with issue
-SEVERITY (one broken template is one problem, not fifty). A layer still reaches 0 when
-enough DISTINCT gates are broken for their capped costs to sum past 100 — seven or more
-independently-broken gates is enough on its own. A brand-new site missing a `<title>`,
-canonical tag, JSON-LD, and `llms.txt` will commonly hit this on the GEO layer, since
-each missing artifact is a separate broken gate.
+SEVERITY (one broken template is one problem, not fifty). As of v0.4.0, the summed capped
+cost is then divided by the layer's own scoring SURFACE — how many distinct gates could
+move that layer's score — so a layer reaches 0 only when **every one of its registered
+gates** is maxed, not after a fixed number of them. `seo` ships 24 scoring gates, and
+measured directly: 7 maxed gates score `71`, 12 score `50`, 16 score `33`, and only all
+24 reach `0`. A small layer still floors easily — `security` ships only 2 scoring gates
+(`mixed-content`, `https-redirect`; its four header gates are `info` and excluded from
+the surface entirely) — but a big layer like `seo` no longer floors on a handful of
+broken gates the way it used to.
 </details>
 
 More questions, including secrets handling and where the JSON report schema lives, are