Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,19 @@ def _product_signature(entry: dict[str, Any]) -> tuple[str, str]:
return (_package_identity(entry), str(entry.get("product") or ""))


#: The product Vulnogram writes into ``affected[]`` when it allocates a CVE ID.
ALLOCATION_PLACEHOLDER_PRODUCT = "[VENDOR] [PRODUCT]"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

correctness: unverified placeholder format

No source rule covers this. It is an assumption about upstream Vulnogram JSON output.

In CVE 5.0 JSON, affected[] entries typically split vendor and product. If Vulnogram emits {"vendor": "[VENDOR]", "product": "[PRODUCT]"} on allocation, strict equality with "[VENDOR] [PRODUCT]" will miss it and continue to refuse the first push.

We should confirm the exact payload from a live allocation or support both "[VENDOR] [PRODUCT]" and product == "[PRODUCT]" (with no package) to be safe.



def _is_placeholder_entry(entry: Any) -> bool:
"""Return ``True`` for an entry with the allocation placeholder product and no package."""
return (
isinstance(entry, dict)
and entry.get("product") == ALLOCATION_PLACEHOLDER_PRODUCT
and not _package_identity(entry)
)


def _diff_affected_products(
current: list[dict[str, Any]],
new: list[dict[str, Any]],
Expand All @@ -191,16 +204,19 @@ def _diff_affected_products(
is read from ``packageURL`` or ``packageName``, whichever the entry
carries, so the two identifier shapes do not read as a rename.
Otherwise returns one line per dropped, added, or renamed signature so
the caller can decide whether to refuse or warn.
the caller can decide whether to refuse or warn. Allocation placeholders
in ``current`` are not authored content and are left out (#1736).
"""
current_sigs = {_product_signature(entry) for entry in current if isinstance(entry, dict)}
current_sigs = {
_product_signature(entry)
for entry in current
if isinstance(entry, dict) and not _is_placeholder_entry(entry)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

correctness: state-unconstrained exemption

No source rule covers this. It is a safety-boundary issue in the product-change guard.

The original issue (#1736) requested exempting the placeholder specifically for "state RESERVED, placeholder product".

Leaving the placeholder out of current_sigs unconditionally means any record in REVIEW, READY, or PUBLIC that carries this string bypasses the rename guard. Please thread current_state into the diff check so the exemption only fires when current_state in ("RESERVED", "DRAFT", None).

}
new_sigs = {_product_signature(entry) for entry in new if isinstance(entry, dict)}
if current_sigs == new_sigs:
return []
# First-time population (e.g. a freshly-allocated RESERVED record with an
# empty ``affected[]``) is not a product *change* — there is no prior
# signature to be renamed away from. Treat it as a non-diff so the guard
# does not force --allow-product-change for the normal first push.
# An empty ``affected[]`` is first-time population, not a product
# *change* — there is no prior signature to be renamed away from.
if not current_sigs:
return []
diffs: list[str] = []
Expand Down Expand Up @@ -254,7 +270,8 @@ def apply_merge_mode_guards(
current record and the new emission, raise
:class:`MergeModeRefused` with a diff so the caller can decide
whether the change is intentional (e.g. broadening the scope
to add a new package) or a regression.
to add a new package) or a regression. Replacing Vulnogram's
allocation placeholder is exempt.

With ``forbid_promotion_to_public`` (the vetted-ops path always sets
it), a push that would move the record to ``PUBLIC`` from any other
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,8 @@
**Merge mode (default on)** — before the POST, the script fetches
the record's current state and applies three safety nets against
the document about to be pushed: state-downgrade refusal,
references-by-URL merge, product/package-change refusal. See
references-by-URL merge, product/package-change refusal
(the first push after allocation is exempt). See
:mod:`vulnogram_api.merge_mode` for the full rules, and the
``--allow-state-downgrade`` / ``--replace-references`` /
``--allow-product-change`` / ``--full-replace`` flags below for the
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -510,6 +510,7 @@ When the state fetch fails, pass `fetched_state: "unknown"`.
Any other outcome (an interrupted run) may or may not have saved: push again, which the idempotence note below makes safe.

For Vulnogram, the push merges into the live record and refuses to rename an affected product or package.
The first push after allocation, which replaces Vulnogram's placeholder product, needs no override.
Two explicit overrides exist; use them only after confirming with the user:

- `--replace-references` — replace `references[]` instead of keeping existing URLs.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,19 @@ def _product_signature(entry: dict[str, Any]) -> tuple[str, str]:
return (_package_identity(entry), str(entry.get("product") or ""))


#: The product Vulnogram writes into ``affected[]`` when it allocates a CVE ID.
ALLOCATION_PLACEHOLDER_PRODUCT = "[VENDOR] [PRODUCT]"


def _is_placeholder_entry(entry: Any) -> bool:
"""Return ``True`` for an entry with the allocation placeholder product and no package."""
return (
isinstance(entry, dict)
and entry.get("product") == ALLOCATION_PLACEHOLDER_PRODUCT
and not _package_identity(entry)
)


def _diff_affected_products(
current: list[dict[str, Any]],
new: list[dict[str, Any]],
Expand All @@ -191,16 +204,19 @@ def _diff_affected_products(
is read from ``packageURL`` or ``packageName``, whichever the entry
carries, so the two identifier shapes do not read as a rename.
Otherwise returns one line per dropped, added, or renamed signature so
the caller can decide whether to refuse or warn.
the caller can decide whether to refuse or warn. Allocation placeholders
in ``current`` are not authored content and are left out (#1736).
"""
current_sigs = {_product_signature(entry) for entry in current if isinstance(entry, dict)}
current_sigs = {
_product_signature(entry)
for entry in current
if isinstance(entry, dict) and not _is_placeholder_entry(entry)
}
new_sigs = {_product_signature(entry) for entry in new if isinstance(entry, dict)}
if current_sigs == new_sigs:
return []
# First-time population (e.g. a freshly-allocated RESERVED record with an
# empty ``affected[]``) is not a product *change* — there is no prior
# signature to be renamed away from. Treat it as a non-diff so the guard
# does not force --allow-product-change for the normal first push.
# An empty ``affected[]`` is first-time population, not a product
# *change* — there is no prior signature to be renamed away from.
if not current_sigs:
return []
diffs: list[str] = []
Expand Down Expand Up @@ -254,7 +270,8 @@ def apply_merge_mode_guards(
current record and the new emission, raise
:class:`MergeModeRefused` with a diff so the caller can decide
whether the change is intentional (e.g. broadening the scope
to add a new package) or a regression.
to add a new package) or a regression. Replacing Vulnogram's
allocation placeholder is exempt.

With ``forbid_promotion_to_public`` (the vetted-ops path always sets
it), a push that would move the record to ``PUBLIC`` from any other
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,8 @@
**Merge mode (default on)** — before the POST, the script fetches
the record's current state and applies three safety nets against
the document about to be pushed: state-downgrade refusal,
references-by-URL merge, product/package-change refusal. See
references-by-URL merge, product/package-change refusal
(the first push after allocation is exempt). See
:mod:`vulnogram_api.merge_mode` for the full rules, and the
``--allow-state-downgrade`` / ``--replace-references`` /
``--allow-product-change`` / ``--full-replace`` flags below for the
Expand Down
31 changes: 24 additions & 7 deletions tools/cve-tool-vulnogram/oauth-api/src/vulnogram_api/merge_mode.py
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,19 @@ def _product_signature(entry: dict[str, Any]) -> tuple[str, str]:
return (_package_identity(entry), str(entry.get("product") or ""))


#: The product Vulnogram writes into ``affected[]`` when it allocates a CVE ID.
ALLOCATION_PLACEHOLDER_PRODUCT = "[VENDOR] [PRODUCT]"


def _is_placeholder_entry(entry: Any) -> bool:
"""Return ``True`` for an entry with the allocation placeholder product and no package."""
return (
isinstance(entry, dict)
and entry.get("product") == ALLOCATION_PLACEHOLDER_PRODUCT
and not _package_identity(entry)
)


def _diff_affected_products(
current: list[dict[str, Any]],
new: list[dict[str, Any]],
Expand All @@ -191,16 +204,19 @@ def _diff_affected_products(
is read from ``packageURL`` or ``packageName``, whichever the entry
carries, so the two identifier shapes do not read as a rename.
Otherwise returns one line per dropped, added, or renamed signature so
the caller can decide whether to refuse or warn.
the caller can decide whether to refuse or warn. Allocation placeholders
in ``current`` are not authored content and are left out (#1736).
"""
current_sigs = {_product_signature(entry) for entry in current if isinstance(entry, dict)}
current_sigs = {
_product_signature(entry)
for entry in current
if isinstance(entry, dict) and not _is_placeholder_entry(entry)
}
new_sigs = {_product_signature(entry) for entry in new if isinstance(entry, dict)}
if current_sigs == new_sigs:
return []
# First-time population (e.g. a freshly-allocated RESERVED record with an
# empty ``affected[]``) is not a product *change* — there is no prior
# signature to be renamed away from. Treat it as a non-diff so the guard
# does not force --allow-product-change for the normal first push.
# An empty ``affected[]`` is first-time population, not a product
# *change* — there is no prior signature to be renamed away from.
if not current_sigs:
return []
diffs: list[str] = []
Expand Down Expand Up @@ -254,7 +270,8 @@ def apply_merge_mode_guards(
current record and the new emission, raise
:class:`MergeModeRefused` with a diff so the caller can decide
whether the change is intentional (e.g. broadening the scope
to add a new package) or a regression.
to add a new package) or a regression. Replacing Vulnogram's
allocation placeholder is exempt.

With ``forbid_promotion_to_public`` (the vetted-ops path always sets
it), a push that would move the record to ``PUBLIC`` from any other
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,8 @@
**Merge mode (default on)** — before the POST, the script fetches
the record's current state and applies three safety nets against
the document about to be pushed: state-downgrade refusal,
references-by-URL merge, product/package-change refusal. See
references-by-URL merge, product/package-change refusal
(the first push after allocation is exempt). See
:mod:`vulnogram_api.merge_mode` for the full rules, and the
``--allow-state-downgrade`` / ``--replace-references`` /
``--allow-product-change`` / ``--full-replace`` flags below for the
Expand Down
61 changes: 61 additions & 0 deletions tools/cve-tool-vulnogram/oauth-api/tests/test_merge_mode.py
Original file line number Diff line number Diff line change
Expand Up @@ -457,6 +457,67 @@ def test_first_time_population_does_not_require_allow_product_change(self):
assert merged["containers"]["cna"]["affected"][0]["packageName"] == "apache-foo"


# ---------------------------------------------------------------------------
# Allocation placeholder (issue #1736)
# ---------------------------------------------------------------------------

PLACEHOLDER = {"product": "[VENDOR] [PRODUCT]"}
REAL_ENTRY = {"packageURL": "pkg:maven/org.apache.foo/foo-core", "product": "Apache Foo"}


def _allocated(**kwargs) -> dict:
"""A record as Vulnogram leaves it right after allocation."""
kwargs.setdefault("state", "DRAFT")
kwargs.setdefault("references", [])
return _current(**kwargs)


class TestAllocationPlaceholder:
def test_first_push_replaces_the_untouched_placeholder(self):
merged = apply_merge_mode_guards(
_allocated(affected=[PLACEHOLDER]),
_new(state="DRAFT", affected=[REAL_ENTRY]),
)
assert merged["containers"]["cna"]["affected"] == [REAL_ENTRY]

def test_dropping_a_placeholder_next_to_an_authored_entry_is_allowed(self):
merged = apply_merge_mode_guards(
_allocated(affected=[PLACEHOLDER, REAL_ENTRY]),
_new(state="DRAFT", affected=[REAL_ENTRY]),
)
assert merged["containers"]["cna"]["affected"] == [REAL_ENTRY]

def test_renaming_an_authored_entry_next_to_a_placeholder_still_refuses(self):
with pytest.raises(MergeModeRefused, match="product"):
apply_merge_mode_guards(
_allocated(affected=[PLACEHOLDER, REAL_ENTRY]),
_new(state="DRAFT", affected=[{"packageName": "apache-bar", "product": "Apache Bar"}]),
)

def test_placeholder_product_with_a_package_still_refuses(self):
with pytest.raises(MergeModeRefused, match="product"):
apply_merge_mode_guards(
_allocated(affected=[{"packageName": "apache-foo", "product": "[VENDOR] [PRODUCT]"}]),
_new(state="DRAFT", affected=[REAL_ENTRY]),
)

def test_a_vendor_on_the_placeholder_does_not_matter(self):
merged = apply_merge_mode_guards(
_allocated(affected=[{"vendor": "Apache Software Foundation", **PLACEHOLDER}]),
_new(state="DRAFT", affected=[REAL_ENTRY]),
)
assert merged["containers"]["cna"]["affected"] == [REAL_ENTRY]

def test_second_push_after_population_keeps_the_guard(self):
with pytest.raises(MergeModeRefused, match="product"):
apply_merge_mode_guards(
_allocated(affected=[REAL_ENTRY]),
_new(
state="DRAFT", affected=[{"packageName": "apache-foo-bar", "product": "Apache Foo Bar"}]
),
)


# ---------------------------------------------------------------------------
# Composition + edge cases
# ---------------------------------------------------------------------------
Expand Down
1 change: 1 addition & 0 deletions tools/skill-evals/evals/security-issue-sync/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ Columns and rules: [Recording runs](../../README.md#recording-runs).

| Date | Ref | Model | Runs | Result | Notes |
|---|---|---|---|---|---|
| 2026-11-18 | `57480dcf` (fix/1736-allocation-placeholder) | `opus[1m]` | 1 | pass | |
| 2026-11-18 | `add4ee52` (feat/security-skills-vetted-cve-ops) | `opus[1m]` | 2 | pass | |
| 2026-11-18 | `9e979ff3` (feat/security-skills-vetted-cve-ops) | `opus[1m]` | 1 | pass | |
| 2026-11-13 | `c22d5769` (feat/cve-tool-interface) | `claude-sonnet-5-5` | 10 | `step-2b-proposed-changes/case-2-milestone-missing-create` failed 1 of 2; `step-2b-proposed-changes/case-5-pre-announce-distributors` failed 2 of 2; `step-2b-proposed-changes/case-6-reporter-unresponsive` errored 1 of 2 | cross-model check for #1800; cross-model check for #1800 (re-run of infra failures) |
1 change: 1 addition & 0 deletions tools/skill-evals/evals/security-issue-sync/runs.jsonl
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,4 @@
{"branch": "feat/security-skills-vetted-cve-ops", "cases": 2, "cli": "claude -p", "cost": {"cli_calls": 2, "grader_calls": 2, "wall_seconds": 36.6}, "date": "2026-11-18", "dirty": true, "errored": 0, "errored_cases": [], "failed": 0, "failed_cases": [], "graded_cases": ["step-5b-judged-gates/case-4-private-scanner-credit", "step-5b-judged-gates/case-5-clean-push-fix-released"], "grader_cli": "claude -p --model haiku", "manual": 0, "model": "opus[1m]", "model_source": "settings:user", "notes": "", "passed": 2, "path": "tools/skill-evals/evals/security-issue-sync/step-5b-judged-gates", "ref": "9e979ff3", "schema": 1, "skill": "security-issue-sync", "tags": [], "timestamp": "2026-11-18T01:05:21Z"}
{"branch": "feat/security-skills-vetted-cve-ops", "cases": 2, "cli": "claude -p", "cost": {"cli_calls": 2, "grader_calls": 2, "wall_seconds": 36.6}, "date": "2026-11-18", "dirty": true, "errored": 0, "errored_cases": [], "failed": 0, "failed_cases": [], "graded_cases": ["step-5b-judged-gates/case-4-private-scanner-credit", "step-5b-judged-gates/case-5-clean-push-fix-released"], "grader_cli": "claude -p --model haiku", "manual": 0, "model": "opus[1m]", "model_source": "settings:user", "notes": "", "passed": 2, "path": "tools/skill-evals/evals/security-issue-sync/step-5b-judged-gates", "ref": "add4ee52", "schema": 1, "skill": "security-issue-sync", "tags": [], "timestamp": "2026-11-18T01:24:36Z"}
{"branch": "feat/security-skills-vetted-cve-ops", "cases": 2, "cli": "claude -p", "cost": {"cli_calls": 2, "grader_calls": 2, "wall_seconds": 36.0}, "date": "2026-11-18", "dirty": true, "errored": 0, "errored_cases": [], "failed": 0, "failed_cases": [], "graded_cases": ["step-5b-judged-gates/case-4-private-scanner-credit", "step-5b-judged-gates/case-5-clean-push-fix-released"], "grader_cli": "claude -p --model haiku", "manual": 0, "model": "opus[1m]", "model_source": "settings:user", "notes": "", "passed": 2, "path": "tools/skill-evals/evals/security-issue-sync/step-5b-judged-gates", "ref": "add4ee52", "schema": 1, "skill": "security-issue-sync", "tags": [], "timestamp": "2026-11-18T02:36:27Z"}
{"branch": "fix/1736-allocation-placeholder", "cases": 2, "cli": "claude -p", "cost": {"cli_calls": 2, "grader_calls": 2, "wall_seconds": 27.0}, "date": "2026-11-18", "dirty": false, "errored": 0, "errored_cases": [], "failed": 0, "failed_cases": [], "graded_cases": ["step-5b-judged-gates/case-4-private-scanner-credit", "step-5b-judged-gates/case-5-clean-push-fix-released"], "grader_cli": "claude -p --model haiku", "manual": 0, "model": "opus[1m]", "model_source": "settings:user", "notes": "", "passed": 2, "path": "tools/skill-evals/evals/security-issue-sync/step-5b-judged-gates", "ref": "57480dcf", "schema": 1, "skill": "security-issue-sync", "tags": [], "timestamp": "2026-11-18T06:53:14Z"}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

consistency: eval journal ref mismatch

The recorded ref here and in README.md is 57480dcf, whereas the commit on this branch is 28cddb28. Please re-render the journal entry with the matching commit ref so eval provenance is accurate.

1 change: 1 addition & 0 deletions tools/spec-loop/specs/cve-tooling.md
Original file line number Diff line number Diff line change
Expand Up @@ -265,6 +265,7 @@ reviewable.
default, refuses a `CNA_private.state` downgrade from `PUBLIC`, refuses
an `affected[].product` / package change, and merges `references[]` by
URL so a hand-added reference is kept.
Replacing Vulnogram's allocation placeholder is not a product change (#1736).
Each guard has an explicit override (`--allow-state-downgrade`,
`--allow-product-change`, `--replace-references`, or the umbrella
`--full-replace`).
Expand Down
Loading