Conversation
Serve Lightwell advisories through an osv.dev-compatible API so OSV
tooling (e.g. osv-scanner) can consume our content until it is accepted
upstream by osv.dev. The mock is public (no x-rh-identity), rooted at
/demo/osvdev, and gated by the LightwellOsvDemo feature flag (off by
default).
Implements the query API (POST /v1/query, POST /v1/querybatch,
GET /v1/vulns/{id}) plus the GCS-style static export
(<ecosystem>/all.zip, <id>.json, ecosystems.txt). Advisories are grouped
by AdvisoryID into OSV records via pkg/lightwell/osv, backed by a new
LightwellAdvisoryDao.ListForOsv method that returns full models with
timestamps.
Note: the ListForOsv mock in pkg/dao/dao_mock.go was hand-added in the
generated style; regenerate with `make mock` before merge.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an osv.dev-compatible mock API under
/demo/osvdevthat serves our Lightwell advisory data in the OSV format, so OSV tooling (e.g.osv-scanner) can consume our content by pointing at this host instead ofapi.osv.dev— until the data is accepted upstream by osv.dev.The endpoint is public / unauthenticated (external scanners send no
x-rh-identity), rooted at/demo/osvdev, and gated behind a newLightwellOsvDemofeature flag (off by default).Endpoints
Query API (mirrors
api.osv.dev):POST /demo/osvdev/v1/query—{"vulns":[…]}, or{}on no matchPOST /demo/osvdev/v1/querybatch— index-alignedresults,{id, modified}stubsGET /demo/osvdev/v1/vulns/{id}— full record, 404 if unknownStatic export (mirrors
storage.googleapis.com/osv-vulnerabilities):GET /demo/osvdev/{ecosystem}/all.zip,GET /demo/osvdev/all.zipGET /demo/osvdev/{ecosystem}/{id}(serves<id>.json)GET /demo/osvdev/ecosystems.txtHow it works
lightwell_advisoriesrows →LightwellAdvisoryDao.ListForOsv→pkg/lightwell/osv.BuildRecords(grouped byAdvisoryID, oneaffected[]entry per row) → serialized as OSV JSON / zipped in-memory. No files are stored on disk; records are built per request.Files
pkg/api/osvdev.gopkg/lightwell/osv/converter.gopkg/handler/osvdev.goListForOsv)pkg/dao/lightwell_advisory.go,pkg/dao/interfaces.gopkg/handler/api.go/demo/osvdevprefix)pkg/middleware/enforce_identity.gopkg/config/config.goThis PR wires up the API, but a few things are not yet done and are required for the endpoint to return anything useful in a demo/stage/prod environment. Flagging them explicitly so reviewers can decide what belongs in this PR vs. follow-ups.
1. The
lightwell_advisoriestable is empty in demo/localThe mock reads
lightwell_advisories, which is populated only by thesync-lightwell-advisoriescronjob. That job:pkg/external_repos/lightwell_repos.json(viaLoadLightwellAllowlist()), notlightwell_demo_repos.json, andosv_path(currently justlightwell/java/remediatedandlightwell/java/predisclosure).Consequences:
osv_path, and the sync ignores it anyway./demo/osvdevreturns{}/ empty zips.SUSPEND_SYNC_LIGHTWELL_ADVISORIESdefaults tofalse), assuming the OSV source is reachable.Options (pick one, follow-up):
lightwell_advisories(only lightwell vulnerabilities are seeded today) soLoadLightwellDemo/ local dev returns data.osv_pathentries tolightwell_demo_repos.jsonand teachsync-lightwell-advisoriesto use the demo allowlist when in demo mode.sync-lightwell-advisoriesrun for demos.2. Ecosystem is hardcoded to
"Red Hat"— likely wronglightwell_advisorieshas no ecosystem column, sopkg/lightwell/osv.DefaultEcosystemcurrently returns a fixed"Red Hat". But the real advisories come from repos typedmaven(Java) /python— so the correct OSV ecosystem isMaven/PyPI. The ecosystem is derivable fromRepoName(lightwell/java/…,lightwell/python/…).TODO: derive ecosystem from
RepoName(java→Maven, python→PyPI, fallback default), or add an ecosystem column and thread it through the advisory sync.3. Lossy OSV → advisory → OSV round-trip
Advisories are themselves ingested from OSV files (
processOSVForEntryinsync_lightwell_advisories.go). We map OSV →lightwell_advisories(dropping fields not stored, e.g. severity vectors, aliases, per-rangeintroduced) → back to OSV. Fields not persisted can't be reconstructed. If fidelity matters, consider serving from (or enriching with) the original OSV payloads.4. Deployment/config wiring for the flag
features.lightwell_osv_demo.enabledhas no ClowdApp parameter yet. To toggle it per-environment (stage/prod), add an env var + template wiring indeployments/. Not needed for local demo.5. Regenerate mocks with the pinned mockery
The
ListForOsvmock inpkg/dao/dao_mock.gowas hand-added in the repo's existing generated style.make mocklocally pulled a newer mockery that reformats every mock (interface{}→any, addsHelper()), which was reverted as unrelated churn. Please regenerate with the CI-pinned mockery version before/at merge so the generated output is authoritative.Testing steps
Automated:
go test ./pkg/lightwell/osv/...— converter unit tests (grouping, ranges,Matches, version compare)go test ./pkg/handler/ -run TestOsvDevSuite— handler suite (query hit/miss, batch index alignment + stubs,vulns/{id}200/404,all.zip, flag-off ⇒ 404, no identity header required, DAO error). Needs a Postgres (like all handler tests).golangci-lint run --timeout=5m— 0 issuesgit diff --exit-code api/openapi.json— no drift (routes intentionally omit swagger annotations)Manual (requires data — see "Missing pieces #1"):
features.lightwell_osv_demo.enabled: trueinconfigs/config.yaml.lightwell_advisories(seed / manual insert /sync-lightwell-advisories).:8000), then without any identity header:curl -sXPOST localhost:8000/demo/osvdev/v1/query -d '{"package":{"name":"<pkg>"},"version":"1.0.0"}'curl -s localhost:8000/demo/osvdev/v1/vulns/<AdvisoryID>curl -sXPOST localhost:8000/demo/osvdev/v1/querybatch -d '{"queries":[{"package":{"name":"<pkg>"},"version":"1.0.0"},{"commit":"deadbeef"}]}'curl -sO localhost:8000/demo/osvdev/all.zip && unzip -l all.zip🤖 Generated with Claude Code