-
Notifications
You must be signed in to change notification settings - Fork 43
LWLP-977: Add purl and coordinates to package versions endpoint #1703
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
etsien
wants to merge
59
commits into
content-services:main
Choose a base branch
from
etsien:add-purl-and-coordinates-to-package_versions-endpoint
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
59 commits
Select commit
Hold shift + click to select a range
ee17fa2
LWLP-35: create db schema for clearinghouse ui
TenSt e8e00c8
Merge branch 'pr/1635' into LWLP-5-Lightwell-Network-API
etsien f0048ec
build out phase 1: advisories, packages, repositories, package-versio…
etsien 212ca7b
Merge branch 'main' into LWLP-5-Lightwell-Network-API
etsien e990645
Update .gitignore
etsien 3373e92
v2 of API buildout Phase 1
etsien d18fec3
add tests
etsien bcdc0d2
Create advisories.sql.go
etsien ec38125
Merge branch 'main' into LWLP-5-Lightwell-Network-API
etsien a71bf04
revert docs changes
etsien 0d66a2f
Revert "revert docs changes"
etsien e79f887
update docs and api
etsien 8827f3f
re-add tests
etsien cc7b447
update migrations
etsien 949fedd
bugfix
etsien 9501f5d
Update migrations.latest
etsien 7e92db7
Update store_test.go
etsien 38da107
LWLP-5: add Lightwell advisory schema, sqlc queries, and store tests
etsien 7344968
LWLP-5: add Lightwell advisories and packages API
etsien d6351b9
LWLP-5: regenerate OpenAPI spec for Lightwell endpoints
etsien 5d2ed76
Merge branch 'LWLP-5-Lightwell-Network-API' of https://github.com/ets…
etsien 0b5baac
LWLP-5: add Lightwell advisory schema, sqlc queries, and store tests
etsien 5e7a921
LWLP-5: add Lightwell advisories and packages API
etsien 32508fa
LWLP-5: regenerate OpenAPI spec for Lightwell endpoints
etsien f2c37d9
Merge branch 'LWLP-5-Lightwell-Network-API' of https://github.com/ets…
etsien 500fa1c
revert and delete duplicates sql files
etsien 8aa9d3d
update calls to use DAO package
etsien fe1c192
LWLP-5: add Lightwell advisory schema, sqlc queries, and store tests
etsien 57cb035
LWLP-5: add Lightwell advisories and packages API via DAO layer
etsien 7dad8d6
LWLP-5: regenerate OpenAPI spec for Lightwell endpoints
etsien e3e7cf4
LWLP-5: regenerate mocks
etsien 8148581
LWLP-5: fix spec deviations and test isolation from merge resolution
etsien dfbaaff
Merge remote branch origin/LWLP-5-Lightwell-Network-API
etsien f9d9539
LWLP-5: resolve merge conflict with upstream main
etsien e939ceb
LWLP-5: align with upstream main changes
etsien b9413dd
more rebase fixes
etsien 8923b2a
remove repeated call
etsien e7f7514
fix swagger annotation
etsien 3b1a2e6
add the v2 API spec
etsien a4b8f6d
fix linting issues
etsien f749048
update api docs
etsien 42f813b
Update interfaces.go
etsien 872eed6
rename template
etsien 96fb90f
Merge branch 'main' into LWLP-5-Lightwell-Network-API
etsien 7214c79
bring back accidentally removed rebased stuff
etsien 2215df8
reuse existing counts, remove redundant fields
etsien 2d1bfb4
regenerate docs
etsien 3c375a4
Merge upstream/main into LWLP-5-Lightwell-Network-API
etsien 40d31f4
fix: rename advisory migration to follow upstream timestamp
etsien cfab2a2
Create handler-test-patterns.mdc
etsien 168e52e
add properties to PackageVersion
etsien 349cb8c
add PURL and coordinate builders, adjust CVE filters to use repo name…
etsien f11d6e9
add tests
etsien de0e4aa
Merge branch 'main' into add-purl-and-coordinates-to-package_versions…
etsien 228baa5
add re-generated docs
etsien 1a333f8
Merge branch 'add-purl-and-coordinates-to-package_versions-endpoint' …
etsien 631b703
Merge upstream/main into add-purl-and-coordinates-to-package_versions…
etsien a3eac03
regenerate docs
etsien 4333b10
fix: remove duplicate severity migration superseded by upstream
etsien File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -34,6 +34,8 @@ func RegisterLightwellPackageRoutes(engine *echo.Group, daoReg *dao.DaoRegistry, | |
| } | ||
| addRepoRoute(engine, http.MethodGet, "/lightwell/packages", h.listPackages, rbac.RbacVerbRead) | ||
| addRepoRoute(engine, http.MethodGet, "/lightwell/package_versions", h.listPackageVersions, rbac.RbacVerbRead) | ||
| addRepoRoute(engine, http.MethodGet, "/lightwell/repositories/:repository_name/packages", h.listRepoPackages, rbac.RbacVerbRead) | ||
| addRepoRoute(engine, http.MethodGet, "/lightwell/repositories/:repository_name/package_versions", h.listRepoPackageVersions, rbac.RbacVerbRead) | ||
|
Comment on lines
+37
to
+38
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same here |
||
| } | ||
|
|
||
| // listLightwellPackages godoc | ||
|
|
@@ -221,7 +223,6 @@ func (h *LightwellPackagesHandler) fetchPackagesFromRepo(ctx context.Context, re | |
| return nil, err | ||
| } | ||
|
|
||
| // Fetch all packages from this repo (no server-side pagination — small datasets) | ||
| pageOpts := tangy.PageOptions{Offset: 0, Limit: MaxLimit} | ||
|
|
||
| switch repo.ContentType { | ||
|
|
@@ -355,19 +356,22 @@ func (h *LightwellPackagesHandler) filterVersionsByResolvingCve(ctx context.Cont | |
| return nil, err | ||
| } | ||
|
|
||
| fixedSet := make(map[string]map[string]bool) | ||
| type repoPackage struct{ repo, name string } | ||
| fixedSet := make(map[repoPackage]map[string]bool) | ||
| for _, m := range matches { | ||
| if fixedSet[m.PackageName] == nil { | ||
| fixedSet[m.PackageName] = make(map[string]bool) | ||
| key := repoPackage{repo: m.RepoName, name: m.PackageName} | ||
| if fixedSet[key] == nil { | ||
| fixedSet[key] = make(map[string]bool) | ||
| } | ||
| for _, v := range m.FixedVersions { | ||
| fixedSet[m.PackageName][v] = true | ||
| fixedSet[key][v] = true | ||
| } | ||
| } | ||
|
|
||
| var result []api.LightwellPackageVersionResponse | ||
| for _, item := range items { | ||
| if versions, ok := fixedSet[item.Name]; ok && versions[item.Version] { | ||
| key := repoPackage{repo: item.Repository, name: item.Name} | ||
| if versions, ok := fixedSet[key]; ok && versions[item.Version] { | ||
| result = append(result, item) | ||
| } | ||
| } | ||
|
|
@@ -382,27 +386,65 @@ func (h *LightwellPackagesHandler) filterVersionsByVulnerableCve(ctx context.Con | |
| return nil, err | ||
| } | ||
|
|
||
| affectedPackages := make(map[string]bool) | ||
| fixedSet := make(map[string]map[string]bool) | ||
| type repoPackage struct{ repo, name string } | ||
| affectedPackages := make(map[repoPackage]bool) | ||
| fixedSet := make(map[repoPackage]map[string]bool) | ||
| for _, m := range matches { | ||
| affectedPackages[m.PackageName] = true | ||
| if fixedSet[m.PackageName] == nil { | ||
| fixedSet[m.PackageName] = make(map[string]bool) | ||
| key := repoPackage{repo: m.RepoName, name: m.PackageName} | ||
| affectedPackages[key] = true | ||
| if fixedSet[key] == nil { | ||
| fixedSet[key] = make(map[string]bool) | ||
| } | ||
| for _, v := range m.FixedVersions { | ||
| fixedSet[m.PackageName][v] = true | ||
| fixedSet[key][v] = true | ||
| } | ||
| } | ||
|
|
||
| var result []api.LightwellPackageVersionResponse | ||
| for _, item := range items { | ||
| if affectedPackages[item.Name] && !fixedSet[item.Name][item.Version] { | ||
| key := repoPackage{repo: item.Repository, name: item.Name} | ||
| if affectedPackages[key] && !fixedSet[key][item.Version] { | ||
| result = append(result, item) | ||
| } | ||
| } | ||
| return result, nil | ||
| } | ||
|
|
||
| // --- PURL / coordinate builders --- | ||
|
|
||
| func buildPURL(contentType, group, name, version string) string { | ||
| switch contentType { | ||
| case config.ContentTypeMaven: | ||
| return fmt.Sprintf("pkg:maven/%s/%s@%s", group, name, version) | ||
| case config.ContentTypePython: | ||
| return fmt.Sprintf("pkg:pypi/%s@%s", name, version) | ||
| case config.ContentTypeNpm: | ||
| if group == "-" || group == "" { | ||
| return fmt.Sprintf("pkg:npm/%s@%s", name, version) | ||
| } | ||
| scope := strings.TrimPrefix(group, "@") | ||
| return fmt.Sprintf("pkg:npm/%%40%s/%s@%s", scope, name, version) | ||
| default: | ||
| return "" | ||
| } | ||
| } | ||
|
|
||
| func buildCoordinates(contentType, group, name string) string { | ||
| switch contentType { | ||
| case config.ContentTypeMaven: | ||
| return fmt.Sprintf("%s:%s", group, name) | ||
| case config.ContentTypePython: | ||
| return name | ||
| case config.ContentTypeNpm: | ||
| if group == "-" || group == "" { | ||
| return name | ||
| } | ||
| return fmt.Sprintf("%s/%s", group, name) | ||
| default: | ||
| return "" | ||
| } | ||
| } | ||
|
|
||
| // --- mapping helpers --- | ||
|
|
||
| func mapMavenToLightwellPackages(resp tangy.MavenPackageListResponse, repo api.RepositoryResponse) []api.LightwellPackageResponse { | ||
|
|
@@ -477,6 +519,8 @@ func expandMavenVersions(resp tangy.MavenPackageListResponse, repo api.Repositor | |
| Ecosystem: config.ContentTypeMaven, | ||
| Repository: repo.Name, | ||
| RepositoryUUID: repo.UUID, | ||
| Purl: buildPURL(config.ContentTypeMaven, item.GroupID, item.ArtifactID, v), | ||
| Coordinates: buildCoordinates(config.ContentTypeMaven, item.GroupID, item.ArtifactID), | ||
| } | ||
| if rel, ok := relMap[v]; ok { | ||
| ver.Release = rel.Release | ||
|
|
@@ -499,6 +543,8 @@ func expandPythonVersions(resp tangy.PythonPackageListResponse, repo api.Reposit | |
| Ecosystem: config.ContentTypePython, | ||
| Repository: repo.Name, | ||
| RepositoryUUID: repo.UUID, | ||
| Purl: buildPURL(config.ContentTypePython, "", item.NameNormalized, v), | ||
| Coordinates: buildCoordinates(config.ContentTypePython, "", item.NameNormalized), | ||
| } | ||
| if info, ok := verMap[v]; ok { | ||
| ver.CreatedAt = info.CreatedAt | ||
|
|
@@ -522,6 +568,8 @@ func expandNpmVersions(resp tangy.NpmPackageListResponse, repo api.RepositoryRes | |
| Ecosystem: config.ContentTypeNpm, | ||
| Repository: repo.Name, | ||
| RepositoryUUID: repo.UUID, | ||
| Purl: buildPURL(config.ContentTypeNpm, scope, name, v), | ||
| Coordinates: buildCoordinates(config.ContentTypeNpm, scope, name), | ||
| } | ||
| if info, ok := verMap[v]; ok { | ||
| ver.CreatedAt = info.CreatedAt | ||
|
|
@@ -641,6 +689,20 @@ func npmVersionMap(versions []tangy.NpmVersionInfo) map[string]versionCreatedAt | |
| return m | ||
| } | ||
|
|
||
| // --- nested repo-scoped alias handlers --- | ||
|
|
||
| func (h *LightwellPackagesHandler) listRepoPackages(c echo.Context) error { | ||
| repoName := c.Param("repository_name") | ||
| c.QueryParams().Set("repository", repoName) | ||
| return h.listPackages(c) | ||
| } | ||
|
|
||
| func (h *LightwellPackagesHandler) listRepoPackageVersions(c echo.Context) error { | ||
| repoName := c.Param("repository_name") | ||
| c.QueryParams().Set("repository", repoName) | ||
| return h.listPackageVersions(c) | ||
| } | ||
|
|
||
| // --- sort helpers --- | ||
|
|
||
| func sortLightwellPackages(items []api.LightwellPackageResponse, sortBy string) { | ||
|
|
||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this route got leftover from a rebase?