Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion CHANGELOG.MD
Original file line number Diff line number Diff line change
Expand Up @@ -39,4 +39,9 @@
Removed graphql dependency.

2.0.2
Made robust to missing gemspec ownership.
Made robust to missing gemspec ownership.

2.0.3
Scope libyear-merged dependencies to the analyzed workspace so cross-workspace
deps are no longer uploaded with blank versions (LIBTRACK-136). Distinguish an
undeterminable resolved set (pnpm failure) from an empty one.
55 changes: 41 additions & 14 deletions lib/library_version_analysis/pnpm.rb
Original file line number Diff line number Diff line change
Expand Up @@ -108,9 +108,9 @@ def source_name_for_workspace(workspace_path)

# Analyze a single workspace
def get_versions_for_workspace(workspace_path, source) # rubocop:disable Metrics/AbcSize, Metrics/MethodLength
all_libraries = {}
puts("\tPNPM [#{source}] adding all libraries") if LibraryVersionAnalysis.dev_output?
all_libraries = add_all_libraries(workspace_path)
resolved_libraries = add_all_libraries(workspace_path)
all_libraries = resolved_libraries || {}

puts("\tPNPM [#{source}] running libyear") if LibraryVersionAnalysis.dev_output?

Expand All @@ -120,11 +120,17 @@ def get_versions_for_workspace(workspace_path, source) # rubocop:disable Metrics
exit(-1)
end

# Snapshot the resolved dependency set for THIS workspace BEFORE merging libyear.
# add_all_libraries returns only the analyzed workspace's resolved dependencies, whereas
# libyear (whole-monorepo union) and Dependabot can inject names that belong to other
# workspaces. Filtering back to this set drops that bleed instead of uploading it with a
# blank current_version. (parse_libyear returns all_libraries as the same object, so the
# snapshot must be taken before it adds libyear-only keys.) nil means the resolved set
# could not be determined (pnpm failed) -> the filter is skipped rather than wiping data.
workspace_package_names = resolved_libraries&.keys&.to_set

puts("\tPNPM [#{source}] parsing libyear") if LibraryVersionAnalysis.dev_output?
parsed_results, meta_data = parse_libyear(libyear_results, all_libraries)
# Snapshot before Dependabot: parse_libyear returns all_libraries as parsed_results (same object),
# so any keys Dependabot injects into parsed_results also appear in all_libraries.
workspace_package_names = parsed_results.keys.to_set

puts("\tPNPM [#{source}] dependabot") if LibraryVersionAnalysis.dev_output?
add_dependabot_findings(parsed_results, meta_data, @github_repo, "pnpm")
Expand All @@ -145,9 +151,9 @@ def get_versions_for_workspace(workspace_path, source) # rubocop:disable Metrics
end

def get_versions(source) # rubocop:disable Metrics/AbcSize, Metrics/MethodLength
all_libraries = {}
puts("\tPNPM adding all libraries") if LibraryVersionAnalysis.dev_output?
all_libraries = add_all_libraries
resolved_libraries = add_all_libraries
all_libraries = resolved_libraries || {}

puts("\tPNPM running libyear") if LibraryVersionAnalysis.dev_output?

Expand All @@ -157,11 +163,14 @@ def get_versions(source) # rubocop:disable Metrics/AbcSize, Metrics/MethodLength
exit(-1)
end

# Snapshot the resolved dependency set for this repo BEFORE merging libyear (see the
# workspace variant for the rationale): libyear and Dependabot can inject names beyond the
# resolved set, and parse_libyear mutates all_libraries in place. nil means the resolved
# set could not be determined (pnpm failed) -> skip the filter rather than wipe data.
workspace_package_names = resolved_libraries&.keys&.to_set

puts("\tPNPM parsing libyear") if LibraryVersionAnalysis.dev_output?
parsed_results, meta_data = parse_libyear(libyear_results, all_libraries)
# Snapshot before Dependabot: parse_libyear returns all_libraries as parsed_results (same object),
# so any keys Dependabot injects into parsed_results also appear in all_libraries.
workspace_package_names = parsed_results.keys.to_set

puts("\tPNPM dependabot") if LibraryVersionAnalysis.dev_output?
add_dependabot_findings(parsed_results, meta_data, @github_repo, source)
Expand Down Expand Up @@ -223,11 +232,23 @@ def add_dependency_graph(parsed_results, workspace_path = nil) # rubocop:disable

private

# Restrict parsed_results to the analyzed workspace's resolved dependency set, dropping names
# injected by libyear (whole-monorepo union) or Dependabot that are not actually dependencies
# of this workspace. Those would otherwise upload with a blank current_version.
def filter_to_workspace_packages(parsed_results, workspace_package_names, source)
# nil means the resolved set could not be DETERMINED (pnpm list failed). Skip the filter
# so we don't wipe libyear data. An EMPTY set means the workspace was resolved and has no
# registry-versioned direct deps; filter normally so the whole-monorepo libyear union is
# dropped instead of uploaded with blank versions.
if workspace_package_names.nil?
warn "PNPM [#{source}] could not determine resolved dependencies; skipping workspace-scope filter"
return
end

injected = parsed_results.keys.reject { |name| workspace_package_names.include?(name) }
return if injected.empty?

puts("\tPNPM [#{source}] removing #{injected.count} Dependabot alerts not in this workspace") if LibraryVersionAnalysis.dev_output?
puts("\tPNPM [#{source}] removing #{injected.count} libraries not in this workspace's resolved dependencies") if LibraryVersionAnalysis.dev_output?
injected.each { |name| parsed_results.delete(name) }
end

Expand Down Expand Up @@ -288,21 +309,27 @@ def run_libyear_open3
# rendered tree output: the text rendering depends on the terminal (e.g. the
# `├── ` prefix is absent in non-TTY/CI runs), which previously caused every
# current_version to come back blank.
# Returns a { name => Versionline } hash of the workspace's resolved direct
# dependencies, or nil when the resolved set could not be DETERMINED (pnpm list
# failed, unparseable output, or no matching workspace entry). A workspace that
# genuinely has no resolvable direct deps returns an empty hash, not nil — the
# distinction lets callers drop the libyear union for empty workspaces while not
# wiping data when pnpm itself failed.
def add_all_libraries(workspace_path = nil)
all_libraries = {}

results = run_pnpm_list_depth0
return all_libraries if results.nil?
return nil if results.nil?

begin
json = JSON.parse(results)
rescue JSON::ParserError
return all_libraries
return nil
end

packages = json.is_a?(Array) ? json : [json]
package = select_workspace_package(packages, workspace_path)
return all_libraries if package.nil?
return nil if package.nil?

%w(dependencies devDependencies).each do |group|
(package[group] || {}).each do |name, info|
Expand Down
2 changes: 1 addition & 1 deletion lib/library_version_analysis/version.rb
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
module LibraryVersionAnalysis
VERSION = "2.0.2".freeze
VERSION = "2.0.3".freeze
end
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-06-10
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
## Context

After `3a76c5e`, `Pnpm` resolves current versions per workspace from `pnpm list --depth=0 --json` in `add_all_libraries`, then `parse_libyear` merges libyear latest-version/drift data, then `server_data` uploads `libraries[].version = row.current_version`.

Two facts about the libyear input make `parse_libyear` produce blank-version, foreign library records:

1. **libyear carries no installed version.** libyear 0.8.0's JSON per dependency is `{ dependency, drift, pulse, releases, major, minor, patch, available }` — `available` is the latest version; there is no installed/current field. So `parse_libyear` cannot populate `current_version` and never tries; for any dependency not already in `all_libraries` it inserts `new_version_line("")`.
2. **The "per-workspace" libyear files are the whole-monorepo union.** libyear runs `pnpm list --depth=0 --json` and lodash-`merge`s all project objects, so `libyear_<ws>.txt` for every workspace contains the union (~248) of all workspaces' direct dependencies. `add_all_libraries`, by contrast, correctly resolves only the analyzed workspace's deps.

The mismatch: `parse_libyear` injects ~248 names into every workspace; only the handful that intersect that workspace's resolved set carry a current version, and the rest upload blank.

The existing `filter_to_workspace_packages` does not help, because its "known" snapshot is taken **after** `parse_libyear`:

```ruby
parsed_results, meta_data = parse_libyear(libyear_results, all_libraries)
workspace_package_names = parsed_results.keys.to_set # already includes the libyear union
add_dependabot_findings(...)
filter_to_workspace_packages(parsed_results, workspace_package_names, source) # only strips Dependabot adds
```

## Goals / Non-Goals

**Goals**
- Stop uploading library records for dependencies that are not in the analyzed workspace's resolved dependency set.
- Eliminate blank `current_version` rows for pnpm repos that arise from libyear-only dependencies.
- Preserve per-workspace current-version resolution, libyear enrichment of in-workspace libraries, the `a..b` range form, and vulnerability/dependency-graph handling.

**Non-Goals**
- Changing how libyear files are generated in `jobber-frontend` (the "option B" source fix). Considered and deferred — see Decisions.
- Backfilling current versions for transitive or other-workspace dependencies (they should not be uploaded per-workspace as standalone libraries).
- Cleaning existing rows in Library Tracking (separate `library_tracking`-owned change).
- The equivalent `parse_libyear` pattern in `npm.rb` / `gemfile.rb`.

## Decisions

**1. Scope the upload to the resolved workspace set (option A), in the gem.**
Snapshot `all_libraries.keys` returned by `add_all_libraries` (the authoritative resolved dependency set for the analyzed workspace) **before** calling `parse_libyear`, then filter `parsed_results` to that set after the libyear merge and Dependabot injection. libyear continues to enrich libraries that are in the set; libyear-only names are dropped instead of uploaded blank.

Rationale: the resolved set from `pnpm list --json` is already correct and authoritative per workspace (that is exactly what `3a76c5e` fixed). Filtering to it removes both the blank versions and the cross-workspace mis-attribution in one move, entirely within the gem, with no dependency on libyear's generation behavior.

**2. Implement by extending `filter_to_workspace_packages`, not rewriting `parse_libyear`.**
Move/duplicate the snapshot so it captures the resolved set before `parse_libyear`, and reuse the existing filter to delete out-of-set names. Keeps `parse_libyear` unchanged (it still safely enriches; any entries it adds get filtered out afterward), and unifies libyear-bleed removal with the Dependabot-bleed removal the filter already does.

**2a. Distinguish "could not determine the resolved set" from "resolved but empty".**
`add_all_libraries` returns `nil` when the resolved set cannot be determined (pnpm list failed, unparseable output, or no matching workspace entry) and an (possibly empty) hash on success. The caller snapshots `resolved&.keys&.to_set` (so `nil` propagates) and `filter_to_workspace_packages` skips only when the snapshot is `nil`. An **empty** set still filters — dropping the whole-monorepo libyear union for a workspace that genuinely has no registry-versioned direct deps (e.g. `link:`-only packages like `packages/tsconfig`). Local validation against `jobber-frontend` proved this matters: without the distinction, `apps/harbour`, `packages/tsconfig`, and `packages/graphql-depth-limit-plugin` each re-emitted the full ~215-entry union as blanks; with it, they correctly contribute zero libraries and all 24 workspaces report 0 blank versions.

**3. Option B (per-workspace libyear at the source) rejected for this change.**
Making `libyear_<ws>.txt` genuinely per-workspace would require working around libyear 0.8.0's `pnpm list --json` + `merge(...)` union behavior in `jobber-frontend`, is cross-repo and more invasive, and still would not retroactively fix data. Option A fixes the ticket symptom and the mis-attribution with a smaller, gem-local blast radius. Option B remains a reasonable future data-pipeline cleanup but is not required.

**4. Empty current versions remain valid only for in-workspace deps.**
A dependency that is in the workspace's resolved set but has no semver (e.g. `link:` / `workspace:` — already skipped by `current_version_from_info`) is governed by existing spec behavior. The new constraint only removes records for names absent from the resolved set; it does not change handling of in-workspace deps.

## Risks / Trade-offs

- **Outdated transitive or other-workspace deps are no longer surfaced per workspace.** This is intended: they are not dependencies of the analyzed workspace and were previously uploaded as blank-version noise. Aggregate libyear metrics (`meta_data`) are computed in `parse_libyear` before filtering and are unaffected.
- **Reliance on `add_all_libraries` correctness.** Filtering to the resolved set means a mis-resolution would drop libraries. Mitigated by the `nil` (could-not-determine → skip) vs empty (resolved-but-empty → filter) distinction in Decision 2a: a pnpm failure preserves prior behavior instead of wiping, while a genuinely empty workspace correctly drops the foreign union.
- **A workspace with deps that fail to install locally resolves to empty and contributes zero libraries.** Observed for `apps/harbour` in a partial local worktree (`pnpm list --depth=0 --json` returned no installed deps). This is the safe outcome (zero rows beats blank rows); in CI with a full install the workspace resolves its real deps normally.
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
## Why

The LIBTRACK-136 fix (`3a76c5e`, "Fix blank pnpm current versions…") resolved current versions from structured `pnpm list --json`, so each workspace's own direct/dev dependencies now upload real versions. The reporter confirmed it got "better but not fully fixed": ~89% of uploaded rows for `jobber-frontend` (repository 34) still carry a blank version.

The remaining blanks are a **different defect that the prior fix explicitly scoped out** (its design.md lists "Changing libyear sourcing" as a Non-Goal). `Pnpm#parse_libyear` mints a standalone library record with an empty `current_version` for every dependency in the libyear report that is not in the analyzed workspace's resolved dependency set:

```ruby
vv = all_libraries[line["dependency"]]
if vv.nil?
vv = new_version_line("") # blank current_version
all_libraries[line["dependency"]] = vv
end
```

This is fatal in `jobber-frontend` because the "per-workspace" `libyear_<ws>.txt` files are actually the **merged union of every workspace's direct dependencies**: libyear 0.8.0 runs `pnpm list --depth=0 --json` and lodash-`merge`s all project objects together, and its JSON omits the installed version entirely (only `dependency, drift, pulse, releases, major, minor, patch, available`). So every one of the 24 workspaces is uploaded ~248 foreign packages that do not belong to it, each with a blank version.

Evidence — Jun 10 nightly (post-fix) CSV, repository 34:

- 6,725 rows; **5,975 (89%) blank version**.
- The blank set is a near-constant ~248 names present in **every** workspace.
- Each name carries a real version only in the 1–3 workspaces where it is genuinely a dependency (e.g. `@fullcalendar/core` → versioned in `apps/jobber-online`, blank in 24 others).

## What Changes

- Restrict the libraries carried into the upload to the **analyzed workspace's resolved dependency set**. libyear data SHALL only enrich libraries already resolved from that workspace's `pnpm list --json` output (`add_all_libraries`). libyear-reported dependencies that are absent from the workspace SHALL NOT be added as standalone library records.
- Implementation: snapshot the resolved workspace package names from `add_all_libraries` **before** `parse_libyear` runs, then filter `parsed_results` down to that set — extending the existing `filter_to_workspace_packages` (which today only removes Dependabot-injected names; its snapshot is taken *after* `parse_libyear`, so the libyear bleed is already "known" and survives).
- Preserve everything the prior fix established: per-workspace current-version resolution (`3a76c5e`), libyear latest-version/drift enrichment for in-workspace libraries, the `a..b` multi-version range form, and vulnerability / dependency-graph handling for retained libraries.

## Capabilities

### Modified Capabilities

- `pnpm-version-analysis`: adds the constraint that uploaded libraries are limited to the analyzed workspace's resolved dependencies, so libyear's whole-monorepo report cannot introduce foreign, version-less library records.

## Impact

- **Code**: `lib/library_version_analysis/pnpm.rb` — capture the resolved workspace package set before `parse_libyear`; extend `filter_to_workspace_packages` to drop libyear-only entries. Callers `get_versions` (single-package) and `get_versions_for_workspace` (monorepo).
- **Behavior**: per-workspace uploads carry only that workspace's real dependencies, each with a version. Blank-version rows stop being produced. The ~248-foreign-package-per-workspace mis-attribution is eliminated.
- **Tests**: `spec/pnpm_spec.rb` — cover that libyear-only deps are dropped, and that in-workspace deps appearing in libyear are retained and enriched (current + latest version).
- **Prerequisite — new branch**: this change must be built on a branch cut from latest `origin/master`, which includes fix `3a76c5e` and the OpenSpec scaffolding. The prior task worktree branch (`46292689…`) predates both and has no `openspec/` directory. This proposal was authored on `libtrack-136-scope-libyear-deps-to-workspace` (off `origin/master`).
- **Companion change (separately owned)**: a code fix does not retroactively clean rows already in Library Tracking. Existing blank/foreign rows are addressed by the `library_tracking` change `cleanup-blank-library-versions`.
- **Related, out of scope**: the same blank-minting `parse_libyear` pattern exists in `npm.rb` and `gemfile.rb` (other repos/sources). Not changed here; noted for follow-up.
Loading
Loading