Skip to content

Document parallel metadata fetching across SOUP parsers#323

Merged
ydesgagn merged 1 commit into
masterfrom
update-20260515-111455
May 15, 2026
Merged

Document parallel metadata fetching across SOUP parsers#323
ydesgagn merged 1 commit into
masterfrom
update-20260515-111455

Conversation

@ydesgagn
Copy link
Copy Markdown
Contributor

Summary

Updates docs/architecture.md to reflect the parallelized parser HTTP
fetching introduced in #320. The documentation now describes the shared
HttpClient::THREAD_COUNT constant, the parallel gem dependency added
to each parser, and the parallel metadata fetching algorithm.

Key changes:

  • Document HttpClient::THREAD_COUNT (set to Etc.nprocessors) and the new etc external dependency in the HttpClient section
  • Update Bundler, Gradle, NPM, PIP, SPM, and Yarn parser descriptions to note metadata is fetched in parallel via Parallel.map(..., in_threads: HttpClient::THREAD_COUNT)
  • Add parallel to the External Dependencies list for each affected parser
  • Add a new "Parallel Metadata Fetching Algorithm" section under critical algorithms describing purpose, location, and implementation

Types of changes

  • Bugfix (fixes an issue)
  • New feature (adds functionality)
  • Refactoring (improves code without changing functionality)
  • Breaking change (incompatible changes)
  • Build or security update (updates dependencies, libraries, or security patches)
  • Code style or documentation update (formatting, renaming, or documentation changes)
  • Other (please describe):

Checklist

  • Unit tests added to validate my fix/feature
  • I have manually tested my change
  • I did not add automation test. Why ?: Documentation-only change
  • Database changes requiring migration with downtime or reprocessing of existing data
  • The SOUP file lists the risk Level, requirements and verification reasoning associated with each library
  • readme.md includes sections on introduction, installation, usage, and contributing
  • docs/architecture.md includes sections on the architecture diagram, software units, software of unknown provenance, critical algorithms and risk controls related to PII and security
  • Impact on PII, privacy regulations (CCPA/GDPR/PIPEDA), CIS benchmarks or security (availability/confidentiality/integrity); management must be notified

@ydesgagn ydesgagn requested a review from a team as a code owner May 15, 2026 11:15
@ydesgagn ydesgagn enabled auto-merge (squash) May 15, 2026 11:16
@ydesgagn ydesgagn merged commit a2fb767 into master May 15, 2026
16 checks passed
@ydesgagn ydesgagn deleted the update-20260515-111455 branch May 15, 2026 11:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants