Add "merge --input-file-list NAME ..." as a way to exceed CLI limits

[jimklimov]

...or to handle filenames in paths which have spaces.

Not tested yet, and C# is not my native language, so please bear with me :)
UPDATE: Now tested, as detailed in comments below.

So far, posting to make sure CI passes on all platforms with this trivial change. Not sure how to test this properly (with an additional file to list SBOM files) - suggestions or maintainer commits are welcome :)

[sonatype-lift]

Sonatype Lift is retiring

Sonatype Lift will be retiring on Sep 12, 2023, with its analysis stopping on Aug 12, 2023. We understand that this news may come as a disappointment, and Sonatype is committed to helping you transition off it seamlessly. If you’d like to retain your data, please export your issues from the web console.
We are extremely grateful and thank you for your support over the years.

📖 Read about the impacts and timeline

[jimklimov]

Now actually tested locally, at least. Outputs posted below (made a bit noisier to see what's happening with parameter handling).

• With a CRLF-separated list file:

:; cyclonedx.exe merge --input-files top*json --input-files-list top*txt \
    --output-file myProject_deploystd~da72200.json
Got 1 individual input file name(s): ['topSBOM.myProject_deploystd~da72200.CollectedComponents.json']
Got 1 file(s) with actual input file names: ['topSBOM.myProject_deploystd~da72200.CollectedComponents.list.txt']
Adding to input file list from topSBOM.myProject_deploystd~da72200.CollectedComponents.list.txt
Got 301 entries from topSBOM.myProject_deploystd~da72200.CollectedComponents.list.txt
Processing input file topSBOM.myProject_deploystd~da72200.CollectedComponents.json
    Contains 18 components
Processing input file common-service-discovery-14.0.0-cyclonedx.json
    Contains 12 components
Processing input file db-12.0.0-cyclonedx.json
    Contains 119 components
...

• Similar for a NUL-separated file (as would be provided by find ... -print0):

:; tr -d '\r' < top*txt | tr '\n' '\0' > top*txt0
:; cyclonedx.exe merge --input-files top*json --input-files-nul-list top*txt0 \
    --output-file myProject_deploystd~da72200.json
Got 1 individual input file name(s): ['topSBOM.myProject_deploystd~da72200.CollectedComponents.json']
Got 1 file(s) with NUL-separated actual input file names: ['topSBOM.myProject_deploystd~da72200.CollectedComponents.list.txt0']
Adding to input file list from topSBOM.myProject_deploystd~da72200.CollectedComponents.list.txt0
Got 301 entries from topSBOM.myProject_deploystd~da72200.CollectedComponents.list.txt0
Processing input file topSBOM.myProject_deploystd~da72200.CollectedComponents.json
    Contains 18 components
Processing input file common-service-discovery-14.0.0-cyclonedx.json
    Contains 12 components
Processing input file db-12.0.0-cyclonedx.json
    Contains 119 components
...

• UPDATE: option renamed to --input-files-nul-list to avoid conflicts in C# Options parser

[jimklimov]

Updated report style:

:; cyclonedx.exe merge --input-files top*json --input-files-list top*txt \
    --output-file myProject_deploystd~da72200.json
Got 1 individual input file name(s): ['topSBOM.myProject_deploystd~da72200.CollectedComponents.json']
Processing 1 file(s) with list of actual input file names: ['topSBOM.myProject_deploystd~da72200.CollectedComponents.list.txt']
Adding to input file list from topSBOM.myProject_deploystd~da72200.CollectedComponents.list.txt
Got 295 new entries from topSBOM.myProject_deploystd~da72200.CollectedComponents.list.txt
Determined 296 input files to merge
Processing input file topSBOM.myProject_deploystd~da72200.CollectedComponents.json
    Contains 18 components
...
Writing output file...
    Total 26726 components

[jimklimov]

FYI: This PR is part of a series I've opened 3 years ago, and stalled in review for whatever reasons including the under-the-hood use of reflection to implement inspection of arbitrary BOM entities to merge them, including the ones still open at this time:

• Json/Validator: add findNamedElements() to detect duplicate "bom-ref" definitions cyclonedx-dotnet-library#240
• FlatMerge should deduplicate "equivalent" entries and squash their data cyclonedx-dotnet-library#245
• Bom: initialize its own metadata cyclonedx-dotnet-library#256
• Introduce "cyclonedx rename-entity" command (bis) #396 (revised re-open of Introduce "cyclonedx rename-entity" command #346)
• MergeCommand: use Bom.BomMetadataUpdate()… #340
• MergeCommand: add options to --validate-output(-relaxed) … #335
• MergeCommand.cs: ensure a timestamp and use new cyclonedx-dotnet-library features… #334
• Add "merge --input-file-list NAME ..." as a way to exceed CLI limits #328

I (and my dayjob's budget) have recently employed Claude AI to pick up where I left long ago, and rewrite those changes in idiomatic C#, rebased over current upstream achievements.

After some internal dev-testing shows that this rewrite is successful, I hope these old PRs will be supplanted by a new series with new technological base and same or better feature set as what I was stuck with using (slowly bit-rotting over the years). Hopefully the new set of PRs would be less questionable for an upstream merge :)

I'll add this note to all impacted PRs listed above.

[jimklimov]

Superseded by #512, same feature (--input-files-list/--input-files-nul-list) redone on top of the current merge-strategy stack. Closing this in favor of #512.