[Feature]: Build a risk metric

[ReneLombard]

Background and motivation

Potential future possibilities

Risk metrics

When you add a new NuGet package to your software solution, you introduce risk. Ideally, you'd want to reduce that risk as much as possible or at the very least, understand what kind of risk you're introducing.

One way to improve decision-making is to calculate a risk metric for each package and use that as an input before the package is integrated.

But before control measures can be put in place, we first need to determine the risk.

There are multiple layers and facets to this, including areas like threat modeling, which typically happens at a higher level. Below are some practical risk categories to consider at the package level:

Risk Dimensions

Legal Risk

When you introduce a NuGet package, you want to know early in the Software Development Lifecycle (SDLC) whether it’s non-compliant. (This already covered by the tool)

Security Risk

The third-party DLL might contain known security vulnerabilities.
It’s important to note:

• Actively mitigate the risk (Build controls in the system to avoid the risk)
• Not all vulnerabilities are applicable in every context or you might be importing a vulnerable package, but not using the vulnerable functionality. . (false positive)
• Some risks may be acceptable depending on your use case and threat model. (Acceptable state)

Known Issues

There might be unintentional risk introduced by existing issues — for example, problems mentioned in changelogs or tracked in issue trackers. These should be surfaced before use.

Open/Closed Source

Closed-source software introduces risk when you don’t have the right to audit or inspect the source code.

Questions worth asking on closed source:

What is the country of origin?
Do they have a support policy?
Could sensitive data be handled by a third-party vendor?

Open Source

Is the project actively maintained? When was the last release
Is the community responsive?
Is the license permissive?

Minimum Hardware/Software Requirements

Does your operating environment meet the minimum operating requirements of the package?

Software Required for Proper Operation

Are there hidden dependencies — like additional software required for the package to run?
Does it bind your project to specific platforms (e.g., Windows-only)?

Costs

Are there licensing costs, subscriptions, or one-time purchases required?

Version Activity

When was the last release published?
Stale packages might indicate abandonment or slower response to issues.

ECCN

Is there an ECCN (Export Control Classification Number) available?

Risk Assessment Categories for NuGet Packages

So you could potentially build out a model with the following

Compliance Risk

• Legal Risk
  • Licensing compatibility
  • Early SDLC compliance checks
• Export Control (ECCN)
  • Is there an ECCN available?
  • Country of origin considerations
• Cost
  • Licensing costs, subscriptions, or one-time fees
  • Budget and policy alignment

2. Security Risk

• Known Vulnerabilities
  • CVEs in third-party DLLs
  • Mitigation strategies (e.g., build-time controls)
  • Context-based risk evaluation (false positives)
• Source Code Transparency
  • Open Source
    • Actively maintained?
    • License permissiveness
    • Community responsiveness
  • Closed Source
    • Auditability concerns
    • Country of origin
    • Vendor support & data handling policies
• Threat Modeling Inputs
  • How does the package interact with sensitive data?
  • Does it elevate attack surface?

3. Operational Risk

• Known Issues
  • Changelogs and open issue trackers
  • Bug severity and frequency
• Version Activity
  • Last release date
  • Signs of project abandonment
• Hardware/Software Requirements
  • Minimum system requirements met?
  • Dependency on specific OS/platforms (e.g., Windows-only)
• Hidden Dependencies
  • Additional required software or services
  • Lock-in risks
• Support & Maintenance
  • Is the vendor or community responsive?
  • Are there SLAs or formal support channels?

Acceptable Risk Threshold

• Use scoring across the above categories
• Define thresholds per organization/team/project
• Incorporate into SDLC decision-making process

How to actually achieve this:

You could potentially use the same mechanism in visual studio to determine the vulnerabilites

sequenceDiagram
    participant Dev as Developer
    participant CI as CI/CD Pipeline
    participant Parser as PackageParser
    participant Risk as RiskEvaluator
    participant MD as MarkdownParser
    participant VulnDB as VulnerabilityDB
    participant Report as ReportGenerator

    Dev->>CI: Commit with .csproj changes
    CI->>Parser: Parse <PackageReference> from .csproj
    Parser-->>CI: List of NuGet packages

    CI->>Risk: Evaluate packages
    loop For each package
        Risk->>MD: Parse compliance.md
        Risk->>MD: Parse security.md
        Risk->>MD: Parse operational.md
        MD-->>Risk: Return structured risk data

        Risk->>VulnDB: Query CVEs for package version
        VulnDB-->>Risk: Return known vulnerabilities
    end

    Risk->>Risk: Aggregate risk data and compute score
    Risk-->>CI: Return pass/fail or warnings

    CI->>Report: Generate risk report (HTML/JSON/Markdown)
    Report-->>Dev: Attach report to pipeline output

    alt If risk exceeds threshold
        CI-->>Dev: Fail build with reason
    else
        CI-->>Dev: Pass build
    end

Loading

When reading this you might think that this already exists, or atleast partially, if you use Veracode or Sync you generally upload your source code. Which is generally not ideal and they cover different facets to the problem.

Alternative Concerns

No response

Are you willing help with a pull-request?

No