Skip to content

Add validation in hyoka tests for minimum Azure.Identity version detection #312

@KarishmaGhiya

Description

@KarishmaGhiya

Reference: PR #301, Comment

Add a validation point to hyoka testing to check whether LLM-based skills correctly detect and use the required Azure.Identity package version (>= 1.16.0) when the code uses DefaultAzureCredential or DefaultAzureCredential.DefaultEnvironmentVariableName. This will help ensure errors are not introduced for developers using older versions, and that guidance is clear whenever these new features are referenced in code or documentation.

Acceptance criteria:

  • Add a test (or update existing test) in hyoka that asserts the LLM suggests the proper minimum Azure.Identity version based on usage of new features in example code or skill documentation.
  • Test this out with other packages as well if the appropriate version of the package gets generated when a feature is supported from a particular version
  • Make sure the installation works as expected depending on your language
  • Document the validation in hyoka's test plan/readme so the requirement is tracked for new or updated skills.
  • Coordination with reviewers to ensure such checks are enforced for future changes affecting Azure.Identity usage.

Related context:

/cc @KarishmaGhiya @JonathanCrd

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions