Skip to content

Improving metrics/logs when task credentials are expired - #5134

Draft
singholt wants to merge 1 commit into
aws:devfrom
singholt:dev
Draft

Improving metrics/logs when task credentials are expired #5134
singholt wants to merge 1 commit into
aws:devfrom
singholt:dev

Conversation

@singholt

@singholt singholt commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Summary

The IMDS info file reports a Code per credential. Success means the provider assumed the role and wrote a credential file; AssumeRoleUnauthorizedAccess means it was not authorized to assume it. The latter could happen for cases such as customer misconfiguring a role during a task's lifecycle (that uses the role). Or the customer's account is suspended/closed but the task is still running due to delay in resource cleanup.

This PR updates the scanner to parse the Code for each credential and keep a track of roles with AssumeRoleUnauthorizedAccess issue in the credentials manager. It also updates the Docker-based agent in this repository to log this info for debugging, since there is no metrics wired for emission overall for the agent.

Implementation details

Parse the Code so a scan returns unauthorized roles alongside retrieved credentials, and track those roles in the credentials manager keyed by credentials id. An unauthorized role leaves any credentials already held in place, and is cleared when credentials arrive for the id or the id is removed.

An unrecognized Code is logged and its credential fetched anyway, so a usable credential is never discarded.

Testing

New tests cover the changes: yes, added unit and integration tests

Description for the changelog

Enhancement - Track unauthorized roles from IMDS scans (currently disabled in production)

Additional Information

Does this PR include breaking model changes? No.

Does this PR include the addition of new environment variables in the README? No.

Licensing

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

The IMDS info file reports a Code per credential. Success means the
provider assumed the role and wrote a credential file;
AssumeRoleUnauthorizedAccess means it was not authorized to assume it.

Parse the Code so a scan returns unauthorized roles alongside retrieved
credentials, and track those roles in the credentials manager keyed by
credentials id. An unauthorized role leaves any credentials already held
in place, and is cleared when credentials arrive for the id or the id is
removed.

An unrecognized Code is logged and its credential fetched anyway, so a
usable credential is never discarded.
@singholt
singholt changed the base branch from master to dev September 11, 2026 23:36
@singholt singholt changed the title Improving credential expiry metrics Improving metrics/logs when task credentials are expired Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants