Repository navigation
Add azure_aks_workload_security: pod security contexts and network po… - #384
Conversation
…licies from the AKS Kubernetes API Reads each Entra-integrated AKS cluster's Kubernetes API with a token for the AKS server application, using the API server address and CA certificate from listClusterUserCredential; no kubectl and no stored kubeconfig. Pods are grouped into workloads, and each container's effective security context is recorded: non-root, privilege escalation, read-only root filesystem, capabilities, privileged and seccomp profile, with host namespaces and host path volumes. Namespaces carry their Pod Security Admission levels and network policy coverage, and the cluster whether a policy engine enforces those policies. System namespaces and Windows workloads are listed but not judged. A list the API refuses leaves its counts null and fails the run; an API server that does not answer is tried once and reported as unreachable; a stopped cluster reads as running nothing. AzureRestClient gains get_url for a GET outside ARM. A workload running two pod templates, as during a rollout, is represented by its least restricted one. A runAsUser of 0 is root whatever runAsNonRoot says. Windows pods are recognised by the os field or the node selector. No token is sent to an API server address that is not https. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D7SnHUNrjyxciAGKLGQrgq
bruceweygandt
left a comment
There was a problem hiding this comment.
The fetcher is well built. Pagination, the https check before a token is sent, and the null handling in the summary are correct. All checks pass: discovery, contracts, unit tests, ruff and mypy.
There is one must-fix and one should-fix.
1. Must-fix: an API server that does not answer is retried for about 5 minutes
fetchers/azure/_shared/azure_rest.py:150-155, reached from kube_list through get_url (fetchers/azure/aks_workload_security/fetcher.py:381)
Problem. The PR description says that an API server that does not answer is tried once. It is not. requests.ConnectTimeout is a subclass of requests.ConnectionError. _send catches ConnectionError first and retries it. I ran a real AzureRestClient with a session that raises ConnectTimeout. It made 5 attempts with a 60 s timeout each, plus 15 s of backoff. That is about 315 s for one cluster.
Why this is a problem. A cluster with authorized IP ranges that do not include the runner drops the connection. Each attempt then waits for the full timeout. The runner stops a fetcher after 600 s by default (framework/runner/executor.py:70). Two such clusters in one subscription take more than 600 s. The runner then stops the fetcher, and no evidence is written for any cluster, including the clusters that were read.
Fix. Do not retry connection failures for a call to a Kubernetes API server. For example:
- Add a
retry_connect: bool = Trueargument to_send. When it isFalse, raise on the firstConnectionError. - Call it with
retry_connect=Falsefromget_url. - Use a short connect timeout in
get_url, such astimeout=(10, HTTP_TIMEOUT).
Keep the retries for 429 and 5xx responses. The server answered those requests, so a retry can help.
2. Should-fix: the unreachable test does not use the retry loop
tests/test_azure_aks_workload_security.py:303 and :393
Problem. test_an_unreachable_private_api_server_is_tried_once_and_fails_as_unreachable replaces get_url with a fake that raises ConnectionError immediately. The call never goes through AzureRestClient._send.
Why this is a problem. The test passes while the retry problem in item 1 exists. It does not prove what its name says.
Fix. Add a test that uses a real AzureRestClient with a fake session. Make the session raise requests.ConnectTimeout, and patch azure_rest.time.sleep. Assert that the session is called once, and that the failure code is target_unreachable.
Review follow-up: - requests.ConnectTimeout is a ConnectionError, so AzureRestClient._send retried an unreachable Kubernetes API server 5 times at the full 60 s timeout plus 15 s of backoff, about 315 s per cluster; two such clusters passed the runner's 600 s limit and lost every cluster's evidence. _send takes retry_connect (default True); get_url passes False and a 10 s connect timeout. 429 and 5xx answers are still retried. - The unreachable test compared a set of paths, so 5 requests read as one. It now checks the list of requests and that nothing slept. New tests go through a real AzureRestClient with a session that raises ConnectTimeout or ConnectionError, and check a 503 is still retried. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MBcaFn4pMcbUNLVCX6bo1j
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MBcaFn4pMcbUNLVCX6bo1j
|
Both confirmed and fixed. get_url now passes retry_connect=False to _send and uses a 10s connect timeout, so an unreachable API server costs one attempt (about 10s) instead of 5 with 60s timeouts. 429/5xx from a reachable server are still retried, and ARM calls are unchanged. |
bruceweygandt
left a comment
There was a problem hiding this comment.
Both review items are fixed. A Kubernetes API server that does not answer is now tried once with a 10 s connect timeout, and the new tests go through the real AzureRestClient. Approved.
…licies from the AKS Kubernetes API
Reads each Entra-integrated AKS cluster's Kubernetes API with a token for the AKS server application, using the API server address and CA certificate from listClusterUserCredential; no kubectl and no stored kubeconfig. Pods are grouped into workloads, and each container's effective security context is recorded: non-root, privilege escalation, read-only root filesystem, capabilities, privileged and seccomp profile, with host namespaces and host path volumes. Namespaces carry their Pod Security Admission levels and network policy coverage, and the cluster whether a policy engine enforces those policies.
System namespaces and Windows workloads are listed but not judged. A list the API refuses leaves its counts null and fails the run; an API server that does not answer is tried once and reported as unreachable; a stopped cluster reads as running nothing. AzureRestClient gains get_url for a GET outside ARM.
A workload running two pod templates, as during a rollout, is represented by its least restricted one. A runAsUser of 0 is root whatever runAsNonRoot says. Windows pods are recognised by the os field or the node selector. No token is sent to an API server address that is not https.