Bug Report: Policy initiative defines unused zoneId for AKS

[juanandmsft]

Describe the bug
The ESLZ policy initiative defines "azureKubernetesManagementPrivateDnsZoneId": "privatelink.{regionName}.azmk8s.io", in the "dnsZoneNames" parameter, but then is not used. AKS PE is kind of special and does not have a groupId that policy can rely on, and managed by cluster creation.

[Springstone]

@juanandmsft what is the workaround for this issue?

[juanandmsft]

@Springstone ,
As far as I know, there is no workaround other than allow workload teams access to the "privatelink.{regionName}.azmk8s.io" zone during AKS deployment, or use set private dns zone at "none" (i.e. az cli "--private-dns-zone none") and add DNS record manually.

The challenge with granting access to "privatelink.{regionName}.azmk8s.io" is that different workload teams may interfere with each other, but fortunately AKS supports subzones, so can use "team1.privatelink.{regionName}.azmk8s.io", "team2.privatelink.{regionName}.azmk8s.io", etc. and get separated RBAC for different teams/workloads.

[davidzenisu]

Hi @juanandmsft!

We're currently investigating a potential workaround because this has been giving us huge headaches when it comes to the policies to ensure centralized private DNS (Deny, DINE) that we have deployed on our platform.
Basically, we are considering instructing workload teams to use AKS subzones as well (similar to your suggestion), but this leaves us with the challenge of manually managing these zones and the individual permissions in the hub.

We considered an alternative approach where the creation of the private DNS (sub)zone is deployed in the spoke of the workload team and then centralize it via policies. This would look something like this:

• Workload team deploys cluster with private DNS subzones (not linked to any VNet since the spoke VNet uses the central private DNS resolver anyway).
• A specific DINE policy creates a VNet link to the hub (where the centralized private DNS resolver is located).

This would require the changes of the following policies:

• Deny-Private-DNS-Zones (updated): Still deny the creation of any private DNS zones with the exception of the pattern "{subzone}.privatelink.{regionName}.azmk8s.io".
• DINE-Private-DNS-VNet-Link (new): Watch for any private DNS zones being deployed with this naming pattern and linking them to the hub VNet.

Would this be a feasible approach or are there some fundamental design flaws present?

PS: Sorry if this is out of scope of this issue but I haven't been able to find any discussion regarding this specific topic elsewhere.

[juanandmsft]

I tried a custom policy in the past using the "classic" DINE policy for PEs relying on groupId and privateLinkServiceId but unfortunately did not work for me.

[Springstone]

@juanandmsft @davidzenisu Sorry for the delayed response. Priorities have changed for everyone :)
I'm not sure if we are in a better position today, as things being discussed are complex to implement for most customers. I'm wondering if we have better options as I understand a lot is changing in the AKS networking space (retirement of kubenet, transition to CNI Overlay, etc).
I'll try look into this futher.