Dynamic Service Principal Creation Failures

[Suhas-Umashankar]

Having issues with Dynamic SP creation consistently. Creating new secrets for exisiting service principals works as expected but the dynamic one fails.

Vault Version:

Version  Installation Time     Build Date
-------  -----------------     ----------
1.21.1   2025-12-11T09:32:22Z  2025-11-18T13:04:32Z

Azure plugin version:
azure                                secret      v0.23.0+builtin

Reproduction Steps:

Following guide here

1. Enable Azure secrets engine
2. Configure Azure engine
3. Provide permissions as needed to the SP, mainly: Application.ReadWrite.All and GroupMember.ReadWrite.All. Provide User Access Administrator role on the subscription to allow role assignment access to the SP.
4. Create a dynamic role using:

vault write azure/roles/edu-app ttl=1h azure_roles=-<<EOF
    [
      {
        "role_name": "Contributor",
        "scope": "/subscriptions/$SUBSCRIPTION_ID/resourceGroups/vault-education"
      }
    ]
EOF

5. Read using

vault read azure/creds/edu-app

Output is:

Error reading azure/creds/edu-app: Error making API request.

URL: GET http://127.0.0.1:8200/v1/azure/creds/edu-app
Code: 500. Errors:

* 1 error occurred:
        * error creating service principal: 2 errors occurred:
        * Resource '3e600671-c682-43fd-95a5-85bb804af996' does not exist or one of its queried reference-property objects are not present.
        * Resource '3e600671-c682-43fd-95a5-85bb804af996' does not exist or one of its queried reference-property objects are not present.

Whereas the Service Principal/Enterprise Application actually exists with that ID.

Some logs that might be helpful from azure activity logs:

Event initiated by                 development-vault-azure-engine
Error code                         PrincipalNotFound
Message                            Principal 3e600671c68243fd95a585bb804af996 does not exist in the directory xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx. Check that you have the correct principal ID. If you are creating this principal and then immediately assigning a role, this error might be related to a replication delay. In this case, set the role assignment principalType property to a value, such as ServicePrincipal, User, or Group. See https://aka.ms/docs-principaltype

Might be related to - microsoftgraph/msgraph-bicep-types#193. The solution there is to wait for a bit for replication.

[xjantoth]

Our setup to generate Dynamic credentials in Azure just stopped without any intervention. We have even added the highest credentials rights to secret engine temporary, which worked for about a week and is suddenly stopped. Help needed!

[stano-sk]

After some changes were applied to our Azure tenant by Microsoft and after creating a free Azure tenant for testing, we face the same issue on both of them. We are unable to create a new service principal with Azure roles. It ends up with the same error as described in this issue.

Also need help with that. It is not even working with 1.21.4.
Thank you for your support in advance.

[novotmich]

Bumping up the issue, same experience here. MS do seem to describe it as a default behavior here https://devblogs.microsoft.com/identity/designing-for-eventual-consistency-for-microsoft-entra/

[paulawangai]

Also experiencing the same issue ...

[bateau84]

We are also facing this issue, and is breaking all pipelines in production.

[chrishiner]

As a workaround support had me set persist_app=true on my backend roles.
It took a few tries for it to successfully create the azure app for each one, but then I'm able to get dynamic credentials.
Hopefully they'll get a better fix soon.

[Cleymax]

Same issue on our side. Pipelines broke today.

[bateau84]

Are you using dynamic serviceprincipals @chrishiner ? and how does this affect regressions later?

[chrishiner]

Yes, I'm using dynamic service principals.
With that option set to true it creates the app when the backend role is created.
It then generates new dynamic credentials on the app for each request. Instead of making a new app/SP for each request.

Better than the errors I'm getting on one of our tenants. Our other tenants still work, but I'm expecting that to change.

[vpo-cdc]

Hi, we did not use the persist_app property but directly used an existing service principal to get dynamic credentials. It was working until recently when we suddenly faced "409" errors ("Directory_ConcurrencyViolation"). I suppose the plugin should implement retries with exponential backoff correctly...

[ghudgens]

persist_app doesn't work for us. Trying to write a new role with persist_app=true just sits there for 10-15 seconds before returning the same "does not exist or one of its queried reference-property objects are not present" error.