[Bug] Authentication fails using Azure.Identity 1.13.*

[lenniehwtw-new]

Library version used

= 4.66.1 (that is the dependency from Azure.Identity 1.13.1) But the dependency in 1.13.0 is >= 4.65.0

.NET version

8.0

Scenario

ManagedIdentityClient - managed identity

Is this a new or an existing app?

The app is in production, I haven't upgraded MSAL, but started seeing this issue

Issue description and reproduction steps

We have a .Net app that we run on a VM in an AzureML Workspace.
When we upgraded to Azure.Identity v 1.13.0 or 1.13.1 the authentication failed.
I have raised the issue on their board and they think is is a regression issue over here

Azure.Identity Issue 46932

Relevant code snippets

DefaultAzureCredentialOptions options = new()
 {
     Diagnostics =
     {
         IsLoggingContentEnabled = true,
         IsLoggingEnabled = true,
         LoggedHeaderNames = { "x-ms-request-id" },
         LoggedQueryParameters = { "api-version" },
     },
 };
 var credentials = new DefaultAzureCredential(options);

Expected behavior

Would expect the code that works with Azure.Identity 1.12.1 to continue to work

Identity provider

Microsoft Entra ID (Work and School accounts and Personal Microsoft accounts)

Regression

No response

Solution and workarounds

From the Azure.Identity people:

The problem seems to be that Azure ML Studio uses similar environment variables to other hosting environments such as Azure App Service. In our previous implementation, we attempted to detect Azure App Service's environment variables before CloudShell. Because CloudShell uses a subset of the environment variables, it will be wrongly detected if it is attempted first in an AzureML environment and fails to add the value of the MSI_SECRET env var in a required header named secret.

[rayluo]

@neha-bhargava , fyi, at one point the AzureML was believed to be only needed in Python, so we implemented in MSAL Python here and there. You can use that implementation as a reference, and help cross-check that implementation.

[neha-bhargava]

Azure SDK had implementation for App Service 2017, which read env variables MSI_ENDPOINT and MSI_SECRET. And the complete flow is same. So it worked with Azure SDK. In MSAL we skipped the implementation for App service 2017 as it was retired. We can add support for Azure ML, it is similar to App service just the env variables for endpoint and secret are different.

Thanks @rayluo for pointing out the code in MSAL Python.

Azure SDK implementation of App service 2017 can be found here