Monitoring with PowerShell: Monitoring legacy authentication logons

This is going to be the last blog for a couple of weeks, as I’m going to be enjoying some vacation time. I’ll be seeing you all soon! 🙂

So Microsoft has announced a while back that legacy authentication is no longer going to be supported, due to COVID we had some extra time to prepare our clients for this change as its been postponed to July of 2021. We’ve helped all of our clients move to modern authentication last year but I understood there is still a bit of a struggle for other MSPs to achieve this.

With a P1 subscription retrieving the Legacy Authentication reports is very straightforward, but I’ve created these methods for users without P1 in mind. You have to keep in mind that this method is slightly less accurate than using the P1 sign in report.

There’s two scripts in this case; one to detect if Azure Security Defaults are enabled, which disables Legacy Authentication for you. And another to detect if there are apps being used that might use legacy authentication. You’ll still have to do some manual investigation and move users over to newer apps, but it should help you tackle modern auth in no time. 🙂

Detecting if Azure Security Defaults are enabled

Before running this script, you’ll have to add some permissions to your Secure Application Model, do that by performing this:

Go to the Azure Portal.
Click on Azure Active Directory, now click on “App Registrations”.
Find your Secure App Model application. You can search based on the ApplicationID.
Go to “API Permissions” and click Add a permission.
Choose “Microsoft Graph” and “Application permission”.
Search for “Reports” and click on “Policy.read.all”. Click on add permission
Do the same for “Delegate Permissions”.
Finally, click on “Grant Admin Consent for Company Name.

######### Secrets #########
$ApplicationId = 'AppID'
$ApplicationSecret = 'AppSecret' | ConvertTo-SecureString -Force -AsPlainText
$TenantID = 'Your-TenantID'
$RefreshToken = 'VeryLongRefreshToken'
$UPN = "UPN-Used-To-Generate-Tokens"
######### Secrets #########
write-host "Generating token to log into Azure AD. Grabbing all tenants" -ForegroundColor Green
$credential = New-Object System.Management.Automation.PSCredential($ApplicationId, $ApplicationSecret)
$aadGraphToken = New-PartnerAccessToken -ApplicationId $ApplicationId -Credential $credential -RefreshToken $refreshToken -Scopes 'https://graph.windows.net/.default' -ServicePrincipal -Tenant $tenantID
$graphToken = New-PartnerAccessToken -ApplicationId $ApplicationId -Credential $credential -RefreshToken $refreshToken -Scopes 'https://graph.microsoft.com/.default' -ServicePrincipal -Tenant $tenantID
Connect-AzureAD -AadAccessToken $aadGraphToken.AccessToken -AccountId $upn -MsAccessToken $graphToken.AccessToken -TenantId $tenantID | Out-Null
$tenants = Get-AzureAdContract -All:$true
Disconnect-AzureAD
$SecDefaults = foreach ($Tenant in $Tenants) {
    write-host "Processing $($Tenant.DisplayName)"
    $CustGraphToken = New-PartnerAccessToken -ApplicationId $ApplicationId -Credential $credential -RefreshToken $refreshToken -Scopes "https://graph.microsoft.com/.default" -ServicePrincipal -Tenant $tenant.CustomerContextId
    $Header = @{
        Authorization = "Bearer $($CustGraphToken.AccessToken)"
    }
    $Enabled = (Invoke-RestMethod -Uri "https://graph.microsoft.com/beta/policies/identitySecurityDefaultsEnforcementPolicy" -Headers $Header -Method get -ContentType "application/json").IsEnabled
    [PSCustomObject]@{
        TenantName                  = $tenant.DisplayName
        'Security Defaults Enabled' = $enabled
    }
}

$SecDefaults

Detecting Legacy Application Usage

So finding legacy application usage is the biggest step in finding out if users are still using Legacy Authentication, for that use the following script:

######### Secrets #########
$ApplicationId = 'AppID'
$ApplicationSecret = 'AppSecret' | ConvertTo-SecureString -Force -AsPlainText
$TenantID = 'Your-TenantID'
$RefreshToken = 'VeryLongRefreshToken'
$UPN = "UPN-Used-To-Generate-Tokens"
######### Secrets #########
write-host "Generating token to log into Azure AD. Grabbing all tenants" -ForegroundColor Green
$credential = New-Object System.Management.Automation.PSCredential($ApplicationId, $ApplicationSecret)
$aadGraphToken = New-PartnerAccessToken -ApplicationId $ApplicationId -Credential $credential -RefreshToken $refreshToken -Scopes 'https://graph.windows.net/.default' -ServicePrincipal -Tenant $tenantID
$graphToken = New-PartnerAccessToken -ApplicationId $ApplicationId -Credential $credential -RefreshToken $refreshToken -Scopes 'https://graph.microsoft.com/.default' -ServicePrincipal -Tenant $tenantID
Connect-AzureAD -AadAccessToken $aadGraphToken.AccessToken -AccountId $upn -MsAccessToken $graphToken.AccessToken -TenantId $tenantID | Out-Null
$tenants = Get-AzureAdContract -All:$true
Disconnect-AzureAD
$LegacyAuth = foreach ($Tenant in $Tenants) {
    write-host "Processing tenant $($tenant.displayname)"
    $CustGraphToken = New-PartnerAccessToken -ApplicationId $ApplicationId -Credential $credential -RefreshToken $refreshToken -Scopes "https://graph.microsoft.com/.default" -ServicePrincipal -Tenant $tenant.CustomerContextId
    $Header = @{
        Authorization = "Bearer $($CustGraphToken.AccessToken)"
    }
    $VersionReport = (Invoke-RestMethod -Uri "https://graph.microsoft.com/beta/reports/getEmailAppUsageVersionsUserCounts(period='D7')" -Headers $Header -Method get -ContentType "application/json") | ConvertFrom-Csv
    $LegacyClients = if ($versionreport.'Outlook 2007' -or $versionreport.'Outlook 2010' -or $versionreport.'Outlook 2013') {
        $VersionReport
    }
    $AppReports = (Invoke-RestMethod -Uri "https://graph.microsoft.com/beta/reports/getEmailAppUsageAppsUserCounts(period='D7')" -Headers $Header -Method get -ContentType "application/json") | ConvertFrom-Csv

    $LegacyApplications = if ($AppReports.'Other For Mobile' -or $AppReports.'POP3 App' -or $AppReports.'SMTP App' -or $AppReports.'IMAP4 App' -or $AppReports.'Mail For Mac') {
        $AppReports
    }
    $UserDetails = (Invoke-RestMethod -Uri "https://graph.microsoft.com/beta/reports/getEmailAppUsageUserDetail(period='D7')" -Headers $Header -Method get -ContentType "application/json") | ConvertFrom-Csv

    [PSCustomObject]@{
        Tenant        = $tenant.DisplayName
        LegacyClients = $LegacyClients
        LegacyApps    = $LegacyApplications
        UserDetails   = $UserDetails
    }

}

if ($LegacyAuth.LegacyClients -or $LegacyAuth.LegacyApps) {
    write-host "Unhealthy - Clients with legacy authenticaiton or Legacy clients have been detected"
    $LegacyAuth | Where-Object {$_.LegacyClients -ne $null -or $_.LegacyApps -ne $null}
}

And that’s it! It might not be as accurate as the normal P1 report, but it surely helps in capturing Legacy Authentication. As always, Happy PowerShelling!