Category Archives: Automation

Monitoring and Documenting with PowerShell: End of year review

Hi! So this is the final post of this year. I’m going to be enjoying some well deserved holidays and spend Christmas with my family. The past year has been pretty cool. I’ve been doing so many cool projects.

I figured I also would list the top blogs of this year by views, and just generally some stuff I’m proud of, so lets get started:

Top blogs

The most viewed blog this year is my Functional PowerShell for MSPs webinar, which is pretty amazing because it was only posted 3 months ago. I still see the views racking up on the Teams Live Event recording and I am going to be giving another (albeit slightly shorter one) the 16th of december.

The runner up in this is the start of the Documenting with PowerShell series. That entire series seems to have been a favorite for most people. The third place is going to the unofficial IT-Glue backup script.

My personal favorite has to be a recent blog; either the Secure Application Model blog or the OneDrive monitoring script which uses user impersonation.

Documenting with PowerShell series

The documenting with PowerShell series has been a hit. I’ve taken a small break from it to reorganise and make it a little more “Eye-candy” focussed as this was the primary question I’ve been getting. I love how some of you have adapted the scripts. Most of them were made as an example so it’s cool to see all different variations of it. I will be continuing this series at the start of next year. If you have any wishes, let me know!

Monitoring with PowerShell series

The monitoring with PowerShell series is still my baby, I love doing it and showing all the different methods of using PowerShell over SNMP, or PowerShell over generic WMI monitoring. Next year I hope to still be posting at least 1 blog a week. I’ve recently been mailed some question that I will be picking up next year too.

Special thanks

This year was great! I especially want to thank my peers in MSP’r’Us that always help me find new ideas. I also want to thank Datto for the way we have been collaborating and adding my blogs and script to their product. It’s been a great adventure.

And that’s it! I wish all my readers an amazing Christmas, and of course a happy new year. As always, Happy PowerShelling!

Monitoring with PowerShell: Monitoring the creation of new teams.

Since my last blog about the Secure App Model I’ve been playing with using it in different cases such as collecting automatic documentation, but also using it as an early warning tool for events our helpdesk might want to monitor.

One of these tasks is monitoring the creation of new Teams. We like keeping our clients environments in check but of course we also love the teams structure that anyone can create a team at any moment to jump right into their work. The issue with this is often one of compliance and security – We need to be aware of creation to apply the correct processes to the Team and maybe assist the end-user in further setup.

To do this I’ve made the following script; The script uses the Graph API to find all current teams. It then alerts on any team that has been created in the last day. You can of course modify this to your own needs by changing the $MonitorDate variable.

All Tenants script:

##############################
$MonitorDate = (get-date).AddDays(-1)
$ApplicationId = 'XXXX-XXXX-XXXX-XXX-XXX'
$ApplicationSecret = 'YourApplicationSecret' | Convertto-SecureString -AsPlainText -Force
$TenantID = 'YourTenantID.Onmicrosoft.com'
$RefreshToken = 'VeryLongRefreshToken'
##############################
$credential = New-Object System.Management.Automation.PSCredential($ApplicationId, $ApplicationSecret)
write-host "Generating access tokens" -ForegroundColor Green
$graphToken = New-PartnerAccessToken -ApplicationId $ApplicationId -Credential $credential -RefreshToken $refreshToken -Scopes 'https://graph.microsoft.com/.default' -ServicePrincipal -Tenant $tenantID 
write-host "Connecting to MSOLService" -ForegroundColor Green
Connect-MsolService -AdGraphAccessToken $aadGraphToken.AccessToken -MsGraphAccessToken $graphToken.AccessToken
write-host "Grabbing client list" -ForegroundColor Green
$customers = Get-MsolPartnerContract -All
write-host "Connecting to clients" -ForegroundColor Green

foreach ($customer in $customers) {
    write-host "Generating token for $($Customer.name)" -ForegroundColor Green
    $graphToken = New-PartnerAccessToken -ApplicationId $ApplicationId -Credential $credential -RefreshToken $refreshToken -Scopes 'https://graph.microsoft.com/.default' -ServicePrincipal -Tenant $customer.TenantID
    $Header = @{
        Authorization = "Bearer $($graphToken.AccessToken)"
    }
    write-host "Grabbing Teams for $($Customer.name)" -ForegroundColor Green
    $GroupUri = "https://graph.microsoft.com/v1.0/Groups?`$top=999"
    $Groups = (Invoke-RestMethod -Uri $GroupUri -Headers $Header -Method Get -ContentType "application/json").value | Where-Object { $_.resourceProvisioningOptions -eq "Team" }
    $NewGroups = foreach ($group in $Groups | Where-Object { [datetime]$_.CreatedDateTime -gt $MonitorDate }) { 
        "$($Group.displayName) has been created on $($group.createdDateTime)"
    
    }
}
if(!$NewGroups){ $NewGroups = "Healthy. No New groups have been created."} 

this script grabs the new-teams for each of the tenants under your administration and alerts into $NewGroups is a group is newer than the date you’ve given.

Single Tenant script:

##############################
$MonitorDate = (get-date).AddDays(-1)
$ApplicationId = 'XXXX-XXXX-XXXX-XXX-XXX'
$ApplicationSecret = 'YourApplicationSecret' | Convertto-SecureString -AsPlainText -Force
$TenantID = 'TenantYouWantTOLogInto.Onmicrost.com'
$RefreshToken = 'VeryLongRefreshToken'
##############################
$credential = New-Object System.Management.Automation.PSCredential($ApplicationId, $ApplicationSecret)
write-host "Generating access tokens" -ForegroundColor Green
$graphToken = New-PartnerAccessToken -ApplicationId $ApplicationId -Credential $credential -RefreshToken $refreshToken -Scopes 'https://graph.microsoft.com/.default' -ServicePrincipal -Tenant $tenantID 
write-host "Connecting to MSOLService" -ForegroundColor Green
Connect-MsolService -AdGraphAccessToken $aadGraphToken.AccessToken -MsGraphAccessToken $graphToken.AccessToken
    $Header = @{
        Authorization = "Bearer $($graphToken.AccessToken)"
    }
    write-host "Grabbing Teams" -ForegroundColor Green
    $GroupUri = "https://graph.microsoft.com/v1.0/Groups?`$top=999"
    $Groups = (Invoke-RestMethod -Uri $GroupUri -Headers $Header -Method Get -ContentType "application/json").value | Where-Object { $_.resourceProvisioningOptions -eq "Team" }
    $NewGroups = foreach ($group in $Groups | Where-Object { [datetime]$_.CreatedDateTime -gt $MonitorDate }) { 
        "$($Group.displayName) has been created on $($group.createdDateTime)"
    
    }
if(!$NewGroups){ $NewGroups = "Healthy. No New groups have been created."}

And that’s it! hope it helps, and as always, Happy PowerShelling.

Connect to Exchange Online automated when MFA is enabled (Using the SecureApp Model)

So in the past months Microsoft has been forcing CSPs and MSPs to use MFA, something I strongly encourage and am glad with. The only issue with this was that Microsoft made this move without accounting for automation and automated jobs that need to run, especially jobs that run unattended and over multiple delegates.

To make sure that we could be able to use this I’ve been advocating and discussing this with Microsoft a lot, including a public part here, but after several months and great help from
Janosch Ulmer at Microsoft I’ve been able to compose a script for everyone to use, to connect to all Microsoft resources using the secure application model, including Exchange Online.

Compiling all of these scripts took me quite some time. If you have questions or issues just let me know!

The Script

Param 
( 
    [Parameter(Mandatory = $false)]
    [switch]$ConfigurePreconsent,
    [Parameter(Mandatory = $true)]
    [string]$DisplayName,
    [Parameter(Mandatory = $false)]
    [string]$TenantId
)

$ErrorActionPreference = "Stop"

# Check if the Azure AD PowerShell module has already been loaded.
if ( ! ( Get-Module AzureAD ) ) {
    # Check if the Azure AD PowerShell module is installed.
    if ( Get-Module -ListAvailable -Name AzureAD ) {
        # The Azure AD PowerShell module is not load and it is installed. This module
        # must be loaded for other operations performed by this script.
        Write-Host -ForegroundColor Green "Loading the Azure AD PowerShell module..."
        Import-Module AzureAD
    } else {
        Install-Module AzureAD
    }
}

try {
    Write-Host -ForegroundColor Green "When prompted please enter the appropriate credentials... Warning: Window might have pop-under in VSCode"

    if([string]::IsNullOrEmpty($TenantId)) {
        Connect-AzureAD | Out-Null

        $TenantId = $(Get-AzureADTenantDetail).ObjectId
    } else {
        Connect-AzureAD -TenantId $TenantId | Out-Null
    }
} catch [Microsoft.Azure.Common.Authentication.AadAuthenticationCanceledException] {
    # The authentication attempt was canceled by the end-user. Execution of the script should be halted.
    Write-Host -ForegroundColor Yellow "The authentication attempt was canceled. Execution of the script will be halted..."
    Exit
} catch {
    # An unexpected error has occurred. The end-user should be notified so that the appropriate action can be taken.
    Write-Error "An unexpected error has occurred. Please review the following error message and try again." `
        "$($Error[0].Exception)"
}

$adAppAccess = [Microsoft.Open.AzureAD.Model.RequiredResourceAccess]@{
    ResourceAppId = "00000002-0000-0000-c000-000000000000";
    ResourceAccess =
    [Microsoft.Open.AzureAD.Model.ResourceAccess]@{
        Id = "5778995a-e1bf-45b8-affa-663a9f3f4d04";
        Type = "Role"},
    [Microsoft.Open.AzureAD.Model.ResourceAccess]@{
        Id = "a42657d6-7f20-40e3-b6f0-cee03008a62a";
        Type = "Scope"},
    [Microsoft.Open.AzureAD.Model.ResourceAccess]@{
        Id = "311a71cc-e848-46a1-bdf8-97ff7156d8e6";
        Type = "Scope"}
}

$graphAppAccess = [Microsoft.Open.AzureAD.Model.RequiredResourceAccess]@{
    ResourceAppId = "00000003-0000-0000-c000-000000000000";
    ResourceAccess =
        [Microsoft.Open.AzureAD.Model.ResourceAccess]@{
            Id = "bf394140-e372-4bf9-a898-299cfc7564e5";
            Type = "Role"},
        [Microsoft.Open.AzureAD.Model.ResourceAccess]@{
            Id = "7ab1d382-f21e-4acd-a863-ba3e13f7da61";
            Type = "Role"}
}

$partnerCenterAppAccess = [Microsoft.Open.AzureAD.Model.RequiredResourceAccess]@{
    ResourceAppId = "fa3d9a0c-3fb0-42cc-9193-47c7ecd2edbd";
    ResourceAccess =
        [Microsoft.Open.AzureAD.Model.ResourceAccess]@{
            Id = "1cebfa2a-fb4d-419e-b5f9-839b4383e05a";
            Type = "Scope"}
}

$SessionInfo = Get-AzureADCurrentSessionInfo

Write-Host -ForegroundColor Green "Creating the Azure AD application and related resources..."

$app = New-AzureADApplication -AvailableToOtherTenants $true -DisplayName $DisplayName -IdentifierUris "https://$($SessionInfo.TenantDomain)/$((New-Guid).ToString())" -RequiredResourceAccess $adAppAccess, $graphAppAccess, $partnerCenterAppAccess -ReplyUrls @("urn:ietf:wg:oauth:2.0:oob","https://localhost","http://localhost","http://localhost:8400")
$password = New-AzureADApplicationPasswordCredential -ObjectId $app.ObjectId
$spn = New-AzureADServicePrincipal -AppId $app.AppId -DisplayName $DisplayName


    $adminAgentsGroup = Get-AzureADGroup -Filter "DisplayName eq 'AdminAgents'"
    Add-AzureADGroupMember -ObjectId $adminAgentsGroup.ObjectId -RefObjectId $spn.ObjectId

write-host "Installing PartnerCenter Module." -ForegroundColor Green
install-module PartnerCenter -Force
write-host "Sleeping for 30 seconds to allow app creation on O365" -foregroundcolor green
start-sleep 30
write-host "Please approve General consent form." -ForegroundColor Green
$PasswordToSecureString = $password.value | ConvertTo-SecureString -asPlainText -Force
$credential = New-Object System.Management.Automation.PSCredential($($app.AppId),$PasswordToSecureString)
$token = New-PartnerAccessToken -ApplicationId "$($app.AppId)" -Scopes 'https://api.partnercenter.microsoft.com/user_impersonation' -ServicePrincipal -Credential $credential -Tenant $($spn.AppOwnerTenantID) -UseAuthorizationCode
write-host "Please approve Exchange consent form." -ForegroundColor Green
$Exchangetoken = New-PartnerAccessToken -ApplicationId 'a0c73c16-a7e3-4564-9a95-2bdf47383716' -Scopes 'https://outlook.office365.com/.default' -Tenant $($spn.AppOwnerTenantID) -UseDeviceAuthentication
write-host "Last initation required: Please browse to https://login.microsoftonline.com/$($spn.AppOwnerTenantID)/adminConsent?client_id=$($app.AppId)"
write-host "Press any key after auth. An error report about incorrect URIs is expected!"
[void][System.Console]::ReadKey($true)
Write-Host "================ Secrets ================"
Write-Host "`$ApplicationId         = $($app.AppId)"
Write-Host "`$ApplicationSecret     = $($password.Value)"
Write-Host "`$TenantID              = $($spn.AppOwnerTenantID)"
write-host "`$RefreshToken          = $($token.refreshtoken)" -ForegroundColor Blue
write-host "`$Exchange RefreshToken = $($ExchangeToken.Refreshtoken)" -ForegroundColor Green
Write-Host "================ Secrets ================"
Write-Host "    SAVE THESE IN A SECURE LOCATION     " 

Update: Please note that you should NOT run this script in the PowerShell ISE as it will not work. Also note that when running the script with an MFA whitelist via the portal, the script fails. You must remove this whitelisting beforehand.

The following script creates a new application, and connects to all resources. At the end you will receive several private keys. Store these in a secure location for future usage such as a Azure Keyvault or IT-Glue. With these keys you can connect to all your delegated resources without MFA. There currently are some slight issues on the Azure side with performing consent, and as thus you’ll have to click a couple of times. Sorry about that 🙂

So, now that you have these keys you can use the following scripts to connect to the correct resources:

MSOL-Module
$ApplicationId         = 'xxxx-xxxx-xxxx-xxxx-xxx'
$ApplicationSecret     = 'YOURSECRET' | Convertto-SecureString -AsPlainText -Force
$TenantID              = 'xxxxxx-xxxx-xxx-xxxx--xxx' 
$RefreshToken          = 'LongResourcetoken'
$ExchangeRefreshToken  = 'LongExchangeToken'
$credential = New-Object System.Management.Automation.PSCredential($ApplicationId, $ApplicationSecret)

$aadGraphToken = New-PartnerAccessToken -ApplicationId $ApplicationId -Credential $credential -RefreshToken $refreshToken -Scopes 'https://graph.windows.net/.default' -ServicePrincipal -Tenant $tenantID 
$graphToken = New-PartnerAccessToken -ApplicationId $ApplicationId -Credential $credential -RefreshToken $refreshToken -Scopes 'https://graph.microsoft.com/.default' -ServicePrincipal -Tenant $tenantID 

Connect-MsolService -AdGraphAccessToken $aadGraphToken.AccessToken -MsGraphAccessToken $graphToken.AccessToken
AzureAD Module
$ApplicationId         = 'xxxx-xxxx-xxxx-xxxx-xxx' 
$ApplicationSecret     = 'YOURSECRET' | Convertto-SecureString -AsPlainText -Force
$TenantID              = 'xxxxxx-xxxx-xxx-xxxx--xxx'
$RefreshToken          = 'LongResourcetoken'
$ExchangeRefreshToken  = 'LongExchangeToken'
$credential = New-Object System.Management.Automation.PSCredential($ApplicationId, $ApplicationSecret)

$aadGraphToken = New-PartnerAccessToken -ApplicationId $ApplicationId -Credential $credential -RefreshToken $refreshToken -Scopes 'https://graph.windows.net/.default' -ServicePrincipal -Tenant $tenantID 
$graphToken = New-PartnerAccessToken -ApplicationId $ApplicationId -Credential $credential -RefreshToken $refreshToken -Scopes 'https://graph.microsoft.com/.default' -ServicePrincipal -Tenant $tenantID 

Connect-AzureAD -AadAccessToken $aadGraphToken.AccessToken -AccountId 'VALIDEMAILADDRESS' -MsAccessToken $graphToken.AccessToken -TenantId $tenantID
Exchange Online

For the Exchange Online module we’ll need to do a little more effort – You will need the tenantid of the client you are connecting too. I’ll show you how to perform a specific action for each client you have delegated access too.

$ApplicationId         = 'xxxx-xxxx-xxx-xxxx-xxxx'
$ApplicationSecret     = 'TheSecretTheSecrey' | Convertto-SecureString -AsPlainText -Force
$TenantID              = 'YourTenantID'
$RefreshToken          = 'RefreshToken'
$ExchangeRefreshToken  = 'ExchangeRefreshToken'
$upn                   = 'UPN-Used-To-Generate-Tokens'
$credential = New-Object System.Management.Automation.PSCredential($ApplicationId, $ApplicationSecret)

$aadGraphToken = New-PartnerAccessToken -ApplicationId $ApplicationId -Credential $credential -RefreshToken $refreshToken -Scopes 'https://graph.windows.net/.default' -ServicePrincipal -Tenant $tenantID 
$graphToken = New-PartnerAccessToken -ApplicationId $ApplicationId -Credential $credential -RefreshToken $refreshToken -Scopes 'https://graph.microsoft.com/.default' -ServicePrincipal -Tenant $tenantID 

Connect-MsolService -AdGraphAccessToken $aadGraphToken.AccessToken -MsGraphAccessToken $graphToken.AccessToken
$customers = Get-MsolPartnerContract -All
foreach($customer in $customers){
    $token = New-PartnerAccessToken -ApplicationId 'a0c73c16-a7e3-4564-9a95-2bdf47383716'-RefreshToken $ExchangeRefreshToken -Scopes 'https://outlook.office365.com/.default' -Tenant $customer.TenantId
    $tokenValue = ConvertTo-SecureString "Bearer $($token.AccessToken)" -AsPlainText -Force
    $credential = New-Object System.Management.Automation.PSCredential($upn, $tokenValue)
    $customerId = $customer.DefaultDomainName
    $session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri "https://ps.outlook.com/powershell-liveid?DelegatedOrg=$($customerId)&BasicAuthToOAuthConversion=true" -Credential $credential -Authentication Basic -AllowRedirection
    Import-PSSession $session
#From here you can enter your own commands
get-mailbox 
#end of commands
Remove-PSSession $session
}

Refreshing the tokens

Use the following script to refresh the tokens

$ApplicationId = 'ApplicationID'
$ApplicationSecret = 'Secret' | Convertto-SecureString -AsPlainText -Force
$TenantID = 'YourTenantID'
$credential = New-Object System.Management.Automation.PSCredential($ApplicationId, $ApplicationSecret)
$token = New-PartnerAccessToken -ApplicationId $ApplicationID -Scopes 'https://api.partnercenter.microsoft.com/user_impersonation' -ServicePrincipal -Credential $credential -Tenant $TenantID -UseAuthorizationCode
$Exchangetoken = New-PartnerAccessToken -ApplicationId 'a0c73c16-a7e3-4564-9a95-2bdf47383716' -Scopes 'https://outlook.office365.com/.default' -Tenant $TenantID -UseDeviceAuthentication
Write-Host "================ Secrets ================"
Write-Host "`$ApplicationId         = $($applicationID)"
Write-Host "`$ApplicationSecret     = $($ApplicationSecret)"
Write-Host "`$TenantID              = $($tenantid)"
write-host "`$RefreshToken          = $($token.refreshtoken)" -ForegroundColor Blue
write-host "`$ExchangeRefreshToken  = $($ExchangeToken.Refreshtoken)" -ForegroundColor Green
Write-Host "================ Secrets ================"
Write-Host "    SAVE THESE IN A SECURE LOCATION     " 

And that’s it! I hope it helps MSPs, CSPs, etc. 🙂 as always, Happy PowerShelling!

Monitoring with PowerShell: Monitoring Office365 admin password changes

So when I was at Dattocon I was approached by an MSP that was using his RMM system to alert on changes of the local admin password, as he wanted to be updated every time a local admin got a new password. He did this by using an older script of mine below.

Monitoring Local Admin Password changes

$LastDay = (Get-Date).addhours(-24)
$AdminGroup = Get-LocalGroupMember -SID "S-1-5-32-544"
foreach($Admin in $AdminGroup){
$ChangedAdmins = get-localuser -sid $admin.sid | Where-Object {$_.PasswordLastSet -gt $LastDay}
}

But he came to me telling me that recently he had a need to start using this to alert on that a password needed to be updated in his documentation system to complete a process, but he was missing this for Office365 environments. I figured I would give him a hand and made the following script

Monitoring Office365 Global Admin Password changes – All tenants

$LastDay = (Get-Date).addhours(-24)
$credential = Get-Credential
Connect-MsolService -Credential $credential
$customers = Get-msolpartnercontract -All
$ChangedUsers = @()
foreach($customer in $customers){
write-host "getting users for $($Customer.Name)" -ForegroundColorGreen
$adminemails = Get-MsolRoleMember -TenantId $customer.tenantid -RoleObjectId(Get-MsolRole-RoleName"CompanyAdministrator").ObjectId
$Users = $adminemails | get-msoluser-TenantId$customer.TenantId
foreach($User in $Users){
if($User.LastPasswordChangeTimestamp -gt $LastDay){$ChangedUsers += "$($User.UserPrincipalName)has changed his password in the last 24 hours.Please update documentation to reflect.`n"}
}
}

 

Monitoring Office365 Global Admin Password Changes – Single tenant

$TenantName = "YourTenantName.onmicrosoft.com"
$LastDay = (Get-Date).addhours(-24)
$credential = Get-Credential
Connect-MsolService -Credential $credential
$Customer=Get-msolpartnercontract -All | Where-Object{$_.DefaultDomainName -eq $TenantName}
$ChangedUsers=@()
write-host"getting users for $($Customer.Name)" -ForegroundColorGreen
$adminemails = Get-MsolRoleMember -TenantId$customer.tenantid -RoleObjectId (Get-MsolRole -RoleName "CompanyAdministrator").ObjectId
$Users= $adminemails | get-msoluser-TenantId $customer.TenantId
foreach($User in $Users){
if($User.LastPasswordChangeTimestamp -gt $LastDay){$ChangedUsers +="$($User.UserPrincipalName) has changed his password in the last 24 hours.Please update documentation to reflect.`n"}
}

 

This script checks if a password has been changed in the last day, and if so alerts on it, notifying you that a global admin password has been updated and needs to be changed in the documentation. You can also use this as a warning system if you do not have anyone that should be changing these passwords.

Anyway, hope it helps, and as always. Happy PowerShelling!

Dattocon!

Hi all,

I’ll be presenting at Dattocon next week, so I will not be able to release the new blogs about monitoring with PowerShell. If you’re coming to Dattocon feel free to join my session. you can find information about the session here. The description is a little bit off as I will be talking mostly about PowerShell and automation at MSPs.

I’ll upload all resources to this blog after, including some documentation, examples to use during scripting, and the slide deck & recording.

The normal blogging schedule will resume directly after the PowerShell for MSP’s webinar. To get tickets for that, click here.

Function: Write-DRRMAlert

New-DattoRMMAlert was shared to me by Stan Lee at Datto, Stan also loves the DRY principle of coding and as such I’m also sharing it with you. You can only alert single line items, and not arrays or multiline contents as Datto does not support this.

function write-DRRMAlert ($message) {
    write-host '<-Start Result->'
    write-host "Alert=$message"
    write-host '<-End Result->'
    }

Function: Write-DRMMDiag

Similar to the top one, but generated by myself is the diagnostics printing module. We can feed this anything from objects, to arrays, to single string items 🙂

function&nbsp;write-DRMMDiag&nbsp;($messages)&nbsp;{
write-host&nbsp; '&lt;-Start&nbsp;Diagnostic-&gt;'
foreach($Message&nbsp;in&nbsp;$Messages){&nbsp;$Message}
write-host&nbsp;'&lt;-End&nbsp;&nbsp;Diagnostic-&gt;'
}

Monitoring with PowerShell: The Windows Firewall

In a lot of situations where we take over server management from clients we often see bad security practices, where the client does not understand the inherent risk and just wants everything to work. Some administrators that don’t know what they are doing often just disable the entire firewall and hope that their application works at that moment. We even see suppliers of large applications such as Microsoft Dynamics and SQL server applications kill the Windows Firewall because of a lack of knowledge.

We try to help these suppliers and administrators setting up correct Windows Firewall rules when we notice this happens, but to make sure that we are able to notice it we need to have monitoring on our servers for when someone disables the firewall. We also have seen bad actors disable the Windows Firewall after penetrating other layers.

To start, we’ll first check if the simplest part of the Windows Firewall is configured correctly: we check if the Firewall profile is enabled

$FirewallProfiles = Get-NetFirewallProfile | Where-Object { $_.Enabled -eq $false}
If(!$FirewallProfiles) { $ProfileStatus = "Healthy"} else { $ProfileStatus = "$($FirewallProfiles.name) Profile is disabled"}

The issue with just monitoring this is pretty obvious: What if someone has the firewall enabled, but changed the configuration to “inbound connections that do not match a rule are allowed”, So for that we’ll add two simple lines:

$FirewallProfiles = Get-NetFirewallProfile | Where-Object { $_.Enabled -eq $false}
If(!$FirewallProfiles) { $ProfileStatus = "Healthy"} else { $ProfileStatus = "$($FirewallProfiles.name) Profile is disabled"}
$FirewallAllowed = Get-NetFirewallProfile | Where-Object { $_.DefaultInboundAction -ne "NotConfigured"}
If(!$FirewallAllowed) { $DefaultAction = "Healthy"} else { $DefaultAction = "$($FirewallAllowed.name) Profile is set to $($FirewallAllowed.DefaultInboundAction) inbound traffic"}

Hope this helps making your environments a little safer, and as always Happy PowerShelling!

Monitoring with PowerShell: Monitoring Office C2R updates

This blog might be a little shorter than normally, I’ve been a bit swamped with work so if you have any questions, let me know!

This time we’re going to monitor the update status of Microsoft Office that’s been installed using C2R. C2R installers do not get updates from the Microsoft Update services and thus RMM systems often can’t update these. Seeing as C2R is now the standard for all Office Installations we’ll need to start monitoring this separately from Windows Updates. We also want all our of clients to be in the same update channel.

$ReportedVersion = Get-ItemPropertyValue -Path "HKLM:\SOFTWARE\Microsoft\Office\ClickToRun\Configuration" -Name "VersionToReport"
$Channel = (Get-ItemPropertyValue -Path "HKLM:\SOFTWARE\Microsoft\Office\ClickToRun\Configuration" -Name "CDNBaseUrl") -split "/" | Select-Object -Last 1

If (!$Channel) { 
    $Channel = "Non-C2R version or No Channel selected."
}
else {
    switch ($Channel) { 
        "492350f6-3a01-4f97-b9c0-c7c6ddf67d60"  { $Channel = 'Current ("Monthly")' }
        "64256afe-f5d9-4f86-8936-8840a6a4f5be"  { $Channel = "Current Preview (`"Monthly Targeted`"/`"Insiders`")" }
        "7ffbc6bf-bc32-4f92-8982-f9dd17fd3114"  { $Channel = "Semi-Annual Enterprise (`"Broad`")" }
        "b8f9b850-328d-4355-9145-c59439a0c4cf"  { $Channel = "Semi-Annual Enterprise Preview (`"Targeted`")" }
        "55336b82-a18d-4dd6-b5f6-9e5095c314a6"  { $Channel = "Monthly Enterprise" }
        "5440fd1f-7ecb-4221-8110-145efaa6372f"  { $Channel = "Beta" }
        "f2e724c1-748f-4b47-8fb8-8e0d210e9208"  { $Channel = "LTSC" }
        "2e148de9-61c8-4051-b103-4af54baffbb4"  { $Channel = "LTSC Preview" }
    }
}

To monitor on the versions we want to support by checking this page by Microsoft. We also monitor the Channel by alerting on anything that is not “Monthly Channel”, as soon as we see an agent that has the incorrect channel we fix it by running the following command

"C:\Program Files\Common Files\Microsoft Shared\ClickToRun\OfficeC2RClient.exe" /changesetting Channel=Monthly

When a client is not up to date, we force the latest update via the following command, this updates the client specifically to the version we want.

 "C:\Program Files\Common Files\Microsoft Shared\ClickToRun\OfficeC2RClient.exe"  /update USER displaylevel=False updatetoversion=16.0.7341.2029

If you want to update to any update that is available, for the channel the installation is in.

 "C:\Program Files\Common Files\Microsoft Shared\ClickToRun\OfficeC2RClient.exe"  /update USER displaylevel=False

And that’s it! you can now use this to update to the latest versions, and monitor the minimum required version you need installed. As always, Happy PowerShelling!

Update: Updated as Microsoft changed the channel names, one of my friends had an updated version of the version table for me so I added that. 🙂

Monitoring with PowerShell: UPS Status (APC, Generic, and Dell)

So we’re using several types of UPS’s at our clients, and sometimes bump into generic USB UPS systems too. To monitor these we use a couple of methods that all have benefits and downsides. Let’s get started.

If a generic USB UPS is installed, Windows Server recognizes this as a Battery Unit. The status is sent to the server by using a generic Windows Driver called “Microsoft Compliant Control Method Battery” which is quite the mouthfull. The good thing is that with this driver we can use a couple of small PowerShell commands to find the exact status of the battery.

USB UPS systems

 $Battery = Get-CimInstance -ClassName win32_battery
Switch ($Battery.Availability) {
    1  { $Availability = "Other" ;break}
   2  { $Availability =  "Not using battery" ;break}
   3  { $Availability = "Running or Full Power";break}
   4  {$Availability =  "Warning" ;break}
   5  { $Availability = "In Test";break}
   6  { $Availability = "Not Applicable";break}
   7  { $Availability = "Power Off";break}
   8  { $Availability = "Off Line";break}
   9  { $Availability = "Off Duty";break}
   10  {$Availability =  "Degraded";break}
   11  {$Availability =  "Not Installed";break}
   12  {$Availability =  "Install Error";break}
   13  { $Availability = "Power Save - Unknown";break}
   14  { $Availability = "Power Save - Low Power Mode" ;break}
   15  { $Availability = "Power Save - Standby";break}
   16  { $Availability = "Power Cycle";break}
   17  { $Availability = "Power Save - Warning";break}
    }

$BatteryStatus = $Battery.Status
$BatteryName = "$($Battery.name)"
$Remaining = $Battery.EstimatedChargeRemaining
$EstRunTimeMinutes = $Battery.EstimatedRunTime
$BatAvailability = $Availability

The script gets the battery status out of WMI, it shows if the machine is running on battery or not, and you can alert on this. We’ve set our systems up to make sure that when the battery status changes from anything but “Not using battery” it alerts, and possibly shuts down the machine.

Another thing to pay attention to is the Battery Status – Most APCs and Dell’s connected to USB even tell the OS if the battery is in a warning state or failed, you should alert on anything but “OK” for the status.

We can’t really monitor network UPS systems with this, as they do not get their data in w32_battery, so we’ll have to use a couple of different solutions for this. I’ll try covering this in a future blog. As always, happy PowerShelling!

Documenting with PowerShell Chapter 4: Network documentation for IT-Glue.

In the last couple of blogs we spoke about how to handle passwords, passwords objects and tagging, and how to start documenting your servers. Today, We’re starting on the network side of documentation. Within IT-Glue it’s important to follow the IT-Glue standards so your documentation works with the relational database it was designed on.

There are a lot of tools that make automatic documation possible but pass around the fact that IT-Glue works best if you develop your documentation based on their best practices. This is why the documentation script we’ll make for networking has those in place: The script tags all available resources it can find, that are related to the network.

We currently tag all devices based on their primary IP in IT-Glue. You will need these filled in correctly to make sure the documentation script picks these up.

The Script

The script creates a flexible asset for you, with the required fields if the asset is not yet available. The flexible Asset will have 6 fields: the subnet, the gateway, the DNS servers, the DHCP servers, and the tagged devices. It also has a single textbox where a copy of the network scan will be added.

The script uses the PSnmap module for the network scan, it will automatically download this from the PSGallery. More info about PSNmap can be found here.

    #####################################################################
    $APIKEy =  "ITGLUEAPIKEYHERE"
    $APIEndpoint = "https://api.eu.itglue.com"
    $orgID = "ORGIDHERE"
    #Tag related devices. this will try to find the devices based on the MAC, Connected to this network, and tag them as related devices.
    $TagRelatedDevices = $true
    $FlexAssetName = "ITGLue AutoDoc - Network overview v2"
    $Description = "a network one-page document that shows the current configuration found."
    #####################################################################
    $ConnectedNetworks = Get-NetIPConfiguration -Detailed | Where-Object {$_.Netadapter.status -eq "up"}

    If(Get-Module -ListAvailable -Name "ITGlueAPI") {Import-module ITGlueAPI} Else { install-module ITGlueAPI -Force; import-module ITGlueAPI}
    If(Get-Module -ListAvailable -Name "PSnmap") {Import-module "PSnmap"} Else { install-module "PSnmap" -Force; import-module "PSnmap"}
        #Settings IT-Glue logon information
        Add-ITGlueBaseURI -base_uri $APIEndpoint
        Add-ITGlueAPIKey $APIKEy
    foreach($Network in $ConnectedNetworks){ 
    $DHCPServer = (Get-CimInstance -ClassName Win32_NetworkAdapterConfiguration | Where-Object { $_.IPAddress -eq $network.IPv4Address}).DHCPServer
    $Subnet = "$($network.IPv4DefaultGateway.nexthop)/$($network.IPv4Address.PrefixLength)"
    $NetWorkScan = Invoke-PSnmap -ComputerName $subnet -Port 80,443,3389,21,22,25,587 -Dns -NoSummary 
    $HTMLFrag = $NetworkScan | Where-Object {$_.Ping -eq $true} | convertto-html -Fragment -PreContent "<h1> Network scan of $($subnet) <br/><table class=`"table table-bordered table-hover`" >" | out-string
    #Tagging devices
    $DeviceAsset = @()
    If($TagRelatedDevices -eq $true){
        Write-Host "Finding all related resources - Matching on IP at local side, Primary IP on IT-Glue side."
        foreach($hostfound in $networkscan | Where-Object { $_.Ping -ne $false}){
        $DeviceAsset +=  (Get-ITGlueConfigurations -page_size "1000" -organization_id $orgID).data | Where-Object {$_.Attributes."Primary-IP" -eq $($hostfound.ComputerName)}
        }
        }
    
    $FlexAssetBody = 
    @{
        type = 'flexible-assets'
        attributes = @{
                name = $FlexAssetName
                traits = @{
                    "subnet-network" = "$Subnet"
                    "subnet-gateway" = $network.IPv4DefaultGateway.nexthop
                    "subnet-dns-servers" = $network.dnsserver.serveraddresses
                    "subnet-dhcp-servers" = $DHCPServer
                    "scan-results" = $HTMLFrag
                    "tagged-devices" = $DeviceAsset.ID
                }
        }
    }
    

    #Checking if the FlexibleAsset exists. If not, create a new one.
    $FilterID = (Get-ITGlueFlexibleAssetTypes -filter_name $FlexAssetName).data
    if(!$FilterID){ 
        $NewFlexAssetData = 
        @{
            type = 'flexible-asset-types'
            attributes = @{
                    name = $FlexAssetName
                    icon = 'sitemap'
                    description = $description
            }
            relationships = @{
                "flexible-asset-fields" = @{
                    data = @(
                        @{
                            type       = "flexible_asset_fields"
                            attributes = @{
                                order           = 1
                                name            = "Subnet Network"
                                kind            = "Text"
                                required        = $true
                                "show-in-list"  = $true
                                "use-for-title" = $true
                            }
                        },
                        @{
                            type       = "flexible_asset_fields"
                            attributes = @{
                                order          = 2
                                name           = "Subnet Gateway"
                                kind           = "Text"
                                required       = $false
                                "show-in-list" = $false
                            }
                        },
                        @{
                            type       = "flexible_asset_fields"
                            attributes = @{
                                order          = 3
                                name           = "Subnet DNS Servers"
                                kind           = "Text"
                                required       = $false
                                "show-in-list" = $false
                            }
                        },
                        @{
                            type       = "flexible_asset_fields"
                            attributes = @{
                                order          = 4
                                name           = "Subnet DHCP Servers"
                                kind           = "Text"
                                required       = $false
                                "show-in-list" = $false
                            }
                        },
                        @{
                            type       = "flexible_asset_fields"
                            attributes = @{
                                order          = 5
                                name           = "Tagged Devices"
                                kind           = "Tag"
                                "tag-type"     = "Configurations"
                                required       = $false
                                "show-in-list" = $false
                            }
                        },
                        @{
                            type       = "flexible_asset_fields"
                            attributes = @{
                                order          = 6
                                name           = "Scan Results"
                                kind           = "Textbox"
                                required       = $false
                                "show-in-list" = $false
                            }
                        }
                    )
                    }
                }
                  
           }
    New-ITGlueFlexibleAssetTypes -Data $NewFlexAssetData 
    $FilterID = (Get-ITGlueFlexibleAssetTypes -filter_name $FlexAssetName).data
    } 
    #Upload data to IT-Glue. We try to match the Server name to current computer name.
    $ExistingFlexAsset = (Get-ITGlueFlexibleAssets -filter_flexible_asset_type_id $Filterid.id -filter_organization_id $orgID).data | Where-Object {$_.attributes.name -eq $Subnet}
    #If the Asset does not exist, we edit the body to be in the form of a new asset, if not, we just upload.
    if(!$ExistingFlexAsset){
    $FlexAssetBody.attributes.add('organization-id', $orgID)
    $FlexAssetBody.attributes.add('flexible-asset-type-id', $FilterID.id)
    Write-Host "Creating new flexible asset"
    New-ITGlueFlexibleAssets -data $FlexAssetBody
    } else {
    Write-Host "Updating Flexible Asset"
    $ExistingFlexAsset = $ExistingFlexAsset[-1]
    Set-ITGlueFlexibleAssets -id $ExistingFlexAsset.id  -data $FlexAssetBody}
    }

And that’s it! this script will document parts of your network for you, and is easy to edit to anything you’d like it to be. and as always, happy PowerShelling!

Monitoring with PowerShell: Monitoring Dell device updates

I’m a big fan of Dell’s Command Update utility. Dell Command update is a program that makes updating Dell based devices super easy, a single utility that you can install on any workstation to update all devices is great. We always deploy Dell Command update with any machine we hand out to clients.

The next issue that occurs is that we need to know if the updates are running well. For this, I’ve made a monitoring set. To make sure that you don’t just monitor without action, we also created a set that automatically remediates.

The monitoring script

The monitoring script downloads a zip file with the Dell Command Update utility. You can create this zip-file yourself by installing Dell Command Update and simply zipping the install location. It then unzips the downloaded file, and runs the DCU-cli with the Report Parameter, I would advise to only run this set on an hourly or even daily schedule, using your RMM system of course.

#Replace the Download URL to where you've uploaded the ZIP file yourself. We will only download this file once. 
$DownloadURL = "https://www.cyberdrain.com/wp-content/uploads/2019/09/DCU.zip"
$DownloadLocation = "$($Env:ProgramFiles)\DCU"
#Script: 
$TestDownloadLocation = Test-Path $DownloadLocation
if(!$TestDownloadLocation){
new-item $DownloadLocation -ItemType Directory -force
Invoke-WebRequest -Uri $DownloadURL -OutFile "$($DownloadLocation)\DCU.zip"
Expand-Archive "$($DownloadLocation)\DCU.zip" -DestinationPath $DownloadLocation -Force
}
#We start DCU with a reporting parameter set. We wait until the report has been generated.
Start-Process "$($DownloadLocation)\DCU-CLI.exe" -ArgumentList "/report `"$($DownloadLocation)\Report.xml`"" -Wait
$XMLReport = get-content "$($DownloadLocation)\Report.xml"

$BIOSUpdates        = ($XMLReport.updates.update | Where-Object {$_.type -eq "BIOS"}).name.Count
$ApplicationUpdates = ($XMLReport.updates.update | Where-Object {$_.type -eq "Application"}).name.Count
$DriverUpdates      = ($XMLReport.updates.update | Where-Object {$_.type -eq "Driver"}).name.Count
$FirmwareUpdates    = ($XMLReport.updates.update | Where-Object {$_.type -eq "Firmware"}).name.Count
$OtherUpdates       = ($XMLReport.updates.update | Where-Object {$_.type -eq "Other"}).name.Count
$PatchUpdates       = ($XMLReport.updates.update | Where-Object {$_.type -eq "Patch"}).name.Count
$UtilityUpdates     = ($XMLReport.updates.update | Where-Object {$_.type -eq "Utility"}).name.Count
$UrgentUpdates      = ($XMLReport.updates.update | Where-Object {$_.Urgency -eq "Urgent"}).name.Count

As this is a number monitor, if something is 0 you are completely up to date, we monitor all type of updates. We also like knowing if an update is urgent, which has a separate category.

Remediation

So remediation can be done quickly, In theory we would only have to run a single command, which is the following script

$DownloadLocation = "$($Env:ProgramFiles)\DCU"
Start-Process "$($DownloadLocation)\DCU-CLI.exe" -Wait

The problem with running this script directly that by default all updates that the DCU finds will be installed, and you cannot set a classification to be excluded. If you would like to exclude specific update types such as BIOS updates or utility software, you’ll have to do this:

  • Open DCU on your administrator workstation
  • click on the cog in the top right corner
  • update filter:, unselect the updates you want to exclude.
  • Export/Import: and export the MySettings.xml file.
  • Add this MySettings.xml file to your self-hosted DCU zip file.

If you’ve done this small list of tasks, then use the following script to install the updates instead:

$DownloadLocation = "$($Env:ProgramFiles)\DCU"
Start-Process "$($DownloadLocation)\DCU-CLI.exe" -ArgumentList "/import /policy `"$($DownloadLocation)\MySettings.xml`"" -Wait
Start-Process "$($DownloadLocation)\DCU-CLI.exe" -Wait

When executing Thunderbolt or BIOS updates. You will also need to suspend Bitlocker. You can use the following script for this. My advice would be to execute the reboot immediately in this case – and only use this if you are certain that the device is in a secure environment during execution.

$DownloadLocation = "$($Env:ProgramFiles)\DCU"
Start-Process "$($DownloadLocation)\DCU-CLI.exe" -ArgumentList "/import /policy `"$($DownloadLocation)\MySettings.xml`"" -Wait
Suspend-BitLocker -MountPoint 'C:' -RebootCount 1
Start-Process "$($DownloadLocation)\DCU-CLI.exe" -Wait

the AMP file can be found here. As always, Happy PowerShelling!

Documenting with PowerShell Chapter 3: Local Administrator Passwords solution

As a good administrator does we always try to change the local administrator password on computers that we hand-out to clients, and disable it, so we only have to enable it when it’s required. Unfortunately changing it is sometimes forgotten during any process. Microsoft makes this easy when implementing LAPS. LAPS is a solution by Microsoft that helps you in randomizing Local Administrator Passwords, unfortunately LAPS relies on a domain environment. With more and more clients going Cloud-only this is not something we can use.

To resolve this issue you can use the script below as a LAPS alternative. Included are two versions as always: one for IT-Glue, and one to modify in any way you see fit.

The script generates a random password via Powershell of 24 alphanumeric characters, sets it for the local administrator called “Administrator”. There is also an option to rename this user to something else, as I strongly advise. We run the script at the end of our installation sequence or at initialisation of our RMM tool, whichever comes first.

IT-Glue version

#####################################################################
$APIKEy = "ITGLUEAPIKEYHERE"
$APIEndpoint = "https://api.eu.itglue.com"
$orgID = "ORGIDHERE"
$ChangeAdminUsername = $true
$NewAdminUsername = "Tits"
#####################################################################
#Grabbing ITGlue Module and installing.
If(Get-Module -ListAvailable -Name "ITGlueAPI") {Import-module ITGlueAPI} Else { install-module ITGlueAPI -Force; import-module ITGlueAPI}
#Settings IT-Glue logon information
Add-ITGlueBaseURI -base_uri $APIEndpoint
Add-ITGlueAPIKey $APIKEy
add-type -AssemblyName System.Web
#This is the process we'll be perfoming to set the admin account.
$LocalAdminPassword = [System.Web.Security.Membership]::GeneratePassword(24,5)
If($ChangeAdminUsername -eq $false) {
Set-LocalUser -name "Administrator" -Password ($LocalAdminPassword | ConvertTo-SecureString -AsPlainText -Force) -PasswordNeverExpires:$true
} else {
$ExistingNewAdmin = get-localuser | Where-Object {$_.Name -eq $NewAdminUsername}
if(!$ExistingNewAdmin){
write-host "Creating new user" -ForegroundColor Yellow
New-LocalUser -Name $NewAdminUsername -Password ($LocalAdminPassword | ConvertTo-SecureString -AsPlainText -Force) -PasswordNeverExpires:$true
Add-LocalGroupMember -Group Administrators -Member $NewAdminUsername
Disable-LocalUser -Name "Administrator"
}
else{
    write-host "Updating admin password" -ForegroundColor Yellow
   set-localuser -name $NewAdminUsername -Password ($LocalAdminPassword | ConvertTo-SecureString -AsPlainText -Force)
}
}
if($ChangeAdminUsername -eq $false ) { $username = "Administrator" } else { $Username = $NewAdminUsername }
#The script uses the following line to find the correct asset by serialnumber, match it, and connect it if found. Don't want it to tag at all? Comment it out by adding #
$TaggedResource = (Get-ITGlueConfigurations -organization_id $orgID -filter_serial_number (get-ciminstance win32_bios).serialnumber).data | Select-Object -Last 1
$PasswordObjectName = "$($Env:COMPUTERNAME) - Local Administrator Account"
$PasswordObject = @{
    type = 'passwords'
    attributes = @{
            name = $PasswordObjectName
            username = $username
            password = $LocalAdminPassword
            notes = "Local Admin Password for $($Env:COMPUTERNAME)"
    }
}
if($TaggedResource){ 
    $Passwordobject.attributes.Add("resource_id",$TaggedResource.Id)
    $Passwordobject.attributes.Add("resource_type","Configuration")
}

#Now we'll check if it already exists, if not. We'll create a new one.
$ExistingPasswordAsset = (Get-ITGluePasswords -filter_organization_id $orgID -filter_name $PasswordObjectName).data
#If the Asset does not exist, we edit the body to be in the form of a new asset, if not, we just upload.
if(!$ExistingPasswordAsset){
Write-Host "Creating new Local Administrator Password" -ForegroundColor yellow
$ITGNewPassword = New-ITGluePasswords -organization_id $orgID -data $PasswordObject
} else {
Write-Host "Updating Local Administrator Password" -ForegroundColor Yellow
$ITGNewPassword = Set-ITGluePasswords -id $ExistingPasswordAsset.id -data $PasswordObject
}

So this uploads the password object and tags it to the correct device. if the device is not found, it still uploads the password but as an untagged password.

Note: updated at 28/02 as matching could fail in some circumstances causing an error when tagging the password.

General version

The generalised version is located below. Please remember my warning as with previous blogs: This script prints the plain-text version of the password to the console. Modify this to upload the script to any general environment but be careful with this, as storing plaintext is just bad practice.

#####################################################################
$ChangeAdminUsername = $false
$NewAdminUsername = "CompanyAdmin"
#####################################################################
add-type -AssemblyName System.Web
#This is the process we'll be perfoming to set the admin account.
$LocalAdminPassword = [System.Web.Security.Membership]::GeneratePassword(24,5)
If($ChangeAdminUsername -eq $false) {
Set-LocalUser -name "Administrator" -Password ($LocalAdminPassword | ConvertTo-SecureString -AsPlainText -Force) -PasswordNeverExpires:$true
} else {
New-LocalUser -Name $NewAdminUsername -Password ($LocalAdminPassword | ConvertTo-SecureString -AsPlainText -Force) -PasswordNeverExpires:$true
Add-LocalGroupMember -Group Administrators -Member $NewAdminUsername
Disable-LocalUser -Name "Administrator"
}
if($ChangeAdminUsername -eq $false ) { $username = "Administrator" } else { $Username = $NewAdminUsername }
write-host "$($Username) now has password $($LocalAdminPassword)"

And that’s it for today! The AMP can be found here, and as always, Happy PowerShelling.

Note on AMP file: It has come to my attention that the IT-Glue API key is too long for the Script Runner in N-Central to handle. The file now has the IT-Glue API integrated in the script itself. Please enter the key there. Remember: Right-click->Save Link as to download the file.

Monitoring with PowerShell: SMART status via CrystalDiskInfo

In a peer-group that I am a member of recently we’ve had a small discussion about monitoring the SMART status of hard drives. We all agreed that the issue with SMART monitoring is that often it is unreliable when using RMM systems. This is due to RMM systems using only the Windows SMART output which lacks some critical values you should monitor. SMART itself could be a pretty decent early warning system when using all values supplied.

To resolve this, I’ve created a set that uses CrystalDiskInfo. A tool made by CrystalMark which presents the values to you in a nice overview. We’ve used this in the past to troubleshoot or check disks for predictive failures manually, but figured we should also try the same automated. This piece of PowerShell makes SMART monitoring more agile and reliable, because we alert on more information than just the predicted failure values.

The script relies on Invoke-expression, and expand-archive, as such at least Windows 8.1 will be required.

The script

As always, the script is self-explanatory. Please upload the zip file to your own web server or location to where the latest version of CrystalDiskInfo is hosted. This also creates a folder in program program files directory and unzips itself there.

#Replace the Download URL to where you've uploaded the ZIP file yourself. We will only download this file once. 
$DownloadURL = "http://rwthaachen.dl.osdn.jp/crystaldiskinfo/71535/CrystalDiskInfo8_3_0.zip"
$DownloadLocation = "$($Env:ProgramFiles)\CrystalDiskInfo"
#Script: 
$TestDownloadLocation = Test-Path $DownloadLocation
if(!$TestDownloadLocation){
new-item $DownloadLocation -ItemType Directory -force
Invoke-WebRequest -Uri $DownloadURL -OutFile "$($DownloadLocation)\CrystalDiskInfo.zip"
Expand-Archive "$($DownloadLocation)\CrystalDiskInfo.zip" -DestinationPath $DownloadLocation -Force
}
#We start CrystalDiskInfo with the COPYEXIT parameter. This just collects the SMART information in DiskInfo.txt
Start-Process "$($Env:ProgramFiles)\CrystalDiskInfo\DiskInfo64.exe" -ArgumentList "/CopyExit" -wait
$DiskInfoRaw  = get-content "$($Env:ProgramFiles)\CrystalDiskInfo\DiskInfo.txt" | select-string "-- S.M.A.R.T. --------------------------------------------------------------" -Context 0,16
$diskinfo = $DiskInfoRaw -split "`n" | select -skip 2 | Out-String | convertfrom-csv -Delimiter " " -Header "NOTUSED1","NOTUSED2","ID","RawValue" | Select-Object ID,RawValue

[int64]$CriticalWarnings = "0x" + ($diskinfo | Where-Object { $_.ID -eq "01"}).rawvalue
[int64]$CompositeTemp = "0x" + ($diskinfo | Where-Object { $_.ID -eq "02"}).rawvalue -273.15
[int64]$AvailableSpare = "0x" +($diskinfo | Where-Object { $_.ID -eq "03"}).rawvalue
[int64]$ControllerBusyTime ="0x" + ($diskinfo | Where-Object { $_.ID -eq "0A"}).rawvalue
[int64]$PowerCycles ="0x" + ($diskinfo | Where-Object { $_.ID -eq "0B"}).rawvalue
[int64]$PowerOnHours = "0x" + ($diskinfo | Where-Object { $_.ID -eq "0C"}).rawvalue
[int64]$UnsafeShutdowns = "0x" +($diskinfo | Where-Object { $_.ID -eq "0D"}).rawvalue
[int64]$IntegrityErrors ="0x" + ($diskinfo | Where-Object { $_.ID -eq "0E"}).rawvalue
[int64]$InformationLogEntries ="0x" + ($diskinfo | Where-Object { $_.ID -eq "0F"}).rawvalue

The output variables will always contain data, this data can be used to threshold against in your RMM system. The thresholds I would use are:

  • $CriticalWarnings = 0
  • $CompositeTemp = 55 (this is 55 degrees celsius)
  • $AvailableSpare = 50 (This means there are 50 reallocation blocks available. This is extremely preventive so you might want to tune it to your personal preference)
  • $ControllerBusyTime = Not monitored, currently only log this for reporting purposes
  • $PowerCycles = Not monitored, currently only log this for reporting purposes
  • $PowerOnHours = 40000 (This is around 5 years of constant runtime.)
  • $UnsafeShutdowns = 365 (I like to know if users are not shutting down their computers normally. This could also point at other software related problems.)
  • $IntegrityErrors = 1 (This is what Windows normally reports on. We want to know as soon as these issues arise)
  • $InformationLogEntries = 1 (How many events have been generated related to disk SMART events)

I hope this helps MSPs that are having issues with SMART monitoring in their RMM systems, anyway – As always, Happy PowerShelling!

Monitoring with PowerShell: Monitoring log on of specific users.

Hi Guys, This’ll be the last blog before I go on holidays, So enjoy it and see you all in two weeks.

This time we’re going to talk about montoring the logon of specific users. We use named accounts for all our engineers and want to alert if another account that is unnamed has been logged onto in an interactive session. To do this, we’ll use the WMI instrumentation Win32_LoggedOnUser.

Just as an extra disclaimer: Please remember that in co-managed or environments that belong to others you won’t be able to always perform all best practices. This script will mostly be used to monitor those messy environments, and give you a little bit more sense of extra security. My personal advice would always be to disable accounts that are no longer allowed to login, use managed service accounts without interactive permissions for services, and delete accounts of ex-employees directly after the leave the company.

The script

Let’s get started on the script. First we’ll have to define which accounts we do not want to have in interactive sessions:

$ForbiddenList = @("Cyberdrain","cyber","migration","administrator","admin","service-QuickBooks*","svc-QB*","ExEmployee1")

So in this list, we name all accounts that are forbidden. You can add any user you would like to this list, We manage a lot of servers, and sometimes after an engineer leaves our company servers are still logged in with the user that we’ve disabled/deleted. With this script we also monitor those situations and log out the deleted user from all servers.

The next step is getting a list of the active users and comparing them:

$ActiveUsers = (Get-CimInstance Win32_LoggedOnUser).antecedent | Select-Object -Unique | Where-Object {$_.name -in $ForbiddenList}

So here we get all users that are currently logged on to the machine, and compare them to our forbidden list, our $activeUsers variable only gets filled if there is a match in the list.

if(!$ActiveUsers){$ActiveUsers = 'false'}

And this last line says that if $ActiveUsers is empty, so no users have been found that are logged in and in our list, it will say “false”. The complete script is just 3 lines and can be found below.

$ForbiddenList = @("Cyberdrain","cyber","migration","administrator","admin","service-QuickBooks*","svc-QB*","ExEmployee1")
$ActiveUsers = (Get-CimInstance Win32_LoggedOnUser).antecedent | Select-Object -Unique | Where-Object {$_.name -in $ForbiddenList}
if(!$ActiveUsers){$ActiveUsers = 'false'}

And that’s it. Some monitoring for situations you do not want to end up in. Remember to always follow security best practices first. Only use these scripts as an early warning system that someone, somewhere has made a mistake. 🙂 As always, Happy PowerShelling!

Documenting with PowerShell: Chapter 2 – Documenting Bitlocker keys

Our RMM system currently does not have support to securely store the bitlocker key inside of the RMM system itself. I’ve subscribed to the school of bitlocking everything that passes through my company, So also computers that sometimes never get connected to Azure AD, Active Directory to store the key in. We also get users that lost the USB drive or piece of paper that the key was stored on.

As we use a documentation system (IT-Glue) to store all our passwords, I figured why not try to also store our Bitlocker keys there, while tagging the device too so we can always find which device belongs to which key easily.

First for the none IT-Glue users I’ll generate a HTML file. With some small adaptation you can upload this to Confluence, ITBoost, or any other system you use. After that example, we’ll get onto IT-Glue again. So let’s get started!

Base script

The base script is the part of the script that captures the data that we want. In our case This will be the Bitlocker key, and output it an HTML file in C:\Temp\Temp.html You can use this script however you’d like.

$BitlockVolumes = Get-BitLockerVolume
#Some HTML to make the page pretty.
$head = @"
<script>
function myFunction() {
    const filter = document.querySelector('#myInput').value.toUpperCase();
    const trs = document.querySelectorAll('table tr:not(.header)');
    trs.forEach(tr => tr.style.display = [...tr.children].find(td => td.innerHTML.toUpperCase().includes(filter)) ? '' : 'none');
  }</script>
<title>Audit Log Report</title>
<style>
body { background-color:#E5E4E2;
      font-family:Monospace;
      font-size:10pt; }
td, th { border:0px solid black; 
        border-collapse:collapse;
        white-space:pre; }
th { color:white;
    background-color:black; }
table, tr, td, th {
     padding: 2px; 
     margin: 0px;
     white-space:pre; }
tr:nth-child(odd) {background-color: lightgray}
table { width:95%;margin-left:5px; margin-bottom:20px; }
h2 {
font-family:Tahoma;
color:#6D7B8D;
}
.footer 
{ color:green; 
 margin-left:10px; 
 font-family:Tahoma;
 font-size:8pt;
 font-style:italic;
}
#myInput {
  background-image: url('https://www.w3schools.com/css/searchicon.png'); /* Add a search icon to input */
  background-position: 10px 12px; /* Position the search icon */
  background-repeat: no-repeat; /* Do not repeat the icon image */
  width: 50%; /* Full-width */
  font-size: 16px; /* Increase font-size */
  padding: 12px 20px 12px 40px; /* Add some padding */
  border: 1px solid #ddd; /* Add a grey border */
  margin-bottom: 12px; /* Add some space below the input */
}
</style>
"@

foreach($BitlockVolume in $BitlockVolumes) {
$HTMLTop = @"
    <h1>Bitlocker Information</h1>
    <b>Computername: </b>$($BitlockVolume.ComputerName)<br>
    <b>Encryption Method:</b>$($BitlockVolume.EncryptionMethod)<br>
    <b>Volume Type:</b>$($BitlockVolume.VolumeType)<br>
    <b>Volume Status:</b>$($BitlockVolume.VolumeStatus)<br>
"@
$HTML += $BitlockVolume.KeyProtector | convertto-html -Head $head -PreContent "$HTMLTop <br> <h1>Keys for $($ENV:COMPUTERNAME) - $($BitlockVolume.Mountpoint)</h1>"
}
$html | Out-File C:\Temp\temp.html

Now, that’s cool. This gives us a good ol’ HTML file. We now have a choice, use the previous script found here and adapt it to upload it to IT-Glue as a Flexible Asset or make the choice to upload it as an embedded password and tag the correct device. That sounds cooler to me!

This script looks for a configuration in your IT-Glue database based on the computer’s serial number. If it finds a match it uploads the bitlocker key as an embedded password, with the name “COMPUTERNAME – DRIVE:” as an example for my computer “DESKTOP-U3984 – C:” – We do this because the hostname might change over time and you’d want the keys to be uploaded separately.

IT-Glue script

#####################################################################
$APIKEy =  "APIKEYHERE"
$APIEndpoint = "https://api.eu.itglue.com"
$orgID = "ORGIDHERE"
#####################################################################
#Grabbing ITGlue Module and installing,etc
If(Get-Module -ListAvailable -Name "ITGlueAPI") {Import-module ITGlueAPI} Else { install-module ITGlueAPI -Force; import-module ITGlueAPI}
#Settings IT-Glue logon information
Add-ITGlueBaseURI -base_uri $APIEndpoint
Add-ITGlueAPIKey $APIKEy
#This is the data we'll be sending to IT-Glue. 
$BitlockVolumes = Get-BitLockerVolume
#The script uses the following line to find the correct asset by serialnumber, match it, and connect it if found. Don't want it to tag at all? Comment it out by adding #
$TaggedResource = (Get-ITGlueConfigurations -organization_id $orgID -filter_serial_number (get-ciminstance win32_bios).serialnumber).data
foreach($BitlockVolume in $BitlockVolumes) {
$PasswordObjectName = "$($Env:COMPUTERNAME) - $($BitlockVolume.MountPoint)"
$PasswordObject = @{
    type = 'passwords'
    attributes = @{
            name = $PasswordObjectName
            password = $BitlockVolume.KeyProtector.recoverypassword[1]
            notes = "Bitlocker key for $($Env:COMPUTERNAME)"

    }
}
if($TaggedResource){ 
    $Passwordobject.attributes.Add("resource_id",$TaggedResource.Id)
    $Passwordobject.attributes.Add("resource_type","Configuration")
}

#Now we'll check if it already exists, if not. We'll create a new one.
$ExistingPasswordAsset = (Get-ITGluePasswords -filter_organization_id $orgID -filter_name $PasswordObjectName).data
#If the Asset does not exist, we edit the body to be in the form of a new asset, if not, we just upload.
if(!$ExistingPasswordAsset){
Write-Host "Creating new Bitlocker Password" -ForegroundColor yellow
$ITGNewPassword = New-ITGluePasswords -organization_id $orgID -data $PasswordObject
} else {
Write-Host "Updating Bitlocker Password" -ForegroundColor Yellow
$ITGNewPassword = Set-ITGluePasswords -id $ExistingPasswordAsset.id -data $PasswordObject
}
}

This script can also be found as an AMP file here, that’s it! as always, happy PowerShelling!

IT-Glue unofficial backup script.

The last couple of weeks I’ve been focused on some API efforts for IT-Glue and was asked by a couple of partners if I couldn’t solve the problem that IT-Glue does not have a backup feature available. This makes it difficult to move away from the product, but also access you data in case of an emergency.

So to make sure that IT-Glue partners get some form of data portability and backups I’ve created the following script. The script connects to the IT-Glue API and creates a HTML and CSV export of all Flexible Assets and Passwords in IT-Glue.

It does not currently backup documents, as those are not exposed by the API. It also does not download attachments included on Flexible Assets. The export always creates 2 files; one HTML file for quick viewing, and a CSV file with all the information included.

I’ve also decided not to directly download all configurations as we store no data in there. If anyone wants to also download the configurations, let me know and I’ll edit the script a little.

Warning: The HTML files contain all your documentation, in plain-text format. Store this in a safe location or adapt the script to upload the data to your Azure Key Vault or secondary password management tool instead. Do not store these in a public location.

All you have to do is change the variables for your environment, APIKey, APIEndpoint, and the Export Directory.

The Unofficial IT-Glue Backup Script.

#####################################################################
#The unofficial Kelvin Tegelaar IT-Glue Backup script. Run this script whenever you want to create a backup of the ITGlue database.
#Creates a file called "password.html" with all passwords in Plain-text. please only store in secure location.
#Creates folders per organisation, and copies flexible assets there as HTML table  &amp; CSV file for data portability.
$APIKEy =  "APIKEYHERE"
$APIEndpoint = "https://api.eu.itglue.com"
$ExportDir = "C:\Hello\ITGBackup"
#####################################################################
Write-Host "Creating backup directory" -ForegroundColor Green
if (!(Test-Path $ExportDir)) { new-item $ExportDir -ItemType Directory }

#Header for HTML files, to make it look pretty.
$head = @"
<script>
function myFunction() {
    const filter = document.querySelector('#myInput').value.toUpperCase();
    const trs = document.querySelectorAll('table tr:not(.header)');
    trs.forEach(tr => tr.style.display = [...tr.children].find(td => td.innerHTML.toUpperCase().includes(filter)) ? '' : 'none');
  }</script>
<title>Audit Log Report</title>
<style>
body { background-color:#E5E4E2;
      font-family:Monospace;
      font-size:10pt; }
td, th { border:0px solid black; 
        border-collapse:collapse;
        white-space:pre; }
th { color:white;
    background-color:black; }
table, tr, td, th {
     padding: 2px; 
     margin: 0px;
     white-space:pre; }
tr:nth-child(odd) {background-color: lightgray}
table { width:95%;margin-left:5px; margin-bottom:20px; }
h2 {
font-family:Tahoma;
color:#6D7B8D;
}
.footer 
{ color:green; 
 margin-left:10px; 
 font-family:Tahoma;
 font-size:8pt;
 font-style:italic;
}
#myInput {
  background-image: url('https://www.w3schools.com/css/searchicon.png'); /* Add a search icon to input */
  background-position: 10px 12px; /* Position the search icon */
  background-repeat: no-repeat; /* Do not repeat the icon image */
  width: 50%; /* Full-width */
  font-size: 16px; /* Increase font-size */
  padding: 12px 20px 12px 40px; /* Add some padding */
  border: 1px solid #ddd; /* Add a grey border */
  margin-bottom: 12px; /* Add some space below the input */
}
</style>
"@

#ITGlue Download starts here
If(Get-Module -ListAvailable -Name "ITGlueAPI") {Import-module ITGlueAPI} Else { install-module ITGlueAPI -Force; import-module ITGlueAPI}
#Settings IT-Glue logon information
Add-ITGlueBaseURI -base_uri $APIEndpoint
Add-ITGlueAPIKey $APIKEy
$i = 0
#grabbing all orgs for later use.
do {
    $orgs += (Get-ITGlueOrganizations -page_size 1000 -page_number $i).data
    $i++
    Write-Host "Retrieved $($orgs.count) Organisations" -ForegroundColor Yellow
}while ($orgs.count % 1000 -eq 0 -and $orgs.count -ne 0)
#Grabbing all passwords.

Write-Host "Creating backup directory per organisation" -ForegroundColor Green
foreach($org in $orgs){
if (!(Test-Path "$($ExportDir)\$($org.attributes.name)")) { 
    $org.attributes.name = $($org.attributes.name).Replace('\W'," ")
    new-item "$($ExportDir)\$($org.attributes.name)" -ItemType Directory | out-null }
}

$i = 0
$Passwords = @()
Write-Host "Getting passwords" -ForegroundColor Green
    do {
        $i++
        $PasswordList += (Get-ITGluePasswords -page_size 1000 -page_number $i).data
        Write-Host "Retrieved $($PasswordList.count) Passwords" -ForegroundColor Yellow
    }while ($PasswordList.count % 1000 -eq 0 -and $PasswordList.count -ne 0)
    Write-Host "Processing Passwords. This might take some time." -ForegroundColor Yellow
foreach($PasswordItem in $passwordlist){
    $Passwords += (Get-ITGluePasswords -show_password $true -id $PasswordItem.id).data
}
Write-Host "Processed Passwords. Moving on." -ForegroundColor Yellow

    Write-Host "Getting Flexible Assets" -ForegroundColor Green
    $FlexAssetTypes = (Get-ITGlueFlexibleAssetTypes -page_size 1000).data
    foreach($FlexAsset in $FlexAssetTypes){
    $i = 0
    do {
        $i++
        Write-Host "Getting FlexibleAssets for $($Flexasset.attributes.name)" -ForegroundColor Yellow
        $FlexibleAssets += (Get-ITGlueFlexibleAssets -filter_flexible_asset_type_id $FlexAsset.id -page_size 1000 -page_number $i).data
        Write-Host "Retrieved $($FlexibleAssets.count) Flexible Assets" -ForegroundColor Yellow
    }while ($FlexibleAssets.count % 1000 -eq 0 -and $FlexibleAssets.count -ne 0)
    }
 
#Generate single HTML file with all passwords.
$Passwords.attributes | select-object 'organization-name',name,username,password,url,created-at,updated-at | convertto-html -head $head -precontent '<h1>Password export from IT-Glue</h1><input type="text" id="myInput" onkeyup="myFunction()" placeholder="Search for content.." title="Type a query">' | out-file "$($ExportDir)\passwords.html"

foreach($FlexibleAsset in $FlexibleAssets.attributes){
$HTMLTop = @"
<h1> Flexible Asset Information </h1>
<b>organization ID: </b>$($FlexibleAsset."organization-id") <br>
<b>organization Name:</b>$($FlexibleAsset."organization-name")<br>
<b>name:</b>$($FlexibleAsset.name)<br>
<b>created-at </b>$($FlexibleAsset."created-at")<br>
<b>updated-at </b>$($FlexibleAsset."updated-at")<br>
<b>resource url: </b>$($FlexibleAsset."resource-url")<br>
<b>flexible-asset-type-id: </b>$($FlexibleAsset."flexible-asset-type-id")<br>
<b>flexible-asset-type-name: </b>$($FlexibleAsset."flexible-asset-type-name")<br>
"@ 

$HTMLTables = $FlexibleAsset.traits | convertto-html -Head $head -PreContent "$HTMLTop <br> <h1> Flexible Asset Traits</h1>"

write-host "Ouputting $outputpath" -ForegroundColor Yellow
$OutputPath = "$($ExportDir)\$($flexibleasset.'organization-name')"
$outputfilename = "$($Flexibleasset.'flexible-asset-type-name') - $($Flexibleasset.name)"
$outputfilename = $outputfilename -replace '[\W]',''
write-host "Ouputting $outputpath\$outputfilename" -ForegroundColor Yellow
$HTMLTables | out-file "$outputpath\$outputfilename HTML.html"
$FlexibleAsset.traits  | export-csv -path "$outputpath\$outputfilename CSV.csv" -NoTypeInformation
}