Our clients often want us to monitor specific network connections, such as VPN tunnels that need to be online, services that always need to be reachable, or even simply to report on internet connection speeds. To do this, we mostly use our network controller software and default RMM sets. In rare cases, that is not enough, so we’ve developed some monitoring sets for our RMM to help us with this.

To start, we have a RMM monitoring set that uses the Test-Connection cmdlet to ping multiple hosts entered in our RMM system. We define these per client, so their most important resources are checked constantly.

$State = "Healthy"
$IPsToPing = $IPsToPing.split(",")
try{
$ConnectionTest = test-connection $IPsToPing -count 3 -ErrorAction stop -Verbose
}
catch [System.Management.Automation.ActionPreferenceStopException]            
{            
try {            
throw $_.exception            
}                 
catch [System.Net.NetworkInformation.PingException] {            
$state = "$($error[0])"            
}                      
catch {            
$state = "$($error[0])"           
}            
}

$EndResult = $ConnectionTest | measure-Object -Property ResponseTime -Average -sum -Maximum -Minimum

$AVGMS = $EndResult.Average.ToString(00)
$MaxMS = $EndResult.Maximum.ToString(00)
$MinMs = $EndResult.Minimum.ToString(00)

$AVGMS shows the average of MS with 3 pings, $MaxMS shows the highest reached MS during 3 pings, and $MinMs, you’ve guessed it – Shows the fastests pings in the west 😉

Next to ping monitoring we also check the health state of the internal network on Windows Servers. We see when doing take overs of infrastructure that in a lot of situations Network Location Awareness does not function or start correctly.

The network location awareness service actually is the service that tells your OS what network profile it should use, like “public”, “private” or “domain”. Not having it running can cause a myriad of issues such a SSPI issues on SQL servers, Firewalling issues, and much more!

Most of our environments use $Domainname.com or when doing takeovers $domainname.local. The common denominator in this is that the network profile contains a period and from there we compare it to the actual Network Category. If these do not match, we alert and see if we need to recover the Network Location Awareness service.

$NetworkProfile = Get-NetConnectionProfile

if ($NetworkProfile.Name -contains ".")
{ $NLAState = "Domain Authentiction might not work properly."  }else{ $NLAState = "Healthy" }

foreach ($NetProfile in $NetworkProfile | where {$_.Name -match "."}) {
 if ($NetProfile.NetworkCategory -ne "DomainAuthenticated")
 { $DomainState = "Network is not set to DomainAuthenticated."  }else{ $DomainState = "Healthy" }
}

if(!$NLAState){ $NLAState = "healthy }

And that’s the blog for today! enjoy and as always, Happy PowerShelling!

For some clients that still have on-site servers running IIS we have to monitor the SSL status to make sure that things such as the Remote Desktop Gateway, or SSTP VPN are always available. To make sure that we are able to replace SSL certifcates on time we need to know when specific certificates expire.

To do this, we will use the Get-IISSite cmdlet. This cmdlet exposes all of the IIS configuration in an easy to use format. First, we’ll get all sites with bindings like this;

(Get-IISSite).bindings

Now if you run this, you won’t see that much information. To make sure we get a list of all possible items we have a couple of choices, but the simplest is using the select statement:

(Get-IISSite).bindings | select *

Here we’re telling PowerShell we want to select every possible object returned by our command. With this you’ll see we have a lot more information about the bindings. Including that we can filter on the protocol used. Because we only want information about the pages secured with a certificated we can run the same command again, but this time filtering on the protocol “HTTPS”

(Get-IISSite).bindings | where-object {$_.Protocol -eq "https"}

Now that we have the information that we want! our next step is easy; we will find the bound certificate in the local certificate store and check the expiry date. We like to be warned 14 days before expiry, but you can change the days to anything you’d like 🙂

$Days = (Get-Date).AddDays("14")
$CertsBound = (Get-IISSite).bindings | where-object {$_.Protocol -eq "https"}
foreach($Cert in $CertsBound){
$CertFile = Get-ChildItem -path "CERT:LocalMachine\$($Cert.CertificateStoreName)" | Where-Object -Property ThumbPrint -eq $cert.RawAttributes.certificateHash
if($certfile.NotAfter -lt $Days) { $CertState += "$($certfile.FriendlyName) will expire on $($certfile.NotAfter)" }
}

if(!$certState){$CertState = "Healthy"}

And thats it! if the certificate will expire in 14 days, this set will start alerting. Happy PowerShelling!

We all know that bad actors often create accounts for repeat access when they gain access to your network. To make sure that we are aware of these situations we user PowerShell monitoring.

For security measures we monitor 3 types of user creation:

• All created domain users,
• All users added privileged groups(e.g. Domain Admins)
• All created local users.
• All (Temporary) users created that contain specific keywords.

Temporary users Monitoring:

The temporary users monitoring set is only used on active directory users, not local users. as we’re already monitoring all local user creation this would be redudant.

$ArrayOfNames = @("test", "tmp","skykick","mig", "migwiz","temp","-admin","supervisor")

foreach($name in $ArrayOfNames){
$filter =  'Name -like "*'+ $($name) + '*"'
$Users = Get-ADUser -Filter $filter
if($users -ne $null){
foreach($user in $users){
$TemporaryUser += "$($user.name) has been found, created at $($user.whencreated)`n"
}
}
}
if(!$TemporaryUser){$Userlist = "No Temporary Accounts Found" }

Created Domain Users

For domain users, we alert on all users created in the last 24 hours. after 24 hours this monitoring set goes back to a healthy state.

$When = ((Get-Date).AddDays(-1)).Date
$GetUsers = Get-ADUser -Filter {whenCreated -ge $When} -Properties whenCreated

foreach($user in $GetUsers){
$userchanges += "$($user.name) has been created at $($user.whencreated) `n"
}

if($UserChanges -eq $Null) { $UserChanges = "No Changes Detected"}

All created local users

Local users do not have a creation date, that makes it a little more difficult to monitor. To solve this we create a file with the results of get-localuser, We update that file every 24 hours. Then from there on, we compare the file to the most recent results of the output of get-localuser. We are alerted if the compare finds newer content, Meaning a user has been created or removed in the local user database.

$Outputfile = "C:\Windows\temp\LocalUsers.txt"
$Localusers = Get-LocalUser | Select-Object *
Test-Path $Outputfile | %{if($_ -eq $false){new-item $Outputfile} }
$OutPutFileContents = Get-Content $Outputfile | ConvertFrom-Json

If((get-item $Outputfile).LastWriteTime -lt (Get-Date).AddDays((-1))){    $Localusers | ConvertTo-Json |out-file $Outputfile  }

$Compare = Compare-Object -DifferenceObject $OutPutFileContents.name -ReferenceObject $Localusers.name

if(!$Compare) { $Compare = "Healthy"}

Privileged Group Changes

We monitor the privileged groups by using Ashley McGlone’s script to get the changes. You can find that full script here.

Function Get-PrivilegedGroupChanges {            
    Param(            
        $Server = "localhost",            
        $Hour = 24            
    )            
                
        $ProtectedGroups = Get-ADGroup -Filter 'AdminCount -eq 1' -Server $Server            
        $Members = @()            
                
        ForEach ($Group in $ProtectedGroups) {            
            $Members += Get-ADReplicationAttributeMetadata -Server $Server `
                -Object $Group.DistinguishedName -ShowAllLinkedValues |            
             Where-Object {$_.IsLinkValue} |            
             Select-Object @{name='GroupDN';expression={$Group.DistinguishedName}}, `
                @{name='GroupName';expression={$Group.Name}}, *            
        }            
                
        $Members |            
            Where-Object {$_.LastOriginatingChangeTime -gt (Get-Date).AddHours(-1 * $Hour)}            
                
    }     
    
    $ListOfChanges = Get-PrivilegedGroupChanges
    
    if($ListOfChanges -eq $Null) 
    {
    $GroupChanges = "No Changes Detected"
    }
    else {
    foreach($item in $ListOfChanges){
    $GroupChanges += "Group $($ListOfChanges.GroupName) has been changed - $($listofchanges.AttributeValue) has been added or removed"
    }
    }
    
    if(!$groupChanges) { $GroupChanges = "No Changes Detected"}

With the advent of Windows 10 all MSP’s are faced with a new challenge: How do we manage the different Windows 10 Feature versions and how do we make sure we can automatically upgrade our clients to the latest version of the Windows 10 OS? Microsoft has not made feature updating very straightforward, and sometimes the automatic updates error out.

A lot of RMM systems claim they do version upgrades perfectly, unfortunately I have not seen any RMM that gracefully upgrades machines without too many issues and without user intervention. as an MSP its key to automate as much as possible and prevent engineers from having to bother users to perform machine upgrades, so here is PowerShell to the rescue:

Checking the Windows version and comparing it to your standards.

We’ve decided we want all of our users on the same feature level, because of this we’ve created a monitoring set that alerts us when Windows 10 computers have a lower version then one we’ve centrally set. Quite simply our monitoring set only returns the current release ID we pull from the registry:

$ReleaseID = (Get-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion" -Name releaseid).releaseid

Now we alert on this monitoring set, when it is below a expected value. In our case anything lower than 1903 generates an alert. Our system then sees that this alert has been generated and performs a remediation script during the next maintenance cycle we’ve agreed with the client.

Remediation

Remediation for having the wrong feature update is easy. Run the update and done. The problem is that most RMM systems don’t handle this very cleanly. To resolve this we’ve created a PowerShell script that grabs the ISO from a networkshare, copies it to a temporary location, and runs the upgrade from there.

The script also checks if there is enough space on the GPT boot partition, and if not deletes the unnecessary font files.

If the network share is not available, It will download the ISO from a web server you specify, which is great for a mobile workforce. All you have to change in this script is $ISOPath and $Webserver.

$ISOPath = "\\Servername\Netlogon\Windows10.iso"
$WebServer = "http://YOURWEBSERVER/Windows10.iso"
$OSDiskIndex = gwmi -query "Select * from Win32_DiskPartition WHERE Bootable = True" | Select-Object -ExpandProperty DiskIndex
$PartTypeFull = gwmi -query "Select * from Win32_DiskPartition WHERE Index = 0" | Select-Object -ExpandProperty Type
$PartTypeMid = $PartTypeFull.Substring(0,3)
$PartType = Out-String -InputObject $PartTypeMid
if ($PartType -like "*GPT*")
{
write-output ("System has a GPT partition, clearing EFI fonts....");
cmd.exe /c "mountvol b: /s"
Remove-Item b:\efi\Microsoft\Boot\Fonts\*.* -force
cmd.exe /c "mountvol b: /d"
}
if (Test-Path $ISOPath)
{
write-output ("Mounting ISO from: "+$ISOPath);
}
else
{
write-output ("Warn: ISO not found at: "+$ISOPath);
write-output ("Downloading ISO from webserver....");
mkdir c:\temp
$ISOPath = "c:\temp\Windows_10_upgrade.iso";
invoke-webrequest $Webserver -OutFile $ISOPath
}
Mount-DiskImage -ImagePath $ISOPath
$ISODrive = Get-DiskImage -ImagePath $ISOPath | Get-Volume | Select-Object -ExpandProperty DriveLetter
write-output ("Mounted ISO on drive: "+$ISODrive)
$Exe = ":\setup.exe"
$Arguments = "/auto upgrade /quiet /noreboot"
$ExePath = $ISODrive + $Exe
write-output ("Running setup from ISO: " + $ExePath)
Start-Process $ExePath $Arguments

And tada! thats it. The upgrade will run, but the machine will not reboot until the user performs the reboot. We schedule more tasks during our maintenance cycle, so we’d rather have the RMM system handle the reboot.

Happy PowerShelling!