Category Archives: Series: PowerShell Monitoring

Dattocon!

Hi all,

I’ll be presenting at Dattocon next week, so I will not be able to release the new blogs about monitoring with PowerShell. If you’re coming to Dattocon feel free to join my session. you can find information about the session here. The description is a little bit off as I will be talking mostly about PowerShell and automation at MSPs.

I’ll upload all resources to this blog after, including some documentation, examples to use during scripting, and the slide deck & recording.

The normal blogging schedule will resume directly after the PowerShell for MSP’s webinar. To get tickets for that, click here.

Function: New-DattoRMMAlert

New-DattoRMMAlert was shared to me by Stan Lee at Datto, Stan also loves the DRY principle of coding and as such I’m also sharing it with you. You can only alert single line items, and not arrays or multiline contents as Datto does not support this.

function write-DRRMAlert ($message) {
    write-host '<-Start Result->'
    write-host "Alert=$message"
    write-host '<-End Result->'
    }

Function: New-DattoRMMAlert

Similar to the top one, but generated by myself is the diagnostics printing module. We can feed this anything from objects, to arrays, to single string items ūüôā

 function write-DRMMDiag ($messages) {
    write-host  '&lt;-Start Diagnostic->'
   foreach($Message in $Messages){ $Message}
    write-host '&lt;-End  Diagnostic->'
    }  

Monitoring with PowerShell: Monitoring Office365 Azure AD Sync

We deploy Azure AD Sync for all of our clients that have hybrid environments. Sometimes the Office365 Azure AD Sync might break down, due to the Accidental Deletion Threshold or no longer perform passwords syncs due to other problems. The Azure AD sync client does tend to break from time to time.

To make sure you are alerted when this happens and can jump in on it early, there are a couple of solutions. In the Office365 portal you can easily set up Office365 to send you an email when this happens. I just don’t like receiving emails for critical infrastructure, and our RMM system has the ability to monitor cloud systems.

The scripts below can be used to monitor the Office365 Azure Active Directory Sync for one tenant, or all tenants in one go.

Single tenant script

$TenantName = "ClientDomain.onmicrosoft.com"
$AlertingTime = (Get-Date).AddHours(-24)
$credential = Get-Credential
Connect-MsolService -Credential $credential
$customer = Get-msolpartnercontract | Where-Object {$_.DefaultDomainName -eq $TenantName}

    $DirectorySynchronizationEnabled = (Get-MsolCompanyInformation -TenantId $customer.TenantId).DirectorySynchronizationEnabled
    if ($DirectorySynchronizationEnabled -eq $true) {
        $LastDirSyncTime = (Get-MsolCompanyInformation -TenantId $customer.TenantId).LastDirSyncTime
        $LastPasswordSyncTime = (Get-MsolCompanyInformation -TenantId $customer.TenantId).LastPasswordSyncTime
        If ($LastDirSyncTime -lt $AlertingTime) { $LastDirSync += "Dirsync Failed for $($Customer.DefaultDomainName) - Last Sync time was at $LastDirSyncTime `n" }
        If ($LastPasswordSyncTime -lt $AlertingTime) { $LastPasswordSync += "Password Sync Failed for $($Customer.DefaultDomainName) - Last Sync time was at $LastPasswordSyncTime `n" }
    }

if(!$LastDirSync){ $LastDirSync = "Healthy"}
if(!$LastPasswordSync){ $LastPasswordSync = "Healthy"}

Multiple tenants script

$AlertingTime = (Get-Date).AddHours(-24)
$credential = Get-Credential
Connect-MsolService -Credential $credential
$customers = Get-msolpartnercontract -All
foreach ($customer in $customers) {
    $DirectorySynchronizationEnabled = (Get-MsolCompanyInformation -TenantId $customer.TenantId).DirectorySynchronizationEnabled
    if ($DirectorySynchronizationEnabled -eq $true) {
        $LastDirSyncTime = (Get-MsolCompanyInformation -TenantId $customer.TenantId).LastDirSyncTime
        $LastPasswordSyncTime = (Get-MsolCompanyInformation -TenantId $customer.TenantId).LastPasswordSyncTime
        If ($LastDirSyncTime -lt $AlertingTime) { $LastDirSync += "Dirsync Failed for $($Customer.DefaultDomainName) - Last Sync time was at $LastDirSyncTime `n" }
        If ($LastPasswordSyncTime -lt $AlertingTime) { $LastPasswordSync += "Password Sync Failed for $($Customer.DefaultDomainName) - Last Sync time was at $LastPasswordSyncTime `n" }
    }
}

if(!$LastDirSync){ $LastDirSync = "Healthy"}
if(!$LastPasswordSync){ $LastPasswordSync = "Healthy"} 

And that’s it! With this monitoring set you’ve created a cloud-sided monitoring set that can show you exactly where your Office365 Azure AD Sync fails. As always, Happy PowerShelling.

Monitoring with PowerShell: External port scanning

So I like knowing exactly what ports are open on my clients network, and have the ability to alert on specific ports that are opened. The problem with most port-scan utilities, and the PowerShell Test-netconnection cmdlet is that they always scan the internal network. In the case that you do enter the external IP whitelisting might allow you to connect anyway and give you some false positives.

To resolve this I’ve created a php page to be used in conjunction with a PowerShell script. The reason I’ve created the page is that I do not like relying on external web based API IP scans. Also I don’t want to be stuck in any subscription model for something as simple as a port scan. With this method you are also completely in control of the source.

So let’s get started! First off you’ll have to upload the following file as “scan.php” to any PHP host. You can browse to the page and it should show you some JSON information regarding the scan it performs on your IP. Scan.php is based on this Github script.

[
<?php
$host = $_SERVER['REMOTE_ADDR'];
$ports = array(21, 25,80,3389,1234,3333,3389,33890,3380);
foreach ($ports as $port)
{
    $connection = @fsockopen($host, $port, $errno, $errstr, 2);
    if (is_resource($connection))
    {
        echo '{' . '"Port":' . $port . ',' . '"status" : "open"' . "},";
        fclose($connection);
    }
    else
    {
        echo '{' . '"Port":' . $port . ', "status" : "closed"},';
    }
}
?>
{ "result": "done" }
]

I’ve converted the original github page to return only JSON. The good thing is that we can use the Invoke-restmethod cmdlet straight away, without having to convert anything, The PowerShell script can be edited to alert only on specific ports that are opened, or on all open ports.

$Results = invoke-restmethod -uri "http://YOURWEBHOST.COM/ip/scan.php"
$OpenPorts = $Results | Where-Object { $_.status -eq "open"}
$ClosedPorts = $Results | Where-Object { $_.status -eq "closed"}

if(!$OpenPorts) {
$PortScanResult = "Healthy"
} else {
$PortScanResult = $OpenPorts
}

And that’s it! as always Happy PowerShelling!

Monitoring with PowerShell: Monitoring Cipher suites (And get a SSLLabs A rank)

I always like getting the maximum achievable rank on websites such as SSLLabs, or the Microsoft Secure Score, because I know I’ve done all that a manufacturer says I need to do to protect their product. The SSL cipher suites are one of these things.

You can run the following script on both Windows Servers that are running IIS to achieve a SSLLabs A rank, but also you can run this script on client machines to increase the security so they will not use older ciphers when requested.

The monitoring script

Monitoring the cipher suites is fairly straightforward. First we’ll check if TLS1.0 and TLS1.1 are disabled and if TLS1.2 is enabled, After that, we check if old know “bad” ciphers are no longer used.

$SChannel&nbsp;=&nbsp;"HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols"
$TLS1ServerEnabled&nbsp;=&nbsp;Get-ItemProperty&nbsp;-Path&nbsp;"$($SChannel)\TLS&nbsp;1.0\Server"&nbsp;-Name&nbsp;Enabled&nbsp;-ErrorAction&nbsp;SilentlyContinue
$TLS11ServerEnabled&nbsp;=&nbsp;Get-ItemProperty&nbsp;-Path&nbsp;"$($SChannel)\TLS&nbsp;1.1\Server"&nbsp;-Name&nbsp;Enabled&nbsp;-ErrorAction&nbsp;SilentlyContinue
$TLS11ClientEnabled&nbsp;=&nbsp;get-ItemProperty&nbsp;-Path&nbsp;"$($SChannel)\TLS&nbsp;1.1\Client"&nbsp;-Name&nbsp;Enabled&nbsp;-ErrorAction&nbsp;SilentlyContinue
$TLS12ServerEnabled&nbsp;=&nbsp;get-ItemProperty&nbsp;-Path&nbsp;"$($SChannel)\TLS&nbsp;1.2\Server"&nbsp;-Name&nbsp;Enabled&nbsp;-ErrorAction&nbsp;SilentlyContinue
$TLS12ClientEnabled&nbsp;=&nbsp;get-ItemProperty&nbsp;-Path&nbsp;"$($SChannel)\TLS&nbsp;1.2\Client"&nbsp;-Name&nbsp;Enabled&nbsp;-ErrorAction&nbsp;SilentlyContinue

If(!$TLS1ServerEnabled&nbsp;-or&nbsp;$TLS1ServerEnabled&nbsp;-eq&nbsp;1)&nbsp;{&nbsp;$TLS1ServerEnabled&nbsp;=&nbsp;"TLS&nbsp;Server&nbsp;1.0&nbsp;could&nbsp;be&nbsp;enabled.&nbsp;Please&nbsp;investigate"&nbsp;&nbsp;}&nbsp;else&nbsp;{&nbsp;$TLS1ServerEnabled&nbsp;=&nbsp;"Healthy"&nbsp;}
If(!$TLS11ServerEnabled&nbsp;-or&nbsp;$TLS11ServerEnabled&nbsp;-eq&nbsp;1)&nbsp;{&nbsp;$TLS11ServerEnabled&nbsp;=&nbsp;"TLS&nbsp;Server&nbsp;1.1&nbsp;could&nbsp;be&nbsp;enabled.&nbsp;Please&nbsp;investigate"&nbsp;&nbsp;}&nbsp;&nbsp;else&nbsp;{&nbsp;$TLS11ServerEnabled&nbsp;=&nbsp;"Healthy"&nbsp;}
If(!$TLS11ClientEnabled&nbsp;-or&nbsp;$TLS11ClientEnabled&nbsp;-eq&nbsp;1)&nbsp;{&nbsp;$TLS11ClientEnabled&nbsp;=&nbsp;"TLS&nbsp;Client&nbsp;1.1&nbsp;could&nbsp;be&nbsp;enabled..&nbsp;Please&nbsp;investigate"&nbsp;&nbsp;}&nbsp;&nbsp;else&nbsp;{&nbsp;$TLS11ClientEnabled&nbsp;=&nbsp;"Healthy"&nbsp;}
If(!$TLS12ServerEnabled&nbsp;-or&nbsp;$TLS12ServerEnabled&nbsp;-eq&nbsp;0)&nbsp;{&nbsp;$TLS12ServerEnabled&nbsp;=&nbsp;"TLS&nbsp;Server&nbsp;1.2&nbsp;could&nbsp;be&nbsp;disabled.&nbsp;Please&nbsp;investigate"&nbsp;&nbsp;}&nbsp;&nbsp;else&nbsp;{&nbsp;$TLS12ServerEnabled&nbsp;=&nbsp;"Healthy"&nbsp;}
If(!$TLS12ClientEnabled&nbsp;-or&nbsp;$TLS12ClientEnabled&nbsp;-eq&nbsp;0)&nbsp;{&nbsp;$TLS12ClientEnabled&nbsp;=&nbsp;"TLS&nbsp;Client&nbsp;1.2&nbsp;could&nbsp;be&nbsp;disabled&nbsp;Please&nbsp;investigate"&nbsp;&nbsp;}&nbsp;&nbsp;else&nbsp;{&nbsp;$TLS12ClientEnabled&nbsp;=&nbsp;"Healthy"&nbsp;}

$OldCipherSuites&nbsp;=
@(
&nbsp;&nbsp;&nbsp;&nbsp;"TLS_DHE_RSA_WITH_AES_256_CBC_SHA"
&nbsp;&nbsp;&nbsp;&nbsp;"TLS_DHE_RSA_WITH_AES_128_CBC_SHA"
&nbsp;&nbsp;&nbsp;&nbsp;"TLS_RSA_WITH_AES_256_GCM_SHA384"
&nbsp;&nbsp;&nbsp;&nbsp;"TLS_RSA_WITH_AES_128_GCM_SHA256"
&nbsp;&nbsp;&nbsp;&nbsp;"TLS_RSA_WITH_AES_256_CBC_SHA256"
&nbsp;&nbsp;&nbsp;&nbsp;"TLS_RSA_WITH_AES_128_CBC_SHA256"
&nbsp;&nbsp;&nbsp;&nbsp;"TLS_RSA_WITH_AES_256_CBC_SHA"
&nbsp;&nbsp;&nbsp;&nbsp;"TLS_RSA_WITH_AES_128_CBC_SHA"
&nbsp;&nbsp;&nbsp;&nbsp;"TLS_RSA_WITH_3DES_EDE_CBC_SHA"
&nbsp;&nbsp;&nbsp;&nbsp;"TLS_DHE_DSS_WITH_AES_256_CBC_SHA256"
&nbsp;&nbsp;&nbsp;&nbsp;"TLS_DHE_DSS_WITH_AES_128_CBC_SHA256"
&nbsp;&nbsp;&nbsp;&nbsp;"TLS_DHE_DSS_WITH_AES_256_CBC_SHA"
&nbsp;&nbsp;&nbsp;&nbsp;"TLS_DHE_DSS_WITH_AES_128_CBC_SHA"
&nbsp;&nbsp;&nbsp;&nbsp;"TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA"
&nbsp;&nbsp;&nbsp;&nbsp;"TLS_RSA_WITH_RC4_128_SHA"
&nbsp;&nbsp;&nbsp;&nbsp;"TLS_RSA_WITH_RC4_128_MD5"
&nbsp;&nbsp;&nbsp;&nbsp;"TLS_RSA_WITH_NULL_SHA256"
&nbsp;&nbsp;&nbsp;&nbsp;"TLS_RSA_WITH_NULL_SHA"
&nbsp;&nbsp;&nbsp;&nbsp;"TLS_PSK_WITH_AES_256_GCM_SHA384"
&nbsp;&nbsp;&nbsp;&nbsp;"TLS_PSK_WITH_AES_128_GCM_SHA256"
&nbsp;&nbsp;&nbsp;&nbsp;"TLS_PSK_WITH_AES_256_CBC_SHA384"
&nbsp;&nbsp;&nbsp;&nbsp;"TLS_PSK_WITH_AES_128_CBC_SHA256"
&nbsp;&nbsp;&nbsp;&nbsp;"TLS_PSK_WITH_NULL_SHA384"
&nbsp;&nbsp;&nbsp;&nbsp;"TLS_PSK_WITH_NULL_SHA256"
)
$SuitesEnabled&nbsp;=&nbsp;@()
foreach($Suite&nbsp;in&nbsp;$OldCipherSuites){
$SuitesEnabled&nbsp;+=&nbsp;get-TlsCipherSuite&nbsp;-name&nbsp;$Suite
}
if(!$SuitesEnabled){
$SuitesEnabled&nbsp;=&nbsp;"Healthy.&nbsp;No&nbsp;old&nbsp;cipher&nbsp;suites&nbsp;found"
}&nbsp;else&nbsp;{
$SuitesEnabled&nbsp;=&nbsp;"Possible&nbsp;old&nbsp;cipher&nbsp;suites&nbsp;found"
} 

After you run this script, you can alert on the contents of $SuitesEnabled to see if old cipher suites are enabled. You also should alert on the content of the following five variables to make sure that you have them all in a “Healthy” state

$TLS1ServerEnabled
$TLS11ServerEnabled
$TLS11ClientEnabled 
$TLS12ServerEnabled
$TLS12ClientEnabled

The remediation script

the remediation is actually very similar to the script above, but we change to create the registry keys this time, and to disable the cipher suites using disable-Tlsciphersuite.

$SChannel = "HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols"
New-Item "$($SChannel)\TLS 1.2\Server" -Force
New-Item "$($SChannel)\TLS 1.2\Client" -Force
New-Item $SChannel -Name "TLS 1.0"
New-Item "$($SChannel)\TLS 1.0" -Name Server
New-Item "$($SChannel)\TLS 1.1\Server" ‚Äďforce
New-Item "$($SChannel)\TLS 1.1\Client" ‚Äďforce
New-ItemProperty -Path "$($SChannel)\TLS 1.0\Server" -Name Enabled -Value 0 -PropertyType DWORD
New-ItemProperty -Path "$($SChannel)\TLS 1.1\Server" -Name Enabled -Value 0 -PropertyType DWORD
New-ItemProperty -Path "$($SChannel)\TLS 1.1\Server" -Name DisabledByDefault -Value 0 -PropertyType DWORD
New-ItemProperty -Path "$($SChannel)\TLS 1.1\Client" -Name Enabled -Value 0 -PropertyType DWORD
New-ItemProperty -Path "$($SChannel)\TLS 1.1\Client" -Name DisabledByDefault -Value 0 -PropertyType DWORD
New-ItemProperty -Path "$($SChannel)\TLS 1.2\Server" -Name Enabled -Value 1 -PropertyType DWORD
New-ItemProperty -Path "$($SChannel)\TLS 1.2\Server" -Name DisabledByDefault -Value 0 -PropertyType DWORD
New-ItemProperty -Path "$($SChannel)\TLS 1.2\Client" -Name Enabled -Value 1 -PropertyType DWORD
New-ItemProperty -Path "$($SChannel)\TLS 1.2\Client" -Name DisabledByDefault -Value 0 -PropertyType DWORD
$OldCipherSuites =
@(
    "TLS_DHE_RSA_WITH_AES_256_CBC_SHA"
    "TLS_DHE_RSA_WITH_AES_128_CBC_SHA"
    "TLS_RSA_WITH_AES_256_GCM_SHA384"
    "TLS_RSA_WITH_AES_128_GCM_SHA256"
    "TLS_RSA_WITH_AES_256_CBC_SHA256"
    "TLS_RSA_WITH_AES_128_CBC_SHA256"
    "TLS_RSA_WITH_AES_256_CBC_SHA"
    "TLS_RSA_WITH_AES_128_CBC_SHA"
    "TLS_RSA_WITH_3DES_EDE_CBC_SHA"
    "TLS_DHE_DSS_WITH_AES_256_CBC_SHA256"
    "TLS_DHE_DSS_WITH_AES_128_CBC_SHA256"
    "TLS_DHE_DSS_WITH_AES_256_CBC_SHA"
    "TLS_DHE_DSS_WITH_AES_128_CBC_SHA"
    "TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA"
    "TLS_RSA_WITH_RC4_128_SHA"
    "TLS_RSA_WITH_RC4_128_MD5"
    "TLS_RSA_WITH_NULL_SHA256"
    "TLS_RSA_WITH_NULL_SHA"
    "TLS_PSK_WITH_AES_256_GCM_SHA384"
    "TLS_PSK_WITH_AES_128_GCM_SHA256"
    "TLS_PSK_WITH_AES_256_CBC_SHA384"
    "TLS_PSK_WITH_AES_128_CBC_SHA256"
    "TLS_PSK_WITH_NULL_SHA384"
    "TLS_PSK_WITH_NULL_SHA256"
)
foreach($Suite in $OldCipherSuites){
disable-TlsCipherSuite -name $Suite -ErrorAction SilentlyContinue
}

And that’s it! I hope you’ve enjoyed and as always, Happy PowerShelling

Monitoring with PowerShell: The Windows Firewall

In a lot of situations where we take over server management from clients we often see bad security practices, where the client does not understand the inherent risk and just wants everything to work. Some administrators that don’t know what they are doing often just disable the entire firewall and hope that their application works at that moment. We even see suppliers of large applications such as Microsoft Dynamics and SQL server applications kill the Windows Firewall because of a lack of knowledge.

We try to help these suppliers and administrators setting up correct Windows Firewall rules when we notice this happens, but to make sure that we are able to notice it we need to have monitoring on our servers for when someone disables the firewall. We also have seen bad actors disable the Windows Firewall after penetrating other layers.

To start, we’ll first check if the simplest part of the Windows Firewall is configured correctly: we check if the Firewall profile is enabled

$FirewallProfiles = Get-NetFirewallProfile | Where-Object { $_.Enabled -eq $false}
If(!$FirewallProfiles) { $ProfileStatus = "Healthy"} else { $ProfileStatus = "$($FirewallProfiles.name) Profile is disabled"}

The issue with just monitoring this is pretty obvious: What if someone has the firewall enabled, but changed the configuration to “inbound connections that do not match a rule are allowed”, So for that we’ll add two simple lines:

$FirewallProfiles = Get-NetFirewallProfile | Where-Object { $_.Enabled -eq $false}
If(!$FirewallProfiles) { $ProfileStatus = "Healthy"} else { $ProfileStatus = "$($FirewallProfiles.name) Profile is disabled"}
$FirewallAllowed = Get-NetFirewallProfile | Where-Object { $_.DefaultInboundAction -ne "NotConfigured"}
If(!$FirewallAllowed) { $DefaultAction = "Healthy"} else { $DefaultAction = "$($FirewallAllowed.name) Profile is set to $($FirewallAllowed.DefaultInboundAction) inbound traffic"}

Hope this helps making your environments a little safer, and as always Happy PowerShelling!

Functional PowerShell for MSPs (Beginner course)

Hi guys,

I’m organising another PowerShell event. Joining the event can be done here. It’ll be a webinar about PowerShell.

The session is mostly oriented for beginners, We’ll have a public Q&A and everyone will be able to enter content during the presentation if you have questions about specific scripts or other issues.

The session will not focus on the theoretical parts of PowerShell. This will be a completely functional session in which you’ll pick up the following:

  1. Configuring your IDE(5-10 minutes.)
  2. Gathering information you want using PowerShell
  3. Finding the correct module for your job.
  4. Passing information to different systems(RMM, Documentation, etc)
  5. Q&A

I hope you’ll find the time to join me! Happy PowerShelling.

Monitoring with PowerShell: Monitoring Office C2R updates

This blog might be a little shorter than normally, I’ve been a bit swamped with work so if you have any questions, let me know!

This time we’re going to monitor the update status of Microsoft Office that’s been installed using C2R. C2R installers do not get updates from the Microsoft Update services and thus RMM systems often can’t update these. Seeing as C2R is now the standard for all Office Installations we’ll need to start monitoring this separately from Windows Updates. We also want all our of clients to be in the same update channel.

$ReportedVersion = Get-ItemPropertyValue -Path "HKLM:\SOFTWARE\Microsoft\Office\ClickToRun\Configuration" -Name "VersionToReport"
$Channel = Get-ItemPropertyValue -Path "HKLM:\SOFTWARE\Microsoft\Office\ClickToRun\Configuration" -Name "CDNBaseUrl"

If(!$Channel) { 
    $Channel = "Non-C2R version or No Channel selected."
} else {
    switch ($Channel) { 
        "http://officecdn.microsoft.com/pr/492350f6-3a01-4f97-b9c0-c7c6ddf67d60" {$Channel = "Monthly Channel"} 
        "http://officecdn.microsoft.com/pr/64256afe-f5d9-4f86-8936-8840a6a4f5be" {$Channel = "Insider / Monthly Channel (Targeted)"} 
        "http://officecdn.microsoft.com/pr/7ffbc6bf-bc32-4f92-8982-f9dd17fd3114" {$Channel = "Semi-Annual Channel"} 
        "http://officecdn.microsoft.com/pr/b8f9b850-328d-4355-9145-c59439a0c4cf" {$Channel = "Semi-Annual Channel (Targeted)"} 
    }
}

To monitor on the versions we want to support by checking this page by Microsoft. We also monitor the Channel by alerting on anything that is not “Monthly Channel”, as soon as we see an agent that has the incorrect channel we fix it by running the following command

"C:\Program Files\Common Files\Microsoft Shared\ClickToRun\OfficeC2RClient.exe" /changesetting Channel=Monthly

When a client is not up to date, we force the latest update via the following command, this updates the client specifically to the version we want.

 "C:\Program Files\Common Files\Microsoft Shared\ClickToRun\OfficeC2RClient.exe"  /update USER displaylevel=False updatetoversion=16.0.7341.2029

If you want to update to any update that is available, for the channel the installation is in.

 "C:\Program Files\Common Files\Microsoft Shared\ClickToRun\OfficeC2RClient.exe"  /update USER displaylevel=False

And that’s it! you can now use this to update to the latest versions, and monitor the minimum required version you need installed. As always, Happy PowerShelling!

Monitoring with PowerShell: UPS Status (APC, Generic, and Dell)

So we’re using several types of UPS’s at our clients, and sometimes bump into generic USB UPS systems too. To monitor these we use a couple of methods that all have benefits and downsides. Let’s get started.

If a generic USB UPS is installed, Windows Server recognizes this as a Battery Unit. The status is sent to the server by using a generic Windows Driver called “Microsoft Compliant Control Method Battery” which is quite the mouthfull. The good thing is that with this driver we can use a couple of small PowerShell commands to find the exact status of the battery.

USB UPS systems

 $Battery = Get-CimInstance -ClassName win32_battery
Switch ($Battery.Availability) {
    1  { $Availability = "Other" ;break}
   2  { $Availability =  "Not using battery" ;break}
   3  { $Availability = "Running or Full Power";break}
   4  {$Availability =  "Warning" ;break}
   5  { $Availability = "In Test";break}
   6  { $Availability = "Not Applicable";break}
   7  { $Availability = "Power Off";break}
   8  { $Availability = "Off Line";break}
   9  { $Availability = "Off Duty";break}
   10  {$Availability =  "Degraded";break}
   11  {$Availability =  "Not Installed";break}
   12  {$Availability =  "Install Error";break}
   13  { $Availability = "Power Save - Unknown";break}
   14  { $Availability = "Power Save - Low Power Mode" ;break}
   15  { $Availability = "Power Save - Standby";break}
   16  { $Availability = "Power Cycle";break}
   17  { $Availability = "Power Save - Warning";break}
    }

$BatteryStatus = $Battery.Status
$BatteryName = "$($Battery.name)"
$Remaining = $Battery.EstimatedChargeRemaining
$EstRunTimeMinutes = $Battery.EstimatedRunTime
$BatAvailability = $Availability

The script gets the battery status out of WMI, it shows if the machine is running on battery or not, and you can alert on this. We’ve set our systems up to make sure that when the battery status changes from anything but “Not using battery” it alerts, and possibly shuts down the machine.

Another thing to pay attention to is the Battery Status – Most APCs and Dell’s connected to USB even tell the OS if the battery is in a warning state or failed, you should alert on anything but “OK” for the status.

We can’t really monitor network UPS systems with this, as they do not get their data in w32_battery, so we’ll have to use a couple of different solutions for this. I’ll try covering this in a future blog. As always, happy PowerShelling!

Monitoring with PowerShell: Monitoring Security state

After the last couple of blogs I’ve been asked how I monitor the security state of Windows Servers, so I figured I would create a blog about monitoring some security advisement. Of course there is another disclaimer involved.

Disclaimer: Monitoring these security settings is only a small part of what your entire security monitoring suite should look like. There are a lot more settings and changes you’d need to monitor than just these, but these are items that can be used as a early warning system.

Now that we’ve got that out of the way we can start our monitoring script. We will dissect the script together and have the complete version at the bottom of the page.

The Script

First we will start on monitoring debuggers. This can be done both on both workstations and on servers. Debuggers are often used to secretly start a different process with elevated credentials, or you can have a executable start without the user ever clicking on it.

$debug = Get-Childitem -Path "HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\" -Recurse | Where-Object { $_.Property -eq "Debugger" } | Where-Object { $_.pschildname -ne "DeviceCensus.exe" }
if(!$debug) { 
    $DebuggerFound = "Healthy - No Debugers found"
} else {
foreach($key in $debug){
$DebuggerFound += "$($key.pschildname) is debugged `n"
}
}

Using this we find exactly which process has a debugger attached. DeviceCensus.exe always has a debugger attached so we can ignore this executable. Next we’ll be moving on to WDigest monitoring.

WDigest was a protocol that was introduced in the Windows XP time, the idea at that time was that this was to be used for web based authentication. Wdigest is enabled by default from server 2003 until Server 2012R2. The problem is that to have wdigest run correctly plain-text passwords got stored in LLASS. To resolve this Microsoft released an update to make sure you can disable wdigest on systems.

The problem is that all that is required to enable wdigest again is to change a registry key, We are going to monitor this key with two simple PowerShell commands. You want these items to be set to 0. If not, you should resolve by setting it to 0.

$WDigestNegotiate           = get-childitem -path "HKLM:\System\CurrentControlSet\Control\SecurityProviders\WDigest" | Where-Object {$_.Property -eq "Negotiate"}
$WDigestUseLogonCredential  =  get-childitem -path "HKLM:\System\CurrentControlSet\Control\SecurityProviders\WDigest" | Where-Object {$_.Property -eq "UseLogonCredential"}

Next up is Cached Credentials Account monitoring, again this can be used on both workstations and servers. Cached Credentials are used to logon when the domain controller is not available. For servers and workstations I would advise to lower this to 0. On laptops that is more difficult as users need to be able to work offline, currently we set it to 3 (2 for system logons, and 1 for the actual user account).

$CachedCredentialsAllowed   = (Get-ItemProperty "HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon").CachedLogonsCount

And as a last check we monitor the LM Compatibility Level. This declares what types of authentications can be used on the device. For more information on NTLM, LM Compatibility, and kerberos check this blog from Microsoft. We always completely go to the maximum security level of 5.

$NTLMCompatibilityLevel     = (Get-ItemProperty "HKLM:\SYSTEM\CurrentControlSet\Control\Lsa").lmcompatibilitylevel

And that’s it. Monitoring these items make your environment a little bit more secure, and protects you against most forms of Pass The Hash. The full script can be found below.

Full Script

$debug = Get-Childitem -Path "HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\" -Recurse | Where-Object { $_.Property -eq "Debugger" } | Where-Object { $_.pschildname -ne "DeviceCensus.exe" }
if(!$debug) { 
    $DebuggerFound = "Healthy - No Debugers found"
} else {
foreach($key in $debug){
$DebuggerFound += "$($key.pschildname) is debugged <br>`n"
}
}
$WDigestNegotiate           = get-childitem -path "HKLM:\System\CurrentControlSet\Control\SecurityProviders\WDigest" | Where-Object {$_.Property -eq "Negotiate"}
$WDigestUseLogonCredential  =  get-childitem -path "HKLM:\System\CurrentControlSet\Control\SecurityProviders\WDigest" | Where-Object {$_.Property -eq "UseLogonCredential"}
$CachedCredentialsAllowed   = (Get-ItemProperty "HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon").CachedLogonsCount
$NTLMCompatibilityLevel     = (Get-ItemProperty "HKLM:\SYSTEM\CurrentControlSet\Control\Lsa").lmcompatibilitylevel

Monitoring with PowerShell: Monitoring Dell device updates

I’m a big fan of Dell’s Command Update utility. Dell Command update is a program that makes updating Dell based devices super easy, a single utility that you can install on any workstation to update all devices is great. We always deploy Dell Command update with any machine we hand out to clients.

The next issue that occurs is that we need to know if the updates are running well. For this, I’ve made a monitoring set. To make sure that you don’t just monitor without action, we also created a set that automatically remediates.

The monitoring script

The monitoring script downloads a zip file with the Dell Command Update utility. You can create this zip-file yourself by installing Dell Command Update and simply zipping the install location. It then unzips the downloaded file, and runs the DCU-cli with the Report Parameter, I would advise to only run this set on an hourly or even daily schedule, using your RMM system of course.

#Replace the Download URL to where you've uploaded the ZIP file yourself. We will only download this file once. 
$DownloadURL = "https://www.cyberdrain.com/wp-content/uploads/2019/09/DCU.zip"
$DownloadLocation = "$($Env:ProgramFiles)\DCU\"
#Script: 
$TestDownloadLocation = Test-Path $DownloadLocation
if(!$TestDownloadLocation){
new-item $DownloadLocation -ItemType Directory -force
Invoke-WebRequest -Uri $DownloadURL -OutFile "$($DownloadLocation)\DCU.zip"
Expand-Archive "$($DownloadLocation)\DCU.zip" -DestinationPath $DownloadLocation -Force
}
#We start DCU with a reporting parameter set. We wait until the report has been generated.
Start-Process "$($DownloadLocation)\DCU-CLI.exe" -ArgumentList "/report `"$($DownloadLocation)\Report.xml`"" -Wait
[xml]$XMLReport = get-content "$($DownloadLocation)\Report.xml"

$BIOSUpdates        = ($XMLReport.updates.update | Where-Object {$_.type -eq "BIOS"}).name.Count
$ApplicationUpdates = ($XMLReport.updates.update | Where-Object {$_.type -eq "Application"}).name.Count
$DriverUpdates      = ($XMLReport.updates.update | Where-Object {$_.type -eq "Driver"}).name.Count
$FirmwareUpdates    = ($XMLReport.updates.update | Where-Object {$_.type -eq "Firmware"}).name.Count
$OtherUpdates       = ($XMLReport.updates.update | Where-Object {$_.type -eq "Other"}).name.Count
$PatchUpdates       = ($XMLReport.updates.update | Where-Object {$_.type -eq "Patch"}).name.Count
$UtilityUpdates     = ($XMLReport.updates.update | Where-Object {$_.type -eq "Utility"}).name.Count
$UrgentUpdates      = ($XMLReport.updates.update | Where-Object {$_.Urgency -eq "Urgent"}).name.Count

As this is a number monitor, if something is 0 you are completely up to date, we monitor all type of updates. We also like knowing if an update is urgent, which has a separate category.

Remediation

So remediation can be done quickly, In theory we would only have to run a single command, which is the following script

$DownloadLocation = "$($Env:ProgramFiles)\DCU\"
Start-Process "$($DownloadLocation)\DCU-CLI.exe" -Wait

The problem with running this script directly that by default all updates that the DCU finds will be installed, and you cannot set a classification to be excluded. If you would like to exclude specific update types such as BIOS updates or utility software, you’ll have to do this:

  • Open DCU on your administrator workstation
  • click on the cog in the top right corner
  • update filter:, unselect the updates you want to exclude.
  • Export/Import: and export the MySettings.xml file.
  • Add this MySettings.xml file to your self-hosted DCU zip file.

If you’ve done this small list of tasks, then use the following script to install the updates instead:

$DownloadLocation = "$($Env:ProgramFiles)\DCU\"
Start-Process "$($DownloadLocation)\DCU-CLI.exe" -ArgumentList "/import /policy `"$($DownloadLocation)\MySettings.xml`"" -Wait
Start-Process "$($DownloadLocation)\DCU-CLI.exe" -Wait

When executing Thunderbolt or BIOS updates. You will also need to suspend Bitlocker. You can use the following script for this. My advice would be to execute the reboot immediately in this case – and only use this if you are certain that the device is in a secure environment during execution.

$DownloadLocation = "$($Env:ProgramFiles)\DCU\"
Start-Process "$($DownloadLocation)\DCU-CLI.exe" -ArgumentList "/import /policy `"$($DownloadLocation)\MySettings.xml`"" -Wait
Suspend-BitLocker -MountPoint 'C:' -RebootCount 1
Start-Process "$($DownloadLocation)\DCU-CLI.exe" -Wait

the AMP file can be found here. As always, Happy PowerShelling!