Monitoring with PowerShell Chapter 2: Hyper-V replication and remediation

Hi All,

As I’ve explained in my previous the series is taking a bit of a turn here and we’re going to start some blogs about remediation instead of just monitoring. I’ll link back to a previous blog and will explain how we automatically react to these issues within our RMM, if you do not have an RMM – Don’t worry! We’ll include the monitoring + remediation script so you can combine the scripts any way you’d like.

The first monitoring and remediation we’re getting on is Hyper-V replication, where we will try to resolve the replication if the server was unreachable for some time, force a resync, and clear the statistics.

We’re getting started on monitoring the current Hyper-V Replication state, to do this we use the following small script:

$VMList = Get-VMReplication  -ErrorAction SilentlyContinue

ForEach ($VM in $VMList)	{
if ($VM.health -eq "Critical")	{ $critical = $critical + 1	}
if ($VM.health -eq "Warning")	{ $warning = $warning + 1   }
if ($VM.state -ne "Replicating"){ $Critical = $Critical + 1 }
                                }
if($critical -gt 0){$Critical = "unhealthy"}
if($warning -gt 0){$warning = "unhealthy"}

if(!$critical ){ $Critical  = "Healthy"}
if(!$warning ){ $warning  = "Healthy"}

Now this script gives us the result we need to start our automatic remediation path; We know that VM’s are not replicating if we have a state of $Critical = unhealthy, and we know that we have Replication Warnings (Most likely related to statistics) when $warning is set to unhealthy. At this point in time we’ll get a fork in the road and some options; do we want our monitoring script to also be our remediation script, or do we want separate scripts that are launched based on the condition of the replication?

If you want the former, the script only needs a couple of extra lines, but monitors AND resolves the issues immediately when found, I do not like this approach very much as it can give you skewed results and you do not control when/how the remediation runs.

$VMList = Get-VMReplication  -ErrorAction SilentlyContinue

ForEach ($VM in $VMList)	{
                if ($VM.state -match "WaitingForStartResynchronize"){ Resume-VMReplication -VMName $VM.Name -Resynchronize -Verbose   }
		if ($VM.health -eq "Critical")	{ $critical = $critical + 1	}
		if ($VM.health -eq "Warning")	{ $warning = $warning + 1   }
                if ($VM.state -ne "Replicating"){ $Critical = $Critical + 1 }
                               }                     
if($critical -gt 0){$Critical = "unhealthy"}
if($warning -gt 0){$warning = "unhealthy"}

if(!$critical ){ $Critical  = "Healthy"}
if(!$warning ){ $warning  = "Healthy"}


ForEach ($State in $ReplState)
{
$Name = $State.Name 
$CurrentState = $State.State
}
} 

After running this script you will most likely notice that it’s still giving you a warning, you can decide to wait for the replication to run long enough to resolve this on its own, or you can decide to reset the statistics right before self-healing:

$VMList = Get-VMReplication  -ErrorAction SilentlyContinue

ForEach ($VM in $VMList)	{
					write-host $VM.name is $VM.health
                    if ($VM.state -match "WaitingForStartResynchronize"){
                    Reset-VMReplicationStatistics -VMName $Name
                    Resume-VMReplication -VMName $Name -Resynchronize -Verbose  
                    }
					if ($VM.health -eq "Critical")	{ $critical = $critical + 1	}
					if ($VM.health -eq "Warning")	{ $warning = $warning + 1   }
                    if ($VM.state -ne "Replicating"){ $Critical = $Critical + 1 }
           

if($critical -gt 0){$Critical = "unhealthy"}
if($warning -gt 0){$warning = "unhealthy"}

if(!$critical ){ $Critical  = "Healthy"}
if(!$warning ){ $warning  = "Healthy"}


ForEach ($State in $ReplState)
{

$Name = $State.Name 
$CurrentState = $State.State



}
} 

If you decided to split the scripts as per my advice, you can use the first script in this blog post to use as your monitoring set, and the following script as your automated resolution script, remember that this is only the resolution in case the replication failed due to the other side being unreachable, and does not resolve certificate errors or other errors.


$ReplState = Get-VMReplication
ForEach ($State in $ReplState) {
$ping = Test-Connection -computername $State.ReplicaServerName
if (-Not $ping ) { write-host "Cannot Ping $($state.ReplicaServerName)" }
else {
$Name = $State.Name 
$CurrentState = $State.State
if ($State -match "WaitingForStartResynchronize"){
Reset-VMReplicationStatistics -VMName $State.Name
Resume-VMReplication -VMName $State.Name -Resynchronize -Verbose
}
}
}
 

And that’s it! Have fun 😊

2 thoughts on “Monitoring with PowerShell Chapter 2: Hyper-V replication and remediation

  1. James

    Hi Kelvin,

    Firstly, thanks for creating all of the scripts you do! I’m trying to implement as many as I can into our current workflows etc

    However, I’m having trouble with the remediation part of this script. The monitor works as expected but the remediation doesn’t do anything.

    When I run it manually, nothing happens unless I run the below manually:

    if ($State -match “WaitingForStartResynchronize”){
    Reset-VMReplicationStatistics -VMName $State.Name
    Resume-VMReplication -VMName $State.Name -Resynchronize -Verbose
    }

    Obviously, that only then takes the last machine from the foreach loop.

    Do you have any thoughts as to why it’s not working?

    Thanks in advance,

    James

    Reply
    1. James

      For anyone having the same problem, I’ve managed to sort this.

      if ($State -match “WaitingForStartResynchronize”) needs to be:

      if ($CurrentState -match “WaitingForStartResynchronize”)

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.