Recently, I had a customer request the ability to quickly report on the status of two AAD Connect servers.

Since these two servers operate independently, it is up to the administrator to ensure the servers are healthy and they are operating in the correct configuration modes with respect to each other.

Typically, if you’re going to spend money operating two AAD connect servers, it make sense they both are enabled with their import cycles but only one runs in ‘Normal’ mode (i.e. exporting) and the other in ‘Staging’ mode (i.e. not exporting but ready to take over if needed).

This customer had a full import & full sync. time of almost two full days (!), so it was essential the second staging mode AAD Connect server was operating correctly (in Staging mode and with its cycle enabled) to take over operations.

Since AAD Connect is based on the architecture of the Synchronization Engine of Microsoft Forefront Identity Manager (formerly known as MIIS), clustering is not an option.

The ‘Get-ADSyncScheduler’ AAD Connect PowerShell commands is well documented by Microsoft, and we’ve posted a few articles on using that command recently at this blog as well.

My customer had a few requirements:

  • Be able to quickly gather the status of both AAD Connect servers once an administrator has logged into at least one of them
  • Pool the status of both server’s ‘Staging’ mode status and its cycle status (either enabled or disabled)
  • Warn administrators if two servers are operating in ‘normal’ mode or are otherwise mis-configured

On the third point, if you attempted to bring a second AAD Connect server out of ‘Staging’ mode, there’s nothing on the server or via Azure authentication that prevent you doing that.  Microsoft strongly warn you during the installation process that you should be wary of other AAD Connect servers and their staging mode status.

I briefly tested dropping a second server out of Staging Mode in a Test environment resulting in two AAD Connect servers operating in ‘normal’ (import/export) and whilst I didn’t see any immediate issue, I strongly recommend not doing this.  I also only had a test Office 365 tenancy of only a handful of objects to test against so it wasn’t a true reflection of what could happen in a more production environment with more features (like near real time password hash sync.) and more objects.  I honestly thought I’d run into a blocking message preventing me making that configuration.

After developing the script, I went down the path of using string matching to determine the results of the ‘Get-ADSyncScheduler’ command.  This had the following impacts:

  • In order to simplify the script, I wanted the objects for ‘Staging’ and ‘Cycle’ to have three results: ‘null’, ‘true’ or ‘false’.
  • In order to filter the results of the ‘get-ADSyncScheduler’ command, I converted it into a string, then performed a string matching query against the whole string for the results of the Staging and Cycle enabled options.
  • Instead of string matching, this example command would return the actual value of ‘Staging mode’ and could have been used instead:
    • Get-ADSyncScheduler | select -ExpandProperty SyncCycleEnabled

Once you have the results of the status of the Staging and Cycle properties for both servers(true/false/null), you can then report on them collectively to either indicate an ‘OK’ status, a ‘Warning’ status or an ‘Alert’ status.

I broke up those reporting categories into the following:

  • OK Status – One ‘Normal’ mode AAD Connect server with Cycle enabled, one ‘Staging’ AAD Connect server with Cycle enabled
  • Warning Status – One ‘Normal’ mode AAD Connect server with cycle enabled, with the other AAD Connect server with its cycle disabled (but still configured to be in Staging Mode)
  • Offline Alert Status – the remote AAD Connect server cannot be contacted i.e.. there’s a null object when the text is searched for the status
  • Alert Status – Both servers can be contacted but no AAD connect servers are operating in normal mode (ie. both are in ‘Staging mode’) or two servers are operating in normal mode.

 

This script was installed onto both of the AAD Connect servers with a shortcut provided to the ‘All Users’ desktop.  Both copies of the script were then modified with the specific server name of their respective remote AAD Connect servers, ie. ‘Server B’ in ‘Server As’ script and vice versa, at this location:
$remoteAAD = Invoke-Command -ComputerName <remote server name here> -ScriptBlock { Get-ADSyncScheduler }
This script could be enhanced with:
  • Running on an administrator’s workstation instead and both AAD Connect servers treated with remote PowerShell commands (and script updated and tested for compatibility running remotely with AAD Connect PowerShell commandlets)
  • Enhanced with email alerting if it was to run in conjunction with Windows Scheduler
A warning: this script has not been tested against two servers operating in normal mode or offline so some of the alerts are ‘theoretical’ at this stage.  Let me know in the comments if you find any bugs etc.

 

cls

# Set variables and constants

$localStagingStatus = $null
$remoteStagingStatus = $null
$localCycleStatus = $null
$remoteCycleStatus = $null

$StagingTrue = """StagingModeEnabled"":true"
$SyncCycleEnabledTrue = """SyncCycleEnabled"":true"
$StagingFalse = """StagingModeEnabled"":false"
$SyncCycleEnabledFalse = """SyncCycleEnabled"":false"

# Review local AAD Scheduler details

$localAAD = Get-ADSyncScheduler
$localAADstr = $localAAD.ToString()

if ($localAADstr -match $StagingTrue) {write-host -ForegroundColor DarkYellow "Staging mode ENABLED locally"
$localStagingStatus = "true" }

if ($localAADstr -match $SyncCycleEnabledTrue) {write-host -ForegroundColor DarkYellow "Sync Cycle ENABLED locally"
$localCycleStatus = "true" }

if ($localAADstr -match $StagingFalse) {write-host -ForegroundColor DarkYellow "Staging mode DISABLED locally"
$localStagingStatus = "false" }

if ($localAADstr -match $SyncCycleEnabledFalse) {write-host -ForegroundColor DarkYellow "Sync Cycle DISABLED locally"
$localCycleStatus = "false" }

# Connect to remote AAD connect server

$remoteAAD = Invoke-Command -ComputerName servername -ScriptBlock { Get-ADSyncScheduler }
$remoteAADstr = $remoteAAD.ToString()

if ($remoteAADstr -match $StagingTrue) {write-host -ForegroundColor DarkYellow "Staging mode ENABLED remotely"
$remoteStagingStatus = "true"}

if ($remoteAADstr -match $StagingFalse) {write-host -ForegroundColor DarkYellow "Staging mode DISABLED remotely"
$remoteStagingStatus = "false"}

if ($remoteAADstr -match $SyncCycleEnabledTrue) {write-host -ForegroundColor DarkYellow "Sync Cycle ENABLED remotely"
$remoteCycleStatus = "true"}

if ($remoteAADstr -match $SyncCycleEnabledFalse) {write-host -ForegroundColor DarkYellow "Sync Cycle DISABLED remotely"
$remoteCycleStatus = "false"}

if ($debug) {
write-host "local staging status:" $localStagingStatus
write-host "local cycle status:" $localCycleStatus
write-host "remote staging status:" $remoteStagingStatus
write-host "remote cycle status:" $remoteCycleStatus
}

# Interpret results

write-host "---------------------------------------------------------------"
write-host "Summary of Results from AAD Connect server:" $env:computername
write-host "---------------------------------------------------------------"

# OK

if ($localStagingStatus -eq "true" -and $localCycleStatus -eq "true" -and $remoteStagingStatus -eq "false" -and $remoteCycleStatus -eq "true") { write-host -foregroundcolor Green "OPERATIONAL STATUS: OK. Local server operating in ACTIVE STANDBY mode. Remote server operating in active production mode."}
if ($localStagingStatus -eq "false" -and $localCycleStatus -eq "true" -and $remoteStagingStatus -eq "true" -and $remoteCycleStatus -eq "true") { write-host -foregroundcolor Green "OPERATIONAL STATUS: OK. Local server operating in ACTIVE PRODUCTION mode. Remote server operating in active standby mode."}

# Warning

if ($localStagingStatus -eq "true" -and $localCycleStatus -eq "false" -and $remoteStagingStatus -eq "false" -and $remoteCycleStatus -eq "true") { write-host -foregroundcolor Yellow "OPERATIONAL STATUS: Warning. Local server operating in OFFLINE STANDBY mode. Remote server operating in ACTIVE PRODUCTION mode."}
if ($localStagingStatus -eq "false" -and $localCycleStatus -eq "true" -and $remoteStagingStatus -eq "null" -and $remoteCycleStatus -eq "null") { write-host -foregroundcolor Yellow "OPERATIONAL STATUS: Warning. Local server operating in ACTIVE PRODUCTION mode. Remote server cannot be contacted, could be OFFLINE"}

# Offline Alert

if ($remoteStagingStatus -eq "null" -and $remoteCycleStatus -eq "null") { write-host -foregroundcolor Yellow "OPERATIONAL STATUS: Alert. Local server operating in STANDBY mode. Remote server cannot be contacted, could be OFFLINE"}

# Major Alert, confirmed configuration issue

if ($localCycleStatus -eq "false" -and $remoteCycleStatus -eq "false") {write-host -foregroundcolor Red "OPERATIONAL STATUS: Both servers have their cycles disabled. Review immediately."}
if ($localStagingStatus -eq "true" -and $remoteStagingStatus -eq "true") { write-host -foregroundcolor Red "OPERATIONAL STATUS: Both servers are in Staging mode. Review immediately."}
if ($localStagingStatus -eq "false" -and $localCycleStatus -eq "true" -and $remoteCycleStatus -eq "true" -and $remoteStagingStatus -eq "false") { write-host -foregroundcolor Red "OPERATIONAL STATUS: Alert. Both servers are in ACTIVE PRODUCTION mode. This violates Microsoft best practice and could cause replication problems. Review immediately."}

 

 

 

Category:
Identity and Access Management, Office 365

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: