Outlook Anywhere NTLM SSO with UAG 2010 KCD

Outlook Anywhere can be configured with two authentication methods – Basic and NTLM. Outlook Anywhere NTLM authentication has always been a bit of a tricky beast when using a pre-authenticating reverse proxy like TMG or UAG. The benefit it can bring is that a user signed on to a domain joined computer with a domain account can get seamless SSO (Single Sign On) without entering a password. This can happen if the user is on the corporate network or if they are remote using Outlook Anywhere (RPC over HTTP) and cached credentials. This sounds like a great solution but the reality is that it often causes more problems than it is worth when using TMG or UAG as they have to use Kerberos Constrained Delegation (KCD) to connect to Exchange.

This post will cover a bit of background around NTLM Outlook Anywhere and KCD and a seemingly hidden configuration change you need to make to successfully use Outlook Anywhere NTLM authentication with UAG and a hardware load balancer. I could not find this referenced anywhere else, so am hopeful that it will save somebody a lot of troubleshooting time. Bear with me as this post gets a little detailed. If you are impatient (and we have all been in that situation where we only care about the solution) then jump to the fix.

Symptoms

On a recent customer project, we had a UAG array that needed to connect to a Hardware Load Balancer (HLB) Virtual IP address (VIP) which load balanced an Exchange Client Access Server (CAS) array. The experience for remote Outlook clients using Outlook Anywhere is they keep getting authentication prompts when trying to connect or create a new Outlook profile. Putting in the correct user name and password will just prompt again and again.

Publishing Outlook Anywhere with NTLM

A white paper by Greg Taylor from the Exchange Product Group was released by Microsoft in January 2011 Publishing Outlook Anywhere Using NTLM Authentication With Forefront TMG or Forefront UAG which covered the steps for TMG and UAG to enable NTLM Outlook Anywhere authentication. This configuration guide covered a lot of the issues people encountered and worked fine if you used a single Exchange Client Access Server (CAS) or wanted UAG to perform the load balancing with a server farm.

Adding a Hardware Load Balancer to the Solution

The problem comes when trying to use a Hardware Load Balancer (HLB) between UAG and the CAS Array. As UAG will only connect to a single published name that is configured on the ‘Addresses’ section of the ‘Web Servers’ tab and will be using KCD, Exchange needs to have a Service Principal Name (SPN) for the published name and UAG must be allowed to delegate authentication. However an SPN should not exist twice in an Active Directory forest, so the CAS array members cannot all have the SPN added. If UAG tries to connect to the server using the SPN, authentication will fail. This was covered in detail in an Exchange Team Blog Combining Web Farm publishing with Software or Hardware Based Load Balanced CAS arrays.

Alternate Service Account

A solution for CAS Array Kerberos authentication was introduced with Exchange 2010 Service Pack 1 in the form of the Alternate Service Account (ASA) which is covered in Configuring Kerberos Authentication for Load-Balanced Client Access Servers and Using Kerberos with a Client Access Server Array or a Load-Balancing Solution. All Client Access Servers use a shared computer account (or use account if required) via the Microsoft Exchange Service Host service and SPNs are added to this account to allow clients to authenticate to any CAS using the CAS Array FQDN. The steps to configure the ASA are:

1. Create a computer account for the ASA
2. Add the SPNs to the ASA
3. Convert the Offline Address Book to an application using ConvertOABDir.ps1
4. Deploy the ASA account to the Exchange CAS array members using RollAlternateserviceAccountPassword.ps1

The first link above states:

“External or Internet-based clients that use Outlook Anywhere won’t use Kerberos authentication. Therefore, the fully qualified domain names that are used by these clients don’t have to be added as SPNs to the ASA credential.”

This seems to be the root of the problem with Outlook Anywhere and UAG, as UAG needs to use Kerberos authentication to perform Kerberos Constrained Delegation (KCD).

KCD, S4U2Self and S4U2Proxy

In order for the client to authenticate to the Client Access Server via UAG, the high level flows are:

1. UAG takes the user credentials in non-Kerberos form, for example Basic or NTLM
2. UAG uses the Kerberos protocol transition extension S4U2Self (Service for User to Self) to obtain a Kerberos ticket on behalf of the user, as if the user had obtained it directly
3. UAG uses the Kerberos Constrained Delegation (KCD) extension S4U2Proxy (Service for User to Proxy) to impersonate the user and obtain a Kerberos ticket to access the Exchange CAS
   1. The UAG server needs to be allowed to delegate authentication to the SPN Exchange uses
4. The Exchange CAS responds to UAG as if the user had authenticated directly with Kerberos
5. UAG sends the response back to the user device

More information on Kerberos Constrained Delegation and the S4U2Self and S4U2Proxy Kerberos extensions is available from:

http://msdn.microsoft.com/library/cc246071(PROT.13).aspx

http://technet.microsoft.com/en-US/library/cc738207(v=ws.10)

Usual Exchange and UAG Configuration

• Follow the ASA guide mentioned above Configuring Kerberos Authentication for Load-Balanced Client Access Servers
• Enable Outlook Anywhere on each server (or all at once)

[sourcecode language=”powershell”]
Get-OutlookAnywhere -server serverone | Set-OutlookAnywhere -ClientAuthenticationMethod NTLM
[/sourcecode]

• Add SPNs for additional UAG published names if you are using custom ports as mentioned in my previous blog about custom port redirect problems.

[sourcecode language=”powershell”]
setspn -s http/exchange.domain.com CAS-ASA$
[/sourcecode]

• Configure UAG Outlook Anywhere application for KCD as per the White Paper above
• Configure Active Directory to allow UAG to use KCD to the ASA computer account SPNs that are published by UAG. Rather than exporting the LDIF file from UAG, you can run the following script to setup delegation where ‘exchange.domain.com’ is what is entered on the UAG application ‘Addresses’ tab

[sourcecode language=”powershell”]
$SPNs = "http/exchange.domain.com","http/exchange"
$UAGservers = "UAG01","UAG02","UAG03","UAG04"
Import-Module -Name ActiveDirectory
foreach ($UAGserver in $UAGservers) {
$TempComputer = $null
$Tempcomputer = Get-ADComputer $UAGserver
Set-ADObject $Tempcomputer.DistinguishedName -Add @{"msDS-AllowedToDelegateTo" = $SPNs}
Set-ADObject $Tempcomputer.DistinguishedName -Replace @{"userAccountControl" = 16781312}
}
[/sourcecode]

At this point you will have done everything mentioned for Exchange and UAG in the Microsoft documentation but it will not work and Outlook users will keep getting prompted for credentials.

Cause

Outlook Anywhere uses the RPC application in IIS which uses the DefaultAppPool Application Pool.

The DefaultAppPool is different to all of the Exchange specific application pools as it uses the ApplicationPoolIdentity identity, not LocalSystem as can be seen below.

As it is not using LocalSystem this application pool cannot use the Exchange Alternate Service Account as it does not have access to the Exchange Service Host SPNs. That means UAG cannot delegate authentication and Kerberos Constrained Delegation will not work. KCD will not fall back to a lower authentication mechanism on authentication failure.

How to Make It Work

The fix is pretty simple – modify DefaultAppPool to LocalSystem or create a new application pool that uses LocalSystem. The first option is not recommended as there are other applications using the DefaultAppPool and doing so may give more rights than are required. Creating a new application pool is possible in the IIS console, but the DefaultAppPool has quite a few non-standard settings. So the easiest way is to use the IIS PowerShell module to copy the application pool. This was my first attempt at using the IIS PowerShell module and I could not get it consistently copy the DefaultAppPool across multiple Exchange versions and Windows patch levels. In the script below there is a dirty hack that creates the application pool, deletes it and then copy. Without the precreation I would get the error ‘Copy-Item : Object reference not set to an instance of an object’ when trying to copy. Note the iisreset at the end – this will disconnect users so plan ahead.

[sourcecode language=”powershell”]
Import-Module webadministration
### Need to stop DefaultAppPool and precreate the new app pool to get it to work consistently
Stop-WebAppPool DefaultAppPool
New-WebAppPool MSExchangeRpcAppPool
Remove-WebAppPool MSExchangeRpcAppPool
copy IIS:\AppPools\DefaultAppPool IIS:\AppPools\MSExchangeRpcAppPool
Start-WebAppPool DefaultAppPool
### Modify new app pool to LocalSystem
$RpcAppPool = dir IIS:\AppPools | where {$_.name -eq "MSExchangeRpcAppPool"}
$RpcAppPool.processModel.identityType = "LocalSystem"
$RpcAppPool | set-item
### Change RPC applications to use the new app pool
set-itemproperty "IIS:\Sites\Default Web Site\Rpc" ApplicationPool MSExchangeRpcAppPool
set-itemproperty "IIS:\Sites\Default Web Site\RpcWithCert" ApplicationPool MSExchangeRpcAppPool
iisreset
[/sourcecode]

You end up with a new application pool using LocalSystem that only contains the RPC applications.

This process has been tested and works with Exchange 2010 Service Pack 2 and Service Pack 3 on Windows 2008 R2. Exchange 2013 and Windows 2012 have not been validated.

Bonus Information – KCD and Trusts

Attempting to work out if Outlook Anywhere NTLM with UAG and KCD was supported with a trust sent us down many conflicting documents and references. This UAG page in particular made me think it would not work Configuring single sign-on with Kerberos constrained delegation when it says:

“The following are the requirements for Kerberos constrained delegation:

• The Forefront UAG server must be part of a domain.
  
• You must define only one authentication server for the trunk to which the application belongs.
  
• All domain controllers in the internal network must be running Windows Server 2003.
  
• Users must be part of the same Active Directory forest as the Forefront UAG server and the application servers.
  
• Forefront UAG servers and application servers must be part of the same domain”
  

However Summary (Kerberos Protocol Transition and Constrained Delegation) says:

“The accounts of users accessing the services do not have to be in the same domain as the services”

It wasn’t clear if that meant it could be in a different domain in the same forest, or if it could cross forest boundaries. Many other documents and articles confused the issue and I could not find a nice concise link stating it would work. This is probably the closest but it is for ISA and IAG (the predecessors of TMG and UAG) KCD with Cross-Forest Accounts.

The bottom line is that it does work – users from trusted forest A can access their linked mailboxes from Outlook Anywhere via UAG in forest B and get seamless SSO.

For KCD to work, UAG and the resource (Exchange in this case) have to be in the same domain. For S4U2Self protocol transition to work, there must be a two way forest trust (which means Windows 2003 Forest functional level or later) and the UAG servers need to be able to resolve and connect directly to the trusted forest Active Directory Domain Contollers for authenticating end users. In our situation we had to add an additional domain suffix to the UAG server internal network interfaces and open up firewall ports.