Skype for Business Online to On-Premises Migration

Okay guys – you’ve been told “lets move everyone back from the cloud! We need Enterprise Voice for our users” This will go against most Microsoft sales materials as we should be looking towards cloud.

If you are part of an organisation that has been birthed out of Skype for Business Online (SFBO) as part of your Office 365 subscription, it would make sense that you would have never had on-premises Lync or SFB servers in your Active Directory domain. Very little configuration is needed in SFBO and a busy administrators would have loved enabling the license SKU for SFBO for each users and then wiped their hands of it. Its just that easy, enable and forget.

The main limitation around SFBO is the need for an IP-PBX and/or PSTN connectivity. The time may have come for your organisation to leverage your Microsoft agreement even further and look to your existing technology/application catalogue and see that Skype for Business can fill the requirements of your aging PBX. This trigger point is usually when the PBX asset has reached capacity and there is a cost trade off;

  • Throw more money at the dusty old PBX box for extra expanision cards and possibly cabling to terminal
  • Spend the money to start new in the world of VOIP, but look to an existing technology that can provide this functionality (and hopefully more) before looking elsewhere

Telephones have been around for a long time, its nothing new. Picking up, making, transferring calls is all pretty standard stuff we have been doing for few decades now. If I’m going to invest money in something I should be asking for more! more! more! How do I make sure that my choice keeps my organisation staying relevant in the way it communicates for the next chapter?

Hello Skype for Business as PBX replacement.

That’s enough of ramble. You have come here to understand the process of moving users back from the cloud because there is less documented about this procedure than too the cloud.

Current Environment

  • On-premises Domain Services
  • DirSync/AADSync Server
  • Office 365 Tenant
    • Skype for Business Online SKU enabled
  • DNS for my domain.com.au
    • Lyncdiscover.domain.com CNAME webdir.online.lync.com
    • SIP.domain.com. CNAME sipdir.online.lync.com
    • _sipfederationtls._tcp.domain.com SERVICE sipfed.online.lync.com
    • _sip._tls.domain.com SERVICE sipdir.online.lync.com

Current_Environment

In this scenario we will look at the steps needed to specifically enable Hybrid and move users back to on-premises.

  1. Add on-premises infrastructure
  2. Connect Hybrid SFB with Office 365 and on-premises
  3. Move Enterprise Voice users back

Add On-premises Infrastructure

  • Add your Front End, Edge and Reverse Proxy Infrastructure
    1. Build the Servers as per TechNet, but leave the SIP Address’s DNS zones to not effect internal and external clients just yet
    2. All discover records should still point to Microsoft (sipdir, webdir etc)
  • Confiure your Edge and Reverse Proxy with Public Certificates
    1. Test the port authentication as best you can;
      1. Telnet to Edge Ports
      2. Test Reverse Proxy URLs
      3. Remote Connectivity Analyzer for Edge
  • Configure Edge for Federation
    1. Assign your Edge as the Federation Route in your Topology Builder
    2. Configure Edge Specific Configuration

[code]Set-CSAccessEdgeConfiguration -AllowOutsideUsers 1 -AllowFederatedUsers 1 -UseDnsSrvRouting -EnablePartnerDiscovery $true[/code]

  • Recreate Allowed Federated Domains in On-premises
    1. If you do a get-csalloweddomain in Office 365 you may not get all the correct information specific for your tenant back to you.
      1. If you have federation with only allowed/block lists, you may need to recreate these as there is no nice way of piping the cmdlets from ‘get’ to ‘new’.
      2. Allow open federation to accommodate for all traffic is the simplist approach for migration
  • Set the Global Remote Access and Federation User Policy to Allow

[code]Get-CsExternalAccessPolicy -Identity Global | Set-CsExternalAccessPolicy -EnableFederationAccess $True -EnableOutsideAccess $True[/code]

 

Connect Hybrid Skype for Business

  • Remove existing Lync/Skype for Business Hosting Rule

[code]Get-CSHostingProvider -Identity <SFB/Lync Online> | Remove-CSHostingProvider[/code]

  • Recreate the Provider with Hybrid Specific Configuration

[code]New-CSHostingProvider -Identity SFBOnline -ProxyFqdn "sipfed.online.lync.com" -Enabled $true -EnabledSharedAddressSpace $true -HostsOCSUsers $true -VerificationLevel UseSourceVerification -IsLocal $false -AutodiscoverUrl https://webdir.online.lync.com/Autodiscover/AutodiscoverService.svc/root[/code]

  • Update External/Public DNS Records
    1. Remember that only updating external DNS records means that your internal users can functions ‘as-is’ until your happy with the progress
      1. Edge Names (SIP Access/Web Conference/ AV  FQDNs)
      2. External Web Services FQDN
      3. Dialin FQDN
      4. Meeting FQDN
      5. LyncDiscover FQDN
      6. SRV _sipfederationtls._tcp.domain.com
      7. SRV _sip._tls.domain.com
    2. Remote users that aren’t previously authenticated could have an issue logging in at the time of the change

Test Process

Join On-premises Pilot User with Online Account

This makes your on-premises deployment aware of active directory accounts that are currently cloud enabled.

  • Run the following cmdlet in SFB Management Shell connected to on-premises servers to test a user

[code]Enable-CsUser -Identity &lt;accountname&gt; -SipAddress "sip:<sipaddress>" -HostingProviderProxyFqdn "sipfed.online.lync.com" -verbose[/code]

  • Synchronise AADSync/DirSync
    1. Login to Directory Sync Server
    2. Run >  Delta Import And Delta Sync on the Active Directory Connector
    3. You will see an update count that includes your object

Move a Pilot Users back

This step will actually move the user from SFB Online back to your on-premises pool with contact lists intact. This is initiated from the on-premises server and will need authentication for the Office 365 tenant to perfom the task.

  • Run the following cmdlet in Powershell connected to both on-premises and online sessions

[code]Import-Module LyncOnlineConnector
$credential = Get-Credential
$session = New-CsOnlineSession -Credential $credential
Import-PSSession $session -AllowClobber[/code]

  • Get the Online Admin URL for your tenant
    1. Log into Office 365 Portal
    2. Check the URL presented in the address bar, will be admin0x. Where x = a letter specific for you
  • Move the User back

[code]Move-CsUser -Identity <UPN> -Target <FE Pool Name> -Credential $cred -HostedMigrationOverrideURL https://admin0f.online.lync.com/HostedMigration/hostedmigrationservice.svc[/code]

Enable All Users

If the above to Pilot Tests worked we need to scale up our migration batches. We need to mass produce the following cmdlet in SFB Management Shell connecting on-premises user accounts to the corresponding online account;

[code]Enable-CsUser -Identity <accountname> -SipAddress "sip:&lt;sipaddress&gt;" -HostingProviderProxyFqdn "sipfed.online.lync.com" -verbose[/code]

To do this practically I used the UPN value which I knew that would resolve to the correct users values in on-premises and Office 365 because they are synced from the source. I also could then understand the logic that the users UPN is in this case the primary SMTP/Mail value and therefor matching SIP Address for Skype for Business that I needed.

  • Get all the Office 365 users that are enabled for Skype for Business

[code]Get-CSOnlineUser | ? {$_.SipAddress -notlike $Null} | Select SipAddress, DisplayName | Export-CSV -Path C:\temp\OnlineUsers.csv -NoTypeInformation[/code]

  1. This will give you a list of ‘real’ SFBO users that are licensed but are also registered SFB logins
  2. Review the list for deleted users that haven’t been removed properly, there SIPAddress will include GUID style login, these lines can be removed as we do not wish to migrate them.

Lets leverage this list of known online users and enable their joining in on-premises with a ForEach loop example below;

[code language=”powershell” firstline=”1″]

$Users = Import-Csv C:\updates\OnlineUsers.csv

ForEach($User in $Users)
{
$SipAddress = $user.sipaddress
$UPN = $SipAddress.replace("sip:", "")
$Enable = Enable-CsUser -Identity $UPN -SipAddress $SipAddress -HostingProviderProxyFqdn "sipfed.online.lync.com"
}
[/code]

  • Update Azure Active Directory of the changes by another AADSync/DirSync > Delta Import & Delta Sync
  • Update Internal DNS to point all associated SFB records to on-premises Skype for Business Server(s)
    1. SRV _sipinternaltls._tcp.domain.com.au
    2. Lyncdiscoverinternal.domain.com.au
    3. SIP.domain.com.au
  • Add Additional A Records
    1. Meet.domain.com.au
    2. Dialin.domain.com.au
    3. Pool Name
    4. SFB Web Service URL Names
    5. Admin URL

Visual Indication of Success

Log into your on-premises SFB Admin Control Panel and run a blank user search to discover all users. Noticed the ‘Homed’ field should say ‘SFBOnline’ 

Move All Users

Leveraging the same list of users, run the move cmdlet like the example;

[code language=”powershell” firstline=”1″]

ForEach($User in $Users)
{

$Displname = $user.displayname
$SipAddress = $user.sipaddress
$UPN = $SipAddress.replace("sip:", "")
$Move = Move-CsUser -Identity $UPN -Target &amp;amp;amp;amp;amp;amp;lt;FE Pool Name&amp;amp;amp;amp;amp;amp;gt; -Credential $credential -HostedMigrationOverrideURL https://admin0f.online.lync.com/HostedMigration/hostedmigrationservice.svc -Confirm:$false
if($Move -eq $False)
{
Write-host "User $SipAddress didn’t move!!"
}
}
[/code]

Status

To get visual status while you move all the users, log into your Office 365 Skype for Business Administration Portal and view the details. Continually refresh the page to see the value for “users synced and homed online” go down as each user becomes enabled on-premises.

SFB_User_Stats

 

 

 

Log into your on-premises SFB Admin Control Panel and run a blank user search with a additional filter for Homed or Registrar Pool / is equal to / <registrar server name>.

Client Experience

The client should be unknowning of your changes being made in Office 365 and on-premises until you perform the move-csuser request for their account. During this period a redirect message will be sent to the client with a new registrar server FQDN and a automated logout and login will happen. If the user doesn’t have their client in the forground of their desktop, then this will happen silently in the background. The redirect in my move request had the users logged out and back in within about 1-2 seconds.

Hybrid Exchange 2007/2013 and Lync EWS Integration

I came across an interesting issue recently with a client currently running Exchange 2007 and looking to migrate to Exchange Online. Since Update Rollup 10 for Exchange 2007 Service Pack 3, it has become possible to coexist Exchange 2oo7 and Exchange 2013.

After installing Exchange 2013 as the Hybrid server, this particular client ran into an issue with the Lync 2013 client losing EWS integration with any mailboxes that still reside on Exchange 2007. The net effect of this is that any users that hadn’t been migrated from Exchange 2007 to Exchange 2013 or Office 365 would have to rely on Outlook MAPI (and Outlook being open) for anything to do with the Personal Information Manager feature of Lync.

Microsoft have confirmed this is a known issue and there is no workaround or hotfix. It’s worth noting that the issue is isolated to a 2013 Hybrid configuration in coexistence with Exchange 2007 only, so a Hybrid Exchange 2010/2007 environment might be the more appropriate solution here.
Update: Microsoft have released a KB article stating that the workaround is to migrate the Exchange 2007 mailboxes to Exchange 2013 http://support.microsoft.com/kb/2941221

Symptoms:

The Lync client will try to connect to Exchange but will prompt with the following:

Legacy versions of Exchange are unable to proxy requests to Exchange 2013, however Exchange 2013 can proxy to legacy versions. This means that a key step of a Hybrid implementation is pointing the Autodiscover service to Exchange 2013, which is when this issue manifested.

In the image below you can clearly see there is an error with the conversation icon and the alert in the bottom right corner is an EWS integration error. The Calendar icon is also missing from the top row.

In Lync “Configuration Information” you’ll notice that there is no server listed for the “Contact List Provider” and under “EWS Information” it shows the status as “EWS not deployed”

Delving into this a little further, looking at the IIS logs on the Exchange 2013 Hybrid server (or the Exchange 2013 server hosting the CAS role), you can see that there is a successful 200 response to the initial authentication request:

2014-03-26 00:36:48 192.168.30.5 POST /autodiscover/autodiscover.svc &CorrelationID=<empty>;&cafeReqId=680dcd6d-2d78-452c-acc2-7659fdc97343; 443 CONTOSO\test1 192.168.10.100 OC/15.0.4569.1503+(Microsoft+Lync) – 200 0 0 15

However when you get to the Autodiscover logs, you can see that the Autodiscover XML file is never actually downloaded by the Lync client due to the mailbox being on Exchange 2007.

2014-03-26T00:36:48.196Z,680dcd6d-2d78-452c-acc2-7659fdc97343,15,0,847,30,,Negotiate,True,test@contoso.com,,OC/15.0.4569.1503 (Microsoft Lync),192.168.10.100,EXCHANGE13,EXCHANGE13.CONTOSO.LOCAL,GetUserSettings,200,InvalidRequest,0,0,1,,,,,GlobalThrottlingPolicy_d74d65fa-cdb4-4e06-be99-6f259ea1158a,,,0,8,0,8,,0,15.63,ADSessionSettingsFromAddress=0;ADRecipientSessionFindBySid=15.63;RequestedUser=test@contoso.com;S:ServiceCommonMetadata.RequestSize=1527;S:WLM.Bal=300000;S:WLM.BT=Ews;S:BudgetMetadata.MaxConn=27;S:BudgetMetadata.MaxBurst=300000;S:BudgetMetadata.BeginBalance=300000;S:BudgetMetadata.Cutoff=3000000;S:BudgetMetadata.RechargeRate=900000;S:BudgetMetadata.IsServiceAct=False;S:BudgetMetadata.LiveTime=00:01:00.2078770;S:BudgetMetadata.EndBalance=300000;Dbl:WLM.TS=15.63;I32:ATE.C[DC01.CONTOSO.LOCAL]=1;F:ATE.AL[DC01.CONTOSO.LOCAL]=0;I32:ADS.C[DC01]=1;F:ADS.AL[DC01]=4.9612,,ErrorMessage=The SOAP-based AutoDiscover service is not available for mailboxes on Exchange 2007.;

Office Web Apps Server – just say no to Windows Update Automatic Updates

Office Web Apps Server 2013 is a standalone Microsoft product that is leveraged by Lync 2013, SharePoint 2013 and Exchange 2013 for web based document viewing and editing using the WOPI (Web app Open Platform Interface) protocol. Office Web Apps Server used be called Web Application Companion (WAC) and that is what all of the Lync 2013 pre-release software and documentation called it. In my opinion, Office Web Apps Server is a very confusing name as Exchange Outlook Web App (or Access) has owned the OWA acronym since 1997 with Exchange 5.0 SP 1. I have seen some people refer to it as OWA Server, but I am trying to reduce confusion and am on a campaign to have everyone call it OWAS.

An overview of OWAS from Microsoft explains how it works and how it integrates with Lync 2013, SharePoint 2013 and Exchange 2013 http://technet.microsoft.com/en-us/library/jj219437(v=office.15)

For Lync 2013 OWAS is an optional component, but it is required if you want to use the PowerPoint sharing feature of Lync. If OWAS is not deployed you can still share the desktop or share PowerPoint as a program and meeting participants see what the presenter is showing them. However you cannot choose the ‘PowerPoint’ option seen below which uploads a presentation and lets participants experience the full PowerPoint features such as embedded video, transitions, animations and skipping back or forward on the slide deck out of sync with the presenter.

Windows Update Automatic Updates is Not Supported

The point of this post is to call out something that we have seen at two Lync 2013 clients so far who deployed OWAS. The documentation Plan Office Web Apps Server states:

“Applying Office Web Apps Server updates by using the Microsoft automatic updates process isn’t supported with Office Web Apps Server”

Apply software updates to Office Web Apps Server explains the correct process of how to deploy updates for Office Web Apps Server, including server farms, and cautions:

“Applying Office Web Apps Server updates by using the automatic updates process isn’t supported with Office Web Apps Server. This is because updates to an Office Web Apps Server must be applied in a specific way, as described in this article. If Office Web Apps Server updates are applied automatically, users may be unable to view or edit documents in Office Web Apps. If this happens, you have to rebuild your Office Web Apps Server farm”

What Goes Wrong

If the advice above is not followed and automatic installation of Windows Updates is enabled, when an update for OWAS like the March 2013 KB2760445 or April 2013 KB2810007 updates (at +- 585 MB each) are added to Windows Update, the server will install the update and OWAS functionality will break. Lync users will not be able to use PowerPoint sharing in meetings.  The server CPU will spike to 100% (thanks to the many OWAS applications) and sit there indefinitely. Additionally, the Office Web Apps Server Farm will be in a broken state. You will get some Application Event Log errors logged for:

  • ‘.NET Runtime’ Event ID 1026 and
  • ‘Application Error’ Event ID 1000 about some of the viewers such as ‘Faulting application name: pptviewerfrontendwatchdog.exe, version: 15.0.4481.1000’

The following commands will state that ‘It does not appear that this machine is part of an Office Web Apps Server farm’:

[sourcecode language=”powershell”]
Import-Module OfficeWebApps
Get-OfficeWebAppsFarm
Get-OfficeWebAppsMachine
[/sourcecode]

In order to get these commands to even respond, you may have to stop the ‘Office Web Apps’ WACSM service.

You may be thinking to yourself that if Office Web Apps Server updates should not be automatically installed – then the updates should not be published on Windows Update at all. Makes sense to me!

Recreate The Farm

The way to fix the broken OWAS farm is to recreate it using the same commands used when the server was installed to create the Office Web Apps Farm. The article mentioned above about applying updates says you should run:

[sourcecode language=”powershell”]
Remove-OfficeWebAppsMachine
[/sourcecode]

However this can be bypassed by running the original commands and confirming at the editing and overwrite prompts, or adding ‘-force’ to the command. Recreating the farm would be similar to the following:

[sourcecode language=”powershell”]
Import-module OfficeWebApps
New-OfficeWebAppsFarm -InternalUrl "https://owas01.domain.local" -ExternalUrl "https://owas.domain.com.au" –CertificateName "OWAS_Internal" –EditingEnabled
[/sourcecode]

The output of this is a successfully created farm and the server should return to normal CPU levels. If the CPU is still 100%, restart the server.

More information on the New-OfficeWebAppsFarm syntax is available http://technet.microsoft.com/en-us/library/jj219436.aspx

Note that the command above can take about 10 minutes to complete as it starts the WACSM service if it is stopped. After that, make sure you disable automatic Windows Update installation or you will have to go through the process again in the future.

Follow ...+

Kloud Blog - Follow