Azure ExpressRoute Public and Microsoft peering changes, notes from the field

I’ve been trying to piece all this together and get a single, concise blog post that covers all bases around the changes that have happened and are going to be happening for Microsoft ExpressRoute peering. That’s been a bit of a challenge because, I hope I don’t harp on this too much, but, communication could be a bit better from the product group team. With that said, though, it’s no secret for those that use ExpressRoute, Microsoft is looking to simply it’s configuration. Good news I guess?
The main change that I’m going to delve into here comes by way of merging Microsoft Peering and Public peering into a single Microsoft Peer. Microsoft announced this at the Ignite 2017 conference:

“To simplify ExpressRoute management and configuration we merged public and Microsoft peering”.

Fast forward from September 2017, there’s not been much communication around this shift in ExpressRoute config. I’ve been scouring the interwebs for publicly available confirmation; and all I could find is a blog post that highlighted that:

“As of April 1, 2018, you cannot configure Public peering on new ExpressRoute circuits.”

Searching the Twitterverse for the hashtag #PublicPeering, we get the following confirmation only a few days later on April 5th:

So, we have confirmation that this change in ExpressRoute Public peering is happening; followed by a confirmation that as of April 1st, 2018 (no this wasn’t a joke), any new ExpressRoute circuits provisioned on or after that April fools date cannot have Public Peering. Well, given the breadth of Microsoft, communication is in a grey area. Apart from that Japanese TechNet blog post, there’s really only suggestions and recommendations budging customers to Microsoft peering. Here’s two examples:

  1. Microsoft peering is the preferred way to access all services hosted on Azure. (Source)
  2. All Azure PaaS services are also accessible through Microsoft peering. We recommend you to create Microsoft peering and connect to Azure PaaS services over Microsoft peering. (Source)

I know I’m banging on about this for too long, but, for me this is a grey area and better communication is required!
 

Migration

If you’re currently using Public peering and need to move to Microsoft peering, there’s some pretty good guidance from Microsoft on how to Migrate – available here.

NOTE: Microsoft peering of ExpressRoute circuits that were configured prior to August 1, 2017 will have all service prefixes advertised through Microsoft peering, even if route filters are not defined. Microsoft peering of ExpressRoute circuits that are configured on or after August 1, 2017 will not have any prefixes advertised until a route filter is attached to the circuit. (Source)

For many customers, and recently a customer I’ve been working with, they’ve had ExpressRoute for several years now. This change has culminated in some interesting circumstances. For this customers migration process, they were actually after upgrading to a faster network carriage and faster ExpressRoute circuit. This meant we could line up the new environment in parallel to the legacy and in configuring peering on the new service, we just configured it as Microsoft Peering only, no more Public peering.
This is all well and good, but, using a legacy ExpressRoute circuit that was configured in ASM/Classic, there’s now also the consideration of Route Filters. In the legacy or Classic ExpressRoute deployment, BGP Communities were not used. Routes were advertised as soon as the peer came on line and, ARP done and eBGP session established between Azure and the customer.
In the ARM ExpressRoute deployment model, Azure Route Filters are a requirement for Microsoft peering (only). Note that this is an Azure side config, not a customer side which can confuse people when talking BGP Route Filters. Similar concept, similar name, much confuse.
ExpressRoute Microsoft peering, out of the box, no routes are advertised from Azure to the customer until such time that a Route Filter is associated with the peer. Inside of that Route Filter, BGP Community tags for the relevant services also need to be defined.
Again, just need to highlight that Route Filters are only required for Microsoft Peering, not for Private peering.
Here’s a few more relevant references to ExpressRoute, Route Filters and BGP Communities:

 

Changes to Azure AD

Recently Microsoft gave everyone that used ExpressRoute public peering about a 45-day notice that from August 2018 Azure AD authentication and authorisation traffic will no longer be routable via Public peering. This functionality is still available if you use Office 365 over ExpressRoute, simply create a Route Filter and assign the BGP Community “Other Office 365 Services”.
To get access to that BGP Community, its much like any Office 365 service being accessed via ExpressRoute- that will need to have your Microsoft TAM approve the request as the Microsoft stance on using ExpressRoute for Office 365 traffic seesaw has swung again in the “you should really use the internet for Office 365, unless maybe Skype for Business/Teams latency is a problem”- again this is my experience.
 

Summary

  • ExpressRoute public peering has been on the radar to be deprecated for some time now
  • If you create new ExpressRoute circuits in parallel to your legacy ones, don’t expect to have the new ones work the same as legacy
    • I’ve even had the Azure product group “restore service” on a ASM/Classic ExpressRoute circuit that had Public peering, which did not restore service at all
    • We essentially spun up Microsoft peering and added the relevant Azure Route Filter
  • ARM ExpressRoute
    • Microsoft Peering has merged with Public peering so Microsoft peering does everything it did before + Public peering
    • Microsoft Peering requires RouteFilters to be applied to advertise routes from Azure to the customer
      • BGP Community tags are used inside of RouteFilters
    • As of August 1st 2018, ExpressRoute Public peering will no longer advertise Azure AD routes
      • This can be accessed via Microsoft Peering, using a Route Filter and the BGP Community tag of “Other Office 365 services”
    • No changes to Private peering at all – woohoo! (as of the date of writing this blog)
  • ASM/Classic ExpressRoute
    • You can’t provision a Classic ExpressRoute circuit anymore
    • If you have one, you’ve likely been bumped up to ARM, given the ASM portal is deprecated
    • Legacy ExpressRoute circuits that have been in-place since prior to August 1 2017, enjoy it while it lasts!
      • Any changes that you might need may be difficult to arrange- you’ll likely need to change the service to comply to current standards

Enjoy!

PowerShell gotcha when connecting ASM Classic VNETs to ARM ExpressRoute

Recently I was working on an Azure ExpressRoute configuration change that required an uplift from a 1GB circuit to a 10Gb circuit. Now thats nothing interesting, but, of note was using some PowerShell to execute a cmdlet.
A bit of a back story to set the scene here; and I promise it will be brief.
You can no longer provision Azure ExpressRoute circuits in the Classic or ASM deployment model. All ExpressRoute circuits that are provisioned now are indeed Azure Resource Manager (ASM) deployments. So there is a very grey area between what cmdlets to run on which PowerShell module.

The problem

The environment I was working with had a mixture of ASM and ARM deployments. The ExpressRoute circuit was re-created in ARM. When attempting to follow the Microsoft documentation to connect a VNET to that new circuit (available on this docs.microsoft.com page), I ran the following command–

Get-AzureDedicatedCircuitLink
New-AzureDedicatedCircuitLink -ServiceKey "[XXXX-XXXX-XXXX-XXXX-XXXX]" -VNetName "[MyVNet]"

–but PowerShell threw out the following error (even after doing the end to end process in a new PS ISE session):

Get-AzureDedicatedCircuit : Object reference not set to an instance of an object.
At line:1 char:1 +
Get-AzureDedicatedCircuit
+ ~~~~~~~~~~~~~~~~~~~~~~~~~     
+ CategoryInfo          : NotSpecified: (:) [Get-AzureDedicatedCircuit], NullReferenceException     
+ FullyQualifiedErrorId : System.NullReferenceException,Microsoft.WindowsAzure.Commands.ExpressRoute.GetAzureDedicatedCircuitCommand

Solution

This is actually quite a tricky one as there’s little to no documentation that outlines this specifically. It’s not too difficult though as theres only two things you need to do:

  1. Ensure you have the version 5.1.1 of the Azure PowerShell modules
    • Ensure you also have the Azure ExpressRoute PowerShell module
  2. Execute the all Import-Module commands and in the right order as per bellow

Note: If you have a newer version of the Azure PowerShell modules, like I did (version 5.2.0), import the older module as per bellow

Import-Module "C:\Program Files\WindowsPowerShell\Modules\Azure\5.1.1\Azure.psd1"
Import-Module "C:\Program Files\WindowsPowerShell\Modules\Azure\5.1.1\ExpressRoute\ExpressRoute.psd1

Then you can execute the New-AzureDedicatedCircuitLink command and connect your ASM VNET to ARM ExpressRoute.
Cheers!


Originally posted on Lucian.Blog. Follow Lucian on Twitter @LucianFrango.

Migrating resources from AWS to Microsoft Azure

Kloud receives a lot of communications in relation to the work we do and the content we publish on our blog. My colleague Hugh Badini recently published a blog about Azure deployment models from which we received the following legitimate follow up question…

So, Murali, thanks for letting us know you’d like to know more about this… consider this blog a starting point :).

Firstly though…

this topic (inter-cloud migrations), as you might guess, isn’t easily captured in a single blog post, nor, realistically in a series, so what I’m going to do here is provide some basics to consider. I may not answer your specific scenario but hopefully provide some guidance on approach.

Every cloud has a silver lining

The good news is that if you’re already operating in a cloud environment then you have likely had to deal with many of the fundamental differences between traditional application hosting and architecture and that of cloud platforms.

You will have dealt with how you ensure availability of your application(s) across outages; dealing with spikes in traffic via use of elastic compute resources; and will have come to recognise that is many ways, Infrastructure-as-a-Service (IaaS) in the cloud has many similarities to the way you’ve always done things on-prem (such as backups).

Clearly you have less of a challenge in approaching a move to another cloud provider.

Where to start

When we talk about moving from AWS to Azure we need to consider a range of things – let’s take a look at some key ones.

Understand what’s the same and what’s different

Both platforms have very similar offerings, and Microsoft provides many great resources to help those utilising AWS to build an understanding of which services in AWS map to which services in Azure. As you can see the majority of AWS’ services have an equivalent in Azure.

Microsoft’s Channel 9 is also a good place to start to learn about the similarities, with there being no better place than the Microsoft Azure for Amazon AWS Professional video series.

So, at a platform level, we are pretty well covered, but…

the one item to be wary of in planning any move of an existing application is how it has been developed. If we are moving components from, say, an EC2 VM environment to an Azure VM environment then we will probably have less work to do as we can build our Azure VM as we like (yes, as we know, even Linux!) and install whatever languages, frameworks or runtimes we need.

If, however, we are considering moving an application from a more Platform-as-a-Service capability such AWS Lambda we need to look at the programming model required to move its equivalent in Azure – Azure Functions. While AWS Lambda and Azure Functions are functionally the same (no pun intended) we cannot simply take our Lambda code and drop it into an Azure Function and have it work. It may not even make sense to utilise Azure Functions depending on what you are shifting.

It’s also important to consider the differences in the availability models in use today in AWS and Azure. AWS uses Availability Zones to help you manage the uptime of your application and it’s components. In Azure we manage availability at two levels – locally via Availability Sets and then geographically through use of Regions. As these models differ it’s an important area to consider for any migration.

Tools are good, but are no magic wand

Microsoft provides a way to migrate AWS EC2 instances to Azure using Azure Site Recovery (ASR) and while there are many tools for on-prem to cloud migrations and for multi-cloud management, they mostly steer away from actual migration between cloud providers.

Kloud specialises in assessing application readiness for cloud migrations (and then helping with the migration), and we’ve found inter-cloud migration is no different – understanding the integration points an application has and the SLAs it must meet are a big part of planning what your target cloud architecture will look like. Taking into consideration underlying platform services in use is also key as we can see from the previous section.

If you’re re-platforming an application you’ve built or maintain in-house, make sure to review your existing deployment processes to leverage features available to you for modern Continuous Deployment (CD) scenarios which are certainly a strength of Azure.

Data has a gravitational pull

The modern application world is entirely a data-driven one. One advantage to cloud platforms is the logically bottomless pit of storage you have at your disposal. This presents a challenge, though, when moving providers where you may have spent years building data stores containing Terabytes or Petabytes of data. How do you handle this when moving? There are a few strategies to consider:

  • Leave it where it is: you may decide that you don’t need all the data you have to be immediately available. Clearly this option requires you to continue to manage multiple clouds but may make economic sense.
  • Migrate via physical shipping: AWS provides Snowball as a way to extract data out of AWS without needing to pull it over a network connection. If your solution allows it you could ship your data out of AWS to a physical location, extract that data, and then prepare it for import into Azure, either over a network connection using ExpressRoute or through the Azure Import/Export service.
  • Migrate via logical transfer: you may have access to a service such as Equinix’s Cloud Exchange that allows you to provision inter-connects between cloud and other network providers. If so, you may consider using this as your migration enabler. Ensure you consider how much data you will transfer and what, if any, impact the data transfer might have on existing network services.

Outside of the above strategies on transferring of data, perhaps you can consider a staged migration where you only bring across chunks of data as required and potentially let older data expire over time. The type and use of data obviously impacts on which approach to take.

Clear as…

Hopefully this post has provided a bit more clarity around what you need to consider when migrating resources from AWS to Azure. What’s been your experience? Feel free to leave comments if you have feedback or recommendations based on the paths you’ve followed.

Happy dragon slaying!

Implementing Microsoft (Office365) Peering for ExpressRoute

Notes from the Field

I have recently been involved with an implementation of Microsoft Peering for Expressroute with a large Australian customer and thought I would share the experience with you.

Firstly, and secondly, make sure that you read the specific guidance from Microsoft regarding prerequisites for Microsoft Peering. (See below)

Configure Microsoft peering for the circuit

Make sure that you have the following information before you proceed.

  • A /30 subnet for the primary link. This must be a valid public IPv4 prefix owned by you and registered in an RIR / IRR.
  • A /30 subnet for the secondary link. This must be a valid public IPv4 prefix owned by you and registered in an RIR / IRR.
  • A valid VLAN ID to establish this peering on. Ensure that no other peering in the circuit uses the same VLAN ID.
  • AS number for peering. You can use both 2-byte and 4-byte AS numbers.
  • Advertised prefixes: You must provide a list of all prefixes you plan to advertise over the BGP session. Only public IP address prefixes are accepted. You can send a comma separated list if you plan to send a set of prefixes. These prefixes must be registered to you in an RIR / IRR.
  • Customer ASN: If you are advertising prefixes that are not registered to the peering AS number, you can specify the AS number to which they are registered. This is optional.
  • Routing Registry Name: You can specify the RIR / IRR against which the AS number and prefixes are registered.
  • An MD5 hash, if you choose to use one. This is optional.

https://azure.microsoft.com/en-us/documentation/articles/expressroute-howto-routing-classic/#microsoft-peering

Let me address the items in bold above.

AS Number for Peering: A public AS number is required in order to implement Microsoft Peering. The customer I was working with, didn’t own a public AS, and as a result, were required to apply to APNIC for a public AS. APNIC are the registrar for AS numbers in Asia Pacific

The prerequisites for applying for a public AS number in Australia are as follows, and have been taken from the APNIC site.

Your organization is eligible for an AS Number assignment if:

  • it is currently multihomed, or
  • it holds previously-allocated provider independent address space and intends to multihome in the future.

An organization will also be eligible if it can demonstrate that it will meet the above criteria upon receiving an AS Number (or within a reasonably short time afterwards).

I have heard along the grapevine from colleagues of mine, that this process can be quite time consuming, particularly when the above requirements are not clear, however this wasn’t the case for the customer in question, and the AS number was approved in a timely manner.

Advertised prefixes: This is the list of pubic IP prefixes which azure will receive requests from.

Important Note: Microsoft will run a validation process once the dedicatedcircuit for Microsoft Peering is established. I attempted to find out detailed information regarding the validation process however I was not able to obtain this information. What I was able to find out, is that a validation check is run against the Public AS number to verify that the customer who owns the tenant, also owns the AS. A similar validation check is completed against the AdvertisedPrefixes.

In our case, even though the customer owned both the AS, and the AdvertisedPrefixes, automatic validation failed.

In order to fix this issue, we opened a support call with Azure support who were able to perform a manual validation within about 10 mins of being in contact.

Once the virtual circuit was up, and the AdvertisedPrefixes value showing “configured”, we moved onto configuring the virtual circuit in the Equinix Cloud Exchange.

The customers’ ISP configured the virtual circuits correctly, however we were not able to successfully establish BGP Peering. Another support call was created, with all vendors involved and after some troubleshooting, it was discovered the Primary and Secondary Peer subnets had been configured in reverse and this was causing a mac address mismatch with Azure.

Fortunately, the solution to this issue was to simply recreate the virtual circuit via the cloud exchange portal using the same service key as was used originally. Once this was completed, BGP peering was successfully established.

Premium Sku

Although not documented anywhere (that I could find), Microsoft Peering requires a Premium sku type in order to be enabled.

If you attempt to enable Microsoft Peering on a virtual circuit with a “standard” sku type, you will be promptly rewarded with a powershell error telling you exactly this.

Quick Shoutout: Thanks to our Microsoft Technology Strategist “Scott Turner” for point this out to me ahead of time.

Implementation Process

Create Virtual Circuit

#Import Azure Expressroute modules
Import-Module 'C:\Program Files (x86)\Microsoft SDKs\Azure\PowerShell\ServiceManagement\Azure\Azure.psd1'
Import-Module 'C:\Program Files (x86)\Microsoft SDKs\Azure\PowerShell\ServiceManagement\Azure\ExpressRoute\ExpressRoute.psd1'

#Creating a new circuit for Sydney O365
$Bandwidth = 500
$CircuitName = "EquinixO365Sydney"
$ServiceProvider = "Equinix"
$Location = "Sydney"
New-AzureDedicatedCircuit -CircuitName $CircuitName `
-ServiceProviderName $ServiceProvider `
-Bandwidth $Bandwidth -Location $Location -Sku Premium

#Creating a new circuit for Melbourne O365
$Bandwidth = 500
$CircuitName = "EquinixO365Melbourne"
$ServiceProvider = "Equinix"
$Location = "Melbourne"
New-AzureDedicatedCircuit -CircuitName $CircuitName `
-ServiceProviderName $ServiceProvider `
-Bandwidth $Bandwidth -Location $Location -Sku Premium

Get your Service Keys

Get-AzureDedicatedCircuit

Enable BGP Peering

Set-AzureBGPPeering -AccessType Microsoft `
-ServiceKey "SYD Service Key" -PrimaryPeerSubnet "x.x.x.x/30" `
-SecondaryPeerSubnet "x.x.x.x/30" -VlanId "Enter Vlan Id" `
-PeerAsn "Enter Public ASN" `
-AdvertisedPublicPrefixes "Enter Advertised Prefixes" -RoutingRegistryName "APNIC"

Set-AzureBGPPeering -AccessType Microsoft `
-ServiceKey "MEL Service Key" -PrimaryPeerSubnet "x.x.x.x/30" `
-SecondaryPeerSubnet "x.x.x.x/30" -VlanId "Enter Vlan Id" `
-PeerAsn "Enter Public ASN" `
-AdvertisedPublicPrefixes "Enter Advertised Prefixes" -RoutingRegistryName "APNIC"

Important Note: Advertising default routes

Default routes are permitted only on Azure private peering sessions. In such a case, we will route all traffic from the associated virtual networks to your network. Advertising default routes into private peering will result in the internet path from Azure being blocked. You must rely on your corporate edge to route traffic from and to the internet for services hosted in Azure.

To enable connectivity to other Azure services and infrastructure services, you must make sure one of the following items is in place:

  • Azure public peering is enabled to route traffic to public endpoints
  • You use user defined routing to allow internet connectivity for every subnet requiring Internet connectivity.

Note: Advertising default routes will break Windows and other VM license activation. Follow instructions here to work around this.

https://azure.microsoft.com/en-us/documentation/articles/expressroute-routing/

Do not advertise the default route to Microsoft Peering as this will break things!

I hope this helps some of you down the track as at the time we completed implementation, there was very little documentation available on the Web for Microsoft Peering.

 

Azure ExpressRoute in Australia via Equinix Cloud Exchange

Microsoft Azure ExpressRoute provides dedicated, private circuits between your WAN or datacentre and private networks you build in the Microsoft Azure public cloud. There are two types of ExpressRoute connections – Network (NSP) based and Exchange (IXP) based with each allowing us to extend our infrastructure by providing connectivity that is:

  • Private: the circuit is isolated using industry-standard VLANs – the traffic never traverses the public Internet when connecting to Azure VNETs and, when using the public peer, even Azure services with public endpoints such as Storage and Azure SQL Database.
  • Reliable: Microsoft’s portion of ExpressRoute is covered by an SLA of 99.9%. Equinix Cloud Exchange (ECX) provides an SLA of 99.999% when redundancy is configured using an active – active router configuration.
  • High Speed speeds differ between NSP and IXP connections – but go from 10Mbps up to 10Gbps. ECX provides three choices of virtual circuit speeds in Australia: 200Mbps, 500Mbps and 1Gbps.

Microsoft provided a handy table comparison between all different types of Azure connectivity on this blog post.

ExpressRoute with Equinix Cloud Exchange

Equinix Cloud Exchange is a Layer 2 networking service providing connectivity to multiple Cloud Service Providers which includes Microsoft Azure. ECX’s main features are:

  • On Demand (once you’re signed up)
  • One physical port supports many Virtual Circuits (VCs)
  • Available Globally
  • Support 1Gbps and 10Gbps fibre-based Ethernet ports. Azure supports virtual circuits of 200Mbps, 500Mbps and 1Gbps
  • Orchestration using API for automation of provisioning which provides almost instant provisioning of a virtual circuit.

We can share an ECX physical port so that we can connect to both Azure ExpressRoute and AWS DirectConnect. This is supported as long as we use the same tagging mechanism based on either 802.1Q (Dot1Q) or 802.1ad (QinQ). Microsoft Azure uses 802.1ad on the Sell side (Z-side) to connect to ECX.

ECX pre-requisites for Azure ExpressRoute

The pre-requisites for connecting to Azure regardless the tagging mechanism are:

  • Two Physical ports on two separate ECX chassis for redundancy.
  • A primary and secondary virtual circuit per Azure peer (public or private).

Buy-side (A-side) Dot1Q and Azure ExpressRoute

The following diagram illustrates the network setup required for ExpressRoute using Dot1Q ports on ECX:

Dot1Q setup

Tags on the Primary and Secondary virtual circuits are the same when the A-side is Dot1Q. When provisioning virtual circuits using Dot1Q on the A-Side use one VLAN tag per circuit request. This VLAN tag should be the same VLAN tag used when setting up the Private or Public BGP sessions on Azure using Azure PowerShell.

There are few things that need to be noted when using Dot1Q in this context:

  1. The same Service Key can be used to order separate VCs for private or public peerings on ECX.
  2. Order a dedicated Azure circuit using Azure PowerShell Cmdlet (shown below) and obtain the Service Key and use the this to raise virtual circuit requests with Equinix.https://gist.github.com/andreaswasita/77329a14e403d106c8a6

    Get-AzureDedicatedCircuit returns the following output.Get-AzureDedicatedCircuit Output

    As we can see the status of ServiceProviderProvisioningState is NotProvisioned.

    Note: ensure the physical ports have been provisioned at Equinix before we use this Cmdlet. Microsoft will start charging as soon as we create the ExpressRoute circuit even if we don’t connect it to the service provider.

  3. Two physical ports need to be provisioned for redundancy on ECX – you will get the notification from Equinix NOC engineers once the physical ports have been provisioned.
  4. Submit one virtual circuit request for each of the private and public peers on the ECX Portal. Each request needs a separate VLAN ID along with the Service Key. Go to the ECX Portal and submit one request for private peering (2 VCs – Primary and Secondary) and One Request for public peering (2VCs – Primary and Secondary).Once the ECX VCs have been provisioned check the Azure Circuit status which will now show Provisioned.expressroute03

Next we need to configure BGP for exchanging routes between our on-premises network and Azure as a next step, but we will come back to this after we have a quick look at using QinQ with Azure ExpressRoute.

Buy-side (A-side) QinQ Azure ExpressRoute

The following diagram illustrates the network setup required for ExpressRoute using QinQ ports on ECX:

QinQ setup

C-TAGs identify private or public peering traffic on Azure and the primary and secondary virtual circuits are setup across separate ECX chassis identified by unique S-TAGs. The A-Side buyer (us) can choose to either use the same or different VLAN IDs to identify the primary and secondary VCs.  The same pair of primary and secondary VCs can be used for both private and public peering towards Azure. The inner tags identify if the session is Private or Public.

The process for provisioning a QinQ connection is the same as Dot1Q apart from the following change:

  1. Submit only one request on the ECX Portal for both private and public peers. The same pair of primary and secondary virtual circuits can be used for both private and public peering in this setup.

Configuring BGP

ExpressRoute uses BGP for routing and you require four /30 subnets for both the primary and secondary routes for both private and public peering. The IP prefixes for BGP cannot overlap with IP prefixes in either your on-prem or cloud environments. Example Routing subnets and VLAN IDs:

  • Primary Private: 192.168.1.0/30 (VLAN 100)
  • Secondary Private: 192.168.2.0/30 (VLAN 100)
  • Primary Public: 192.168.1.4/30 (VLAN 101)
  • Secondary Public: 192.168.2.4/30 (VLAN 101)

The first available IP address of each subnet will be assigned to the local router and the second will be automatically assigned to the router on the Azure side.

To configure BGP sessions for both private and public peering on Azure use the Azure PowerShell Cmdlets as shown below.

Private peer:

Public peer:

Once we have configured the above we will need to configure the BGP sessions on our on-premises routers and ensure any firewall rules are modified so that traffic can be routed correctly.

I hope you’ve found this post useful – please leave any comments or questions below!

Read more from me on the Kloud Blog or on my own blog at www.wasita.net.

Connection Options When Building An Azure Hybrid Cloud Solution

If your business is migrating workloads to Azure the chances are at some point you will probably want to create a form of private interconnect with Azure. There is more than one way to achieve this, so in this post I’ll take a look at what options you have and the most appropriate scenarios for each.

We’ll work through the connection types from simplest (and quickest to provision) to more complex (where you’ll need IP networking expertise and hardware).

Hybrid Connection

This is your baseline interconnect option and is tied to the BizTalk Services offering within Azure. At time of writing the only Azure-based services that can leverage Hybrid Connections are Web Apps (formerly Websites) and Mobile Apps (formerly Mobile Services).

Hybrid Connections are a great way to quickly get access to on-premises resources without the complexity involved in firewall or VPN setups. If you look at the official documentation you’ll see there is no mention of firewall rules or VPN setup!

Your on-premises resources must be running on Server 2008 R2 or above in order to leverage this service offering which at its most restricted can work over standard HTTP(S) ports and nothing more.

  • Benefits:
    • Quick to setup (requires no changes on-prem)
    • Typically “just works” with most corporate network edge configurations
    • Doesn’t require a Virtual Network to be configured in Azure
    • Can be shared between multiple Web and Mobile Apps
    • Good for exposing single on-prem resources to consumers in Azure (i.e. DB on-prem / web in Azure).
  • Drawbacks:
    • Your security team may be unhappy with you 🙂
    • Performance may not meet your needs beyond simple use cases
    • TCP services requiring dynamic ports aren’t supported (think FTP)
    • Tied to BizTalk Services and utilises a range of other Azure services such as ACS and Azure SQL Database
    • In Preview (no SLA) and not available in all Azure Regions
    • Limited use cases in Azure (Web and Mobile Apps).

Point-to-Site VPN

The next step up from Hybrid Connections is Point-to-Site (P2S) VPN connections. These connections allow you to use a downloaded client to provide an SSTP VPN between a single on-premises (or Azure based) resource and a Virtual Network (VNet) in Azure.

This VPN setup is a good way to test out simple-to-medium complexity hybrid scenarios or proof-of-concepts without the need for dedicated VPN hardware in your corporate environment.

When setting up a P2S VPN you have a few items you need to succeed:

  • the IP address range that will be used for clients when they connect
  • a subnet defined on your VNet for the Azure Gateway that will host VPN connections
  • a running Gateway instance that will allow your VPN clients to connect
  • a valid x509 certificate that will be used by the VPN client.

As you can see, there are quite a few extra steps involved beyond the Hyrbid Connection! You can run up to 128 on-prem clients connected to an Azure VNet if needed.

  • Benefits:
    • Does not require dedicated on-premises networking hardware
    • SSTP can usually connect OK with most corporate network edge configurations
    • Can co-exist with Site-to-Site connections
    • Allows you expose services on a single on-prem resource to an entire Azure VNet.
  • Drawbacks:
    • You’ll need to understand IP networking to setup a VNet in Azure
    • Performance is still relatively limited due to the nature of SSTP
    • Isn’t an ‘always on’ proposition as requires an interactive user session on-prem to run the client
    • Only supports connection from a single on-prem resource running on Windows
    • You’ll need an x509 certificate for use with the VPN client.

Site-to-Site VPN

Now we start to get serious!

If you want to run a Site-to-Site (S2S) connection you will need to have dedicated VPN hardware or Windows Server 2012 (or above) running RRAS on-prem and some private IP address space for your Azure environment that doesn’t overlap with the on-premises network you’ll be connecting with.

This option is the first to really offer you a true hybrid environment where two networks can connect via the VPN. This is often the first step we see many enterprises take when adopting Azure as it is relatively quick to stand up and typically most customers have the necessary devices (or ones that meet Azure’s VPN requirements) available already.

When you setup your Gateway in Azure, the Azure platform will even handily provide you with a configuration script/template for whichever on-prem device you’ve selected.

  • Benefits:
    • Provides full network-to-network connectivity
    • Supports a growing number of standard VPN appliances
    • Foundation of support for multi-site connectivity
    • Can use Windows Server 2012 RRAS if you don’t have an appliance.
  • Drawbacks:
    • Maximum throughput of 100 Mbps
    • Doesn’t support redundant single site to single VNet connections.

Be aware: Forced Tunnelling

Before we move on to the next Azure connection type we do need to talk about Forced Tunelling. The current generation Azure VNet has a default route for all public Internet traffic which is out over Azure’s managed Internet infrastructure (it’s just there and you can’t manage it or turn it off). On some other public cloud platforms you can disable public internet traffic by not adding an Internet Gateway – on Azure that option isn’t currently available.

In order to mitigate some challenges around controlling the path public traffic takes from an Azure VNet, Microsoft introduced Forced Tunelling which can be used to force traffic bound for the Internet back over your VPN and into your on-prem environment.

You must plan your subnets appropriately and only apply Forced Tunelling to those where required. This is especially important if you will consume any of Azure’s PaaS offerings other than Web or Worker Roles which can be added to an Azure VNet.

Almost all of Azure’s PaaS services (even Blob Storage) are exposed as secured public Internet endpoints which means any call to these from a VNet configured for Forced Tunelling will result in that traffic heading back to your on-prem network and out your own Internet Gateway. Performance will take a hit and you will pay data egress charges on those calls as well as they will appear to originate from your on-prem Internet Gateway.

ExpressRoute

The grandpappy of all of them – and the one that requires the most planning and commitment. If you find yourself starting your hybrid journey here then either you have an existing investment in an MPLS WAN or you’re already co-located in an exchange that is providing ExpressRoute services.

The are two connection options for ExpressRoute:

  • Network Service Provider (NSP): utilises a new or existing MPLS WAN cross-connect into one or more Azure Region. Speeds 10 Mbps to 1 Gbps supported.
  • Exchange Provider (IXP): uses a new paired cross-connect in a data centre location when the IXP and Microsoft’s routers are co-located. Speeds 200 Mbps to 10 Gbps supported.

The officially support list of NSPs and IXPs is pretty small, but you can quite often work with your existing provider to get a connection into an IXP, or look to leverage offerings such as Equinix’s Cloud Exchange as a shortcut (for example, in Sydney 130+ network service providers provide services into Equinix).

Once you’re operating at this level you will definitely need the networking team in your organisation involved as you’ll be doing heavy lifting that requires solid knowledge of IP networking and specifically BGP.

  • Benefits:
    • A single ExpressRoute circuit can connect to multiple Azure VNets (up to 10) and across multiple Azure Subscriptions (also up to 10)
    • Redundant connection by default (a pair is provided when you connect)
    • Two peers provided: one for Azure public services and one for your private services. You can choose to not use either peer if you wish
    • Can support bursting to higher bandwidth levels (provider depending)
    • Offers an SLA for availability.
  • Drawbacks:
    • Requires that you have a relationship with an NSP or IXP in addition to Azure.
    • NSP bandwidth maximum is 1 Gbps
    • Maximum 4,000 route prefixes for BGP on all peers on a connection.

If you’re unsure how to get started here, but you have an existing WAN or co-location facility it may be worth talking to them about how to get a connection into Azure.

Be Aware: Default Routes and Public Peering

This topic falls under the same category as the earlier section on Forced Tunnelling for S2S VPNs.

When using ExpressRoute you can use BGP to advertise the default route for all your Azure VNets to be back over your ExpressRoute connection to your on-prem environment. Unlike the VPN connection scenario though, where all Azure PaaS services will route back over your on-prem Internet gateway, with ExpressRoute’s peering you can use the public peer as the shortcut back to Azure.

While this is a better option than you get with VPN it still means you are pushing Azure calls back to your ExpressRoute gateway so you will potentially see a performance hit and will see the data included if you are using an IXP connection.

Conclusion

So there we have it – a quick rundown of the techniques you have at your disposal when looking to create a private hybrid network environment that allows you to connect your existing locations with Azure.

HTH.

Follow Us!

Kloud Solutions Blog - Follow Us!