Akamai Cloud based DNS Black Magic

Let us start with traditional DNS hosting with any DNS hoster or ISP. How does traditional DNS name resolution works? When you type a human readable name http://www.anydomain.com on the address bar of internet explorer, that name is resolved to an Internet Protocol (IP) address hosted by an Internet Service Provider (ISP). The browser presented the website to the user. By doing so, the website exposed the public IP address to everyone. The good and bad guys know the IP address and can trace globally. A state sponsored hackers or private individual can launch a denial of service attack also known as DDoS on a website whose publicly IP address is known and traceable. The bad guys can send overwhelming number of fake service request to the original IP address of the human readable name http://www.anydomain.com and shut the website. In this situation, DNS server hosting DNS record http://www.anydomain.com will stop serving genuine DNS request resulting distributed denial of service (DDoS).

Akamai introduced Fast DNS that is dynamic DNS located almost every country, state, territory and regions to mitigate such risk of DDoS and DNS hijack.

Akamai Fast DNS offloads domain name resolution from on-premises infrastructure and traditional domain name provider to an intelligent, secure and authoritative DNS service.  Akamai has successfully prevented DDoS attack, DNS forgery and manipulation by complex dynamic DNS hosting and spoof IP addresses.

As of today Akamai has more than 150000+ servers located in more than 2000+ locations in the world that are very well connected in 1200+ networks in 700+ cities in 92 countries and in most cases, an Akamai edge server is just a hop away from the end users.

How does it works?

  1. User request http://www.anydomain.com
  2. User’s ISP respond the DNS name query http://www.anydomain.com
  3. User’s ISP resolves http://www.anydomain.com DNS Name to http://www.anydomain.com.edgekey.net hosted by Akamai
  4. Akamai Global DNS checks the CNAME http://www.anydomain.com.edgekey.net and the region of the request coming from
  5. Akamai Global then forward the request to the Akamai regional DNS Server for example Sydney, Australia
  6. Akamai regional DNS server forward the request to nearest Akamai edge server of the user location for example Melbourne, Australia
  7. Akamai local DNS server for example Melbourne, Australia resolve the original CNAME http://www.anydomain.com to http://www.anydomain.com.edgekey.net
  8. http://www.anydomain.com.edgekey.net resolve to cached (if cached) website http://www.anydomain.com by Akamai which then presented to user’s browser

Since Akamai uses dynamic DNS server, it is extremely difficult for a bad guy to track down the real IP address of the website and origin host of the website. In Akamai terminology, .au or .uk means that the website is hosted in that country (au or uk) but the response of the website is coming to the user from his/her geolocation hence IP address of the website will always be presented from the Akamai edge server of the user’s geolocation. In plain English, origin host and IP address is vanished in the complex dynamic DNS servers of Akamai. For example,

  1. http://www.anydomain.com.edgekey.net resolve to a spoof IP address hosted by Akamai DNS server
  2. The original IP address of http://www.anydomain.com is never resolved by Akamai DNS server or the ISP hosting the http://www.anydomain.com

Implementing Akamai Fast DNS:

  1. Create a Host A record in your ISP http://www.anydomain.com and point to 201.17.xx.xx public IP (VIP of Azure Web Services or any web services)
  2. Create an origin host record or CNAME record http://www.anydomain.com and point to xyz9013452bcf.anaydomain.com
  3. Now request Akamai to black magic http://www.anydomain.com and point to http://www.anydomain.com.edgekey.net
  4. Once Akamai completes the black magic, request your ISP to create another CNAME record xyz9013452bcf.anydomain.com and point to http://www.anydomain.com.edgekey.net

Testing Akamai Fast DNS: I am using http://www.akamai.com as the DNS name instead of a real DNS of record of any of my client.

Go to mxtoolbox.com and DNS lookup, http://www.akamai.com you will see

CNAME http://www.akamai.com  resolve to http://www.akamai.com.edgekey.net

Open command Prompt and ping http://www.akamai.com.edgekey.net

Since I am pinging from Sydney Australia, my ping responded by the Akamai edge server Sydney, result is

Ping http://www.akamai.com.edgekey.net

Pinging e1699.dscc.akamaiedge.net [118.215.118.16] with 32 bytes of data:

Reply from 118.215.118.16: bytes=32 time=6ms TTL=56

Reply from 118.215.118.16: bytes=32 time=3ms TTL=56

Open a browser and go to http://www.kloth.net/services/dig.php and trace e1699.dscc.akamaiedge.net

; <<>> DiG 9 <<>> @localhost e1699.dscc.akamaiedge.net A

; (1 server found)

;; global options: +cmd

.                                            375598   IN            NS           d.root-servers.net.

.                                            375598   IN            NS           c.root-servers.net.

.                                            375598   IN            NS           i.root-servers.net.

.                                            375598   IN            NS           j.root-servers.net.

.                                            375598   IN            NS           k.root-servers.net.

.                                            375598   IN            NS           m.root-servers.net.

.                                            375598   IN            NS           a.root-servers.net.

.                                            375598   IN            NS           l.root-servers.net.

.                                            375598   IN            NS           e.root-servers.net.

.                                            375598   IN            NS           f.root-servers.net.

.                                            375598   IN            NS           b.root-servers.net.

.                                            375598   IN            NS           g.root-servers.net.

.                                            375598   IN            NS           h.root-servers.net.

;; Received 228 bytes from 127.0.0.1#53(127.0.0.1) in 3 ms

net.                                       172800   IN            NS           a.gtld-servers.net.

net.                                       172800   IN            NS           b.gtld-servers.net.

net.                                       172800   IN            NS           c.gtld-servers.net.

net.                                       172800   IN            NS           d.gtld-servers.net.

net.                                       172800   IN            NS           e.gtld-servers.net.

net.                                       172800   IN            NS           f.gtld-servers.net.

net.                                       172800   IN            NS           g.gtld-servers.net.

net.                                       172800   IN            NS           h.gtld-servers.net.

net.                                       172800   IN            NS           i.gtld-servers.net.

net.                                       172800   IN            NS           j.gtld-servers.net.

net.                                       172800   IN            NS           k.gtld-servers.net.

net.                                       172800   IN            NS           l.gtld-servers.net.

net.                                       172800   IN            NS           m.gtld-servers.net.

;; Received 512 bytes from 2001:7fd::1#53(2001:7fd::1) in 8 ms

akamaiedge.net.                  172800   IN            NS           la1.akamaiedge.net.

akamaiedge.net.                  172800   IN            NS           la3.akamaiedge.net.

akamaiedge.net.                  172800   IN            NS           lar2.akamaiedge.net.

akamaiedge.net.                  172800   IN            NS           ns3-194.akamaiedge.net.

akamaiedge.net.                  172800   IN            NS           ns6-194.akamaiedge.net.

akamaiedge.net.                  172800   IN            NS           ns7-194.akamaiedge.net.

akamaiedge.net.                  172800   IN            NS           ns5-194.akamaiedge.net.

akamaiedge.net.                  172800   IN            NS           a12-192.akamaiedge.net.

akamaiedge.net.                  172800   IN            NS           a28-192.akamaiedge.net.

akamaiedge.net.                  172800   IN            NS           a6-192.akamaiedge.net.

akamaiedge.net.                  172800   IN            NS           a1-192.akamaiedge.net.

akamaiedge.net.                  172800   IN            NS           a13-192.akamaiedge.net.

akamaiedge.net.                  172800   IN            NS           a11-192.akamaiedge.net.

;; Received 504 bytes from 2001:503:a83e::2:30#53(2001:503:a83e::2:30) in 14 ms

dscc.akamaiedge.net.          8000       IN            NS           n7dscc.akamaiedge.net.

dscc.akamaiedge.net.          4000       IN            NS           n0dscc.akamaiedge.net.

dscc.akamaiedge.net.          6000       IN            NS           a0dscc.akamaiedge.net.

dscc.akamaiedge.net.          6000       IN            NS           n3dscc.akamaiedge.net.

dscc.akamaiedge.net.          4000       IN            NS           n2dscc.akamaiedge.net.

dscc.akamaiedge.net.          6000       IN            NS           n6dscc.akamaiedge.net.

dscc.akamaiedge.net.          4000       IN            NS           n5dscc.akamaiedge.net.

dscc.akamaiedge.net.          8000       IN            NS           n1dscc.akamaiedge.net.

dscc.akamaiedge.net.          8000       IN            NS           n4dscc.akamaiedge.net.

;; Received 388 bytes from 184.85.248.194#53(184.85.248.194) in 8 ms

e1699.dscc.akamaiedge.net. 20        IN            A             23.74.181.249

;; Received 59 bytes from 77.67.97.229#53(77.67.97.229) in 5 ms

Now tracert 23.74.181.249 on a command prompt

Tracert 23.74.181.249

Tracing route to a23-74-181-249.deploy.static.akamaitechnologies.com [23.74.181.249]

over a maximum of 30 hops:

1     1 ms     1 ms     1 ms  172.28.67.2

2     4 ms     1 ms     4 ms  172.28.2.10

3     *        *        *     Request timed out.

4     *        *        *     Request timed out.

5     *        *        *     Request timed out.

6                     *     Request timed out.

7     *        *        *     Request timed out.

8     *      125 ms    75 ms  bundle-ether1.sydp-core04.sydney.reach.com [203.50.13.90]

9   172 ms   160 ms   165 ms  i-52.tlot-core02.bx.telstraglobal.net [202.84.137.101]

10   152 ms   192 ms   164 ms  i-0-7-0-11.tlot-core01.bi.telstraglobal.net [202.84.251.233]

11   163 ms   183 ms   176 ms  gtt-peer.tlot02.pr.telstraglobal.net [134.159.63.182]

12   151 ms   157 ms   155 ms  xe-2-2-0.cr2-lax2.ip4.gtt.net [89.149.129.234]

13   175 ms   160 ms   154 ms  as5580-gw.cr2-lax2.ip4.gtt.net [173.205.59.18]

14   328 ms   318 ms   317 ms  ae21.edge02.fra06.de.as5580.net [78.152.53.219]

15   324 ms   325 ms   319 ms  78.152.48.250

16   336 ms   336 ms   339 ms  a23-74-181-249.deploy.static.akamaitechnologies.com [23.74.181.249]

Now open hosts file of windows machine C:\WINDOWS\system32\drivers\etc\hosts and add Akamai spoof IP 172.233.15.98   http://www.akamai.com (reference)

Browse http://www.akamai.com website on internet explorer that will point you to 172.233.15.98

Open command prompt, nslookup 172.233.15.98

Server:  lon-resolver.telstra.net

Address:  203.50.2.71

Name:    a172-233-15-98.deploy.static.akamaitechnologies.com

Address:  172.233.15.98

In conclusion, Akamai tricked web browser to go to Akamai edge server Sydney Australia instead of original Akamai server hosted in USA. An user will never know the original IP address of the http://www.akamai.com website. Abracadabra the original IP address is vanished…

Exchange Server 2016 in Azure

I recently worked on a project where I had to install Exchange Server 2016 on an Azure VM and I chose a D2 sized Azure VM (2 cores, 7GB RAM) thinking that will suffice, well that was a big mistake.

The installation made it to the last step before a warning appeared informing me that the server is low on memory resources and eventually terminated the installation, leaving it incomplete.

Let this be a warning to the rest of you, choose a D3 or above sized Azure VM to save yourself a whole lot of agony.

To try and salvage the Exchange install I attempted to re-run the installation as it detects an incomplete installation and tries to pick up where it failed previously, this did not work.

I then tried to uninstall Exchange completely by running command: “Setup.exe /mode:Uninstall /IAcceptExchangeServerLicenseTerms”. This also did not work as it was trying to uninstall an Exchange role that never got installed, this left me one option manually remove Exchange from Active Directory and rebuild the Azure VM. 

To remove the Exchange organisation from Active Directory I had to complete the following steps;

  1. On a Domain Controller | Open ADSI Edit
  2. Connect to the Configuration naming contextconfig-naming-context
  3. Expand Services
  4. Delete CN=Microsoft Exchange and CN=Microsoft Exchange Autodiscoverconfig-exchange-objects
  5. Connect to the Default naming contextdefault-naming-context
  6. Under the root OU delete OU=Microsoft Exchange Security Groups and CN=Microsoft Exchange System Objects delete-exchange-objects
  7. Open Active Directory Users and Computers
  8. Select the Users OU
  9. Delete the following:
    • DiscoverySearchMailbox{GUID}
    • Exchange Online-ApplicationAccount
    • Migration.GUID
    • SystemMailbox{GUID}ad-exchange-objects

After Exchange was completely removed from Active Directory and my Azure VM was rebuilt with a D3 size I could successfully install Exchange Server 2016.

Exchange Server 2016 install error: “Active Directory could not be contacted”

I recently worked on a project where I had to install Exchange Server 2016 on an Azure VM and received error “Active Directory could not be contacted”.

To resolve the issue, I had to complete the following steps;

  1. Remove the Azure VM public IP address
  2. Disable IPv6 on the NICipv6-disabled
  3. Set the IPv4 DNS suffix to point to your domain. If a public address is being used it will be set to reddog.microsoft.com by default.dns-suffix

Once done the installation could proceed and Active Directory was contactable.

Break down your templates with Linked Templates (Part 1)

Templated deployment is one of the key value propositions of moving from the Azure classic to Resource Manager (ARM) deployment model.  This is probably one key feature that made a big stride towards Infrastructure as a Code (IAC).  Personally, I have been looking forward to this feature since it’s a prominent feature on the other competing platform.

Now that this feature is live for a while, one aspect which I found interesting is the ability to link templates in Azure Resource Manager.  This post is part of a three-part series highlighting ways you could deploy linked templates.  The first part of the series describes the basics – building a template that creates the base environment.  The second part will demonstrate the linked template.  Third part will delve into a more advanced scenario with KeyVault and how we can secure our linked templates.

Why linked templates?  We could get away with one template to build our environment, but is it a good idea?  If your environment is pretty small, sure go ahead.  If the template becomes un-manageable for you which is mostly the case if you are serious about ‘templating’, then you come to the right place – linking or nesting templates is a way for you to de-couple your templates to manageable chunks.  This allows you to ‘branch’ out templates development, especially when you have more than one person; resulting in smaller, manageable templates than a one big template.  Plus, this would make testing / debugging a little easier as you have a smaller scope to work off.

Linking or nesting templates is a way for you to de-couple your templates to manageable chunks.  This allows you to ‘branch’ out templates development, especially when you have more than one person; resulting in smaller, manageable templates than a one big template.  Plus, this would make testing / debugging a little easier as you have a smaller scope to work off.

OK, so let’s start with the scenario that demonstrates linked templates.  We will build a two-tier app consisting of two web servers and a database server.  This includes placing the two web servers in an availability set with a virtual network (VNET) and storage account to host the VMs.  To decouple the components, we will have three templates, one template that defines the base environment (VNET, subnets, and storage account).  The second template will build the virtual machines (web servers) in the front-end subnet, and finally the 3rd template will build the database server.

AzTemplate-1

First template (this blog post)

AzTemplate-2

Second template

AzTemplate-3

Third template

AzTemplate-4

The building process

First, create a resource group, this example relies on a resource group for deployment.  You will need to create the resource group now or later during deployment.

Once you have a resource group created, set up your environment – SDK, Visual Studio, etc. You can find more info here.  Visual Studio is optional but this demo will use this to build & deploy the templates.

Create a new project and select Azure Resource Group.  Select Blank templateAzTemplate-5

This will create a project with PowerShell deployment script and two JSON files: azuredeploy and azuredeploy.parameters.  The PowerShell script is particularly interesting as it has everything you need to deploy the Azure templates.

AzTemplate-6

We will start by defining the variables in the azuredeploy template.   Note this can be parameterised for a better portability.

Then we define the resources – storage account and virtual network (including subnets).

We then deploy it – in Visual Studio (VS) the process is simply by right-clicking project and select New Deployment… then Deploy.  If you didn’t create a resource group, you can create it here also.

AzTemplate-7

The template will deploy and a similar message should be displayed once it’s finished.

Successfully deployed template ‘w:\github-repo\linkedtemplatesdemo\linkedtemplatesdemo\templates\azuredeploy-base.json‘ to resource group ‘LinkedTemplatesDemo‘.

On the Azure portal you should see the storage account and virtual network created.

AzTemplate-8AzTemplate-9

The next part in the series describes how we modify this template to call the web and db servers templates (linked templates).  Stay tuned :).

 

 

 

How to create custom images for use in Microsoft Azure

In this post I will discuss how we can create custom virtual machine images and deploy them to the Microsoft Azure platform. To complete this process you will need an Azure Subscription, the Azure PowerShell module installed and a pre-prepared VHD which you would like to use (VHDX is not supported at present.)

You can sign up for a free trial of Microsoft Azure here if you don’t currently hold a subscription.

Completing this process will allow you take advantage of platforms which aren’t offered “out of the box” on Microsoft Azure eg, Server 2003 and Server 2008 for testing and development. Currently Microsoft offers Server 2008 R2 as the minimum level from the Azure Image Gallery.

What do I need to do to prepare my image?

To complete this process, I built a volume license copy of Windows Server 2008 Standard inside a generation one Hyper-V guest virtual machine. Once the installation of the operating system completed I installed Adobe Acrobat Reader. I then ran sysprep.exe to generalise the image. This is important, if you don’t generalise your images, they will fail to deploy on the Azure platform.

I will detail the steps carried out after the operating system install below.

  1. Log into the newly created virtual machine
  2. Install the Hyper-V virtual machine additions (if your guest doesn’t already have it installed)
  3. Install any software that is required in your image (I installed Acrobat Reader)
  4. From an Administrative command prompt, navigate to %windir%\system32\sysprep and then execute the command “sysprep.exe”

  1. Once the SysPrep window has opened, select Enter System Out of Box Experience (OOBE) and tick the Generalize check box. The shutdown action should be set to Shutdown, this will shut down the machine gracefully once the sysprep process has completed.
  2. Once you are ready, select OK and wait for the process to complete.

I built my machine inside a dynamically expanding VHD, the main reason for doing so was to avoid having to upload a file size which was larger than necessary. As a result of this, I chose to compact the VHD before moving on to the next step by using the disk wizard inside the Hyper-V management console. To complete this process, follow the steps below.

  1. From the Hyper-V Host pane, select Edit Disk
  2. Browse to the path of VHD we were working on, in my case it is “C:\VHDs\Server2008.vhd” and select Next
  3. Select Compact and Finish.
  4. Wait for the process to complete. Your VHD file is now ready to upload.

What’s next?

We are now ready to upload the virtual machine image, to complete this process you will need access to the Azure PowerShell cmd-lets and a storage account for the source VHD. If you do not already have a storage account created, you can follow the documentation provided by Microsoft here.

IMPORTANT: Once you have a storage account in Azure, ensure that you have a container called VHDs. If you don’t have a container you can create on by selecting Add from the bottom toolbar, name it vhds and ensure the access is set to Private (container shown below.)


We are now ready to connect to the Azure account to kick off the upload process. To do so, launch an Administrative Azure PowerShell console and follow the following steps.

  1. Run the cmd-let Add-AzureAccount, this will present a window which will allow you to authenticate to Azure.

  1. On the next screen, enter your Password. The PowerShell session is now connected.
  2. To verify that the session connected successfully, run the cmd Get-AzureAccount, you should see your account listed below.

We are now ready to commence the upload process. You will need your storage blob URL. You can find this on the container page we visited previously to create the vhds container.

The complete command is as follows.

Add-AzureVhd -Destination “<StorageBlobURL>/vhds/Server2008.vhd” -LocalFilePath “C:\VHDs\Server2008.vhd”

Once you have executed the command, two things happen..

  1. The VHD file is indexed by calculating the MD5 hash

  1. Once the indexing process is completed, the upload starts.


This is very neat, as the demo gods often fail us… (my upload actually failed part way through.) Thankfully I was able to re-execute the command, which resumed the upload process where the first pass left off (see below.)

  1. Wait for the upload process to complete.

Creating the Image in the Azure console.

Now that our upload has completed, we are ready to create an image in the Azure console. This will allow us to easily spawn virtual machines based on the image we uploaded earlier. To complete this process you will need access to the Azure console and your freshly uploaded image.

  1. Select Virtual Machines from the management portal.
  2. Select Images from the virtual machines portal.
  3. Select Create an Image

  1. A new window titled Create an image from a VHD will pop up. Enter the following details (as shown below.)
  • Name
  • Description
  • VHD URL (from your storage blob)
  • Operating System Family


Ensure you have ticked I have run Sysprep on the virtual machine or you will not be able to proceed.

  1. The Image will now appear under MY IMAGES in the image gallery.

Deploying the image!

All the work we have completed so far won’t be much use if the deployment phase fails. In this part of the process we will deploy the image to ensure it will work as expected.

  1. Select Virtual Machines from the management portal.
  2. Select New > Compute > Virtual Machine > From Gallery
  3. From the Choose an Image screen, select MY IMAGES. You should see the image that we just created in the gallery (shown below.)

  1. Select the Image and click Next
  2. Complete the Virtual Machine Configuration with your desired settings.
  3. Wait for the virtual machine to complete deployment provisioning.

Connecting to the virtual machine.

The hard work is done! We are now ready to connect to our newly deployed machine to ensure it is functioning as expected.

  1. Select Virtual Machines from the management portal.
  2. Select the Virtual Machine and then click Connect from the toolbar down the bottom. This will kick-off a download for the RDP file which will allow you to connect to the virtual machine.
  3. Launch the RDP file, you will be asked to authenticate. Enter the credentials you specified during the deployment phase and click OK


  1. You will now be presented with your remote desktop session, connected to your custom image deployed on Microsoft Azure.

I went ahead and activated my Virtual Machine. To prove there is no funny business involved, I have provided one final screenshot showing the machine activation status (which details the Windows version) and a snip showing the results of the ipconfig command. This lists the internal.cloudapp.net addresses showing that machine is running on Microsoft Azure.

Enjoy!