The quickest way to create new VMs in Azure from existing VM snapshots, mostly with PowerShell

Originally posted on Lucian’s blog @clouduccino.com. Follow Lucian on Twitter @LucianFrango.


There’s probably multiple ways to do this, both right and wrong, but, here’s a process that I’ve been using for a while that I’ve recently tweaked to take advantage of new Azure Managed Disks.

Sidebar – standard managed disk warning

Before I go on though, I wanted to issue a quick warning about the differences between standard unmanaged and managed disks. Microsoft will be pushing you to you Managed Disks more and more. Yes, its a great feature that makes the management of VM disks simpler. The key bit of information though is as follows:

  • If you provision an unmanaged disk that is 1Tb in size, but, only use 100Gb, you are paying for 100Gb of storage costs. So you’re only paying for what you use. [1. Unmanaged disk cost – Azure Documentation ]
  • If you provision a managed disk that is 1Tb in size, but, only you 10Mb, you will be paying for the privilege of the whole 1Tb disk [2. Managed disk cost – Azure Documentation ]
  • Additionally, Premium disks, you’re paying for what you provision no matter if its managed or unmanaged

That aside, Managed Disks are a pretty good feature that makes disk and storage account management considerably simpler. If you’re frugal with your VM allocation and have the process to manage people and technology correctly, Managed Disks are great.

The Process

tl;dr

  • Create a snapshot in Azure
  • Copy the snapshot from snapshot storage location to Blob storage
  • Create a new VM instance based on the blob.vhd file
    • This blob post outlines the use of managed disks
    • However, mounting direct from Blob can also be done

The actual process

I’ve gone through this recently and updated it so that it’s as streamlined, for me, as possible. Again, this is skewed towards managed disk usage, but, can easily be extended to be used with unmanaged disks as well. Lets begin:

Step 0 ?

If you’re wanting to do this to create copies of your VM instances, to scale out your workload, remember to generalise or sysprep your VM instance prior to Step 1. In the example I go into below, my use case was to create a copy of a server from a production environment (VNET and subscription) and move it to different and seperate non-production environment (seperate VNET and subscription).

Step 1 – Create a snapshot of your VM disk(s)

The first thing we need to do is actually power off your virtual machine instance. I’ve seen that snapshots can happen while the VM instance is running, but, I guess you can call me a a little bit more old school, a little bit more on the cautious side when it comes to these sorts of things. I’ve been bitten by this particular bug in the past, unpleasant it was; so i’m inclined to err on the side of caution.

Once the VM instance is offline, go to the Azure Portal and search for “Snapshots”. Create a new snapshot.

  • NOTE: snapshots in Azure are done per DISK and not per VM INSTANCE
  • Name the snapshot
  • Select the subscription where the VM instance is located
  • Select the resource group you want to save the snapshot to
    • Or create a new one
  • Select the snapshot location
  • Select the source disk
    • If you earlier selected the same resource group where your VM instance is contained, the disk selection will display the resource group member VM instance disks first in the list
  • Select the storage type- standard or premium for your snapshot
    • I usually just use standard as I’ve not had the need for faster speed premium as yet (that will change one day for sure)
  • Create the snapshot

One the snapshot is created, complete this quick next step to generate an export access URL (we’ll need this in step 2):

  • Select the snapshot
  • From the top menu, select Export
  • You’ll be presented with a menu item with a time interval (based in seconds)
  • The default is 3600 or 1 hour
  • That is fine, but, I like to make that 36000 (add another 0) so that I have a whole day to do this again and again if need be
  • Save the generated URL to notepad for later!

Step 2 – Copy the snapshot to Blob

The next part relies on PowerShell. Update the following PowerShell script with your parameters to copy the snapshot to Blob:

$storageAccountName = "<storage account name>"
$storageAccountKey = “<storage account key>
$absoluteUri = “https://blahblahblah.blob.core.windows.net/blahblahblah/........
$destContainer = “<container>”
$blobName = “server.vhd

$destContext = New-AzureStorageContext –StorageAccountName $storageAccountName -StorageAccountKey $storageAccountKey
Start-AzureStorageBlobCopy -AbsoluteUri $absoluteUri -DestContainer $destContainer -DestContext $destContext -DestBlob $blobName

Just for your info, heres a quick explanation of the above:

  • Storage account name = there storage account where you want to store the VHD
  • The storage account key = either the primary or secondary  which is used for authentication for accessing the storage account
  • The absolute URI = this is the snapshot URI we generated at the end of step 1
  • The destination container = where you want to store the VHD. Usually this is either “vhds”,  or maybe create one called “snapshots”
  • The blob name = the file name of the VHD itself (remember to only use lowercase)

Step 2.5 – Moving around the blob if need be

Before we actually create a new VM instance based on this snapshot blob, there is an additional option we could take. That is, perhaps it would make sense to move the blob to a different subscription. This is particularly handy when you would have a development environment that you would want to move to production. Other use cases might be the inverse- making a replica of a production system for development purposes.

The absolute fastest way to do this, as I don’t like being inefficient here is with the Azure Storage Explorer (ASE) tool. Its an application that provides a quick GUI for completing storage actions. If you add in both the storage accounts in the ASE, you can as easily as this:

  • Select the blob from storage account A (in subscription A)
  • Select copy from the top menu
  • Go to the your second storage account (storage account B in perhaps subscription B)
  • Go to the relevant container
  • Select paste from the top menu
  • Wait for the blob to copy
  • DONE

It can’t get any simpler or faster than that. I’m sure if you’re command line inclined, you have a quick go to PowerShell cmdlet for that, but, for me, I’ve found that to be pretty damn quick. So it isn’t broken, why fix it.

Step 3 – Create a new VM with a managed disk based on the snapshot we’ve put into Azure Blob

The final piece of the puzzle, as the cliche would go, is to create a new virtual machine instance. Again, as the wonderfully elusive and vague title of this blog post states, we’ll use PowerShell to do this. Sure, ARM templates would work and likely the Azure Portal can get you pretty far as well. However, again I like to be efficient and I’ve found that the following PowerShell script does this the best.

Additionally, you can change this up to mount the VHD from blob, vs create a new managed disk as well. So, for purpose of creating a new machine, PowerShell is as flexible as it is fast and convenient.

Here’s the script you’ll need to create the new VM instance:

#Prepare the VM parameters 
$rgName = "<resource-group-name>"
$location = "australiaEast"
$vnet = "<virtual-network>"
$subnet = "/subscriptions/xxxxxxxxx/resourceGroups/<resource-group-name>/providers/Microsoft.Network/virtualNetworks/<virtual-network>/subnets/<subnet>"
$nicName = "VM01-Nic-01"
$vmName = "VM01"
$osDiskName = "VM01-OSDisk"
$osDiskUri = "https://<storage-account>.blob.core.windows.net/<container>/server.vhd"
$VMSize = "Standard_A1"
$storageAccountType = "StandardLRS"
$IPaddress = "10.10.10.10"

#Create the VM resources
$IPconfig = New-AzureRmNetworkInterfaceIpConfig -Name "IPConfig1" -PrivateIpAddressVersion IPv4 -PrivateIpAddress $IPaddress -SubnetId $subnet
$nic = New-AzureRmNetworkInterface -Name $nicName -ResourceGroupName $rgName -Location $location -IpConfiguration $IPconfig
$vmConfig = New-AzureRmVMConfig -VMName $vmName -VMSize $VMSize
$vm = Add-AzureRmVMNetworkInterface -VM $vmConfig -Id $nic.Id

$osDisk = New-AzureRmDisk -DiskName $osDiskName -Disk (New-AzureRmDiskConfig -AccountType $storageAccountType -Location $location -CreateOption Import -SourceUri $osDiskUri) -ResourceGroupName $rgName
$vm = Set-AzureRmVMOSDisk -VM $vm -ManagedDiskId $osDisk.Id -StorageAccountType $storageAccountType -DiskSizeInGB 128 -CreateOption Attach -Windows
$vm = Set-AzureRmVMBootDiagnostics -VM $vm -disable

#Create the new VM
New-AzureRmVM -ResourceGroupName $rgName -Location $location -VM $vm

Again, let me explain a little the parameters we’ve set that the start of the script:

  • $rgName = the resource group where you want to deploy the VM instance
  • $location = the Azure region
  • $vnet = the virtual network where you want to deploy the VM instance
  • $subnet = the subnet where you want to deploy the VM instance
  • $nicName = the name of the NIC of the server
  • $vmName = the name of the VM instance, the server name
  • $osDiskName = the OS disk name
  • $osDiskUri = the direct URI/URL to the VHD in your storage account
  • $VMSize = the VM size or the service plan for the VM
  • $storageAccountType = what type of storage you would like to have
  • $IPaddress = the static IP address of the server as I like to do this in Azure, rather than use dynamic IP’s

And that is pretty much that!

Conclusion

It’s Friday in Sydney. Its the pre-kend and it’s a gloomy, cold 9th day of Winter 2017. I hope that the above content helps you out of a jam or gives you the insight you need to run through this process quickly and efficiently. That feeling of giving back, helping. Thats that feeling that should warm me up and get me to lunch time! Counting down!

Cheers!

Create a new Active Directory Forest using Desired State Configuration

Originally posted on Nivlesh’s blog @ nivleshc.wordpress.com

Desired State Configuration (DSC) is a declarative language in which you state “what” you want done instead of going into the nitty gritty level to describe exactly how to get it done. Jeffrey Snover (the inventor of PowerShell) quotes Jean-Luc Picard from Star Trek: The Next Generation to describe DSC – it tells the servers to “Make it so”.

In this blog, I will show you how to use DSC to create a brand new Active Directory Forest. In my next blog post, for redundancy, we will add another domain controller to this new Active Directory Forest.

The DSC configuration script can be used in conjunction with Azure Resource Manager to deploy a new Active Directory Forest with a single click, how good is that!

A DSC Configuration script uses DSC Resources to describe what needs to be done. DSC Resources are made up of PowerShell script functions, which are used by the Local Configuration Manager to “Make it so”.

Windows PowerShell 4.0 and Windows PowerShell 5.0, by default come with a limited number of DSC Resources. Don’t let this trouble you, as you can always install more DSC modules! It’s as easy as downloading them from a trusted location and placing them in the PowerShell modules directory! 

To get a list of DSC Resources currently installed, within PowerShell, execute the following command

Get-DscResource

Out of the box, there are no DSC modules available to create an Active Directory Forest. So, to start off, lets go download a module that will do this for us.

For our purpose, we will be installing the xActiveDirectory PowerShell module (the x in front of ActiveDirectory means experimental), which can be downloaded from https://gallery.technet.microsoft.com/scriptcenter/xActiveDirectory-f2d573f3  (updates to the xActiveDirectory module is no longer being provided at the above link, instead these are now published on GitHub at https://github.com/PowerShell/xActiveDirectory  . For simplicity, we will use the TechNet link above)

Download the following DSC modules as well

https://gallery.technet.microsoft.com/scriptcenter/xNetworking-Module-818b3583

https://gallery.technet.microsoft.com/scriptcenter/xPendingReboot-PowerShell-b269f154

After downloading the zip files, extract the contents and place them in the PowerShell modules directory located at $env:ProgramFiles\WindowsPowerShell\Modules folder

($env: is a PowerShell reference  to environmental variables).

For most systems, the modules folder is located at C:\ProgramFiles\WindowsPowerShell\Modules

However, if you are unsure, run the following PowerShell command to get the location of $env:ProgramFiles 

Get-ChildItem env:ProgramFiles

Tip: To get a list of all environmental variables, run Get-ChildItem env:

With the pre-requisites taken care of, let’s start creating the DSC configuration script.

Open your Windows PowerShell IDE and let’s get started.

Paste the following into your PowerShell editor and save it using a filename of your choice (I have saved mine as CreateNewADForest.ps1)

Configuration CreateNewADForest {
param(
      [Parameter(Mandatory)]
      [String]$DomainName,

      [Parameter(Mandatory)]
      [System.Management.Automation.PSCredential]$AdminCreds,

      [Parameter(Mandatory)]
      [System.Management.Automation.PSCredential]$SafeModeAdminCreds,

      [Parameter(Mandatory)]
      [System.Management.Automation.PSCredential]$myFirstUserCreds,

      [Int]$RetryCount=20,
      [Int]$RetryIntervalSec=30
)

The above declaration defines the parameters that will be passed to our  configuration function.  There are also two “constants” defined as well. I have named my configuration function CreateNewADForest which coincides with the name of the file it is saved in – CreateNewADForest.ps1

Here is some explanation regarding the above parameters and constants

We now have to import the DSC modules that we had downloaded, so add the following line to the above Configuration Script

Import-DscResource -ModuleName xActiveDirectory, xNetworking, xPendingReboot

We now need to convert $AdminCreds the format domain\username. We will store the result in a new object called $DomainCreds

The next line states where the DSC commands will run. Since we are going to run this script from within the newly provisioned virtual machine, we will use localhost as the location

So enter the following

Node localhost
{

Next, we need to tell the Local Configuration Manager to apply the settings only once, reboot the server if needed (during our setup) and most importantly, to continue on with the configuration after reboot. This is done by using the following lines

LocalConfigurationManager
{
     ActionAfterReboot = 'ContinueConfiguration'
     ConfigurationMode = 'ApplyOnly'
     RebootNodeIfNeeded = $true
}

If you have worked with Active Directory before, you will be aware of the importance of DNS. Unfortunately, during my testing, I found that a DNS Server is not automatically deployed when creating a new Active Directory Forest. To ensure a DNS Server is present before we start creating a new Active Directory Forest, the following lines are needed in our script

WindowsFeature DNS
{
     Ensure = "Present"
     Name = "DNS"
}

The above is a declarative statement which nicely shows the “Make it so” nature of DSC. The above lines are asking the DSC Resource WindowsFeature to Ensure DNS (which refers to DNS Server) is Present. If it is not Present, then it will be installed, otherwise nothing will be done. This is the purpose of the Ensure = “Present” line. The word DNS after WindowsFeature is just a name I have given to this block of code.

Next, we will leverage the DSC function xDNSServerAddress (from the xNetworking module we had installed) to set the local computer’s DNS settings, to its loopback address. This will make the computer refer to the newly installed DNS Server.

xDnsServerAddress DnsServerAddress
{
     Address        = '127.0.0.1'
     InterfaceAlias = 'Ethernet'
     AddressFamily  = 'IPv4'
     DependsOn = "[WindowsFeature]DNS"
}

Notice the DependsOn = “[WindowsFeature]DNS” in the above code. It tells the function that this block of code depends on the [WindowsFeature]DNS section. To put it another way, the above code will only run after the DNS Server has been installed. There you go, we have specified our first dependency.

Next, we will install the Remote Server Administration Tools

WindowsFeature RSAT
{
     Ensure = "Present"
     Name = "RSAT"
}

Now, we will install the Active Directory Domain Services feature in Windows Server. Note, this is just installation of the feature. It does not create the Active Directory Forest.

WindowsFeature ADDSInstall
{
     Ensure = "Present"
     Name = "AD-Domain-Services"
}

So far so good. Now for the most important event of all, creating the new Active Directory Forest. For this we will use the xADDomain function from the xActiveDirectory module.

Notice above, we are pointing the DatabasePath, LogPath and SysvolPath all on to the C: drive. If you need these to be on a separate volume, then ensure an additional data disk is added to your virtual machine and then change the drive letter accordingly in the above code. This section has a dependency on the server’s DNS settings being changed.

If you have previously installed a new Active Directory forest, you will remember how long it takes for the configuration to finish. We have to cater for this in our DSC script, to wait for the Forest to be successfully provisioned, before proceeding. Here, we will leverage on the xWaitForADDomain DSC function (from the xActiveDirectoy module we had installed).

xWaitForADDomain DscForestWait
{
     DomainName = $DomainName
     DomainUserCredential = $DomainCreds
     RetryCount = $RetryCount
     RetryIntervalSec = $RetryIntervalSec
     DependsOn = "[xADDomain]FirstDC"
}

Viola! The new Active Directory Forest has been successfully provisioned! Let’s add a new user to Active Directory and then restart the server.

xADUser FirstUser
{
     DomainName = $DomainName
     DomainAdministratorCredential = $DomainCreds
     UserName = $myFirstUserCreds.Username
     Password = $myFirstUserCreds
     Ensure = "Present"
     DependsOn = "[xWaitForADDomain]DscForestWait"
}

The username and password are what had been supplied in the parameters to the configuration function.

That’s it! Our new Active Directory Forest is up and running. All that is needed now is a reboot of the server to complete the configuration!

To reboot the server, we will use the xPendingReboot module (from the xPendingReboot package that we installed)

xPendingReboot Reboot1
{
     Name = "RebootServer"
     DependsOn = "[xWaitForADDomain]DscForestWait"
}

Your completed Configuration Script should look like below (I have taken the liberty of closing off all brackets and parenthesis)

You can copy the above script to a newly create Azure virtual machine and deploy it manually.

However, a more elegant method will be to bootstrap it to your Azure Resource Manager virtual machine template. One caveat is, the DSC configuration script has to be packaged into a zip file and then uploaded to a location that Azure Resource Manager can access (that means it can’t live on your local computer). You can upload it to your Azure Storage blob container and use a shared access token to give access to your script or upload it to a public repository like GitHub.

I have packaged and uploaded the script to my GitHub repository at https://raw.githubusercontent.com/nivleshc/arm/master/CreateNewADForest.zip

The zip file also contains the additional DSC Modules that were downloaded. To use the above in your Azure Resource Manager template, create a PowerShell DSC Extension and link it to the Azure virtual machine that will become your Active Directory Domain Controller. Below is an example of the extension you can create in your ARM template (the server that will become the first domain controller is called DC01 in the code below).

Below is an except of the relevant variables

And here are the relevant parameters

That’s it folks! Now you have a new shiny Active Directory Forest, that was created for you using a DSC configuration script.

In the second part of this two-part series, we will add an additional domain controller to this Active Directory Forest using DSC configuration script.

Let me know what you think of the above information.

Passing Parameters to Linked ARM Templates

Recently, my workmate Vic wrote some great posts regarding to Azure Linked Templates. This is, a supplementary post to his ones, to show how to share parameters across the linked templates.

Scripts and templates used in this post can be found at: https://github.com/devkimchi/Linked-ARM-Templates-Sample

parametersLink and parameters Properties

We have a master template, master-deployment.json, and it looks like:

Each nested template has a parameter called environment that has the same value as the one in the master template. Each template also has a corresponding parameter file. Therefore, as above, we use both parametersLink and parameters properties to handle both. However, life is not easy.

Oooops! We can’t use both parametersLink and parameters at the same time. We have to use one or the other. This is by design at the time of this writing. Because the environment parameter is a common denominator across all the nested templates, it is natural to think that the parameter can be passed from master to nested ones.

How can we work this out then? There are several workarounds.

  1. Add the common parameters to each parameter JSON file and upload it to Azure Storage Account programatically.
  2. Create a set of parameter files for all possible combinations.
  3. Use something to refer during the template deployment, whose value will be changing over time during the deployment but the reference point remains the same.

Update Parameters Programatically

The easiest way is the first option. It can be simply achieved by running a separate PowerShell script before the deployment. Let’s have a look.

This is merely to upload nested template files to Azure Storage Account. It excludes parameter files because they need to be updated before being uploaded. Let’s have a look at the following PowerShell script.

The core part of the script is:

  • To read parameter file as a JSON object,
  • To add the environment property to the JSON object, and
  • To overwrite the parameter file.

Then this updated parameter files are uploaded to Azure Storage Account. Now our Storage Account has got all nested templates and their parameter files. Let’s run the master template again without the parameters property.

Tada~! It all works fine!

So far, we’ve taken a look how to sort out the restriction that we can’t use both parametersLink and parameters properties at the same time. We obviously need a help of another script to run linked templates with their parameters by updating it. It’s definitely not an ideal scenario but certainly it works. If this update is done by hand, it’s tedious, which should be avoided. However, in CI/CD pipelines, it wouldn’t be an issue because we can automate it.

Adding/Removing User Office365 Licences using PowerShell and the Azure AD Graph RestAPI

In a recent blog post here I posted about the Azure AD v2.0 Preview Powershell cmdlets that are currently in preview. These update the functionality the current MSOL cmdlets provide whilst also supporting features they don’t (such as managing users with MFA).

The Azure AD v2.0 cmdlets interface with the Azure AD Graph API and this week I tried using the Set-AzureADUserLicense cmdlet to add/remove licenses from users in a test tenant. With no sample documentation for syntax I didn’t kick any goals so I figured I’d just go straight to using the Azure AD Graph API to get the job done direct from Powershell instead.

In this post I’m going to show you how to add/remove Office365 licenses from users using PowerShell and the Azure AD Graph API.

As per my other post linked above if you’ve installed the Azure AD Preview Powershell module you’ll have the Microsoft.IdentityModel.Clients.ActiveDirectory.dll which we can leverage via Powershell to then connect to the Azure AD Graph API. Chances are you’ll have Microsoft.IdentityModel.Clients.ActiveDirectory.dll though if you also have the AzureRM Modules installed or TFS. Just search your Program Files sub-directories.

# the default path to where the Azure AD Preview PS Module puts the Libs
'C:\Program Files\WindowsPowerShell\Modules\AzureADPreview\1.1.143.0\Microsoft.IdentityModel.Clients.ActiveDirectory.dll'
# TFS Path
'C:\Program Files\Common Files\microsoft shared\Team Foundation Server\14.0\Microsoft.IdentityModel.Clients.ActiveDirectory.dll'
# Azure RM Cmdlets
'C:\Program Files\WindowsPowerShell\Modules\AzureRM.ApiManagement\1.1.2\Microsoft.IdentityModel.Clients.ActiveDirectory.dll'

Below you’ll find what you need to script your connection to Azure AD via the GraphAPI using the dll discussed above. Change $tenantID $username and $password to reflect you tenant and credentials.

Now that we’ve authenticated let’s enumerate our licenses. Bascially we make a RestAPI call to “https://graph.windows.net/{0}/subscribedSkus?api-version=1.6” as below. The particular license I want to add in my tenant is ExchangeStandard_Student. A key difference from the MSOL cmdlets though is adding or removing a license via the Graph API we reference the License skuId rather than skuPartNumber. The last line gets the skuId for my license.

Now to find the users that need to have the license assigned. Below in line 4 I search for users that are account enabled using this URI “https://graph.windows.net/{0}/users?$filter=accountEnabled eq true &api-version=1.6″  You can modify the filter for your criteria.

I then iterate through the users and find the users that aren’t assigned the license identified earlier above. I also exclude the AADConnect account for this tenant.

The GraphAPI expects the body with the info of licenses to be added or removed in a hashtable. So create the hashtable and convert it to JSON which is the format for Azure AD Graph API requires. Then for each of the unlicensed users we call the Azure AD GraphAPI URI “https://graph.windows.net/myorganization/users/$usertolicense`/assignLicense?api-version=1.6” and assign the license . In my environment it processed users at just over 1 user/sec.

A similar approach to remove licenses from users. A subtle difference is you only need to specify the skuId for removal as shown below. Note: you can add and remove licenses in the same call (if say you’re switching users over from one license plan to another).

All together for quick copy and paste. Update it for your tenant and creds. And remark out the Add or Remove depending on what you’re looking to do. If you’re looking to add/remove multiple licenses just add them to the hashtable/array.

Follow Darren on Twitter @darrenjrobinson

 

Entity Framework 7 Data Migration through KUDU

From DevOps perspective, everything needs to be automated in regards to application setup and deployment. There’s no exception for database migration. If database schema change occurs, it should be automatically applied before/after the application deployment. Unlike Entity Framework 6.x using PowerShell cmdlets for database migration, Entity Framework 7 (EF7) uses DNX for it.

Applying Database Migration with EF7

In EF7, updating database change can be done by running the following command:

If your DbContext is located in another project and your web application has a reference to it, then you can run the following command:

By running the command above within your build/deployment pipeline, your database change is easily applied to the existing database. Connectionstrings are defined in appsettings.json in your ASP.NET Core application.

Visit https://docs.efproject.net for more details.

In most cases, there’s no issue to access to Azure SQL Database from your build/deployment server, as long as Azure SQL Database has a proper firewall setup. But what if your enterprise firewall doesn’t allow to connect to Azure SQL Database, like blocking the TCP port of 1433? Then we can’t run this command from our build/deployment server.

REST API in KUDU

KUDU is basically a backend service engine for deployment tied to your Azure Website. If you are running any Azure App Service, your KUDU can be accessible via https://your-azure-website.scm.azurewebsites.net. It provides REST API for website maintenance and one of its endpoint is command. Therefore, we can write a script, say db-migration.cmd, deploy it at the same time when the application is deployed, and run it through this REST API. The db-migration.cmd might look like:

So, the application is ready for database migration. Let’s write a PowerShell script to run the command. Make sure that we are using Azure Service Management (ASM) cmdlets.

NOTE: You should login to ASM with appropriate subscription first.

NOTE: The dir property is where the actual command is run, which is the relative path to %HOME% in Azure App Service.

Running the PS script above will bring you to database migration completed within KUDU. Make sure that the $result object has an exit code of 0 by examining $result.ExitCode. If the exit code is other than 0, database migration has come to fail.

So far, we have briefly looked at KUDU for Azure SQL Database migration. KUDU actually has many useful functions for monitoring, so it would be worth taking a look.

Simultaneously Start|Stop all Azure Resource Manager Virtual Machines in a Resource Group

Problem

How many times have you wanted to Start or Stop all Virtual Machines in an Azure Resource Group ? For me it seems to be quite often, especially for development environment resource groups. It’s not that difficult though. You can just enumerate the VM’s then cycle through them and call ‘Start-AzureRMVM’ or ‘Start-AzureRMVM’. However, the more VM’s you have, that approach running serially as PowerShell does means it can take quite some time to complete. Go to the Portal and right-click on each VM and start|stop ?

There has to be a way of starting/shutting down all VM’s in a Resource Group in parallel via PowerShell right ?

Some searching and it seems common to use Azure Automation and Workflow’s to accomplish it. But I don’t want to run this on schedule or necessarily mess around with Azure Automation for development environments, or have to connected to the portal and kickoff the workflow.

What I wanted was a script that was portable. That lead me to messing around with ‘ScriptBlocks’ and ‘Start-Job’ functions in PowerShell. Passing variables in for locally hosted jobs running against Azure though was painful. So I found a quick clean way of doing it, that I detail in this post.

Solution

I’m using the brilliant Invoke-Parallel Powershell Script from Cookie.Monster, to in essence multi-thread and run in parallel the Virtual Machine ‘start’ and ‘stop’ requests.

In my script at the bottom of this post I haven’t included the ‘invoke-parallel.ps1’. The link for it is in the paragraph above. You’ll need to either reference it at the start of your script, or include it in your script. If you want to keep it all together in a single script include it like I have in the screenshot below.

My rudimentary PowerShell script takes two parameters;

  1. Power state. Either ‘Start’ or ‘Stop’
  2. Resource Group. The name of the Azure Resource Group containing the Virtual Machines you are wanting to start/stop. eg. ‘RG01’

<

p style=”background:white;”>Example: .\AzureRGVMPowerGo.ps1 -power ‘Start’ -azureResourceGroup ‘RG01’ or PowerShell .\AzureRGVMPowerGo.ps1 -power ‘Start’ -azureResourceGroup ‘RG01’

Note: If you don’t have a session to Azure in your current environment, you’ll be prompted to authenticate.

Your VM’s will simultaneously start/stop.

What’s it actually doing ?

It’s pretty simple. The script enumerates the VM’s in the Resource Group you’ve specified. It looks to see the status of the VM’s (Running or Deallocated) that is the inverse of the ‘Power’ state you’ve specified when running the script. It’ll start stopped VM’s in the Resource Group when you run it with ‘Start’ or it will stop all started VM’s in the Resource Group when you run it with ‘Stop’. Simples.

This script could also easily be updated to do other similar tasks. Like, delete all VM’s in a Resource Group.

Here it is

Enjoy.

Follow Darren Robinson on Twitter

Azure Internal Load Balancing – Setting Distribution Mode

I’m going to start by saying that I totally missed that the setting of distribution mode on Azure’s Internal Load Balancer (ILB) service is possible. This is mostly because you don’t set the distribution mode at the ILB level – you set it at the Endpoint level (which in hindsight makes sense because that’s how you do it for the public load balancing too).

There is an excellent blog on the Azure site that covers distribution modes for public load balancing and the good news is that they also apply to internal load balancing as well. Let’s take a look.

In the example below we’ll use the following parameters:

  • Cloud Service: apptier
    containing
  • Two VMS: apptier01, apptier02
    on
  • VNet subnet with name of ‘appsubnet’
    adding a
  • load balancer with static IP address of 192.168.1.25
    which
  • balances HTTP traffic based on Source and Destination IP.

Here’s the PowerShell to achieve this setup.


# Assume you have setup PS subscription and user Account.

# Add Load Balancer to Cloud Service wrapping VMs
Add-AzureInternalLoadBalancer -ServiceName apptier `
-InternalLoadBalancerName apptierplb -SubnetName appsubnet `
-StaticVNetIPAddress 192.168.1.25

# Add Endpoints to VMs
# VM1
Get-AzureVM -ServiceName apptier -Name apptier01 | `
Add-AzureEndpoint -LBSetName 'HttpIn' -Name 'HttpIn' `
-DefaultProbe -InternalLoadBalancerName 'apptierplb' -Protocol tcp `
-PublicPort 80 -LocalPort 80 -LoadBalancerDistribution sourceIP | `
Update-AzureVM

# VM2
Get-AzureVM -ServiceName apptier -Name apptier02 | `
Add-AzureEndpoint -LBSetName 'HttpIn' -Name 'HttpIn' `
-DefaultProbe -InternalLoadBalancerName 'apptierplb' -Protocol tcp `
-PublicPort 80 -LocalPort 80 -LoadBalancerDistribution sourceIP | `
Update-AzureVM

# You can check what distribution mode is set
Get-AzureVM –ServiceName apptier –Name apptier01 | Get-AzureEndpoint

HTH.

Moving resources between Azure Resource Groups

The concept of resource groups has been around for a little while, and is adequately supported in the Azure preview portal. Resource groups are logical containers that allow you to group individual resources such as virtual machines, storage accounts, websites and databases so they can be managed together. They give a much clearer picture to what resources belong together, and can also give visibility into consumption/spending in a grouped matter.

However, when resources are created in the classic Azure portal (e.g. virtual machines, storage accounts, etc.) there is no support for resource group management, which results in a new resource group being created for each resource that you create. This can lead to a large number of resource groups that are unclear and tedious to manage. Also, if you do tend to use resource groups in the Azure preview portal there is no way to perform housekeeping or management of these resource groups.

With the latest Azure PowerShell cmdlets (v0.8.15.1) we now have the ability to move resources between resource groups. You can install the latest version of the PowerShell tools via the Web Platform Installer:

wpi azure powershell

After installation of this particular version we now have the following PowerShell commands available that will assist us in moving resources:

  • New-AzureResourceGroup
  • Move-AzureResource
  • Remove-AzureResourceGroup
  • Get-AzureResource
  • Get-AzureResourceGroup
  • Get-AzureResourceLog
  • Get-AzureResourceGroupLog

Switch-AzureMode AzureResourceManager

After launching a Microsoft Azure Powershell console we need to switch to Azure Resource Manager mode in order to manage our resource groups:

Switch-AzureMode AzureResourceManager

Get-AzureResourceGroup

Without any parameters this cmdlet gives a complete list of all resource groups that are deployed in your current subscription:

When resources are created in the classic Azure portal they will appear with a new resource group name that corresponds to the name of the object that was created (e.g. virtual machine name, storage account name, website name, etc.).

Note that we have a few default resource groups for storage, SQL and some specific resource groups corresponding to virtual machines. These were automatically created when I built some virtual machines and a Azure SQL server database in the classic Azure portal.

New-AzureResourceGroup

In order to group our existing resources we’re going to create a new resource group. It’s important to note that resource groups reside in a particular region which needs to be specified upon creation:

You’d think that resources can only be moved across resource groups that reside in the same region. However, I’ve successfully moved resources between resource groups that reside in different regions. This doesn’t affect the actual location of the resource so I’m not sure what the exact purpose of specifying a location for a resource group is.

Get-AzureResourceGroup

The Get-AzureResourceGroup cmdlet allows you to view all resources within a group, including their respective types and IDs:

Move-AzureResourceGroup

To move resources from the existing resource groups we need to provide the Move-ResourceGroup cmdlet a list of resource IDs. The cmdlet accepts the resource ID(s) as pipeline input parameters, so we can use the Get-AzureResource cmdlet to feed the list of resource IDs. The following script moves a cloud service, virtual machine and storage account (all residing in the same region) to the newly created resource group:

The Get-AzureResource cmdlet allows you to further filter based on resource type, or individual resource name. The Move-ResourceGroup cmdlet automatically removed the original resource group in case there are no resources associated after moving them.

Unfortunately at the time of writing there was an issue with moving SQL database servers and databases to other resource groups:

Trying to move the SQL server only does not raise any errors, but doesn’t result in the desired target state and leaves the SQL server and database in the original resource group:

The cmetlets Get-AzureResourceLog and Get-AzureResourceGroupLog provide a log of all the performed operations on resources and resource groups, but couldn’t provide any further information regarding the failure to move resources to the new group.

Now we have successfully moved our virtual machine and storage account to the new resource group we can get insight into these resources through the resource group:

Resource Group

Publish to a New Azure Website from behind a Proxy

One of the great things about Azure is the ease of which you can spin up a new cloud based website using Powershell. From there you can quickly publish any web-based solution from Visual Studio to the Azure hosted site.

To show how simple this is; After configuring PowerShell to use an Azure Subscription, I’ve created a new Azure hosted website in the new Melbourne (Australia Southeast) region:

Creating a new website in PowerShell

Creating a new website in PowerShell

That was extremely easy. What next? Publish your existing ASP.NET MVC application from Visual Studio to the web site. For this test, I’ve used Microsoft Visual Studio Ultimate 2013 Update 3 (VS2013). VS2013 offers a simple way from the built-in Publish Web dialogue to select your newly created (or existing) websites.

publish

Web Publish to Azure Websites

This will require that you have already signed in with your Microsoft account linked with a subscription, or you have already imported your subscription certificate to Visual Studio (you can use the same certificate generated for PowerShell). Once your subscription is configured you can select the previously created WebSite:

Select Existing Azure Website

Select Existing Azure Website

The Publish Web dialogue appears, but at this point you may experience failure when you attempt to validate the connection or publish the WebSite. If you are behind a proxy; then the error will show as destination not reachable.

Unable to publish to an Azure Website from behind a proxy

Unable to publish to an Azure Website from behind a proxy

Could not connect to the remote computer ("mykloudtestapp.scm.azurewebsites.net"). On the remote computer, make sure that Web Deploy is installed and that the required process ("Web Management Service") is started. Learn more at: http://go.microsoft.com/fwlink/?LinkId=221672#ERROR_DESTINATION_NOT_REACHABLE. Unable to connect to the remote server

The version of Web Deploy included with VS2013 is not able to publish via a Proxy. Even if you configure the msbuild.exe.config to have the correct proxy settings as documented by Microsoft, it will still fail.

Luckily in August 2014 Web Deploy v3.6 BETA3 was released that fixes this issue. To resolve this error, you can download the Web Deploy beta and patch your VS2013 installation. After patching Visual Studio; you can modify the proxy settings used by msbuild.exe (msbuild.exe.config) to use the system proxy:

<system.net>
	<defaultProxy useDefaultCredentials="true" />
</system.net>

You should now be able to publish to your Azure WebSite from behind a proxy with VS2013 Web Deploy.

Get Azure Virtual Networks with PowerShell

I needed to make my life easier the other day as a colleague and I worked through setting up a Azure IaaS network topology to connect to an enterprise production network. One of our clients requirements meant that whilst we created the network sites, subnets and segments we needed to report on what we had created to verify it was correct. This simple task of viewing network names and associated subnets is currently missing from the Azure cmdlets, so we have pieced together this quick bit of re-usable code. There isn’t currently a nice way to do a ‘Get-AzureVirtualNetwork’ and instead you only have the portal(s) or a downloadable XML. This isn’t great if you would like to do some validation checks or even a basic CSV dump to report on current configuration to a network administrator.

I’ve only been able to test this on a few VNET configurations but it seems to do the job for me so far. I’m sure that you can add your own trickery to it if you like, but by the time I invest in doing so, Microsoft will have probably released an updated module that includes this functionality. So here it is in all its glory.

The output will give you a nice table that you can export as per below;

Get-AzureVirtualNetwork | FT

Filter to find reference to matching CIDR blocks;

Get-AzureVirtualNetwork | ? {$_.AddressPrefix -like "10.72.*"} | FT

Store the networks and re-use with other Azure cmdlets;

$APSites = (Get-AzureVirtualNetwork | ? {$_.AddressPrefix -like "10.72.*"})

Export the current configuration for a report;

Get-AzureVirtualNetwork | Export-CSV -Path "c:\Temp\AzureVirtualNetworkConfiguration.csv"

If you notice the code snippet I threw in another handy export for all local routes;

Get-AzureVirtualNetworkRoutes

This will be useful to see if you are missing a local route when the list starts getting large.