Hands Free VM Management with Azure Automation and Resource Manager – Part 1

Over the past six months, Microsoft have launched a number of features in Azure to enable you to better manage your resources hosted there.

In this two part series, I will show how we can leverage two of these new features – Azure Automation and Azure Resource Manager – to schedule the shutting down of tagged Virtual Machines in Microsoft Azure.

• In Part 1 we will walk through tagging resources using the Azure Resource Manager features and
• In Part 2 we will setup Azure Automation to schedule a runbook to execute nightly and shutdown tagged VM resources.

About Azure Resource Manager and Azure Automation

Azure Resource Manager (ARM) provides the capability to create reusable deployment templates and provide a common way to manage the resources that make up your deployment. ARM is the foundation of the ALM and DevOps tooling being developed by Microsoft and investments in it are ongoing.

Another key service in this space is the Azure Automation service which provides the ability to create, monitor, manage, and deploy resources using runbooks based on Windows PowerShell workflows. Automation assets can be defined to share credentials, modules, schedules and runtime configuration between runbooks and a runbook gallery provides contributions from both Microsoft and the community that can be imported and used within your Automation account.

Operating infrastructure effectively in the cloud is the new holy grail of today’s IT teams – scaling out to meet demand, back to reduce unnecessary operating costs, running only when needed and de-allocating when not. Automating elastic scale has been one area cloud providers and 3rd party tooling ISVs have invested in and the capability is pretty solid.

The broader resource lifecycle management story is not so compelling. Shutting down resources when they are not needed is still largely a manual process unless heavy investments are made into infrastructure monitoring and automation tools or via 3rd party SaaS offerings such as Azure Watch.

Objectives

We will start by identifying and tagging the resource groups we want to ensure are automatically shutdown overnight. Following this we will author an Azure Automation runbook that looks for tagged resources and shuts them down, configuring it to run every night.

Setting this up is not as straight forward as you would think (and we will need to bend the rules a little) so I will spend some time going through these steps in detail.

Tagging Resource Groups

Azure Resource Management features are currently only available via the preview portal and command line tools (PowerShell and cross-platform CLI). Using the preview portal we can manually create tags and assign them to our resource groups. A decent article on the Azure site walks you through how to perform these steps. Simon also introduced the ARM command line tools in his post a few months ago.

Performing these tasks in PowerShell is a little different to previous service management tasks you may be use to. For starters, the Azure Resource Manager cmdlets cannot be used in the same PS session as the “standard” Azure cmdlets. We must now switch between the two modes. The other main difference is the way we authenticate to the ARM API. We no longer use service management certificates with expiry periods of two to three years but user accounts backed by Azure Active Directory (AAD). These user tokens have expiry periods of two to three days so we must constantly refresh them.

Note: Switch “modes” just removes one set of Azure modules and imports others. So switching to Azure Resource Manager mode removes the Azure module and imports the Azure Resource Manager module and Azure Profile module. We need this understanding when we setup our Azure Automation later on.

Let’s walk through the steps to switch into ARM mode, add our user account and tag the resource groups we want to automate.

Before you start, ensure you have the latest drop of the Azure PowerShell cmdlets from GitHub. With the latest version installed, switch to Azure Resource Manager mode and add your AAD account to your local PS profile

[code language=”PowerShell”]
# Switch to Azure Resource Manager mode
Switch-AzureMode AzureResourceManager

# Sign in to add our account to our PS profile
Add-AzureAccount

# Check our profile has the newly gathered credentials
Get-AzureAccount

[/code]

Add-AzureAccount will prompt us to sign-in using either our Microsoft Account or Organisational account with service management permissions. Get-AzureAccount will list the profiles we have configured locally.

Now that we are in ARM mode and authenticated, we can examine and manage our tag taxonomy using the Get-AzureTag Cmdlet

[code language=”PowerShell”]
#Display current taxonomy
Get-AzureTag
[/code]

This displays all tags in our taxonomy and how many resource groups they are assigned to. To add tags to our taxonomy we use the New-AzureTag cmdlet. To remove, we use the Remove-AzureTag cmdlet. We can only remove tags that are not assigned to resources (that is, have a zero count).

In this post we want to create and use a tag named “autoShutdown” that we can then assign a value of “True” against resource groups that we want to automatically shutdown.

[code language=”PowerShell”]
# Add Tag to Taxonomy
New-AzureTag -Name "autoShutdown"
[/code]

Now, let’s tag our resource groups. This can be performed using the preview portal as mentioned above, but if you have many resource groups to manage PowerShell is still the best approach. To manage resource groups using the ARM module we use the following cmdlets

[code language=”PowerShell”]
# Assign tag to DEV/TEST resource groups
Get-AzureResourceGroup | ? { $_.ResourceGroupName -match "dev-cloud" } | `
Set-AzureResourceGroup -Tag @( @{ Name="autoShutdown"; Value=$true } )
[/code]

The above statement finds all resource groups that contain the text “dev-cloud” in the name (a naming convention I have adopted, yours will be different) and sets the autoShutdown tag with a value of True to that resource group. If we list the resource groups using the Get-AzureResourceGroups cmdlet we can see the results of the tagging.

Note: Our resource group contains many resources. However, for our purposes we are only interested in managing our virtual machines. These resource types shown above will come in handy when we look to filter the tagged resources to only return our VMs.

We can also un-tag resource groups using the following statement

[code language=”PowerShell”]
# Reset tags on DEV/TEST resource groups
Get-AzureResourceGroup | ? { $_.ResourceGroupName -match "dev-cloud" } | `
Set-AzureResourceGroup -Tag @{}
[/code]

We can see these tagged resource groups in the preview portal as well…

…and if we drill into one of the resource groups we can see our tag has the value True assigned

In this post we have tagged our resource groups so that they can be managed separately. In Part 2 of the post, we will move on to creating an Azure Automation runbook to automate the shutting down of our tagged resources on a daily schedule.

Considerations

At the time of writing there is no support in the preview portal for tagging individual resources directly (just Resource Groups). The PowerShell Cmdlets suggest it is possible however I always get an error indicating setting the tags property is not permitted yet. This may change in the near future and provide more granular control.