22nd of February, 2016 / / 14 Comments

This post is details my method for automating the creation of AzureRM virtual machines for use in a development environment. I’m using this process to quickly standup an environment for testing configurations on.

In summary this process;

  • parallel creation of the AzureRM Virtual Machines
  • All machines have the same configuration
    • NIC, Disks etc
  • All machines are created in a new Resource Group, with associated Virtual Network

Simultaneous Creating the AzureRM Virtual Machines for MIM 2016

For my MIM 2016 Lab I’m going to create 5 Virtual Machines. Their roles are;

  • Active Directory Domain Controllers (2)
  • MIM 2016 Portal Servers (2)
  • MIM 2016 Synchronization Server & collocated SQL Server (1)

In order to stand up the Virtual Machines as quickly as possible I spawn the VM Create process in parallel leveraging the brilliant Invoke-Parallel Powershell Script from Cookie.Monster just as I did for my Simultaneous Start|Stop all AzureRM VM’s script detailed here.

VM Creation Script

In my script at the bottom of this post I haven’t included the ‘invoke-parallel.ps1’. The link for it is in the paragraph above. You’ll need to either reference it at the start of your script, or include it in your script. If you want to keep it all together in a single script include it like I have in the screenshot below.

Parts you’ll need to edit

The first part of the script defines what, where and names associated with the development environment. Where will the lab be deployed (‘Australia East’), what will be the Resource Group name (‘MIM2016-Dev10’), the virtual network name in the resource group (‘MIM-Net10’), the storage account name (‘mimdev16021610’). Update each of these for the name you’ll be using for your environment.

#Global Variables
# Where do we want to put the VM’s
$global:locName = ‘Australia East’
# Resource Group name
$global:rgName = ‘MIM2016-Dev10’
# Virtual Network Name
$global:virtNetwork = ‘MIM-Net10’
# Storage account names must be between 3 and 24 characters in length and use numbers and lower-case letters only
$global:stName = ‘mimdev16021610’
# VMName
$global:NewVM = $null

The VM’s that will be deployed are added to an array. The names used here will become the ComputerName for each VM. Change for the number of machines and the names you want.

# MIM Servers to Auto Deploy
$VMRole = @()
$VMRole += ,(‘MIMPortal1’)
$VMRole += ,(‘MIMPortal2’)
$VMRole += ,(‘MIMSync’)
$VMRole += ,(‘ADDC1’)
$VMRole += ,(‘ADDC2’)

The script will then prompt you to authN to AzureRM.

# Authenticate to the Azure Portal
Add-AzureRmAccount

The script will then prompt you provide a username and password that will be associated with each of the VM’s.

# Get the UserID and Password info that we want associated with the new VM’s.
$global:cred = Get-Credential -Message “Type the name and password for the local administrator account that will be created for your new VM(s).”

The Resource Group will be created. The environment subnet and virtual network will be created in the resource group.
# Create Resource Group
New-AzureRmResourceGroup -Name $rgName -Location $locName
# Create RG Storage Account
$storageAcc = New-AzureRmStorageAccount -ResourceGroupName $rgName -Name $stName -Type “Standard_GRS” -Location $locName
# Create RG Subnet
$singleSubnet = New-AzureRmVirtualNetworkSubnetConfig -Name singleSubnet -AddressPrefix 10.0.0.0/24
# Create RG Network
$global:vnet = New-AzureRmVirtualNetwork -Name $virtNetwork -ResourceGroupName $rgName -Location $locName -AddressPrefix 10.0.0.0/16 -Subnet $singleSubnet

A configuration for each of the VM’s that we listed in the array earlier will be created and added to a new array $VMConfig

Update the -VMSize “Standard_A1” for a different Azure VM Spec and  DiskSizeInGB for a different data disk size.

# VM Config for each VM
$VMConfig = @()
# Create VMConfigs and add to an array
foreach ($NewVM in $VMRole) {
# ******** Create IP and Network for the VM ***************************************
# *****We do this upfront before the bulk create of the VM**************


$pip = New-AzureRmPublicIpAddress -Name “$NewVM-IP1″ -ResourceGroupName $rgName -Location $locName -AllocationMethod Dynamic
$nic = New-AzureRmNetworkInterface -Name “$NewVM-NIC1″ -ResourceGroupName $rgName -Location $locName -SubnetId $vnet.Subnets[0].Id -PublicIpAddressId $pip.Id
$vm = New-AzureRmVMConfig -VMName $NewVM -VMSize “Standard_A1”
$vm = Set-AzureRmVMOperatingSystem -VM $vm -Windows -ComputerName $NewVM -Credential $cred -ProvisionVMAgent -EnableAutoUpdate
$vm = Set-AzureRmVMSourceImage -VM $vm -PublisherName MicrosoftWindowsServer -Offer WindowsServer -Skus 2012-R2-Datacenter -Version “latest”
$vm = Add-AzureRmVMNetworkInterface -VM $vm -Id $nic.Id
# VM Disks. Deploying an OS and a Data Disk for each
$osDiskUri = $storageAcc.PrimaryEndpoints.Blob.ToString() + “vhds/WindowsVMosDisk$NewVM.vhd”
$DataDiskUri = $storageAcc.PrimaryEndpoints.Blob.ToString() + “vhds/WindowsVMDataDisk$NewVM.vhd”
$vm = Set-AzureRmVMOSDisk -VM $vm -Name “windowsvmosdisk” -VhdUri $osDiskUri -CreateOption fromImage
$vm = Add-AzureRmVMDataDisk -VM $vm -Name “windowsvmdatadisk” -VhdUri $DataDiskUri -CreateOption Empty -Caching ‘ReadOnly’ -DiskSizeInGB 10 -Lun 0

# Add the Config to an Array
$VMConfig += ,($vm)
# ******** End NEW VM ***************************************
}

And finally BAM, we let it loose. Using the Invoke-Parallel script each of the VM’s are created in AzureRM simultaneously.

# In Parallel Create all the VM’s
$VMConfig | Invoke-Parallel -ImportVariables -ScriptBlock { New-AzureRmVM -ResourceGroupName $rgName -Location $locName -VM $_ }

The screenshot below is near the end of the create VM process. The MIMPortal1 VM that was first in the $VMConfig array is finished and the others are close behind.

The screenshot below shows all resources in the resource group. The Storage Account, the Network, each VM, its associated NIC and Public IP Address.

For me all that (including authN to AzureRM and providing the ‘username’ and ‘password’ for the admin account associated with each VM) executed in just under 13 minutes. Nice.

The full script? Sure thing, here it is.

Follow Darren Robinson on Twitter

Category:
Azure Infrastructure, Azure Platform, Cloud Infrastructure, PowerShell
Tags:
, ,

Join the conversation! 14 Comments

  1. Adam Langley

    Hi,

    I have recently done something very similar. I am creating 300 virtual machines in parallel for product load testing purposes – however I came across a problem, and in searching, found your site.
    What I am experiencing, is the parallel loop blocks in batches of 5 machines – meaning that I cannot create 300 machines truly in parallel. Because it takes 7 minutes to fully create 5 machines, it will actually require 7 hours to finish creating 300 machines. Any idea why this is, and how to get around it?

    Thanks

    • Darren Robinson

      Hi Adam. What happens if you specify a value for the -throttle switch on invoke-parallel ?

      • shantanu89

        Hi Darren,
        just came across the script , great one!! thanks a lot
        to get invoke-variable?
        i just need to download it and how do i give reference of the that script because when i run the above one it saysInvoke-Parallel : The term ‘Invoke-Parallel’ is not recognized as the name of a cmdlet, function, script file, or operable program. Check the
        spelling of the name, or if a path was included, verify that the path is correct and try again

      • Darren Robinson

        Yes, Invoke-Parallel is a dependency as per the blog post. The link to it is also still valid https://github.com/RamblingCookieMonster/Invoke-Parallel

      • shantanu89

        thanks Darren,
        I am trying to map new virtual machines to an existing VNET, i sit possible to that tweaking your script i have been debugging it from last couple of day’s but no luck. Here is what i am trying to do.

        $singleSubnet = Add-AzureRmVirtualNetworkSubnetConfig -Name ‘pla-nareg-sg1’ -VirtualNetwork ‘MyPnet’ -NetworkSecurityGroup ‘pla-nareg-sg1-nsg’ -AddressPrefix 10.40.0.0/24

        $global:vnet = Get-AzureRmVirtualNetwork -Name $virtNetwork -ResourceGroupName ‘MyPNetwork’

        however i always landup getting error.

        Thoughts?

      • Darren Robinson

        It sure is. You need to Get the existing network info and then create the new NIC and IP. That is pretty much what I do here in this post https://blog.kloud.com.au/2016/08/25/how-to-quickly-recover-from-a-failed-azurerm-virtual-machine-using-powershell/

        See these lines

        # VirtNet
        $VirtNet = Get-AzureRmVirtualNetwork -ResourceGroupName $ResourceGroup
        # Subnet
        $SubnetID = (Get-AzureRmVirtualNetworkSubnetConfig -VirtualNetwork $VirtNet -Name $brokenVMSubnet).Id

        # Create new Public IP and create a new NIC
        $pip = New-AzureRmPublicIpAddress -Name “$VMName-IP2” -ResourceGroupName $ResourceGroup -Location $location -AllocationMethod Dynamic
        $nic = New-AzureRmNetworkInterface -Name “$VMName-NIC2” -ResourceGroupName $ResourceGroup -Location $location -PublicIpAddressId $pip.Id -SubnetId $SubnetID
        # Add the new NIC to the new VM
        $NewVM = Add-AzureRmVMNetworkInterface -VM $NewVM -Id $nic.Id

  2. Vishal Kalal

    Hi Darren,

    Thanks for the article. I was looking for same thing but in a difference manner. I am planning to prepare a power-shell script to create multiple VM’s simultaneously. I have a CSV file where I will populate the data and import to the PowerShell scripts. Based on the details given in the CSV file powershell scripts will perform below actions-

    1. Check RSG for Compute resource, if not available go ahead and create it.
    2. Check RSG for Storage account, if not available go ahead and create it.
    3. Check Storage account, if not available go ahead and create it.
    4. Check Availability Set, if not available go ahead and create it.
    5. Create Nic cards
    6. Create VMs.

    I tried to prepare in the invoke-Parallel script but it create only one VM for me. Could you please guide me, how to achieve this.

  3. vkalal

    Hello Darren,

    Thanks for the article. I was looking for same thing similar but in a difference manner. I am planning to prepare a power-shell script to create multiple VM’s simultaneously. I have a CSV file where I will populate the data and import to the PowerShell scripts. Based on the details given in the CSV file powershell scripts will perform below actions-

    1. Check RSG for Compute resource, if not available go ahead and create it.
    2. Check RSG for Storage account, if not available go ahead and create it.
    3. Check Storage account, if not available go ahead and create it.
    4. Check Availability Set, if not available go ahead and create it.
    5. Create Nic cards
    6. Create VMs.

    I tried to prepare in the invoke-Parallel script but it create only one VM for me. Could you please guide me, how to achieve this. Let me know if required I can share power-shell script and CSV file.

    • Darren Robinson

      Hey Vishal, That is what this script and my post details. You need to create an array with the details of each VM then pass that to the invoke-parallel function. DR

  4. Gulab Pasha

    Hi Guys,

    I looking for a script where I can use my customized VHD and deploy multiple vms parallely with the following.

    1 – Resource Group
    20 VMs with 20 NIC cards, 20 Public IP, 20 storage accounts, 1 network security group and 1 Virtual Network

    Same as above but with my own VHD as source.

    Looking forward to your support.

    Thanks,

  5. Gulab Pasha

    Hi Simon,

    Thanks for your reply, I’m looking for a PS script where I can execute and create multiple vms in one go using my customized Windows/Linux VHD (Source).

    Example script is attached.

    #Global Variables

    # Where do we want to put the VM’s

    $global:locName = ‘Central India’

    # Resource Group name

    $global:rgName = ‘DBTraining’

    # Virtual Network Name

    $global:virtNetwork = ‘DBTrng-Net10’

    # Storage account names must be between 3 and 24 characters in length and use numbers and lower-case letters only

    $global:stName = ‘dbtgngstg’

    # VMName

    $global:NewVM = $null

    # MIM Servers to Auto Deploy

    $VMRole = @()

    $VMRole += ,(‘Attendee01’)

    $VMRole += ,(‘Attendee02’)

    $VMRole += ,(‘Attendee03’)

    $VMRole += ,(‘Attendee04’)

    $VMRole += ,(‘Attendee05’)

    # Authenticate to the Azure Portal

    Add-AzureRmAccount

    # Get the UserID and Password info that we want associated with the new VM’s.

    $global:cred = Get-Credential -Message “Type the name and password for the local administrator account that will be created for your new VM(s).”

    $SubscriptionName = Get-AzureRmSubscription | sort SubscriptionName | Select SubscriptionName

    $SubscriptionName = $SubscriptionName.SubscriptionName

    Select-AzureRmSubscription -SubscriptionName $SubscriptionName

    # Create Resource Group

    New-AzureRmResourceGroup -Name $rgName -Location $locName

    # Create RG Storage Account

    $storageAcc = New-AzureRmStorageAccount -ResourceGroupName $rgName -Name $stName -Type “Standard_GRS” -Location $locName

    # Create RG Subnet

    $singleSubnet = New-AzureRmVirtualNetworkSubnetConfig -Name singleSubnet -AddressPrefix 10.0.0.0/24

    # Create RG Network

    $global:vnet = New-AzureRmVirtualNetwork -Name $virtNetwork -ResourceGroupName $rgName -Location $locName -AddressPrefix 10.0.0.0/16 -Subnet $singleSubnet

    # VM Config for each VM

    $VMConfig = @()

    # Create VMConfigs and add to an array

    foreach ($NewVM in $VMRole) {

    # ******** Create IP and Network for the VM ***************************************

    # *****We do this upfront before the bulk create of the VM**************

    $pip = New-AzureRmPublicIpAddress -Name “$NewVM-IP” -ResourceGroupName $rgName -Location $locName -AllocationMethod Dynamic

    $nic = New-AzureRmNetworkInterface -Name “$NewVM-NIC” -ResourceGroupName $rgName -Location $locName -SubnetId $vnet.Subnets[0].Id -PublicIpAddressId $pip.Id

    $vm = New-AzureRmVMConfig -VMName $NewVM -VMSize “Standard_A1”

    $vm = Set-AzureRmVMOperatingSystem -VM $vm -Windows -ComputerName $NewVM -Credential $cred -ProvisionVMAgent -EnableAutoUpdate

    $vm = Set-AzureRmVMSourceImage -VM $vm -PublisherName MicrosoftWindowsServer -Offer WindowsServer -Skus 2012-R2-Datacenter -Version “latest”

    $vm = Add-AzureRmVMNetworkInterface -VM $vm -Id $nic.Id

    # VM Disks. Deploying an OS and a Data Disk for each

    $osDiskUri = $storageAcc.PrimaryEndpoints.Blob.ToString() + “vhds/WindowsVMosDisk$NewVM.vhd”

    $DataDiskUri = $storageAcc.PrimaryEndpoints.Blob.ToString() + “vhds/WindowsVMDataDisk$NewVM.vhd”

    $vm = Set-AzureRmVMOSDisk -VM $vm -Name “windowsvmosdisk” -VhdUri $osDiskUri -CreateOption fromImage

    $vm = Add-AzureRmVMDataDisk -VM $vm -Name “windowsvmdatadisk” -VhdUri $DataDiskUri -CreateOption Empty -Caching ‘ReadOnly’ -DiskSizeInGB 10 -Lun 0

    # Add the Config to an Array

    $VMConfig += ,($vm)

    # ******** End NEW VM ***************************************

    }

    # In Parallel Create all the VM’s

    $VMConfig | Foreach-Parallel -ImportVariables -ScriptBlock {

    New-AzureRmVM -ResourceGroupName $rgName -Location $locName -VM $_

    }

    Thanks,
    Gulab Pasha

  6. Gulab Pasha

    As I’m new to Powershell and Azure, I tried to execute the above script to test and create multiple VM’s, I get the following error.

    Environment : AzureCloud
    Account : “my account”
    TenantId : “my Tenant ID”
    SubscriptionId : “my Subscription ID”
    SubscriptionName : Free Trial
    CurrentStorageAccount :

    WARNING: Unable to acquire token for tenant ‘Common’

    Account : “my account”
    TenantId : “my Tenant ID”
    SubscriptionId : “my Subscription ID”
    Environment : AzureCloud

    ResourceGroupName : DMTraining
    Location : centralindia
    ProvisioningState : Succeeded
    Tags :
    TagsTable :
    ResourceId : /subscriptions/mysubsID/resourceGroups/DMTraining

    WARNING: The output object type of this cmdlet will be modified in a future release.
    WARNING: The output object type of this cmdlet will be modified in a future release.
    WARNING: The output object type of this cmdlet will be modified in a future release.
    WARNING: The output object type of this cmdlet will be modified in a future release.
    WARNING: The output object type of this cmdlet will be modified in a future release.
    WARNING: The output object type of this cmdlet will be modified in a future release.
    WARNING: The output object type of this cmdlet will be modified in a future release.
    WARNING: The output object type of this cmdlet will be modified in a future release.
    WARNING: The output object type of this cmdlet will be modified in a future release.
    WARNING: The output object type of this cmdlet will be modified in a future release.
    WARNING: The output object type of this cmdlet will be modified in a future release.
    ForEach-Object : Cannot bind parameter ‘RemainingScripts’. Cannot convert the “-ImportVariables” value of type “System.String” to type
    “System.Management.Automation.ScriptBlock”.
    At line:139 char:13
    + $VMConfig | Foreach -ImportVariables – {
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : InvalidArgument: (:) [ForEach-Object], ParameterBindingException
    + FullyQualifiedErrorId : CannotConvertArgumentNoMessage,Microsoft.PowerShell.Commands.ForEachObjectCommand

    PS C:\Users\gulab>>

    The script has successfully created 1 Storage account, 1 VNet and 5 NIC and 5 Public IPs but it failed to created VM’s

    Thanks,
    Gulab Pasha

  7. The Chad

    Just came across this article now…..
    Great blog post to demonstrate how one can increase their deployment efficiency using the invoke-parallel function….thanks for sharing DR 😉

Comments are closed.