Just before re:Invent, AWS launched a new feature for CloudFormation … Create Stack with existing resources! This gives a lot of options to start to manage existing resources as code, if they were created via GUI or CLI. Apart from that though, it now gives you the ability to move resources between stacks, rename stacks, etc.
There is a pretty good AWS blog, but there are some things the blog really didn’t cover too well. I’ll go into some of the wins and discoveries I’ve made in testing this out.
I’ll start off with the easiest option, the basic import. In this example I’m using an EC2 instance, but the techniques are the same for any resource that isn’t currently managed by CloudFormation.
Step 1. I’m not going to show this here, but create a CloudFormation template that EXACTLY matches the resource(s) you want to import. I have a generic template for a Windows server. I could still use several of the parameters, but I also choose to hard code some of the items, mainly the tags. Initially, I left in all my standard tags, and while the import was successful, the extra tags in my template caused a drifted status. Not ideal, but interesting from a learning perspective.
The second big requirement is that you need to have a DeletionPolicy for each resource, best to have that set to Retain.
Step 2. Now that you have the prep, it is time to get started. Go to “Create stack” drop-down and select “With existing resources”
You’ll get a window explaining that you need a template and giving some information. I didn’t bother to include that as it’s just a lot of text and nothing to enter. After you get through that screen is the one to enter your template. This is not too dissimilar to the standard, but you have another infobox reminding you that you need the DeletionPolicy set.
Step 3. Once your template is loaded, you get this new screen. For each resource listed in your template, you’ll be asked to enter the appropriate identifier. In this case, it’s the InstanceID of the instance. Enter in that value and hit Next.
Step 4. This next screen is again pretty familiar. Enter in the stack name and any parameters. Nothing unusual here, but remember, the parameters need to match the existing resource.
Step 5. That’s everything that you need to enter. At this step, you are presented with the overview. Along with that, it also shows the change(s) that will happen.
Step 6. Assuming everything is all good, CloudFormation goes off and does the import of the resources. The other thing it does, as the keen-eyed may see in the below, is to add the AWS stack-level tags to each resource, just as it would if you were creating the resources. Note: This is the only addition/update it will allow.
You will also see the new stack listed.
While importing a resource into a new stack is actually pretty straightforward, assuming you have the deletion policy and match the current resource, there were other things I wanted to test.
Note: All my examples were done with an EC2 resource, but similar questions and issues can be applied to other resource types.
So, what if I have an EC2 instance that was created via CloudFormation, but someone manually added an extra drive/volume? Could I use this new functionality to update my template?
This turned out to be a bit more of a challenge, and I tested various things.
Firstly, along with the new option to create a stack from existing resources, you can choose stack Actions to import resources to your current stack. Yay! … ish.
Attempt 1. For my first attempt, I had a look at the Drift report. It showed a new BlockDeviceMapping, so I tried to just update that section with a new Ebs volume.
As can be seen, while I was adding a new Ebs volume, in this method I’m not directly importing a new resource. I sort of suspected this, but it was worth a shot.
Attempt 2. This time, I tried to create a new volume via AWS::EC2::Volume and then add this to the instance via the Volume property. Again, a fail.
While it would have been happy to create the volume, because I’d updated the configuration of the AWS::EC2::Instance resource, the import function wasn’t happy.
How to fix?
So, how do you fix this situation? You can’t. If you’ve modified your resource in a way that a standard “Update” won’t fix, you can’t do it this way.
What bloody good is this then? Well:
- start by going back and updating your stack to have a DeletionPolicy: Retain for each resource
- make sure you have a copy of your template, or just copy it from the Template tab of your stack and take note of the Parameters
- delete your stack! With the deletion policy, your resources should be untouched.
- update your template to include the new resource (in my case the extra volume as a BlockDeviceMappings entry)
- do a new create stack with existing resources!
Why Import into stack?
OK, so if I had to delete and recreate the stack, what’s the good of the “Import resources into stack”? This is really for when you have, or want, multiple resources in your stack. In this simple example, what if I wanted to include that Security Group into my stack? I can do that with the Import into stack. I can also tidy up my CloudFormation to now use a Ref to the Security Group rather than passing it as a Parameter.
These new features really help you clean up an environment that either started poorly or drifted over time. If the stack doesn’t exist, you can now create one from existing resources. If you want to restructure your stacks, you can do that too. Just make sure you have that DeletionPolicy set and you are good to go! Now, get out there and start cleaning up those messy environments.