In this blog post we explore a recursive pattern for AWS Lambda. This pattern allows us to tackle potentially long running tasks such as ones requiring processing multiple items in the input or tracking the progress of a long running task.
Lets take a practical example. The Center for Internet Security (CIS) Foundations for AWS recommends us to “Ensure the default security group of every VPC restricts all traffic”. Here is the detailed description:
A VPC comes with a default security group whose initial settings deny all inbound traffic, allow all outbound traffic, and allow all traffic between instances assigned to the security group. If you don’t specify a security group when you launch an instance, the instance is automatically assigned to this default security group. Security groups provide stateful filtering of ingress/egress network traffic to AWS resources. It is recommended that the default security group restrict all traffic.
The default VPC in every region should have it’s default security group updated to comply. Any newly created VPCs will automatically contain a default security group that will need remediation to comply with this recommendation.
So we need to iterate over all the regions of our AWS account and delete the default security group rules. As AWS Lambda has a maximum timeout of 5 minutes, it would be difficult to get this done in a single invocation.
The approach we are going to take is to have our Lambda function retrieve the regions and then iterate over the regions deleting the rules in default security groups along the way. But it does this by recursively calling itself processing one region at a time. This is what this blog post is about.
Here is an example of code for the lambda handler function:
Here is whats going on in the handler function:
– We first check if the event passed to us has the regions
– If not, we get the regions data using ec2.describeRegions call
– Pop one region from the regions data and call the deleteRulesFromDefaultSG function with it.
Here is the (admittedly kludgy) code for the deleteRulesFromDefaultSG function:
– Once the function returns, we call the same lambda function again with the event object using the invokeLambda function. But the event object now has one region less! This is where we establish the recursive pattern. Popping and processing one region per invocation, eventually we will process all the regions and the handler function exits.
Here is the code for this invokeLambda function:
The complete Lambda code is available at: https://gist.github.com/kmkale/acfd0d99cd2ffb4e34670222a9768550
One reminder, the role under which this lambda function will run will need to have following permissions at a minimum:
- “lambda:InvokeFunction”
- “logs:CreateLogGroup”
- “logs:CreateLogStream”
- “logs:PutLogEvents”
- “ec2:DescribeRegions”
- “ec2:DescribeSecurityGroups”
- “ec2:RevokeSecurityGroupEgress”
- “ec2:RevokeSecurityGroupIngress”
By processing only one region per invocation, the lambda function can easily stay under the maximum 5 minutes timeout limitation. This pattern is reusable in many situations where you have to process a number of items in the input event or have to track the progress of some long running task. Hope you liked it. Let me know your thoughts in the comments.