I recently implemented an AWS site-to-site VPN for a customer to connect their on-premise network to their newly deployed AWS account.
The requirement was network level connectivity from their on-premise network to their management VPC. Support of production VPC resources would be carried out from bastion hosts in the management VPC.
The setup of this was simple from an AWS perspective. With Cloud Formation we deployed a Customer Gateway (CGW) using the IP address of their on-premise firewall, created a Virtual Private Gateway (VPG) and then the VPN Gateway (VPN).
The on-premise configuration took a bit of research to get right but once configured correctly it worked as expected.
However, after deployment it was determined that an on-premise server needed to connect to a Production VPC resource.
We already have a VPC peer connection between the management and production VPCs, but a VPN will only route traffic to the VPC it is connected to and VPC peer connections are not ‘transitive’,
For a more detailed explanation of this see; https://docs.aws.amazon.com/vpc/latest/peering/invalid-peering-configurations.html
The solutions considered to allow access from on-premise to the production VPC resource were;
- Create another VPN connection from the on-premise datacenter to the production VPC
- Deploy an application proxy in the management VPC
- Deploy the newly announced AWS Transit Gateway service
The customer wasn’t keen on adding VPN connections, as it would add configuration and complexity to the on-premise firewall, and we weren’t confident that an application proxy would work, so we decided on the new Transit Gateway service.
AWS Transit Gateway
AWS Transit Gateway was release at the end of 2018. It will allow our customer to connect their on-premise network to both of their AWS VPCs, and any future VPCs, without having to configure and support multiple VPN endpoints on their on-premise firewall and support multiple VPN gateways in AWS.
The steps to implement this were fairly simple, however Cloud Formation doesn’t cover all of the steps;
Deploy the ‘Transit Gateway’ and ‘Transit Gateway Attachment’ for the VPCs
This Cloud Formation template assumes that two VPCs already exist, and each have one subnet.
The VPC and subnet Ids need to be entered into the parameters section. The on-premise VPN endpoint is setup after the AWS VPN setup, but the IP address is added to the VPNAddress parameter. This also assumes a non BGP on-premise endpoint.
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Transit Gateway",
"Parameters": {
"VPCIdMgmt": {
"Type": "String",
"Description": "Management VPC Id",
"Default": "vpc-12345678901234567"
},
"VPCIdProd": {
"Type": "String",
"Description": "Production VPC Id",
"Default": "vpc-9876543210987654"
},
"MgmtPrivateAzASubnetId": {
"Type": "String",
"Description": "Az A Subnet in Mgmt VPC",
"Default": "subnet-23456789012345678"
},
"ProdPrivateAzASubnetId": {
"Type": "String",
"Description": "Az A Subnet in Prod VPC",
"Default": "subnet-34567890123456789"
},
"VPNAddress": {
"Type": "String",
"Description": "On-premise VPN endpoint",
"Default": "201.65.1.1"
}
},
"Resources": {
"CustomerGateway": {
"Type": "AWS::EC2::CustomerGateway",
"Properties": {
"Type": "ipsec.1",
"BgpAsn": "65000",
"IpAddress": {
"Ref": "VPNAddress"
}
}
},
"TransitGateway": {
"Type": "AWS::EC2::TransitGateway",
"Properties": {
"AmazonSideAsn": 65001,
"DefaultRouteTableAssociation": "enable",
"DefaultRouteTablePropagation": "enable",
"Description": "Transit Gateway",
"DnsSupport": "enable",
"VpnEcmpSupport": "enable"
}
},
"TransitGatewayMgmtAttachment": {
"Type": "AWS::EC2::TransitGatewayAttachment",
"Properties": {
"SubnetIds": [{
"Ref": "MgmtPrivateAzASubnetId"
}],
"TransitGatewayId": {
"Ref": "TransitGateway"
},
"VpcId": {
"Ref": "VPCIdMgmt"
}
}
},
"TransitGatewayProdAttachment": {
"Type": "AWS::EC2::TransitGatewayAttachment",
"Properties": {
"SubnetIds": [{
"Ref": "ProdPrivateAzASubnetId"
}],
"TransitGatewayId": {
"Ref": "TransitGateway"
},
"VpcId": {
"Ref": "VPCIdProd"
}
}
}
},
"Outputs": {
"CustomerGateway": {
"Description": "CustomerGateway Id",
"Value": {
"Ref": "CustomerGateway"
},
"Export": {
"Name": "TransitGateway-CustomerGatewayId"
}
},
"TransitGateway": {
"Description": "TransitGateway Id",
"Value": {
"Ref": "TransitGateway"
},
"Export": {
"Name": "TransitGateway-TransitGatewayId"
}
}
}
}
After the Cloud Formation stack is deployed, the ‘Outputs’ section will list the CustomerGatewayId and TransitGatewayId, these are needed in the next steps.
Create the site to site VPN
This step is completed in the AWS CLI as Cloud Formation doesn’t support it yet. Change customer-gateway-id and transit-gateway-id to the values in the output section of the Cloud Formation stack, or look it up in the AWS console.
aws ec2 create-vpn-connection --customer-gateway-id cgw-045678901234567890
--transit-gateway-id tgw-56789012345678901 --type ipsec.1 --options "{\"StaticRoutesOnly\":true}"
Create the VPN Transit Gateway route
The attached VPCs have had routes added by default, but as we are using non-BGP on-premise endpoint, the VPN needs routes specifically added.
The route we are adding here is the CIDR of the on-premise network e.g. 172.31.0.0/16
Get the Id of the ‘Transit Gateway Route Table’ and VPN’s ‘Transit Gateway Attachment Id’ from the AWS console under ‘Transit Gateway Route Tables’ and ‘Transit Gateway Attachments’
aws ec2 create-transit-gateway-route --destination-cidr-block 172.31.0.0/16
--transit-gateway-route-table-id tgw-rtb-67890123456789012 --transit-gateway-attachment-id tgw-attach-7890123456789012
Configure VPC subnet routing
The routing that you will see configured on the Transit Gateway is only used within the Transit Gateway itself. So we now need to manually add routes to VPC subnets that you want to use the VPN.
In our case we are leaving VPC-VPC traffic to use the VPC peer, and only adding an on-premise network to the subnet routes.
Get the Transit Gateway Id from the Cloud Formation template output, and get the route VPC subnet’s route table Id;
aws ec2 create-route --route-table-id rtb-89012345678901234 --destination-cidr-block 172.31.0.0/16 --transit-gateway-id tgw-56789012345678901
Summary
Using a Transit Gateway can make site-to-site VPNs simpler and less messy by allowing a single VPN connection to AWS that can reach more than one VPC.
One important limitation is that Transit Gateway doesn’t yet support security groups.
If you use security groups over VPC peer connections, and switch from VPC peer connections to Transit Gateway, you will see your security groups become listed as ‘stale’, you would need to re-add them as IP based rules.
The Transit Gateway FAQs states the following;
- Q: Which Amazon VPC features are not supported in the first release?
- A: Security Group Referencing on Amazon VPC is not supported at launch. Spoke Amazon VPCs cannot reference security groups in other spokes connected to the same AWS Transit Gateway.
This implies that it will be supported in the future, but for now resource access between VPCs using security groups must remain over VPC Peer connections.
More information on Transit Gateway
https://docs.aws.amazon.com/vpc/latest/tgw/working-with-transit-gateways.html