What is AWS Transit Gateway (TGW) for Direct Connect (DX)?
AWS released Transit Gateway (TGW) at re:Invent 2018. It was a major enhancement in how VPCs can connect to each other. In the same event, they also promised Direct Connect(DX) for TGW to be released in the first quarter of 2019. Now, DX support is finally available and we’ve now had a chance to deploy it and get a first-hand look. In this article, we are going to share our learnings about using DX with TGW, and share what we think the advantages and disadvantages are.
Having the ability to connect DX directly into TGW, allows Direct Connect users to take advantage of the high-speed dedicated link between their on-premises and their AWS VPCs that are attached to Transit Gateway. This is a good addition to the current TGW offering, but it also comes with limitations that are important to understand.
At a high level, in order to connect your DX to TGW, you need to follow these steps:
- Create a Direct Connect Gateway
- Create and/or connect your Transit Virtual Interface (VIF) to the Direct Connect Gateway.
- Associate your Direct Connect Gateway with your Transit Gateway, as seen below
- Optional: This attachment goes to the default route table by default. You can modify attachment from AWS TGW
Once everything is connected and built, the connection topology looks like this:
Aviatrix TGW Orchestrator help simplifies various aspects of deployment and management of all elements of TGW deployment. At Aviatrix, we believe in using “cloud native” where it makes sense. As such, the Aviatrix Controller has the ability to deploy and manage Direct Connect association with TGW, in a centrally managed and automated fashion (Available in R4.6). This alleviates some of the pains that go with orchestrating DX connectivity to TGW, however, other underlying limitations will still apply that Cloud Architects need to be aware of – and the Aviatrix Controller with TGW Orchestration helps overcome these other limitations as well.
What are the limitations of Direct Connect for Transit Gateway?
While having the ability to connect DX directly to AWS Transit Gateway is advantageous from the perspective of simplicity, practitioners should be aware that some important limitations pertinent to Direct Connect apply in this scenario. The major limitations that need to be highlighted are as follows:
- Propagation from TGW VPC CIDRs to on-premises is completely manual
- Maximum number. of BGP routes advertised from on-premises to cloud is capped at 100
- Maximum number of routes advertised from TGW to on-premises is capped at 20
- Maximum of 1 transit virtual interface per direct connect
(For more information, please refer to the table below for the full list of Direct Connect limitation as of the time of this writing.)
Source: https://docs.aws.amazon.com/directconnect/latest/UserGuide/limits.html
How do these limitations affect a deployment?
At first glance, these limitations may not seem problematic. However, they can become disruptive down the line if not considered carefully. Let’s look at how these limitations can adversely impact a hybrid cloud deployment that leverages AWS Transit Gateway.
Manual and limited route propagation from cloud to on-prem
Let’s start with the most impactful drawback here, which is the need to manually configure routes to be propagated from your AWS Cloud/TGW to on-premises. As you can see in Figure 2, the dialogue for associating a Direct Connect Gateway to AWS Transit Gateway shows that you need to manually enter the CIDRs required for advertising VPC CIDRs to Direct Connect Gateway, which then will be sent down to your on-premises edge routers.
After all, manual configuration of a system that can change often goes against the cloud model. Anything that’s configured manually results in inelasticity and is error prone.
The other issue is that you can only send 20 routes from cloud to on-premises.
How about a gigantic summary route?
One may argue that you can summarize the routes at a high level (i.e at class boundary), and then you never have to worry about reconfiguring the routes later. The problem with this method of summarization is that it can lead to routing issue, such as asymmetric routing and suboptimal routing. For example, if you have two data centers that are connecting to your cloud, then you have to make complex routing decisions on your data center edge router to avoid potential problems. In some ways, this is even worse than manually configuring specific routes.
In the figure below, you can see an example of a topology where this level of summarization will lead to big routing problems with your hybrid cloud connection. In this case, without using tedious routing policies on the edge Routers, a connection can traverse from the cloud → data center in the west region, and then return packet can go up from data center up to the east region.
Transit Virtual Interface Limitation
As mentioned earlier, in order to connect your direct connect to AWS transit gateway, you need to connect to DX Gateway using a transit VIF. A single direct connect line is going to support a single transit VIF. If you previously split your direct connect to individual VIFs, this will not be supported by transit VIF. Therefore, you will need to plan accordingly to ensure this limitation does not cause a surprise down the line.
Can’t use Direct Connect Gateway to bridge multiple TGWs
Another interesting learning for us was that DX gateway can’t connect multiple TGWs, even though it can connect to up to 3 TGWs. That sounds unintuitive, but it is true. For instance, if you terminate your transit VIF into a DX gateways, and then connect that DX gateway to three TGWs, the VPCs that are connected to TGW 1 can’t talk to VPCs that are attached to TGW 2 through the DX gateway. Therefore, you can’t use your DX gateways to do transit routing. For that, you will need another solution, such as Aviatrix transit solution, to connect TGWs together.
How do you avoid the limitations of Direct connect
Aviatrix can help avoid limitations of AWS Transit Gateway. With Aviatrix, you will be able to leverage the privacy and performance of the Direct Connect, while you can avoid the limitations of Direct Connect route propagation. Aviatrix will help you orchestrate connectivity using Direct Connect to TGW, and with the help of Aviatrix Edge Gateways and On-premises appliance, you will be able to learn the routes from your on-premises router and propagate them to other route tables in your TGW, based on the connection policies that you have defined in Aviatrix. Our “InsaneMode” high performance encryption technology will help you get full in-transit encryption, while achieving near line-rate performance of Direct Connect.
Additional Benefits of Aviatrix Controller and TGW Orchestration
Aside from connectivity to on-premises, Aviatrix Controller will also orchestrate and manage various elements of AWS Transit Gateway itself, that helps simplify deployment and management of various routing elements in the cloud. Some of the important features of Aviatrix TGW Orchestration are as follows:
Become the cloud networking hero of your business.
See how Aviatrix can increase security and resiliency while minimizing cost, skills gap, and deployment time.