During my work on Subaru of America's Gen 2 telematics platform, we encountered a significant challenge related to network routing and vendor costs. Our communications vendor could easily handle vehicle communications to internet endpoints with static IP addresses as part of the existing agreement. However, requirements emerged for vehicles to connect to several external vendors whose services were hosted on dynamic IP addresses. This dynamic routing capability was a premium service for the hardware in use, with a significant additional cost, presenting a substantial obstacle to enabling these necessary vendor connections.

The main technical hurdle was twofold. First, how do you consistently route traffic to an endpoint whose IP address changes? Second, how do you do that when all the traffic is HTTPS encrypted end-to-end? You can't just inspect the encrypted payload to see where it's supposed to go. My initial thoughts gravitated towards a proxy-based solution. Standard proxying is straightforward, but the dynamic nature of the target IPs and the encrypted HTTPS traffic presented difficulties. Simply setting up individual proxies for each vendor would be cumbersome and potentially costly, especially if requiring dedicated static IPs via services like AWS Global Accelerator for each one. But I realized that even within an encrypted HTTPS session, the initial TLS handshake includes the Server Name Indication (SNI) field in clear text, revealing the intended destination domain name. I identified HAProxy as a powerful software solution capable of inspecting this SNI field upon receiving traffic, allowing it to act as an intelligent proxy without needing to decrypt the payload.

The next challenge was directing the vehicles to send traffic to the HAProxy cluster instead of attempting direct connections to the vendors. I leveraged the fact that our communications vendor allowed specification of the DNS servers used by the vehicles. I designed and deployed custom, auto-scaling DNS servers within AWS using the standard named (BIND) software. These DNS servers were configured to resolve most domain names normally. However, for the specific domain names of the vendors requiring dynamic routing, the DNS servers were configured to return the static IP address(es) of the HAProxy infrastructure. This created a seamless flow: the vehicle would request the vendor's domain, receive my proxy's IP from my custom DNS, initiate an HTTPS connection to the proxy, the proxy would read the SNI, determine the correct destination, and forward the encrypted traffic appropriately.

This architecture provided a secure and scalable solution using primarily native AWS services (like Global Accelerator, Auto Scaling, etc.) alongside well-established open-source software (HAProxy, BIND). The only customer-managed components were the DNS and proxy instances themselves, designed for high availability and automatic scaling based on load. Crucially, the estimated annual operational cost for this AWS-based infrastructure was projected to be around $50,000 a year which was an order of magnitude smaller than the proposal.

By presenting this solution and its associated low cost, it demonstrated that we had a viable path forward without relying on their expensive dynamic routing service. This fundamentally shifted the negotiation. Consequently, we were able to get a significantly revised offer which reduced the fee to something closer to our proposed solution. At this new price point, it made vendor's managed service a reasonable option and we ultimately chose that path for operational simplicity. Although the solution wasn't deployed in production, it successfully fulfilled the strategic objective: an operationally viable alternative that helped avoid an significant vendor cost and ensured Subaru obtained the required functionality at a reasonable price. And it was pretty cool to design and build : )