I've gone from writing about SD

This spring, my wife and I moved from the Denver suburbs into the Colorado Rockies. The high-altitude retreat presented a welcome reprieve from our otherwise tech-dominated lives. As remote workers, however, we faced certain challenges, and technology was the only reason our move was possible in the first place. Prior to my time at El Reg, I, among other things, breathlessly covered a subset of the networking industry, which as it happens had become big business following the COVID-19 outbreak and the pandemic that followed. I’m speaking, of course, about software defined wide-area-networking (SD-WAN). Little did I know, a mere six years later, this technology would become essential to my wife and me. REG AD Had it not been for 5G and Starlink, the move might as well have been time travel. To put it into perspective just how far into the foothills we’d ventured, the fastest wired connection at our disposal was DSL. CenturyLink promised, but notably did not guarantee, up to 20 Mbps down and a staggering 1.5 Mbps up. REG AD Compared to the symmetrical gigabit fiber line we’d enjoyed in the Denver suburbs for the past six years, relying on DSL alone felt positively medieval. To those, including many of my neighbors, living this reality, you have my sympathies. Thankfully, DSL wasn’t our only option. For better or worse, 5G and Starlink are helping to close the digital divide in a meaningful way — but individually they’re still far from perfect. Where we live, T-Mobile’s 5G internet service delivers near-gigabit download speeds and uplink speeds ranging from 30–40 Mbps. Meanwhile, Starlink promised between 100 and 400 Mbps downlink depending on the plan, with uplinks that, at least for us, capped out at around 30 Mbps. Either would be fast enough for our needs, but reliable? Turns out there’s still some room for improvement. In the past two months we’ve experienced no fewer than a dozen dropped connections lasting anywhere from a minute all the way to five. Making matters worse, the majority of these drops took place during working hours. With a relatively clear view of the sky, we’ve found Starlink to be more reliable, but even SpaceX isn’t immune to the occasional outage. That’s not to mention the threat of space junk kicking off a Kessler syndrome-like event. SD-WAN to the rescue The easy way to approach multi-WAN would have been to set up two networks, each with its own SSID. In the event one went down, we’d switch to the other. While simple, it’s far from perfect, and does nothing to mitigate disruptions to the voice and video calls on which my wife and I spend a considerable amount of our week. With two WiFi access points, we’d also have to contend with interference from overlapping channels. REG AD Wrangling multiple LANs, wireless SSIDs, and firewall rules isn’t exactly ideal. Thankfully, there’s a better way, and we just so happened to have all the equipment we needed. SD-WAN was developed in the early 2010s to solve a very specific problem: as SaaS apps like Office 365, Salesforce, and Google Apps took off, more and more enterprise traffic was going out to the internet. This was a problem for a lot of enterprise WANs, which often relied on expensive MPLS links designed to connect satellite offices to servers running back at headquarters or regional datacenters. In many cases, these WANs were built to prioritize reliability over bandwidth, and due to their topology required backhauling traffic tens or hundreds of miles just to access the internet. SD-WAN gateways offered an alternative. These devices could route internal traffic over the private WAN while SaaS and other internet-bound traffic could be piped over whatever local ISP served businesses in that area. SD-WAN is an entire can of worms in itself, but for my purposes two key capabilities stood out: multi-WAN and policy-based routing support. Our router, a Ubiquiti UniFi Dream Machine SE (UDM-SE), already supported both. The hardware The UDM-SE is not Ubiquiti’s newest nor even its most capable gateway. Introduced in 2022, the all-in-one appliance combines a 3.5 Gbps router (with IPS/IDS enabled) with an 8-port gigabit PoE switch, an integrated network video recorder, and critically two WAN ports, one good for 2.5 Gbps and another capable of 10 Gbps. Ubiquiti’s SD-WAN implementation is somewhat basic compared to gateways from HPE, Cisco, or Extreme. However, of the bevy of networking and security features SD-WAN encompasses, we only really needed a handful – multi-WAN and policy-based routing were the main ones – and Ubiquiti is kind enough not to gate this functionality behind an enterprise license. REG AD The setup itself was about as simple as it gets: plug each ISP-provided gateway into one of UDM-SE’s WAN ports and tell it whether you want to load-balance traffic across the two, or failover in the event of an outage or dropped connection. In practice there were a few extra steps, including disabling the 5G and Starlink gateways' onboard routing and WiFi functionality. Starlink supports a bypass mode, but the T-Mobile modem+router combo didn’t. We were able to disable the onboard WiFi, but only using an onboard utility. Putting it to the test With everything connected, we opted to configure the UDM-SE for WAN failover with T-Mobile as the primary and Starlink as the backup, rather than load-balancing traffic across the two, in part because the 5G link was so much faster. The first few days after setting everything up, my wife and I were questioning whether we even needed Starlink. But then we saw the notification: “Internet connection WAN1 (T-Mobile USA) on port 9 is down and WAN2 (Starlink) is now active,” and then a few minutes later, “Internet connection WAN1 (T-Mobile USA) on port 9 is restored after failing over to WAN2 (Starlink).” Both notifications were dated two hours earlier. We just hadn’t noticed. In fact, despite at least a dozen dropped connections over the past two months, the UDM-SE failed over fast enough that it was never an issue. No dropped calls, no buffering video, no timed-out web pages. Perhaps the most surprising discovery was that while Starlink didn’t offer the fastest or lowest-latency connectivity, it was remarkably reliable. Despite less than perfect alignment and view of the sky, the logs show just two dropped connections in the past month, one of which happened in the middle of the night when no one was awake to notice. While WAN failover kept us online, it didn’t do anything to guard against network congestion. The UDM-SE offers a preconfigured quality-of-service (QoS) policy that in theory should detect Zoom or MS Teams traffic and prioritize it over something like Windows Update or YouTube. But since we had two WANs, we could go one better and take advantage of another SD-WAN favorite: policy-based routing. This allowed us to force certain devices or services to prioritize one ISP over the other. For example, because Starlink had proven to be the more reliable of the two links, I wrote a policy to route traffic from my wife’s work laptop over the satellite link first and fail over to T-Mobile in the event of an outage. This also had the benefit of ensuring that if I happened to kick off a large game or AI model download while she was in the middle of a meeting, I wouldn’t get myself in trouble for turning her Zoom call into a pixelated mess. In the more rural parts of the US where wired connectivity isn’t a given, cellular home internet and low-Earth orbit (LEO) satellite communications, like Starlink, have gone a long way toward closing the digital divide — particularly for those working from home. However, individually they’re still far from perfect. Dropped connections weren’t a daily occurrence, but they still occurred more frequently and for longer than we would have liked. Five minutes is a long time to wait wondering whether the person on the other end of the Zoom call will be there when you eventually get back online. Technologies like SD-WAN can help to mitigate the problem by allowing multi-WAN failover, load balancing, and policy based routing, but to take advantage of them also requires paying for two ISPs and hundreds of dollars of equipment, which may or may not require monthly subscriptions or annual licenses. We were fortunate that the hardware we already owned supported these features and didn’t lock them behind a pricy enterprise license. Even still, we’re paying more for internet than we’d like. Between T-Mobile and Starlink, we’re spending around $125 a month to get online. Along with the higher price, network latency is also a lot higher than we’d like. Compared to our old fiber line, which routinely achieved latencies under 5 ms, T-Mobile and Starlink ranged from 15 ms at best to as much as 80 ms at worst. The only online game I play regularly is a real-time strategy that’s old enough to drink. I’m not playing CS:GO or whatever twitch shooter is popular these days, so the higher latency wasn’t that big of a deal, but if you're a competitive gamer, a wired connection is still a must. The good news is T-Mobile and Starlink may not be our only options for high-speed internet for much longer. There are numerous ongoing efforts across the US and other nations to improve access to fiber connectivity, including one slowly working its way down the highway in our general direction. On a personal note, while it’s neat to experience a technology firsthand that I dedicated years of my life to writing about, the moment fiber-to-the-home comes, both our Starlink and 5G home internet subscriptions are getting the boot. ®