Ivo Ivanov is the CEO of DE-CIX.
Seemingly all of our technology is smart in 2025, but not all of it feels smart. You can ask Gemini to summarize a news article, get ChatGPT to act as your interior designer or create photorealistic images and videos from a single prompt. This year’s Consumer Electronics Show (CES) showcased an LLM-powered humanoid robot called Aria capable of engaging in real-time conversations. BMW unveiled Panoramic iDrive, an in-vehicle AI-powered assistant projected onto the driver’s windshield that offers live navigation and real-time driving assistance.
We’ve lived with voice assistants for a while, and carry LLMs around in our pockets, and they work well for the most part. But when you’re talking about vehicle-to-cloud driving assistance or getting real-time translations during a conversation, the cracks begin to show. The technology is there, and demonstrations in isolation are impressive, but do we have the connectivity infrastructure to roll these technologies out at scale?
A delay while a chatbot generates a text-based answer or some buffering while you’re watching Netflix is tolerable, if annoying. But any noticeable lag in any of the applications showcased at CES would render them unusable. More often than not, with these AI-powered applications, that lag has less to do with your device or Wi-Fi and more to do with something we rarely consider: geography.
Data takes time to travel. We’ve always known that distance matters when it comes to streaming or online gaming, but when it comes to AI—especially the kind that interprets voice, vision or motion in the moment—those distances become more crucial. The further you are from where your AI model “lives,” and the more inefficient the pathway to your device, the slower it feels. Milliseconds matter, and latency is the new currency.
The Triangle Of Inference
For AI to feel truly “real time,” whether it’s translating speech in smart glasses or alerting a driver about an obstacle ahead, three components need to work in perfect harmony. Imagine a triangle. On one corner, you have the AI model itself: the trained neural network that understands your voice, decodes your surroundings or generates a response. These models are often stored in powerful data centers or, increasingly, on edge servers closer to users. On the second corner is the device: the glasses, the car, the robot or the sensor capturing the information. The third corner is the connection: the digital thread that links them together, whether via fiber-optic cables, mobile networks like 5G or even low Earth orbit satellites.
The magic of real-time AI happens only when this triangle is tight, when the model is close enough to the device and the network between them is fast and stable. If the connection is spotty or the model is hosted hundreds of miles away, even the most advanced AI stumbles. A visual translation might lag a few seconds too long. These pauses break the illusion of seamless intelligence. The triangle doesn’t need to collapse completely to cause issues. A slight wobble—a jittery network, an overloaded compute node or a data center that’s too far away—is enough to disrupt the entire experience.
And here’s the real catch: This fragility isn’t something you can fix with a software update or a faster phone. It’s a systems-level problem, rooted in where infrastructure physically exists and how data is routed between devices and AI engines. Today, many consumer AI experiences are built on networks optimized for throughput, not proximity. But AI, particularly inference-based AI that reacts to the world in real time, doesn’t just need data to arrive. It needs it to arrive now. This design challenge will define which innovations make the leap from stages like the one at CES to daily life.
Why Geography Matters
It’s easy to assume that the internet has made location irrelevant. After all, you can stream, scroll and speak to AI from just about anywhere. But when it comes to real-time intelligence, geography still matters—a lot. That’s because most of today’s network infrastructure was built for reach instead of responsiveness. Data often travels through long-haul fiber connections, passing through towns and cities that offer no local breakout or interconnection points. These places have become digital “flyover zones,” where the infrastructure exists but can’t be accessed in a meaningful way. Even if your home has fast broadband or 5G coverage, your AI data might still be taking the long road to a server hundreds of miles away before coming back to your device.
This creates delays that aren’t always visible, but are increasingly noticeable in AI-driven experiences. The answer to frustrating online experiences used to be a faster connection or greater bandwidth, but now it’s smarter placement and optimization of infrastructure. We need smaller interconnection hubs that sit closer to end users, allowing devices and AI models to communicate without crossing half the country.
That shift in mindset is already starting to happen behind the scenes. Engineers and infrastructure planners are rethinking what the next layer of connectivity should look like. Ideally, this will be a distributed web of smaller, smarter interconnection points placed within 50 to 100 miles of where people actually live and work. These hubs don’t need to be complex; in some cases, they’re no bigger than a server rack. But they provide a crucial local handshake between devices, data and intelligence.
Whether it’s powering AI assistants in cars, routing language models for instant translation or coordinating machines in a smart warehouse, the principle is the same: Bring the intelligence closer, and everything starts to work better. When AI is expected to live in our homes, cars and cities—and in our pockets—connectivity needs to be more than just fast. It needs to be close.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?