This video provides a detailed explanation of the sophisticated system that streams the real-time locations of millions of drivers and riders while keeping the Uber app running smoothly. Uber overcame the problems of the initial polling approach and introduced push-based communication and the H3 hexagonal spatial index to maximize efficiency. The key innovations are edge servers and dead reckoning technology to minimize mobile app network latency and improve user experience.

1. Uber's Early System: The Limits of Polling

When a rider requests a trip on the Uber app, the rider's location is shared with Uber while nearby driver locations are shared with the rider. Behind this seemingly simple process lies enormous complexity to keep the system running smoothly for hundreds of millions of users. Early Uber used a polling approach, where the app (client) periodically asked the server if there was new data. It was like repeatedly asking: "Any new locations for nearby drivers? Any new locations? Any new locations?"

"Hey, are there new locations for nearby drivers? Are there new locations? Are there new locations?"

But this approach soon hit major problems:

• Unnecessary server load: Clients kept sending requests even when there was no new data, creating unnecessary server load.
• Battery drain and overhead: Each request added extra headers, increasing battery consumption and network overhead.
• 80% of network requests were polling: Eventually, 80% of all network requests hitting Uber's servers were polling calls.
• Slow app startup time: Multiple polling calls competing simultaneously interfered with UI rendering, causing cold startup times to spike.

These problems made Uber realize polling couldn't scale, and they began searching for new solutions.

2. Transition to Push-Based Communication: Enter Ramen

To solve polling's problems, Uber transitioned to push-based communication. Instead of clients requesting data, the server 'pushes' data to clients whenever new data arrives. For this, Uber developed Ramen.

"Ramen stands for realtime asynchronous messaging network."

While theoretically much more efficient, several challenges remained — specifically the logic for deciding 'when, what, and how' to push to clients. Uber separated these responsibilities into three microservices for efficient scaling.

2.1. Firewall: The Push Decision Maker

Firewall is the microservice that decides when to push. It receives all types of events and determines whether an update is worth pushing to the client.

• Key events: Rider trip requests, driver acceptances, new rider or driver locations.
• Smart decisions: Location information can change very frequently, but not every tiny change needs to be sent. Firewall compares new and previous locations and only sends updates when they've changed 'sufficiently.'

When Firewall determines a push is needed, it transmits the information to Uber's API Gateway.

2.2. API Gateway: Data Aggregation

The API Gateway takes the minimal push data from Firewall and collects all the data needed by the client — adding information like the user's locale settings and operating system.

2.3. Ramen Server: Payload Push

Finally, the client connects directly to the Ramen server, which pushes the final payload determined through the API Gateway. Initially, Ramen was built on SSE (Server-Sent Events) supporting 'at least once delivery.' Events pushed to the client are guaranteed to arrive at least once.

It has since migrated to gRPC, enabling bidirectional messaging between client and server. Now the server can push data to the client and the client can share data with the server, enabling more flexible communication.

3. Efficient Location Processing: Spatial Partitioning and H3

With the push-based system handling data delivery efficiently, a major challenge remained. Uber receives real-time updates from millions of locations per minute — how can they efficiently send only nearby driver locations to a specific rider?

The simplest approach would be calculating distances from the rider to all drivers and showing only those within a certain radius. But computing millions of distances for millions of riders and drivers would put enormous load on servers.

So Uber devised a much more ingenious approach: spatial partitioning.

3.1. The Concept of Spatial Partitioning

The core idea is dividing geographic space into smaller regions. This allows mapping driver locations to specific regions, and instead of calculating millions of distances, only checking drivers in the rider's region and neighboring regions.

However, simply dividing the world into squares had problems. Squares don't have equal distances to all neighbors, creating corner bias — diagonal squares can be farther away than adjacent ones. Ideally, a method was needed that could query nearby drivers based at least on an approximate radius.

3.2. H3: Hexagonal Spatial Index

To solve this problem, Uber developed H3, an open-source hexagonal spatial index.

• Honeycomb hexagons: H3 divides the world into honeycombs of hexagons instead of squares.
• Uniform distances: Each hexagon has nearly identical distances to all its neighbors.
• H3 index: Given any GPS coordinate, H3 calculates which hexagon it belongs to and returns the corresponding H3 index.
• K-ring queries: Instead of calculating distances to all drivers, a K-ring is requested from the hexagonal cell where the rider is located. A K-ring means all hexagons within K steps of the rider's cell.
  • K=1: Includes the first neighbor layer, totaling 7 cells.
  • K=2: Includes through the second neighbor layer.

This new system revolutionized time complexity. Previously it was O(N) proportional to millions of active drivers, but now it's O(K^2 + M), where K is the radius (k-ring size) and M is the number of nearby drivers. If there are 100 nearby drivers, Uber now only needs to check those 100 instead of potentially millions!

"If there are 100 nearby drivers, Uber now only has to iterate over those 100 drivers instead of possibly millions."

The H3 system is used not just for finding nearby drivers but also for dynamic pricing zone setup, ETA prediction, demand forecasting, and other critical Uber functions, optimizing server load and providing faster client responses.

4. Optimization for Poor Network Conditions

Due to the nature of mobile apps, internet connections are never consistently stable. Ride-sharing app users are primarily outside using mobile data, likely on cellular networks rather than stable Wi-Fi. How did Uber engineers optimize the app to work smoothly even in poor network conditions?

4.1. Minimizing Network Latency: Edge Servers

To keep network traffic fast, Uber adopted edge servers.

• Globally distributed servers: Hundreds to thousands of edge servers are distributed worldwide, each serving nearby users.
• Geographic proximity: For example, a Berlin user communicates with a geographically close server like Frankfurt instead of one in California.
• Data caching: These edge servers serve as first entry points for nearby drivers and user requests, and cache relevant data for even faster data access.

In typical 4G conditions, this change alone can speed up request processing by about 100 milliseconds — an enormous difference compared to communicating with a server 10,000km away!

4.2. Smooth Map Movement: Dead Reckoning & Kalman Filters

No matter how close the server is, mobile apps must always prepare for connection drops. There's no guarantee that new location data arrives reliably at consistent intervals. If driver location updates come only three times in 20 seconds, we need to prevent the driver icon from jumping between coordinates on the map.

For this, Uber uses dead reckoning technology.

• Position prediction: Dead reckoning predicts where a driver should be based on last known speed and direction. It estimates the vehicle's current position even without new location information.
• Combined with Kalman Filters: These predicted coordinates are combined with Kalman Filters. The Kalman Filter uses both predicted and actual measured coordinates so that even when new measurements differ significantly from predictions, the vehicle marker moves smoothly on screen.

5. An Interesting Controversy: Phantom Cars

Here's an interesting side note. There have been real claims and discussions that nonexistent fake vehicles may appear on the Uber app. If a vehicle doesn't exist, it wouldn't need location updates from the server, and it could prevent an area from looking like it has 'no cars available,' keeping riders from switching to competitor apps by making it appear there are more drivers than there actually are.

Uber has denied the phantom cars phenomenon, but this story demonstrates how incredibly complex the backend systems behind even the simplest-looking interfaces are, especially when massive scaling is required.

Closing

Uber's real-time map system goes far beyond simple map display. It's the product of countless advanced technologies and careful engineering: overcoming polling limitations, push-based communication via Ramen, efficient spatial indexing via H3, and mobile environment optimization through edge servers and dead reckoning. Thanks to this complex and sophisticated system, we can quickly and smoothly use vehicle services through the Uber app.