Changing Data Capture: The Critical Link for Airbnb, Netflix, and Uber

Couldn't attend Transform 2022? Check out all the summit sessions in our on-demand library now! Look here.

The modern data stack (MDS) is fundamental to digital disruptors. Think Netflix. The company pioneered a new business model around video as a service, but much of its success is based on real-time streaming data.

They use analytics to provide highly relevant recommendations to viewers. They monitor data in real time to maintain constant visibility into network performance. They sync their database of movies and shows with Elasticsearch to make it quick and easy for users to find what they're looking for.

It needs to be real-time and 100% accurate. The old fashioned Extract, Transform, Load (ETL) is just too slow. To address this need, Netflix has created a change data capture (CDC) tool called DBLog which captures changes to MySQL, PostgreSQL and other data sources and then streams those changes to target data stores at research and analysis purposes.

Netflix required high availability and real-time synchronization. They also needed to minimize the impact on operational databases. CDC deletes database logs, replicating changes to target databases in the order they occur, so it captures changes as they happen, without locking records or bog down the source database.

Event

MetaBeat 2022

MetaBeat will bring together thought leaders to advise on how metaverse technology will transform the way all industries communicate and do business on October 4 in San Francisco, CA.

register here

Data is at the heart of what Netflix does, but they are not alone in this regard. Companies like Uber, Amazon, Airbnb and Meta thrive because they really know how to leverage data. Data management and analytics are strategic pillars for these organizations, and CDC technology plays a central role in their ability to carry out their core missions.

The same can be said of almost any business operating at the top of its game in today's business environment. If you want your business to perform like a top performer, you need to modernize and master your data. Your competitors are probably already doing this.

Onboarding in less than a second is the new normal at Airbnb and Uber

In today's world, a strong customer experience requires real-time data feeds. Airbnb has recognized the value of CDC technology in creating excellent CX for its customers and hosts. They too have built their own CDC platform, which they call SpinalTap. Airbnb's dynamic pricing, listing availability, and reservation status require uncompromising accuracy and consistency across all systems. When an Airbnb guest books a visit, they expect workflows to be very fast and 100% accurate.

For Uber, immediacy is arguably even more important. Whether a customer is waiting to be driven to the airport or ordering food delivery, timing is everything. Just like Netflix and Airbnb, they have developed their own CDC platform to sync data across multiple data stores in real time. Again, a common set of requirements emerged. Uber needed their solution to be extremely fast and fault-tolerant, with no data loss. They also needed a solution that wouldn't reduce the performance of their source databases.

Change data capture for the rest of us

Once again, CDC fits the bill. Previously, overnight batch mode ETL might have been sufficient to provide daily management update or operational reports. Today, real time is increasingly the norm. If information is power, then immediate access to information is turbo power...