Writes, 3 ways: Postgres, Apache Kafka® and Apache Iceberg™
Session Abstract
Learning new things is hard, but a useful way to think about new things is by comparing them to things you already know. In this talk, we’ll compare writes between 3 different popular data services: Postgres, Apache Kafka and Apache Iceberg. In doing so, we’ll learn a bit about the evolution of how we’ve thought of data storage as developers.
Session Description
The world of data services is evolving rapidly, with adoption of open table formats like Apache Iceberg™ picking up steam quickly. But “data services” is a pretty broad category, and none of these services is quite like the other.
In this talk we’ll take a step back to look at three data services: Postgres, Apache Kafka and Apache Iceberg, and how they each handle writes. In doing so, we’ll trace a history through how data services have evolved in the world of distributed systems and big data. We’ll understand the key differences and similarities between these services. Finally, we’ll take a look at what’s coming next in the world of open source data, from Postgres and beyond.
This session is meant as a refresher for existing data engineers as well as a primer for junior engineers: Most developers know a bit about Postgres but they might not fully understand the internals, and many engineers are getting heavily involved in Iceberg, but might not understand why it’s relevant.
