Speaker: Mahesh Balakrishnan
Location: Soda 430-438, Woz Lounge
Date: August 26, 2022
Time: 12-1pm PST
Title:
Log-Structured Distributed Systems
Abstract:
In this talk, I propose an hourglass architecture for distributed systems with the shared log abstraction as its narrow waist. The top of the hourglass consists of diverse applications layered above the shared log API: metadata services, transactional databases, key-value stores, publish-subscribe layers, and even blockchain apps. At the bottom of the hourglass, we have a similar diversity of consensus mechanisms hidden by the shared log API, including protocols such as Paxos variants, Raft, ZAB, LogDevice, etc. A log-based hourglass architecture accelerates innovation by allowing applications and consensus protocols to be developed and deployed independently: for example, we can run a ZooKeeper clone and a SQL database on a single codebase and operational platform; which in turn can dynamically switch its consensus engine between Paxos and Raft. I’ll describe the origins of this paradigm at Microsoft Research (Corfu/Tango in NSDI 2012 and SOSP 2013); its evolution at VMware Research and Yale; its practical validation as Meta’s control plane storage (Delos in OSDI 2020 and SOSP 2021); and future directions.
Bio: Mahesh Balakrishnan is a distributed systems researcher at Confluent. Prior to this, he led a team at Meta until 2021 that built and deployed Delos, a storage system which currently supports Meta’s internal infrastructure. Mahesh was an Associate Professor (on term) at Yale University from 2015 to 2019; and a researcher at VMware Research and Microsoft Research Silicon Valley before that, where he co-led the Corfu project (later commercialized by VMware as CorfuDB). He obtained his PhD from Cornell University in 2008. His work has received best paper awards at OSDI and ASPLOS. He is currently the program co-chair for NSDI 2023, and has co-chaired SoCC and LADIS in the past.
https://docs.google.com/spreadsheets/d/17wNqXemFl-EeQHMyZ_0e1LTN1VRJdujMN-Y_CuB9Llk/edit#gid=0