Book Review: Designing Data-Intensive Applications

After seeing major recommendations from a lot of respected folks¹, I finally decided to read DDIA this year. And right off the bat, I’ll just reaffirm many of the things I heard/read — I agree that it should be mandatory reading for engineers. I also agree that it should be read multiple times, once every few years as you progress through your career. I for one feel quite unlucky not to have come across this gem of a book sooner, but at the same time really glad that I did now!

For me the greatest value the book provided was in the form of formalization & reaffirmation — as a Tech Lead with almost 6 years of professional experience working with distributed systems, albeit more on the user end of the spectrum, I’ve collected a lot of incidental knowledge that typically comes with such experience. And reading through DDIA, one of my main observations was that Kleppmann does a really great job at formalizing this knowledge. Working at the same organization for the past few years, I have left a long trail of decisions in my wake. For me DDIA provided great validation for some of those decisions — giving me more confidence in my ability in making them, often with a healthy dose of “Whew! I sure managed to dodge that invisible bullet!” In other words, DDIA helps you reflect on your knowledge and decisions through a very clear lens of formalized knowledge.

DDIA helps you reflect on your knowledge and decisions through a very clear lens of formalized knowledge

If you’re someone with relatively less experience, a lot of things might not click the way they would if you had more experience, but just trust me when I say that you’ll be much better off knowing them now than later. The fact that the concepts & knowledge I picked up from the book have already helped me multiple times in the past few months, a couple times in very critical situations, should be evident enough. And I’m fairly positive that they’ll continue to do so for years to come.

DDIA also teaches you what you should beware of — acting as a sort of a reference of things that can and will go wrong when working with modern systems. This aspect of it is invaluable in my opinion. Imagine being able to spend just a few hours of your life to gain knowledge of things that folks only get to learn over years, often through ending up in very tricky situations. This knowledge allows you to create and improve a personal framework for architectural decision making.

Imagine being able to spend just a few hours of your life to gain knowledge of things that folks only get to learn over years

Another impact it had on me was igniting an interest in the construction and internals of distributed systems, such as databases, brokers etc. I have a feeling that this is likely to have a major impact on my career trajectory in the near future. It also widened the sphere of my understanding — I recently attended P99 Conf, and was pleasantly surprised by how much of it I was able to absorb; the same goes for Dist Sys blogs I’ve been reading recently.

All this said, the book isn’t perfect in every way — at times the focus seems to deviate from things that most require it — but that might just be an artefact of my personal experiences and preconceptions. I also found it to be a demanding read in some sections, either by virtue of trying to cram too much into too little, or at times the structuring of contents requiring tracing too many references in order to grasp some concepts. Chapter 9 comes to mind as the main offender in this regard². But note that this is only relative to the rest of the book — DDIA is by far the easiest and smoothest to read technical book I’ve read. Kleppmann (along with Chris Riccomini) is working on the second edition³ currently, and in addition to being excited by all the new concepts that’ll be covered, I’ll also look forward to some of these being addressed to make for an even better book.

… inviting you to break through the traditional approaches of designing distributed systems and give a fair consideration to the alternatives, lest they be missed in the noise

Another major goal that Kleppmann seems to accomplish is expanding your worldview on how to view and design distributed systems, inviting you to break through the traditional approaches and give a fair consideration to the alternatives, lest they be missed in the noise. I suspect his ideas played an important role in the development of things such as AtProto⁴, indicating the continuing achievement of this goal. The discussion about ethical considerations in the end was also a welcome surprise: Kleppmann didn’t have to, but he still did, which I respect and admire. It will push me to consider the consequences of the things I build or help build in a more holistic manner, and also give some bandwidth to reflecting, both internally and externally, on such consequences in my body of (future) work.

As I hinted at the start — DDIA is definitely the kind of book that gives you more every time you read it, the most deserving of this claim, at least of all the books I’ve read. I’m certainly planning to revisit it in a couple years or so. And I think you should too.

Thanks for reading! Feel free to share this with others on the butterfly site, the elephant site, the bird unicode character site, or maybe even the orange site?