Architectural Engineer and Elixir expert Paul Schoenfelder will be speaking at The Big Elixir Conference November 8 in New Orleans. We caught up with him to get a preview of his presentation, and hear about his latest Elixir development project. Check him out on stage at the conference, and dive more into his Elixir insights on the DockYard blog.
Update: Watch Paul’s presentation below.
Can you provide a brief overview of your presentation?
My topic is, “Building reliable distributed stateful systems in Elixir.” I’ll be discussing distributed state and how it typically finds its way into systems, as well as common implementation problems. I’ll give a brief tour of consistency/consensus models, the tools typically used in Elixir, and then dive into an explanation of the new library I’ve created, which approaches some of these problems and attempts to provide a more generally useful abstraction/solution.
What made you decide to build this library?
The approach to developing the library is interesting. I started by building a simulator, which would allow me to deterministically run a simulated cluster and inject faults of different types, such as disk/power failure, network partitions, clock skew, and more. This made it possible to test whether those faults result in undesirable application behavior. Then, using the simulator, I built a framework for developing different types of key/value storage with pluggable consistency models.
The store and models can be built separately from one another, allowing an application to mix and match persistence, consistency, and stores based on the needs of that application. The simulator allows us to verify that the models are correct. This has the potential to be a huge benefit in the development of applications which require distributed state and is something that, until now, hasn’t existed in Elixir.
Can you elaborate on the future of Elixir?
The future of Elixir and how this new library ties into it is a great question. In my opinion, one of the big problems facing the development community right now is how to take an application from single-node to multi-node, and how to deal with the resulting distributed state problems. The existing tools are hard to understand, or have varied APIs, or make assumptions about your application; and they are almost universally unverified. So there is no way to know if they really work the way they are supposed to.
What I’ve built is an attempt at solving all of those things: I want to provide a unified API for the most common class of distributed state (key/value-based storage), which allows developers to focus on the application rules without having to think about distribution or consistency guarantees.
My approach simultaneously allows other developers to build consistency models for different use cases, which can be plugged into the library. Engineers can make use of the provided simulation framework to ensure that the model behaves the way they expect in different failure modes, particularly the ability to specify properties of the system that must hold under all conditions. So it becomes a soft form of verification.
What are you most looking forward to hearing about at Big Elixir?
I’m very much looking forward to hearing Josh Adams’ talk On Building Systems as well as Going Multi-Node by Eric Oestrich. And the Big Elixir Conference is such a wonderful gathering of leaders in the Elixir community, I’m sure there will be many other great talks, as well!
DockYard is a digital product agency offering exceptional user experience, design, full stack engineering, web app development, software, Ember, Elixir, and Phoenix services, consulting, and training.
