Elixir and the BEAM: the stack that was ready for AI

Elixir and the Beam runtime upon which it relies are truly special. On this channel, I've shown how to build projects with certain aspects of the language, but never stepped back to show the full picture. Recently, folks have been gluing together tons of technology to solve multi-agent orchestration, while the Beam has had most of what's been needed for decades. Frankly, it's sad to see that more folks don't know what's available within this runtime. So, let me take you on a trip through Elixir, what it's made of, and why it's a perfect fit for the past, present, and future of software. By the end of this video, most of these concepts at the top should be familiar to you, and hopefully it'll change the way that you think about developing these systems. Rather than running on a single threaded event loop or manually forking to set up and coordinate OS threads, applications running on the Beam are composed of many lightweight processes. Processes have a small memory footprint, so you can fit millions of them on a single box. They have their own heap, so there's no global garbage collector pause, something you might be familiar with in Java or Go. And they're preemptively scheduled across every CPU core you give them. For this reason, it's not uncommon for companies who switch to Elixir to reduce the number of servers they run because they get greater utilization out of their hardware. Most importantly though, processes have an ID and a mailbox. For the most part, Elixir applications operate on the actor model, many independent processes that pass messages to each other rather than sharing memory. If a process wants to send a message to another one, it just calls send with the destination processes ID. To receive, we use a built-in receive keyword, which will pause execution until a message comes in or a timeout occurs. If you're a gopher, this isn't dissimilar to a go routine with a globally registered inbox channel, but as you'll see, there's more to it than that. The process is one of the core primitives, but most of the time we work at higher levels of abstraction. A common one is the generic server or gen server for short. These are still processes, but they're programmed with a few callbacks that might manipulate a process local state. Gen servers can be called, where a process sends a message and asks for a response to be sent back. They can be cast, which is just a direct send of a message, or they can handle info, which is a catch-all for other messages that come into a process mailbox. We'll talk about some of those other message types in a bit. I added a very simple example here for a get an increment call which might take in an amount and the current state and then reply to the caller with the existing state and save a new one in that final tupil. In elixir the last expression in a function is its return value. Now gen servers have a specific life cycle. They initialize, they run handlers in a loop as messages come in and finally they terminate. All processes can terminate normally as part of a shutdown or with an abnormal value that might contain an error message. While it's common to catch and recover from common error scenarios like calling an API that has rate limiting where maybe you'll do a retry with back off, you may have also heard the expression let it crash. In the face of unexpected errors, it's normal for a beam process to exit abnormally. Unlike a node application, this doesn't take down the whole machine, just that one process. This is typically acceptable because of another pattern called supervision. Every Elixir application has a tree of processes that take care of each other. In addition to having an ID, a mailbox, and local state, processes can be monitored and link to each other. If a parent process calls process.mmonitor with the P ID of a child, the runtime will send a down message if the child terminates. This ends up in the parents mailbox and can be handled just like any other message. If it's a gen server, this will come in through handle info. And if it's just a regular process, this will come in through a receive. Now, maybe the parent is going to restart the child or trigger an alert. It can do anything because this is a part of your application code and it's all managed by the runtime. The alternative to monitoring is linking where two processes are tied together such that if one dies, the other probably should too. There's a distinction here, though. A normal exit won't cause a linked process to terminate. So, in practice, links are temporary while a process is doing work for another one. Process supervision is so common that there's another special type of process abstraction that comes in two flavors. supervisor and dynamic supervisor. A supervisor is a process whose only job is to start up and look after a static ordered list of predefined children. If one of the children fails, it'll follow one of three strategies to recover configured when the supervisor was started. One for one will only restart the crashed child, while one for all will restart all children if one of them dies. And the reason that a supervisor is defined with an order of children is for the rest for one strategy where if one process fails, all of the ones that come after it in the list will be restarted one by one in the face of an error. On the other hand, a dynamic supervisor is for processes that get created and stopped at runtime. These might represent an agent, an HTTP handler, a game server, anything really that you can't predict at the start of a server. And no matter which supervisor you're using in a branch of the tree, child processes are defined through child specifications. These contain an identifier, a start definition, which is what are we running, a restart declaration, and a shutdown behavior. Permanent processes are always restarted unless the entire server is shutting down. On the other hand, temporary processes are never restarted. Their supervision is mainly present for shutdown propagation and unique registration. And in between these concepts is a transient process which is only restarted if it exits abnormally. All of this happens automatically, allowing an Elixir application to gracefully recover from unexpected errors and have predictable shutdown behavior. It's tricky to get similar semantics out of other systems requiring a ton of hard coding or large libraries. The beam has all of this stuff built in. But you might be wondering, how do processes discover each other's IDs? How can a system like this stay organized? For this, we use registries. A registry, unsurprisingly, is a process. Specifically, it's a process that accepts lookup, register, dispatch, and count requests. When a process is registered with a registry, it'll be stored in a table with a name, an ID reference, and optional metadata. The registry process will also monitor those processes and automatically remove it if that process terminates. Registries can either be unique or duplicate. A unique registry will reject the registration of a process with the same name, while duplicate allows it. And since processes work through their mailbox messages one at a time, there's no race conditions. Lookup and count are pretty obvious operations, but dispatch is particularly interesting. It's like a for loop that walks through each entry in the registry with a specific name, provides the PD, and can be used to do things like a broadcast. Perfect for building publish subscribe messaging systems without much fuss. You might be looking at this table and wondering where it's stored though. In addition to process local heap storage, the beam provides something called Erling term storage or ETS for short. An ETS table is not a process but it is owned by a process. It offers key value storage similar to Reddus but it's provided natively within the runtime. XT tables can enforce uniqueness. That's how registries pull it off behind the scenes. They can also be accessed directly in parallel without messaging overhead with three visibility levels restricting read and write to the owner process or being open to all processes. A write optimized public ETS table would be great for collecting tracing data from many parallel processes while a cache might be best modeled as a protected read optimized one. Now, like I mentioned earlier, a single server can run millions of processes without an issue, but at the end of the day, it's important to distribute workloads across multiple servers. Typically, this kind of thing would require an external job queue, maybe Reddis, PubSub, maybe a database with polling, Rabbit MQ, or SQS. But I was hiding something when I first introduced process messaging. The P2.456 456.0 is special. Well, this P isn't, but that two at the front is. See, beam runtimes do great alone, but they do even better in clusters, which can be assembled through a service discovery system or a static list of servers, something that works in all container orchestration systems if you're Docker inclined. So, the trick is that processes can freely pass messages between node boundaries. A P ID contains three segments. The node identifier, a serial number, and a creation number that's almost always zero. But if a PD starts with zero, it's for the local node. All other identifiers get natively mapped to the destination server. Messages aren't just simple notifications. Sure, they can include numbers. They can include strings, but they could also contain code for RPCs, or they could contain references to a network port or a file system descriptor to directly stream data across a network. This is one of the most impressive parts of the beam. And honestly, I take it for granted sometimes. That is until I start working in another language and have to build things the hard way. Now over the years solutions have been developed to bring single node concepts to distributed environments including distributed registries and distributed tables and combining all of these native capabilities. The ecosystem has been given some incredible libraries. Fenix pubsub and channels provide websocket and long pulling messaging to clients. Massive applications like whatnot rely on this every single day. Phoenix Live View brings server-driven dynamic web applications that operate by sending tiny HTML diffs over the wire with state and messaging managed in a process on the server. This inspired things like Laravel Livewire and Rails Hotwire. Broadway is a scalable concurrent data ingestion and processing system that efficiently works through huge messaging workloads. Obon is a job Q system with retries and scheduling across nodes and GTO is a new scalable fault tolerant agent framework that's really starting to gain momentum. After learning about what the beam provides, the concepts directly can be applied to AI orchestration problems. Web soockets could easily be backed by processes. Tokens can be streamed and agents can coordinate via native messaging. Vault tolerance is trivial with supervision trees and monitoring. Parallelization can be achieved through the actor model and at scale a distributed cluster can efficiently serve many users. And if you need caching anywhere, Ets is nearby. I didn't even mention zero downtime hot code reloading. Anyway, contrast that with the multiple services and glue code required to do something similar with other stacks, and I hope you can see why folks who have used Elixir are always so excited to tell you about it. Vendors are starting to catch on to the value of these primitives and have been partially implementing them on their own platforms. They're going to overstate their capabilities and try to lock you in. If you want a battle tested runtime that delivers on its promises all with a simple local development process that can be replicated in production without much fuss, there's truly no place like the beam. This has been code and stuff. Thanks for watching.

Elixir, and the BEAM runtime upon which it relies, have built-in primitives that other ecosystems are stitching together. There's no runtime like the BEAM for workloads past, present, and future. Links Screen recording software I use (affiliate): https://screen.studio/@Yy75o Elixir: https://elixir-lang.org/ Language Tour: https://elixir-language-tour.swmansion.com/ Phoenix Channels: https://hexdocs.pm/phoenix/channels.html Broadway: https://elixir-broadway.org/ Oban: https://oban.pro/ Jido: https://jido.run/ Timestamps What? - 00:00 GenServer - 01:53 Supervision - 03:33 Registries - 06:37 ETS - 07:52 Distribution - 08:39 Ecosystem - 10:33 Why this matters now - 11:21