Building Beacon #3 - Loader

Not ready to devote the time or resources to onboarding a new Elixir engineer? Book a free consult today to learn how we can help.

Continuing our Building Beacon series, let’s take a look at Beacon’s Loader system. The primary goal of this system is to ensure Beacon content is available to clients with minimal wait time. To show what makes Beacon unique, we’ll first consider the most naive approach.

The Naive Approach

Let’s imagine a hypothetical framework “NaiveCMS” (for Elixir + Phoenix LiveView):

• An admin creates a new page via some form in a dashboard
• This form data is saved to the database
• When a user attempts to access that page, NaiveCMS queries the database for the data
• This data needs to be converted to a %Phoenix.LiveView.Rendered{} struct in order to meet the expectations for serving a page via LiveView

For this final step of returning Rendered, the logic looks like:

assigns = %{greeting: "hello"}

template = ~S"""
<h1><%= @greeting %></h1>
"""

opts = [
  engine: Phoenix.LiveView.TagEngine,
  line: 1,
  indentation: 0,
  file: "nofile",
  caller: __MODULE__,
  source: template,
  trim: true,
  tag_handler: Phoenix.LiveView.HTMLEngine
]

quoted = EEx.compile_string(template, opts)
{rendered, _} = Code.eval_quoted(quoted, [assigns: assigns], __ENV__)

Now, this implementation will work as expected. However, sites built with NaiveCMS will score poorly on SEO (search engine optimization) because this process is so slow, and has to run for every page request on the site. Why so slow? It isn’t the database call, or compiling the string to AST, but rather the final line: Code.eval_quoted/3 - evaluating arbitrary AST at runtime is slow, because the AST doesn’t exist in the BEAM VM yet. Therefore this function simply does not meet the performance needs for serving web requests.

Possible Improvements?

If the above approach is too slow, is there something we can do to speed it up?

One thought might be to replace the database call with some in-memory storage solution such as ETS. It’s true, loading the page data from ETS will reduce the time needed to serve a page. But as mentioned earlier, this is usually not the bottleneck. Even if we load the data entirely from memory, Code.eval_quoted/3 will still take so long to complete, that the response time is still unacceptable.

What if we take it to the next level and store the result of Code.eval_quoted/3 (a Rendered struct) in-memory? That seems like it would bypass the bottleneck, and in fact it does. However, that result would have the assigns we originally provided, and any updates to those assigns would not have any effect. We would lose all “live” functionality on our pages!

So, if we need Rendered, but it’s too slow to create on-demand, and can’t be cached, how is it possible to serve these page requests? For that, let’s see how LiveView itself accomplishes this.

The LiveView Way

In a regular LiveView module, you have callbacks which build/update state (e.g. mount/3, handle_params/3, handle_event/3), and then a render/1 callback which contains a HEEx template:

defmodule MyAppWeb.PageLive do
  use MyAppWeb, :live_view

  def mount(_params, _session, socket) do
    {:ok, assign(socket, greeting: "hello")}
  end

  def render(assigns) do
    ~H"""
    <h1><%= @greeting %></h1>
    """
  end
end

We can compare this approach directly with the one shown previously in NaiveCMS:

# This is set in `mount/3` and stored in the LiveView process' state.
assigns = %{greeting: "hello"}

# This template is wrapped in an ~H sigil inside `render/1`
template = ~S"""
<h1><%= @greeting %></h1>
"""

opts = [
  ...
]

# The ~H sigil automatically sets the opts and compiles the template
quoted = EEx.compile_string(template, opts)

Notice the biggest piece of the puzzle - Code.eval_quoted/3 - is missing. That’s because the quoted content has already been loaded into the BEAM when you booted your app. By defining the template in an Elixir module, it is available at compile-time, when the Elixir compiler does this “heavy lifting” in advance. So by the time a user requests the page, all that needs to be done is a simple function call MyAppWeb.PageLive.render(assigns).

To recap what we’ve covered so far:

• Code.eval_quoted/3 is used to call a function which does not yet exist in the BEAM
• This takes quite a bit of time, resulting in poor performance serving web requests
• Phoenix LiveView avoids this issue by expecting developers to define templates in Elixir Modules in the code repository
• This makes those templates available at compile-time, when they are automatically loaded into the BEAM for quick access
• A CMS cannot use this approach because new templates are defined at runtime (after the app has already compiled/booted)

With these points in mind, we’re finally ready to see how Beacon solves this problem.

The Beacon Way

We’ve seen that we can avoid calling Code.eval_quoted/3 if the quoted content has already been loaded into the BEAM in an Elixir Module. But how can we do this if the app has already started? The Elixir compiler has a public function to do just that: :elixir_compiler.quoted/3. We can call this function at any time to load a new Module into the BEAM:

module_ast = Beacon.Loader.Page.build_ast(site, page)
[{module, _}] = :elixir_compiler.quoted(module_ast, "nofile", fn _, _ -> :ok end)

Once this completes, we can call any of the functions in that module without any performance trouble, similarly to how LiveView calls the render/1 function each time the assigns change.

Building Module ASTs

In the previous section we saw that Module ASTs can be compiled into a BEAM instance at runtime. But how do we build these ASTs? Let’s look at a simplified example of what a CMS page Module might look like:

module_name = :"MyApp.DynamicPage#{page_id}"

template = ~S"""
<h1><%= @greeting %></h1>
"""

opts = [
  ...
]

quoted = EEx.compile_string(template, opts)

quote do
  defmodule unquote(module_name) do
    import Phoenix.LiveView
    ...

    def render(var!(assigns)) do
      unquote(quoted)
    end
  end
end

This code will generate the complete AST for a module, which can then be loaded with :elixir_compiler.quoted/3, and page rendering will be as simple as calling module_name.render(assigns).

Lazy Loading

Now that we’ve seen how Beacon’s Loader builds and loads dynamic Modules, let’s discuss the “when” and “why.” If we attempt this on boot for every published page, large sites will take a long time (and possibly a lot of memory) to complete the boot stage. With that approach - which we call “eager loading” - a site could theoretically grow so large that it can no longer boot at all, due to hardware limitations. Therefore, we instead use an approach called “lazy loading”, where most¹ pages are left unloaded until the first request for that page, when the page will be loaded on-demand. Fortunately, Erlang/OTP provides a built-in mechanism for lazy loading Modules: :error_handler.

Every process in the BEAM has a flag which sets an ErrorHandler module. If the process attempts to call a Module/function that doesn’t exist, the ErrorHandler is automatically invoked. The default ErrorHandler will simply raise a RuntimeError, but a custom ErrorHandler can be set to run whatever code we want. For Beacon, we want to call the Loader. Assuming the Loader is successful, no error will be raised and the page will continue to be served as expected.

Avoiding Loader Conflicts

One additional concern with dynamically loading Modules is when multiple processes request the same missing module and they all try to compile the module at the same time. The Elixir compiler was not meant to allow this, so we must prevent it in order to maintain stability and reliability for all requests. In Beacon we accomplish this by using a Registry with unique keys. Any calls to load a module must first register the caller with the Registry, and only one process can be registered at a time. Concurrent requests from other processes will simply wait for the registered process to finish.

case Registry.register(Beacon.Registry, module, module) do
  {:ok, _pid} ->
    # this is the first request, so it's OK to compile + load
    load_module(module)
    Registry.unregister(Beacon.Registry, module)
    module

  {:error, {:already_registered, pid}} ->
    # another process is already loading the module, let's wait for it
    Process.monitor(pid)

    receive do
      {:DOWN, _ref, :process, _pid, {:shutdown, :loaded}} -> module
    after
      timeout -> :error
    end
  end
end

Regardless of whether we loaded or waited, the above function returns the module and we can assume it’s now available.

Conclusion

Now you’ve seen how Beacon’s Loader works with the strengths and limitations of the BEAM to make a CMS a reality for Elixir. Stay tuned for the next installment of the Building Beacon series!

Footnotes