Conference experience reports used to be a thing, back in the days of people doing lots of blogging. Why not be a bit retro, and put one together?

This year’s Elixir Conf in Lisbon was the first conference I’ve attended in-person since Prague in 2019. For various reasons, including the negotiations for the Deliveroo acquisition of Cultivate1, I spent a lot of the 2019 conference preparing the slides for my talk. This time I was an attendee so could relax and take in the other talks.

This is a brief summary of what I saw and from what I can decipher of my notes. It was a 2.5 track conference; I did not attend every talk.

José Valim keynote

Elixir 1.15 is to be the most boring Elixir version release so far, but in a good way. There are to be developer-experience improvements for large codebases, consisting of up to 40% faster compilations when used with (the upcoming) OTP 26. There’ll also be improvements in OTP 26 for the experience of developing on Windows. One of the speed improvements for maps in OTP 26 could break anyone relying on key order when iterating maps (which no-one should be doing as it has never been formally guaranteed).

He also mentioned the work on bringing “Set Theoretic Types” to Elixir, which is progressing very well and a the documentation / feedback stage but mostly pointing to the later talk by Guillaume Duboc. Development will start later this year.

Improvements to the developer experience include Mix.install, dbg2, plugins for mix format. A promising area is the introduction of code fragments which enable tools (eg Elixir LS) can work better with code that can not currently compile.

The rest of the keynote was some demonstrations of cool LiveBook features, such as Kino.Process.render_seq_trace/2 for visualising function calls with a sequence diagram, or Kino.Process.render_sup_tree/2 to show a supervision tree.

José also demonstrated some of the new LiveBook machine learning capabilities in the Smart Cells and also how Smart Cells can be used for extension by converting to code then changing.

There was a great answer to an audience question: why make LiveBook when Jupyter notebooks already exists? Partly it was a proof of the the capabilities of LiveView. It was also that Jupyter uses global state across all users, when LiveBooks are designed to be shared but have individual state for each user/session. There are other advantages of being focussed on a single functional language, such as Smart Cells and the ability to detect whether an update makes another cell stale.

A Domain Specific Language for Impact — Simon de Haan and Federico Meini

Turn.io is a chat product aimed to social impact. The principal is that WhatsApp is available and used widely especially in developing countries. The automatic chat is configured with a powerful External (I think) Domain Specific Language (DSL).

The cool part is that the DSL is similar enough to Elixir that they do not need a parser. Code.string_to_quoted/2 will turn the user’s DSL to AST which can then be transformed by a bit of code into Structs that can be saved to the database. Even cooler they can go the other way by turning their Structs to AST then running Macro.to_string/1.

As well as the DSL they have a visual way of building the automation flow. For that they convert the visual representation to JSON that conforms to the Flow Interoperability Standard which is a JSON representation of a state machine.

They do need to do their own parsing for part of the flow. That is for guard expressions such as making a decision base on the age of the contact eg “contact.patient_age > 18”. For this they use Nimble Parsec. There was a comment disguised as a comment, suggesting that the Abacus maths parsing hexicle should be used rather doing their own parsing. The Abacus Github README overview is not included in its Hex documentation.

Remote Debugging with Livebook — Luca Dei Zotti

This was a nice little demo of debugging an issue on a “production” instance with tools like dbg with trace port remotely from a LiveBook rather than a remote console. Also included some scary code replacement by compiling on the server, without deploying; I am unsure about this part but “needs must as the devil drives” I guess.

I can see how LiveBook might beat a remote console for this, in that there is a clear path of what has already been investigated.

Optimising LiveView for Realtime Applications — Marius Saraiva

Marius is the person behind Surface, which I haven’t used, and his examples used Surface and the ~F sigil which confused and distracted me a little until I realised. I am dumb.

I loved all the tips in this talk. While Marius started by saying that not all Live View needs to be optimised, most of the techniques he demonstrated now seem more like good practice rather than premature optimisation.

Marius started by reminding us of how LiveView, in particular HEEx works: splitting the dynamic and static parts of a template and after the initial rendering only sending the dynamic parts that have changed to the client. The tips were all about minimising those changes.

Replace function calls with components

In the olden times of a few years ago, it was common to call functions from within templates to return strings to display.

eg something like

<div>
    <%= score(@score) %>
</div>

defp score(score) do
   "You have #{@score} points"
end

Then all the text will be sent every time the score changes. If this is replaced with a component function then only the score will be sent on a change.

<div>
    <.score score={@score} />
</div>

defp score(assigns) do
   ~H"""
    You have #{@score} points
    """
end

Generally calling out to functions from a template is now a red flag.

Use CSS selectors instead of dynamically creating classes

Particularly now that Tailwind has become the default CSS framework for Phoenix and Live View, it is common to dynamically assign classes to reflect some changing state such as a button being disabled. Rather then send the large list of classes down the wire, this can be done purely in CSS by selecting on the disabled state….

eg

<button class="mybtn" disabled={@disabled}>Do it!</button>

in CSS

.mybtn {
    @apply: bg-green-500 text-white font-bold py-3 px-4 rounded-lg text-center

.mybtn[disabled] {
    @apply: bg-gray-500 text-white opacity-50 font-bold py-3 px-4 rounded-lg text-center
}

Data and aria attributes can be used for more general selecting. I love this tip which is as cunning as a fox who’s just been appointed Professor of Cunning at Oxford University3.

Avoid accessing whole objects when parts of the object are frequently updated

Marius demonstrated that when passing an object (ie map / struct) to a function component then the whole component is re-rendered and with all the data being passed to client when any part changes. The example was a full name component which was re-rendered whenever the same map’s updated_at value (displayed elsewhere) changed. Something like

<.full_name user={@user}/>

I am unable to reproduce this with vanilla Phoenix (1.7.1) LiveView (0.18.6). I could be getting the issue wrong but I do wonder if this is something to do with Surface.

Debounce / Batch very frequent updates

Last Marius demonstrated displaying simulated messages to a user that come at a ridiculous rate. Over a throttled connection downstream messages to the user become queued up, probably leading to eventual resource issues on the server. Batching the messages prevents this backlog and looks at least as well as sending individual messages.

I have probably spent too long on this talk. It was clear, actionable, with great examples.

Quantum Doodle: Digital Twins for Everyday Activities — Paul Valckenaers

I was intrigued by this talk and I am still intrigued. My understanding is that a Digital Twin is highly granular representation of a real world object, such as an aeroplane engine, that is continuously updated with information from sensors etc… It is used for things like monitoring and diagnosis of issues.

Paul was suggesting introducing digital twins for activities, such as student examinations. Honestly he lost me a lot. I left confused about what the difference would be between a Digital Twin for (say) items being shipped around the world and how that would be currently modelled in existing software.

Powerful Machine Learning at Your Fingertips — Jonatan Kłosko

This was a great introduction of machine learning capabilities in Elixir. Jonatan gave an introduction to pre-trained models from Hugging Face and Bumblebee.

Livebook featured heavily in this demonstration including using Smart Cells to demonstrate and generate code to explore.

Also featured was running an AI chat in a distributed cluster with LiveView on Hugging Face Spaces.

Lively LiveView with Membrane — Lars Wikman

While watching Lars’ presentation I first became aware that it was automatically transcribing his words, then that when he said certain words it would load the next slide. He later showed that it was coded in LiveView, and using Whisper with Bumblebee. It also used Membrane for (I think) creating a waveform in SVG from the audio and Evision for facial recognition.

The talk itself was largely philosophical and posed a question:

Are things hard because cool things are hard, or are hard things cool?

The answer is probably yes.

Lars wrote his conference experience up here.

Bringing Types to Elixir — Guillaume Duboc

This was on at the same time as the Lively LiveViw with Membrane talk, but I cheated and caught up on a rewound live-stream.

Guillame walked through the Set Theoretic Types coming to Elixir. The concept of set types fits with how functions can work with pattern matching: if a function can take, say, User.t() or an integer() (denoting an id) then that is the set of acceptable input types.

The Set Theoretic Types (I’ll use STT from now on) seem to build on what is available already with Typespecs but are more capable and are planned to be more integrated with the compiler: an end to digging through lots of code to figure out why dialyzer is complaining that function “has no local return”.

One interesting enhancement is interesection types. An example was a function negate/1 that returns the negative of an integer and the opposite of a boolean. Rather than specifying this as a union type

@spec negate(integer() | boolean()) :: integer() | boolean()

with STT, we could be more specific

$(integer() -> integer()) and (boolean() -> boolean())

Whether it’s good practice to vary the return type depending on the input type is another matter.

If I understand things, other features will include

  • guard clauses being integrated with the type system, showing more warnings (and errors) at compile time4
  • Composing protocols, eg $ type traversable(a) = Enumerable.t(a) and Collectable.t()
  • Parametric polymorphism — ie being able to define a parameter as an Enumerable of integers Enumerable.t(integer())
  • Support for dynamic types, so the codebases will not have to go strongly typed all at once.

There was no mention of how types will work with message passing between processes. I guess that is another reason for dynamic types.

Lightning talks

I gave one of the 5 minute talks at the end of the day. I am glad I submitted slides rather than tried to do something with LiveBook, as I was considering — swapping laptops for those that needed was (as is usual) problematic.

Erlang Ecosystem Foundation - community engagement — Alistair Woodman and Francesco Cesarini

This was a compelling pitch for joining the EEF. I’ve just signed up.

Optimizing Software Delivery on-the-fly — Tom Calloway

A super pitch for Tom’s product, Kanbran which is “like Kanban but with more fibre”.

Hiring Elixir Devs in 2023 — Arjun Gillard

A talk by Arjun Gillard, a recruiter, on how to hire and retain Elixir Developers. Worth watching when the videos come out, if that’s what you need to do.

PS - you could maybe also hire me.

Introducing Fedecks, for easy communication between Nerves and your Phoenix Server — ME

I talked about a set of libraries5 I have just released to make it very easy to set up a durable Websocket connection between a Nerves device and a Phoenix Server living in the cloud.

They are Fedecks Server and Fedecks Client. I’ll make a post launching this a few days from posting this.

LiveView goes k6 — Sebastian Göttschkes

About the pK6 framework for load testing from Grafana and how it integrates with Phoenix and LiveView.

Quick Tricks with the .iex.exs file — Daniils Petrovs

Enhancing your iex experience by loading lots of things from your .iex.exs file.

https://gist.github.com/DaniruKun/ccbaad8720c203fd6d86a39722c63c51

Doomguy visits the BEAM — André Albuquerque

This was an amazing demonstration of supervision strategies with the Doom video game. Using what was clearly magic, loads an instance of Doom with monsters spawned by supervisors. :one_for_one, :rest_for_one, and :one_for_all was clearly demonstrated by killing the monsters with a gun and observing the respawning. Dynamic supervisors involved audience members joining via LiveView but didn’t work quite as expected as ngrok failed.

The code is here.

An Introduction to Property Based Testing — Roland Tritsch

Digital nomad, Roland, gie a quick overview of property testing.

Improving democracy with a petitions platform with delegation — Hector Perez Arenas

Hector showed us his You Congress, a better petitions site with the ability to downvote as well as upvote, and the ability to delegate your votes to someone trusted.

Lessons from using Elixir and Phoenix to build a city software infrastructure — Shankar Dhanasekaren

This was an unusual, but fascinating, keynote to kick off day 2. Elixir Conf keynotes are, as far as I remember, heavily technology focussed. Shankar is CTO of Auroville. I had not heard of the city before, but Wikipedia tells us that it is an experimental township in India, founded by Mirra Alfassa, a spiritual guru.

Previously much of the software was developed with Drupal but that has (is being?) replaced with Elixir and Phoenix which fit the city’s constraints:

  • They only have a small team of developers
  • They need the flexibility to release, and then add more features later6.
  • Overheads have to be low
  • The user interface needs to be rich
  • The budget is limited

Scale is not an issue but Elixir + Phoenix (with LiveView) is a great fit as it is so productive.

Something notable, but not mentioned, was that they write all their own software rather than use services. For instance one of the many products is a booking system for their guest house. I would have liked to have known whether buying or renting in a commercial system was considered and what was the decision making process. I did not raise my hand to ask as I could not work out how to phrase the question without it coming across as a criticism.

It was also interesting to see that they are experimenting with growing their own food with Farm Bots, which I guess are a commercial product.

Safer DB Migrations with excellent_migrations — Artur Sulej

Excellent Migrations is a library to detect potentially problematic issues with a migration, such as setting an existing column to be “not null”. These are the issues that may not appear when testing the migration but blow up in production with heavily populated tables. Nice points are that:

  • It is implemented as a Credo check making it easy to add to what is likely to be an existing workflow
  • It is easy to configure, such as ignoring all migrations before a certain date as they’ve already gone to production or removing particular checks which may no longer be an issue with the database / version being used.
  • Checks can be ignored by adding a structured comment, if for example we know that the table in question is small.

Artur gave a nice run through of some of how it is implemented by pattern matching on the AST.

Phoenix beyond Cowboy — Mat Trudel

Bandit is an alternative to using Cowboy with Phoenix. As I’ve felt a bit iffy about Cowboy since I found out the origin of its name, I will definitely be trying it out. Other good reasons are that it is written in Elixir using OTP, and Mat showed Bandit coming out ahead in benchmarks that he’d run. I have found Cowboy’s code hard to follow, partly because it’s Erlang but also as it does its own process lifecycle management rather than using GenServers7.

Prior to 1.7, Phoenix was tied to Cowboy because of the way websockets were implemented. While standard requests were built atop Plug making it possible to swap out the underlying server, websocket support was built directly using Cowboy’s Websocket handler. Now, thanks to Mat, Phoenix uses Websock and Websock Adapter which is kind-of like Plug for websockets.

Not only did Mat build Bandit and Thousand Island, he built the Websock and Websock Adapter, then worked on integrating it with Phoenix. What a hero!

Websock makes custom websocket integration way more flexible in Phoenix. Previously the only sane8 way of doing so was to implement a Phoenix.Socket.Transport and add it to the endpoint, as I described here. Now we have the option to call WebSockAdapter.upgrade/4 within a Plug or even a Phoenix Controller. This gives full access to the connection before upgrade; only limited information is passed to the Phoenix.Transport.Connect.connect/1 implementation.

I wish I had known about Websock with Phoenix 1.7 earlier. Fedecks Server currently uses the Phoenix.Socket.Transport approach and a customer header for authentication on connection. This opens the opportunity for a cleaner approach.

Don’t Fight the Monolith — Peter Ullrich

This was ostensibly about techniques for keeping monolithic architectures healthy but was more about defining your Bounded Contexts using DDD’s Event Storming and their relationships with Context Mapping. This was fine but a bit of a bait and switch.

Peter did touch on two experiences: one with a company that found extracting their monolith into microservices9 and his current company that does not. He thought that a monolith is better for a company that is in “hypergrowth” and services for those that are more mature. I’m a little unconvinced of the argument.

Peter did also mention that their code is organised by bounded context / team, by namespace. He implied that the method of communication between contexts is controlled. I imagine that one team can not reach into the innards of the another context’s code but must (by enforced convention) go through specific “API modules”10 or possibly further disintermediation; I would also guess (hope) that direction of communication is also enforced. I would roll my eyes if they turn out to be loading the API implementation module for each boundary from config.

They have considered Saša Jurić’s Boundary package but have not used it yet; I suspect that this may be a universal experience.

Change Data Capture with Elixir and Debezium — Michal Gibowski and Vanessa Loviton

I was reminded of the existence of database transaction logs (tlogs), and how tlogs propagation from the primary database is used to keep replica databases in sync. I learned that Debezium is a project that can listen in on transaction logs, converting the tlog to a common format.

They use this approach as a Strangler Fig for replacing a legacy application, including changing the structure of the database. Debezium is able to send changes to the database down to the new database over Kafka so it can be kept up to date with changes made through the old application. Pretty clever!

One issue mentioned is that each table is sent over a different topic. Sometimes child records can be received before the parent is created. They curently deal with this with a sleep and retry as it is not a bottleneck, but they could do something more sophisticated.

Telemetry: Now what? — Zac Barnes

This was a good basic introduction to using Telemetry in your (specifically Phoenix) application.

Chris McCord Keynote

As of time of writing, I think Chris’ talk is the only one video publicly indexed and is here.

Chris started with a call out to Fly.io saying it provided a Heroku-like experience from before when Heroku stopped working on things. That was funny.

Chris focussed the rest of his talk on LiveView. When LiveView was first conceived ambitions were much less than what has now been achieved. Chris expected it be limited to basic Apps, but now it can support most things that we would previously have used a client-side Javascript framework for - but with a lot less effort.

He spent some time explaining Slots in the new component model, how they came from Surface, and how much more flexible it makes components.

Live Navigation is a feature that they do not talk about enough - loading page changes over a websocket makes things more efficient.

Uploads are also easy and great, and allow uploading.

Other things included an infinite scrolling demo, “streams” for avoiding keeping large lists in memory, solving the previous issues with dynamically sized embedded items in a form. The latter involves very cheeky use of checkboxes.

Chris’s talk was the final one. I really enjoyed the conference. There were great talks and I enjoyed connecting an reconnecting with people.

Elixir Forum thread for any comments is here

  1. I may write about this one day. ↩

  2. Which had passed me by up until now(!). ↩

  3. Blackadder ↩

  4. I think ↩

  5. Hexicles ↩

  6. You could argue that is is true of any software or framework. It may be that Phoenix tends to a better architecture than others; it would be less true of buying / renting a service. ↩

  7. As far as I remember ↩

  8. Alternatively you could implement Cowboy’s websocket handler yourself, which I have done. I found it too hard to configure this within Phoenix though so would end up creating a separate endpoint with Plug Cowboy and have to manage listening on different ports. Bit of a nightmare. ↩

  9. I think for the purposes of this talk microservices mean services, ie not so micro ↩

  10. Application in DDD terminology - but that gets confusing in BEAM ↩