This week I have been mainly iterating (aka going round in circles) on application deployment, but I did say that I would talk a bit about the backend of the memorable password generator.

It’s just an ETS table

On reflection, there is not much to say on the backend. It is essentially an ETS table. It is owned by a GenServer, WordList.

WordList is initialised with a reference (for access) and an Enum containing all the words. In production this is a FileStream to a text list of just under 5,000 common words which I picked up from various places. For testing it is just a short list of words.

WordList’s GenServer creates the table in its init/1.

 def init({reference, words}) do
    table = :ets.new(reference, [:named_table, :set, read_concurrency: true])
    {:ok, {table, words}, {:continue, :add_words}}
  end

Note that:

  • It’s a named table and its name is the reference.
  • The default table setting is protected meaning that only the owning process can write to the table but, crucial for concurrent access, any process can read.
  • It is read concurrent

It’s also a set, meaning the keys are unique, but that is not important in this case.

Reading the contents into the table carries on in a handle_continue

def handle_continue(:add_words, {table, words}) do
    words
    |> Stream.map(&String.trim/1)
    |> Stream.with_index()
    |> Enum.each(fn {w, i} -> :ets.insert(table, {i, w}) end)

    {:noreply, {}}
end

The words a trimmed to get rid of the trailing \n that comes in the filestream. The word is added to the the table with an index as the key.

The WordList module provides a convenience method to retrieve a word from the table by its table reference, and word index, word_at/2.

@spec word_at(word_list_ref(), pos_integer()) :: {:error, :invalid_index} | {:ok, String.t()}
def word_at(reference, index) do
  case :ets.lookup(reference, index) do
    [{^index, word}] -> {:ok, word}
    _ -> {:error, :invalid_index}
  end
end

Not that this takes place in the caller’s process, allowing concurrent access. If there had been a GenServer.call to the owning process then this would cause a bottleneck.

The size of the table is easy enough to get:

@spec size(word_list_ref()) :: pos_integer()
def size(reference) do
  :ets.info(reference, :size)
end

So, getting a random word is also super easy:

@spec random_word(word_list_ref()) :: String.t()
def random_word(reference) do
  size = size(reference)
  index = :rand.uniform(size) - 1
  {:ok, word} = word_at(reference, index)
  word
end

Other bits and pieces of the backend involve getting a list of random words matching the minimum words and characters specifications) and turning that list into a password with the decoration and separation options.

Iterating on the deployment

I explored using packer to creat an AMI containing the release on top of a basic image. It turned out to be both easy and a poor approach: it is painfully slow and uses up lots of EBS storage.

I went back to minimal provisioning with Terraform. Right now I’m using the file provisioner to get a gzipped tar of the release onto the box. The remote exec provisioner then untars and starts the service.

Rust meetup

On Thursday evening I attended the remote Edinburgh Rust meetup, which was an account of a setup to automatically irrigate tomatoes in a greenhouse. I enjoyed the talk, though I have only dabbled in Rust. Obviously I would have used Nerves for such a project, but Rust was a fine choice. Neil’s code is here.

An interesting snippet was using Rust targeted at Web Assembly for graphing sensor information. Rust for front-end web visuals seems to be a valuable emerging niche for the language.