Fuzzy find with Ecto in Phoenix LiveView

Fuzzy find is both a simple and a complex thing. Even though though its simple to implement, its hard to get right from a UX perspective. Luckily, I can help you get going with the technological implementation.

Postgres has some build in functionality for fuzzy search. First there is a extension called pg_trgm.

A trigram is a group of three consecutive characters taken from a string. So, by measuring the similarity of trigrams between two strings, it is possible to estimate how similar they are on a scale between 0 and 1. This allows for fuzzy matching, by setting a similarity threshold above which strings are considered to match.

Another built in function for comparing strings is the Levenshtein function. This function calculates the Levenshtein distance between two strings. That is the number of single character deletions, insertions, or substitutions required to transform one string into the other.

So, the starting point for this is a view with a list of products. I would like to add a search or filter with fuzzy functionality.

STEP 1 - Install the needed extensions

I need to create a migration where I activate the needed extensions.

defmodule ShopTest.Repo.Migrations.EnableFuzzyExtensions do
  use Ecto.Migration

  def up do
    execute "CREATE EXTENSION pg_trgm"
    execute "CREATE EXTENSION fuzzystrmatch"
  end

  def down do
    execute "DROP EXTENSION fuzzystrmatch"
    execute "DROP EXTENSION pg_trgm"
  end
end

STEP 2 - Building the search query

A good search function is a science in itself and it depends on use case, dataset and user base. However, In this example, I go about it in this way:

• I want the starting characters to match. So if I type an "A", I only want to select phrases that starts with "A". For this I extract the first character and use it in an ILIKE.
• Next, I only take search results where thare are some similarity. So SIMILARITY between the strings are about 0.
• Last, I order the strings according to LEVENSHTEIN function.

One thing to note here is that the search result will be more correct, the more a user types in the search box.

So, in the products context, I add this serach function:

def search_products(search_phrase) do
  start_character = String.slice(search_phrase, 0..1)

  from(
    p in Product,
    where: ilike(p.name, ^"#{start_character}%"),
    where: fragment("SIMILARITY(?, ?) > 0",  p.name, ^search_phrase),
    order_by: fragment("LEVENSHTEIN(?, ?)", p.name, ^search_phrase)
  )
  |> Repo.all()
end

Next step will be to add the actual interface to the search

STEP 3 - Add search to LiveView

As I mentioned above, I already have a products LiveView. So I will start with adding the search form. So, in the index template on the correct spot, I add:

# lib/shop_test_web/live/product_live/index.html.leex

<%= f = form_for @changeset, "#", phx_change: "search", as: "search" %>
  <%= label f, :search_phrase, class: "tag-label" do %>
    <div class="tag-icon">
      <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="11" cy="11" r="8"></circle><line x1="21" y1="21" x2="16.65" y2="16.65"></line></svg>
      <%= text_input f, :search_phrase, class: "tag-input",  phx_debounce: 500, placeholder: "Search product" %>
    </div>
  <% end %>
</form>

NOTE that I have added a debounce there on 500ms.

As soon as I type something I send an event to the ProductLive.Index module.

# lib/shop_test_web/live/product_live/index.ex

@impl true
def handle_event("search", %{"search" => search}, socket) do
  search
  |> search_changeset()
  |> case do
    %{valid?: true, changes: %{search_phrase: search_phrase}} ->
      {:noreply, assign(socket, :products, Products.search_products(search_phrase))}
    _ ->
      {:noreply, socket}
  end
end

Note that I have a changeset here. I want to have a schema less changeset. That makes it possible to add some validations to the actual search phrase before actually putting it in the query.

# lib/shop_test_web/live/product_live/index.ex

@types %{search_phrase: :string}

defp search_changeset(attrs \\ %{}) do
  cast(
    {%{}, @types},
    attrs,
    [:search_phrase]
  )
  |> validate_required([:search_phrase])
  |> update_change(:search_phrase, &String.trim/1)
  |> validate_length(:search_phrase, min: 2)
  |> validate_format(:search_phrase, ~r/[A-Za-z0-9\ ]/)
end

One more thing I want to add, is to reset the list if the search phrase changes to an empty string.

# lib/shop_test_web/live/product_live/index.ex

@impl true
def handle_event("search", %{"search" => %{"search_phrase" => search_phrase}}, socket) when search_phrase == "" do
  {:noreply, assign(socket, :products, list_products())}
end

FINAL RESULT