Tutorial

CSV Import file upload with preview in LiveView

Phoenix 1.7 fileupload CSV liveview

A very common scenario in web applications is the need to import a larger set of data. This could be done with importing it from an API but that is not always, or even not often possible. A solution that is almost always possible to import records through a CSV file that has been exported from Excel or any other spread sheet software.

In this tutorial, I will go through how to upload and import a CSV file with Phoenix LiveView and, show how easy it is to preview the imports before actually storing them in the database.

Setup

Before starting, I need to install the CSV package. So, first I can open up the mix-file and install

# mix.exs
{:csv, "~> 3.0"},

Step 1: Create an Importer Module

We'll start by creating an Importer module to handle our CSV data. This module will include a function empty_csv_data which provides a template CSV file with headers but no data. This will serve as a sample file for users to download and populate with their data for import.

# lib/tutorial/customers/importer.ex

defmodule Tutorial.Customers.Importer do
  def empty_csv_data do
    [["First Name", "Last Name", "Email", "Phone"]]
    |> CSV.encode()
    |> Enum.to_list()
  end
end

Next, we'll set up a route in our router file that will handle requests to download the sample CSV file:

# lib/tutorial_web/router.ex

scope "/", TutorialWeb do
  pipe_through :browser

  get "/sample-csv", CsvController, :index
end

The /sample-csv route will trigger the index action in our CsvController. In the index action, we generate the sample CSV file and send it for download:

defmodule TutorialWeb.CsvController do
  use TutorialWeb, :controller

  alias Tutorial.Customers.Importer

  def index(conn, _params) do
    csv_data = Importer.empty_csv_data()

    send_download(
      conn,
      {:binary, csv_data},
      content_type: "application/csv",
      filename: "sample-import.csv"
    )
  end
end

Now if I visit https://localhost:4000/sample-csv I can download the sample csv.

Step 2: Implement the LiveView

Now Its time to turn our attention to the Import LiveView. I will start by creating a new live route, /customers/import, that will handle the process of importing customer data:

live "/customers/import", CustomerLive.Import, :import

This route corresponds to the TutorialWeb.CustomerLive.Import LiveView module. Here, we allow file uploads for :sample_csv in the mount function and set up empty assigns for storing parsed rows, imported customers, and uploaded files:

# lib/tutorial_web/live/customer_live/import.ex
defmodule TutorialWeb.CustomerLive.Import do
  use TutorialWeb, :live_view

  @impl true
  def mount(_params, _session, socket) do
    {:ok,
     socket
     |> assign(:parsed_rows, [])
     |> assign(:imported_customers, [])
     |> assign(:sample_customers, [])
     |> assign(:uploaded_files, [])
     |> allow_upload(:sample_csv, accept: ~w(.csv), max_entries: 1)}
  end

  @impl true
  def handle_event("reset", _, socket) do
    {
      :noreply,
      socket
      |> assign(:parsed_rows, [])
      |> assign(:imported_customers, [])
      |> assign(:sample_customers, [])
      |> assign(:uploaded_files, [])
    }
  end
end

In our LiveView template, we provide a form that allows users to download the sample CSV file and upload their own CSV file for parsing:

<!-- lib/tutorial_web/live/customer_live/import.html.heex -->
<.header>
  Import Customers
  <:subtitle>
    Open up the file below in Excel, Google sheets or a similar
  </:subtitle>
  <:actions>
    <.link href={~p"/sample-csv"} target="_blank" class="text-sm font-semibold leading-6 text-zinc-900 hover:text-zinc-700">
      Download Sample CSV
    </.link>
  </:actions>
</.header>

<div class="my-12">
  <form :if={length(@parsed_rows) == 0} id="upload-form" phx-submit="parse" phx-change="validate" class="space-y-2">
    <div class="p-4 border border-zinc-200 rounded" phx-drop-target={@uploads.sample_csv.ref}>
      <.live_file_input upload={@uploads.sample_csv} />
    </div>
    <.button type="submit">Upload</.button>
  </form>
  <div :if={length(@parsed_rows) > 0} class="flex items-center space-x-2">
    <h3 class="text-base font-semibold">Upload a new csv file</h3>
    <.button type="button" phx-click="reset">Reset</.button>
  </div>
</div>

Now there is functionality for both downloading the sample file and upload it back.

Step 3: Extend the Importer Module

In the next step, we'll extend our Importer module with preview and import functions. The preview function reads the first five rows from the CSV, transforms their keys to snake case, and prepares them for import. The import function does the same for all rows in the CSV.

# lib/tutorial/customers/importer.ex
defmodule Tutorial.Customers.Importer do
	alias Tutorial.Customers
  alias Tutorial.Customers.Customer  

  # ...

  def preview(rows) do
    rows
    |> Enum.take(5)
    |> transform_keys()
    |> Enum.map(fn attrs ->
      Customers.change_customer(%Customer{}, attrs)
      |> Ecto.Changeset.apply_changes()
    end)
  end

  def import(rows) do
    rows
    |> transform_keys()
    |> Enum.map(fn attrs ->
      Customers.create_customer(attrs)
    end)
  end

  # "First Name" => "first_name"
  defp transform_keys(rows) do
    rows
    |> Enum.map(fn row ->
      Enum.reduce(row, %{}, fn {key, val}, map ->
        Map.put(map, underscore_key(key), val)
      end)
    end)
  end

  defp underscore_key(key) do
    key
    |> String.replace(" ", "")
    |> Phoenix.Naming.underscore()
  end
end

Step 4: Handle File Upload Events

Next, we'll add handlers for "validate", "parse", and "import" events in our LiveView module.

In the "validate" event handler, we do nothing for now. We could add validation logic here in the future.

In the "parse" event handler, we read the uploaded CSV file, decode its contents, and assign the resulting rows to :parsed_rows. We also generate a preview of the first five rows and assign it to :sample_customers.

Finally, the "import" event handler imports all parsed rows and assigns the resulting Customer structs to :imported_customers.

  # lib/tutorial_web/live/customer_live/import.ex

  def handle_event("validate", _params, socket) do
    {:noreply, socket}
  end

  def handle_event("parse", _, socket) do
    parsed_rows = parse_csv(socket)

    {
      :noreply,
      socket
      |> assign(:parsed_rows, parsed_rows)
      |> assign(:sample_customers, Importer.preview(parsed_rows))
      |> assign(:uploaded_files, [])
    }
  end

  def handle_event("import", _, socket) do
    imported_customers = Importer.import(socket.assigns.parsed_rows)

    {
      :noreply,
      socket
      |> assign(:sample_customers, [])
      |> assign(:imported_customers, imported_customers)
    }
  end

  defp parse_csv(socket) do
    consume_uploaded_entries(socket, :sample_csv, fn %{path: path}, _entry ->
      rows =
        path
        |> File.stream!()
        |> CSV.decode!(headers: true)
        |> Enum.to_list()

      {:ok, rows}
    end)
    |> List.flatten()
  end

Finally update the template with preview and import button

<div :if={@sample_customers != []} class="space-y-2">
  <h3 class="text-xl font-bold">Preview</h3>
  <.table id="customers" rows={@sample_customers}>
    <:col :let={customer} label="First name"><%= customer.first_name %></:col>
    <:col :let={customer} label="Last name"><%= customer.last_name %></:col>
    <:col :let={customer} label="Email"><%= customer.email %></:col>
    <:col :let={customer} label="Phone"><%= customer.phone %></:col>
  </.table>
  <p>This is a sample of the data to import</p>
  <.button type="button" phx-click="import">Import Now</.button>
</div>

<div :if={@imported_customers != []} class="space-y-2">
  <h3 class="text-xl font-bold">Import Result</h3>
  <p>Success: <%= length(for {:ok, _} = row <- @imported_customers , do: row) %> imports</p>
  <p>Failed: <%= length(for {:error, _} = row <- @imported_customers , do: row) %> imports</p>
</div>

Final result. Fileupload with preview and import with validation:

Related Tutorials

Published 12 Feb - 2022

CSV Export with Phoenix and LiveView

A common need in web apps is to export data to different file formats. One common, or even maybe the most common format, is exporting CSV files. CSV..

Published 25 Nov - 2022
Updated 29 Nov - 2022
Phoenix 1.7

Sortable lists with Phoenix LiveView and SortableJS

A very common user interface pattern on the web is to have sortable elements. In this tutorial I will go through how to accomplish sortable lists wi..