How to setup recurring jobs with Oban in Elixir

Oban has proven itself to be the most versatile job processing library in Elixir and Phoenix. Coming from Sidekiq, it feels very familiar and supports scenarios with async jobs, jobs to be performed in a certain time in the future, and recurring jobs based on a cron schedule.

The latter is especially great if you want to setup a daily digest email to yourself or your users.

In this tutorial we will go through how to add Oban and create a daily digest email that I send out with Bamboo.

Step 1 - Download a premade boilerplate

As with many of the tutorials, I need some prerequisites. In this case, I want a premade Phoenix app with Pow authentication and Bamboo email.

Generate a boilerplate

After the boilerplate is downloaded, I need to run

mix deps.get
mix ecto.setup
yarn install --cwd assets

Step 2 - Install Oban

Oban is a well documented library and the installation instructions are pretty straight forward. First add it to the mix file.

# mix.exs
def deps do
  [
    {:oban, "~> 2.6.1"}
  ]
end

Oban is dependent on storing the job queues in a Postgres database so after the deps are fetched, I need to generate the migration file for it.

mix deps.get
mix ecto.gen.migration add_oban_jobs_table

And paste in this code that comes directly from the installation instructions:

# priv/repo/migrations/*_add_oban_jobs_table.exs
defmodule MyApp.Repo.Migrations.AddObanJobsTable do
  use Ecto.Migration

  def up do
    Oban.Migrations.up()
  end

  # We specify version: 1 in down, ensuring that we'll roll all the way back
  # down if necessary, regardless of which version we've migrated up to.
  def down do
    Oban.Migrations.down(version: 1)
  end
end

Run the migrations by

mix ecto.migrate

These are the basic configuration recommended in the get started guide. You could tweak the queue prioritizations so it fits your needs.

Most likely, you don’t want to start Oban to work off queues in tests so turn it off in the test config by adding:

# config/test.exs
config :my_app, Oban, queues: false, plugins: false

With the configurations in place, add Oban inside the children list so it’s started when the application starts.

# lib/my_app/application.ex
defmodule MyApp.Application do
  use Application

  def start(_type, _args) do
    children = [
      MyApp.Repo,
      MyAppWeb.Telemetry,
      {Phoenix.PubSub, name: MyApp.PubSub},
      MyAppWeb.Endpoint,
      {Oban, oban_config()} # Add this line
    ]

    opts = [strategy: :one_for_one, name: MyApp.Supervisor]
    Supervisor.start_link(children, opts)
  end

  defp oban_config do # Add this function
    Application.fetch_env!(:my_app, Oban)
  end
end

That should conclude the basic installation and setup of Oban in an Elixir application.

Step 3 - Add the daily digest mailer

Next step is to setup the daily digest email. This will be a very simple daily digest. I just want to receive an email on all users that have signed up in the last 24 hours.

# lib/my_app/users.ex
defmodule MyApp.Users do
  @moduledoc """
  The Users context.
  """

  import Ecto.Query, warn: false
  alias MyApp.Repo
  alias MyApp.Users.User

  def list_users_from_last_day do
    from(
      u in User,
      where: u.inserted_at > ago(1, "day")
    )
    |> Repo.all()
  end
end

Since I want the email to be sent at the same time every day, I don’t need to specify the time in the query.

Since I picked a boilerplate with Bamboo installed, I already have the mailer and some emails setup. But I need to add my digest_email that takes the users:

# lib/my_app/emails.ex
defmodule MyApp.Email do
  import Bamboo.Email

  def digest_email(users) do
    base_email()
    |> subject("Daily Digest")
    |> to("myself@example.com")
    |> render("daily_digest.html", title: "Daily Digest Email", users: users)
  end

  defp base_email do
    new_email()
    |> from(@from)
    |> put_html_layout({MyAppWeb.LayoutView, "email.html"})
    |> put_text_layout({MyAppWeb.LayoutView, "email.text"})
  end
end

I want the content of the email in a template file:

<!-- lib/my_app_web/templates/email/daily_digest.html.eex -->
<%= if Enum.count(@users) == 0 do %>
  <h1>No users signed up since yesterday</h1>
<% else %>
  <h1>These users have joined since yesterday</h1>

  <%= for user <- @users do %>
    <%= user.email %><br>
  <% end %>
<% end %>

I can now test this in IEx:

MyApp.Users.list_users_from_last_day() |> MyApp.Emails.digest_email() |> MyApp.Mailer.deliver_now()

And I should get the variant that no users have signed up since I haven’t added any users.

However, if I start the app and register a user:

MyApp.Users.list_users_from_last_day() |> MyApp.Emails.digest_email() |> MyApp.Mailer.deliver_now()

If I run the code again, then I can see that I have a new user registered since yesterday. And with that part working, I will need to move that code from IEx to a worker and have that triggered by Oban.

Step 4 - Configure Oban to run daily

First I want to store my workers in a separate folder. So I create a workers folder and add the daily digest worker in there. I will do the exact same thing as the command I ran in the IEx console.

# lib/my_app/workers/daily_digest_worker.ex
defmodule MyApp.Workers.DailyDigestWorker do
  use Oban.Worker

  alias MyApp.{Users, Emails, Mailer}

  @impl Oban.Worker
  def perform(_job) do
    Users.list_users_from_last_day()
    |> Emails.digest_email()
    |> Mailer.deliver_now()

    :ok
  end
end

The last piece of the puzzle is to configure Oban to run the worker daily at a specific time. Oban supports the normal cron syntax for this. In this case, I want to run this at 08:00 (server time) every day.

# config/config.exs
config :my_app, Oban,
  repo: MyApp.Repo,
  queues: [default: 10, mailers: 20, events: 50, low: 5],
  plugins: [
    Oban.Plugins.Pruner,
    {Oban.Plugins.Cron,
     crontab: [
       {"0 8 * * *", MyApp.Workers.DailyDigestWorker},
     ]}
  ]