Elixir Phoenix 1.4 Deployments with Distillery and Edeliver on Ubuntu

In that past, there have been some pain points around deployments for Elixir and Phoenix 1.4, and after managing some production deployments we realized that the available tools, such as Distillery and Edeliver, have greatly reduced the complexity of deployments.

In this article we're going to explore how to deploy a Phoenix 1.4 application backed by a Postgres database to Digital Ocean on a generic Ubuntu 18.04 droplet without using Docker.  We'll use Distillery to create releases, and edeliver to handle deployment related tasks, and use Nginx as a proxy server to handle requests.

There's no shortage of set up and configuration needed for this article, but it should all be very familiar if you've worked with Ubuntu based servers in the past.

Assumptions

  • You are familiar with Ubuntu systems and are able to SSH into a server to run commands and install/configure software.
  • You have some working knowledge of how to create Elixir and Phoenix applications.

The Plan

Step 1: Generate new Phoenix 1.4 Application

We're going to start off with a very simple Application using the well known commands.  

On your local machine, navigate to your work directory and run these commands:

$ mix phx.new phxroad
  ## Hit enter to install frontend dependencies

$ cd phxroad
$ mix ecto.create
$ mix phx.server

Visit http://localhost:4000 and you should see the beautiful landing page:

Step 2:  Create Target Server on Digital Ocean

We're going to use Digital Ocean for our production server.  Their UI and tools are very powerful, but much simpler to manage than some other services.  

First things first,  create a small droplet based on Ubuntu 18.04 in Digital Ocean.  If you don't have an account sign up here.  

Add your SSH key to the new server, and login as root.

NOTE: use 2GB droplet minimum so things run smoothly

Step 3: Target Server - Initial Setup

Before we go too much further, let's let our default editor to vim.  If you prefer nano, you can skip this step.

As root on the target server:

$ sudo update-alternatives --config editor

Enter 3 for vim.basic and hit enter.

Update locale

We'll also set some system locale variables to prevent some warnings as we progress:

$ sudo update-locale LC_ALL=en_US.UTF-8
$ sudo update-locale LANGUAGE=en_US.UTF-8

Create new deploy user


The next step is to create a non-root user on the server which will own our application and handle deployments.  We'll configure the .ssh directory for the user, configure the user to have passwordless sudo access, and disabled password login to harden the server a bit.

Let's create the user deploy.

As root on the target server:

$ adduser deploy

Fill out the data as required.

Next, run this series of commands to configure the user's .ssh directory:

sudo mkdir -p /home/deploy/.ssh
sudo touch /home/deploy/.ssh/authorized_keys
sudo chmod 700 /home/deploy/.ssh
sudo chmod 644 /home/deploy/.ssh/authorized_keys
sudo chown -R deploy:deploy /home/deploy

As root on the target server:

$ sudo vim /etc/ssh/sshd_config

Verify that password authentication is set to "no":

PasswordAuthentication no

Add deploy to sudoers

Finally we'll add the user to sudo.

As root on the target machine:

$ visudo

This will open the sudoers file where we can add the deploy users privileges below root like this:

# User privilege specification
root     ALL=(ALL:ALL) ALL
deploy   ALL=(ALL) NOPASSWD: ALL

Add ssh public key to authorized_keys

In another terminal window, on your local machine:

$ cat ~/.ssh/id_rsa.pub

On the target server, paste the key into authorized_keys:

$ sudo vim /home/deploy/.ssh/authorized_keys

Now we can safely log in and work as the deploy user on the target machine without having to enter a password.

Step 4: Target Server - Install asdf

Sometimes, you'll need to install specific versions or Erlang, Elixir or Node, and we'll use asdf, a version manager to tackle this complex task.

asdf is an extendable version manager with support for Ruby, Node.js, Elixir, Erlang & more.  

To install it, we'll first switch over to our newly created deploy user.

On the target server as root:

$ su deploy
$ cd # move into deploy's home path

Install asdf

Install dependencies:

Ensure that you are using the deploy account, and clone the repo:

$ git clone https://github.com/asdf-vm/asdf.git ~/.asdf

Then add the shims and source the changes:

$ echo -e '\n. $HOME/.asdf/asdf.sh' >> ~/.bashrc
$ echo -e '\n. $HOME/.asdf/completions/asdf.bash' >> ~/.bashrc

$ source ~/.bashrc

We're now ready to use asdf on our system to manage versions for Erlang, Elixir and Node.

Enter asdf in the console to check if in works.

Step 5: Target Server - Install Erlang/Elixir

Because we need our Phoenix project to run on both the local development machine and the production server, we'll need to install the same languages and tools in both places.  Erlang 21.1, and Elixir 1.7.4.

Install Erlang

We're going to use asdf to install Erlang.  I uses plugins for different libraries, so let's add the plugin:

As deploy on the target machine:

$ asdf plugin-add erlang

Install Erlang/OTP 21.1 (or whichever version your app needs)

$ asdf install erlang 21.1
$ asdf global erlang 21.1

Install Elixir

As deploy add the plugin:

$ asdf plugin-add elixir

Install Elixir and make it global:

$ asdf install elixir 1.7.4
$ asdf global elixir 1.7.4

Now you can open a new terminal and try erl:

$ erl
Erlang/OTP 21 [erts-9.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V9.3  (abort with ^G)
1> 

Or start Erlang Observer by erl -s observer start.

And you can try `iex`:

$ iex Erlang/OTP 21 [erts-9.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false]  Interactive Elixir (1.7.4) - press Ctrl+C to exit (type h() ENTER for help) iex(1)>

Use asdf .tool-versions file to manage which version is active on each of your projects.

Use Mix to install Hex.

$ mix local.hex

When prompted to confirm the installation, enter Y.

OutputAre you sure you want to install "https://repo.hex.pm/installs/1.5.0/hex-0.17.1.ez"? [Yn] Y
* creating .mix/archives/hex-0.17.1

We'll still need to install some other tools like Postgresql and NodeJS, but you should be familiar with this process.

Step 6: Target Server - Install Nodejs

Installing Node is straightforward with a few commands.

On the target server:

$ sudo apt install curl
$ curl -sL https://deb.nodesource.com/setup_8.x | sudo bash -
$ sudo apt-get install -y nodejs

Step 7: Target Server - Install Postgresql

We're going to install Postgresql 10 for this article.

As root on the target server:

$ sudo app update
$ sudo apt install -y postgresql postgresql-contrib

Create new database user

During the postgres installation, a postgres root user postgres was created, but we don't want to connect to the server with this user. Let's create a separate postgres user for our app.

You can switch to the postgres user with su postgres than back to root with exit

On the target server, switch to the postgres user, create a phx database user and set the new user's password:

$ su postgres
$ createuser phx --pwprompt
Enter password for new role: 
Enter it again:

NOTE: Remember these credentials as they'll be used once we configure our Phoenix application for production.

Create production database:

We'll need to create our production database manually as edeliver only manages migrations.

As postgres on the target server:

$ createdb phx_prod

Then log in to pgsql:

$ pgsql

Ensure that you are logged into the postgres cli tool and run:

GRANT ALL PRIVILEGES ON DATABASE phx_prod TO phx;

This gives our new phx user access to the newly created database.

Database credentials:

To re-cap:

  • username: phx
  • password: <yourpass>
  • database: phx_prod

Step 8: Target Server - Production Configuration

Create prod.secret.exs

You may have noticed that Phoenix created a config/prod.secret.exs file.  This is imported by config/prod.exs but is ignored by git by default.  We'll need to create this file on the target machine so edeliver can symlink to it during the build process.

We're also going to store our applications in  ~/apps/<appname> format which is the equivalent of /home/deploy/apps/<appname>.

Let's switch over to our deploy user.  On the target machine:

$ su deploy

Make a new directory for our secrets:

$ mkdir -p apps/phxroad/secret

Then create a new prod.secret.exs file in that directory:

$ vim ~/apps/phxroad/secret/prod.secret.exs

Add this content to it:

You can generate a new secret key by using mix phx.gen.secret on your local machine.

IMPORTANT: Update the file to have a production ready secret key, and the database credentials you set from earlier.

Step 9: Install  Distillery and edeliver

As previously mentioned, Distillery compiles our Phoenix application into releases, and edeliver uses ssh and scp to build and deploy the releases to our production server.

On your local machine, open mix.exs and add 2 deps:

      {:edeliver, ">= 1.6.0"},
      {:distillery, "~> 2.0", warn_missing: false},

And add :edeliver to extra_applications in the application block:

  def application do
    [   
      mod: {Phxroad.Application, []},
      extra_applications: [:logger, :runtime_tools, :edeliver]
    ]   
  end 

Use mix to install deps:

$ mix deps.get

Initialize Distillery

Distillery requires a build configuration file that is not generated by default.

Let's generate it with:

$ mix release.init

This generates configuration files for Distillery in the rel directory.  We don't need to make any changes to the default config.

Configure edeliver

Create a .deliver directory in your project folder and add the config file:

These are mostly environment variables use by Edeliver in the shell scripts.  Go through them one by one and try to understand what they're doing.

You can read more about the configuration variables in their Wiki.

Don't forget to update your host configuration

The custom functions are hooks that run during different phases of the deployment.

You can also read more about running additional tasks in their Wiki.

Project prod configuration

We'll need to make some changes to the default production configuration in our project.

Open config/prod.exs and update it to this:

These are settings recommended by Distillery.  You might notice that we're setting the system port using the PORT environment variable.   This needs to be available during the build process and we can add it to the ~/.profile file on our production server.

Add env Variables to Target Server

Login as deploy to the target server.

$ vim ~/.profile

And add these lines:

export MIX_ENV=prod
export PORT=4000

This will ensure that our system runs on port 4000, and that the environment is set to prod.

We're using port 4000 because we'll be routing traffic to it using Nginx as a proxy later in the article.

Step 10: Deployment

We're finally ready to build and deploy our first release.  The default version is 0.1.0 so let's just keep that for now.

Warning: the commands for building and deploy can seem complicated at first but once you become familiar with them, you will see that they use a natural language style.

Build the production release:

In your project directory:

$ mix edeliver build release production

This builds the release on the target server, and stores the archive in your local .edeliver/releases directory.

Deploy the reload to production:

In your project directory:

$ mix edeliver deploy release to production

This uploads the release archive to the specified directory and extracts it, and starts the production server.

Admin commands

There are many other commands available for managing the deployment.  Here are some examples:

You can read more about Edeliver in their documentation.

Test it in the browser

You should now be able to access your project at your IP or domain and port 4000:

http:/phxroad.com:4000

Step 11: Target Server - Install Nginx

Nginx is a powerful HTTP server and reverse proxy which is widely adopted and straightforward to install and configure.

Log in as root to the target server and install Nginx:

$ sudo apt update
$ sudo apt install -y nginx

Remove default configurations:

$  rm /etc/nginx/sites-enabled/default /etc/nginx/sites-available/default

Create a new phxroad.com file (use your own domain)

$ sudo vim /etc/nginx/sites-available/phxroad.com

Add add this content:

Test Nginx config:

$ sudo nginx -t

If everything looks good, restart Nginx:

$ sudo service nginx restart

You can now access your new server at: <yourdomain.com> and nginx will route requests to your Phoenix application running on port 4000.

Wrapping up

We've covered many different aspects of deploying a Phoenix 1.4 application, and the major tools that helped us accomplish it, namely Distillery and Edeliver.

Though we're just skimming the surface of how powerful OTP applications are, and how these tools can help customize deployments, this has been a good start to getting your applications into production.

Feel free to explore the features of Distillery and Edeliver more as your needs inevitably expand.

If you have any issues feel free to reach out, and I'll try to help!

UPDATE:

We're planning a follow up article to this to plug CircleCI into the deployment and test process.  

Follow me on twitter for updates: @tmartin8080