Researching Rails Performance Issues - Memory

As anyone experienced with Rails production applications might know, there are times when certain features or parts of an application cause high memory usage and/or unacceptably slow response times.

There are many contributing factors to why rails memory usage can get unwieldy, but as one of our roles as Ruby developers, we need to be able to figure out what they are, and what causes them.

Thankfully, there are some tools available to us that will help investigate and resolve performance issues.  

Basics of Ruby Memory

Ruby memory is an enormous topic which should be researched in greater depth, but for the purposes of this simple guide and without going into too much detail here are a few things to keep in mind:

Ruby is a "managed language" that uses Garbage Collection (GC) to automatically detect and free the memory which has become unnecessary.  
Garbage Collection only cleans up objects.  If an object is assigned to a CONSTANT it will not be garbage collected.
If you have created more objects than Ruby can fit into memory, it will allocate additional memory.
Requesting memory from the operating system is an expensive operation, so Ruby will allocate a larger chunk of memory than it needs rather than a few KB at a time.
Because allocating memory is expensive, Ruby holds onto it for some time. The memory will be freed gradually, but slowly.

This is obviously not an exhaustive outline, and I would recommend doing some further research into how Ruby manages memory as it will help when analyzing memory related performance issues in Ruby and Rails.

Getting Started

Let's start by adding the following gems to the Gemfile:

gem 'memory_profiler'
gem 'derailed_benchmarks'

derailed_benchmarks is a gem written by Richard Schneeman which in his own words contains:

A series of things you can use to benchmark a Rails or Ruby app.

With our laymen's understanding of Ruby memory and these gems in place the first place we can start to look for memory issues is with our gems.

Static Benchmarking

The following section covers how to get memory information from gems in your Gemfile without having to actually boot your app.

Analyzing Gem Memory at Require

Before we begin looking through our app code, we're going to analyze gems included in our Gemfile for memory issues.

Each gem you add to your project can increase your memory at boot. You can get visibility into the total memory used by each gem in your Gemfile by running:

$ bundle exec derailed bundle:mem
TOP: 28.582 MiB
  rails/all: 24.6602 MiB
    rails: 10.8672 MiB (Also required by: active_record/railtie, active_model/railtie, and 7 others)
      active_support/railtie: 4.2461 MiB
        active_support/i18n_railtie: 4.1914 MiB
          active_support/file_update_checker: 3.5273 MiB (Also required by: rails/application/configuration)

The output displays the total memory used required for all the gems, lists them from highest to lowest, and if there is a "bad gem" (using this term loosely) that is allocating too much memory at require time, it will be easy to detect at this stage.

You can read more about bundle:mem here.

Once you have identified a possible "bad gem" that creates a large amount of memory,  you can add it to a separate Gemfile and get detailed information about it:

$ bundle exec derailed bundle:objects
This information can be used by contributors and library authors to identify and eliminate object creation hotspots.

Armed with insight into the memory usage of each gem at require time you can quickly determine if memory issue is caused by a particular gem.  

If nothing seems out of place here, can we move on to Dynamic Benchmarking.

Dynamic Benchmarking

This section requires you to be able to boot your rails app in production mode locally.

You may need to change your config and env variables temporarily.

First, ensure that

$ RAILS_ENV=production rails server

will run, and address any issues.  

Memory over Time

Derailed comes with a very useful tool that tracks memory usage over time.  

Something important to note about this is that not every memory related issue is an actual "Memory Leak".  

If your app appears to be leaking ever increasing amounts of memory, you'll want to first verify if it's an actual unbound "leak" or if it's just using more memory than you want. A true memory leak will increase memory use forever, most apps will increase memory use until they hit a "plateau".

To diagnose this you can run:

bundle exec derailed exec perf:mem_over_time
$ bundle exec derailed exec perf:mem_over_time
Booting: production
Endpoint: "/"
PID: 78675
103.55078125
178.45703125
179.140625
180.3671875
182.1875
182.55859375
# ...
183.65234375
183.26171875
183.62109375

This example levels off at 183 MiB which means that there is no "unbound memory leak".

Customizing the test

There are some ways to customize test for different endpoints, and here are a few things that normally need to change:

  • TEST_COUNT=5000 - increasing test count
  • PATH_TO_HIT - change the endpoint
  • USE_AUTH - enable the use of auth

If you enable auth, and use devise create a perf.rake file in the root dir:

# perf.rake
DerailedBenchmarks.auth.user = -> { User.find_by(email: 'your@email.com') }

Now the full command would be something like:

TEST_COUNT=10_000 USE_AUTH=true bundle exec derailed exec perf:mem_over_time

There are many more details for the tools included in the documentation for Derailed gem such as customizing the endpoints and adjust the counts.  It might be a good idea to take some time to read about the different features and configuration options to help in assessing issues in your app.

Analyzing Memory Leaks

If you find an endpoint/process responsible for a memory leak, or you simply want to see where your memory use is coming from, there is another tool included in derailed_benchmarks that will help with this.

$ bundle exec derailed exec perf:objects
This task hits your app and uses memory_profiler to see where objects are created. You'll likely want to run once, then run it with a higher TEST_COUNT so that you can see hotspots where objects are created on EVERY request versus just maybe on the first.
$ TEST_COUNT=10 bundle exec derailed exec perf:objects

With this data, you can see which lines of your application allocate the most objects when you hit an endpoint.

allocated memory by file
-----------------------------------
      7104  /usr/local/bundle/gems/rack-2.0.7/lib/rack/utils.rb
      4680  /usr/local/bundle/gems/activesupport-5.2.3/lib/active_support/hash_with_indifferent_access.rb
      3880  /usr/local/bundle/gems/actionpack-5.2.3/lib/action_dispatch/http/response.rb
      3570  /usr/local/bundle/gems/actionview-5.2.3/lib/action_view/helpers/tag_helper.rb
      3288  /usr/local/bundle/gems/rack-2.0.7/lib/rack/mock.rb

Conclusion

This guide is intended as a basic introduction into assessing memory related issues in Ruby and Rails.  We introduced some tools and strategies for getting started and though we have barely scratched the surface on the subject, I hope you find this introductory guide useful.  

As always if you have any questions for issues about what we've covered feel free to reach out any time @tmartin8080.

Resources