Database Options

📣 NEW! Precompiled Mysql2 Gem for Lambda for RDS Proxy.

RDS Proxy MySQL Setup
Using Aurora Serverless
Using DynamoDB
Other Resources

RDS Proxy Step by Step with MySQL

Thanks to the recent announcement of RDS Proxy becoming generally available, you can now run Rails in AWS Lambda with either MySQL or PostgreSQL. Lamby makes this really easy with MySQL since we have precompiled and statically linked libmysqlclient in an easy to use gem named mysql2-lambda. Assuming you have already run through our Quick Start guide, here are the steps needed to setup RDS Proxy with MySQL.

Create Your RDS Instance & Proxy
Update Your Lamby Project
Configure AWS SAM
The DATBASE_URL Environment Variable
Deploy & Test
Local MySQL Development
What About Migrations?

1️⃣ Create Your RDS Instance & Proxy

We have created a CDK stack at customink/lamby-rds-proxy which can quickly get you up and running with RDS Proxy. For most, the default VPC in your aws account will do. However if needed, we created the customink/lamby-vpc CDK stack to create a fresh VPC.

Before running these RDS Proxy CDK commands, log into your AWS console, navigate to Services -> VPC -> Your VPCs to get the the value for the VPC_ID where your RDS instance & proxy will be created. Set the DB_NAME value to something that makes sense for you. Like the name of your app.

$ git clone https://github.com/customink/lamby-rds-proxy.git
$ cd lamby-rds-proxy
$ ./bin/bootstrap && ./bin/setup
$ DB_NAME=myapp \
  VPC_ID=vpc-01a23b45c67d89e01 \
  ./bin/deploy

Once completed the stack will output useful information about the resources it created. Make a note of the MyDbProxyDbUrlParameterName whose value will be needed later on for your DATABASE_URL. You will also need the subnet groups and security group(s) in later steps. You can find these in the "Proxy configurations" section in the AWS Console as seen in the image to the right.

These stacks are for learning purposes & create resources in public subnets. Consult the project's README for security details.

2️⃣ Update Your Lamby Project

Add the precompiled for Amazon Linux mysql2-lambda to your Gemfile.

gem 'mysql2-lambda'

Enable ActiveRecord. Open config/application.rb and uncomment this line.

require "active_record/railtie"

Add this to your config/environments/development.rb file.

# Raise an error on page load if there are pending migrations.
config.active_record.migration_error = :page_load
# Highlight code that triggered database queries in logs.
config.active_record.verbose_query_logs = true

Add this to your config/environments/production.rb file.

# Do not dump schema after migrations.
config.active_record.dump_schema_after_migration = false

Create the app/models/application_record.rb base model.

class ApplicationRecord < ActiveRecord::Base
  self.abstract_class = true
end

Open the bin/build-rails file and remove this line from the cleanup artifacts section.

./gems/activerecord-*

One last thing! It will be nice to know this all works after our next deploy. Open the app/views/application/index.html.erb file and change the <h1> tag's welcome to Lamby message to:

<h1>
  RDS Proxy Active: <%= ApplicationRecord.connection.active? %>
</h1>

3️⃣ Configure AWS SAM

We are going to need a few changes to your AWS SAM template.yaml file. We need a policy for your Lambda to access your VPC. AWS SAM makes this easy with the AWSLambdaVPCAccessExecutionRole policy template which can be added to a Policies property of your Rails Lambda. Next you will need a VpcConfig using the subnets and security group(s) created in step 1️⃣ above. Both of these should be added underneath the Properties section, for example:

RailsLambda:
  # ...
  Properties:
    # ..
    Policies:
      - AWSLambdaVPCAccessExecutionRole
    VpcConfig:
      SubnetIds:
        - subnet-09792e6cd06dd59ad
        - subnet-0501f3136415021da
      SecurityGroupIds:
        - sg-07be99aff5fb14557

4️⃣ The DATBASE_URL Environment Variable

We now need to set the DATBASE_URL environment variable so Rails can connect to the RDS Proxy. The CDK stack in step 1️⃣ above created this string which contains everything from the DBs user/password to the host in an AWS Service called Systems Manager Parameter Store. Open up your bin/build-rails file and add the code below under the RAILS_MASTER_KEY line in the "Environments & Configuration" section. But please change myapp-dburl with the value from MyDbProxyDbUrlParameterName when you deployed your CDK stack above.

DATABASE_URL=$(aws ssm get-parameter \
  --name "myapp-dburl" \
  --with-decryption \
  --query "Parameter.Value" \
  --output text 2> /dev/null | echo "$(</dev/stdin)"
)
echo "DATABASE_URL=$DATABASE_URL" >> ".env.$RAILS_ENV"

This works because our starter project uses Dotenv. This variable will be stored in a .env.production file which is packaged as part of your Lambda's code and loaded when Rails boots.

5️⃣ Deploy & Test

With all these changes done, you can now run the bin/deploy script again. If everything is working, the <h1> tag should say "RDS Proxy Active: true". If not or you see an error, check your CloudWatch logs for hints as to what went wrong.

6️⃣ Local MySQL Development

The Lamby cookiecutter makes heavy use of Docker for local development. Adding a MySQL container to our Docker Compose setup is pretty straight forward. When finished, all existing scripts will automatically start your MySQL server which can actually be shared with other projects too. Here are the steps to update your project.

Open your docker-compose.yml file and replace with this. Essentially we are adding a depends_on and links to the cicd service to use a new mysql service. In this example we are using MySQL v5.7. This version can be changed to match your needs.

version: '3.7'
services:
  cicd:
    build: .
    environment:
      - RUBYOPT=-W0
      - RAILS_ENV=${RAILS_ENV-development}
      - SAM_CLI_TELEMETRY=0
      - AWS_PROFILE=${AWS_PROFILE-default}
      - AWS_DEFAULT_REGION=${AWS_DEFAULT_REGION-us-east-1}
    volumes:
      - ~/.aws:/root/.aws:delegated
      - ~/.gitconfig:/root/.gitconfig:ro
      - .:/var/task:delegated
    depends_on:
      - mysql
    links:
      - mysql
  mysql:
    image: mysql:5.7
    container_name: mysql57
    restart: always
    environment:
      - MYSQL_ROOT_PASSWORD=root
    ports:
      - '3307:3306'
    volumes:
      - 'dbdata:/var/lib/mysql'
volumes:
  dbdata:

Create a config/database.yml file in your project with the following contents. Change myapp database prefixes or any default configs that make sense for you. The host must match the service/links in the docker compose file.

default: &default
  adapter: mysql2
  encoding: utf8mb4
  charset: utf8mb4
  collation: utf8mb4_unicode_ci
  host: mysql
  username: root
  password: root

development:
  <<: *default
  database: myapp_development

test:
  <<: *default
  database: myapp_test

Open your project's bin/_setup file and add this Rails task right after the bundle/yarn install section.

echo '== Preparing database =='
./bin/rails db:setup

You are now all set up! You will need to run through your bootstrap & setup scripts again to get everything working.

$ ./bin/bootstrap
$ ./bin/setup
$ ./bin/server

7️⃣ What About Migrations?

Good question. You simply can not rely on legacy systems that have deploys & schema migrations happening at the same time with Lambda. A better system typically involves changing your schema in a way that works before & after your code changes happen.

That said, there has to be a better way of running a migration vs resorting to DDL statements sent directly to your RDS cluster from a jump server. In the future I will be looking at a tool like active_record_migration_ui or maybe some ActiveJob event triggers with SQS.

Using Aurora Serverless

Are you someone with a pet Rails project running on a Free, Hobby, or Professional Heroku plan? Perhaps your company or freelance gig has a valuable, but infrequently used, Rails application? Such applications make great candidates for both AWS Lambda & Aurora Serverless.

This is a simple ActiveRecord Mysql2 adapter extensions to allow Rails to use AWS Aurora Serverless via the Aws::RDSDataService::Client interface. Perfect for infrequently accessed applications.

Using DynamoDB

In some cases Rails with DynamoDB is an excellent choice. Some might say in all cases! If this sounds right for you, I highly recommend using the Aws::Record gem which leverages the aws-sdk-dynamodb in a very Rails like ActiveModel way. Please share your stories with us.

Other Resources

Databases with Lambda is a big fun topic. Here are a few links to help you learn more.

ActiveJob & Lambda →