Grape Gem Tutorial: How To Build A REST-Like API In Ruby

[pt_text]As Ruby On Rails developers, we often need to extend our applications with API endpoints to support JavaScript-heavy Rich Internet clients or native iPhone and Android apps. There are also some cases where the application’s sole purpose is to serve the iPhone/Android apps via a JSON API.
In this tutorial, I demonstrate how to use Grape – a REST-like API micro-framework for Ruby – to build backend support in Rails for a JSON API. Grape is designed to run as a mountable rack engine tocomplement our web applications, without interfering with them.
Web API in Ruby using Grape Gem

Use Case

The use case we will focus on for this tutorial is an application capable of capturing and reviewing pair programming sessions. The application itself will be written for iOS in ObjectiveC and will need to communicate with a backend service for storing and retrieving the data. Our focus in this tutorial is on creation of a robust and secure backend service that supports a JSON API.
The API will support methods for:

  • Logging in to the system
  • Querying pair programming session reviews

NOTE: In addition to providing the ability to query pair programming session reviews, the real API would also need to provide a facility for submitting pair programming reviews for inclusion in the database. Since supporting that via the API is beyond the scope of this tutorial, we will simply assume that the database has been populated with a sample set of pair programming reviews.
Key technical requirements include:

  • Every API call must return valid JSON
  • Every failed API call must be recorded with adequate context and information to subsequently be reproducible, and debugged if necessary

Also, since our application will need to serve external clients, we will need to concern ourselves with security. Toward that end:

  • Each request should be restricted to a small subset of developers we track
  • All requests (other than login/signup) need to be authenticated

Test Driven Development And RSpec

We will use Test Driven Development (TDD) as our software development approach to help ensure the deterministic behavior of our API.
For testing purposes we will use RSpec, a well known testing framework in the RubyOnRails community. I will therefore refer in this article to “specs” rather than “tests”.
A comprehensive testing methodology consists of both “positive” and “negative” tests. Negative specs will specify, for example, how the API endpoint behaves if some parameters are missing or incorrect. Positive specs cover cases where the API is invoked correctly.

Getting Started

Let’s put down the foundation for our backend API. First, we need to create a new rails application:

rails new toptal_grape_blog

Next, we’ll install RSpec by adding rspec-rails into our gemfile:

group :development, :test do
  gem 'rspec-rails', '~> 3.2'
end

Then from our command line we need to run:

rails generate rspec:install

We can also leverage some existing open source software for our testing framework. Specifically:

Step 1: Add these into our gemfile:

...
gem 'devise'
...
group :development, :test do
...
  gem 'factory_girl_rails', '~> 4.5'
...
end

Step 2: Generate a user model, initialize the devise gem, and add it to the user model (this enables the user class to be used for authentication).

rails g model user
rails generate devise:install
rails generate devise user

Step 3: Include the factory_girl syntax method in our rails_helper.rb file in order to use the abbreviated version of user creation in our specs:

RSpec.configure do |config|
  config.include FactoryGirl::Syntax::Methods

Step 4: Add the grape gem to our DSL and install it:

gem 'grape'
bundle

User Login Use Case And Spec

Our backend will need to support a basic login capability. Let’s create the skeleton for our login_spec, assuming that a valid login request consists of a registered email address and password pair:

require 'rails_helper'
describe '/api/login' do
  context 'negative tests' do
    context 'missing params' do
      context 'password' do
      end
      context 'email' do
      end
    end
    context 'invalid params' do
      context 'incorrect password' do
      end
      context 'with a non-existent login' do
      end
    end
  end
  context 'positive tests' do
    context 'valid params' do
    end
  end
end

If either of the parameters is missing, the client should receive an HTTP return status code of 400 (i.e., Bad Request), along with an error message of ‘email is missing’ or ’password is missing’.
For our test, we will create a valid user and set the user’s email and password as original parameters for this test suite. Then we will customize this parameter hash for every specific spec by either omitting the password/email or overriding it.
Let’s create the user and the parameter hash at the beginning of the spec. We will put this code after the describe block:

describe '/api/login' do
  let(:email) { user.email }
  let(:password) { user.password }
  let!(:user) { create :user }
  let(:original_params) { { email: email, password: password } }
  let(:params) { original_params }
  ...

We can then extend our ‘missing params’/’password’ context as follows:

let(:params) { original_params.except(:password) }
it_behaves_like '400'
it_behaves_like 'json result'
it_behaves_like 'contains error msg', 'password is missing'

But instead of repeating the expectations across the ‘email’ and ‘password’ contexts, we can use the same shared examples as expectations. For this, we need to uncomment this line in our rails_helper.rb file:

Dir[Rails.root.join('spec/support/**/*.rb')].each { |f| require f }

We then need to add the 3 RSpec shared examples into spec/support/shared.rb:

RSpec.shared_examples 'json result' do
  specify 'returns JSON' do
    api_call params
    expect { JSON.parse(response.body) }.not_to raise_error
  end
end
RSpec.shared_examples '400' do
  specify 'returns 400' do
    api_call params
    expect(response.status).to eq(400)
  end
end
RSpec.shared_examples 'contains error msg' do |msg|
  specify "error msg is #{msg}" do
    api_call params
    json = JSON.parse(response.body)
    expect(json['error_msg']).to eq(msg)
  end
end

These shared examples are calling the api_call method which enables us to define the API endpoint only once in our spec (in keeping with the DRY principle). We define this method as follows:

describe '/api/login' do
...
  def api_call *params
    post "/api/login", *params
  end
...

We will also need to customize the factory for our user:

FactoryGirl.define do
  factory :user do
    password "Passw0rd"
    password_confirmation { |u| u.password }
    sequence(:email) { |n| "test#{n}@example.com" }
  end
end

And finally, before running our specs we need to run our migrations:

rake db:migrate

Bear in mind, though, that the specs will still fail at this point, since we haven’t yet implemented our API endpoint. That’s next.

Implementing The Login API Endpoint

For starters, we’ll write an empty skeleton for our login API (app/api/login.rb):

class Login < Grape::API
  format :json
  desc 'End-points for the Login'
  namespace :login do
    desc 'Login via email and password'
    params do
      requires :email, type: String, desc: 'email'
      requires :password, type: String, desc: 'password'
    end
    post do
    end
  end
end

Next, we’ll write an aggregator class which aggregates the API endpoints (app/api/api.rb):

class API < Grape::API
  prefix 'api'
  mount Login
end

OK, now we can mount our API in the routes:

Rails.application.routes.draw do
...
  mount API => '/'
...
end

Now let’s add the code to check for the missing parameters. We can add that code to api.rb by rescuing from Grape::Exceptions::ValidationErrors.

rescue_from Grape::Exceptions::ValidationErrors do |e|
  rack_response({
    status: e.status,
    error_msg: e.message,
  }.to_json, 400)
end

For the invalid password, we will check if the http response code is 401 which means unauthorized access. Let’s add this to the ‘incorrect password’ context:

let(:params) { original_params.merge(password: 'invalid') }
it_behaves_like '401'
it_behaves_like 'json result'
it_behaves_like 'contains error msg', 'Bad Authentication Parameters'

The same logic is then added to the ‘with a non-existent login’ context as well.
We then implement the logic which handles the invalid authentication attempts into our login.rb as follows:

post do
  user = User.find_by_email params[:email]
  if user.present? && user.valid_password?(params[:password])
  else
    error_msg = 'Bad Authentication Parameters'
    error!({ 'error_msg' => error_msg }, 401)
  end
end

At this point all of the negative specs for the login api will behave properly, but we still need to support the positive specs for our login api. Our positive spec will expect the endpoint to return an HTTP response code of 200 (i.e., success) with valid JSON and a valid token:

it_behaves_like '200'
it_behaves_like 'json result'
specify 'returns the token as part of the response' do
  api_call params
  expect(JSON.parse(response.body)['token']).to be_present
end

Let’s also add the expectation for the response code 200 to spec/support/shared.rb:

RSpec.shared_examples '200' do
  specify 'returns 200' do
    api_call params
    expect(response.status).to eq(200)
  end
end

In case of successful login we are going to return the first valid authentication_token together with the user’s email in this format:

{‘email’:<the_email_of_the_user>, ‘token’:<the users first valid token>}

If there is no such token yet then we will create one for the current user:

...
if user.present? && user.valid_password?(params[:password])
  token = user.authentication_tokens.valid.first || AuthenticationToken.generate(user)
  status 200
else
...

In order for this to work, we will need an AuthenticationToken class which belongs to the user. We will generate this model, then run the corresponding migration:

rails g model authentication_token token user:references expires_at:datetime
rake db:migrate

We also need to add the corresponding association to our user model:

class User < ActiveRecord::Base
  has_many :authentication_tokens
end

Then we will add the valid scope to the AuthenticationToken class:

class AuthenticationToken < ActiveRecord::Base
  belongs_to :user
  validates :token, presence: true
  scope :valid,  -> { where{ (expires_at == nil) | (expires_at > Time.zone.now) } }
end

Note that we used Ruby syntax in the where statement. This is enabled by our use of the squeel gem which enables support for Ruby syntax in activerecord queries.
For a validated user, we are going to create an entity that we’ll refer to as the “user with token entity”, leveraging the features of the grape-entity gem.
Let’s write the spec for our entity and put it in the user_with_token_entity_spec.rb file:

require 'rails_helper'
describe Entities::UserWithTokenEntity do
  describe 'fields' do
    subject(:subject) { Entities::UserWithTokenEntity }
    specify { expect(subject).to represent(:email)}
    let!(:token) { create :authentication_token }
    specify 'presents the first available token' do
      json = Entities::UserWithTokenEntity.new(token.user).as_json
      expect(json[:token]).to be_present
    end
  end
end

Next let’s add the entities to user_entity.rb:

module Entities
  class UserEntity < Grape::Entity
    expose :email
  end
end

And finally, add another class to user_with_token_entity.rb:

module Entities
  class UserWithTokenEntity < UserEntity
    expose :token do |user, options|
      user.authentication_tokens.valid.first.token
    end
  end
end

Since we don’t want tokens to remain valid indefinitely, we have them expire after one day:

FactoryGirl.define do
  factory :authentication_token do
    token "MyString"
    expires_at 1.day.from_now
    user
  end
end

With this all done, we can now return the expected JSON format with our newly written UserWithTokenEntity:

...
user = User.find_by_email params[:email]
if user.present? && user.valid_password?(params[:password])
  token = user.authentication_tokens.valid.first || AuthenticationToken.generate(user)
  status 200
  present token.user, with: Entities::UserWithTokenEntity
else
...

Cool. Now all of our specs are passing and the functional requirements of the basic login api endpoint are supported.

Pair Programming Session Review API Endpoint: Getting Started

Our backend will need to allow authorized developers who have logged in to query pair programming session reviews.
Our new API endpoint will be mounted to /api/pair_programming_session and will return the reviews belonging to a project. Let’s begin by writing a basic skeleton for this spec:

require 'rails_helper'
describe '/api' do
  describe '/pair_programming_session' do
    def api_call *params
      get '/api/pair_programming_sessions', *params
    end
    context 'invalid params' do
    end
    context 'valid params' do
    end
  end
end

We will write a corresponding empty API endpoint (app/api/pair_programming_sessions.rb) as well:

class PairProgrammingSessions < Grape::API
  format :json
  desc 'End-points for the PairProgrammingSessions'
  namespace :pair_programming_sessions do
    desc 'Retrieve the pairprogramming sessions'
    params do
      requires :token, type: String, desc: 'email'
    end
    get do
    end
  end
end

Then let’s mount our new api (app/api/api.rb):

...
  mount Login
  mount PairProgrammingSessions
end

Let’s expand the spec, and the API endpoint, against the requirements one by one.

Pair Programming Session Review API Endpoint: Validation

One of our most important non-functional security requirements was to restrict API access to a small subset of developers we track, so let’s specify that:

...
def api_call *params
  get '/api/pair_programming_sessions', *params
end
let(:token) { create :authentication_token }
let(:original_params) { { token: token.token} }
let(:params) { original_params }
it_behaves_like 'restricted for developers'
context 'invalid params' do
...

Then we will create a shared_example in our shared.rb to confirm that the request is coming from one of our registered developers:

RSpec.shared_examples 'restricted for developers' do
  context 'without developer key' do
    specify 'should be an unauthorized call' do
      api_call params
      expect(response.status).to eq(401)
    end
    specify 'error code is 1001' do
      api_call params
      json = JSON.parse(response.body)
      expect(json['error_code']).to eq(ErrorCodes::DEVELOPER_KEY_MISSING)
    end
  end
end

We will also need to create an ErrorCodes class (in app/models/error_codes.rb):

module ErrorCodes
  DEVELOPER_KEY_MISSING = 1001
end

Since we expect our API to expand in the future, we are going to implement an authorization_helper which can be reused across all API endpoints in the application to restrict access to registered developers only:

class PairProgrammingSessions < Grape::API
  helpers ApiHelpers::AuthenticationHelper
  before { restrict_access_to_developers }

We are going to define the method restrict_access_to_developers in the ApiHelpers::AuthenticationHerlpermodule (app/api/api_helpers/authentication_helper.rb). This method will simply check if the key Authorization under the headers contains a valid ApiKey. (Every developer wanting access to the API will require a valid ApiKey. This could either be provided by a system administrator or via some automated registration process, but that mechanism is beyond the scope of this article.)

module ApiHelpers
  module AuthenticationHelper
    def restrict_access_to_developers
      header_token = headers['Authorization']
      key = ApiKey.where{ token == my{ header_token } }
      Rails.logger.info "API call: #{headers}\tWith params: #{params.inspect}" if ENV['DEBUG']
      if key.blank?
        error_code = ErrorCodes::DEVELOPER_KEY_MISSING
        error_msg = 'please aquire a developer key'
        error!({ :error_msg => error_msg, :error_code => error_code }, 401)
        # LogAudit.new({env:env}).execute
      end
    end
  end
end

We then need to generate the ApiKey model and run the migrations: rails g model api_key token rake db:migrate
With this done, in our spec/api/pair_programming_spec.rb we can check if the user is authenticated:

...
it_behaves_like 'restricted for developers'
it_behaves_like 'unauthenticated'
...

Let’s also define an unauthenticated shared example which can be reused across all specs (spec/support/shared.rb):

RSpec.shared_examples 'unauthenticated' do
  context 'unauthenticated' do
    specify 'returns 401 without token' do
      api_call params.except(:token), developer_header
      expect(response.status).to eq(401)
    end
    specify 'returns JSON' do
      api_call params.except(:token), developer_header
      json = JSON.parse(response.body)
    end
  end
end

This shared example needs the token in the developer header so let’s add that to our spec (spec/api/pair_programming_spec.rb):

...
describe '/api' do
  let(:api_key) { create :api_key }
  let(:developer_header) { {'Authorization' => api_key.token} }
...

Now, in our app/api/pair_programming_session.rb, let’s attempt to authenticate the user:

...
class PairProgrammingSessions < Grape::API
  helpers ApiHelpers::AuthenticationHelper
  before { restrict_access_to_developers }
  before { authenticate! }
...

Let’s implement the authenticate! method in the AuthenticationHelper (app/api/api_helpers/authentication_helper.rb):

...
module ApiHelpers
  module AuthenticationHelper
    TOKEN_PARAM_NAME = :token
    def token_value_from_request(token_param = TOKEN_PARAM_NAME)
      params[token_param]
    end
    def current_user
      token = AuthenticationToken.find_by_token(token_value_from_request)
      return nil unless token.present?
      @current_user ||= token.user
    end
    def signed_in?
      !!current_user
    end
    def authenticate!
      unless signed_in?
        AuditLog.create data: 'unauthenticated user access'
        error!({ :error_msg => "authentication_error", :error_code => ErrorCodes::BAD_AUTHENTICATION_PARAMS }, 401)
      end
    end
...

(Note that we need to add the error code BAD_AUTHENTICATION_PARAMS to our ErrorCodes class.)
Next, let’s spec what happens if the developer calls the API with an invalid token. In that case the return code will be 401 signaling an ‘unauthorized access’. The result should be JSON and an auditable should be created. So we add this to spec/api/pair_programming_spec.rb:

...
context 'invalid params' do
  context 'incorrect token' do
    let(:params) { original_params.merge(token: 'invalid') }
    it_behaves_like '401'
    it_behaves_like 'json result'
    it_behaves_like 'auditable created'
    it_behaves_like 'contains error msg', 'authentication_error'
    it_behaves_like 'contains error code', ErrorCodes::BAD_AUTHENTICATION_PARAMS
  end
end
...

We will add the “auditable created”, “contains error code”, and “contains error msg” shared examples to spec/support/shared.rb:

...
RSpec.shared_examples 'contains error code' do |code|
  specify "error code is #{code}" do
    api_call params, developer_header
    json = JSON.parse(response.body)
    expect(json['error_code']).to eq(code)
  end
end
RSpec.shared_examples 'contains error msg' do |msg|
  specify "error msg is #{msg}" do
    api_call params, developer_header
    json = JSON.parse(response.body)
    expect(json['error_msg']).to eq(msg)
  end
end
RSpec.shared_examples 'auditable created' do
  specify 'creates an api call audit' do
    expect do
      api_call params, developer_header
    end.to change{ AuditLog.count }.by(1)
  end
end
...

We also need to create an audit_log model:

rails g model audit_log backtrace data user:references
rake db:migrate

Pair Programming Session Review API Endpoint: Returning Results

For an authenticated and authorized user, a call to this API endpoint should return a set of pair programming session reviews grouped by project. Let’s modify our spec/api/pair_programming_spec.rb accordingly:

...
context 'valid params' do
it_behaves_like '200'
it_behaves_like 'json result'
end
...

This specifies that a request submitted with valid api_key and valid parameters returns an HTTP code of 200 (i.e., success) and that the result is returned in the form of valid JSON.
We are going to query and then return in JSON format those pair programming sessions where any of the participant is the current user (app/api/pair_programming_sessions.rb):

...
get do
sessions = PairProgrammingSession.where{(host_user == my{current_user}) | (visitor_user == my{current_user})}
sessions = sessions.includes(:project, :host_user, :visitor_user, reviews: [:code_samples, :user] )
present sessions, with: Entities::PairProgrammingSessionsEntity
end
...

The pair programming sessions are modeled as follows in the database:

  • 1-to-many relationship between projects and pair programming sessions
  • 1-to-many relationship between pair programming sessions and reviews
  • 1-to-many relationship between reviews and code samples

Let’s generate the models accordingly and then run the migrations:

rails g model project name
rails g model pair_programming_session project:references host_user:references visitor_user:references
rails g model review pair_programming_session:references user:references comment
rails g model code_sample review:references code:text
rake db:migrate

Then we need to modify our PairProgrammingSession and Review classes to contain the has_manyassociations:

class Review < ActiveRecord::Base
  belongs_to :pair_programming_session
  belongs_to :user
  has_many :code_samples
end
class PairProgrammingSession < ActiveRecord::Base
  belongs_to :project
  belongs_to :host_user, class_name: :User
  belongs_to :visitor_user, class_name: 'User'
  has_many :reviews
end

NOTE: In normal circumstances, I would generate these classes by writing specs for them first ,but since that is beyond the scope of this article, I will skip that step.
Now we need to write those classes which are going to transform our models to their JSON representations (referred to as grape-entities in grape terminology). For simplicity, we will use 1-to-1 mapping between the models and grape-entities.
We begin by exposing the code field from the CodeSampleEntity (in api/entities/code_sample_entity.rb):

module Entities
  class CodeSampleEntity < Grape::Entity
    expose :code
  end
end

Then we expose the user and associated code_samples by reusing the already defined UserEntity and the CodeSampleEntity:

module Entities
  class ReviewEntity < Grape::Entity
    expose :user, using: UserEntity
    expose :code_samples, using: CodeSampleEntity
  end
end

We also expose the name field from the ProjectEntity:

module Entities
  class ProjectEntity < Grape::Entity
    expose :name
  end
end

Finally, we assemble the entity into a new PairProgrammingSessionsEntity where we expose the project, the host_user, the visitor_user and the reviews:

module Entities
  class PairProgrammingSessionsEntity < Grape::Entity
    expose :project, using: ProjectEntity
    expose :host_user, using: UserEntity
    expose :visitor_user, using: UserEntity
    expose :reviews, using: ReviewEntity
  end
end

And with that, our API is fully implemented!

Generating Test Data

For testing purposes, we’ll create some sample data in db/seeds.rb. This file should contain all the record creation needed to seed the database with its default values. The data can then be loaded with rake db:seed(or created with the db when db:setup is invoked). Here’s an example of what this could include:

user_1 = User.create email: '[email protected]', password: 'password', password_confirmation: 'password'
user_2 = User.create email: '[email protected]', password: 'password', password_confirmation: 'password'
user_3 = User.create email: '[email protected]', password: 'password', password_confirmation: 'password'
ApiKey.create token: '12345654321'
project_1 = Project.create name: 'Time Sheets'
project_2 = Project.create name: 'Toptal Blog'
project_3 = Project.create name: 'Hobby Project'
session_1 = PairProgrammingSession.create project: project_1, host_user: user_1, visitor_user: user_2
session_2 = PairProgrammingSession.create project: project_2, host_user: user_1, visitor_user: user_3
session_3 = PairProgrammingSession.create project: project_3, host_user: user_2, visitor_user: user_3
review_1 = session_1.reviews.create user: user_1, comment: 'Please DRY a bit your code'
review_2 = session_1.reviews.create user: user_1, comment: 'Please DRY a bit your specs'
review_3 = session_2.reviews.create user: user_1, comment: 'Please DRY your view templates'
review_4 = session_2.reviews.create user: user_1, comment: 'Please clean your N+1 queries'
review_1.code_samples.create code: 'Lorem Ipsum'
review_1.code_samples.create code: 'Do not abuse the single responsibility principle'
review_2.code_samples.create code: 'Use some shared examples'
review_2.code_samples.create code: 'Use at the beginning of specs'

Now our application is ready for use and we can launch our rails server.

Testing the API

We will use Swagger to do some manual browser-based testing of our API. A few setup steps are required for us to be able to make use of Swagger though.
First, we need to add a couple of gems to our gemfile:

...
gem 'grape-swagger'
gem 'grape-swagger-ui'
...

We then run bundle to install these gems.
We also need to add these to assets to our assets pipeline (in config/initializers/assets.rb):

Rails.application.config.assets.precompile += %w( swagger_ui.js )
Rails.application.config.assets.precompile += %w( swagger_ui.css )

Finally, in app/api/api.rb we need to mount the swagger generator:

...
  add_swagger_documentation
end
...

Now we can take advantage of Swagger’s nice UI to explore our API by simply going to https://localhost:3000/api/swagger.
Swagger presents our API endpoints in a nicely explorable way. If we click on an endpoint, Swagger lists its operations. If we click on an operation, Swagger shows its required and optional parameters and their data types.
One remaining detail though before we proceed: Since we restricted use of the API developers with a valid api_key, we won’t be able to access the API endpoint directly from the browser because the server will require a valid api_key in the HTTP header. We can accomplish this for testing purposes in Google Chrome by making use of the Modify Headers for Google Chrome plugin. This plugin will enable us to edit the HTTP header and add in a valid api_key (we’ll use the dummy api_key of 12345654321 that we included in our database seed file).
OK, now we’re ready to test!
In order to call the pair_programming_sessions API endpoint, we first need to log in. We’ll just use of the email and password combinations from our database seed file and submit it via Swagger to the login API endpoint, as shown below.

As you can see above, the token belonging to that user is returned, indicating that the login API is working as functioning properly as intended. We can now use that token to successfully perform the GET /api/pair_programming_sessions.json operation.

As shown, the result is returned as a properly-formatted hierarchical JSON object. Notice that the JSON structure reflects two nested 1-to-many associations, since the project has multiple reviews, and a review has multiple code samples. If we wouldn’t return the structure in this way, then the caller of our API would need to separately request the reviews for each project which would require submitting N queries to our API endpoint. With this structure, we therefore solve the N+1 query performance issue.

Wrap-up

As shown herein, comprehensive specs for your API help ensure that the implemented API properly and adequately addresses the intended (and unintended!) use cases.
While the example API presented in this tutorial is fairly basic, the approach and techniques we have demonstrated can serve as the foundation for more sophisticated APIs of arbitrary complexity using the Grapegem. This tutorial has hopefully demonstrated that Grape is a useful and flexible gem that can help facilitate implementation of a JSON API in your Rails applications. Enjoy!

[/pt_text]

[pt_text]
[/pt_text][pt_testimonials_balloon name=”About this post” quote=”This is a shared post. Originally posted on “https://www.toptal.com/“. All rights reserved to their editors. I just shared this post on Original Author’s demand to share this post on my blog.” minheight=”170″]

Related Posts