In this tutorial, I demonstrate how to use Grape – a REST-like API micro-framework for Ruby – to build backend support in Rails for a JSON API. Grape is designed to run as a mountable rack engine tocomplement our web applications, without interfering with them.
Use Case
The use case we will focus on for this tutorial is an application capable of capturing and reviewing pair programming sessions. The application itself will be written for iOS in ObjectiveC and will need to communicate with a backend service for storing and retrieving the data. Our focus in this tutorial is on creation of a robust and secure backend service that supports a JSON API.
The API will support methods for:
- Logging in to the system
- Querying pair programming session reviews
NOTE: In addition to providing the ability to query pair programming session reviews, the real API would also need to provide a facility for submitting pair programming reviews for inclusion in the database. Since supporting that via the API is beyond the scope of this tutorial, we will simply assume that the database has been populated with a sample set of pair programming reviews.
Key technical requirements include:
- Every API call must return valid JSON
- Every failed API call must be recorded with adequate context and information to subsequently be reproducible, and debugged if necessary
Also, since our application will need to serve external clients, we will need to concern ourselves with security. Toward that end:
- Each request should be restricted to a small subset of developers we track
- All requests (other than login/signup) need to be authenticated
Test Driven Development And RSpec
We will use Test Driven Development (TDD) as our software development approach to help ensure the deterministic behavior of our API.
For testing purposes we will use RSpec, a well known testing framework in the RubyOnRails community. I will therefore refer in this article to “specs” rather than “tests”.
A comprehensive testing methodology consists of both “positive” and “negative” tests. Negative specs will specify, for example, how the API endpoint behaves if some parameters are missing or incorrect. Positive specs cover cases where the API is invoked correctly.
Getting Started
Let’s put down the foundation for our backend API. First, we need to create a new rails application:
rails new toptal_grape_blog
Next, we’ll install RSpec by adding rspec-rails into our gemfile:
group :development, :test do
gem 'rspec-rails', '~> 3.2'
end
Then from our command line we need to run:
rails generate rspec:install
We can also leverage some existing open source software for our testing framework. Specifically:
- Devise – a flexible authentication solution for Rails based on Warden
- factory_girl_rails – provides Rails integration for factory_girl, a library for setting up Ruby objects as test data
Step 1: Add these into our gemfile:
...
gem 'devise'
...
group :development, :test do
...
gem 'factory_girl_rails', '~> 4.5'
...
end
Step 2: Generate a user model, initialize the devise gem, and add it to the user model (this enables the user class to be used for authentication).
rails g model user
rails generate devise:install
rails generate devise user
Step 3: Include the factory_girl syntax method in our rails_helper.rb file in order to use the abbreviated version of user creation in our specs:
RSpec.configure do |config|
config.include FactoryGirl::Syntax::Methods
Step 4: Add the grape gem to our DSL and install it:
gem 'grape'
bundle
User Login Use Case And Spec
Our backend will need to support a basic login capability. Let’s create the skeleton for our login_spec, assuming that a valid login request consists of a registered email address and password pair:
require 'rails_helper'
describe '/api/login' do
context 'negative tests' do
context 'missing params' do
context 'password' do
end
context 'email' do
end
end
context 'invalid params' do
context 'incorrect password' do
end
context 'with a non-existent login' do
end
end
end
context 'positive tests' do
context 'valid params' do
end
end
end
If either of the parameters is missing, the client should receive an HTTP return status code of 400 (i.e., Bad Request), along with an error message of ‘email is missing’ or ’password is missing’.
For our test, we will create a valid user and set the user’s email and password as original parameters for this test suite. Then we will customize this parameter hash for every specific spec by either omitting the password/email or overriding it.
Let’s create the user and the parameter hash at the beginning of the spec. We will put this code after the describe block:
describe '/api/login' do
let(:email) { user.email }
let(:password) { user.password }
let!(:user) { create :user }
let(:original_params) { { email: email, password: password } }
let(:params) { original_params }
...
We can then extend our ‘missing params’/’password’ context as follows:
let(:params) { original_params.except(:password) }
it_behaves_like '400'
it_behaves_like 'json result'
it_behaves_like 'contains error msg', 'password is missing'
But instead of repeating the expectations across the ‘email’ and ‘password’ contexts, we can use the same shared examples as expectations. For this, we need to uncomment this line in our rails_helper.rb file:
Dir[Rails.root.join('spec/support/**/*.rb')].each { |f| require f }
We then need to add the 3 RSpec shared examples into spec/support/shared.rb:
RSpec.shared_examples 'json result' do
specify 'returns JSON' do
api_call params
expect { JSON.parse(response.body) }.not_to raise_error
end
end
RSpec.shared_examples '400' do
specify 'returns 400' do
api_call params
expect(response.status).to eq(400)
end
end
RSpec.shared_examples 'contains error msg' do |msg|
specify "error msg is #{msg}" do
api_call params
json = JSON.parse(response.body)
expect(json['error_msg']).to eq(msg)
end
end
These shared examples are calling the api_call method which enables us to define the API endpoint only once in our spec (in keeping with the DRY principle). We define this method as follows:
describe '/api/login' do
...
def api_call *params
post "/api/login", *params
end
...
We will also need to customize the factory for our user:
FactoryGirl.define do
factory :user do
password "Passw0rd"
password_confirmation { |u| u.password }
sequence(:email) { |n| "test#{n}@example.com" }
end
end
And finally, before running our specs we need to run our migrations:
rake db:migrate
Bear in mind, though, that the specs will still fail at this point, since we haven’t yet implemented our API endpoint. That’s next.
Implementing The Login API Endpoint
For starters, we’ll write an empty skeleton for our login API (app/api/login.rb):
class Login < Grape::API
format :json
desc 'End-points for the Login'
namespace :login do
desc 'Login via email and password'
params do
requires :email, type: String, desc: 'email'
requires :password, type: String, desc: 'password'
end
post do
end
end
end
Next, we’ll write an aggregator class which aggregates the API endpoints (app/api/api.rb):
class API < Grape::API
prefix 'api'
mount Login
end
OK, now we can mount our API in the routes:
Rails.application.routes.draw do
...
mount API => '/'
...
end
Now let’s add the code to check for the missing parameters. We can add that code to api.rb by rescuing from Grape::Exceptions::ValidationErrors.
rescue_from Grape::Exceptions::ValidationErrors do |e|
rack_response({
status: e.status,
error_msg: e.message,
}.to_json, 400)
end
For the invalid password, we will check if the http response code is 401 which means unauthorized access. Let’s add this to the ‘incorrect password’ context:
let(:params) { original_params.merge(password: 'invalid') }
it_behaves_like '401'
it_behaves_like 'json result'
it_behaves_like 'contains error msg', 'Bad Authentication Parameters'
The same logic is then added to the ‘with a non-existent login’ context as well.
We then implement the logic which handles the invalid authentication attempts into our login.rb as follows:
post do
user = User.find_by_email params[:email]
if user.present? && user.valid_password?(params[:password])
else
error_msg = 'Bad Authentication Parameters'
error!({ 'error_msg' => error_msg }, 401)
end
end
At this point all of the negative specs for the login api will behave properly, but we still need to support the positive specs for our login api. Our positive spec will expect the endpoint to return an HTTP response code of 200 (i.e., success) with valid JSON and a valid token:
it_behaves_like '200'
it_behaves_like 'json result'
specify 'returns the token as part of the response' do
api_call params
expect(JSON.parse(response.body)['token']).to be_present
end
Let’s also add the expectation for the response code 200 to spec/support/shared.rb:
RSpec.shared_examples '200' do
specify 'returns 200' do
api_call params
expect(response.status).to eq(200)
end
end
In case of successful login we are going to return the first valid authentication_token together with the user’s email in this format:
{‘email’:<the_email_of_the_user>, ‘token’:<the users first valid token>}
If there is no such token yet then we will create one for the current user:
...
if user.present? && user.valid_password?(params[:password])
token = user.authentication_tokens.valid.first || AuthenticationToken.generate(user)
status 200
else
...
In order for this to work, we will need an AuthenticationToken class which belongs to the user. We will generate this model, then run the corresponding migration:
rails g model authentication_token token user:references expires_at:datetime
rake db:migrate
We also need to add the corresponding association to our user model:
class User < ActiveRecord::Base
has_many :authentication_tokens
end
Then we will add the valid scope to the AuthenticationToken class:
class AuthenticationToken < ActiveRecord::Base
belongs_to :user
validates :token, presence: true
scope :valid, -> { where{ (expires_at == nil) | (expires_at > Time.zone.now) } }
end
Note that we used Ruby syntax in the where statement. This is enabled by our use of the squeel gem which enables support for Ruby syntax in activerecord queries.
For a validated user, we are going to create an entity that we’ll refer to as the “user with token entity”, leveraging the features of the grape-entity gem.
Let’s write the spec for our entity and put it in the user_with_token_entity_spec.rb file:
require 'rails_helper'
describe Entities::UserWithTokenEntity do
describe 'fields' do
subject(:subject) { Entities::UserWithTokenEntity }
specify { expect(subject).to represent(:email)}
let!(:token) { create :authentication_token }
specify 'presents the first available token' do
json = Entities::UserWithTokenEntity.new(token.user).as_json
expect(json[:token]).to be_present
end
end
end
Next let’s add the entities to user_entity.rb:
module Entities
class UserEntity < Grape::Entity
expose :email
end
end
And finally, add another class to user_with_token_entity.rb:
module Entities
class UserWithTokenEntity < UserEntity
expose :token do |user, options|
user.authentication_tokens.valid.first.token
end
end
end
Since we don’t want tokens to remain valid indefinitely, we have them expire after one day:
FactoryGirl.define do
factory :authentication_token do
token "MyString"
expires_at 1.day.from_now
user
end
end
With this all done, we can now return the expected JSON format with our newly written UserWithTokenEntity:
...
user = User.find_by_email params[:email]
if user.present? && user.valid_password?(params[:password])
token = user.authentication_tokens.valid.first || AuthenticationToken.generate(user)
status 200
present token.user, with: Entities::UserWithTokenEntity
else
...
Cool. Now all of our specs are passing and the functional requirements of the basic login api endpoint are supported.
Pair Programming Session Review API Endpoint: Getting Started
Our backend will need to allow authorized developers who have logged in to query pair programming session reviews.
Our new API endpoint will be mounted to /api/pair_programming_session and will return the reviews belonging to a project. Let’s begin by writing a basic skeleton for this spec:
require 'rails_helper'
describe '/api' do
describe '/pair_programming_session' do
def api_call *params
get '/api/pair_programming_sessions', *params
end
context 'invalid params' do
end
context 'valid params' do
end
end
end
We will write a corresponding empty API endpoint (app/api/pair_programming_sessions.rb) as well:
class PairProgrammingSessions < Grape::API
format :json
desc 'End-points for the PairProgrammingSessions'
namespace :pair_programming_sessions do
desc 'Retrieve the pairprogramming sessions'
params do
requires :token, type: String, desc: 'email'
end
get do
end
end
end
Then let’s mount our new api (app/api/api.rb):
...
mount Login
mount PairProgrammingSessions
end
Let’s expand the spec, and the API endpoint, against the requirements one by one.
Pair Programming Session Review API Endpoint: Validation
One of our most important non-functional security requirements was to restrict API access to a small subset of developers we track, so let’s specify that:
...
def api_call *params
get '/api/pair_programming_sessions', *params
end
let(:token) { create :authentication_token }
let(:original_params) { { token: token.token} }
let(:params) { original_params }
it_behaves_like 'restricted for developers'
context 'invalid params' do
...
Then we will create a shared_example in our shared.rb to confirm that the request is coming from one of our registered developers:
RSpec.shared_examples 'restricted for developers' do
context 'without developer key' do
specify 'should be an unauthorized call' do
api_call params
expect(response.status).to eq(401)
end
specify 'error code is 1001' do
api_call params
json = JSON.parse(response.body)
expect(json['error_code']).to eq(ErrorCodes::DEVELOPER_KEY_MISSING)
end
end
end
We will also need to create an ErrorCodes class (in app/models/error_codes.rb):
module ErrorCodes
DEVELOPER_KEY_MISSING = 1001
end
Since we expect our API to expand in the future, we are going to implement an authorization_helper which can be reused across all API endpoints in the application to restrict access to registered developers only:
class PairProgrammingSessions < Grape::API
helpers ApiHelpers::AuthenticationHelper
before { restrict_access_to_developers }
We are going to define the method restrict_access_to_developers in the ApiHelpers::AuthenticationHerlpermodule (app/api/api_helpers/authentication_helper.rb). This method will simply check if the key Authorization under the headers contains a valid ApiKey. (Every developer wanting access to the API will require a valid ApiKey. This could either be provided by a system administrator or via some automated registration process, but that mechanism is beyond the scope of this article.)
module ApiHelpers
module AuthenticationHelper
def restrict_access_to_developers
header_token = headers['Authorization']
key = ApiKey.where{ token == my{ header_token } }
Rails.logger.info "API call: #{headers}\tWith params: #{params.inspect}" if ENV['DEBUG']
if key.blank?
error_code = ErrorCodes::DEVELOPER_KEY_MISSING
error_msg = 'please aquire a developer key'
error!({ :error_msg => error_msg, :error_code => error_code }, 401)
# LogAudit.new({env:env}).execute
end
end
end
end
We then need to generate the ApiKey model and run the migrations: rails g model api_key token rake db:migrate
With this done, in our spec/api/pair_programming_spec.rb we can check if the user is authenticated:
...
it_behaves_like 'restricted for developers'
it_behaves_like 'unauthenticated'
...
Let’s also define an unauthenticated shared example which can be reused across all specs (spec/support/shared.rb):
RSpec.shared_examples 'unauthenticated' do
context 'unauthenticated' do
specify 'returns 401 without token' do
api_call params.except(:token), developer_header
expect(response.status).to eq(401)
end
specify 'returns JSON' do
api_call params.except(:token), developer_header
json = JSON.parse(response.body)
end
end
end
This shared example needs the token in the developer header so let’s add that to our spec (spec/api/pair_programming_spec.rb):
...
describe '/api' do
let(:api_key) { create :api_key }
let(:developer_header) { {'Authorization' => api_key.token} }
...
Now, in our app/api/pair_programming_session.rb, let’s attempt to authenticate the user:
...
class PairProgrammingSessions < Grape::API
helpers ApiHelpers::AuthenticationHelper
before { restrict_access_to_developers }
before { authenticate! }
...
Let’s implement the authenticate! method in the AuthenticationHelper (app/api/api_helpers/authentication_helper.rb):
...
module ApiHelpers
module AuthenticationHelper
TOKEN_PARAM_NAME = :token
def token_value_from_request(token_param = TOKEN_PARAM_NAME)
params[token_param]
end
def current_user
token = AuthenticationToken.find_by_token(token_value_from_request)
return nil unless token.present?
@current_user ||= token.user
end
def signed_in?
!!current_user
end
def authenticate!
unless signed_in?
AuditLog.create data: 'unauthenticated user access'
error!({ :error_msg => "authentication_error", :error_code => ErrorCodes::BAD_AUTHENTICATION_PARAMS }, 401)
end
end
...
(Note that we need to add the error code BAD_AUTHENTICATION_PARAMS to our ErrorCodes class.)
Next, let’s spec what happens if the developer calls the API with an invalid token. In that case the return code will be 401 signaling an ‘unauthorized access’. The result should be JSON and an auditable should be created. So we add this to spec/api/pair_programming_spec.rb:
...
context 'invalid params' do
context 'incorrect token' do
let(:params) { original_params.merge(token: 'invalid') }
it_behaves_like '401'
it_behaves_like 'json result'
it_behaves_like 'auditable created'
it_behaves_like 'contains error msg', 'authentication_error'
it_behaves_like 'contains error code', ErrorCodes::BAD_AUTHENTICATION_PARAMS
end
end
...
We will add the “auditable created”, “contains error code”, and “contains error msg” shared examples to spec/support/shared.rb:
...
RSpec.shared_examples 'contains error code' do |code|
specify "error code is #{code}" do
api_call params, developer_header
json = JSON.parse(response.body)
expect(json['error_code']).to eq(code)
end
end
RSpec.shared_examples 'contains error msg' do |msg|
specify "error msg is #{msg}" do
api_call params, developer_header
json = JSON.parse(response.body)
expect(json['error_msg']).to eq(msg)
end
end
RSpec.shared_examples 'auditable created' do
specify 'creates an api call audit' do
expect do
api_call params, developer_header
end.to change{ AuditLog.count }.by(1)
end
end
...
We also need to create an audit_log model:
rails g model audit_log backtrace data user:references
rake db:migrate
Pair Programming Session Review API Endpoint: Returning Results
For an authenticated and authorized user, a call to this API endpoint should return a set of pair programming session reviews grouped by project. Let’s modify our spec/api/pair_programming_spec.rb accordingly:
...
context 'valid params' do
it_behaves_like '200'
it_behaves_like 'json result'
end
...
This specifies that a request submitted with valid api_key and valid parameters returns an HTTP code of 200 (i.e., success) and that the result is returned in the form of valid JSON.
We are going to query and then return in JSON format those pair programming sessions where any of the participant is the current user (app/api/pair_programming_sessions.rb):
...
get do
sessions = PairProgrammingSession.where{(host_user == my{current_user}) | (visitor_user == my{current_user})}
sessions = sessions.includes(:project, :host_user, :visitor_user, reviews: [:code_samples, :user] )
present sessions, with: Entities::PairProgrammingSessionsEntity
end
...
The pair programming sessions are modeled as follows in the database:
- 1-to-many relationship between projects and pair programming sessions
- 1-to-many relationship between pair programming sessions and reviews
- 1-to-many relationship between reviews and code samples
Let’s generate the models accordingly and then run the migrations:
rails g model project name
rails g model pair_programming_session project:references host_user:references visitor_user:references
rails g model review pair_programming_session:references user:references comment
rails g model code_sample review:references code:text
rake db:migrate
Then we need to modify our PairProgrammingSession and Review classes to contain the has_manyassociations:
class Review < ActiveRecord::Base
belongs_to :pair_programming_session
belongs_to :user
has_many :code_samples
end
class PairProgrammingSession < ActiveRecord::Base
belongs_to :project
belongs_to :host_user, class_name: :User
belongs_to :visitor_user, class_name: 'User'
has_many :reviews
end
NOTE: In normal circumstances, I would generate these classes by writing specs for them first ,but since that is beyond the scope of this article, I will skip that step.
Now we need to write those classes which are going to transform our models to their JSON representations (referred to as grape-entities in grape terminology). For simplicity, we will use 1-to-1 mapping between the models and grape-entities.
We begin by exposing the code field from the CodeSampleEntity (in api/entities/code_sample_entity.rb):
module Entities
class CodeSampleEntity < Grape::Entity
expose :code
end
end
Then we expose the user and associated code_samples by reusing the already defined UserEntity and the CodeSampleEntity:
module Entities
class ReviewEntity < Grape::Entity
expose :user, using: UserEntity
expose :code_samples, using: CodeSampleEntity
end
end
We also expose the name field from the ProjectEntity:
module Entities
class ProjectEntity < Grape::Entity
expose :name
end
end
Finally, we assemble the entity into a new PairProgrammingSessionsEntity where we expose the project, the host_user, the visitor_user and the reviews:
module Entities
class PairProgrammingSessionsEntity < Grape::Entity
expose :project, using: ProjectEntity
expose :host_user, using: UserEntity
expose :visitor_user, using: UserEntity
expose :reviews, using: ReviewEntity
end
end
And with that, our API is fully implemented!
Generating Test Data
For testing purposes, we’ll create some sample data in db/seeds.rb. This file should contain all the record creation needed to seed the database with its default values. The data can then be loaded with rake db:seed(or created with the db when db:setup is invoked). Here’s an example of what this could include:
user_1 = User.create email: 'railssuperhero@email.com', password: 'password', password_confirmation: 'password'
user_2 = User.create email: 'railshero@email.com', password: 'password', password_confirmation: 'password'
user_3 = User.create email: 'railsrookie@email.com', password: 'password', password_confirmation: 'password'
ApiKey.create token: '12345654321'
project_1 = Project.create name: 'Time Sheets'
project_2 = Project.create name: 'Toptal Blog'
project_3 = Project.create name: 'Hobby Project'
session_1 = PairProgrammingSession.create project: project_1, host_user: user_1, visitor_user: user_2
session_2 = PairProgrammingSession.create project: project_2, host_user: user_1, visitor_user: user_3
session_3 = PairProgrammingSession.create project: project_3, host_user: user_2, visitor_user: user_3
review_1 = session_1.reviews.create user: user_1, comment: 'Please DRY a bit your code'
review_2 = session_1.reviews.create user: user_1, comment: 'Please DRY a bit your specs'
review_3 = session_2.reviews.create user: user_1, comment: 'Please DRY your view templates'
review_4 = session_2.reviews.create user: user_1, comment: 'Please clean your N+1 queries'
review_1.code_samples.create code: 'Lorem Ipsum'
review_1.code_samples.create code: 'Do not abuse the single responsibility principle'
review_2.code_samples.create code: 'Use some shared examples'
review_2.code_samples.create code: 'Use at the beginning of specs'
Now our application is ready for use and we can launch our rails server.
Testing the API
We will use Swagger to do some manual browser-based testing of our API. A few setup steps are required for us to be able to make use of Swagger though.
First, we need to add a couple of gems to our gemfile:
...
gem 'grape-swagger'
gem 'grape-swagger-ui'
...
We then run bundle to install these gems.
We also need to add these to assets to our assets pipeline (in config/initializers/assets.rb):
Rails.application.config.assets.precompile += %w( swagger_ui.js )
Rails.application.config.assets.precompile += %w( swagger_ui.css )
Finally, in app/api/api.rb we need to mount the swagger generator:
...
add_swagger_documentation
end
...
Now we can take advantage of Swagger’s nice UI to explore our API by simply going to https://localhost:3000/api/swagger.
Swagger presents our API endpoints in a nicely explorable way. If we click on an endpoint, Swagger lists its operations. If we click on an operation, Swagger shows its required and optional parameters and their data types.
One remaining detail though before we proceed: Since we restricted use of the API developers with a valid api_key, we won’t be able to access the API endpoint directly from the browser because the server will require a valid api_key in the HTTP header. We can accomplish this for testing purposes in Google Chrome by making use of the Modify Headers for Google Chrome plugin. This plugin will enable us to edit the HTTP header and add in a valid api_key (we’ll use the dummy api_key of 12345654321 that we included in our database seed file).
OK, now we’re ready to test!
In order to call the pair_programming_sessions API endpoint, we first need to log in. We’ll just use of the email and password combinations from our database seed file and submit it via Swagger to the login API endpoint, as shown below.
As you can see above, the token belonging to that user is returned, indicating that the login API is working as functioning properly as intended. We can now use that token to successfully perform the GET /api/pair_programming_sessions.json operation.
As shown, the result is returned as a properly-formatted hierarchical JSON object. Notice that the JSON structure reflects two nested 1-to-many associations, since the project has multiple reviews, and a review has multiple code samples. If we wouldn’t return the structure in this way, then the caller of our API would need to separately request the reviews for each project which would require submitting N queries to our API endpoint. With this structure, we therefore solve the N+1 query performance issue.
Wrap-up
As shown herein, comprehensive specs for your API help ensure that the implemented API properly and adequately addresses the intended (and unintended!) use cases.
While the example API presented in this tutorial is fairly basic, the approach and techniques we have demonstrated can serve as the foundation for more sophisticated APIs of arbitrary complexity using the Grapegem. This tutorial has hopefully demonstrated that Grape is a useful and flexible gem that can help facilitate implementation of a JSON API in your Rails applications. Enjoy!
[/pt_text]
[/pt_text][pt_testimonials_balloon name=”About this post” quote=”This is a shared post. Originally posted on “https://www.toptal.com/“. All rights reserved to their editors. I just shared this post on Original Author’s demand to share this post on my blog.” minheight=”170″]