macro photo of red ruby group

Wrapping Rest APIs with Ruby

At PipelineDeals, we have a growing number of satellite apps—we still have one main monolithic Rails app (which we’ve started calling p.core), but ever so slowly, we have ported bits & pieces of functionality out to smaller, separate apps. These apps communicate with p.core and each other in a couple of ways:

  1. pubsub messaging system we built ourselves
  2. Our own APIs!

Essentially, our main Rails app (p.core) broadcasts events that happen in the system to any satellite app that pays attention. So, for instance, if a person’s email address was updated, we would send a pubsub message like person:updated:123456 and the payload would include information about the attributes that changed.

Conversely, when a satellite app needs to make a change to a model in p.core, the satellite app will use our public api to write changes.

When using our own APIs, we’ve grown into a pattern for wrapping them with Ruby classes that feels very clean and predictable to us.

our architecture imagined as a space elevator. Earth is p.core, the end of the elevator a satellite app, and the cable our api. The elevator box itself is composed of our Ruby classes.

How we’ll use it

In the middle of 2014, we upgraded our Google Apps integration to a newer version of Google’s API. As part of this work, we moved the entire integration into a separate app—p.core now knows nothing about our Google integration.

I’ll use this app (which we call p.google) as an example.

As part of its work, p.google needs to list all events on a user’s PipelineDeals calendar. We’d like that to look like this:

1Pipeline::Api::Events.new(user_api_key).list

Where user_api_key uniquely identifies each user in p.core. We store these values in the p.google database.

This looks pretty good, but if a user has more than 200 events, p.core paginates the results. In that case, we’ll want to include a page number.

1Pipeline::Api::Events.new(user_api_key).list(page: 2)

Finally, sometimes p.google just needs to search for certain events within a user’s calendar.

1Pipeline::Api::Events.new(user_api_key).list(conditions: { named: “Pi Day” })

The Pipeline::Api::Events class

Ok, so what should that class look like? First, we need to instantiate it with the single user_api_key argument. Easy enough.

Next, we need to make the list method with a couple of optional named arguments (thanks, Ruby 2!):

Above you can really see the virtues of Ruby 2’s named parameters. The code is easily readable and it’s clear what will happen if you do not pass your own params into this function.

Next we come to the conditions.reverse_merge. The p.core events api always needs these parameters, for our use case. However, if we want to override them, reverse_merge makes sure we can do that.

Then we come to handle_response. I won’t bother showing you the exact code, but handle_response deals with non-200 responses (like 402s or 422s). If the response looks good, it passes right through.

Then we come to the main functionality of the method: Pipeline::Api.get. What happens there? Excitingly boring things!

The Pipeline::Api class

This class lays the foundation for communicating with the p.core API.

I find this class wonderfully readable. I’ll step through it bit by bit, in case you have trouble following.

Class methods as conveniences

We only want to use this class through its class methods. Because this:

1Pipeline::Api.get(‘calendar_entries’, { other: ‘stuff’ })

Reads nicer than

1Pipeline::Api.new(‘calendar_entries’, { other: ‘stuff’ }).get

We decided that we care so much about only using the class methods that we put the def initialize in the private section, which Ruby actually ignores. However, it effectively communicates to future-us the message, please consider only using the class methods.

Even though we want to access the class through its class methods, we made sure to make these methods very small. They do nothing but delegate to instance methods of the same name. Why do this? Because class methods resist refactoring.

ENV vars ftw!

Putting the ApiUrl and app_key into ENV variables has a couple advantages.

  1. We can copy and paste the entire class right into a blog post and not even care
  2. We only need to change these parts for connecting to a locally-running p.core for development and testing (for which we use vcr—does anyone know a more robust way to test between app boundaries?). Using something like Rails secrets makes this robust and easy.

The app_key

Each of our satellite apps gets an app_key. This does a few things for us.

  1. Apps like p.google or our iPhone app use our API on behalf of our users, but the users shouldn’t need to enable API access or even know that an API is a thing. (This helps explain the use of user_api_key above!)
  2. We can track usage of each app, and make sure none of them DoS our main app.
  3. If errors occur in calls with an app_key, we can track down the app author and let them know.

Retries

We baked one other thing into this simple-looking class: retries. The with_retries block has become a pattern that we use all over. It provides a central place in the codebase where we can handle the logic for retrying failed network requests.

Conclusion

another image of a hypothetical space elevator

If you have decided to take the route of breaking apart your rails monolith by using service apps, it’s a good idea for those service apps to use the public API of your core application. This is a powerful form of dogfooding and you will quickly realize any performance limitations or bugs in your API.

We didn’t plan this pattern ahead of time, we grew into it. Now that we’ve arrived here, it feels right. Things that make it feel right:

  • Classes read easily
  • Classes and methods all seem short and focused
  • Changing them when new requirements come up doesn’t take long
  • We have no trouble testing them

What do you think? Do you use a similar pattern? We’d love to hear about it!

Share this post:

Share on facebook
Share on google
Share on twitter
Share on linkedin
Share on pinterest
Share on print
Share on email

Don't miss another post! Sign up here.