text:
An Incremental Migration from Rails Monolithic to Microservices
Posted on by Erin Swenson-Healey in Development, Everything Else, Microservices, Rails, Ruby
Your Rails application has become a monolith. Your test suite takes 30 minutes to run, your models grow to several hundred (or thousand!) lines, and feature development slows to a crawl. New members of your team must read through massive amounts of code before they can feel confident making changes. You've watched Uncle Bob's Architecture the Lost Years, Matt Wynne's Hexagonal Rails and read Martin Fowler's Microservices and are convinced that the time has come to start breaking things up into smaller, simpler, faster components – but you're unsure of where to begin.
In this article, I will demonstrate an approach to breaking a monolithic-style Rails application apart into microservices using plain Ruby classes and a simple RPC framework – all deployed to Heroku. I'll show you how to work through this migration incrementally; you'll be able to decompose things down as far as makes sense for your codebase. As a special bonus, I'll show you how you can use Barrister RPC to alleviate much of the pain associated with documenting and creating API clients for the components of your now-distributed system. I'm skipping the advocacy; if you're here it's because you already love yourself some microservices but don't know how to implement them yourself.
Monolithic Applications
Martin Fowler describes an example monolith as an application that:
will handle HTTP requests, execute domain logic, retrieve and update data from the database, and select and populate HTML views to be sent to the browser.
In concrete terms, this definition easily fits the traditional Rails application consisting of views, models (our proxies to the database) and HTTP request handlers (controllers). It is this type of application – the one that we web developer-folk in the Rails world most commonly work with – that we'll be decomposing into microservices.
So what's a "microservice"? Fowler defines it as:
An out-of-process component that communicates with a mechanism such as a web service request, or remote procedure call.
He makes note that these microservices are organized around business capabilities; some examples would be: a service that sends emails, a service that manages users, or a reporting service.
A Three Phased Attack
Our migration from monolith to microservices will consist of three steps:
- The logical decomposition of our application into in-process services, each exposing a functional, well-defined interface.
- The physical separation of these services into independent Ruby processes, communicating with each other using Redis as a message broker.
- Decomposing our application into separate Heroku applications, communicating through Barrister RPC using HTTP.
Separation Anxiety
Decomposition of our monolith into a system of microservices requires that we publish an interface for each microservice. As many of us who have worked in such an environment can attest, creating and maintaining the documentation required to fully describe the request-response formats of each component can be a heavy burden to bear. This bothersome step in our workflow is oftentimes skipped or overlooked, resulting in discrepancies between our API implementation and its documentation.
The need to document these internally-consumed interfaces can be mitigated somewhat if the maintainer of the component is willing to publish an API client. The burden is then shifted from documenting an API to maintaining the API's client bindings. Adding additional client language-bindings into the mix increases the API maintainer's workload further, and will eventually lead to endless maintenance of boilerplate API-wrapping code.
Barrister RPC to the Rescue
Barrister RPC provides an interesting solution to the problem of documenting a microservice interface: First, a language-agnostic IDL (interface definition language) file is created in which the developer describes the request-response formats for the methods of their component interface. This IDL file is then used by the Barrister server to enforce the requests and responses to and from the API. The documentation can never get out of sync with our microservice interface because it is the microservice interface.
Using Barrister RPC also obviates the need to write a custom API client. Using the provided RPC client, an API proxy can be generated at runtime after issuing a request to our microservice for its IDL. This has two implications:
The result of this IDL-request always represents of the current state of the API. This obviates the need to write and maintain custom client-bindings. The auto-generated API client is in a position to ensure that the request conforms to the IDL before going over the wire. If a programmer attempts to make an unsupported call (or calls with the wrong data, etc.), an error will be raised.
Step One: Creating Microservices
The first step of our migration will consist of five steps in which we'll:
- Identify chunks of our Rails application to split into services, organized around a business capability
- Document our interface in a Barrister IDL file
- Implement our service as a plain Ruby class that wraps the ORM
- Connect our Rails application to our services through Barrister
- Modify existing controllers to consume services (instead of ORM directly)
Problem Space + Baseline Implementation
For the sake of keeping things simple, we'll be working with a Rails application that facilitates the management of users. This system will allow us to create a user and get a list of users already-created. On user-creation, we'll send that user a "welcome" email. Our system has only one model in it,
…and an accompanying controller:
Identifying and Describing Services
From this simple implementation we can extract two pieces of business function: a
UserService – which manages all things user, and a
MailService – which sends email. First step: complete!
Next, we'll create a new root-level directory in our Rails project:
services. Under
services we'll create a folder
mail_service and inside that folder we'll create our first IDL. This IDL will define our first microservice, the
MailService. The
MailService doesn't do much; we define only one method on its interface:
send_email. Then – in order to communicate to the consumer of our API what we expect to receive, and to the Barrister server what we have committed to return – we'll write down the expected types for each of send_email method's parameters:
It is important to note that all messages between Barrister RPC clients and servers are encoded as JSON-RPC 2.0 requests/responses. This constrains the type of data that we can send between system components; you're unable to encode lambdas, for instance. Barrister RPC provides you with four primitive types (
string) from which you can construct your own structs. We'll use them in the IDL for our new
In the example above, you can see that we've defined two structs:
UserProperties and
UserProperties contains three of its own properties and is extended with an additional property to create the User struct. Properties (in both structs and in method return types) can be marked as omittable by including the
[optional] flag. The final piece of tricky syntax is noted in the return type of our
get_all_users method: The brackets indicate that an array of User structs will be returned.
Implementing a Service
We've identified two services and authored IDL files that describe the requests and responses that these services expect to receive and send. Stellar! Now, we need to do some real work and actually implement our services. We'll start with the easiest: the
MailService implementation is a simple Ruby class that implements the interface described in our mail_service.idl file. We'll follow the same pattern in implementing our
We've hidden the
User model behind the public interface of our new
UserService class. Like the
MailService, the
UserService implements the interface defined in its IDL file – method signature and return types alike. We're nearly done with the microservice side of things – we just need to make a modification to our model, moving it to our
user_service tree and removing the old life cycle callback:
Connecting Rails to Your Services
Now that we've hidden our ORM behind the UserService interface, we need to modify our Rails application to consume this interface. We'll do so through the Barrister::Rails::Client, which we'll create in an initializer and expose via a singleton:
Our proxy_services method looks peculiar in that we're instantiating the service that we aim to communicate with. That's fine for now; we're logically separating our Rails application from our services, deferring the physical separation of those services until the second step of our migration.
Finally, we need to update our controller to use our RPC client:
With this modification complete, we've managed to completely isolate our services from each other with relatively small changes to the original codebase. We've decomposed our monolithic application into a system of logically separated services without losing the ability to leverage the Rails views, controllers, helpers, and models that we're familiar with.
A Note on the barrister-rails Gem
Barrister decodes struct-responses from JSON into hashes and those hashes don't play nicely with the Rails view helpers (form_for, *_path, etc.). To circumvent this problem I've written the barrister-rails gem. This purpose of this gem – which wraps the built in Barrister Ruby client – is to decode responses into dynamically-generated classes whose names match the names of the structs in your IDL.
An example: The IDL describes the response from a call to get_all_users as returning an array of
User structs. The Barrister::Rails::Client will transmute the resulting array of hashes into an array of instances of
User – a class it created at runtime. If a
User class already existed, the Barrister::Rails::Client will attempt to use it instead. The dynamically-generated classes adhere to the ActiveModel API enough that things like
form_for :user continue to work, which allows backwards-compatibility in your views.
Step Two: Distributed Microservices with Redis
By the end of Step One, we'd logically separated our application into in-process services that we communicate with through an RPC client. The indirection might have seemed a little silly (we're serializing/deserializing in-process method requests and responses to JSON), but we've set ourselves up nicely for the next step of the migration. Now, we'll explore what it looks like to physically separate these services until their own Ruby processes, communicating with each other using Redis as a message broker – all deployed as a single Heroku app.
Connecting Your Service to Redis
First, head here and read up on provisioning yourself a Redis database. Once you've gone through the steps to create a database, let's create a new file in your
mail_service directory:
In this file, we're creating a Barrister::RedisContainer and passing the
MailService. The container hides the details of retrieving API requests from the Redis database and enqueuing responses originating from the service's methods. Barrister continues to enforce that requests and responses conform to the IDL; we've just swapped the mechanism by which messages are marshalled to and from the service. Note that the implementation of the
MailService did not need to change, nor did its interface definition.
Spinning up the
UserService follows a similar pattern, but requires that we create a proxy for the
MailService API (accessed as
Connecting Your Client to Redis
We'll need to modify the initializer in the main Rails application to use the Barrister::RedisTransport to connect our client to the Redis database instead of the in-process mechanism being used in Step One. The impact of our architectural change (from the perspective of the Rails application) is minimal; the
UsersController remains unmodified.
Running It All with Foreman
Finally, we'll need to modify our Procfile such that Heroku fires up our microservices on application start. Since the
main.rb files serve as the entry point to each service, making the modification to run them as Foreman processes is a snap:
Step Three: Independently-Deployable Microservices
The third and final step of our migration involves splitting our single Heroku application into several, independently-deployable Heroku applications. We'll end up with a single Rails application speaking HTTP to microservices wrapped by Sinatra.
Splitting Into Separate Codebases
A thorough demonstration of how you would partition your existing repository is outside of the scope of this talk. Some things that you'll need to think about:
- How do you want to split apart your VCS repo? If you're using Git, I recommend reviewing the git subtree command, splitting the existing repository up according to the subdirectories of the
- You'll need to decide if you want each service to have its own database.
- You'll need to decide what you want to do with your existing migrations – which up until now have lived in your Rails application's
Swapping Redis for HTTP
Barrister's pluggable transport support makes swapping containers a breeze. For the sake of brevity (ha ha), I'll show just the
MailService's container:
Like the Barrister::RedisContainer, the Barrister::SinatraContainer hides the details of marshalling the JSON-RPC requests/responses from transport to the
MailService as a method call. We simply bind our container to the port provided by Heroku, and tell Sinatra where to start listening for RPC requests.
In the Rails app, the change is even simpler. We simply swap out the custom transport (Barrister::RedisContainer) with a URI that our client will use to connect to the microservice:
That it! You've got microservices.
Final Words
Many thanks if you're still with me by this point! While this article demonstrates an incremental migration from a monolithic-style Rails application to a distributed microservices architecture – several topics are conspicuously absent. In a future (shorter) blog post, I'll be talking about versioning APIs using Barrister RPC and about decentralized messaging and service-discovery using Iris. Be sure to check out the Barrister RPC website and JSON-RPC 2.0 specification for detailed information on how things are working behind the scenes.
Sample Projects
I have several GitHub repos demonstrating the various stages of this migration from monolith to microservices. Feel free to fork, experiment, and improve!
- Original Monolithic Rails app
- Decomposed into in-process services
- Physically-separated services with Redis
- Swapping Redis database for HTTP
Follow me on Twitter: @lasericus
Send me an email: firstname.lastname@example.com


text:
Search This Blog
We provide data, new trends, actors, intelligence, market dynamics (drivers, opportunities, restraints, challenges, etc.), analytics, advisory, technologies (apps, software, systems, algorithm, etc.), services, market data (consumers, revenue, investments, capital, etc.), actionable insights, information, consulting, etc.
Showing posts from April 28, 2016
Large Scale Ceph Storage Microserver Cluster Unveiled
- Get link
- Other Apps

subset: commoncrawl


text:
Ragtales Bo Rabbit
AGE: Birth +
Ragtales Bo Rabbit is made from the softest fur in a beautiful beige colour and looks just like a real bunny, crafted with love and care.
In true Ragtales fashion, Bo Rabbits features jointed legs and a weighted torso, filled with soft beans to help keep its shape and sit upright, and lightly padded to provide your little one with comfort and well-being.
Bo Rabbit measures approximately 40cm in height and is presented in a beautiful cube printed box.
An ideal gift for a little one!
Suitable for newborn infants.


text:
4 Comments (+add yours?)
Did you buy the ironing station from Victorian Trading Co? Do you love it? What is the height?
Thank you so very much! I had a gift certificate from my son and I have been looking at this for quite some time. I don't have a "sewing room" so to speak. I sew in the living room, so not quite sure where I will put it, but I will figure it out. Is it well made? I figured I could order it and see and always send it back if I have to. Thank you for the info!


text:
Oyo state Police Command Monday warned the people and stakeholders in the Local Government administration to steer clear of any act that may lead to breakdown of laws and order across the state.
The Command handed down the warning in a statement issued by the State Police Public Relatiions Officer (PPRO) SP Lawrence Fadeyi made available to newsmen in Ibadan on the resumption into office of sacked local government Chairmen in the state.
In the statement, the PPRO added that "the role of the Police in the current issues surrounding the administration of Local Governments in Oyo State should not be seen as a face-off but as part of the professional Police efforts geared towards the performance of its constitutional responsibilities".
"The Commissioner of Police, Oyo State Command, CP Shina T. Olukolu wishes to remind for the umpteenth time all the people and Stakeholders in the Local Government administration in Oyo State that it is the constitutional responsibilities of the Nigeria Police Force to amongst other duties, protect lives and properties, maintain laws and order, arrest, investigate and prosecute offenders and the due enforcement of all laws and extant enactments etc in accordance with the 1999 Constitution of the Federal Republic of Nigeria as amended and the Police Acts and Regulations" he said.
SP Fadeyi added, "the Commissioner of Police therefore wishes to appeal to all the Stakeholders, leaders and the entire peace loving good citizens of Oyo State involved in the administration of Local Government in Oyo State to be circumspect in their respective efforts at managing their respective demands on the daily administration of these Local Governments and LCDAs and prevent a descent to unpalatable/unintended consequences."
"As the Police continue to strive to provide adequate security at the respective Local Government offices; nonetheless, the Stakeholders are enjoined to refrain from any act(s) or cause of action that may lead to breakdown of laws and order, lawlessness, and subsequent avoidable shedding of blood, injuries or damage to properties."
The PPRO maintained in the statement that "it is our hope that the issues at stake/unintended lock jam will eventually be addressed to the satisfaction of all the groups concerned and in the best interest of the entire people of Oyo State who have no other State that they can call their own."
SP Fadeyi then warned social miscreants/ criminal minded individuals/groups " to stay clear from the premises of the Local Governments and LCDAs as the Police will not allow any such developments" adding "the law abiding members of the Public are however assured of their safety and advised to go about their lawful businesses in an atmosphere of peace and tranquility devoid of any molestation, intimidation or threat whatsoever. In the same breath, they are urged to continue to partner with the Police to enable us collectively deal with the criminal elements in our midst.


text:
Pricing Info
- $5 per day
- $20 per week
$5.00 per day
Pricing Info
Safety 1st Secure Click
Moving from the crib to a "big kid" bed is an exciting step for both you and your child, and a bed rail lets your family make the transition easily and with less worry.
We provide FREE delivery and set-up to your choice of location! Please let us know in the Additional Comments section if you have a preferred room you would like us to set up the equipment in. Please refer to our Delivery, Set-up & Pick up section in the FAQs for more info.
Note: We are unable to install car seats for liability purposes, but we will deliver/pickup from car rental agencies!
All our products are cleaned and sanitised after ever use. Please refer to our Gear section in the FAQs for more info.


text:
Do you think that when Alan Sugar, Richard Branson and J.K Rowling first started out they were successful overnight? Well they weren't. So that answered that. They did not make millions on their first day of business. When Rowling was sat on that train planning Harry Potter, she didn't know it would make her... Continue Reading →
We've got your back!
You're browsing through Facebook or Instagram and you come across someone and think 'Why do they have everything and I don't?' They don't have everything. I promise. It's just not possible. (Unless they are The Queen... are they The Queen?) Your life will never be perfect. Now I'm not saying you'll never be... Continue Reading →
Darkness – The all consuming
The pain cuts through you without any warning sign. An invisible force has presented itself in front of you. You can't see it but you sure as hell can feel it. There's no way around it, through it or over it. It torments and taunts without even uttering a sound. You can... Continue Reading →
My law of attraction in verse…
I recently joined a mindset coaching academy called the 'Bossing It Academy', here they (they being Holly and Caroline, who are incredible and will help you every step of the way) teach you about the Law of Attraction and how to basically boss the s**t out of your life. So my brain has obviously become inspired... Continue Reading →