Introduction to event sourcing in Ruby on Rails
Date published: May 14, 2019
10 min read
1. Introduction
A large number of applications usually have a database where all kind of data is kept while being processed and displayed to the user. Although for the large majority of these apps only the current state of data is important, there are some cases where a detailed history of all the state changes is needed in order to find out how we got to the current state.
In this case, when the state change history is important, we can use the event sourcing pattern, the design pattern that ensures that each step was logged and can be accessed in order to check all the steps involved until the current state.
In this case, when the state change history is important, we can use the event sourcing pattern, the design pattern that ensures that each step was logged and can be accessed in order to check all the steps involved until the current state.
Fig 1. Event Source Rails
Event sourcing (ES) is a design pattern that allows us to build applications that are oriented around domains that are easy to extend. An event can be described as an important change of a certain state. For example, when a user moves a product from point A to point B, the product’s location changes from A to B.
Many applications would just store the current state (the location in our case) of the product but event sourcing allows us to keep track of all the changes made to an object’s state. Let’s say a user moves the same product from point B to point C. In this case, we usually only know the current state (point C) but using event sourcing in rails, we know that the product was moved from point A to point B before reaching point C. Event sourcing in rails Store is a gem that allows us to use an Event-Driven Architecture (EDA) for our Rails applications.
2. Application concept
In this article, we want to show a use case of the event sourcing in Ruby on Rails pattern so we have created an application that is available on GitHub. Based on the supposition that a company has multiple entities (phones, computers, keyboards, etc.) and it keeps all this data in Excel files, the requirement is to create an application that can be used to store all the data and all the states every entity goes through.
Fig 2. Entities list
The main objective of this app is to keep track of the entities in a company. We would like to be able to answer questions like: “where was a certain keyboard over the last 5 months?” (from a store to person X, then being broken, back to the warehouse and so on) or “which entities were deleted so far?” and so on.
The app should be able to store data about every entity, even if this data is modified from one stage to another. Let’s say a computer is given to an operator with certain specifications but for some reason, it had added/removed a RAM card, so when it’s moved we can keep track of these changes as well.
The app should be able to store data about every entity, even if this data is modified from one stage to another. Let’s say a computer is given to an operator with certain specifications but for some reason, it had added/removed a RAM card, so when it’s moved we can keep track of these changes as well.
3. How to configure rails_event_store inside a Rails application
Add gem "rails_event_store" in Gemfile.rb and run bundle.
spring stop # if you use spring
rails generate rails_event_store_active_record:migration
rails db:migrate
Then instantiate a client by adding the code below in your config/application.rb file:
config.event_store = RailsEventStore::Client.new
Add the code below in your config/application.rb file because we will use the AggregateRoot module later:
AggregateRoot.configure do |config| config.default_event_store = Rails.application.config.event_store end
Your application.rb file should look approximately like this:
require_relative 'boot' require 'rails/all' require 'aggregate_root' # Require the gems listed in Gemfile, including any gems # you've limited to :test, :development, or :production. Bundler.require(*Rails.groups) module RailsEventstoreEntitiesTracker class Application < Rails::Application # Initialize configuration defaults for originally generated Rails version. config.load_defaults 5.2 config.autoload_paths += Dir["#{config.root}/app/**/"] config.autoload_paths += Dir["#{config.root}/lib/**/"] config.before_configuration do env_file = File.join(Rails.root, 'config', 'local_env.yml') YAML.load(File.open(env_file)).each do |key, value| ENV[key.to_s] = value end if File.exists?(env_file) end config.event_store = RailsEventStore::Client.new AggregateRoot.configure do |config| config.default_event_store = Rails.application.config.event_store end end end
Then we need to add an event_store.rb file under the config/initializers folder where we will define our subscribers to the events.
Rails.application.config.event_store.tap do |es| #more later end
After that, we need to create some files under the lib/commands/ folder. Firstly we will add the validation_error.rb file so we can use customized errors:
module Commands class ValidationError < StandardError attr_accessor :message, :custom_error def initialize(message, custom_error = nil) @message = message @custom_error = custom_error end end end
Secondly, we will add the base.rb file:
module Commands class Base include ActiveModel::Model include ActiveModel::Validations include ActiveModel::Conversion def initialize(attributes={}) super end def validate! raise ValidationError, errors unless valid? end def persisted? false end end end
All our commands will inherit from this Base class so it can be validated and initialized. In this article, we are only going to present the Create event, but all other events that were implemented - Edit, Delete, Restore - can be checked on GitHub.
Finally, we will add the execute.rb helper file, which will help us to use the execute method in our controllers.
module Commands module Execute def execute(command) command.validate! handler_for(command).call(command) end private def handler_for(command) { Commands::Entities::CreateEntity => CommandHandlers::Entities::CreateEntity.new, Commands::Entities::EditEntity => CommandHandlers::Entities::EditEntity.new, Commands::Entities::DeleteEntity => CommandHandlers::Entities::DeleteEntity.new, Commands::Entities::RestoreEntity => CommandHandlers::Entities::RestoreEntity.new, }.fetch(command.class) end end end
In event sourcing in rails, an event is seen as an action that already happened, therefore, we need to represent it as a verb in the past tense such as EntityCreated, EntityEdited, EntityAddedToObject. Current production databases mostly rely on storing the current state of a record while the event store stores every event in the database.
module Events EntityCreated = Class.new(RailsEventStore::Event) EntityEdited = Class.new(RailsEventStore::Event) EntityDeleted = Class.new(RailsEventStore::Event) EntityRestored = Class.new(RailsEventStore::Event) end
After we have defined the events, the next step would be to define what the action does. This action represents the command. This way we decouple the events that are done from the action. As can be seen in the screenshot below, we created a new module under lib, called commands where all the commands are defined. In order to be saved, each action (command) needs to be validated and checked if it is persisted or not.
Fig 3. Commands and command handlers file structure
For example, `CreateEntity`, defined below represents the definition of create action. As it can be seen it validates the presence of uid and name, and it checks that they are not blank.
create_entity.rb file:
module Commands
module Entities class CreateEntity < Base attr_accessor :uid, :name, :description, :state, :extra_data validates :uid, presence: true, allow_blank: false validates :name, presence: true, allow_blank: false def aggregate_uid uid end end end end
After the command is executed, a command handler is triggered which will manipulate data from the command and it will call methods on the domain model. The command handler is an entry point to your domain.
create_entity.rb handler file:
module CommandHandlers module Entities class CreateEntity def call(command) stream = "Domain::Entity$#{command.aggregate_uid}" entity_data = { uid: command.uid, name: command.name, description: command.description, state: command.state, extra_data: command.extra_data } aggregate = Domain::Entity.new(command.aggregate_uid).load(stream) aggregate.create(entity_data) aggregate.store end end end end
When using event sourcing in rails, your aggregates (domain models) are built based on domain events. To create a new aggregate domain object, we need to include the AggregateRoot module inside domain classes.
module Domain class Entity include AggregateRoot ...
Its underlying data model doesn’t store the current state, but instead a series of domain events that have been applied to that aggregate since the beginning of its creation. The convention proposed by the rails_event_store gem is to use the word apply + underscored event class name:
module Domain class Entity include AggregateRoot attr_reader :uid, :name, :description, :state, :extra_data, :restored def initialize(uid) @uid = uid end def create(data) apply Events::EntityCreated.new(data: data) end private def apply_entity_created(event) @name = event.data[:name] @description = event.data[:description] @state = event.data[:state] @extra_data = event.data[:extra_data] end end end
Using this convention will ensure that the aggregate state will not be exposed and it will allow it to protect its invariants.
Every method called by an aggregate will most likely result in publishing new events.
Note: The only way to change the internal state of an aggregate is by publishing and applying new domain events, to ensure that the aggregate will be rebuilt to the same state from the events.
Once an event is published, it will trigger a subscriber defined in our event_store.rb file:
Rails.application.config.event_store.tap do |es| es.subscribe(Denormalizers::Entities::EntityCreated, to: [Events::EntityCreated]) end
We can add a denormalizer that will manipulate data from the event and insert it into a classic database, which will be used for displaying data (GET requests). A huge benefit of this is that we can get the latest state of an entity in the traditional way without browsing in the stream, therefore when an event in rails is published we take the data and store it in its corresponding table.
Example:
module Denormalizers module Entities class EntityCreated def call(event) ::Entity.create!(event.data) end end end end
3.1 How are data organized?
Each entry into the ES database contains a list of attributes as described in the following paragraphs. There are similarities between the ES database and an ordinary database such as the ID or metadata. When we install the event store gem, it provides us two tables, event_store_events and event_store_events_in_streams.
The `event_store_events` table saves the events in ruby on rails and contains the following attributes: ID, event_type, data, created_at and metadata.
The ID field is unique for each entry, the event_type field is the event class name (Events:: + Event Name), the data field is everything that is passed to the domain when an event is applied, and the created_at field is the timestamp.
Similar to an SQL database, the event store provides the option to store a metadata field as well. By default, it stores the remote ip (which is the IP of the HTTP client that issued the request) and the request id (which is a unique id of the request) but it can be extended to store custom data according to the requirements of the application. Data and metadata fields are blob types.
Fig 4. event_store_events entry sample for EntityDeteled event
Fig 5. Metadata text sample for EntityDeteled event
Fig 6. Data text sample for EntityDeteled event
The `event_store_events_in_streams` table is used to save the streams and has the following attributes: ID, stream, event_id, created_at and position. The id field is an incremented ID, stream is the name of the stream which conventionally is Domain:: + Domain Model Name + $aggregate_uid, event_id is the unique id of the event, the created_at field is the timestamp, and the position is the expected version of the stream, but we are not going to cover that in this article.
Fig 7. event_store_events_in_streams entry sample
The app we created for this article provides four events - EntityCreated, EntityEdited, EntityDeleted, and EntityRestored.
Fig 8. Events list for an entity
Fig9. Event list of a restored entity
We wanted to keep track of all the deleted entities so we used another important tool provided by the gem - link to another stream. When an event is published it can live in multiple streams, this being very powerful when we want to group events with specific particularities. So we linked the EntityDeleted event to another stream called DeletedEntities.
class DeletedEntitiesTracker def call(event) Rails.application.config.event_store.link( event.event_id, stream_name: "DeletedEntities" ) end end
Fig 10. Event list of a deleted entity
4. Conclusions
There are cases in some applications where the user is interested in knowing more than the current state of a certain piece of data. In these cases, event sourcing in ruby on rails design is the perfect fit and for a Rails application, we have the `rails_event_store` gem, which is the best choice to do it.
Our article tried to showcase what the event sourcing in rails pattern is and how it can be used in a Rails application. Although we covered only the creation event, the other events and the entire code base for the application can be checked on GitHub.
The application contains a number of specs as well and you can find out even more about how to integrate cucumber tests with Rails Event Store by reading our colleague’s article on this topic.
