Introduction to event sourcing in Ruby on Rails

A large number of applications usually have a database where all kind of data is kept while being processed and displayed to the user. Although for the large majority of these apps only the current state of data is important, there are some cases where a detailed history of all the state changes is needed in order to find out how we got to the current state.

In this case, when the state change history is important, we can use the event sourcing pattern, the design pattern that ensures that each step was logged and can be accessed in order to check all the steps involved until the current state.

In this article, we want to show a use case of the event sourcing in Ruby on Rails pattern so we have created an application that is available on GitHub. Based on the supposition that a company has multiple entities (phones, computers, keyboards, etc.) and it keeps all this data in Excel files, the requirement is to create an application that can be used to store all the data and all the states every entity goes through.

The main objective of this app is to keep track of the entities in a company. We would like to be able to answer questions like: “where was a certain keyboard over the last 5 months?” (from a store to person X, then being broken, back to the warehouse and so on) or “which entities were deleted so far?” and so on.

The app should be able to store data about every entity, even if this data is modified from one stage to another. Let’s say a computer is given to an operator with certain specifications but for some reason, it had added/removed a RAM card, so when it’s moved we can keep track of these changes as well.

Add gem "rails_event_store" in Gemfile.rb and run bundle.

spring stop # if you use spring
rails generate rails_event_store_active_record:migration
rails db:migrate

Then instantiate a client by adding the code below in your config/application.rb file:

config.event_store = RailsEventStore::Client.new

Add the code below in your config/application.rb file because we will use the AggregateRoot module later:

AggregateRoot.configure do |config|
 config.default_event_store = Rails.application.config.event_store
end

Your application.rb file should look approximately like this:

require_relative 'boot'

require 'rails/all'
require 'aggregate_root'

# Require the gems listed in Gemfile, including any gems
# you've limited to :test, :development, or :production.
Bundler.require(*Rails.groups)

module RailsEventstoreEntitiesTracker
 class Application < Rails::Application
   # Initialize configuration defaults for originally generated Rails version.
   config.load_defaults 5.2
   config.autoload_paths += Dir["#{config.root}/app/**/"]
   config.autoload_paths += Dir["#{config.root}/lib/**/"]

   config.before_configuration do
     env_file = File.join(Rails.root, 'config', 'local_env.yml')
     YAML.load(File.open(env_file)).each do |key, value|
       ENV[key.to_s] = value
     end if File.exists?(env_file)
   end

   config.event_store = RailsEventStore::Client.new
 
   AggregateRoot.configure do |config|
     config.default_event_store = Rails.application.config.event_store
   end
 end
end

Then we need to add an event_store.rb file under the config/initializers folder where we will define our subscribers to the events. 

Rails.application.config.event_store.tap do |es|
	#more later
end

After that, we need to create some files under the lib/commands/ folder. Firstly we will add the validation_error.rb file so we can use customized errors:

module Commands
 class ValidationError < StandardError
   attr_accessor :message, :custom_error

   def initialize(message, custom_error = nil)
     @message = message
     @custom_error = custom_error
   end
 end
end

Secondly, we will add the base.rb file: 

module Commands
 class Base
   include ActiveModel::Model
   include ActiveModel::Validations
   include ActiveModel::Conversion

   def initialize(attributes={})
     super
   end

   def validate!
     raise ValidationError, errors unless valid?
   end

   def persisted?
     false
   end
 end
end

All our commands will inherit from this Base class so it can be validated and initialized. In this article, we are only going to present the Create event, but all other events that were implemented - Edit, Delete, Restore - can be checked on GitHub.

Finally, we will add the execute.rb helper file, which will help us to use the execute method in our controllers.

module Commands
 module Execute
   def execute(command)
     command.validate!
     handler_for(command).call(command)
   end

   private
   def handler_for(command)
     {
       Commands::Entities::CreateEntity => CommandHandlers::Entities::CreateEntity.new,
       Commands::Entities::EditEntity   => CommandHandlers::Entities::EditEntity.new,
       Commands::Entities::DeleteEntity => CommandHandlers::Entities::DeleteEntity.new,
       Commands::Entities::RestoreEntity => CommandHandlers::Entities::RestoreEntity.new,
     }.fetch(command.class)
   end
 end
end

In event sourcing in rails, an event is seen as an action that already happened, therefore, we need to represent it as a verb in the past tense such as EntityCreated, EntityEdited, EntityAddedToObject. Current production databases mostly rely on storing the current state of a record while the event store stores every event in the database.

module Events
 EntityCreated = Class.new(RailsEventStore::Event)
 EntityEdited = Class.new(RailsEventStore::Event)
 EntityDeleted = Class.new(RailsEventStore::Event)
 EntityRestored = Class.new(RailsEventStore::Event)
end

After we have defined the events, the next step would be to define what the action does. This action represents the command. This way we decouple the events that are done from the action. As can be seen in the screenshot below, we created a new module under lib, called commands where all the commands are defined. In order to be saved, each action (command) needs to be validated and checked if it is persisted or not.

For example, `CreateEntity`, defined below represents the definition of create action. As it can be seen it validates the presence of uid and name, and it checks that they are not blank.

 module Entities
   class CreateEntity < Base
     attr_accessor :uid, :name, :description, :state, :extra_data

     validates :uid, presence: true, allow_blank: false
     validates :name, presence: true, allow_blank: false

     def aggregate_uid
       uid
     end
   end
 end
end

After the command is executed, a command handler is triggered which will manipulate data from the command and it will call methods on the domain model. The command handler is an entry point to your domain. 

module CommandHandlers
 module Entities
   class CreateEntity
     def call(command)
       stream = "Domain::Entity$#{command.aggregate_uid}"
       entity_data = {
         uid: command.uid,
         name: command.name,
         description: command.description,
         state: command.state,
         extra_data: command.extra_data
       }

       aggregate = Domain::Entity.new(command.aggregate_uid).load(stream)
       aggregate.create(entity_data)
       aggregate.store
     end
   end
 end
end

When using event sourcing in rails, your aggregates (domain models) are built based on domain events. To create a new aggregate domain object, we need to include the AggregateRoot module inside domain classes. 

module Domain
 class Entity
   include AggregateRoot
...

Its underlying data model doesn’t store the current state, but instead a series of domain events that have been applied to that aggregate since the beginning of its creation. The convention proposed by the rails_event_store gem is to use the word apply + underscored event class name:

module Domain
 class Entity
   include AggregateRoot

   attr_reader :uid, :name, :description, :state, :extra_data, :restored

   def initialize(uid)
     @uid = uid
   end

   def create(data)
     apply Events::EntityCreated.new(data: data)
   end

    private

   def apply_entity_created(event)
     @name = event.data[:name]
     @description = event.data[:description]
     @state = event.data[:state]
     @extra_data = event.data[:extra_data]
   end
 end
end

Using this convention will ensure that the aggregate state will not be exposed and it will allow it to protect its invariants.

Every method called by an aggregate will most likely result in publishing new events. 

Note: The only way to change the internal state of an aggregate is by publishing and applying new domain events, to ensure that the aggregate will be rebuilt to the same state from the events.

Once an event is published, it will trigger a subscriber defined in our event_store.rb file:

Rails.application.config.event_store.tap do |es|
 es.subscribe(Denormalizers::Entities::EntityCreated, to: [Events::EntityCreated])
end

We can add a denormalizer that will manipulate data from the event and insert it into a classic database, which will be used for displaying data (GET requests). A huge benefit of this is that we can get the latest state of an entity in the traditional way without browsing in the stream, therefore when an event in rails is published we take the data and store it in its corresponding table.

module Denormalizers
 module Entities
   class EntityCreated
     def call(event)
       ::Entity.create!(event.data)
     end
   end
 end
end

Each entry into the ES database contains a list of attributes as described in the following paragraphs. There are similarities between the ES database and an ordinary database such as the ID or metadata. When we install the event store gem, it provides us two tables, event_store_events and event_store_events_in_streams.

The `event_store_events` table saves the events in ruby on rails and contains the following attributes: ID, event_type, data, created_at and metadata.

The ID field is unique for each entry, the event_type field is the event class name (Events:: + Event Name), the data field is everything that is passed to the domain when an event is applied, and the created_at field is the timestamp.

Similar to an SQL database, the event store provides the option to store a metadata field as well. By default, it stores the remote ip (which is the IP of the HTTP client that issued the request) and the request id (which is a unique id of the request) but it can be extended to store custom data according to the requirements of the application. Data and metadata fields are blob types.

The `event_store_events_in_streams` table is used to save the streams and has the following attributes: ID, stream, event_id, created_at and position. The id field is an incremented ID, stream is the name of the stream which conventionally is Domain:: + Domain Model Name + $aggregate_uid, event_id is the unique id of the event, the created_at field is the timestamp, and the position is the expected version of the stream, but we are not going to cover that in this article. 

The app we created for this article provides four events - EntityCreated, EntityEdited, EntityDeleted, and EntityRestored. 

We wanted to keep track of all the deleted entities so we used another important tool provided by the gem - link to another stream. When an event is published it can live in multiple streams, this being very powerful when we want to group events with specific particularities. So we linked the EntityDeleted event to another stream called DeletedEntities. 

class DeletedEntitiesTracker
 def call(event)
   Rails.application.config.event_store.link(
     event.event_id,
     stream_name: "DeletedEntities"
   )
 end
end

There are cases in some applications where the user is interested in knowing more than the current state of a certain piece of data. In these cases, event sourcing in ruby on rails design is the perfect fit and for a Rails application, we have the `rails_event_store` gem, which is the best choice to do it. 

Our article tried to showcase what the event sourcing in rails pattern is and how it can be used in a Rails application. Although we covered only the creation event, the other events and the entire code base for the application can be checked on GitHub.

The application contains a number of specs as well and you can find out even more about how to integrate cucumber tests with Rails Event Store by reading our colleague’s article on this topic.