August 14, 2023

Why Cirqus, and Why Now?

I have been working on Evently for a few years now, building out the foundations of the event ledger. Evently has a nice CLI to help developers inspect their ledger, a stable REST API, and a front-end-friendly notification system for real-time event subscriptions. In all, the bones of Evently look pretty good. It’s time to move forward from the “what do you think?” preview mode we have been in and on to a usable service to build applications.

That means we have to add in account and service management capabilities. As I started to design these, it became obvious that this work would be much easier to write as an Event-Sourced application that used CQRS concepts like commands, read models and the event ledger to store everything meaningful. In other words, Evently should use event-sourcing to build up the Evently offering. We need to eat our own dog food, and the dog food of choice is an application framework.

Over the years, I have built CQRS/ES application frameworks in different programming languages and have used several others, so the thought of building a new one is both familiar and exciting. I have made numerous mistakes and learned many lessons from others along the way. Evently has some unique capabilities that don’t quite fit inside existing frameworks, such as Filter Selectors, Atomic Append and Notifications, so what I know today is about 50% of what needs to be known in order to create a useful and compelling framework for Evently.

Naming is hard, so that seems like a good place to start. After a lot of fussing about with whiteboards, sticky notes and talking out loud, I’m settling on calling it Cirqus. The letters CQRS/ES do not turn into a simple term (though I did give Querqus a good go), and finally settled on the common software practice of misspelling things. Evently isn’t a word, either, so it fits the pattern.

Cirqus Key Design Principles

Front-End Friendly

Evently focuses on reaching beyond the server-side, out to the front end. In this day and age, that means Javascript, or at least a JS-friendly interface. The framework will run in the browser as well as in server-side node deployments. It should bear a minimal footprint and use modern concepts as well as pair well with tree-shakers and compilers like Webpack.

Declarative

Nobody likes writing boilerplate; it’s easy to mess up, and it adds no value. We have some really nice options now with decorators and ESM to declare relationships and reactions in code, rather than wiring things together manually. For example, here are some examples of what your application code might look like as declarative reactions.

Read Model

// ESM lets us import JSON files safely
import GuestRegisteredSchema from "./events/guest-registered-schema.json" assert {type: "json"}
// Types generated by json-schema-to-typescript
import {GuestRegistered} from "../../types/guest-registered-schema"
import GuestDeletedSchema from "./events/guest-deleted-schema.json" assert {type: "json"}
import {GuestDeleted} from "../../types/guest-deleted-schema"


// the Read Model type
export type GuestModel = {
  firstName:  string
  lastName:   string
  email:      string
  state:      "registered" | "deleted"
}

export type GuestModelVariables = {
  email: string
}


// marks the class as a read model
@ReadModel({
  name:  "registered guests",
  // Evently selector used to fetch events
  selector: {
    data: {
      guest: {
        // Uses the schema's Title as the event name.
        [GuestRegisteredSchema.title]: "$.email ? (@ == $email)"
        // Notice the '$email' var? This read model reuses the selector statement
        // by using data from other call sites to fill in the request.
      }
    }
  }
})
export class GuestEntityReadModel implements ReadModel<GuestModel, GuestModelVariables> {

  // react to an event in the stream.
  @OnEvent(GuestRegisteredSchema)
  // model is the hydration result
  onGuestRegistered(model: GuestModel, event: GuestRegistered): GuestModel {
    const {firstName, lastName, email} = event
    return {
      firstName,
      lastName,
      email,
      state: "registered"
    }
  }

  @OnEvent(GuestDeletedSchema)
  onGuestDeleted(model: GuestModel, event: GuestDeleted): GuestModel {
    return {
      // spread operator makes hydration easy.
      ...model,
      state: "deleted"
    }
  }
}

The class declares its selector statement as well as the variables needed to query for specific events. Below, look at the CommandHandler implementation where it provides the selector variable email from the Command data.

All the handlers required to build the read model are annotated to indicate which event they react to. Cirqus will handle building the selector and optimizing the calls to only select events that have reactions.

One can imagine keeping the read model instance in memory and updating it when needed by replaying only newer events. This same class can be used to support event notifications to automatically keep the read model up-to-date with relevant ledger events.

Command Handlers

import RegisterGuestSchema from "./commands/register-guest-schema.json" assert {type: "json"}
import {RegisterGuest} from "../../types/register-guest-schema"
import GuestRegisteredSchema from "./events/guest-registered-schema.json" assert {type: "json"}
import {GuestRegistered} from "../../types/guest-registered-schema"


@CommandHandler({
  entity:   "guest",
  command:  RegisterGuestSchema,    // JSON Schema object
  event:    GuestRegisteredSchema,  // JSON Schema object
  model:    GuestModel              // Hydrated by the GuestEntityReadModel class above
})
export class RegisterGuestHandler extends CommandHandler<RegisterGuest, GuestEntityReadModel, GuestRegistered> {
  
  // the read model uses a filter selector to find events that have this command's email value
  selectorVariables(command: RegisterGuest): GuestModelVariables {
    // filter selector is "$ ? (@.email == $email)" so return 'email' variable from command data
    return {
      email: command.email
    }
  }

  execute(command: RegisterGuest, model: GuestModel): CommandResult<GuestRegistered> {
    // model may be undefined, if the email has never been used, or a Guest with this email adress
    // may be in a 'deleted' state
    if (model && model.state !== 'deleted') {
      // instead of throwing an exception, return the rejection message.
      return {
        rejection: `Guest email ${command.email} already registered.`
      }
    }
    
    const {
      firstName, lastName, email
    } = command

    // no state failures, return the event body. Evently takes care of creating an event ID
    // and appending atomically. If the event is rejected due to a race condition, the handler
    // is executed again with an updated Read Model.
    return {
      event: {
        meta: {
          commandId: command.id
        },
        data: {
          firstName, lastName, email
        }
      }
    }
  }
}

Cirqus uses Decorators (newly-promoted to Stage 3 in TC39) to declare types and data about the code rather than requiring the developer to write a bunch of registration code to wire things together. These declarations build on each other so that the runtime safety is optimized. For instance, the event and command declarations are actually JSON Schema objects, which are used internally to validate the commands and events for correctness before use.

This saves the developer from having to defensively check their inputs, and instead invest in higher-quality JSON Schema declarations. In Typescript, this approach goes further by generating Types from the schema so the TS compiler can catch errors in your code early.

Side Effect Free

CQRS/ES frameworks usually disallow interactions with external systems for many reasons, so Cirqus will be making it very hard to do so. By using declarative classes, rather than wiring up instances, developers will have limited options to grab state from other systems or make database calls where they should not. Their code should be side effect free. One nice advantage of this approach will be simpler unit testing. Nothing needs to be mocked, just data in, data out.

Reactive

Evently Notifications bring liveness to applications with event notifications, yet leave the application in charge of when to consume the new events. This approach, while powerful and intuitive, does not have the familiarity one finds in other Event ledgers. Cirqus will make notifications easy to manage and consume so your apps stay in control of their experience. Nothing worse than getting data shoved at you just when your battery is low, or the tunnel looms, or your app is busy doing more important things.

Open Source

This framework will be developed in the open, be free to use under an Open Source license. I’m partial to the MIT license. I will update this post with URLs soon so watch this space for details.

What’s Next

The core of Cirqus is an existing framework I built for another project a few years ago. I’m cleaning that up so it compiles and does a small subset of what it needs to do to tell the story. This will be pushed to a repo, release as a v0 and iterated from there.