Idempotent Services and Guard Rules


The guard rule pattern provides a way to ensure services are idempotent even when their actions aren't. This post shows how to use guard rules in KRL.

Microservices are usually easier to program when responses to an event are idempotent, meaning that they can run multiple times without cumulative effect.

Many operations are idempotent (i.e. adding a ruleset to a pico over and over only results in the ruleset being added once). For operations that aren't naturally idempotent, we can make the rule idempotent using the rule's guard condition. Using a guard condition we can ensure the rule only fires when specific conditions are met.

Unfortunately, there are some functions in KRL (notably in the PCI and RSM modules) that make state changes (i.e. have persistent effect). These modules are used extensively in CloudOS. When these are used in the rule prelude, they cause side effects before the rule's guard condition is executed. This is a design flaw in KRL that I hope to rectify in a future version of the language. These functions should probably be actions rather than functions so that they only operate after the guard condition is met.

In the meantime, a guard rule offers a useful method for assuring idempotency in rules. The basic idea is to create two rules: one that tests a guard condition and one that carries out the rule's real purpose.

The guard rule:

  1. responds to the event
  2. tests a condition that ensures idempotence
  3. raises an explicit event in the postlude for which the second rule is listening

For example, in the Fuse system, we want to ensure that each owner has only one fleet. This condition may be relaxed in a future version of the Fuse system, but for now, it seems a reasonable limitation.

There are several examples in Fuse where a guard rule is used. The following is the guard rule for the Fuse initialization:

rule kickoff_new_fuse_instance {
  select when fuse need_fleet
  pre {
    fleet_channel = pds:get_item(common:namespace(),"fleet_channel");
  if(fleet_channel.isnull()) then
    send_directive("requesting new Fuse setup");
  fired {
    raise explicit event "need_new_fleet"
      with _api = "sky"
       and fleet = event:attr("fleet") || "My Fleet";
  } else {
    log ">>>>>>>>>>> Fleet channel exists: " + fleet_channel;
    log ">> not creating new fleet ";

The guard rule merely looks for a fleet channel (evidence that a fleet already exists) and only continues if the fleet channel is null.

The second rule does the real work of creating a fleet pico and initializing it.

rule create_fleet {
  select when explicit need_new_fleet
  pre {
    fleet_name = event:attr("fleet");
    pico = common:factory({"schema": "Fleet", "role": "fleet"}, meta:eci());
    fleet_channel = pico{"authChannel"};
    fleet = {"cid": fleet_channel};
    pico_id = "Owner-fleet-"+ random:uuid();
  if (pico{"authChannel"} neq "none") then
    send_directive("Fleet created") with
      cid = fleet_channel;
    // tell the fleet pico to take care of the rest of the initialization.
    event:send(fleet, "fuse", "fleet_uninitialized") with
      attrs = {"fleet_name": fleet_name,
               "owner_channel": meta:eci(),
               "schema":  "Fleet",
               "_async": 0  //complete this before we try to subscribe below
  fired {
    // put this in our own namespace so we can find it to enforce idempotency
    raise pds event new_data_available 
      with namespace = common:namespace() 
       and keyvalue = "fleet_channel"
       and value = fleet_channel
       and _api = "sky";
    // make it a "pico" in CloudOS eyes
    raise cloudos event picoAttrsSet
      with picoChannel = fleet_channel 
       and picoName = fleet_name
       and picoPhoto = common:fleet_photo 
       and picoId = pico_id
       and _api = "sky";
    // subscribe to the new fleet
    raise cloudos event "subscribe"
      with namespace = common:namespace()
       and  relationship = "Fleet-FleetOwner"
       and  channelName = pico_id
       and  targetChannel = fleet_channel
       and  _api = "sky";
    log ">>> FLEET CHANNEL <<<<";
    log "Pico created for fleet: " + pico.encode();
    raise fuse event new_fleet_initialized;
  } else {
    log "Pico NOT CREATED for fleet";

When this rule fires, an action sends an event to the newly created fleet pico that causes it to initialize and three events are raised in the postlude that cause further initialization to take place.