A University API

BYU Logo Big

BYU has been in the API and Web Services game for a long time. Kelly Flanagan, BYU's CIO, started promoting the idea to his team almost 10 years ago. The result? BYU has over 900 services in its Web Services registry. Some are small and some are big, but almost everything has a Web service of some kind.

Of course, this is both good news and bad news. Having services for everything is great. But a lot of them are quite tightly coupled to the underlying backend system that gives rise to them. On top of that, the same entity, say a student, will have different identifiers depending on which service you use. A developer writing an application that touches students will have to deal with multiple URL formats, identifiers, data formats, and error messages.

We're aiming to fix that by designing and implementing a University API. The idea is simple: identify the fundamental resources that make up the business of the university and design a single, consistent API around them. A facade layer will sit between the University API and the underlying systems to do the translation and deal with format, identifier, and other issues.

The name "API" reflects an important shift in how we view providing services. When you're providing a service, it's easy to fall into the trap of collecting an ad hoc mish-mash of service endpoints and thinking you're done. The "I" in API is for "interface." When you're providing an interface to the university, not just a collection of services, your mindset shifts. Specifically, in designing the University API we're aiming for something with the following properties:

  • Business-oriented—the API should be understandable to people who understand how a university works without having to understand anything about the underlying implementation. Many more people know how a university works than could ever know about the underlying implementation. An API based on resources familiar to anyone who understands a University makes the API useful even to non-programers.
  • Consistent—a developer should see a consistent pattern in URL formats, identifiers, data formats, and error messages. Consistency allows developers to anticipate how the API will work, even when they're working with a new resource.
  • Completeness—over time the University API ought to be an interface to every thing at the University that works via software (which is to say everything).
  • Obvious—using the API should be obvious to anyone who understands the general principles without needing to rely excessively on documentation.
  • Discoverable—a program should be able to discover allowed state transitions, query parameters, and so on to the extent possible.
  • Long-lived—An API is like a programming language in that it is a notation, not a technology. The goal is to create something that is not only intuitive, but stands the test of time. Designing for long-term use is more difficult than designing for short-term efficiency

The fundamental business of the university doesn't change rapidly. BYU has had students, classes, and instructors for 140 years. Likely, instructors will still be teaching classes to students in 20 years. The API to a university ought to reflect that stability. This doesn't mean it won't change, but ideally the University API will evolve over a period of decades in the same way a language does. Perl 5 is quite different and much more useful than Perl 2, for example, but it's still Perl. This gives the University API an importance that an ad hoc collection of services would be hard pressed to meet.

Building a University API has multiple advantages:

  1. First, and most obviously, making the API consistent and understandable will make it easier for developers to use it in building applications. This includes developers in the Office of Information Technology, BYU's central IT department, as well as developers in other units around campus. Further, there's no reason that students and others shouldn't be able to use the APIs, where authorized (see below), to create new services and GUIs on top of the standard university systems. The University API is the heart of a great university developer program.
  2. Beyond making it easy for developers, a consistent University API eases the pain of changing out underlying systems by introducing a layer of indirection. Once a University API is in place, underlying parts of the system can be changed out and the facade layer adjusted so that the API presented to developers doesn't change, or, more likely, only changes in response to new features.
  3. A third advantage of the University API is that it provides a single place to apply authorization policies. This is a huge advantage because it allows us to apply formal, specified policy parametrically to the API rather than doing it ad hoc. This results in more consistent and accurate data protection.
  4. Finally, a University API serves as a definition for the business of the University that befits the reality that more and more of the university's business is controlled and mediated via software systems. By designing a notation and semantics that matches what people believe the University to be, we document how the University's business is conducted.

How do you get started on such a monumental undertaking? We've created a University API team that is busy discussing, designing, and mocking-up APIs for a small set of interrelated, core resources. We hope to have mock-docs for review in the next few weeks. For now, we're focused on the following set:


Others that will eventually need to be considered include /colleges, /departments, /programs, and so on. There could easily be dozens of top-level resources in a university, but that's much more manageable than 900. And when they're logical and consistent that's especially true. For now we've identified five resources that form the core of the API and touch on activities that the university cares about most.

As you'd expect a GET on any of these resources returns a collection of all the members of that resources.

GET /students

Obviously some of these collections could be very large, so they will usually be filtered and paginated. Take /students for example. By rights, this resource should include not only all the current full-time students, but part-time students, independent study students, and so on—easily tens of thousands of records. Being able to filter this list so that it's the collection of students you want (e.g. all full-time students in the College of Engineering) will be critical.

Performing a GET on a resource with an identifier in the path returns the record for that identifier.

GET /students/:id

Again, the result could be very large. In theory, a student record contains everything the university knows about the student. In his white paper on University APIs, Kin Lane listed 11 types of data that might be in a student resource without even getting to things like transcripts, grades, applications, and so on. There are dozens of sub-resources inside a resource as complicated as /students. In practice, most programmers don't want (and aren't authorized to get) all the data about a particular student. We're attacking this problem by creating useful field sets for the most popular data for any given resource.

We're also dealing with issues such as the following:

  • What are the meta values for the API (e.g. what values are appropriate for a given field) and how should we represent them in the API?
  • How do we handle sub-resources? For example, a class has set of values for prerequisites that is a complex record in it's own right. But prerequisites doesn't deserve to be a top-level resource because it's only meaningful in the context of a class.
  • Many of the identifiers for a resource (like a class) are aggregate identifiers. In the case of a course the identifier is made from the term, department, course number, and section.
  • What is the right boundary between workflow and user interface. For example, when a student drops the class and that has cascading consequences, should the client or server be responsible for ensuring the student understands those consequences?
  • How deep do we go when returning a resource? For example, when we get a class enrollment, do we return links to the student records or the data about the students? If the latter, what does that communicate to developers about what can and can't be updated in the record?

There are new issues that come up all the time. We're still thinking, designing, and planning, so if you have suggestions, we'd love to hear them.

The effort has been fun and we're anxious to make at least part of this design exercise real. Watch for projects that tackle parts of this enterprise over the coming months.

Further reading:

Suggesting Changes to Google Places

Google Maps Icon Buttons

The Fuse app lets drivers record fuel purchases. One of the features it has is populating a pulldown of nearby gas stations so that the driver doesn't have to type all that in. I buy gas at the Costco in Lehi occasionally and I noticed it wasn't in the list while the Costco in Orem was. I got to wondering how the data for Google Places (the API Fuse uses) is collected and updated.

A quick search of the Costco in Lehi and the on in Orem on Google Maps didn't show any difference. There also wasn't an obvious difference when I drilled into the reviews. It was only when I clicked on the "Edit details" link that I noticed that the Costco in Orem had the following categories:

Optician, Discount Store, Gas Station, Department Store, Wholesale Club,...

whereas the one in Lehi only showed this:

Department Store

Fortunately, Google takes user comments on business listings, so I clicked "add category" and added "Gas Station." The change doesn't show up right away since Google reviews these types of changes.

So, the sequence for updating a business categorization:

  1. Find the business on Google Maps
  2. Click the "N reviews" link (or the "Write a review" link)
  3. Click "Edit details" under "Contact Information"
  4. Click the ">" to the left of "Category" to expose the category entry fields
  5. Edit as needed
  6. Submit

All in all, this is not easy to discover without some poking around. I'd tried looking from my phone a few times and never got far enough to make a change. But, once you know the secret, it's not too hard. I'm surprised that Costco doesn't do a better job of categorizing its stores in things like Google Places.

The Dangers of Internet Voting

I Voted!

Current computer operating systems, Internet protocols, and standard computing practices are inherently insecure. Recent news stories about break-ins at Target, UPS, Community Health Systems, and the Nuclear Regulatory Commission point out all too well both the weaknesses in the system and the power of hackers to take advantage of those weaknesses. Groups mounting attacks include both state and criminal actors. Yet in spite of this inherent insecurity, the Internet has become an indispensable tool for myriad activities. Consider why.

In many cases, we’re able to get around the inherent insecurity of the Internet because the value to be received from attacking a weakness isn’t sufficient to attract the attention of those who exploit these weaknesses for gain.

In other cases there is significant value to be gained by attacking a system. Yet despite that, the use of the Internet for commerce, enterprise systems, data dissemination, and other activities continues to grow because the rewards for using the system outweigh the risks and those risks are mitigated by other factors.

Similarly, Internet voting presents a valuable target for hackers. Elections have consequences and the ability to influence an election is enticing to those who have a stake in the outcome of an election. The list of potential attackers is large: individual hackers, political parties, international criminal organizations, hostile foreign governments, or even terrorists have a stake in the outcome of elections and can be expected to use weaknesses in the voting system to gain influence or simply cause mischief.

Proponents of Internet voting point out the great benefits to be gained from making voting easier. They point out that the Internet has been used to great benefit in other activities. Specifically, the refrain, “if we can shop online, why can’t we vote online?” is frequently heard. After all, online shopping and other activities continue to grow in spite of security problems.

Online voting has three properties that, taken together, set it apart from other online activities like shopping:

  1. Secret ballots are required — We require that people register to vote, but how they vote is kept secret from election officials and others. This measure protects the validity of the vote by making it more difficult to coerce or pay people to vote a particular way and by ensuring people that they won’t have to answer to others for their decisions in the polling booth. Internet voting initiatives have to ensure secret ballots.
  2. Computing environment is uncontrolled — Online voting would have to allow people to vote from their own devices in their own homes or businesses to have the desired impact. But a study in 2010 showed that 48% of 22 million scanned computers were infected with a virus and “over a million and a half [were] infected with crimeware/banker trojans.” Any Internet voting system has to be able to run on a collection of computers that is not only not under the control of the voting authorities, but not wholly under the control of the voter either.
  3. Margin for error is very small — Elections are often decided by very small margins. Unlike a business transaction where the likely hood of fraud can be statistically calculated and then factored in to the cost of doing business, there is no margin in a voting scenario to use in mitigating fraud. The margin for fraud has to be very near zero.

These three properties of voting, taken together, make online voting a very different proposition than other activities that we regularly undertake online.

To see why, consider the problem of ensuring the integrity of the vote. Vote integrity is particularly important because people will not trust a government when they don’t believe that the results of elections are valid.

The only reason we know about security breaches at Target and others is because the system is, by design, transparent and auditable. Even if these companies were unable to prevent an attack, it was abundantly clear after the fact that a breach had occurred. In a voting system, however, the secret ballot and uncontrolled computing environment combine to make auditing the validity of the vote impossible.

To make the online commerce scenario analogous to online voting, the online commerce company would know that a customer had bought something but not what she’d bought or how much she’d spent except in aggregate with other purchases. Further, we have to assume the customer never receives any feedback (like a package) and thus can never verify that the order was received correctly. Under these circumstances, there’s almost no way we could ever assure ourselves that the orders the company were receiving had any correlation to the orders customers were placing.

To see why this is a problem, suppose some group claims to have altered the results of an election after the fact. Whether they have or not is immaterial because there would be no way to prove they had not. Voter confidence in the validity of the vote could be undermined without even going to the trouble of mounting an attack.

I do not believe that we can easily overcome any of these problems in the near future. Further I am confident that none of the present commercial offerings solve these problems. Consequently I believe that the risks of Internet voting sharply outweigh the benefits and will for some time to come. But you need not take my word for it. Numerous computer scientists have come out against Internet voting. In addition, an independent panel examined Internet voting for the Province of British Columbia and concluded:

Do not implement universal Internet voting for either local government or provincial government elections at this time. However if Internet voting is implemented, it should be limited to those voters with specific accessibility challenges. If Internet voting is implemented on a limited basis, jurisdictions need to recognize that the risks to the accuracy of the voting results remain substantial.

I strongly urge the committee to curtail Internet voting initiatives for the time being. The pressure to do something might be great, but having studied the issue, we must be the ones to educate others on why Internet voting is not for Utah.

Fuse Version 1 Candidate Release for 20140815

Colour: The spice of life

On Friday I released new code that serves as the latest candidate for Fuse version 1. Here are some of the things that were included:

  • Maintenance API — the maintenance API was completed. The maintenance API contains queries and services for managing reminders, alerts, and maintenance history.
  • Fix Refresh Token Refreshing — Refresh token refreshing is more robust now and integrated with the "fleet as repository for OAuth tokens" model of linking Fuse to Carvoyant.
  • Refactor Weekly Report — The weekly report now uses a separate function to get the fleet summary. This new function will also be used for generating exportable CSV files for taxes and other use cases.
  • Name changes — some query and event names were changed to be more consistent.

There have also been changes to Joinfuse.com:

  • Add status — the provisioning app now shows the status of the link between vehicles and Carvoyant as well as some basic data about the vehicle.
  • Version choice — there are both product and development version of the service. Joinfuse now recognizes which the user is attached to and uses the correct service.

In addition, the javaScript SDK and it's documentation have been updated to match changes to the underlying service.

Extending and Using Fuse

fuse trio

Fuse is a connected car system. You might have noticed that there are a bunch of connected-car products on the market. There are several things that set Fuse apart from other connected-car products, but there's one that's especially important: Fuse is extensible.

When I say "extensible" I don't just mean that Fuse has an API, although it does have that. Fuse is extensible in four important ways:

  1. Fuse has an API — the Fuse API allows anyone to write applications that use Fuse.
  2. The Fuse API is user-extensible — Anyone can write services that Fuse users can install to extend the capabilities of Fuse.
  3. Fuse is open-source — Not only is Fuse open-source, it's based on an open stack including KRE and CloudOS. Because it's open-source, you can replace or modify it at will
  4. Fuse can be self-hosted — being able to self-host means that people have choices about their data and who controls it.

Fuse's extreme extensibility has important implications:

  • User Control of Data — the architecture necessary to create Fuse's extensibility also supports selective data sharing through relationships with it's owner, manufacturer, drivers, and others.
  • Openness and Interoperability — Fuse is a model for an open, interoperable Internet of Things rather than the closed Compuserve of Things that vendors are currently offering.
  • Future Growth — Fuse can change and grow as connected-car products come and go. Fuse owners are not solely dependent on Kynetx to make Fuse work with new ideas, products, and APIs but can take matters into their own hands.

The following describe some of the ways that Fuse can be extended.

Using the Fuse API to Build an App

Fuse has an API that, much like any other API, accesses the core functions of the Fuse platform. Using the Fuse API, developers can add connected-car features to existing applications or even completely replace the stock Fuse app with something more to their liking.

The Fuse API uses OAuth so that developers can let users link their Fuse fleet to another app or service in a standard way. Fuse provides a JavaScript SDK, but you can use any language to access the API so long as it supports HTTP.

The API isn't, strictly speaking, RESTful. Instead it's an event-query API. This is a result of the Fuse architecture and is necessary to support the more advanced forms of extensibility I describe below. We're still experimenting with Event-Query APIs to determine how best to design, use, and document them.

Extending the API with New Services

Fuse is constructed of persistent compute objects, or picos. You can think of a pico as a container for microservices. There is a pico for the owner, for the fleet, and for each of the vehicles in the system as shown below:

fuse microservice overall

Because each of these picos can contain different sets of services, each behaves differently and presents a unique API to the world. Note that this is not just true of a class of picos (i.e. "owner" picos have a different API than a fleet pico) but of each individual pico as well. That is, my Fuse API could be distinct from yours based on the set of services that are installed. In this way, they feel more like personal computers than traditional Web 2.0 web services that present the same, single, non-extensible API for everyone.

Consequently, developers can build and distribute their own services on the Fuse platform. If the Fuse API doesn't do what you want, you can write an extension of the API that your users can install to enable that service.

One example of why you might do this is to add support for devices besides the Carvoyant devices that Fuse is based on now. You could, for example, add a service to Fuse that uses the Automatic device instead. Because each pico is a separate service container, you would then be able to have Carvoyant devices in some of your vehicles and Automatic devices in others and still see them in a single app with consistent vehicle and fleet functionality.

Replacing Existing Services

A direct consequence of the ability to extend the existing API and the fact that it's open source is the ability to replace parts of it wholesale. This allows keeping the old API for interoperability with other apps while completely changing out the code below it.

You could, for example, fork an existing service ruleset on Github and fix bugs, extend it's functionality. If the Fuse maintenance service doesn't suit your needs, but you want keep the existing API so that apps can continue to use it, it's possible to simply replace the maintenance service with one of your own.

Your new maintenance service could be just for you (i.e. installed on your own Fuse picos) or distributed more widely for other Fuse owners to use.

Hosting Fuse

Our goal with Fuse is to create a connected-car platform that belongs completely to the owner of the vehicles. We've architected Fuse to respect privacy and let people choose what happens to the data about their cars and yet supports sharing and data transferability at the same time.

One important component of owner-control is the ability to self-host, even if most people don't take advantage of it. As an analogy, the fact that I can self-host email, if I choose, is an important part of the control I feel over my email account, even though I might choose to let someone host email for me at the moment.

Fuse is open source and is based on an open-source software stack:

  • Fuse and it's underlying pico architecture are based on an open source pico container system called KRE (GPL license) that runs a language called KRL. KRE runs on Apache. Anyone can install and run KRE anywhere they like.

  • Fuse uses a pico-management system we call CloudOS that will be open-source (as soon as I find time to fix some code issues). CloudOS provides core functionality for picos like lifecycle management, communications, and storage.

  • The software that provides Fuse functionality, the Fuse API, is also open-source (MIT License).

  • The Fuse app is not yet open source, but will be once it's done.

Using these freely available packages, you can not just self-host Fuse, but set up your own Fuse system if you choose.

One important benefit of self hosting is users are not even beholden to the account system that we're using for Fuse. They can use the code to create their own accounts in a system they control.


We've tried to make Fuse the most open, extensible connected-car system available. Extensibility is the key to Fuse giving people better control over their data, being interoperable with a wide variety of services and things, and being able to adapt to future changes.

Fuse and Kynetx

Fuse is an open-source project that's supported by Kynetx. Kynetx is behind KRL, KRL, CloudOS, and Fuse. Kynetx makes money by supporting Fuse and through selling Fuse devices and hosting.

Getting Involved

The easiest way to get involved is to use the API. The Fuse App will be opened up to other developers soon and we welcome help in developing it and adding new features. If you're interested in extending the API or running your own Fuse system, contact me and I'll point you in the right direction.

Blockchain and Bearer Tokens

Bitcoin keychain/keyring and key

One of the problems with most substitutes for email is that they fail to implement a concept that Marc Stiegler of HP writes about in his technical report Rich Sharing for the Web. Narc outlines six features of rich sharing that are captured in this short scenario:

Alice, in a race to her next meeting, turns thunder-struck to Bob and says, “Bob, I just remembered I need to get my daughter Carol’s car to Dave’s repair shop. I’ve got to go to this meeting. Can you take Carol’s car over there?”

Marc's thesis is that email has held on so long because it's one of the few systems that supports all six features of rich sharing. But that's not really what this post is about, so I won't describe that further. You can read Marc's excellent white paper from the link above if that interests you.

As I was contemplating this scenario this morning, I was thinking that part of what makes this work is the idea of bearer tokens. A car key is essentially a bearer token. Anyone who has the key can open, start, and drive the car. Hence Carol's daughter can delegate to Carol by giving her the key, as can Carol to Bob, and then Bob to the mechanic.

The problem with bearer tokens is that they can be easily copied. If I give you the "key" to my account at Dropbox in the form of the OAuth2 bearer token that authorizes access, we both have a copy. In fact you could put it on a web site and everyone would be able to get a copy. OAuth2 bearer tokens don't work like car keys.

The key difference is that car keys are fermions, not dataions. That is, they can be in exactly one place at one time whereas bearer tokens can be in multiple places at the same time. Sure we can make a copy of the car key, but that takes work. Exchanging keys (Carol giving it to Bob) takes work. They have to arrange to meet or something else. The concept of work is critical.

One of the key features of the bitcoin blockchain is that it prevents double spending. That is, even though the data representing a coin can be in many places, only one person can spend it. And, importantly, can only spend it once. This seems like a property we'd want bearer tokens to have.

The simple concept is to put bearer tokens in a distributed ledger like the blockchain so that we only allow the current holder of the token to use it. Checking if someone is the current holder of the token is easy since everyone can have a copy of the ledger. But transferring takes work in the same way that transferring a bitcoin takes work (that's what all the "bitcoin mining" is really accomplishing).

In fact we could probably just use bitcoins as tokens. When Alice authorizes Bob to access her account on my system, I'll send a small amount of bitcoin to Bob. When Bob accesses the system, he presents the coin (note, he just has to show it to my system, not spend it or transfer it) and I can check that it's the right coin and that Bob is the current holder. If Carla presents the token (coin), I can check that she doesn't own it and refuse service.

A few thoughts:

  • There's nothing in this system to prevent Bob from transferring the coin to Carla. That is, after Alice gives the token to Bob, she has no control who he transfers it to.
  • This system costs real money. That is, bitcoins, no matter how small, have value. But that's a feature, not a bug, as the saying goes. The cost makes bearer tokens behave more like physical keys.

This is open loop thinking, but I'd appreciate your thoughts and feedback on how a system like this might work better and what problems it might solve or create.

What Happens to the Data

Silos de Trigueros

Nathan Schor pointed me at an article about Metromile that appeared in TechCrunch recently. Metromile is a per-mile insurance company that uses a OBD II device that you plug in your car. It tracks your vehicle stats, similar to Fuse, Automatic, and other connected car services.

The kicker is that it's free because Metromile is making money by selling per-mile insurance. The more users they have using their device the bigger their potential market for selling insurance. That is made evident by the fact that you can only get the free device if you live in a state where they offer insurance (currently CA, OR, and IL). Otherwise, get in line (until they come to your state, presumably).

I don't know how Metromile is implemented, but I wonder what happens to the data. I'm pretty sure they're using a cellular device (rather than Bluetooth) so that the data is always transmitted to their system even if your phone's not in the car or connected. Does all the data about every trip go to the insurance company? Or some aggregation? What's the algorithm?

These questions are relevant because it's unclear who ultimately owns this data. Users aren't paying for the device or the data, just the insurance. As I wrote in The CompuServe of Things, business models that connect devices to non-substituable services threaten to leave users with little control over the things they own and use.

I believe users ought to be customers who own the data and control where and how it's used. That doesn't mean they can't choose to share it with the insurance company, but they ought to know what's being shared and even be able to substitute one insurance company for another. If every connected car device is associated with a different insuarance company, I can't switch without having to give up access to all the data that's been collected about my car and driving.

Data silos with murky policies about data ownership are all too common. Unfortunately, they lead to a future I don't want to live in. And if you think about it, I'll bet you won't want to live there either.

A Microservice for Healing Subscriptions

Such Great Lows

Last week I wrote about how Fuse uses a microservices architecture and the benefits that such an architecture provides. This morning I was faced with a problem that the microservices approach solved handily, so I thought I'd write it up as an example.

Fuse uses webhooks (we think of them as event channels) to receive notifications that a vehicle has started or stopped, has a low battery, etc. If these subscriptions aren't set up for a vehicle nothing works. Most noticably trips aren't recorded. Alex, who's working on the Fuse app, wasn't seeing any trips from his truck. Sure enough, when I checked, it had no Fuse event subscriptions.

I could have just reinitialized Alex's truck, but I figured if it happened to Alex then it's likely to happen to other people, so I created a simple microservice (i.e. KRL rule) that checks the number of subscriptions and if it's lower than the expected number, reinitializes them.

rule check_subscriptions {
  select when fuse subscription_check
  pre {
    vid = carvoyant:vehicle_id(); 
    my_subs = carvoyant:getSubscription(vid);
    should_have = required_subscription_list.length();
  if(my_subs.length() < should_have) then
    send_directive("not enough subscriptions") with
      my_subscriptions = my_subs and
      should_have = should_have
  fired {
    log ">>>> vehicle #{vid} needs subscription check";
    raise fuse event need_initial_carvoyant_subscriptions;
  } else {
    log ">>>> vehicle #{vid} has plenty of subscriptions";

The rule uses a pre-existing function to get the current subscriptions and compares the length of that result to the number we should have. If there aren't enough the rule fires and raises the fuse:need_initial_carvoyant_subscriptions.

This is a really simple rule. One reason is because it makes use of other services that already exist. There's already a working function for getting the current subscriptions for a vehicle. There's already another service (or rule), called initialize subscriptions, that sets up the subscriptions when they're needed.

Another reason the rule is simple is because it doesn't have to figure out which subscriptions are missing and limit itself to only initializing those. If any are missing, it asks for them all. That's because the initialize subscriptions rule is idempotent. You can run it as many times as you like without messing anything up. Of course, I'd rather not put that load on the system if I don't have to, so the check_subscriptions rule checks if something needs to be done before it raises the event.

The primary point is that the microservice architecture is loosely coupled and so setting up a service like this is easy. There's very little code, it makes use of other services, and it's unlikely to break anything. I wired it into the system by raising the event it looks for, fuse:subscription_check when the vehicle profile is updated. That seems like a good compromise between over checking and user control.

Idempotent Services and Guard Rules

Microservices are usually easier to program when responses to an event are idempotent, meaning that they can run multiple times without cumulative effect.

Many operations are idempotent (i.e. adding a ruleset to a pico over and over only results in the ruleset being added once). For operations that aren't naturally idempotent, we can make the rule idempotent using the rule's guard condition. Using a guard condition we can ensure the rule only fires when specific conditions are met.

Unfortunately, there are some functions in KRL (notably in the PCI and RSM modules) that make state changes (i.e. have persistent effect). These modules are used extensively in CloudOS. When these are used in the rule prelude, they cause side effects before the rule's guard condition is executed. This is a design flaw in KRL that I hope to rectify in a future version of the language. These functions should probably be actions rather than functions so that they only operate after the guard condition is met.

In the meantime, a guard rule offers a useful method for assuring idempotency in rules. The basic idea is to create two rules: one that tests a guard condition and one that carries out the rule's real purpose.

The guard rule:

  1. responds to the event
  2. tests a condition that ensures idempotence
  3. raises an explicit event in the postlude for which the second rule is listening

For example, in the Fuse system, we want to ensure that each owner has only one fleet. This condition may be relaxed in a future version of the Fuse system, but for now, it seems a reasonable limitation.

There are several examples in Fuse where a guard rule is used. The following is the guard rule for the Fuse initialization:

rule kickoff_new_fuse_instance {
  select when fuse need_fleet
  pre {
    fleet_channel = pds:get_item(common:namespace(),"fleet_channel");
  if(fleet_channel.isnull()) then
    send_directive("requesting new Fuse setup");
  fired {
    raise explicit event "need_new_fleet"
      with _api = "sky"
       and fleet = event:attr("fleet") || "My Fleet";
  } else {
    log ">>>>>>>>>>> Fleet channel exists: " + fleet_channel;
    log ">> not creating new fleet ";

The guard rule merely looks for a fleet channel (evidence that a fleet already exists) and only continues if the fleet channel is null.

The second rule does the real work of creating a fleet pico and initializing it.

rule create_fleet {
  select when explicit need_new_fleet
  pre {
    fleet_name = event:attr("fleet");
    pico = common:factory({"schema": "Fleet", "role": "fleet"}, meta:eci());
    fleet_channel = pico{"authChannel"};
    fleet = {"cid": fleet_channel};
    pico_id = "Owner-fleet-"+ random:uuid();
  if (pico{"authChannel"} neq "none") then
    send_directive("Fleet created") with
      cid = fleet_channel;
    // tell the fleet pico to take care of the rest of the initialization.
    event:send(fleet, "fuse", "fleet_uninitialized") with
      attrs = {"fleet_name": fleet_name,
               "owner_channel": meta:eci(),
               "schema":  "Fleet",
               "_async": 0  //complete this before we try to subscribe below
  fired {
    // put this in our own namespace so we can find it to enforce idempotency
    raise pds event new_data_available 
      with namespace = common:namespace() 
       and keyvalue = "fleet_channel"
       and value = fleet_channel
       and _api = "sky";
    // make it a "pico" in CloudOS eyes
    raise cloudos event picoAttrsSet
      with picoChannel = fleet_channel 
       and picoName = fleet_name
       and picoPhoto = common:fleet_photo 
       and picoId = pico_id
       and _api = "sky";
    // subscribe to the new fleet
    raise cloudos event "subscribe"
      with namespace = common:namespace()
       and  relationship = "Fleet-FleetOwner"
       and  channelName = pico_id
       and  targetChannel = fleet_channel
       and  _api = "sky";
    log ">>> FLEET CHANNEL <<<<";
    log "Pico created for fleet: " + pico.encode();
    raise fuse event new_fleet_initialized;
  } else {
    log "Pico NOT CREATED for fleet";

When this rule fires, an action sends an event to the newly created fleet pico that causes it to initialize and three events are raised in the postlude that cause further initialization to take place.

Fuse as a Microservice Architecture

I recently ran across the idea of microservices. I don't know why it took me so long to run across it, since they've been discussed for a few years and there are many article written about them (see the end of this post for a link list). They have, like anything new, many different definitions, but I've settled on a few characteristics that I think differentiate a microservice from other styles of architecture:

  1. Organized around a single business capability
  2. Small, generally less than 1000 lines and code and usually much smaller
  3. Event-based and asynchronous
  4. Run in their own process
  5. Independently deployable
  6. Decentralized data storage

What struck me as I reviewed various material on microservices is how much the philosophy and architectural style match what I've been preaching around persistent compute objects (picos). As I worked through the ideas, I came to the conclusion that since you can view each pico as an event bus, we can view each rule installed in the pico as a microservice.

With this lens, Fuse can be seen as an extensible, microservice architecture for connected cars.


Fuse is a connected-car platform. I've written extensively on Fuse in this blog. For the purposes of this post, it's important to understand the following:

  • Fuse uses Carvoyant to manage devices and provide an API from which we use to get vehicle data. The Carvoyant API is a well-designed RESTful API that uses OAuth for user authorization.
  • Picos use a set of pre-built services that collectively I call CloudOS to manage things like creating and destroying picos, pico-to-pico subscriptions, storing profiles, etc.
  • Rules are collected into rulesets that can share function definitions.
  • Each ruleset has a separate persistent key-value store from every other ruleset.
  • Rules are programmed in a language called KRL.
  • When we create a pico for a vehicle, the pico is automatically endowed with an event bus that connects all rules installed in the pico.
  • CloudOS provides a ruleset that functions as a persistent data store for the entire pico called the PDS. The PDS provides a standard profile for each pico. Fuse stores all of the vehicle's configuration and identity information in the pico profile.
  • Other vehicle data is stored by the individual service. For example, the trip service stores information about trips, the fuel service stores information about fuel purchase, and so on.
  • Rules can use HTTP to interface with other Web-based APIs.

Not only do we create a pico for each vehicle, but we also create one for each owner, and one per owner to represent the fleet. They are organized as shown below.

fuse microservice overall

This organization provides essential communication pathways and establishes a structural representation of ownership and control.

Example Interactions

Microservices should be designed to be tolerant of failure. Let me walk through one place that shows up in Fuse. As mentioned above, Carvoyant provides an API that allows Fuse to interact with the OBD II device and it's data. To do that, we have to mirror the vehicle, to some extent, in the Carvoyant API. Whenever we create a vehicle in Fuse, we have to create one in Carvoyant. But there's a small problem: before the vehicle can be created at Carvoyant, the owner needs a Carvoyant account and that account needs to be linked (via OAuth) to their Fuse account.

One way to solve this problem would be to ensure that a vehicle can't be added to Fuse unless a Carvoyant account is active. Another way is to be more tolerant and let users add vehicles any time, even before they've created their Carvoyant account, and sync the two accounts pro actively. This has the added benefit of working the other direction as well. If someone happens to already have a Carvoyant account, they can add their vehicle to Fuse and the two accounts will sync up.

The following diagram shows a few of the microservices (rules) in the vehicle pico that help perform this task.

fuse microservice

(click to enlarge)

As I mentioned above, the vehicle configuration is stored in the PDS profile. Whenever the profile is updated, the PDS raises a pds:profile_updated event. So anytime the user or a process changes any data in the vehicle profile, this event will be raised.

Any number of rules might be listening to that event. One rule that listens for that event is carvoyant initialize vehicle (lower left of the preceding diagram). Carvoyant initialize vehicle is a fairly complicated service that ensures that any vehicles in Fuse and Carvoyant are represented, if possible, in the other service. We'll examine it in more detail below. When it's done, carvoyant initialize vehicle raises the fuse:vehicle_uninitialized if the vehicle is changed.

Also, when the carvoyant initialize vehicle rule contacts Carvoyant to either update or create a vehicle, it does so with an http:post(). Picos are designed to automatically raise an event with the results of the post when it completes. The initialization OK rule (lower center) listens for that event (only with an HTTP 200 status) and store various information about the vehicle at Carvoyant. This is what establishes a link between the vehicle pico and the vehicle at Carvoyant.

Independently, the initialize vehicle rule (upper left in the diagram) is listening for the fuse:vehicle_uninitialized event. Initialize vehicle primarily functions to raise other events and thus sets off a collection of activities that are necessary to bring an uninitialized vehicle into the system and make it function. Since it's now been connected to Carvoyant, the vehicle needs to

  1. subscribe to events from the Carvoyant system including events that communicate the ignition status, fuel level, battery level, and any diagnostic codes,
  2. retrieve the current status of various vehicle systems, and
  3. retrieve specific data about the vehicle's geoposition.

The initialize vehicle rule doesn't ask for all these on it's own, it merely signals that they're needed by raising events. Other services respond by carrying out one simple task.

For example, the update vehicle data rule in the upper right makes calls to the Carvoyant API to gather data from the vehicle, communicates that data to the fleet pico (i.e., raises an external event from the standpoint of the vehicle), and raises various events including the pds:new_data_available event.

The PDS add item rule (lower right corner) is listening for the pds:new_data_available event. It takes the data in the event and stores it in the vehicle PDS. We chose to store the vehicle data in the PDS instead of the update vehicle data rule because it's widely used and is more widely available in the PDS.

Of course, these are only a few of the rules that make up the entire system. There are at least 60 in the vehicle pico and many more in the fleet. Most of these rules are quite simple, but still perform a vital task.

One important note: each vehicle is represented by a unique pico and this has it's own copy of each of these rules, running against the unique data for that vehicle. The rule doesn't know or care about any vehicles except the one that it's installed in. As such, it is a dedicated service for that specific vehicle. In a system with 10,000 vehicles, there will be 10,000 independent carvoyant initialize vehicle microservices running, listening for pds:profile_update events from the PDS in the pico in which it runs.

A Detailed Look at A Service Rule

The carvoyant initialize vehicle rule performs initialization, but it has to be flexible. There are three basic scenarios:

  1. The Fuse vehicle pico has a representation in Carvoyant and thus any changes that have been made in Fuse merely need to be reflected back to Carvoyant.
  2. The Fuse vehicle pico has no Carvoyant representation but there is a Carvoyant vehicle that matches the Fuse vehicle in specific attributes (e.g. the vehicle identification number) and thus should be linked.
  3. The Fuse vehicle pico has no Carvoyant representation and nothing matches, so we should create a vehicle at Carvoyant to represent it.

Here's the code for the rule

rule carvoyant_init_vehicle {

  select when carvoyant init_vehicle
	   or pds profile_updated

   pre {
    cv_vehicles = carvoyantVehicleData();
    profile = pds:get_all_me();
    vehicle_match = cv_vehicles
			       v{"vin"} eq profile{"vin"}  

    existing_vid = ent:vehicle_data{"vehicleId"} || profile{"deviceId"};

    // true if vehicle exists in Carvoyant with same vin and not yet linked
    should_link = not vehicle_match.isnull() 
	       && ent:vehicle_data{"vehicleId"}.isnull();

    vid = should_link                            => vehicle_match{"vehicleId"} 
	| ent:vehicle_data{"vehicleId"}.isnull() => "" 
	|                                           existing_vid;

    config_data = get_config(vid); 
    params = {
	"name": event:attr("name") || profile{"myProfileName"} || "Unknown Vehicle",
	"deviceId": event:attr("deviceId") || profile{"deviceId"} || "unknown",
	"label": event:attr("label") || profile{"myProfileName"} || "My Vehicle",
	"vin": event:attr("vin") || profile{"vin"} || "unknown",
	"mileage": event:attr("mileage") || profile{"mileage"} || "10"

  if( params{"deviceId"} neq "unknown"
   && params{"vin"} neq "unknown"
    ) then
	with ar_label = "vehicle_init";

  fired { 
    raise fuse event vehicle_uninitialized 
      if should_link || event:name() eq "init_vehicle";
    log(">> initializing Carvoyant account with device ID = " + params{"deviceId"});
  } else {
    log(">> Carvoyant account initializaiton failed; missing device ID");

This code is fairly dense, but straightforward. There are four primary pieces to pay attention to:

  1. The select statement at the beginning declaratively describes the events that this rule listens for: carvoyant:init_vehicle or pds:profile_updated
  2. The pre block (or prelude) retrieves the current list of vehicles Carvoyant knows about and filters them to see if any of them match the current vehicle. Then it calculates the vehicle ID (vid) based on that information. The calculated value of vid corresponds to the three scenarios listed above. The code also computes the parameters to post to Carvoyant.
  3. The action block is guarded by a condition that ensures that the deviceId and vin aren't empty. Assuming the guard condition is met, the rule POSTs to carvoyant (the action carvoyant_post has all the logic for making the POST with appropriate OAuth access tokens).
  4. Finally, if the rule fired (i.e. it was selected and it's guard condition was true) the postlude raises the fuse:vehicle_uninitialized event. The raise statement is further guarded to ensure that we don't signal that vehicle is uninitialized event when updating an existing vehicle.


There are a few points I'd like to make about the preceding discussion and example:

Service contracts must be explicit and honored. Because we lean heavily on independent services, knowing which services raise which events and what those events mean is critical. There have been times that I've built services that didn't quite connect because I had two events that meant the same thing and each service was looking for a different event.

Self-healing allows misses. Picos don't guarantee evnt delivery. So consider what happens if, for example, the initialization OK rule misses the http:post event and the Carvoyant vehicle information fails to get stored? The carvoyant initialize vehicle rule is tolerant of failure in that the system will merely find the vehicle again later and link it then.

Idempotency is important to loose coupling. There are multiple places where it's easier to just raise an event and rely on the fact that running a service again does no harm. The logic gets very complicated and tightly coupled when one service has to determine if it should be raising an event because another service isn't designed idempotently.

Scaling by adding services. The carvoyant initialize vehicle rule ensures that Fuse vehicles are linked to their representation at Carvoyant, but it doesn't sync historical data, such as trips. What if something gets disconnected by a bad OAuth token? Because of the microservice architecture, we can add other rules later that look at historic data in the Carvoyant system and sync the Fuse vehicle with that data. These new services can be added as new needs arise without significant changes to existing services

Picos and CloudOS provide infrastructure for creating microservices in the Internet of Things. Picos provide a closure around the services and data for a given entity. In the case of Fuse, a vehicle. By representing an entity and allowing multiple copies to be created for like entities, picos provide persistenly available data and services things whether or not they're online.

Asynchronous, one-way communication is the rule. Note that two-way communication is limited in the preceding example. Rules are not waiting for returned results from other rules. Neither is it right to think of rules "calling" each other. While the KRL system does make some guarantees about execution order, much of the execution proceeds as dictated by the event-flow, which can be conditional and change with circumstances.

Picos cleanly separate data. Picos, representing a specific entity, and microservices, representing a specific business capability within the pico, provide real benefit to the Internet of Things. For example, if you sell you car, you can transfer the vehicle pico to the new owner, after deleting the trip service, and its associated data, leaving untouched the maintenance records, which are stored in the maintenance service.

A microservice architecture means picos are extensible. Microservices provide an easily extensible API, feature set, and data model for the pico. By changing the mix of services in a given pico, the API, functionality, and data model of that pico change commensurately. There is nothing that requires that every vehicle pico be the same. They can be customized by make, model, owner, or any other characteristic.


Viewing rules as microservics within a pico have given me a powerful way to view pico programming. As I've folded this way of thinking into my programming, I've found that many of the ideas and patterns being developed for microservices can equally be applied to KRL programming within a pico.

Further Reading

For more background on microservices, here are some resources: