A Programming Model for the CloudOS


Summary

The CloudOS and the persistent data objects it runs on depend on a specific programming model. KRL is the only programming language that supports that model right now. This blog post discusses how we could expand the underlying rules engine to support running JavaScript programs on the CloudOS.

Programmer's aid

In Programming the Cloud With Persistent Data Objects, I describe why I think KRL is a great language for building persistent data objects (PDOs). What didn't come through in that post strongly enough is that it isn't KRL that's special, rather it's the programming model that KRL supports. In other words, you could write programs for the CloudOS with other languages provided they support the key ideas of the programming model. This post lays out ideas and design for using Javascript, but other languages would work as well.

The Programming Model

Before we discuss how Javascript could be used to create programs for the CloudOS, let's review the programming model for the CloudOS. In the post mentioned above, I wrote about the three important features that KRL has that support the CloudOS programming model that gives rise to event-based PDOs. I've modified this slightly from what appeared in the preceding post so as to better focus on the model instead of the language:

  1. Entity Orientation--The programming model of KRL has identity as a core feature. While it is true that every program regardless of programming language ultimately executes on behalf of a particular entity, in KRL, that entity is known and built into the underlying semantics of the language. The entity orientation of KRL is supported by the underlying KRE (Kynetx Rules Engine) and so is usable by any program running in the engine—even one not written in KRL. The next two features illustrate why identity is crucial to the programming model.

  2. Event Binding--rules in KRL bind event patterns to actions. Event patterns are specified using event expressions. Events and actions are both extensible so that programmers are free to define events and actions that are relevant to their problem space.

    Every event is raised on behalf of a particular entity and thus a selected rule runs on behalf of that same entity. We call this concept "salience." An event is salient for a given entity if that entity has installed a rule that listens for that event.

  3. Persistent Data Values--KRL has a class of variables called "persistent variables" or just "persistents". There are two kinds of persistents: application variables and entity variables. Both are closed over the ruleset they are in, meaning that they are only visible to code executing within the ruleset. Application variables are stored for the ruleset and are available to any entity executing the ruleset. Entity variable values are only visible to the entity for whom they were stored. Think of application variables as roughly analogous to class variables. Entity variables are like instance variables.

    Entity variables, in particular, are a very powerful concept since they provide KRL programmers with the ability to store persistent values without the headache of configuring, linking, and using a database for most things. Because a ruleset represents a closure over its entity variables, every ruleset potentially represents a persistent data object. The Kynetx CloudOS takes advantage of this feature in implementing each of the important services underlying its overall functionality.

These features make up a programming model that is considerably different than the programming model for a Web 2.0 application or a mobile application. This is a programming model for the CloudOS and the PDOs it supports. As a result of these features, the KRL programming model is inherently multi-tenanted and easily gives rise to collections of ultra-lightweight virtual machines that we're calling a PDO. The CloudOS provides a programming API to programs running in one of these PDOs and as long as another programming language supports the programming model, it will also be able to use any or all of the services that the CloudOS provides.

Supporting the KRL Programming Model in JavaScript

I happen to believe that the programming model described above is important for any system that supports the notion of personal clouds. Whether it's built in KRL or on Kynetx technology, I think this architecture has significant benefits that include supporting privacy by design, providing a locus of control for the user, and being particularly attuned to creating large decentralized networks that model real-world situations with greater fidelity.

That said, I don't think that KRL, as a language, is necessary to support the programming model. Let's explore how we might be able to support the features from the last section in JavaScript.

First, let's make the following mappings between terms in KRL and terms in JavaScript:

KRL JavaScript
ruleset object
rule function
action function

Now for some specific ideas around supporting the KRL programming model in JavaScript:

Entity Orientation - We can accomplish entity orientation for JavaScript at the system level by providing a framework for JavaScript programs such that they always execute in an environment that identifies a specific entity. The KRE (Kynetx Rule Engine) could do this. In addition, we would require libraries that support specific entity-specific features (see below for examples).

Event Binding - One way to accomplish this is for JavaScript functions to be associated with specific event expressions. Event expressions are a pattern language for events. An event expression can test for matches to a specific event, test event attributes, and match combinations of events logically, temporally, or in aggregate.

While we could redo the entire expression system for JavaScript, I think that we could incorporate KRL's event expression language inside JavaScript. The simplest way would be to create an array of maps like so (we'll use KOBJ as a namespace in this and the following examples):

KOBJ.bind_events = 
    [{"bank withdrawal where amount > 100" :
      withdrawal_notify_function
     },
     {"count 4 (bank withdrawal) within 24 hours":
      function(result) {... }
     }
    ]

The key is a string representing the event expression. The value associated with each key is a JavaScript function that takes one parameter representing any value calculated by the event expression. In this design, the system would look for the KOBJ.bind_events when the program is registered (installed), parse the event expressions (they get turned into state machines), update the salience graph for the entity, and cache the result so that the event expressions can be run whenever the entity installing the JavaScript program receives a salient event.

Persistent Data Values - Every JavaScript function selected by the event system will need to run in an environment where the entity's persistent variables are defined. One possible solution would be to place those in a KOBJ object that is available to the JavaScript. For example the following example would add 3 to the value in the entity variable y and assign the value to x:

var x = KOBJ.ent.y + 3

System Libraries - Some of system libraries that should be supported for consistency. Some system libraries are required. For example, writing programs for the CloudOS frequently requires access to meta information. Another example is the event.

Raising Events - More important than system libraries is the need for programs in the CloudOS to raise events. This could be accommodated by a raise function in the KOBJ object:

KOBJ.raise("explicit", "watch", {"time" : 5, "rid": "a15x45"});

This would raise the event explicit:watch with attributes time and rid.

Actions - Finally, rules bind events to actions. Consequently, we'll want our JavaScript programs to take action. Sometimes, that action is simply raising another event as we discussed above. Usually the action is context specific. KRL provides a set of pre-defined actions that could be made available as functions in the KOBJ object. KRL also provides the means of defining new actions. This could be accommodated using JavaScript functions that follow a specific pattern similar to user-defined actions in KRL.

While what I've outlined above is just one way we might accommodate JavaScript in KRE, it illustrates that it is possible for JavaScript to support the programming model of the CloudOS. What is also, clear is that programming JavaScript for the CloudOS would feel similar to writing a Node.js application, albeit in the specific environment of the CloudOS

Implementation

When a event is received by KRE, there are four steps required to complete the evaluation of any applicable rules:

  1. Event Handling - parse the event, establish entity information, and build the event environment that will be used during the remainder of the evaluation
  2. Scheduling - Using the salience graph, retrieve the associated event expressions, and execute them. Add the rule (JavaScript function) to the schedule if the event expression succeeds. Event expressions can be evaluated in parallel.
  3. Evaluation - Execute the rules in the schedule. This step can be done in parallel.
  4. Compile Result - create the directive document that results from the evaluations. This might or might not be returned to the event generator depending on whether the event is asynchronous or synchronous.

Here's a picture showing both KRL and JavaScript evaluators:

functional decomposition of KRE

Obviously, we could add evaluators for as many languages as we like. Ideally the evaluators would be easily plugged-in and configured.

Moving Forward

I would love to support JavaScript programs in the CloudOS. There is still significant design work to complete to make this really usable, but there isn't anything technical standing in the way of getting it done. The largest impediment at this point is resources. Since KRE is open source, I'd be open to working with a group of people who are interested in making this happen. Or you can wait. We'll get to it eventually.