Summary

I'm reimplmenting KRE, the evaluation engine that makes picos work. Here's the initial plan and a call for (paid) help.

The mountains, the lake and the cloud

The Kynetx Rules Engine, or KRE, is the evaluator that implements persistent compute objects (picos) and evaluates KRL inside them. Picos are actors that are first-class Internet citizens. Thus, KRE supports an actor-model for programming the Internet of Things. Picos are the underlying infrastructural element for my ideas on social things.

The current implementation of KRE is an Apache module written in Perl5. Apache connects KRE to the Web and functions as a application server. Mark Horstmeier and I wrote it over the past eight years with help from Cid Dennis, Sam Curren, Wade Billings, and others.

I believe that for KRL to be widely used in the Internet of Things, the current implementation will need to be replaced with something much faster. KRE is an excellent test bed and pilot implementation that has hosted multiple production products. But there are some products we avoided due to performance and security issues.

Here are the principles I think should govern a reimplementation:

  • Speed — The1 new evaluator needs to be several orders of magnitude faster. I believe this is possible because KRE is implemented as a naive, AST-directed interpreter with few optimizations.
  • Internet first — The new evaluator must retain KRE's Internet-first stance. That means that picos must remain first-class Internet citizens. Consequently, picos must
    • be capable of receiving events and queries on Internet channels,
    • interact with other picos, even if they're hosted on another instance of the evaluator somewhere else on the Internet, and
    • call and be called from Internet-hosted APIs.
  • Small deployments — KRE is a pretty complicated beast to build and deploy. Not only does it require Apache, but also over 100 Perl modules, memcached, MongoDB, and Java (for ANTLR). The new evaluator should be fully containerized and capable of being deployed in relatively small execution environments such as home WiFi routers
  • Attribute-Based Event Authorization — KRE has a simple authorization scheme that has served the products built on it. An attribute-based event authorization scheme would increase the kinds of uses to which picos could be put.

This reimplementation won't entail significant changes to the overall functionality of either picos or KRL beyond those required to meet the preceding guidelines. There will be some cleaning up of syntactic warts. The current parser is mostly good enough.

I'm inclined to a bytecode interpreter-based implementation. I think targeting the JVM is a good starting point, but I've not done enough research to know if that's a good decision or not. Perhaps some other bytecode interpreter would serve better. I don't want to write my own bytecode interpreter if I can help it.

A bytecode interpreter fits well with KRE's current architecture. For example, KRL rulesets are hosted online (e.g. Github) and compiled and optimized on demand. The engine then uses this cached intermediate representation to respond to events and queries. As a result, the parsing and translation of KRL programs could be easily separated from the evaluation of the bytecode when an event or query is received by a pico. That supports small deployments since compile functions could be hosted in the cloud.

The following picture shows the Compile pipeline. KRL is ingested on one end. Bytecode is produced as a result. In between, KRL is parsed, optimized, and analyzed before being translated into bytecode. In addition, the pipeline creates the salience graph, the data structure the scheduler uses to determine which rules are applicable to a given event.

Compile Pipeline

Once the bytecode is created it would be used by both the Event Evaluation pipeline and the Query Evaluation pipeline. The following diagrams show those pipelines. The event (or query) decoding creates a decode object and evaluation context. The determination of which rules are scheduled or which functions respond is made at runtime. For the Event Evaluation pipeline, the execution of a rule may cause additional rules to be scheduled.

Event Evaluation Pipeline Query Evaluation Pipeline

I have money to proceed with this project. Right now I'm in the planning stage. I'm looking for someone who is interested in helping with the planning and implementation. The position is a contract position that will last for at least one year. I prefer someone who can be local (i.e. sit near me at BYU). If you're interested in getting paid to build a programming language, let me know.