Phil Windley's Technometria

« November 2011 | Main | January 2012 »

December 15, 2011

Notifications in Personal Event Networks

If you're old enough to remember WordPerfect on DOS, it always came with a stack of disks for the printer drivers. That's because DOS had no printer drivers. Every application had to handle the printer itself. Every program treated the printer differently and the results were uneven at best. WordPerfect and other programs succeeded where others failed partly on the strength of their printer drivers. When Windows came along, it included printer drivers. This was a big change; once the OS knew about your printer and how to work with it, any application could print without ever worrying about the printer or how it works. I suspect WordPerfect had a dozen or more people working on printer drivers back in the day. With Windows that burden went away. (Well, not really since WordPerfect couldn't let go of DOS, but that's a different story...the burden could have gone away).

In the same way, providing event APIs for common services and then supplying rulesets that handle those services make personal event networks more powerful and easier to use. Personal event networks unleash tremendous power for apps to cooperate to provide better and more flexible services for their owners and a great deal of leverage for programmers who don't have to program all those services over and over.

A few weeks ago, I blogged about the new Sky API and what it means for KRL programmers. With Sky, KRL developers can raise events that any app a user has activated will see. In this post, I'll discuss a generalized event API for notiications. This is going to be a long post, so hang on. I'll break it up into sections that deal with specific parts of the post. You can probably stop reading after any section and have a good idea of why this matters to a certain level of specificity.

Notification Events

The Notification Event API defines an event domain, notification and two event types, status and log. Today, I'm going to focus on status events.

A status event has several attributes:

• application (optional) - name of the application/ruleset raising the notification event. If not application is given, the current ruleset ID is used.
• subject (required) - subject (title) of the notification. Typically a subject is one line.
• description (optional) - additional information beyond what is in the subject.
• priority (optional) - Every notification has a priority. If not priority is given, the priority is 0. The priorities are:
  • -2 (Very low)
  • -1 (Moderate)
  • 0 (Normal)
  • +1 (High)
  • +2 (Emergency)
• id (optional) - a unique identifier that the event raiser chooses.

Our purpose in defining the Notification Event API is to put a stake in the ground around how notifications messages will be handled in a personal event network. With such a standard, developers can write apps that manage the notifications for a user's personal event network. Other developers can raise notification events knowing that if the user wants to see notifications, they will have activated an application that handles them. In effect, the Notification Event API is the printer interface for a personal event network.

This is the key point: with this development, KRL developers don't ever have to build notifications into their apps or worry about meeting the user's demands with regard to how they want to be notified. That's someone else's job. I anticipate that we'll give everyone with a KBX a basic notification application that they can use and configure. If they don't like that one, they can replace it with another one so long as it handles events as specified in the Notification Event API

For example, the basic notification app will likely make heavy use of email and SMS, but you could imagine someone building and selling an app that uses a service like Urban Airship to send notifications to a custom mobile application that manages them, allows for responses, and so on.

Events in the Personal Event Network

This wouldn't work if every app needing to make a notification had to be configured so it knew the identifier of the app that the user wanted to handle notifications. That's why Sky is so important.

I have a Simple Notification app and a Notification Testing app activated in my Kynetx account. The test app paints a form on a page and when the form is submitted turns the fields in the form into a notifiation:status event. Because the Simple Notification app is listening for notification:status events, it responds. Nothing links them except that I have them both activated in my account. If I delete the Simple Notification app and replace it with another that understands the Notification Event API, that new app will automatically start processing notifications for me. I can have more than one app listening for notification events. moreover, I can activate a dozen or a hundred apps that raise notification:status events and they will all be handled by whatever apps I've activated to listen for those particular events.

By defining the semantics of some common APIs like the one for notifications, we can start to build out flexible services on top of personal event networks. Users are in complete control of which apps they activate to handle those services. I suspect that most people will use the standard ones that we supply, but I also hope other developers see value in creating custom rulesets to handle common events.

This isn't unlike drivers for an operating system or standard services in application server. I'd be hard pressed to overemphasize the importance of this fact. Personal event networks are real and work now.

If you've been following along for the last year, you might be scratching your head and thinking "wait a minute, I thought that the apps I activated in my Kynetx account were 'browser apps' and just affected my browsing experience!" Nope! Those apps are functioning on the personal event network you created when you signed up for a Kynetx account. It just happens that all of the ones you've activated to date, work in the browser. They don't have to.

Processing Status Notifications

The simple notification ruleset has three rules to handle status notifications: one for notifications with priority 2, one for notifications with priority 0 or 1, and one for notifications with priority less than 0.

The meta section declares three libraries we're going to use: prowl, sendgrid, and Google spreadsheets:

use module a16x83 alias prowl with 
  apikey = keys:prowl_test("apikey") and 
  providerkey = keys:prowl_test("providerkey") and
  application = "Kynetx Notification Manager"
          
use module a16x129 version "dev" alias sendgrid with
  api_user = keys:sendgrid("api_user") and 
  api_key = keys:sendgrid("api_key") and
  application = "Kynetx Notification Manager"

use module a163x73 version "dev" alias spreadsheet with
  docskey = keys:docskey();

The ruleset processes event attributes, including setting defaults, in the global block so that they're available for the various rules:

global {
  app = event:attr("application");
  subj = event:attr("subject") || 
     "Status Notification for #{app}";
  priority = event:attr("priority") || 0;
  description = event:attr("description") || 
     "A status notification with priority 
      #{priority} was received from #{app}." ;
}

Low priority notifications (less than 0) are handled by the logonly rule. The logonly rule uses a Google spreadsheet module to merely save the notification in a "logging" spreadsheet to they can be viewed later:

rule low {
  select when notification status priority "^-[12]$"
  spreadsheet:submitrow() with 
      values = [app, priority, subj, description]
}

The result of this rule firing is a logging line in a google spreadsheet that records the attributes of the notification:

(click to enlarge)

Medium priority notifications (0 or 1) are handled by the routine rule. The routine rule uses a SendGrid module to send an email to the owner of the personal event network. Right now, I'm storing the email address in an entity variable and setting it in a configuration step, but eventually that kind of data ought to be routinely available to the personal event network without multiple configurations.

rule routine {
  select when notification status priority "^[01]$"
  sendgrid:send(name, 
                ent:email_address, 
                "#{app}: #{subj}", 
                description);
}

The result of this rule fire is an email that looks like this:

High priority notifications (2) are handled by the urgent rule. The urgent rule uses Prowl and a Prowl module to push notifications to the owner's phone. SMS is more generally available and would be a better choice for widely used ruleset, but I was anxious to try out Prowl. (Kind of a poor man's Urban Airship.)

rule urgent {
  select when notification status priority "^2$"
  prowl:notify(subj, description) with priority = priority;
}

The result of this rule firing is a push notification to my phone:

The ruleset could be improved in a number of ways, including providing more configuration options for choosing what level of notifications get sent where and breaking out all five levels of notifications.

Using the Simple Notification App

As I said, any ruleset that the owner of the personal event network has activated can raise a notification:status event and the simple notification app will handle it according to the rules we just reviewed. To test it, I wrote a ruleset that places a form on a page and then raises notification:status events with the event attributes taken from the form. Here's an example of the form:

Here's the rule that handles the form submission:

rule respond_submit is active {
  select when web submit "#test_form"
  noop();
  always {
    raise notification event status with
      priority = event:attr("priority") and
      application = 
        event:attr("application") || "Kynetx Test" and
      subject = event:attr("subject") || "No Subject" and
      description = event:attr("desc") and
      _api = "sky"
  }
} 

The important part is the rule postlude (the always clause) where a notification:status event is raised with the appropriate attributes (taken from the form). Note that the raise statement does not specify the previous ruleset or any other for handling the event. That is determined automatically by the system based on what apps the owner of the personal event network has activate.

The only tricky part is the special attribute _api that tells the rules engine to use Sky API processing semantics when the event is raised. The current runtime still uses the Blue API and the rules engine tries to maintain consistency--if you start with Blue, it's Blue all the way through...unless you force another API as I have here.

Conclusions

If you're still with me, congratulations. This post has demonstrated the power of common services in a personal event network and shown how they can be implemented and leveraged. To summarize:

• Sky means that any ruleset a personal event network owner activates will see relevant events from other rulesets that have been activated in the same network.
• The Notification Event API defines a generalized service API for personal event networks. More will follow. If you have suggestions of other common services that ought to be standardized with an API, we're listening.
• There will be certain services, like notifications, that every almost personal event network has a ruleset to handle. They can be customized to match the purpose of the personal event network.

2:40 PM | Comments () | Recommend This | Print This

A Web of Things on the Internet of Things

This article by Marshall Kirkpatrick describes why the Internet of Things is real and happening right before our eyes. The Internet of Things is happening because more and more things are being connected to the 'Net. This, of course, is inevitable as Moore's Law incessantly drops the price of such connectivity. Marshall says "real-world devices not traditionally connected to the Internet that will be wired-up into a future Web of Things."

I like the phrase "Web of Things" because "Internet of Things" has come to describe what is known as "M2M" or machine-to-machine connection. Web of Things conveys something different to me. Where M2M is about connecting things together for what ever reason, Web of Things implies a higher level of functionality and something that people will use.

M2M technologies are being used to create services that back up physical products. This is the idea I blogged about recently with products as avatars for a service. Many products are really just physical manifestations of the real product which is an evergreen service. Tivo is a good example. So are phones, Fitbit, and even my Ford truck.

Marshall uses the example of a traffic sign:

If you've ever driven past one of those big signs on a road that show you your own driving speed, you might have wondered who else was seeing that information. Originally, no one else was. A company called AllTraffic sold those signs to government agencies around the US as a hardware play. Stick it on the side of the road and show people how fast they are going - hopefully it will cause speeders to slow down.

About a year and a half ago, though, they connected those blinking signs to a web portal accessible by police headquarters and citizens, using Axeda's connectivity technology. Now they sell access as a subscription and they've changed from being a hardware vendor into a software and service vendor.

From Google X? These Nine Products From the Future Are Real Right Now
Referenced Thu Dec 15 2011 10:14:03 GMT-0700 (MST)

We might view M2M interactions like this:

A manufacturer, GE in this case, has created a network of all the devices they manufacturer of a particular type for purposes of managing those things. They might update the software in your dishwasher, gather operating statistics, and so on. Marshall talks about this:

[Bill] Zujewski [of Axeda] shared a story with me about a dishwasher manufacturer that made a mistake. The company didn't program its rinse cycle to be long enough and was getting thousands of phone calls from customers complaining that the machines weren't working well. The company sent a technician out to reset the rinse cycle timers - but future iterations of the machines saved all those costs by adding read/write cellular connections to the dishwasher computers that could be re-calibrated remotely.

Ovens, dishwashers, all kinds of appliances get shipped from the factory with certain assumptions. Add connectivity and they can be optimized, in the field, remotely.

From Google X? These Nine Products From the Future Are Real Right Now
Referenced Thu Dec 15 2011 10:22:18 GMT-0700 (MST)

But of course, once the dishwasher has connectivity, then all kinds of things are possible. Marshall continues:

[O]nce the computer is on board, you may as well start building apps that add value directly to the consumer as well.

Want to start the pre-heating cycle from your phone, while on your way home? Can't remember if you turned your oven off or not? An application framework layer on those devices enables engagement with the devices themselves via mobile device.

"Because cellular connection capabilities for these devices are coming down from hundreds of dollars, sprinklers, garage doors, smoke detectors, all kinds of things are being experimented with as connected devices," Zujewski says. "It wasn't economic before because consumers wouldn't pay for that connectivity, but if it was a couple cents a month, then they will. Right now, costs are around $50 per month to retrofit devices with connectivity, but if you can do it with a chip involved at the origina of design, it can be around $10 per month."

If you can engineer connectivity right into the product from the start, the price drops dramatically. The sales cycle is long, though, because it takes years to bake connectivity into the core of a device.

From Google X? These Nine Products From the Future Are Real Right Now
Referenced Thu Dec 15 2011 10:24:42 GMT-0700 (MST)

How all this happens is of intense interest to me and my colleagues at Kynetx. I don't just want apps on my phone that control my oven. Our view is bigger than that. We want things--what we call social products--cooperating on behalf of their owner. For this to work people need personal event networks--a private cloud--where their products and services can interact to get things done for them. I don't merely want dashboards so I can turn on my oven from my iPhone or remotely check the temperature. I dream of my things working together with only modest intervention and interaction with me.

The following picture illustrates how I think personal event networks will interrelate to M2M networks that manufacturers are bound to put in place. This creates a Web of Things:

While the manufacturers still have their M2M connectivity so they can manage my things (presumably with my permission), each of the things I own is also part of my personal event network. My personal event network also has access to APIs for Web services and personal data. Applications running in my personal event network do the heavy lifting of scripting interactions for my things. Some of these might come from manufacturers; many come from third-party developers.

Giving me more apps on my phone and more things to manage doesn't appeal to me. Giving companies more control over my life doesn't appeal to me. But giving me a personal network that automates drudgery and enables things that I couldn't do before does. That's the promise of a personal event network and why I think everyone will want one.

10:51 AM | Comments () | Recommend This | Print This

December 6, 2011

APIs That Call You

Ever since I heard Tibco's CEO, Vivek Ranadiv say that a database is like a phone that doesn't ring, I've used it as a way to describe why APIs should be two-way. Another metaphor we've used to describe this is by talking about "full-duplex" APIs, although there's a little mismatch with that.

There are at least three different modes in which an API might send data:

1. Continuous stream - some system need to be in near-constant contact with the other participants. For example, a chat system needs to be ready to send and receive information from the participants at almost any time. Another way to think of these kinds of API interactions is as persistent connections.
2. Broadcast - some systems have data that they must distribute to many participants. A blog post being sent to multiple subscribers is an example of this.
3. Evented - some systems need to push data to one or a few other systems when something happens (i.e. when there's an event). Some APIs are for user (entity) accounts and the data being sent is related to a specific entity and destined for another system where that same user has an account. Some are merely point-to-point connections. The key idea is that there's an event that needs to be seen by a few other systems.

The first and second scenarios have specifications that are widely known, if unequally used. Continuous stream applications can use Cometd or Web sockets. PubSubHubBub (PuSH) is commonly used in broadcast-style applications.

Twilio is probably the best example of an evented API. They're using Webhooks, a good solution for this sort of thing. But many other point-to-point or entity-centry applications are using home grown solutions that tend to be used only by that one API. Take Fitbit, for example. I love my Fitbit, but their "subscription API" is "loosely modeled on PubSubHubbub (PuSH), with simplifications." Don't get me wrong--I'm glad they have one at all--but PuSH probably isn't the best place to start for this application.

Sam Curren and I have proposed the Evented API specification for this sort of thing. Evented APIs are specializations of Webhooks. All APIs that follow the Evented API specification are Webhooks, but not all Webhooks are examples of evented APIs.

Evented APIs are not meant to be used as continuous streams or broadcast subscriptions. They are designed for one-to-one or one-to-a-few interactions. Let's return to Fitbit. I'm a Fitbit user. I have an account. If I want to link my Fitbit account to some other online service where I also have an account, then an Evented API would be a perfect fit (bit).

There's nothing "wrong" with how Fitbit's doing it, but it's not standard. This is an argument about standards, not about the "right way." If we can all agree on a standard for point-to-point and entity-centric API interactions, we can start to create libraries for common programming languages and develop pools of expertise. Sam and I created the Evented API specification to solve this problem. Kynetx supports the Evented API, but the specification is independent and does not rely on any proprietary technology.

APIs that call you are relatively new and consequiently unfamiliar. Undersanding what type of subscription or push interaction you need to support will lead you to the best way to do it. We'll be taking the Evented API specification to a standards body next year. We'd love to get your comments. We'd really love for you to start using it and give us feedback based on real-world experience.

2:33 PM | Comments () | Recommend This | Print This

December 1, 2011

Products as Avatars for a Service

I've read three interesting articles recently that describe an idea I think is very important: more and more, products are merely the physical manifestation of a service. As a result, even traditional product companies are seeing new revenue as they shore up their products with services. They're also discovering the unique requirements of delivering software services.

• Now Every Company Is A Software Company
• Google X? These Nine Products From the Future Are Real Right Now
• Everything is a service

Note that there's a strong M2M thread here. As companies who haven't been in the business of connecting devices to the 'Net before jump in, they're turning to M2M players to provide the infrastructure for that.

The result of this is that we're getting things connected to the Internet. What's not happening--yet--is connecting products to each other. I've taken to calling those "social products" after hearing Marc Benioff use that term. I don't know if that's what he meant, but that's what I mean.

10:28 AM | Comments () | Recommend This | Print This