Phil Windley's Technometria

« April 2011 | Main | June 2011 »

May 27, 2011

Platforms and Lock-in

At Gluecon on Wednesday, Mark Suster said that Force.com & AppExchange are mostly about lock-in. I think that's pretty much an exact quote. I tweeted it and expected to see someone react. No one did--at least on Twitter. But I did have a few people mention it to me with some questions about what Mark meant. The truth is, I don't know and I can't speak for Mark.

Any platform provides some degree of lock-in. APIs lock a developer into a particular interaction mode. When I use Flickr's API, I make decisions that lock me into that platform. I can support others, but each is a certain amount of work and the switching costs can be quite high depending on the depth of the API. Similarly, when I build an app for Facebook, it only works there. I can architect things so that I can redeploy on other platforms, but that's work that locks me in.

At the other end of the spectrum are IaaS plays like AppEngine, Amazon Web Services, and Heroku. To varying degrees, each of these causes some lock-in. Heroku is probably the most general. The Rails code I run there can be pretty much run in any other Rails environment, although I have to live within some constraints to run on Heroku. Sure AWS is just Linux on EC2, but as soon as I use S3, SimpleDB or something else, I've locked myself in. Similarly AppEngine and BigTable.

Given this I'm not quite sure why Mark would single out Salesforce. True, Force.com is largely about building apps that work with Salesforce data and if I've invested time to build an app on Force.com or gotten used to the features of one I bought, it is harder for me to switch, but heck, that's true of almost any platform. I'm pretty hevily invested in OS X at this point, but to say that because of that OS X's ability to run applications is mostly about lock-in would be a stretch. It's part and parcel of providing value.

To say that something is "mostly about lock-in" implies that it provides little other value because lock-in is the quid pro quo we pay on most platforms for the value they provide. The fact that SF's customers buy apps to enhance their experience leads me to believe that they're seeing value and that belies Mark's comment by my reasoning.

So, maybe Mark will write a blog post giving us his thoughts about platforms. Until then, I'm not sure I see the logic.

2:24 PM | Comments () | Recommend This | Print This

May 24, 2011

Error Handling in KRL

If you've been following along, you know that KRL, the Kinetic Rule Language, is an event processing language for processing events on behalf of an individual entity (usually a person). Think of it as the People's Complex Event Processing language.

For a long time, it's been difficult for KRL developers to handle errors. The language had no mechanism for exceptions and the only hope was to turn on debugging and fix them. Too often, however, we don't want to fix errors so much as handle them. Errors are not always problems in the code--bugs to be fixed--but exceptional conditions that a program needs to be ready for and deal with appropriately.

Most modern language have some kind of exception handling mechanism--some more sophisticated than others. I realized that exception handling is usually treated as a small event system within a larger imperative language. KRL, being an event language, is perfectly suited to handling errors--if only the system would raise them as events. As of today, it does.

The most recent update of the Kinetic Rules Engine provides support in KRL for the following features related to errors and error handling in KRL:

• system errors - system errors (like applying an operator to an operand of an inappropriate type in a way that the system can't recover) will raise an event with event domain system and event type error. The event can have the following attributes:
  • msg - the error message as a text string
  • level - the error level, one of error, warn, info, of debug
  • genus - the major categorization of the error according to the event hierarchy shown in the documentation.
  • species - the minor categorization of the error according to the event hierarchy.
  • rid - the ruleset ID that caused the error
  • rule_name - the name of the rule that caused the error

• explicit errors - as we will see below, developers can explicitly cause and error event to be raised. These errors have the event domain user and the event type error can have msg, level, rid, and rule_name attributes.

• error redirection - the meta block now supports the inclusion of the errors to pragma which defines the ruleset that is handling errors for the current ruleset.

  meta {
   ...
   errors to a16x88 version "dev"
   ...
  }
  

  The version clause is optional.

• error handling - the ErrorStack module supports sending errors to the ErrorStack online service. Of course, other actions can also be taken as appropriate.

Raising events on errors is a natural means of dealing with out of bounds conditions in a KRL ruleset since writing rules that respond to errors is a natural extension of the base behavior of the system. When the system raises an error event, or a developer causes one to be raised with an explicit error statement in the rule postlude, that event is evaluated by the engine in the current execution cycle and any rule that selects for that error with it's given event domain, type and attributes will be added to the evaluation schedule.

In the absence of the errors to pragma in the meta block, the rules in the current ruleset will be evaluated for selection. If the errors to pragma is present, the rules in the ruleset identified by that pragma will be be evaluation for selection. If a ruleset does not have any rules to handle the error event and no errors to pragma is present, the error event, like any other unhandled event, has no effect on the execution of the ruleset or the system.

The ErrorStack module is a simple example of how errors can be reported. ErrorStack is a service that provides an API for reporting errors that are then presented to the owner of the error stack with support for categorization, alerting, and visualization. When you sign up for the Error Stack service, you can create any number of different stacks to report errors into. A developer key uniquely identifies the errors stack that the errors are reported to.

The ErrorStack module has a single action defined, send_error, that takes a message. When the module is used in a ruleset (via the use module pragma) it is configured with the stack key that the developer receives from Error Stack. The following code fragment shows a rule that handles errors and reports them to an Error Stack using the send_error action:

rule process_error {
  select when system error
           or user error
  pre{
    genus = event:param("genus");
    species = event:param("species");
  }
  es:send_error("(#{genus}:#{species}) " + event:param("msg"))
       with rule_name = event:param("rule_name")
        and rid = event:param("rid");
}

The rule assumes that the following pragma was present in the ruleset's meta block and the developer key was stored in the ruleset's keychain:

use module a16x104 alias es
      with es_key = keys:errorstack()

Error Stack isn't the only way to handle errors. More generally, you could use the HTTP library to send the notification to some other API like notifio, the Twilio library to send an SMS, or even tweet the error. Alternately, there may be some mitigation to apply.

Developers can choose to raise their own errors using the error statement in the postlude. Technically, since an error is simply an event, developers have been able to do this using Explicit Event and still can. The error statement just makes it more convinient. The syntax of an explicit error statement is:

error <level> <expr>

where <level> is one of error, warn, info or debug and <expr> is an valid KRL expression that results in a string (or something that can be cast as a string such as a number). The following example would raise an event with domain user and type error with level info and a message with the value of a variable named query if the rule fired:

fired {
  error info "query:"+query
}

Every ruleset ought to have a catchall error handling rule like the one above or send their errors to a ruleset that's been created to handle errors. Error handling rules can, of course, be mode more specific with more sophisticated select statements, filtering on error genus and species as well as the error message itself. Error handlers that themselves causes errors won't cause an infinite recursion in any case as the system will catch those and break the event chain.

11:59 AM | Comments () | Recommend This | Print This

May 17, 2011

Install Kynetx for Google Chrome

The Kynetx Browser Extension (KBX) is now available for Google Chrome users in the Chrome Web Store. KBX is a universal browser extension that allows you to run Kynetx applications in any browser. There are dozens of Kynetx browser apps available right now and developers are creating more all the time.

Because the apps run in the cloud, you get the same behavior from apps whether they're running in Chrome, Firefox, or Safari (IE support is comming soon). What's more, even if you only use one browser, the apps behave uniformly across every computer you own and any app state is automatically in sync. For example, I use and love Personal Block List, an app for combatting search spam. When I block a site at work, it's also blocked at home.

Take a minute and install Kynetx for Chrome and grab a few apps. I think you'll like it. If you'd like to know how to use Kynetx to sign up for an account and get started.

Update - KBX is listed in Mozilla Add-ons for Firefox now as well. There's been no review yet, so you can't find it in the directory. If you have any influence on the reviewers, be sure to tell them how great it is. Meanwhile, sign in and leave your own review and rating.

9:34 AM | Comments () | Recommend This | Print This

May 10, 2011

Dealing with JSONP in KRL: Using the Flickr API

When I blog I like to put pictures, like the one to the right of this sentence, in the article when I can. My usual method is to use Flickr because I can find good pictures that have creative commons rights that allow me to use them on my blog. The problem is that after I find the picture, I need to create an image tag with the right size source image that links back to the Flickr picture for attribution purposes.

Creating the tags to do that by hand is a pain, so I've created a simple Kynetx application that automates the process for me. After I've found the picture I want, I just click a bookmarklet and get a box overlaying the bottom right corner of the screen like this:

I simply cut and paste the image into the blog post and it's ready to go.

Making this work required interacting with the Flickr API. The return results in two ways: XML or JSONP. Note that the "P" isn't a typo. JSONP is a method for returning JSON to Javascript programs running in the browser so tha they can simply call them from a script tag and automatically run a magic function. In Flickr's case the magic is called jsonFlickrApi. You define that in your JavaScript before hand to do what you want done with the JSON. That works great in a browser, but if you're calling the API from another programming language on a server, it's a pain. Because I was writing a Kynetx app, that was the case with me.

I defined a function to call the API, strip out the JSONP call, and turn what was left over into JSON.

datasource twitter_picture <- 
     "http://api.flickr.com/services/rest/?format=json";
   
flickr_photos = function (pict_id, method) {
  datasource:twitter_picture(
                          {"photo_id": pict_id,
                           "api_key": keys:flickr_api_keys("key"),
                           "method": method
       }).replace(re/jsonFlickrApi\(([^)]*)\)/, "$1").as("json");
};

I first declare the datasource and then use it inside the function. The function makes use of the Flickr keys I've stored in the meta block. I could use and http:get() instead of the datasource, but wouldn't get the benefit of automatic caching. Probably not a big win for this use case in any event, but nothing wrong with doing it this way.

Using the function couldn't be easier. This is the rule that creates the overlay shown above:

rule create_image_tag {
  select when pageview "flickr.com/photos/" setting ()
  pre {
    id = page:url("path").extract(re#/(\d+)/#);
    small_pict_url = 
        flickr_photos(id[0], 
            "flickr.photos.getSizes").pick("$..size[2].source");
    title = flickr_photos(id[0], 
            "flickr.photos.getInfo").pick("$..title._content");
    image_html = <<
<a href="#{page:env("caller")}">
<img style="margin-top: 10px" 
     src="#{small_pict_url}" 
     border="0" hspace="3" vspace="0" 
     align="right" width="150px" title="#{title}" alt="#{title}" />
</a>
     >>;
     img_tag =  <<
<table>
<tr>
<td>#{image_html}</td>
<td>
<textarea rows="5" cols="67">#{image_html}</textarea>
</td>
</table>
     >>;
  }
  notify("Flickr Image Tags", img_tag) with
     sticky=true and 
     position = "bottom-right" and
     width=550;
}

The rule looks longer than it is because of two big HTML templates in the middle. We grab the id for the picture out of the URL, call the function we defined above twice, once to get the information about the various sizes available and once to get the information about the picture and use a pick operator with a JSONPath expression to pick out the parts of the JSON that we want: the URL for the small image and the title. Then we place them inside the HTML and put it on the page with a notify action.

Now I've got a tool that makes blogging easier and saves me time every time I use it. I also was pleased to note that KRL could deal with the Flickr API regardless of whether it played nice or not.

If you think this app would benefit you, let me know and I'll make it available to you. Also, try out some other Kynetx apps here. I'm anxious to know what you think.

2:45 PM | Comments () | Recommend This | Print This

Someone Clue Delta in About Transactional Integrity

Delta Airline's ticket kiosks apparently were designed and built by 1st year CS students using Visual Basic. I'm guessing that's the case based on the fact that they apparently have never heard that when you sell someone something, it's bad form to take their money and not deliver the product. They don't know that you should wrap such processes in transactions to ensure they're atomic. Someone needs to give them a clue.

Last Friday on our way home from San Jose, Steve and I had the chance to get on an earlier flight for $50 each. I was checking us both in, so I popped in my credit card for the $100 charge. The kiosk said "We're sorry, we can't read your card. Please try again." So, I did. The card failed the second time, so we tried Steve's card. The machine didn't like Steve's card either and after showing the same message, it printed out our original tickets and sent us on way. We went to the counter and bought the upgrade.

This morning, Steve was looking over the credit card bill and, surprise!, the three failed upgrade purchases were charged to our cards even though the machine said the cards were unreadable and didn't deliver the earlier flight.

I might forgive a small startup for not getting transactions right, but Delta must have a few people who have heard of them, understand their importance, and can get the kiosk software right. Maybe I'm giving them too much credit. I couldn't imagine a big company in 2011 that has issues with transactional integrity on simple purchases. I've always assumed they'd get this much right. Maybe I ought to start questioning their ability to safely fly an airplane as well.

9:23 AM | Comments () | Recommend This | Print This