Phil Windley's Technometria

« September 2010 | Main | November 2010 »

October 26, 2010

IIW XI Is Next Week

IIW begins in a week on Tuesday November 2nd.

We are really excited about all the attendee's who are registered so far. The emerging themes we have identified are reflected in the topics proposed:

• Personal Data Ecosystem
• Federated Social Web
• User-Centric Identity applied (OpenID, OAuth, XRD, SAML, InfoCard, Activity Streams, etc.)
• Vendor Relationship Management
• Active Clients (tools in the browser and other clients)
• Identity in the Cloud

It is not to late to register. If you want invite friends to IIW-Nov still you can give them this 10% discount code good for Regular that ends at Thursday at midnight: "IIW-Nov". Then prices go up $100 per ticket type.

We have Demo slots available for Wednesday after lunch. There is more room for your project to share please let Kaliya know (kaliya@mac.com) if you are interested in doing so. Please provide a name, link and 280 character description by Friday October 30th.

Schedule

Tuesday doors will open at 8AM for registration. Phil Windley will give the opening talk at 9am and we will begin agenda creation by 9:30. We will have 5 sessions per day. Dinner on Tuesday and Wednesday will be hosted and at local restaurants.

You can find the schedule online

If you are wondering about how the unconference works please read this post on Kaliya's unconference blog.

One of the key things about IIW is that participants take notes and submit them for composition into a book and to be posted on the wiki. Please download this form for this purpose.

Thank you to our Sponsors

• Microsoft
• Google
• Facebook
• PingID
• OpenID Foundation
• Kynetx

We're also still looking for dinner, lunch, and break sponsors. Contact Phil if you're interested. (Sponsoring an IIW break costs about the same as just attending regular conferences.)

Meetings around IIW

There will be several meetings happening the same week as IIW that you might want to be aware of and attend:

The OpenID Foundation will host a "OpenID Summit" to discuss the progress of its technical working groups on the afternoon of Monday, November 1 at Facebook's Headquarter's in Palo Alto. The OpenID Foundation Board of Directors will have a face-to-face board meeting at the conclusion of the IIW the afternoon of November 4. Please contact Don Thibeau with questions and comments.

The UMA WG will hold a face-to-face meeting on Monday Nov 1 at the Computer History Museum. Contact Eve Maler with questions.

Kynetx will host a DevDays for people interested in knowing more about the evented-web and how to create event-aware applications using the Kynetx developer platform on Friday, Nov 5 at the Computer History Museum. Contact Brad Hintze with questions. Register here (use discount code iiw_kynetx for a 100% IIW affiliate discount)

OpenID Foundation will be having a dedicated session at IIW for retailers preparing for a kickoff OpenID Retail Summit in Q1 at PayPal. Janrain and PayPal are taking the lead on developing an agenda with input from retailers, OIDF members, and other interested parties. More info here:

• Open Retail Advisory Committee
• ORAC Overview Presentation

Other community events of interest

Workshop: Online Constituent Identity Nov 12, Washington, DC; Uniting the identity and citizen engagement software communities Creating sustainable online constituent identity solutions that serve the needs of citizens, advocacy groups, and elected representatives. Here's the link for full details. Early registration discounts expire 10/29. And for more info on the association itself, see the Open Model for Citizen Engagement website.

Identity.Next December 8th The Hague, Netherlands; The Identity.Next is assembling a program with top experts, professionals and industry stakeholders to discuss the world around Digital Identity and best practices. The event is 1/2 programmed conference in the morning and 1/2 unconference in the afternoon. These themes have been identified:

• Social consumer (How will social identity turn around the new value chain?)
• MobileMe (Where is my DigitalMe and who should know?)
• Private Eye (Who owns and control the privacy problem and should we really care?)
• E-citizen (Next generation eGovernment and its identity issues).

Don't Forget to Vote Early since you'll be at IIW on election day!

9:36 AM | Comments () | Recommend This | Print This

CTO Breakfast on Friday

This Friday we will hold October's CTO Breakfast at the usual place, Novell Cafeteria (Mountain View Room) in Provo. We'll start at 8am and go until we're tired or they kick us out. As always, the topics come from you, so come prepared to discuss your favorite tech happenings. You don't have to be a CTO to come, just someone who's interested in technology, high-tech products, and building high-tech businesses.

The next CTO Breakfast will be on Thursday December 2, so mark your calendars.

9:06 AM | Comments () | Recommend This | Print This

October 25, 2010

Voting at API Hack Day: A Kynetx App

Sam Curren and Brad Hitze were at API Hack Day yesterday. In a fit of meta hacking, Sam created a voting app that was, at the same time, an app competing in the hackathon and the app recording the votes for who won. The API Hack Day results we computed and communicated in real time using a twitter account from votes made via phone or SMS. Sam's app combined Twilio, Twitter, and a Google spreadsheet.

Sam reports that a few people gamed it by provisioning and then dropping telephone numbers, but that's OK. It was still fun. Next time he'll take more precautions in hackathon voting (after all it is a hackathon).

This kind of application is a natural in KRL because an event-driven architecture is a no-brainer in this situation. As you'll see in the code below, modeling the interactions by describing event patterns in the select expressions and then writing responses to them provides a readable and easily maintainable application. The flow can be a little hard to visualize if you're not used to thinking in terms of events, but is pretty simple.

Sam defines seven rules in two parallel event chains, one for SMS and one for voice. If the voter is voting by SMS, then the inboundsms is fired and through explicit events leads to recordvote and then to tweetit. If the voter phones in callstart is fired leading to getvote, phonevote, and finally, tweetit:

In both paths through the app, the end result is that the current winner is tweeted. The Google spreadsheet is just doing a form post for updates, which has some limitations in terms of update frequency imposed by Google.

Here's Sam's code, with keys redacted:

ruleset a8x76 {
  meta {
    name "PeoplesChoice"
    description <<
      votes for apihackday
    >>
    author "Sam Curren"
    logging on
    key twitter {
       *redacted*
    }
  }
 
  global {
    dataset winningdemo <- 
     "https://spreadsheets.google.com/pub?key=*redacted*&single=true&gid=4&range=D2&output=txt" 
     cachable for 1 second;
  }
 
  rule inboundsms is active {
    select when twilio sms
    pre {
      messagetext = event:param("Body");
      sender = event:param("From");
      applist = messagetext.extract(re/(\d+)/);
    }
    if(applist.length() > 0) then {
        noop();
    }
    fired{
      raise explicit event recordvote with 
         appvote = applist.head() and 
         voter = sender;
    } else {
      raise explicit event help;
    }
  }
      
  rule help is active {
    select when explicit help
    twilio:sms("Vote for your APIHackDay demo x by texting 'vote x'");
  }
    
  rule recordvote is active {
    select when explicit recordvote
    pre {
      voter = event:param("voter");
      appvote = event:param("appvote");
    }
    {
      http:post("https://spreadsheets.google.com/formResponse?formkey=*redacted*") 
       with params = {"entry.0.single":voter, 
                      "entry.1.single":appvote, 
                      "submit":"Submit", 
                      "pageNumber":"0",
                      "backupCache":""};
      twilio:sms("You voted for app #{appvote}. Only your last vote counts. Thanks! (Powered by Kynetx + Twilio)");
    }
    fired {
      raise explicit event tweet;
    }
  }
        
  rule callstart is active {
    select when twilio callstart
    {
      twilio:say("Thank you for coming to API Hack Day");
      twilio:pause(1);
    }
    fired {
      raise explicit event getvote;
    }
  }
      
  rule getvote is active{
    select when explicit getvote or twilio getvote
    {
      twilio:gather_start("phonevote");
      twilio:say("Enter the number of your favorite app, followed by the pound sign.");
      twilio:gather_stop();
      twilio:redirect("getvote");
    }
  }
      
  rule phonevote is active {
    select when twilio phonevote
    pre {
      appvote = event:param("Digits");
      voter = event:param("Caller");
    }
      {
      http:post("https://spreadsheets.google.com/formResponse?formkey=*redacted*") 
         with params = {"entry.0.single":voter, 
                        "entry.1.single":appvote, 
                        "submit":"Submit", 
                        "pageNumber":"0",
                        "backupCache":""};
      twilio:say("Your vote for app #{appvote} has been recorded. Only your last vote counts. Thank You!");
    }
      fired {
        raise explicit event tweet;
      }
  }
      
  rule tweetit is active {
    select when explicit tweet or pageview ".*"
        pre {
          winner = winningdemo;
          tweet = "Current Voting Leader is App #{winner}! #apihackday";
        }
      twitter:update(tweet);
 
  }
}

The code makes use of the Kynetx webhook endpoint to tie to Twilio. Note that the Consumer and OAuth keys for the Twitter link were redacted so it may not be obvious how the connection to Twitter is made from the source. Sam's going to do a more detailed analysis of the code and describe some of the trickier aspects (like what all the Twilio actions do and how they work). When he does, I'll update this post to point to it.

All in all, not a lot of code to intergrate a couple of APIs and a telephony service. Especially given that it's supporting SMS and voice. A dedicated Google spreadsheet integration in KNS would make this cleaner and improve the app's performance. I'm very impressed with what Sam was able to pull off in a few hours.

2:29 PM | Comments () | Recommend This | Print This

October 22, 2010

Kynetx Platform Update

We've had some changes in the Kynetx platform, KNS, over the last few weeks that I wanted to update people on.

First, a few weeks ago, we cut over to a new parser. Yeah, if you're keeping score, this is number 3. The first parser was based on HOP. We replaced that with Damian Conway's Parse::RecDescent as chronicled in a blog post I did in September of 2008. The third (last?) parser is based on the ANTLR parser generator. Cid Dennis did most of the hard work of creating the new Parser and Mark Horstmeier did the Perl integration using Inline::Java.

We switched for speed. The old parser was starting to choke on large rulesets. Other than a syntax change forced upon us by losing the flexibility of doing lexing and parsing at the same time, the new parser was deployed without any significant problems due to the large suite of tests we created the last time we did this. I don't like ANTLR's input file as much as RecDescent and the fact that it creates Java means we have to jump through some hoops to make it work in our system, but it's several orders of magnitude faster than the old parser. I'm confident the new parser will take us a long way and don't anticipate a changeout.

I wrote about explicit events a few weeks ago. These are proving to be a very powerful addition to the language. As originally written, event names and target rulesets were denoted by name. As I mentioned, we've fixed that by allowing these to both be denoted by expressions. This makes explicit events much more powerful.

We found a similar limitation where we'd required that the keys in a literal hash be strings. This was limiting how developers used them so we relaxed that as well and now the key in a literal hash can be any expression--as long as it evaluates to a string or number.

As I wrote in KRL Patterns: Building Event Intermediaries, often rules that are acting on events don't need actions to be useful. The postlude becomes the rule focus because that's where events are raised. In order to accomodate this pattern without requiring developer to include the "noop" action, we've made actions optional.

Finally, I recently checked-in code to make the rule mode optional. If you're not familiar with rule modes, that's because it's a string you probably just mindlessly cut and paste in your rulesets without really knowing what is does. In the following example rule, the "is active" is the rule's mode:

rule example_1 is active {
  select when pageview ".*"
  ...
}

You can say "is inactive" and the rule will cease to fire without commenting out the entire rule. You can also say "is test" if you're working with an endpoint that can be put in test mode. In test mode, all active and test rules fire. Most rules, however, are active so in a nod to convention over configuration, we've made the rule mode optional, so the previous example can be written:

rule example_2 {
  select when pageview ".*"
  ...
}

With all these optional parts, the simplest rule possible is now:

rule example_3 {
  select when pageview ".*"
}

The only required components of a rule are the name and the event expression. Of course, this rule doesn't do anything, but that's a different matter.

In addition to these, we've also made less visible changes to KNS to make the platform perform as expected (i.e. we fixed a few bugs) and layed the groundwork for some significant changes coming soon. Stay tuned!

2:48 PM | Comments () | Recommend This | Print This

October 21, 2010

Events, Webhooks, and the Realtime Web: Kynetx Dev Days

At Kynetx we're very bullish on the real-time Web. There are several trends that are leading us inexorably toward better use of real-time data including webhooks, Restful APIs, streaming data from sources like Twitter and Facebook, and Internet identity protocols like OAuth. At Kynetx, we use the term "event-driven" to descibe systems and architectures that make use of these ideas. Kynetx is a system for building event-driven applications that make use of webhooks, APIs, stream data, and user-centric identity. As I said in my post on Static Queries, Dynamic Data: Enabling the Real Time Web,

In a traditional database, the data is (relatively) static and the queries are dynamic. An event processing system is the dual of that. In an event processing system, the queries are static and the data is dynamic. Since KNS is an event processing network, or a system of programmable event loops, and KRL is an event processing language, it has provision for expressing static queries that it continuously applies to streams of events.

From Phil Windley's Technometria | Static Queries, Dynamic Data: Enabling the Real Time Web
Referenced Thu Oct 21 2010 07:51:03 GMT-0600 (MST)

We're holding DevDays, or meetups, in Mountain View CA and Broomfield CO (near Boulder) in November to help introduce these ideas and socialize the ideas around event-driven systems.

The Mountain View DevDay will be on November 5th at the Computer History Museum. This is the day after the Internet Identity Workshop, so if you're in town for that, just stay an extra day and see what we're up to. We'll be talking about event-driven architectures and application, webhooks, and using APIs. Twilio will be there to talk about how webhooks are used in their telephony API. You can register for the Mountain View DevDay at Eventbrite.

The Broomfield DevDay will be on November 19th at Acxiom's Broomfield office. This is the day after Defrag and just down the street. Again, we'll be talking about event-driven architectures and application, webhooks, and using APIs. Acxiom will be there to talk about how they're using Kynetx with their Fortune 500 partners to build unique, compelling appliucations that are event-based and mashup data from across the Web. You can register for the Broomfield DevDay on Eventbrite.

At both of these events, we'll be demoing mind-blowing applications like Sam Curren's real-time telephone auction application that he wrote to mash up Twilio, Google spreadsheets, and Twitter to sell a phone at a conference. That's real-time at work. If you're interested in new, different, and powerful programming models and the real-time Web you'll want to come to one of our DevDays. I'm anxious to see what you think.

8:07 AM | Comments () | Recommend This | Print This

October 18, 2010

Matthias Felleisen at BYU CS Colloquium: Adding Types to Untyped Languages

Mattahias Felleisen from Northeastern University will be delivering the CS Colloquium at BYU on Thursday Nov 4 at 11am. Here's the abstract of the talk:

Over the last 15 years, we have experienced a programming language renaissance. Numerous scripting languages have become widely used in industrial and open-source projects. They have supplemented the existing mainstream languages---C++ and Java---and, in contexts such as systems administration and web programming, they have started to play a dominant role. While each scripting language comes with its own philosophy, their designers share an antipathy to types. As a result, these languages come without a static type system. Most script developers initially welcome this freedom, but soon discover that the lack of a type system deprives them of an essential maintenance tool.

My talk explains my team's approach to equip such languages with a type system. The goal of our work is to empower programmers so that they can gradually enrich scripts with types on a module-by-module basis as they perform maintenance work on the system. Naturally, we wish to ensure type soundness so that the type annotations are meaningful, and we wish to accommodate the programming idioms of the original language in order to keep the overhead of type enrichment low.

From Adding Types to Untyped Languages | BYU Computer Science Department
Referenced Mon Oct 18 2010 10:49:38 GMT-0600 (MDT)

Matthias is one of the deep thinkers in the area of programming language design and someone I admire very much. If you're at all interested in programming language design, you'll want to make this talk.

10:53 AM | Comments () | Recommend This | Print This

October 14, 2010

Building a Programming Layer for the Internet: Audio from Impact

We held our Impact conference for 2010 last April. There were a lot of great talks. Some of them we were able to capture and have published on IT Conversations. If you have yes listened to Steve Spencer, Joe Vito, Steve Gillmor, or Jon Udell, take some time and put them on your playlist. They are great talks with great messages.

The audio from my talk from Impact wasn't quite good enough for IT Conversations--there were cutouts in some spots--so we didn't publish it there. I recently got around to cleaning it up and was going to publish it here. Then I decided I ought to link to the slides as well. Then I decided I ought to create a slidecast on SlideShare. Of course, the problem is that that's a lot more work.

Nevertheless, I spent some time yesterday with SlideShare and attached the audio to the slides. The result is less than I'd hoped for, but I learned some things that will help next time. I can't say I'm overly enamored of the SlideShare tools for syncing audio with slides.

12:34 PM | Comments () | Recommend This | Print This

October 13, 2010

Jeff Lindsay on Webhooks

This week Scott and I interviewed Jeff Lindsay, the creator of the term webhooks and an avid promoter of the concepts. I love webhooks, we you'll know if you've read my post Beyond the API: The Event Driven Internet . Here's the IT Conversations write-up:

WebHooks are meant to do something. The concept of a WebHook is simple. It is an HTTP callback: an HTTP POST that occurs when something happens; a simple event-notification via HTTP POST. Developer Jeff Lindsay talks about what a WebHook is and how it works as a programming API. Jeff reviews how WebHooks are being currently used and discusses the technical aspects of their purpose and their implementation.

From IT Conversations | Technometria with Phil Windley | Jeff Lindsay
Referenced Tue Oct 12 2010 20:29:36 GMT-0600 (MST)

Listen up and give webhooks a try. They're very cool.

7:31 AM | Comments () | Recommend This | Print This

October 12, 2010

Kynetx DevDay After IIW on November 5

Kynetx will be hosting an Impact DevDay on November 5th at the Computer History Museum in Mountain View, CA. This is the Friday after Internet Identity Workshop XI at the same place. Impact DevDay is an all-day, intensive training designed for programmers interested in creating apps that are cross-platform, context-aware, cross-browser and event-driven using the Kynetx Rules Lanaguage (KRL).

I've written extensively about KRL and some about why I believe event driven APIs are critical to the Web we all want to build. This is a chance to understand the platform we've built to support this vision and learn how to use it. You'll learn how to build applications that incorporate multiple APIs and work across the Web, mobile, and even mail like the Project Neck Pain demo I wrote about last week.

We're charging $10--mostly to get a commitment--and the day will include snacks, lunch, prizes, and a t-shirt or something else worth $10. Go and check out the agenda and sign up. I'd love to have you there.

11:16 AM | Comments () | Recommend This | Print This

Startups in Non-Traditional Places

Tim Spaulding who runs LibraryThing, a startup in Portland...Maine asked me on Twitter if I had any thoughts on startups in non-traditional places.

The short answer is I've always got thoughts, but I'm not sure how relevant the are. The truth is that while the Wasatch Front isn't Silicon Valley by any stretch, it's got a thriving high-tech community, thousands of software developers working at hundreds of companies, a nascent, but active angel investment community, three strong universities pumping out graduates and ideas, and ahost of support services including legal and PR that understand high-tech business. Just looking at the Utah Tech Events calendar shows that there's things happening here. That puts the Wasatch Front in league with Portland OR or Boulder CO as a thriving tech community that lives outside, but in the orbit of, Silicon Valley.

I'm grateful everyday to have landed in an area that has the support structure that we have in the Wasatch Front. We've leveraged the developer community a great deal in Kynetx with our Free Lunch Fridays, conferences, and developer days. Kynetx has had to raise less money and grown faster because of the support that we get from the local tech community.

Last summer (or was it two summers ago?), Marc Cantor and his family stopped and had lunch with me as he was moving from California to Ohio to "reboot" the region and himself:

There would not be enough space in our print newspaper for me to fully explain what Canter has in mind. So let me take some liberties and boil it down: He plans to reinvent work-force development, by constructing a system that takes unemployed people and teaches them a host of multimedia skills to fill the demand for online video that he sees coming. The plan involves creating a large nonprofit, a new private company, raising millions in grants and private equity, and getting an entire region that has no idea who he is to buy into it.

From Marc's Voice » Valley's one-time godfather of multimedia is leaving for Ohio
Referenced Tue Oct 12 2010 08:58:44 GMT-0600 (MDT)

After he left, I remember remarking to Steve Fulling who came to lunch with us that this was a tough row Marc had chosen to hoe. Even with good universities (which Ohio has), getting the infrastructure built to the point where it's self sustaining is a long haul. Utah has been on this road for 30 years (since IOmega, WordPerfect, and Novell were birthed in the PC revolution). There aren't any shortcuts that I know of.

I think high-tech companies in non-traditional markets have to be prepared to work more at building community and be willing to travel more to get to the places where the tech is. Of course, these companies aren't in business to build a high-tech ecosystem, but I think their efforts in those areas will get payed back. Start simle--sponsor a lunch once a month or something and get people meeting and talking. For example, I started the CTO Breakfast 6 or 7 years ago as a way to get people talking and it's been a lot of fun.

Spend time at the local colleges and universities fostering relationtions with the faculty and students. Hiring in a non-traditional area is one of the hardest things to do.

RightNow Technologies founder Greg Gianforte started his very successful company in Bozeman MT because he wanted to live there. He wrote a book that documents some of his lessons learned: Bootstrapping Your Business: Start And Grow a Successful Company With Almost No Money and started Bootstrap Montana.

Omniture Founder Josh James "decided to initiate a branding campaign for Utah's tech community in the form of Silicon Slopes."

So, as I said, a lot of thoughts, but no silver bullet. Anyone have other tips or ideas about starting high-tech businesses in decidedly low-tech areas?

9:24 AM | Comments () | Recommend This | Print This

Language Design Gives Leverage

Last Friday I spoke at the Utah Open Souce Conference on designing and building domain specific languages (DSLs). The slide from my talk are online (PDF). Our experience building KRL served as the catalyst and gave me concrete examples to talk about even though I'd hesitate to call it a DSL anymore.

Yesteday, I had an experience which reinforced my enthusiasm for building languages. Writing the post on building event intermediaries, thinking through the patterns, and talking to some of the Kynetx engineers who have the most experience with raising explicit events led to thoughts about how to generalize the raise statement and make it even more powerful.

The problem had three parts

1. event names were denoted by a constant (name),
2. target rulesets were denoted by a constant (RID), and
3. only one target ruleset was allowed

The obvious answer was to generalize the statement so that both event names and target rulesets were denoted by expressions. The value of the expression would be used to determine the event name or the target ruleset ID. Further, if the expression evaluated to an array, the members of the array could denote multiple target rulesets.

I sat down yesterday afternoon and in about 2 hours had made the changes and written the tests to confirm the changes worked. The code will be released today. I expect this simple change will provide a significant increase in the expressive power of KRL. Having the target of an explicit event be computed rather than static means that KRL is a much more flexible vehicle for writing event intermediaries.

My point is simple: a language designer has tremendous leverage. Small changes taking a few hours in the language's design and implementation can provide hundreds or even thousands of people a much more powerful tool that they, in turn, will use to accomplish mind-boggling feats. There are few other activities in programming with that kind of reach. More programmers should build languages. Even little languages can be the source of great productivity and satisfaction.

8:41 AM | Comments () | Recommend This | Print This

October 11, 2010

Smartphones as the ProtoPDS

Last week I had an interesting discussion with Sam Curren where we hit on the notion that the smartphone is a prototype personal data store (PDS). If you think about it, a smartphone--be it Andriod, iPhone, or one of the new Win7 devices--provides a common calendar, a common address book, and common photo library (among other things) with APIs for interacting to all of them.

For example, I can change out my calendar app to something like Calvetica (a popular alternate calendar for the iPhone). Doing this doesn't change the underlying calendar data or interfere with any of the other tools that use and access the calendar. Everything else still just works the way it always did. I simply have a different interface.

Or, I might download a new app that uses my calendar and adds things to it. Again, everything else that uses the calendar is richer and better because there's a new actor playing with my calendar.

Of course, programmers are shaking their heads and thinking "duh! this is the way things are supposed to work!" and that's true. But this is a watershed moment for Joe User. Smartphones are training people to think of the calendar not as an app, but as a data service that multiple apps can interact with. They might not use those terms, but the intuition is right. This is the kind of understanding that we need for people to really get the full impact of a PDS.

On the other hand, one of the dark, dusty corners of the "smartphone as protoPDS" idea is syncing. A true personal data service won't be one place where you keep data, but a service that manages data from lots of places. IMAP is a start at this idea, but syncing data across multiple devices is still an ugly stepsister to the flashiness of the whole smartphone idea. I've got MobileMe and I'm not very impressed. Its surprising that Apple hasn't done a better job on this front since iTunes is such a spectacular success at mixing stuff you load in the app with stuff you bought (and really maintain) somewhere else. There's much work to do here.

The Kindle system is another example of an app that does an admirable job of mixing cached content with a service that manages that content across multiple devices. The problem with the Kindle system is that it's closed. I can't write different apps that use that content and upload new content.

Still, with all their limitations, smartphones seem like a conceptual model for how people will interact with their PDS. A true PDS is a richer, more flexible, and full-featured version of the data repositories, services, and APIs that exist now in the thing you carry around in your pocket.

11:04 AM | Comments () | Recommend This | Print This

October 7, 2010

Putting Your Own DNS Servers in ClearOS

I've been using a CentralPointe server as a gateway server in my home for years. DirectPointe (I'm on the board) used to offer these, but no longer does, so it had gotten out of date and wasn't being supported anymore. Fortunately, the same system (based on Point Clark Networks code) is available from the Clear Foundation as ClearOS.

I like this software because it allows me to manage content filtering, etc. for my family centrally rather than relying on filtering software installed on each machine. It also provides intrusion detection and other services. The whole thing is based on Linux, of course, but there's a handy Web interface that makes it easy to set everything up and configure the box. ClearOS was easy to install. I just downloaded the ISO image, burnt it to DVD, and then booted the DVD and followed the intructions. The installation is straightforward.

One problem I had was I don't like relying on the DNS that my ISP provides. I'd rather use OpenDNS. I couldn't figure out how to do this--even though I knew you could--and it wasn't easy to find in the documentation or using a Google search. I finally found it by clicking around and decided to document how I did it here.

The trick is to turn off "automatic DNS" on the external interface and then the Web tool will show you boxes where you can enter the IP addresses of your DNS hosts. So, the first step is to edit the interface:

That page will have a checkbox for automatic DNS that is on by default. Uncheck it.

Once you've unchecked the "automatic DNS" box and clicked "Confirm" you'll return to the interface configuration page and see that the DNS server is now a set of input boxes rather than static text:

You can see that I entered the IP addresses for OpenDNS. Now, I'm using DNS servers of my choosing rather than the default from my ISP. You might wonder why I care. One reason is performance. Another is reliability. Most of the time when my family has complained that "the Internet is down" what's really been at fault is the ISP's DNS. The last reason is the additional protection against phishing and content I don't want in my home that OpenDNS provides. It's just one last measure of protection.

7:25 PM | Comments () | Recommend This | Print This

October 5, 2010

Kynetx and Personal Data Services: Project Neck Pain

One of the technologies that seems to be picking up steam lately is the PDS or personal data service. The PDS goes by other names as well. David Siegel calls it a "personal data locker" in his book Pull. Drummond Reed has determined the right name is personal data service rather than "store." I talked about the problem with the name and enumerated some principles for personal data services after IIW DC. I'm certain that the PDS will be a significant topic of conversation at the upcoming Internet Identity Workshop XI in Mountain View, CA (register here).

Why all the fuss? I believe it's because the PDS is the centerpiece in a new kind of Internet; one where the individual sees significant increased utility from the use of their personal data in their behalf instead of having it used against them. At the same time, the PDS offers increased privacy over the current regime.

That said, I think a PDS might be more of a pain than a help if a PDS doesn't come with accompanying automation. If it's just one more thing to manage, then I don't need it. On the other hand, if the PDS helps me by making mundane tasks less onerous and focuses my attention on the decisions that I really must make, then it's a win.

Of course, if you've been following along, you've probably already guessed that I'm thinking the Kynetx Rule Language, or KRL, is the perfect way to automate PDS tasks.

So, without further ado, here's a five minute video that shows a conceptual demonstration of how a PDS and KRL can work together. This conceptual demo illustrates the opportunities that are available for automating the contextual activities that people undertake every day. At the heart of the demo is a personal data store and Kynetx. The interactions are all done using real Kynetx applications that are plumbed in a realistic manner. The scenario uses 5 different APIs and a dozen individual rulesets in the Kynetx system. In the scenario, Scott Phillips gets bad news from his radiologist: he needs surgery. You'll see that a personal data store and a collection of loosely coupled Kynetx apps automate the frustratingly disjointed activities associated with Scott's bad news and focused his attention so he can complete the tasks with the least amount of effort.

Project Neck Pain

I commissioned this demo in mid August in preparation for Doc Searls' VRM+CRM workshop on August 25, 2010. We called if Project Neck Pain (PNP) and it involved much of the company. The goals were

1. Create a compelling demo of what Kynetx could to to make a PDS useful and worthwhile
2. Produce a large application comprising multiple cooperating rulesets
3. Find out where the Kynetx Network Service lacked features in support of goals (1) and (2)

I couldn't have been happier with the result as it achieve all three goals. The video is proof of (1). I'll talk about (2) below. We extended and polished the platform in some significant ways in support of (3). My recent post on building event intermediaries is one small piece of that.

In what follows, I'll describe the various pieces that went into PNP.

Endpoints

To understand what happened behind the scenes in this video, you need to understand a little of the architecture of Kynetx. In the Kynetx Network Service, or KNS, active clients, called "endpoints," raise events. Rules inside the Kynetx Rules Engine respond to those events to cause an action. When an event is raised one or more rulesets might respond to that event on the user's behalf. In the case of the demo, there were four different endpoints involved:

• a web endpoint in the form of a browser extension,
• an email endpoint that uses IMAP to watch a mailbox,
• a telephony endpoint that uses Twilio to make phone calls and respond to user input, and
• the PDS itself.

The idea that a personal data service be not simply a repository of personal data, but an active participant in coordinating activities in behalf of the user is a key piece of the Kynetx vision for how personal data will transform what we think the Internet is and how it works. In the demo, the PDS is a key actor that not only responds to API requests, but also raises events.

As outlined in our free white paper: The Kynetx Rule Language - The First Internet Application Platform (PDF), there is a vital link between events raised by endpoints and rules. For example, the following rule fires off the phone confirmation when the PDS signals that the appointment has been chosen:

rule start_confirm is active {
  select when kpds update_event key "chosenappt" 
           or kpds create_event key "chosenappt"
  pre {
    phoneNum = (datasource:pds({"key":"phone"}))
                            .pick("$.value.number");
    appt = datasource:pds({"key":"chosenappt"})
                            .pick("$.value");
    confirmed = appt.pick("$.confirmed");
  }
  if((confirmed neq "") && (confirmed == 0)) then {
    http:post("https://secrets:here@api.twilio.com/Calls.json") 
     with params = 
      {"Called":phoneNum,"Caller":"8018954878",
       "Url":"http://webhooks.kynetx.com/h/a8x54/outboundcnct"};
    send_directive("log") with 
      app = "PhoneConfirm" and
      desc = "Confirmation Call Started";
  }
}

This rule is selected when the PDS signals that the appointment has been chosen. Note that there are two different ways that might happen and the select statement takes both into account. It gets Scott Phillips' phone number, the appointment data, and doctor information from the PDS in the rule prelude (pre) and then initiates the phone call with Twilio using an HTTP POST if specific conditions are met. One key feature of the event model is that the PDS doesn't have any idea what will happen when it signals that the appointment is set. The PDS simply raises the event. What happens depends on what apps the user has installed and what rules are in those apps.

APIs

In addition to the various endpoints involved in the demo, it also uses multiple APIs, including the follow:

• PDS - the PDS has an API as you can see in the preceding rule. Not only does the PDS allow data to be queried in a permissioned way (right now using OAuth), but it also allows values to be updated or created. At present our prototype PDS doesn't have a generalized schema beyond what is needed for this demo.
• Twilio - as already noted above, the Twilio API is used to make phone calls and gather responses from the user.
• Google Calendar - we are readying Scott Phillips' schedule from his Google calendar using their API.
• Flickr - the pictures painted on Scott Phillips' dashboard are coming from his FLickr account via the API.
• Weather - the weather data is coming from the Yahoo! weather API

The ability to freely use APIs makes the demo very powerful because of the data that can be pulled in. If the demo were real, there would likely be a dozen more APIs that would be useful in helping choose a doctor and schedule an appointment.

Rulesets

Our vision entails multiple coordinating rulesets cooperating in a loosely coupled manner to help the user. Events allow the system to be loosely coupled because, as mentioned above, the raiser of an event does not know who may be listening or what action might be taken because an event is raised. Multiple rulesets may be listening to and event without being aware of each other.

Our goal in creating the demo was to create apps that we thought might actually exist. We wanted apps that had a good backstory, even if they were demo apps. I'll describe the primary rulesets in this demo and how we envision they may really exist.

• Dashboard - the dashboard in the demo is painted by a ruleset. There's no real web page to speak of. Just a blank page with a few divs for structure. Everything on it from the banner, to the weather and time, are painted on the page by a Kynetx ruleset. This app might be provided by the PDS manufacturer.
• TODO List - the TODO list and its management are a separate app that might be provided by the PDS manufacturer or be a standalone component that the user installs according to their own preference.
• PDS Activity Stream - again this is a standard component for the PDS that shows detailed logging data
• Flickr - reads the Flickr feed of the user and shows pictures. This might come with the dashboard or be something the user has found and installed to work with the dashboard.
• Healthcare Action Items - watches the user's email for messages from the health care provider and adds relevant TODO items to the TODO list in the PDS. You can imagine that the radiologist is using a patient management system (PMS) to communicate with the user and has provided a PDS app to the user that is watching specifically for messages from the PMS. The app might be doing other things as well such as proactively reminding users of medical alerts, looking for appointment reminders, and so on. In the demo, we just looked for text patterns, but RDFa or microformats in a HTML message would provide more opportunity to grab meaningful, structured data from the email.
• Doctor Choices - responds to the TODO item about choosing a surgeon by showing the user some choices and annotating the doctors who are in the user's insurance network. This app might come from the user's health care network or an independent provider.
• Select Doctor - overlays the doctor's professional "about" pages with relevant data, augmenting what's there with recommendations from the user's social network, showing data from other Web sites, and any relevant ratings and reviews. In our scenario, we've imagined this app came from Angie's List or a similar service provider ranking site.
• Set Appointment - compares the user's calendar (Google in this case) with the doctor's free-busy schedule so find the intersection and display three choices for the appointment. The app also sets the appointment in the user's calendar and transmits it to the doctor. Currently we know of no standard way to discover calendar data for a service provider. Such a capability would be critical to an app like this working seamlessly.
• Phone Confirmation - called the user with relevant appointment data and asked for confirmation. Note that this app was calling on behalf of the personal data store. This app might be part of the appointment scheduling system or a more general "confirmation" app that the user has installed to confirm certain changes to PDS data using a secondary channel.
• Medical Information Transfer - responds to requests from medical providers for information from the PDS. This might be specific medical information or part of a more general information request system that requests and records the user's authorizations and data transfer preferences.
• Global Configuration - while it wasn't part of the demo that you saw, there's also a separate app that is used to configure the PDS and reset certain fields for demo purposes.

This set of eleven rulesets cooperate to deliver the experience that we saw in the demo. They are independently using the PDS and working with whatever APIs are relevant to their function. The PDS is the centerpiece of the demo since all the apps and functionality revolve around it. The individual rulesets know nothing of each other for the most part. They just do their piece and use events to message other components.

Events enable loose coupling, but they're not magic. While we've avoided it through design in this demo, loosely coupled apps may interfere with each other and give non deterministic performance in real life. For example, two apps may need to write the same field in the PDS. The PDS will need to incorporate appropriate data isolation techniques and, where ordering is important, apps will need to find ways to serialize themselves. Events can do this, but it's not automatic.

Life Events: Helping Users Achieve Purpose

Back in 2001, when we were envisioning how egovernment would work at the State of Utah, we came up with a concept called "life events." The idea was that most people interacted with government when something happened in their life that compelled them to. Further, those events almost never cleaved along departmental lines. A typical life event, like "moving to Utah" involved multiple interactions with many different government and non-government entities at all sorts of levels.

At the time, we envisioned that we'd build out tools on the Utah.gov portal that would help constituents walk through the various steps. The only one that ever got built out, as far as I know, is the one-stop business registration application. I've used it and it works surprisingly well. The reason that more didn't get built was that a server-side portal is just the wrong technology for coordinating all this.

One reason why a server is the wrong place to do this is that the server is missing all the help that active clients provide--specifically the ability to raise relevant events and respond to directives. As a consequence, the server has to simulate this by requiring the user to enter data, click buttons, and select things. That is, a server turns the user into the active client.

Another reason the server is wrong for this is that it's too tightly coupled. In a way, the whole Web services play was a complicated way of trying to create interactions between APIs on the server side. You could make this all work with Web services but it would be quite brittle. Of course, there are Web services technologies for creating more loosely coupled systems, but they're still working on the server away from the user.

In 2010, I think we're a lot closer to actually being able to help people with the tasks and activities that matter to them--their purpose for being online at any given time. The demo shown above is an example of this.

The answer isn't a big, one-size-fits-all, server-side, portal-based application, but a set of loosely-coupled, cooperating apps coordinating around a personal data service. The overall experience is infinitely customizable because any of the components can be traded out by the user. Further, the experience is incremental, meaning that I don't need all of the apps. I see incremental benefit as I incrementally install apps. The experience grows and changes as I customize my environment to suit my needs. What's more, changing out an app for another one doesn't require upgrading or changing the other apps.

Conclusion

There are several important ideas for me in Project Neck Pain that bear emphasis

• The experience focuses the user's attention on the things that matter, critical decisions, rather than making the user fuss with instigating every interaction and manage the details.
• There was no Web site involved in delivering this experiece except for a very simple Web page served with a few divs for structural purposes. Everything else was painted on the page using a Kynetx app.
• You can write event-driven applications that knit together multiple domains like Web, email, and phone. This is a powerful idea that is difficult without the concepts of an active client and events.
• Developers program loosely coupled applications that are event-driven. KNS with a PDS gives developers a very flexible platform for building all or just pieces of any given experience.
• KRL provides significant help in creating apps that use multiple APIs and interact using events. KRL was designed from the ground up with the idea of making writing such apps easier. Having a domain specific language adds significant mental leverage.
• You can build large applications in Kynetx using multiple cooperating rulesets. We've further tested this with another large app that we built for a customer in the last few weeks.
• A PDS without an accompanying automation system is going to be a pain in the neck. A PDS will be just one more huge, complicated thing to manage if it's not a participant in an app ecosystem.
• The demo shown in the video isn't really about health care, but rather, more abstractly, about picking a service provider and scheduling the service.

If any of this is interesting to you, I invite you to signup for a free Kynetx developer account and start playing around. We're happy to help--remotely or in person--and will gladly share the techniques and even the code we used to create this demo.

1:37 PM | Comments () | Recommend This | Print This

CTO Breakfast Meetup this Thursday

In honor of the Utah Open Source Conference happening this week, we'll be holding a special CTO Breakfast meetup on Thursday morning at 8am. The conference is at the Larry H. Miller campus of SLCC. We'll be in room MFEC 123. Kynetx will supply bagels and juice. We'll be done in time for Sam Curren's talk on Webhooks at 10am (you don't want to miss this talk--the Sammage will be incredible).

The next regularly schedule CTO Breakfast is on Friday, October 29 at 8am in the Novell Cafeteria.

9:31 AM | Comments () | Recommend This | Print This