Phil Windley's Technometria

February 8, 2010

Build 424: Functions and Array Operators

The latest build of the Kynetx Rule Language (KRL) provides a significant upgrade in capability with the addition of functions. We’ve also added some new array operators that take advantage of functions to make using arrays easier.

KRL supports functions as first-class objects in the expression language. KRL supports only anonymous functions, but they can be given names by binding them to a variable in a declaration. Here’s an example:

pre {
  add5 = function(x) {
           x + 5
         };
}

Functions are evaluated statically (e.g. the environment they are defined in, not the environment they are executed in determines the binding of free-variables) and can be recursive. Here’s an example of a recursive function in KRL:

pre {
  fact = function(n) {
            (n <= 0) => 1
                      | n * fact(n-1)
         }
}

Functions are declared using the keyword function and contain optional declarations followed by a single expression that returns the result of the function when executed. To see this, consider the following example which uses Newton’s method to calculate square roots (taken from Section 1.1.8 of Structure and Interpretation of Computer Programs):

sqrt = function(x) {
    average = function(x,y) { (x + y) / 2 };
    good_enough = function(guess, x) {
       v = (guess * guess) - x;
       v < 0.01 && v > -0.01
    };
    improve = function(guess, x) {
       average(guess, (x / guess))
    }
    sqrt_iter = function(guess, x) {
       good_enough(guess, x) => guess
                              | sqrt_iter(improve(guess,x), x)
    };
    sqrt_iter(1.0, x)
}

Functions can return functions as values and functions can be passed as the arguments to other functions and operators in KRL. The following example defined a generalized summation function that sums the numbers from a to b incrementing using inc and applying the function f to each term:

sum = function(f, a, next, b) {
  (a > b) => 0
           | f(a) + sum(f, next(a), inc, b)
};
inc = function(x) { x + 1 };
cube = function(x) { x * x * x };
sum_cubes = function(a, b) {
  sum(cube, a, inc, b)
}

We could define a function that creates incrementor functions. When given a number, it returns a function that increments by that value:

inc_generator = function(n) { function(x){ x + n } };
inc = inc_generator(1);
inc_by_2 = inc_generator(2);
inc_by_25 = inc_generator(25);

Being able to write functions adds significant power. More so with some of the other languages changes we have in mind for the next few months.

Weve also added several new array operators in recent builds. Most notably, array references now work as follows:

a = [1,4,3,6,5];
b = a[1]

This would bind the value 4 to the variable b. Note that array references only work for arrays of one-dimension, so c[1][2] is not allowed (presuming c is an array of arrays).

In addition, there are a number of new operators available for arrays. The following array operators are now available (in addition to length which has been previously available):

sort - sorts the array. With no argument, sorting is done in ascending order. The argument "reverse" causes sorting to happen in descending order. The argument can also be a function that takes two argument and returns a boolean value which will be used as the comparison function for the sort.
filter - filters an array, producing a new array. The operator takes a function argument that takes a single parameter and returns a boolean value. The return array contains elements for which the function returns true.
map - modfies an array from mapping a function to each member of the array. The operator takes a function argument that takes a single parameter and returns any value. The array returned from map is the result of applying the function to each member of the original array in turn, collecting the results into a new array.
head - returns the first element of an array without modifying the array.
tail - returns an array that is identical to the orginal array except without the first member.

You could use these like so:

pre {
  f = function(x) { x < 4 };
  g = function(y) { y * 2 };
  a = [1,4,3,6,5];

  b = a.sort(); // returns [1,3,4,5,6]
  c = a.filter(f); // returns [1,3]
  d = a.head(); // d has the value 1
  e = a.map(g); // e has the value [2,8,6,12,10]
}

Operators are fairly easy to add and handy to have, so if you have ideas for other operators, on arrays, strings, and so on, just let us know.

February 6, 2010

Redirectionless OAuth Credentials Exchange

Image representing Twitter as depicted in Crun...

Image via CrunchBase

Am I missing something here? Twitter is working with select partners to test what is variously being called OAuth delegation or browserless OAuth credentials exchange method (not sure why browserless since it’s not about the browser, it’s about the redirection).

The bottom line is that in an effort to be more user friendly, this removes the redirection to the Twitter site where you authoirize access by letting the third-party site (the site being delegated to) collect and then pass along the user’s username and password to get the OAuth credentials. Abraham Williams captured the POST headers from Seesmic Look and they clearly contain the username and password.

I don’t see how this can be a step forward in secure third-party access to APIs like Twitter. Once users start being allowed—even required—to (again) enter usernames and passwords into third-party sites, they’ll be ripe for phishing attacks. Maybe I’m misunderstanding this based on the scetchy information available, but it looks phishy to me.

February 5, 2010

Subscription Models are Chic

Image representing Dave McClure as depicted in...

Image via CrunchBase

A recent blog post by Dave McClure, the investor in charge of the Founders Fund seed investment program makes the assertion that “subscription models are the new black” and we’ve lost a decade of innovation by people living off the table scraps of Google’s $10B pay-per-click ad system. (Warning: the blog post is pretty raw.)

In a seeming non-sequiter, he moves on to talking about passwords. But pay attention, because what he’s really doing is talking about friction in subscription models and the friction that they inpose. I think it’s interesting that the iPhone app store, for example, still requires that I type a password when I purchase an app on my iPhone given that they have a good identification based on the device. Of course, what they’re doing is using the password for authorization. Making sure it’s me who’s purchasing the app.

January 28, 2010

CTO Breakfast Tomorrow - Free Pizza Tonight

Tomorrow is the CTO Breakfast. I know, kind of late notice, but this one snuck up on me. I’ve been heads down with a major ope-heart surgery of the Kynetx rules engine. I’ll tell you all about it tomorrow at the breakfast—refactoring code provides lots of opportunity for reflection on software development.

The breakfast will be held in the usual place (Novell Cafeteria in Provo - map) at 8am.

Anyone interested in how information technology is used to build products or run companies. Despite it’s name, you don’t have to be a CTO to attend—just interested in technology, where it’s headed, and the problems of starting and building a high-tech business in Utah. I

Also, tonight at 5pm, Kynetx is hosting Kynetx Fireside Chat (code-bash and developer get-together) at our offices in Thanksgiving Point (map). Free pizza, interesting discussions, and good friends. You’re invited, so come on over.

I hope you can make it to one or both of these events. See you there!

January 21, 2010

Starting a High Tech Business: Do What's Important by Embracing the Urgent

ANAHEIM, CA - MARCH 20: Jimmer Fredette #32 o...

Image by Getty Images via Daylife

I’m starting a new business called Kynetx. As I go through some of the things I do, I’m planning to blog them. The whole series will be here. This is the twenty-second installment. You may find my efforts instructive. Or you may know a better way—-if so, please let me know!

Last Saturday I attended the BYU vs Colorado State basketball game and came away with a lesson I hadn’t anticipated. When the game first started, things seemed pretty even matched and you’d have had a hard time seeing significant differences between the teams. They were both executing pretty well and the score was close. Over time, however BYU pulled away so that by half-time they were a dozen or more points ahead and ended the game with BYU ahead 91-47. BYU won because they were executing well as a team and playing good ball. Over time BYU’s small wins compounded to result in resounding defeat.

I think the same thing’s true in business. Watch any set of startups in their early stage and you might not see huge differences—but they’re there. There’s one or two that are executing better and over time, that difference will compound.

Steven Covery talks about “urgent” vs “important” tasks. We see those all the time in business. Urgent tasks, like raising money, often get done at the expense of important tasks, like meeting with customers. On the tech side, we spend time getting new machines configured in the data center or setting up a mail server, rather than adding new features to the product. Over time, some companies are getting the important things done and others aren’t. Guess who succeeds.

The problem is that you can’t simple say “I’ll ignore the urgent tasks.” They’re urgent, after all. You won’t survive if you ignore urgent tasks, but you’ll never thrive until to get past them and move onto important tasks.

Getting past the urgent tasks means dispatching them as quickly and efficiently as possible. In other words, you have to embrace and fully attend to urgent tasks in order to keep them from eating you up. That’s contrarian, but it’s true. Much of this series has been focued on exactly those kinds of issues. Here are a few things we’ve learned:

Automate everthing you can. This is one important task that you can do that will help slay the merely urgent ones. There’s never been a greater set of tools for automating the mundane tasks of administering high-tech products. Kynetx went two years without a fulltime sysadmin and we spent only a few hours per week making all the IT work. An additional benefit of automation is that tasks not only get done, but they get done consistently, improving uptime and reliability. That’s a huge win.
Outsource everything you can. There are all sorts of tasks that you can just give to someone else and largely forget about. This includes, PR, HR, payroll, travel, finance, email, servers, storage, and so on. Sometimes “outsource” means hiring part-time consultants (like PR or HR specialists) and other times it means using a Web service (like GMail or TripIt). Most urgent tasks can be handled this way. Don’t spend your time doing something that isn’t core to the business.
Take care of finances now. We hired a part-time CFO early on for strategic advice, but neglected the books until later. Once we got them set up it was like someone had switched on the lights. Next time, I’ll do it sooner. Having your books set up right and tracking your company’s financial health from the start is critical. The longer you take to get your books set up the hard it will be and then you’ll not have the information you need to make critical decisions.

Have the discipline to do urgent tasks “right” rather than just doing enough to get by. Whenever something urgent pops up, figure out how to accomplish the task in a way that ensures that you won’t have to do it again. That kind of execution is what will separate you from the competition over time. Your ability to spend just a little more time on what’s important each day will compound over time and lead to crushing victories in the end.

January 13, 2010

Using OAuth to Access Twitter from KRL

The latest build (Build 391) of the Kynetx Rule Language (KRL) includes support for accessing Twitter data intrinsically within the language. Integrating interesting data with KRL is an important part of what makes the language so useful for building cross-site applications that mash-up data and user interactions. But what’s really interesting about this release is that we’re using OAuth to access the Twitter API and have built primitives into the language for dealing with the Twitter OAuth interaction to save developers from doing it.

Not only are we making it easy for developers to write apps that use Twitter, but this is also the first time that end users will see significantly different behavior from an app than their friend might. An app that uses the Twitter library will use my Twitter data for me and your Twitter data for you. Using OAuth, Kynetx apps can now be personalized.

Using Twitter data inside a KRL app generally involves two KRL patterns: authorize then use and initialize then populate

In the authorize then use pattern, a rule is put in place to check if the app is authorized to take a certain action and, if not, do what is necessary to complete the authorization. What makes this work is using the rule postlude to ensure that the rest of the rules (which presumably rely on the authorization) don’t run. Here’s an example from an app I wrote to demonstrate using the the new Twitter library:

rule auth is active {
  select using ".*" setting ()
  if(not twitter:authorized()) then
      twitter:authorize()
        with opacity=1.0 and
             sticky = true

  fired {
     last
  }
}

Notice that this rule only fires if the predicate twitter:authorized() is false. The action, twitter:authorize(), is what initiates the OAuth ceremony. The action will pop up a notification in the user’s browser that looks like this:

OAuth ceremony initiated by a KRL application

The postlude of the rule (fired {...}) runs the last statement if the rule fires to ensure that nothing else happens. Of course, if the app is authorized, the rule doesn’t fire, the OAuth ceremony is not initiated, the last is never executed, and the remaining rules in the ruleset are evaluated.

The initialize then populate pattern is important any time you’re working with complex data. With complex data, you will frequently need to do something for each component of an array. That’s what the foreach statement does as part of the rule selector: executes a rule once for each member of an array.

The problem is that if we use a foreach to loop over the tweets we get back and use notify to place them on the page, we’ll end up with one notification box for each tweet. Not very pretty.

A better solution is to use a rule to place the notification box (the initializer) and another rule to loop over the tweets and place them in the notification box (the populater).

Here’s the initialization rule:

rule init_tweetdom is active {
  select using ".*" setting ()

  pre {
    init_div = <<
<div id="tweet_list">
</div>
>>
  }

  notify("Friends Tweets", init_div)
    with sticky=true and
         opacity = 1.0

This is a pretty simple rule that places an empty notification box on the page.

The real work is done by the populating rule:

rule populate_tweetdom is active {
  select using ".*" setting ()
    foreach tweets setting (tweet)
      pre {
        text = tweet.pick("$..text");
        div = "<div style='background-color:#666;margin:2px'>" +
              text + 
              "</div>";
      }
      append("#tweet_list", div)
}

This rule loops over the tweets, grabs data out of them using pick, and appends the result to the div in the notification box.

The tweets variable was set in the global block:

tweets = 
  twitter:authorized() => twitter:friends_timeline({"count": 8})
                        | [];

After you’ve gone through the OAuth ceremony at Twitter, wherever you run this app, you will see a box like this that contains the last eight tweets from your friends timeline on Twitter. Here’s the results for me:

The last eight tweets from my friends timeline on twitter.

Of course, if you run this app, either by card or bookmarket, you’ll see the results from your friends timeline.

Craig Burton has created a nice tutorial about how all this works. There’s some important data in the tutorial about how to get the keys from Twitter for your application. The library is also well documented and the source code for the ruleset is available.

The ability to personalize apps by appealing to personal data elsewhere on the Web is a huge step forward for KRL. Look for other APIs to be embedded in KRL in the near term and, eventually, more general support for OAuth so that developers can use any OAuth protected data source.

January 12, 2010

UtahPolitics.org All Over the Web: A Kynetx App

Since 2003, I’ve been running a site called UtahPolitics.org. The site started off as a blog on which I and others posted articles. When I started UtahPolitics.org there was some speculation about my motives. But my motives are simple: create a place I can experiment with new media in an arena that interests me.

Last year, in an effort to continue the experiment, I put up a retweeter for UtahPolitics.org that would retweet any tweet from friends of the @utahpolitics account that contained the tag #utpol. When I did that I also morphed the web site—imperfectly—to a site that just aggregated those tweets. The idea was to move the discussion off of UtahPolitics.org and onto Twitter.

The problem with this is that it’s sometimes difficult to follow the thread of a conversation, particularly those about particular pages on the Web. To try to solve that problem, I’ve build a Kynetx app that will show all the #utpol tweets about page on that page when you visit it. Here’s a picture to show you what I mean:

UtahPolitics.org app running on a legislative page

The box on the right is showing any tweets that reference the page its on and contain the tag #utpol. Right now it’s tweets by anyone. If that gets to be a problem, I’ll restrict it to friends of @utahpolitics so that I have a way to block people.

I’m hoping that you’ll use this app to follow Utah Politics during the next legislative session and contribute to the Twitter discussion about specific bills or newspaper articles. Using this app and your Twitter account, you can comment on legislative pages, newspaper pages, and so on. Anyone else who has the app will see your tweets and be able to respond on the page.

You can use this app in one of three ways:

Download the AzigoLite Card selector and install the Utah Politics card - this is the most flexible solution since once you have the card selector installed, you can also use other Kynetx apps without any further downloads. Just install the card and go.
Install a browser extension - this is probably the fastest and most convinient way to get going. Just download the right extension for the browser you want to use (or several if you regularly use more than one browser) and you’re good to go. The following extensions are available:
Use a bookmarklet - this is the simplest, but has the drawback of working only when you click on it. You may not always remember to do so and will miss out on people’s comments. To use the bookmarket, drag this link to your bookmark bar at the top of your browser
UtahPolitics Bookmarklet

Once you’ve got something installed, head on over to House Bill 10 on the Utah Legislature’s page to see it in action. I’ve left some test tweets there.

You don’t need to do anything special to use it. Just tweet about a page and include the tag #utpol. All the tweets about a particular page will show up in the app—even from people who don’t have it installed. This is an assymetrical communication tool. Note, if there are no tweets about a particular page, you won’t see anything—the app doesn’t do anything in that case.

If you’re interested in how this was made, I’ve made the source available on the app page. Feel free to ask questions or just copy it and build your own—Kynetx developer accounts are free.

I’m excited to see how this works—I hope it leads to some interesting discussions about Utah politics and gives some insight about how apps like this one—that work on multiple Web sites—can be made more effective.

January 11, 2010

On Science, Society, and Democracy

Image via Wikipedia

A few weeks ago a friend said something about science that helped me understand why many people misunderstand it. He said “science isn’t the only way I have of knowing the truth.” He was looking at science as a way for individuals to know things. Certainly science informs us all—and that’s at the heart of the misunderstanding—but that’s not the point of science. Science is how we as a society can know the truth about the world we live in.

Another example of a societal truth-finding system is the justice system. A court of law is a way to find the truth about events involving several people. While far from perfect, courts are still the best way we have to determine the truth in these situations and as a society we work very hard to ensure that they function well. We all have legally prescribed duties towards them like jury duty.

Science is best viewed the way we view democracy: Democracy is the best way for societies to organize themselves and make decisions in ways that respect and protect individual rights and freedoms. Science is the best way for society to understand the world around us and ourselves. Science and democracy serve each other and niether can long survive without the other.

If society comes to distrust science—and many do—then where will we turn for answers? What other system does our public discourse have for finding out about the world? Our democratic institutions depend on science being healthy and trusted. Democracy is no substitute. It’s a great system for making decisions, but a rotton system for finding the truth. Science is the best societal tool at our disposal for knowing thing about our world.

I’m not arguing that science is infallible. Scientists are people and are consequently subject to all the same foibles as the rest of us. But like a court of law, there are rules in science that ensure that these human failings have limited affect on the result. Do scientists sometimes doctor results and cheat? Sure. Are they sometimes decieved? Sure. But, like a court of law or the democratic process, science is a system for getting good results in spite of these problems.

Politicians frequently misuse science. Anti-intellectual politicians (populists in all parties, but more so in the Republican party than the Democractic party as of late) dismiss it as just another special interest. Progressives and environmentalists (who I’m distinguishing from scientists, although a single person may, obviously, be both) frequently use science to scare us into adopting their agenda. Most people don’t know enough science to distinguish the political wrangling from the science.

This is to bad, because it causes us to distrust science. People will always use social institutions like science, the courts, and democracy to get their way. That is human nature. The way we protect these institutions is by understanding them and being vigilant in their defense.

Most people, despite a dozen years of compulsory education in science as children don’t understand it or how it works. Like democracy, the ultimate defense of science lies in education. My kids learn a lot about science facts. They learn little about the methods of science or how to read reports and think critically about what they read, using their knowledge of science.

I don’t expect politicians to change anytime soon, but I do think more of us need to speak out in defense of science and what it represents for our society. Unless we do so, we will find our society adrift without any means of getting the good, trustworthy information that democracy needs to make good decisions.

Phil Windley's Technometria

Organizations Get the IT They Deserve

Technometria

About Phil

Essays

Software

Topic Guides

Categories

Archives

Utah Tech Organizations

Friends

Copyright

I'm CTO of Kynetx

Free Newsletter!

Search

What I'm Doing

Find Me Elsewhere

Digital Identity

Upcoming Events

Recent Entries