Phil Windley's Technometria

« October 2010 | Main | December 2010 »

November 30, 2010

CTO Breakfast This Friday: Venue Change

We'll be holding the CTO Breakfast for November and December this Friday (Dec 3) at 8am. Whether you're a CTO or not, you're welcome to come. The discussion is about building high-tech products, building companies, and what's hot right now. We'd love to have you join us.

This time we're going to try something new. We will NOT meet at the Novell cafeteria, but rather at Paradise Bakery in American Fork. We'll see how this works. I hope you can make it. January's CTO Breakfast is scheduled for Thursday, January 27, 2011. Put it on your calendar now, or just subscribe.

5:26 PM | Comments () | Recommend This | Print This

November 24, 2010

Podcast: Scott Lemon, Dion Almaer, Ben Galbraith on The State of the Web 2010

The web continues to evolve as new devices appear and new standards are adopted. The iPhone and other mobile devices are now in the mix of places where people go to connect with each other. Dion Almaer and Ben Galbraith call from Belgium where they were attending the Devoxx 2010 conference. They talk with Phil and Scott about internet protocols and social networking and how these are affected by the many different ways people and companies are interacting.

2:07 PM | Comments () | Recommend This | Print This

Readings: Week of November 22, 2010

I saw Paul Kedrosky regularly posts information about what articles he's been reading. I tweet them, but would probably really like to have them on my blog too. So, here goes.

• Clay Shirky: It's Not Information Overload. It's Filter Failure
• Noodlesoft: Hazel
• Geek Love: a Holiday Wishlist of Gizmos, Gadgets, and Gear
• Where do ideas come from?
• Your Internet Driver's License
• iOS 4.3 To Launch In Mid-December With App Subscriptions
• Book: Future Shock
• Adam Savage: TSA saw my junk, missed 12 inch razor blades
• Do Businesses need 63 different types of event?

2:04 PM | Comments () | Recommend This | Print This

November 18, 2010

Clay Loveless: Understanding API Usage

I was in a Point-of-View session at Defrag with Laura Merling of Alcatel-Lucent, Brian Mulloy of Apigee, and Clay Loveless of Mashery. Laura and Brian gave interesting talks, but since they went before me, I was too occupied to take notes. I did take notes for Clay's talk, however, since he went after me. :-)

Clay gave six tips for making an API work:

1. Test it all. Unit tests are just the beginning, but if you don't have them yet, don't start there. Instead, test what your user's experience with end-to-end black box tests. Replaying your access logs is a good way to go since you'll then get tests that mock-up what your users actually do rather than what assumptions you've made. Last thing on testing: validate your return payloads. Getting a response isn't enough since a stack trace isn't the same as getting valid XML, JSON, or what have you.
2. Plan for the future. Use versions on your APIs. This allows you to avoid forklift upgrades that you have to coordinate with all your users. Versions aren't sexy or semantic, but you have to do it anyway. Announce versions well in advance to avoid surprising users.
3. Embrace standards when practical. APIs are better when they're predictable. Standard approaches make tooling and monitoring easier. It's easier to monitor anomalies on non-unique snowflakes. Standards help avoid uncomfortable migrations like the OAutholypse. Standards also enhance runtime validation by allowing you to reject bogus calls earlier in the request pipeline.
4. Monitor everything and be honest. Testing is vital to this effort--you should know before your users. Trends are your friends. To spot trends, you need to be continuously monitoring. Fess up fast--no user wants to think they're your early warning team. Be open by default. Make it part of your nature.
5. Fail well. Well-formed errors win friends. Developers will be more tolerant of failure if you anticipate the possibility and are prepared. Make failures obvious so that monitoring is easy. Try to localize failures so that any given failure doesn't screw everyone. Take care of you big customers.
6. Use a management system. Obviously this is a plug for Mashery, but that doesn't make it wrong. Active monitoring by a service with a support team can be your early warning system. Analytics can help understand how your API is used.

My own talk was about how using events will make users happy. The idea is that client-server architectures require users to "go and get" whereas an event-based architecture lets goodness flow to users without their continual interaction. Web 2.0 was largely about dynamic queries against static data pools whereas the future web will include rivers of information with static queries. My contention is that we need new abstractions for processing these rivers of information. That, of course, leads us to KRL (Kynetx Rule Language) and the event processing layer that Kynetx is building for the Web. I'll do a slide share next week when I'm home and can properly do the voiceover.

4:30 PM | Comments () | Recommend This | Print This

David Weinberger: On Knowing

Dave Weinberger is speaking about "knowing." He starts by asking if "the 'Net is exceptional in the same way of the printing press?"

There are five things that everyone who goes on the Web knows:

• There an abundance of stuff--good and bad
• The 'Net is a permission-free zone
• There is no principle of organization on the 'Net and if there were, it wouldn't be better, it would be worse
• Realize that we built this and its ours
• The 'Net is filled with hyperlinks.

These are unexpected results.

The hyperlink is important because by connecting things, it lets them fall apart. But why do they fall apart? Why do we take advantage of this?

Things that we thought were held together by something important were held together by library paste. Encyclopedias fell apart and become Wikipedia. Newspapers disaggregated. Albums fell apart. These are the easy cases.

Long-form writing, something we believe is vastly improved by being tightly held together is a harder case. Long form writing is a bunch of short pieces held together by deductive reasoning. We proceed from A to Z through a series of steps. There are some weaknesses in this model:

• The author must imagine all the objections and react to them
• The author must avoid digressions that will distract the reader from the path the author wants to lead you down
• Very few long-form writings work

The opposite isn't short-form thought. The opposite is "web-form" thought. Web-form thought is done in public and is constantly being revised--if not by the author, by the things that link to it. The web-form work is never consistent and shapeless.

Knowledge has three shapes:

1. A pyramid: data, information, knowledge, wisdom
2. A foundation, a bottom. Made from hard rock upon which we can build and extend arguments.
3. A rectangle, a book. The media that has contain and communicated knowledge has been a page of a book. Books bound knowledge to brain-sized chuck.

The fact that knowledge is a pyramid means we can reduce it. The foundation means we can resolve arguments. The rectangle means we can master knowledge.

All these characteristics spring from the medium--paper--know knowledge itself. Paper is expensive and libraries are small. Books have to be published and this have an end. Books don't have links and they have ends. Our strategy for mastering knowledge has been one of reduction. Libraries and books don't scale--but the 'Net does.

Knowledge without shape. Clay Shirky's "filter failure" quote is referenced again. In a library the filtering happens when the librarian "doesn't buy" a book. You don't see that. On the 'Net you see the everything so you can look past the filters any time you want. We are not "filtering out" but "filtering forward." This changes the nature of the filter. Filters are "just some kid's playlist."

Traditional knowledge has functioned as a stopping point. These stopping points are backed-up by credentialed sources. This system has the capability of being corrupted because of the power of authority. The 'Net on the other hand is a place of differences and disagreements.

Hyperlinks blow apart the rectangle. Hyperlinks are a new kind of punctuation. Regular punctuation tells you when to stop, but hyperlinks tell you where to go next.

Is the 'Net exceptional? "Hell, yes!"

9:23 AM | Comments () | Recommend This | Print This

November 17, 2010

Dion Hinchcliffe: The Future of Social Analytics

Dion Hinxchcliffe is speaking on the future of social analytics.

• Social analytics is the creation of typed signals by listening to social ecosystems, resulting in the ability to tap into collective intelligence as well as aggregate, mine, and predict outcomes.
• Observable works - transparency for people's work. Getting value from what's observable. Reminds me of Jon Udell's comments from early blogging days about narrating your work.
• Social is how we communicate today. As of last July, 850M using social systems vs 800M using email. Most companies are not here yet. I was speaking to Yammer folks at lunch. Kynetx is a Yammer customer and it cuts down on our email volume considerably. We don't have much Kynetx email. I wonder if startups tend to be more social communicators.
• There are hundreds of public social networks. This leads to fragmenting. There are dozens of internal social channels.
• All of these social trends are increasing social visibility. Social interactions no longer "evaporate." Modern tools keep linkable, searchable information.
• The social universe is a single continuum between employees, partners, customers, and the entire world. All of them might use different channels and tools, but we need to see it as a continuum and gain visibility into it.
• Organizations routinely purge old email for legal and space reasons, but we're starting to realize this isn't always a good idea.
• Knowledge workers spend 20% of their time looking for information they need to do their work.
• Organizations that adopt social tools widely see the amount of data skyrocket after several years leading to management headaches.
• "Information overload is not the problem. It's filter failure" - Clay Shirky
• The people who share their work in organization become the go-to people. They are recognized as contributors.
• Our information landscape is now measured in exabytes. We don't want to delete information, but handling the volume is a real problem.
• Tell me about information shadows. What's the shape of the data? How does it look in aggregate? What does it mean? We don't need to directly see all the information.
• First step is to listen. This is better than crawling and analyzing everything up front. Here the signal. It's the real-time web.
  1. The first filter is the lens of your social capital. Social networking has made your social capital formal. Cultivate it. It's the surface area upon which you can affect change.
  2. Filter 2 is what you care about. Keywords are a early way to do this. Hashtags on Twitter. What's new, important for me.
  3. Filter 3 is what you don't know to ask for. The information you want may find you. Serendipity.
• We need to build, personally and for our organizations, a listening capability that tells us what we need to know. Optionally we might also build a capability to react to this flow--an engagement process. How do we acquire new capabilities. We often gather information but fail to look at it.
• Personally we'll listen in our social environments on our personal devices.
• We will use strategic tools to automate, to scale on aggregate social capital. Generally for businesses. He mentions Friendwheel on Facebook.
• Strategic tools are in their infancy, focus on the outside world, strongly favor new social environments, have limited analytics abilities, don't connect to existing reporting tools well, and are relatively expensive. They also don't exist where you expect them.
• Listening and graphing are easy. Analyzing is harder. Requires a deeper understanding of what you're listening to. Needs semantics to be effective.

One of the themes that seems to be developing at Defrag is one of filtering the information flows. Of course, my take is that filtering is a start, but it's not enough. We need to act on the filtered data. When you change "filter" to "event" you get a bigger, more complete, more sophisticated model for solving the problem.

2:13 PM | Comments () | Recommend This | Print This

Scott Porad: How We Filter 20,000 User Generated Submissions per Day

Scott Porad is talking about how the "I Can Haz Cheeseburger" network of sites filters 20,000 user submissions a day.

• Each site has an "upload" tab. Some have a LOL Builder tag that allows people to compose pictures and text.
• There are about 500,000 submission per month, they publish about 1-2%.
• The name of the game is "quality content" so how do you find the needle in the haystack?
• There is a four step process
  1. Screening - every submission is screened by an employee. There is a system that shows how submission, who submitted, how it was submitted, etc. Screeners look for quality, appropriatness, germanity (no jumbo jets on the cat site), and usually no people. This gets 50% of the submissions because a lot of it is junk.
  2. User moderation - sites have a "vote" tab where users can rate new LOLz. This is the "what's funny" step. They have a five cheeseburger rating widget. What people think is funny changes over time.
  3. User screening - Every entry on the site has a link for things that are incorrect or offensive. Usually this catches inappropriately sourced content.
  4. Editorial curation - Things that make the home page have a final approval by the site editor. Some trends (like the Tiger Woods scandal) can dominate a site. Editorial control is about balanace.
• Technology is used (beyond worrkflow) in only one place where pictures are deduped.
• This is hard to outsource since there's cultural sensitivity that's important.

This is a nice look into the workings of these sites and the problems they face.

12:15 PM | Comments () | Recommend This | Print This

Esther Dyson: On Exploring Yourself

Esther Dyson is speaking about exploring the data about yourself.

• The interesting question is not what you'll die from, but what you'll live with. Our genetics can tell us a lot, but much is still missing or not doable yet.
• Visualizing and explaining data is critical. In order to live in the modern world you need to understand probability and statistic. Otherwise you're uninformed.
• New interfaces (like Fitbit) are changing how we work with data about our selves. Most have warts. None of them interact with each other. (As an aside, this is the problem Kynetx is attacking.)
• Data about us (how many miles we fly, whether we exercise, etc.) is status. How do you design systems, services, and tools where your healthy behavior connotates status.
• Esther wants someone to build the BodyVille app that manages all the data about you.
• Employers will buy and get late adopters to use health apps that create a better, healthier workforce.
• Personal health data is less of a security risk than financial data and we've managed to secure that. The risk is somewhat overblown.

11:46 AM | Comments () | Recommend This | Print This

Stowe Boyd: Social Cognition

Stowe Boyd is talking about social cognition. I always enjoy Stowe's take on things.

• Adding people with high IQ to a small group doesn't add to the groups effectiveness.
• What does make a small group more effecive is adding someone with good social sensibilities. Not correlated with IQ. Correlated with being a woman.
• Groups with good social interaction have less monologues and more interaction.
• Twittering led to students learning more--not just in one class, but across the board.
• Having friendly, personal conversations within a group leads to better performance. Feeling good about the context people are working with is a key to performance. Conversely, confrontation prior to performing a task leads to worse performance.
• Our thoughts, in a sense, are not our own. We think socially. We need to build social tools with an understanding of how people actually behave and what really affects them.

I'm made greater by the sum of my connections and so are my connections.

10:48 AM | Comments () | Recommend This | Print This

Dan Portillo: Scaling Technology Company Cultures

Dan Portillo is talking at Defrag about finding talent to start a company. High-tech does not mirror the larger economy. There's a demand for talent right now and it's hard to hire good people.

• Early start-ups get going and don't worry much about attracting talent--they're just worried about getting going.
• Who are the people in your company who would have a huge impact on your organization if they left. Which or those are at risk of leaving. What are their motivations, drivers, and needs? What's the vision for the organization?
• Find new ways to engage these people. Let them flourish.
• HR functions in companies focus on compliance and transactional issues instead of growing and keeping a great workforce. Process becomes more important than results.
• Scaling has a big effect on the bonds in the company. There's less interaction between the whole team. Make sure people get a chance to co-work, bring people to the mother ship. Be deliberate about communication and be direct and authentic with your employees. As an organization scales, they tend to use "performance reviews" and email blasts to communicate. These aren't enough.
• Give thanks to people. 17% of people quit their job because of inadequate recognition. Let people know their work is valuable. Financial incentive is important, but finding what matters to people is better than a chunk of cash.
• When you're recruiting people, tell a story. Create meaning. Articulate the vision of what the company is doing. What does it mean to work for your company? Position the position. This is bigger than role definition. How does this job relate to the success of the company and why getting the right person is important.
• Create great experiences. Follow through on promises.

As a closing note, Dan started by saying he doesn't do this very often and was nervous. Couldn't tell. He did a good job.

10:01 AM | Comments () | Recommend This | Print This

Alex Wright: Oral Cultures and Social Networks

Alex Wright is opening up Defrag. He's an expert in the history of information and even wrote a book on it: Glut.

• Counting and money begat writing. Commerce was the birthplace or writing.
• People are pre-disposed to classify things hierarchicaly. We don't do well with "tag-cloud" style organization.
• Literacy is fairly recent, so oral traditions are important for how humans have managed information. For example, picture tapestries for teaching religion.
• The 19th century gave rise to the "literate culture." The growth of large "knowledge bureaucracies" in the 20th entury led to a schism of oral and written cultures.
• Oral cultures are additive (stories build on each other), aggretative, participaroty, and situational
• Literate cultures are subirdinative (individual authors assert ideas about the world), analytical, objective, and abstract.
• Facebook has a conversational, fluid, feel. But there is also a hierarchical aspect to it that allows for aggregation.
• The Web provides a place for oral and literate cultures to merge (clash?). Sites with lots of reviews are reflective of oral cultures. The aggregate of the reviews is more interesting than the individual comments generally. At the same time, individual views--especially from experts-- co-exist with these contributed reviews. Yet, they tend to remain separate.
• We're finding ways to encode our relationships. Ebay trust ratings are a shared acknowledgement of meaning that induce people to engage in trust relationships with people they don't know over long distances.

There is a lot of power in looking at the fundamental human impulses the drive us to engage in the online world is specific ways. By understanding thes deeper impulses to commicate orally and with symbols will guide us where we need to keep things apart and where we need to connect.

9:33 AM | Comments () | Recommend This | Print This

November 12, 2010

Gary Crocker: Moving Beyond Start-up Mode

Gary Crocker is speaking at the Utah Technology Council breakfast on building sustainable businesses (in Utah specifically, but the remarks are more general). Gary speaks of several companies that have started in Utah but later moved to other places.

Gary talked about Jim Sorenson calling him and asking him to help him sell his company (Sorenson Research). He doesn't think that had to happen. He thinks that as a leader in fundamental niche (continuous blood pressure monitors) the company could have stayed and grown in Utah.

He tells another story of Ballard Medical selling and the eventual closing of the Draper facility and the jobs being lost. While these exits may be financially good for the founders, they're not good for Utah. What does it take to build sustainable businesses in Utah that can grow and prosper, give the founders good returns, but also keep jobs, technology, and influence in Utah.

Gary asks what financial and cultural characteristics help to create a sustainable business? He gives nine value-maximization and sustainability concepts.

1. Identify, as an explicit part of the business plan, a desire to avoid an early financial exit. Build a sustainable franchise, not a financial play.
2. Hire only those team members whose own internalized goal is to build something that matters. This reminds me of Guy Kawasaki's advice to "build meaning." Many people yearn to do something important with their lives. Find them and hire them. If you take care of this, the financial rewards will come.
3. Invest in those employees committed to long term value creation with long term equity incentives. "Greed in moderation." Reward empowered teams that disdain politicalization and demands product-focused results. "Culture matters." People who aren't focused on "building their career" and are focused on building meaning will deliver results. Start with this and your culture will attract the right kinds of people.
4. Prior to every funding round, identify the value creation events your team must attain to raise funds for the subsequent funding round. Sustainable, long-term players raise money after already having identified the strategy for raising the next round. This ensures you'll spend the money your raise on the right things. Capital planning is a whoafully missing skill in many companies. "If you're not focused on the next round, the current round simply gives you enough money to fail gloriously." This is a lot of work.
5. The CEO and CFO must select financial equity partners who share your long-term commitment to creating exceptional and sustainable institutions--not simply generating short term financial returns. Cultivate your relationships with these partners years before you need them. This takes a lot of time and planning, but if you're going to be sustainable, you need partners who's equity vision is consistent with your own. "Be cautious of those who run OPM." Many funds will force you to exit long before you're ready. Ask "what is the strategic cost of taking this investment?"
6. As part of each funding round, pre-identify which visionary investors will be required to fund the subsequent funding round. Pre-position your value creation vision. Have a schedule for visiting them years before you need their money.
7. Carefully optimize the long term tradeoffs between partnering asset dilution and financial equity dilution--before of "validating partnerships." If you partner an asset, it really isn't your asset anymore to some degree and capital investors will perceive it as not being your asset and will discount it accordingly. If you don't control your value chain, someone else will and they will gain the benefit. Control every part of the value chain of your product. There are substantive costs to "non-dilutive, no cost" partnering. Big company "partners" are seen as "validating" but can be costly. By "assets" include things like "global sales rights," "exclusives," and so on as well as physical and IP assets.
8. Create and insist on a "humble" corporate cultere--be pragmatic and realistic about your capabilities and competitive realities. Understand how your bandwidth and capabilities compare with those of your competitors.
9. It is not about you, it is about the product and long term job and value creation. If you're building something that is long-lasting and sustainable, then you eventually won't be there. It's about product and culture that will long outlast you.

Some ideas from the Q&A session afterwards.

• Build a firm that is growing and you will have all the assets needed to be around long-term. You'll have your currency. Acquire others, don't let them acquire you.
• Even in this tough capital environment, there are partners who will invest and be good advisors. You need BHAG goals. Be audacious and there are people who will invest in the idea. Don't think too small.
• If you want to keep the company in a particular location, hire people who want to live in that place. Kepe a consistent culture.
• Think sustainable from the very first time you start building your business model. Don't try to retrofit it later--it's tough.

9:09 AM | Comments () | Recommend This | Print This

November 11, 2010

The Real-Time Web

Over the last several years, we've witnessed a dramatic shift in how people use the Web. What started as interactive Web applications under the moniker "Web 2.0" has become a firestorm of social applications like Facebook, Twitter, and Foursquare, among others. But underlying these changes is something even more important than the "social" Web: the "real-time" Web.

The real-time Web is a radical shift in how people use the Internet: rather than simply viewing static pages, or even interacting with a Web site, the real-time Web uses dynamic streams of information to present contextual, relevant experiences to user.

These dynamic streams of information include such diverse data flows as Twitter streams and Facebook news to RSS feeds of product recalls. Many of these information streams are enabled and supported by APIs since programmatic access to the data is critical to its reuse in various guises.

Information reuse is a major premise of the real-time Web since much of the information available is not nearly as interesting by itself as it is in combination with other data streams. Further, this reuse is often highly personal since what you want from the information stream and how you want it mixed with other data is different from what I want.

Making streams "real-time" skews design to one favoring interrupts over polling. Protocols like PubSubHubBub and services like push.ly are created with the explicit purpose of taking polling-based technologies like RSS and giving them an interrupt-driven façade.

I think we've only begun to see and imagine the world that will result from real-time access to information streams that are recombined in myriad ways. This is especially true when you start to consider the possibilities that personal data, properly protected and permissionsed, adds to the mix.

9:45 AM | Comments () | Recommend This | Print This

November 9, 2010

The Dawn of the Eventernet

This is a guest blog post by Dave McNamee

The Eventernet is an event architecture overlayed on the Internet, where events are published and applications process those events and take action on behalf of people. This allows for the creation of new value that was not possible under the constraints of the request architecture of the WWW.

The World Wide Web

In the beginning there was HTTP with its collection of methods. GET, POST, PUT, DELETE, etc. These methods, along with the advent of HTML (and browsers that could render HTML) allowed the World Wide Web to emerge and flourish. Suddenly the big-I Internet had a mainstream purpose.

GET and POST are the methods that get the most exercise. Billions of people are using browsers as an interface to GET documents from web servers. Of course, in the fifteen years since the explosion of the WWW, the content of those documents has become more dynamic and sophisticated, but the basic request method has remained the same, and the basic paradigm of requesting documents has not changed much until recently.

Some platforms (hegemonies?) have emerged which are attractive to developers for their functionality, user base, and comScore ratings. These platforms have extended their reach dramatically by publishing APIs. Now users are accessing functionality from these platforms through third-party interfaces rather than through requesting documents directly from the platform providers.

RESTful architectures make API consumption easier because of their fidelity to the methods of HTTP. The ascent of internet-enabled native applications on mobile platforms is adding fuel to the API fire. However, we are still stuck in a request architecture with its inherent limitations.

Why We Need the Eventernet

At this point I am reminded of a discussion that many married couples will recognize. It goes something like this:

Wife: "You didn't do X and I am angry."
Husband: "You never asked me to do X."
Wife: "I shouldn't always have to ask you to do X. You should just know."
Husband: "How am I supposed to know that you want me to do X if you don't ask me?"
Wife: "If you paid attention you would know that X needs to be done."
Husband: "Hmmph, grumble."

The hapless husband in this story is operating under the assumptions of an request architecture. He assumes that if the wife wants something done, she will request it. Unfortunately for him, in this scenario the wife is operating under the assumptions of an event architecture. She expects that the husband will observe events around him, put them into context, and synthesize them into the expected action. Life would be better for both of them if he were better at processing events!

Just like in my example, there are many events that occur in the world that, if synthesized into a desired action, would create great value to individuals and companies alike. Intrepid developers are beginning to use webhooks to message explicit events for processing by an external application, but these systems are still a closed loop: the events published are predetermined and intended for a specific application. It is impossible for the published events to be synthesized by some other system into a valuable output.

The Eventernet

Enter the Eventernet. Eventernet is a term I made up in an effort to describe this architecture. The Eventernet is comprised of a few key components: event publishers, event hubs, event processors, and client applications.

The Eventernet is built on top of HTTP and makes use of the standard methods, however, instead of GETting documents, an event publisher uses GET or POST to publish an event to an event hub (or directly to an event processor). Event processors subscribe to those events in a manner similar to Pubsubhubub and process them in a rules engine to determine the action that should be taken on behalf of individuals or other entities. The event processor transmits instructions, called directives, to client applications that benefit the individual or entity. In some cases, the event publisher may also be the intermediary between the event processor and the client application, managing both up and down communications.

There are different flavors of event publishers. Some are publicly available and are already published through APIs, such as the public tweet stream or stock market values. Others will be directly connected with individuals and are controlled by them, such as cell phone GPS data or expressed interests. Some people believe that Personal Data Exchanges will serve as the event clearinghouse for individuals. Whatever the architecture for publishing the events or data, the fact remains that they are useless unless they are processed in a way that generates value for all involved.

Kynetx and the Eventernet

The Kynetx platform includes elements of this architecture. We have event publishers, the world's only platform-agnostic event processing engine, event hubs, and client applications. We are also developing rudimentary personal data storage functionality as we await the emergence of a complete Personal Data Exchange architecture.

I believe that entirely new industries and value networks will be built upon this architecture. The world will not soon abandon the traditional request architecture of the web, but to evolve we must augment it with an event architecture.

10:49 AM | Comments () | Recommend This | Print This

November 2, 2010

Discovery: Webfinger and OpenID Connect

I'm sitting in a session on webfinger, OpenID Connect, and discovery session. Discovery is a the process of turning a small piece of information (like a user ID) into the URLs and APIs needed to service some specific request. For example, say I tell you my email address is windley@gmail.com, how do you find my profile? Of course, as long as we're talking about one site, like Google, we can just hard code that translation. But how can the discovery problem be generalized?

That's the goal of Webfinger: WebFinger is about making email addresses more valuable, by letting people attach public metadata to them. You can try it yourself at webfinger.org (try it with your GMail address, for example).

There's also the related problem of how to know, for some particular host, where to get the webfinger data. That's the job of the host-meta file, a well-known URL proposal.

For example the host-meta data for Google is here:

http://gmail.com/.well-known/host-meta

and it returns

<XRD xmlns='http://docs.oasis-open.org/ns/xri/xrd-1.0' 
     xmlns:hm='http://host-meta.net/xrd/1.0'>
  <hm:Host xmlns='http://host-meta.net/xrd/1.0'>gmail.com</hm:Host>
  <Link rel='lrdd' 
        template='http://www.google.com/s2/webfinger/?q={uri}'>
    <Title>Resource Descriptor</Title>
  </Link>
</XRD>

This tells us that we can get data about a GMail account from the URL

http://www.google.com/s2/webfinger/?q={uri}

by substituting the GMail address for {uri}. So we can get my webfinger data from

http://www.google.com/s2/webfinger/?q=windley@gmail.com

This URL returns the extensible resource descriptor (XRD) as follows:

<XRD xmlns='http://docs.oasis-open.org/ns/xri/xrd-1.0'>
  <Subject>acct:windley@gmail.com</Subject>
  <Alias>http://www.google.com/profiles/windley</Alias>
  <Link rel='http://portablecontacts.net/spec/1.0' 
        href='http://www-opensocial.googleusercontent.com/api/people/'/>
  <Link rel='http://portablecontacts.net/spec/1.0#me'
        href='http://www-opensocial.googleusercontent.com/api/people/103887646945247867113/'/>
  <Link rel='http://webfinger.net/rel/profile-page' 
        href='http://www.google.com/profiles/windley'  
        type='text/html'/>
  <Link rel='http://microformats.org/profile/hcard' 
        href='http://www.google.com/profiles/windley'  
        type='text/html'/>
  <Link rel='http://gmpg.org/xfn/11'  
        href='http://www.google.com/profiles/windley'  
        type='text/html'/>
  <Link rel='http://specs.openid.net/auth/2.0/provider'  
        href='http://www.google.com/profiles/windley'/>
  <Link rel='describedby'  
        href='http://www.google.com/profiles/windley'  
        type='text/html'/>
  <Link rel='describedby'  
        href='http://www.google.com/s2/webfinger/?q=windley%40gmail.com&fmt=foaf'  
        type='application/rdf+xml'/>
</XRD>

If you look closely, you'll see that there is a subject, and alias, and a lot of URLs that are tagged in ways that tell you categorically what kind of thing they return about the subject. For example, the entry with

rel=http://microformats.org/profile/hcard

tells you where to get HCard data about me.

You might notice that there's a lot of XML here. There are proposals to turn this into JSON, such as JRD, the JSON resource descriptor. Lots of discussion about why this is better, easier, and so on.

One extension is to allow for access tokens to get non-public information. For example, you can get my publicly available information from the profile URL, but what if I've been to your site or app and allowed you access to non-public data? Can you get it using this mechanism. What's the standard for specifying how to pass the OAuth tokens, for example.

So, if I understand correctly, OpenID Connect is a variation on Webfinger that uses JSON, extends it in important ways, and allows OpenID (and other systems) to dynamically put relevant links to services on sites without hard coding them. This allows small players to compete in the NASCAR game. Most service providers won't be big enough to get their button hard coded on a site, discovery allows them to get dynamically added when the site knows that they're relevant.

3:23 PM | Comments () | Recommend This | Print This

Essential Characteristics of a Personal Data Store

I'm in a session at IIW where personal data stores are being discussed. Drummond Reed and Paul Trevethick are moderating.

Someone asked Paul why Facebook isn't a personal data store. Certainly is a store of personal data. Facebook has a rich and powerful mechanism for sharing data with apps. But...

• They can change the terms any time they like
• They monetize my data--who owns or controls the transactions

Mary Hodder gave a "bank vs bar" analogy. Facebook is a bar, not a bank. There are lots of options and things to do, but no fiduciary responsibility or interoperability.

The point is that a PDS ought to have specific characteristics or else everything will be a PDS. Every company will say "we have a PDS!" and it will be as meaningless as "SOA governance."

Here are some characteristics that we've discussed

• Controlled by a person
• Virtually distributed
• Seemless permissioned access to your digital data
• Data portability
• Interoperability
• Provisions for ownership, co-ownership, authority
• Control access to data for service creation

We could probably come up with more, but this is a pretty good list.

12:27 PM | Comments () | Recommend This | Print This

November 1, 2010

Starting a High Tech Business: Finding a Market

When you're starting a business, you'll hear a lot about product/market fit. Marc Andreesen says:

In a great market -- a market with lots of real potential customers -- the market pulls product out of the startup. The market needs to be fulfilled and the market will be fulfilled, by the first viable product that comes along. The product doesn't need to be great; it just has to basically work. And, the market doesn't care how good the team is, as long as the team can produce that viable product.

In short, customers are knocking down your door to get the product; the main goal is to actually answer the phone and respond to all the emails from people who want to buy.And when you have a great market, the team is remarkably easy to upgrade on the fly.

This is the story of search keyword advertising, and Internet auctions, and TCP/IP routers.

From The Pmarca Guide to Startups, part 4: The only thing that matters - pmarca *ARCHIVE* - bit.ly/1z2b3V
Referenced Mon Nov 01 2010 11:52:51 GMT-0600 (MST)

Note: I've quoted a small portion of a long article that is well worth reading.

This is undoubtedly true, but one of the mistakes that many people make in reading statements like this is thinking of "markets" as these things that you can go find, check out, and pick up like a head of lettuce from the supermarket.

In truth, there's no static collection of markets. Further, they're remarkably difficult to analyze and understand to the level of details someone does when they're building a product. In fact, there's only one sure fire way to explore a market and that's to build products and try to sell them.

This is a state space exploration problem that evolutionary algorithms are great at exploiting and on the macro scale, that's exactly what happens when thousands of companies build and sell products. They find the fit.

Of course, if you're starting a business, you don't want to solve the problem on a macro scale, but on a micro scale. Specifically, you want to find a market for your product, or a product for your market depending on how you look at it. And that's the point. You can't afford to look at it from either of these perspectives. You have to consider the problem holisitically.

That's why people make such a big deal about having a good team. A good team, with strong leadership will constantly adjust product and strategy based on signals that are coming back from the world (I specifically don't say "market") about how well their product fits a need.

Consequently, it's difficult to say that team, product, or market is paramount in building a startup. The truth is that you need all three and successful startups are constantly iterating all of them. If you're not getting traction, you'd better adjust and quick. You'll never be smart enough to look at the market, see a need, and build a product that fits, although you may get lucky. What you can do, however, is quickly iterate through variations on a theme to find what works.

12:07 PM | Comments () | Recommend This | Print This