My G+ Profile
« August 2011 | Main | October 2011 »
September 26, 2011
E-Verify Isn't the Answer
A bill that would require employers nationwide to use the Department of Homeland Secutity's E-Verify system to authenticate an individuals legal work status has has made it through the judiciary committee. To many, applying information technology to what seems like a nice, tidy identity problem seems like a quick, easy fix. But, this is not a simple identity problem.
A 2009 independent report (PDF) by Westat of Rockville MD shows that the system is 99.2% accurate. That sounds pretty good, if we're talking about shooting percentages or exam grades. But there are systems where 99.2% isn't good enough. I like my bank to be much more accurate than that, for example.
Here's what the 99.2% accuracy rate means for E-Verify. US employers hire approximately 60 million workers each year. A 1% error rate (rounding) means that 600,000 legal workers will be mistakenly classified as not qualified to work. Once the system misclassifies you, the onus is on you to correct the problem. You are guilty until you prove that you're innocent, so to speak. The study found that 22% of peopled who were classified spent $50 to correct the problem and 13% spent over $100. Half of misclassified workers missed work to correct the problem. These numbers might get better with time as more and more workers correct problems with their records, but I'm skeptical. There will always be documentation mistakes.
The accuracy isn't likely to get better without significant changes to the US government identity regime. One percent is pretty good for a system based on names and social security numbers. Increasing the accuracy would require moving to something like biometric IDs for everyone. India's doing it, but I doubt the US is ready to that step yet and even if we were, it has it's own accuracy problems.
I can hear some people saying "But that's a small price to pay to keep illegal immigrants from stealing jobs from Americans!!!" If only. The problem is on the other side of the accuracy issue. Westat estimated that "54 percent of unauthorized workers screened through E-Verify are incorrectly confirmed by the system, usually because they use borrowed or stolen identity data." That's shocking. The system that Lamar Smith (R-TX) wants to force every business in America to use doesn't even work.
These numbers are all based on a small population of employers who use the system voluntarily and employers from Arizona who are required by state law to use it. The accuracy rates are likely to get worse as the system is used more broadly since voluntary users are more likely to be conscientious about using it that employers who are forced to. Moveover, bad actors will game the system and expand identity fraud.
From a systems perspective, government mandates like this remove flexibility that is necessary for getting things done. Expanding E-Verify isn't an easy fix to solving immigration problems. In fact, it's likely to make things worse for employees and employers while expanding identity fraud.
9:56 AM | Comments () | Recommend This | Print This
September 21, 2011
A Facebook of Things
Marc Benioff has spoken about "social products and services" at multiple events over the last year. He uses Toyota Friend as an example of a social product. Here's a video of a joint Salesforce and Toyota press conference where Benioff talks about what this means (the meat starts at about 4:20)
Marc says "The car needs to become your friend. How does the car speak to me? How does the car integrate with the dealership and the factory? How does the car help me get to the service station? ... I want a relationship with the car in the same way I have a relationship with my friends. ... The car is going to become social. All our products are going to become social." (emphasis mine)
I have to admit when I first heard this, I wasn't sure I understood it. The demos that accompany the Toyota Friend pitches didn't help, because they mostly seemed like dashboards. Do I really want my car sending me tweets or writing on my wall? But as I've thought about it, I think Benioff's vision, while short on implementation details, is a brilliant articulation of a concept I've believed for a long time--what I'm calling the Live Web. Brilliant because it puts things in terms that almost anyone can understand.
The idea of social things (I'm just going to say "social things" from now on as a short hand for "social products and services") raises a few interesting questions:
- What does it mean for things to be social? I think it means more than being chatty.
- What does a Facebook of Things look like and how does it work? Facebook and other social networks were built for people--things need something different.
We understand intuitively what it means for animals to be social because we're social. Insects like bees, ants and termites create societies that are super organisms. Prarie dogs, monkeys, whales, apes, and humans all create societies. When we think of social animals we expect that they will do more than simply communicate with each other. A school of fish is not "social" in the same way that bees are--fish don't cooperate. We expect cooperation in social animals that complete tasks more complicated and sophisticated than any individual can do independently.
Societies are not hierarchical. They are networked. See my last post On Hierarchies and Networks for more on this. I happen to keep bees, so I know a little about how they work. While there is a "queen" and bees have different roles at different stages of life, the overall operation of the hive is remarkably decentralized, being controlled by pheremones that the queen and other bees emit. For example, when a bee detects a threat to the hive, it emits a pheremone that serves as an alarm to other bees, who respond individually--not under any kind of centralized command or control. The guard bee doesn't "command" other bees to go sting the intruder, rather it says "intruder" and other bees know what to do. The rules governing the proper response to an intruder are encapsulated inside each bee.
In a similar way, social things aren't just chatty, telling you about their status. Neither are they making requests or issuing commands. Rather, a social thing cooperates with other things you own on tasks that matter to you--your purposes, intents, and interests. They don't accomplish this because someone has set up an elaborate hierarchy of command and control, but because they are designed to cooperate. Of course, this is easier said than done. We are better at building hierarchical systems than enabling networked systems.
If you've been following my writing, you'll know I have specific ideas about how this can be accomplished--or at least how we can progress toward the ideal of loosely-coupled, cooperating networks of things. There are three key ideas:
- Data in motion is better than data at rest
- People (and their information) matter
- Networks trump hierarchies
I think events play a key role because they enable networked systems, whereas requests work best inside a tightly-coupled, command and control hierarchy. The only way to reduce coupling is to wean ourselves from our dependence on hierachical systems. Events also support semantic encapsulation--the idea that a product or service in the network keeps as much of its semantics to itself as possible. This is critical to reducing coupling between components.
Another important idea is that people are at the center of the interactions between the things they own and use. We can't build a coperating network of things if we place the person they are supposed to be serving on the sidelines. My things are different than your things (yeah, we share some) and they need to cooperate on my behalf. Being at the center of the interactions doesn't mean I'm coordinating them or somehow driving them. For this to work, I need to be involved only in the most critical interactions or when there are decisions only I can make.
If things are social, what is the "Facebook of Things?" What constitutes the network that they use to store their relationships and communicate? This Facebook of Things can't be Facebook. Facebook stores the relationships between people and has a limited set of verbs for how people interact: post, share, poke, message, etc. My car likely wants to do more than "share" with my GPS. My TripIt account doesn't need to merely "poke" my Expensify account.
A network that serves as the Facebook of things needs to support a larger set of events than these few verbs. Events are subject-verb-object triples. Because of the nature of what needs to be communicated, events need to be free-form and flexible. The systems that respond to them should support complex event scenarios to recognize situations more complicated than what a single thing can communicate with a single event. For example, "Phil's leaving on a trip and his car needs gas" is a combination of two events: "Phil's leaving on a trip" and "Phil's car needs gas." These two events mean something important to me when they're linked. Less so when they're not.
This event network must be universal--standards-based and open. I call it the "global event bus" or GEB. The GEB supports raising event and subscribing to specific kinds of events for specific entities. The GEB must also support discovery for event processors and generators. Everything I own and use will eventually tie into the GEB.
The addition of identity to the GEB allows each entity (usually a person) to overlay their own personal event network (PEN) on top of the GEB. A PEN comprises the collection of things that a person owns, personal data she controls, and the apps she uses to manage scenarios (think "take a trip," "find a service provider," etc.). All of these, on the basis of being associated with an identity, link products, services, data, and applications together in a social graph in service of an individual. My PEN is obviously different than yours--it is uniquely individual and personal. The PEN is the "society" or "ecosystem" in which my things interact and provides a new model for integration. Adding new things, new data, and new apps is easy because the entire network is loosely coupled and the architecture supports emergent interaction. (See my living room scenario in On Hierarchies and Networks for more detail on this.)
As I said, this is what I call the Live Web, but if "Facebook of Things" works better for you, go right ahead and think of it that way. I believe such a system is inevitable. While its exact nature and form may differ from what I envision, I don't think the overall idea will be long in coming. We can build this now and it will prove extremely valuable--so it will be built.
3:02 PM | Comments () | Recommend This | Print This
September 16, 2011
On Hierarchies and Networks
JP has been wond'ring aloud about sub-linear and super-linear complexity growth and how it might apply to systems. He's referencing a construct from this remarkable TED talk by Geoffrey West:
West is presenting the surprising math of cities and corporations. The talk is well worth 18 minutes of your life, but if you're impatient, the point that matters in the discussion to follow is that cities (empirically) get more of both the good and the bad things as they get larger. They are super-linear with a slope of about 1.15. Meaning that as a city gets bigger it will have more crime and more innovation than a strictly one-to-one linear slope would suggest. The bigger a city gets, the worse and better it gets.
Companies, on the other hand have economies of scale--and that can be a bad thing. They are sub-linear with a slope of 0.75. A company that gets twice as big doesn't have twice the costs--remember the slope of the line is 0.75, so it will have 1.5 times the cost. Unfortunately, it will also have 1.5 times the innovation, not twice. As a company gets bigger, you need more and more to achieve less and less. Ugh.
JP is wondering how this relates to IT systems.
As a result, soon after TED, I've been trying to formulate a similar argument about super-linearity and sub-linearity from a systems perspective: which costs scale up faster than the rate at which a system "grows", and which costs scale down faster? How do we measure system "growth"? Like West and his team found in the relationships between city populations and many other city measures, is there an equivalent for complex systems?
Most importantly, what is the role of collaboration in all this? How does all this tie up with the Jane Jacobs view of neighbourhoods and sidewalks and boulevards and parks? What does that mean for our understanding of stacks and ecosystems? Of course I have anchors and frames, I know what I want these answers to be, but I want to be able to prove them. Scientifically. Because the data in the research by people like Hagel, Seely Brown and West has some fascinating pointers.
From More Wond'ring Aloud -- confused of calcutta
Referenced Wed Sep 14 2011 16:46:18 GMT-0600 (MDT)
Meanwhile, Dion Hinchcliffe is responding to JP with some interesting ideas.
This fractal aspect of user systems, Web 2.0, and SOA is one that I deeply explored in the 2005-2007 timeframe, and my ideas on this even made the cover story of the SOA/Web Services Journal at one point. This is when we began to see successful, composite systems sprouting up "in the wild" that were eminently natural and highly successful, such as Amazon's, Flickr's, and many others. Innovative new open API-based businesses had definitively emerged and shown us --- in fact, virtually proved to anyone willing to observe --- that business/technology ecosystems could be routinely produced that were far more successful than the ones virtually any traditional business had managed to produce for itself internally with methods like SOA. It was clear something important and new was happening and we tried to learn from it.
...
The Web has also shown us that complexity, as important as it is to address and resolve the many inherent vagaries of dealing with the real world, is largely the enemy, whenever it exists needlessly. Our assumptions of where the complexity should be, in the transports for example, was wrong. It was in the ecosystem itself and we paid dearly for half a decade (and running) based on that misapprehension. But we've learned. SOAP usage has almost completely shifted to REST, complex WS-* stacks have largely moved on to simple standards and methods like Web-Oriented Architecture.
From Exceeding the Benefits of Complexity? A Fractal Model for the Social Business Era | On Web Strategy | Dion Hinchcliffe
Referenced Thu Sep 15 2011 15:02:08 GMT-0600 (MDT)
Dion is making the point that the Web 2.0 systems we see being built for consumer-side Internet services like Amazon and Flickr scale better and are less complex than systems built by IT shops behind the firewall. Dion goes on to reference John Hagel lamenting that "the cultural gap between the enterprise crowd and the Internet crowd is still much too large and is still hindering the move forward for many organizations." Dion, rightly, sees the difference being technology choice: Web services like SOAP and the WS-* specs proved to be too heavyweight to use. Here's a diagram he uses to illustrate this:
So, why am I telling you all this? Like JP and Dion, I think there's more than a mere metaphor between the city/corporation insights and the ways we build software today. I'd argue that social systems, including cities are models for the techniques and technologies we ought to be using. Networked systems scale better and are more flexible than hierarchical systems. They can absorb complexity better without suffering from the debilitating effects of tight coupling that hierarchical systems create. But I'm ready to go further than I think JP and Dion are.
While API-based systems are more flexible than the moribund systems that enterprise IT shops create, they don't go far enough. They are still request-response based. Request-response based systems are hierarchical and create unnecessary coupling. Cities and other social systems do not operate solely using a request-response model.
The dual of request-response is events. Event-driven systems exhibit more of the network architecture that make cities flexible and robust. Let me use the following scenario to illustrate.
Suppose you want a system set up in your living room so that when you get a phone call, your Tivo automatically pauses what you're watching.
![[]](http://farm7.static.flickr.com/6082/6153160836_78c96e6b14.jpg "[]")
In a request-response system the Tivo would have some kind of API and I'd configure my phone--probably by installing an app and pairing it with the Tivo--so that when a phone call came in, it would send the "pause" command to the Tivo. Not so bad. All we need is an API on the Tivo.
But suppose I want more. I want my stereo to mute the sound when I get a phone call. Another API and another app. But I have a DVR from Comcast in the bedroom and I want that to pause too. Another API and another app. I switch out receivers--better find the right app. Buy a new Android phone and you've got to do it all over again. What about the house phone? It doesn't have apps. Sheesh!
Do you see what's going on? I'm creating a hierarchy with my phone at the center. Installing and configuring an app isn't so bad for the first, but do you really want to install and configure apps for every thing (device, appliance, car, and so on) you own? It doesn't scale and the complexity goes through the roof.
Now, imagine an evented system. The phone merely raises and event to an event bus that says "Phil's getting a phone call." The Tivo listens for events and knows that it pauses (that's the natural behavior for a Tivo when the viewer gets distracted by something else) when there's an inbound call. The stereo knows to pause. Lights know to come on. And so on. The phone doesn't know these other systems are there or what their actions will be. The phone doesn't have to understand their APIs, what commands they understand, or be configured when they are added. Any configuration happens on the device itself. If a new DVR shows up, everything else can ignore it. If the stereo component goes away, everything else goes on about their business.
Here's the magic. In a request-response system the requester is obligated to understand the semantics of the APIs it deals with. In event-driven system, the responsibility for semantics falls on the event processor--the receiver. This is called semantic encapsulation. Turning semantic responsibility around leads to looser coupling.
When you think about it, that's how cities work. When a new person moves in the neighborhood, all the neighbors don't have to be configured to interact with them. If you want to start a new business, there are some things you need to know and some conventions you'd better follow, but you do it in relative independence. There is loose coupling between the actors in the system and the semantic responsibilities are encapsulated in the actors--just like our DVR in the evented living room.
Of course, event-driven systems aren't a panacea. Events can be messy and figuring out the flow of control is tricky because there's no central command-and-control system like there is in a hierarchical system. Event-driven systems can be hard to debug when something unexpected happens. Nevertheless, I think there is an architectural imperative in force that will lead to more and more systems being event-driven. We're already starting to see the importance of events in the flows of information that social tools create. Notifications, activity streams, and the like are becoming more important and the natural way to deal with them isn't request-response systems or, in most cases, long-lived sockets, but events.
Events create a networked pattern of interaction with decentralized decision making. Because new players can enter the event system without others having to give permission, be reconfigured, or be reprogrammed, they grow organically. This is how the Internet was meant to be. There's no reason why we have to create hierarchical systems on top of the Internet. Our current interaction patterns on the Web are a legacy of client-server computing--itself a legacy of the corporate structure of the industrial age. Hierarchical interaction isn't the ultimate expression of the Internet, but merely a stepping stone to more powerful systems of the future. I'm convinced they'll be event-based.
I'm writing a book about this called The Live Web. If you'd like advanced access to the PDF, let me know and I'll invite you to the Dropbox.
9:17 AM | Comments () | Recommend This | Print This
September 13, 2011
Schmidt at Dreamforce
I went to Dreamforce a few weeks ago. What a kick. The conference was so big and very fun. It reminded me of conferences from the 1990s. I couldn't believe how big the expo floor was. We had to leave before the closing keynote with Marc Benioff talking to Eric Schmidt. I made sure I watched the You Tube video.
The whole talk is work listening to, but it's over an hour. I was especially taken by a few minutes in the middle (at about minute 25). I showed this clip in my talk to the SCITDA conference. I took the liberty of clipping that section out so I could focus on it:
In this clip, Schmidt talks about the death of client-server computing and the business mode behind it. He talks about the power of the cloud--having the data all referencable from a single place--and how that empowers predictive computing and a new real-time "alerting" model that will transform computing. He says: "You think it's real-time now, it's really going to be real-time."
When Schmidt says "alerts," I hear "events." The transformation that Schmidt speaks of is the Live Web.
8:30 AM | Comments () | Recommend This | Print This
September 12, 2011
SCITDA Fall Conference: The Live Web
I spoke today at the South Carolina IT Director's Association fall conference. The topic was "The Live Web." The subject of my upcoming book. Here's the summary:
The web is moving from the Dynamic Query/Static Data model that has characterized Web 2.0 sites to a Dynamic Data/Static Query model that characterizes many of today's most interesting Internet interactions. What does this mean for your organization and how can you take advantage of this shift?
And here's the slides:
The conference was fun and I'm glad David Elwer invited me.
7:06 PM | Comments () | Recommend This | Print This
September 6, 2011
Tablets
I bought an iPad when they first came out. Lately I picked up a Samsung Galaxy Tab to experience the Android side of things. Tablet computers have changed important parts of my life.
The iPad wasn't my first "tablet." I had a Compaq TC1100 for a while and used it when computing in places that a regular PC didn't work well. The TC1100 was a pen computer and used Windows XP. I bought it, rather than other tablet computers of the error era because the keyboard completely detached and left a slate. That's the mode I used it in 99% of the time. Ultimately, though it was underpowered and I found myself just using my laptop more and more.
The iPad is different. Part of it is design, power, and size. But mostly, I think that the world has changed in some important ways.
For example, I read a lot. Since I got the iPad and the Kindle app, I've almost exclusively cut over to ebooks. The only exception is when a Kindle version isn't available. The fact that the Kindle app is available for every device I own and so my books are with me anywhere I go is a huge part of the appeal. When I owned the TC1100, there was no Kindle app and I was an ebook skeptic. Also, the form factor of the iPad is nicer than most books, I can increase the font size, and the screen is lit. The battery life on the iPad is such that I almost never worry about not having power. The only downside is having to put my book away when the plane is landing (stupid rule, but that's a different blog post).
The Galaxy Tab is a great device. In fact, I wanted to dislike it, but find that I use it more and more. I haven't had it long enough to comment on things like battery life vis 'a vis the iPad, but the platform itself it well made and Android works well--after you get used to it. I have found a few apps that aren't available on it that I like on the iPad (like Zite), but there is a vibrant app community with a lot of choice. What's more, there's a Kindle app, so it has my books.
I recently went on a trip (personal, not business) and just took the iPad. I didn't really need anything else since it had email, a browser, my calendar, and my books. I didn't miss my computer. I don't think I'll travel for business with just an iPad yet, mainly because of presentation and demos. Also, for working all day writing or coding I love a big screen and keyboard.
A few years ago, my Mom and Dad bought a computer. They didn't take to it--although my Mom likes to play solitaire on it. But I can see my Mom using a iPad with a 3G connection for email, reading, and doing Facebook. She would be more comfortable with the form factor, sitting in her chair and using it like a book or tablet.
That brings me to the chief disadvantage. These things aren't cheap. The Galaxy Tab with 32Gb of RAM cost more than what my parents paid for their inexpensive PCs at Wal-Mart.
Still, I can imagine that in a few years, my family will have a bunch of tablets just spread around as general computing devices throughout the house. As we become more and more attached to the 'Net, we're more likely to need it everywhere. Count me as a believer in the tablet revolution.