« September 23, 2004 | Main | September 28, 2004 »

September 27, 2004

Technorati Outages and the Importance of Product Engineers

Dave Sifry gives some details about the Technorati outage this past weekend. Seems an electrical fire in the data center their co-lo at was the culprit. Running a 24/7 Web application reliably isn't easy and it isn't cheap. It took us several years of problems and study to hit on a solution at iMALL. We finally did figure it out and that was a real lightening of my load. One of the answers is product engineers, an engineer on the operations side whose job it is to make the product (not just the server) work. Properly incented, a product engineer will drive all of the emergency and contingency planning, along with ensuring that engineering delivers a system that can be reliably operated.

01:01 PM | Recommend This | Print This

CTO Breakfast Report: Local Exchanges

The topic for Friday's CTO Breakfast was local exchanges. We had a great discussion and several new people Hope to see you there next month. Don't know what an exchange is or why its important? Keep reading.

In the days of railroad, dead end rail lines, called spurs, were built to connect small communities to the main rail line. This allowed farmers to ship their wares to market and small businesses to more easily get supplies. If you were a big business with lots of shipping needs, you didn't locate at the end of a spur, you located in a big city with a railroad switching yard and lines going out to multiple locations.

This situation with the Internet today is similar. There are major switching locations, called exchange points, and there are spurs. Much as we like to refer to the Internet as a "network" it more closely resembles a group of connected trees with lots of dead end branches.

This has worked pretty well, so far, but let's face it, the current network was built for dial up. The main problem with a group of connected trees is not bandwidth--you can always make the branches bigger--its latency. With dial-up, latency isn't really an issue. The latency issues can be ignored because more low-bandwidth applications aren't latency sensitive. Broadband changes that equation however. When I'm viewing live streams or playing an interactive game, latency can ruin the entire experience.

An exchange is a place where networks meet. Some of these meeting places are private, that is they are just an arrangement between two or three players, but some exchanges are more open and neutral. Exchange points are a central physical location where Internet traffic is handed off from the source carrier to the destination carrier. The are primarily used for peering, the free exchange of traffic between Internet backbones.

So why does this matter? If you live in a place without an exchange point, all of the Internet traffic that you generate and consume likely goes through some large city that does have an exchange point. The result is high latency as every packet has to go to this junction first before starting the journey to where its really headed. People who live in places like San Francisco or Washington DC, don't see this, but everyone else does. In Salt Lake, packets go to Denver, LA, or Chicago usually. If that's where I'm headed, no problem. When I'm trying to get to something in Utah however, that can be a problem.

Keeping local traffic local is good for the consumer, but is it good for the carriers? Well, yes and no. The good news for a carrier is that they don't pay money to backhaul traffic out of Utah and then back in. But unless that traffic is significant, the downside of exchange--leveling the playing field between competitors--is too great.

In the past, Utah's population hasn't been large enough for major carriers to want to play ball since the cost of backhauling what traffic there is remains small. The rise of fiber to the home broadband projects in Utah such as Utopia and iProvo, however, could change things in several ways:

  • First, more broadband makes Utah's 2 million users look like 20 million.
  • Second, there are a few large consumers who can create a market if they band together.

With that in mind, I'm going to be putting together a meeting soon to make a game plan for how we can create a marketplace and invite the carriers to participate. We've two of the four big players in Utah on board to give this a go and hope to convince the others soon. It will likely be an open meeting, so if you've got an interest, let me know and I'll put your name on the list.

10:05 AM | Recommend This | Print This

eVoting Round-Up

Here's a round-up of some eVoting articles from the last few days:

  • First, the Deseret News has an article about the press conference on eVoting held at the Univ. of Utah last week. Its short, but gets the main point across: "paperless voting are insecure."
  • An opinion piece from the Christian Science Monitor discusses the first use of electronic voting machines with a paper trail in Nevada's primary this month.
    Perhaps inadvertently, Nevada's success will help squash much of the controversy over the reliability of electronic voting machines. Though states are moving toward e-voting - approximately one-third of voters are expected to use such digital machines on Nov. 2 - many types of them still aren't error-free.

    Too many show vulnerability to hacking or software glitches. A paper trail remains the best interim step as states make their glacial efforts to comply with Congress's 2002 Help America Vote Act
    From E-Voting, With a Paper Trail | csmonitor.com
    Referenced Mon Sep 27 2004 08:57:23 GMT-0600
  • And, if you just want a little humor, Sundays' Non Sequitur cartoon pokes fun at electronic voting.

I particularly hope that Utah's Voting Equipment Selection Committee pays attention to the Nevada results. They have the flexibility to choose a system with a paper audit trail. There's no good reason to avoid paper--and plenty of reasons not to take any chances.

09:29 AM | Recommend This | Print This