Posts with keyword: operations


Engineers Do for $1.00 What Any Fool Can Do for $2.00

Guest post by Wade Billings, VP Operations, Kynetx When we started Kynetx, we had a very limited IT budget due to fact that the company was bootstrapped with the proceeds of selling the founders Piper Turbo Arrow airplane. From the beginning, Kynetx was built upon the conviction to be initially self funded and very strong beliefs regarding system performance and stability, so from day one, we lived inside a paradox: how do you maintain a high level of reliability, accessibility and stability when you have little in the way of resources or capital to expend. Had we started Kynetx
Continue reading {Engineers Do for $1.00 What Any Fool Can Do for $2.00}


Continuous Deployment

This morning Steve gave a presentation on context automation and Kynetx at the Utah Technology Council's CTO P2P forum. The presentation was great and the audience asked a lot of good questions. One thing that came up (I don't even remember why) was the subject of continuous deployment. I decided I'd pull a few URLs out of my head and put them in a blog post for people to mull over. The first URL I think of when I consider continuous deployment is code.flickr.com. If you've never been there, the bottom of the page lists when the last deployment
Continue reading {Continuous Deployment}


The Kynetx Move to CloudFront

Image by chascar via Flickr One of the components of Kynetx Network Service (KNS) is a 30K (compressed) static Javascript library. This is mostly a slightly modified jQuery along with some other components. We set the Expires header so that it is cached in the browser for 24 hours. Even still, it's a significant load on our network bandwidth and, consequently, our budget. When Amazon's CloudFront (CF) was announced, we realized that we could move these kinds of static files to CF as a way to reduce our bandwidth and maybe get a performance improvement. If you aren't familiar
Continue reading {The Kynetx Move to CloudFront}


Infrastructure Automation Increases Operational Efficiency

Last week Jeff Atwood wrote a piece about why throwing hardware at a problem makes sense in some situations. He's careful to avoid the trap of thinking that throwing hardware at a poorly designed program will do more than buy you a little time, but makes a good point: Clearly, hardware is cheap, and programmers are expensive. Whenever you're provided an opportunity to leverage that imbalance, it would be incredibly foolish not to. From Coding Horror: Hardware is Cheap, Programmers are ExpensiveReferenced Sat Dec 27 2008 15:42:36 GMT-0700 (MST) Riffing on that, Gordon Weakliem makes a related and somewhat
Continue reading {Infrastructure Automation Increases Operational Efficiency}


Kynetx Operational Procedures

In the spirit of transparency, Kynetx has published our operational procedures and practices online. As we gain more control over our infrastructure automation, we'll also have real-time operational information and statistics. I'd appreciate feedback on what's there. Keep in mind, this isn't out "dream" but rather we're trying to present a true picture of where we're at. We obviously have goals to make this better and have quite a bit of experience in mature operations so we know where we need to be and where we currently fall short. I don't believe that there's anything there that would compromise
Continue reading {Kynetx Operational Procedures}


Velocity 08: Actionable Logs

Mandi Walls from AOL is talking about creating actionable logs. Actionable logs are logs that provide data that can be used to fix problems. There are a few rules to start with: No nonsense logging Concise, easy to understand Express symptoms of productions issues Any that makes the log needs to be somethings that can be fixed (better signal to noise ratio) Everytime you write to a log file, you're expending resources. The point of logging in production is diagnosing issues. You need to be able to understand the logs at 4am in the morning. The primary goal is
Continue reading {Velocity 08: Actionable Logs}


Velocity Keynote: IT Operations are Unsustainable

Bill Coleman (the "B" in BEA) is giving the opening keynote titled Green Data Centers, but it's really about sustainable operations. He begins by saying that the current way we operate data centers is unsustainable. Operations costs are growing at twice the rate of IT in general. This is the unintended consequence of the success of networked operations. Scale and complexity have grown dramatically. Bill claims 5 orders of magnitude. In some organizations it can take 6 months to get a new server into the data center. If you do it faster than that, good for you. Virtualization is
Continue reading {Velocity Keynote: IT Operations are Unsustainable}


(Not) Scaling MySpace

Larry Dignan has a five-point analysis of MySpace's IT infrastructure and it's scaling issues over at Between the Lines. Very interesting. I know a little about MySpace before they were bought and this scaling issue doesn't surprise me. Read the Baseline article for the details. Bonus: I posted a piece on Tivo's walled garden and Tivoserver at BTL this morning.
Continue reading {(Not) Scaling MySpace}