This is probably the most difficult blog post I’ve had to write. What’s worse is I’ve been sitting on it for months. When I started Noah a few years ago, I had a head full of steam. I had some grand ideas but was trying to keep things realistic. I simply wanted a simple REST-ish interface for stashing nuggets of information between systems and a flexible way to notify interested parties when that information changed.

People may or may not have noticed but I’ve been largely offline for the past 4 weeks or so. This is because I’ve been in the middle of a pretty heavy redesign of a few key parts of our application stack. This also required me to learn Java so I’ve been doubly slammed. As part of that redesign, I worked on what we lovingly refer to internally as the “solo installer”.

This post is something that’s been brewing for a while. While it may sound targeted in tone, it’s more general than that. Let’s just call it an open letter to family, friends and coworkers around the world. One thing that I have the hardest time communicating to friends and family who aren’t in the IT industry is the concept of “production” and what it means to be on-call. Even coworkers have a hard time understanding what it means.

Since I just tweeted about this and I know people would want an explaination, I figured I’d short circuit 140 character hell and explain why I think EBS was the worst thing Amazon ever did to AWS. First time I’ve had to do this but: the following is my personal opinion and in no way reflects any policy or position of my employer A journey through time I remember when EC2 was first unleashed.

So it’s come to my attention that today is the “anniversary” of when I wrote my first “#monitoringsucks” blog post.

It always starts with a tweet. However it normally doesn’t start with a tweet from Cliff Moon. {% blockquote %} Of all the problems to fix in chef or puppet, the diffusion and drift of state that occurs in idiomatic usage seems highest priority. {% endblockquote %} Now for sure what spawned this comment was something unrelated but it got me thinking. Oddly enough Tim Dysinger was either poking around in my head or just had the same idea:

So it looks like Adrian Cockcroft finally spilled the beans on Netflix (no)Operations and sadly it reads like I expected. Netflix still does operations Regardless of what words Adrian uses, Netflix still does operations. John Allspaw summed it up pretty well in this tweet: and here are the things, he mentions: Metrics collection PaaS/IaaS evaluation/investigation Automation (auto-build, auto-recovery) Fault tolerance Availability Monitoring Performance Capex and Opex forecasting Outage response So what does Adrian get wrong?

Recently I saw a tweet from Kohsuke Kawaguchi that really got me frustrated. I’ve addressed this topic a bit before here. At the time it was addressing specifically dynamic languages. However the post that Kohsuke wrote (and the post that inspired it) have led me to a new line attitude. Don’t bother trying to get your packages into upstream vendor distros Wait. What? Let’s step back a sec Let me clarify something first.

So anyone who knows me knows I spend an inordinate amount of time bitching about Maven. I don’t know if it’s the type of companies I end up working for or what but I always seem to find myself ass-deep in Maven. please note that I’m drifiting into deeply unfamiliar territory for me. Someone once told me the best way to learn about something is to write about it. Keep that in mind when making comments?

A few days ago I wrote up some notes on how we’re making Logstash better by adding ZeroMQ as an option for inputs and outputs. That night we decided to take it a bit further and add support for ZeroMQ as a filter plugin as well. I’ve had a lot of people ask me what’s so hot about ZeroMQ. It’s hard to explain but I really would suggest you read the excellent zguide.