I came, I coded, I debugged.

At QCon 2014, LinkedIn gave a talk ( slides ) about their Gobblin projec t (as well as published a  blog post ). Gobblin is a unified data ingestion system, aimed at providing  Camus -like capabilities for sources other than Kafka. While Gobblin is a fascinating piece of engineering, what I find to be no less fascinating is the direction LinkedIn has chosen by going for a system like Gobbblin.

There are a few posts talking about Storm's metrics mechanism, among which you can find  Michael Noll's post ,  Jason Trost's post  and the  storm-metrics-statsd  github project, and last but not least (or is it?)   Storm's documentation . While all of the above provide a decent amount of information, and one is definitely encouraged to read them all before proceeding, it feels like in order to get the full picture one needs to combine them all, and even then a few bits and pieces are left missing. It is these missing bits I'll be rambling about in this post.