Skip to main content

Posts

Showing posts from February, 2014

IntelliJ - You auto complete me (2)

A short one.

I've already posted about IntelliJ in the past, and judging by my enthusiastic PR for IntelliJ IDEA someone might mistake me for a share holder. Well, I'm not, though come to think of it, it might actually be a good IDEA (pardon the pun). I just love working with IDEs that understand developers and make things easier on them, and IntelliJ does just that. I've recently been brushing up my (automatic) refactoring skills, and IntelliJ has some really awesome stuff to offer in that department. Truth be told, there are some issues here and there, but hey, I have my bad days too.

All in all, for me it's done the unbelievable job of making programming in Java ... kinda fun, not a simple task by any means. I even managed to enjoy editing a bash script the other day, I mean, that's unprecedented! All that before I even had a chance to try out the Ultimate edition, as I'm working with their (free!) Community version.

Speaking of community, check out this Obs…

Finding a needle in a Storm-stack

(Crossposting from Outbrain's techblog)

Using Storm for real time distributed computations has become a widely adopted approach, and today one can easily find more than a few posts on Storm's architecture, internals, and what have you (e.g., Storm wiki, Understanding the parallelism of a storm topology, Understanding storm internal message buffers, etc).

So you read all these posts and and got yourself a running Storm cluster. You even wrote a topology that does something you need, and managed to get it deployed. "How cool is this?", you think to yourself. "Extremely cool", you reply to yourself sipping the morning coffee. The next step would probably be writing some sort of a validation procedure, to make sure your distributed Storm computation does what you think it does, and does it well. Here at Outbrain we have these validation processes running hourly, making sure our realtime layer data is consistent with our batch layer data - which we consider to b…