Gonna play with RRDs soon.
February 4th, 2009
It just occurred to me that RRDs might be really well-suited to some general purpose roles as counters, statistics trackers, etc. for use in webapps. There are statistical analysis tools built in, so I could potentially see it as a way to (locally) rate limit API requests, track certain types of errors, etc. It’s also very fast, writes back to disk, and already plays very nicely with a variety of graphing and monitoring tools.
I’ll do some experimentation soon and report back.
That is exactly what we have been doing recently. We update counters in memcache and have a sweeper process that sends the data into ganglia. I wrote a small web service that dumps out the info as sparkline and full graphs. Also have one that makes the data available as a block of json.
Jason: Cool! After I posted this, Leonard sent me an old link to Flickr’s RRD-over-UDP post here: http://code.flickr.com/blog/2008/10/27/counting-timing/