StatsD and You

The easy way to get application level metrics

http://cleverdomain.org/ppw-statsd/

Who am I?

Who are you?

What’s this about?

More sources here and here

StatsD is a metrics aggregation system originated by Flickr and implemented by StatsD a few years ago

What can I do with StatsD?

realtime and historical views both useful when debugging a problem

What can I do with StatsD?

This is a graph of how long some service call takes — point out percentiles

What can I do with StatsD?

This is a graph of the number of hits to some service — see how busy it is and how the load breaks down among different sources

StatsD features

Installing Statsd

npm install -g statsd
statsd exampleConfig.js

Installing Graphite

Installing Graphite

Docker

docker build git://github.com/dotcloud/collectd-graphite

or

docker build git://github.com/silas/docker-graphite

How does it integrate with my app?

Clients for every language means all your apps can send data the same place even if you use lots of languages. Net::Statsd::Client is my module created with help from Shutterstock.

Net::Statsd vanilla

use Net::Statsd;
$Net::Statsd::HOST = "stats.example.com";

sub login {
  # ...
  Net::Statsd::increment("myapp.logins");
}
Global variables are meh :(

Net::Statsd::Client

use Net::Statsd::Client;
my $stats = Net::Statsd::Client->new(
  server => "stats.example.com",
  prefix => "myapp.",
);

sub login {
  # ...
  $stats->increment("logins");
}
Prefix for DRY, namespacing, possiblity of loading config from Dancer config or similar

Net::Statsd timing

use Net::Statsd;
$Net::Statsd::Server = "stats.example.com";
use Time::HiRes qw(tv_interval gettimeofday);

sub get_something {
  my $start = [gettimeofday];
  my $client = Service::Client->new;
  my $data = $client->get("some stuff");
  my $elapsed = tv_interval($start);
  Net::Statsd::timing("myapp.get_something.elapsed", 
      int($elapsed * 1000)); # Seconds to ms
  return $data;
}
Lots of repeated boilerplate around timing. What’s the right multiplication factor? Statsd uses ms.

Net::Statsd::Client timing

use Net::Statsd::Client;
my $stats = Net::Statsd::Client->new(
  server => "stats.example.com",
  prefix => "myapp.",
);

sub get_something {
  my $timer = $stats->timer("get_something.elapsed");
  my $client = Service::Client->new;
  my $data = $client->get("some stuff");
  $timer->finish;
  return $data;
}
Timer object abstracts away timing, warns if →finish or →cancel not called because it indicates an uncovered piece of code

Sampling

Do you have that many events? Okay.

use Net::Statsd::Client;
my $stats = Net::Statsd::Client->new(
  server => "stats.example.com",
  prefix => "myapp.",
  sample_rate => 0.01, # Send 1%
);

Override per-event if you like.

$stats->increment("lines_of_code_executed", 0.0001);
For counters, statsd automatically scales with sample rate so everything is consistent

Planned Features

Graphite Tricks

Graphite Tricks

Demo

Thanks!