Aptible + DataDog DogStatsD

#1

I am in the process of getting DataDog set up to work with our application on Aptible, and so far I have the push-button metrics integration working and I also have APM working by posting traces to a separate Aptible app that’s running the datadog agent image as described here. So far so good.

Now I’m trying to get DogStatsD set up for custom metrics but no luck so far, and I’m wondering if this is even possible? It works over UDP which doesn’t seem to be supported. There is also a Unix Domain Socket option but that would require getting the DataDog agent running in a container on the same host as our application somehow which I also fear is not achievable.

Has anyone figured out a way to use DogStatsD in Aptible? Any advice would be much appreciated. Thanks!

#2

that would require getting the DataDog agent running in a container on the same host as our application

I haven’t tried this (yet), but it has been suggested to me that it might be worth a shot to install the dd agent alongside your application in the same container. While this does violate the separation of concerns principles, it might be worth it in that one might be able to get a lot of the host information (and possibly the custom stats you want?) that are otherwise unavailable via the separate dd agent container set-up. If you do try it, please keep us posted!

#3

thanks Jessica! I’ll let you know if we figure that out.

Really appreciate your earlier posts too, they were incredibly helpful in getting APM up and running.

#4

Hey @lmoore , we had to do this and for us the easiest was to get a DataDog agent running on a micro EC2 instance, and send the UDP traffic over to the instance using VPC peering.

This is the config that the agent would need:

api_key: $your_dd_api_key
use_dogstatsd: true
bind_host: localhost
dogstatsd_non_local_traffic: true

Then point your configs to the private IP of the EC2 instance assuming peering is working, and the instance security group allows traffic. If you want an HA seup, just put an NLB in front (which supports UDP load balancing), packerize the AMI and use an 1-1-1 (min-max-desired) ASG so the app recovers from failures.

If Aptible ever supports NLB endpoints then we could run it on a container on Aptible but meanwhile this is probably the best if you want to avoid colocating DataDog agents with your app containers.

Hope it helps
Alberto.

#5

thanks for sharing, i really appreciate it.