UDP Communication between containers

Is there a way to allow UDP communication between containers or any workaround for it? We need to emit metrics to our agent via StatsD

The AWS LoadBalancers that back Endpoints only support TCP so services running on Aptible cannot receive data via UDP. Your options are:

  • Host the agent in your own AWS VPC and set up VPC peering with your Stacks so that they can send the metrics to the agent over the private network. I believe this is what most users have done if they decide to use the StatsD agent.
  • Run the agent in each container you want to collect metrics for and configure your services to send the data to the agent running in the same container. Connecting to services running within the same container doesn’t require an Endpoint so they can use UDP.
  • Create a custom service that runs the agent and accepts the metrics over TCP before forwarding them to the agent over UDP. There was a user that attempted to set up a service that accepted metrics InfluxDB format, likely using telegraf, then forward them on to the agent but I don’t know if they were successful or not.

I’m not sure if this is the same but the Datadog application performance monitor (APM) agent supports tracing application metrics so if you’re already using the APM in your containers, or would like to, this may be a good option.

Following this - would be interested on any guidance for setting up the datadog APM agent in existing container - currently trying to get this working alongside the dd-trace-rb ruby gem from datadog

Hi @abramf , Thanks for asking the question here and I apologize for not getting to it sooner. I’m going to share @MichaelW 's answer here as well:

The gem is trying to open a TCP connection using the default port and host IP route. You will need to create a TCP Endpoint and then set the tracer hostname to that of the Endpoint.

We cannot provide very much help for 3rd-party services like DataDog, but we’ve seen the DataDog agent be successfully implemented as follows:

  • Run the Datadog Agent as it’s own application, using the prebuilt image from Docker Hub : aptible deploy --app datadog-agent --docker-image=datadog/agent

  • Add three important pieces of configuration to the agent App : aptible config:set --app datadog-agent DD_APM_ENABLED=true DD_APM_NON_LOCAL_TRAFFIC=true DD_API_KEY=your_key_here (see the Datadog documentation for these and other settings)

  • Create an internal TCP Endpoint to allow your instrumented applications to publish APM information to the DD Agent: aptible endpoints:tcp:create --app datadog-agent cmd --default-domain --internal

  • Instrument your Ruby App as documented here: https://docs.datadoghq.com/tracing/setup_overview/setup/ruby/

  • Set the tracer option hostname to that of the above Endpoint : https://docs.datadoghq.com/tracing/setup_overview/setup/ruby/

Then, on your Apps that send data to the agent, you only have to set a couple of environment variables

DD_AGENT_HOST=app-NNNNN.on-aptible.com (the endpoint URL from the aptible endpoints:tcp:create command above)

Hope this information helps!