Configure Datadog

How to send Optimizely Feature Experimentation decisions and conversion events to Datadog using notification listeners.

📘
Note
This is a third-party integration and is not an Optimizely subprocessor. See Optimizely's Third-Party Add-Ons & Platform Integration Terms.

Datadog is a cloud-based platform for monitoring and analyzing applications and infrastructure. Using notification listeners, you can send Feature Experimentation decision and conversion events to Datadog to track how feature flags affect system health. This integration uses the Optimizely Feature Experimentation decision notification listener, which triggers when Optimizely makes a decision for a feature flag.

📘
Note
Notification listeners trigger a callback function that you define when specific actions are activated in the SDK.

Use cases

Integrating Optimizely Feature Experimentation with Datadog lets you correlate feature flag decisions with system health metrics. This helps track the impact of experiments on performance, stability, and user experience. The following are some key use cases:

Detect performance regressions – When rolling out a feature behind a flag, you can monitor system metrics such as CPU usage, memory consumption, and request latency. If a flagged variation introduces performance issues, you can roll it back before it affects a wider audience.
Correlate experiment results with system metrics – Datadog lets you overlay Optimizely decision events with system-level metrics to identify trends, such as whether users in a particular variation experience errors, database slowdowns, or increased cache misses.
Monitor experiment impact on infrastructure costs – Different feature versions can have varying impacts on infrastructure usage, leading to differences in cloud costs. Datadog's Infrastructure Monitoring compares these differences to give insights into how each version affects resource consumption and costs.
Configure proactive alerts – Use Datadog's Actional Alerting to create real-time alerts based on Optimizely's experiment data. For example, you can configure an alert is a specific variation triggers more than n number of errors within a time period.

Example code

The following code samples demonstrate how to send metrics from Optimizely Feature Experimentation to Datadog using the Python SDK and JavaScript (Node) SDK. You can create the same logic using Feature Experimentation's Additional SDKs. At a high level, the sample code completes the following:

Import and configure Optimizely client – Imports necessary modules and classes and initializes the Optimizely client using the provided SDK key.
Datadog client initialization– Configures Datadog to send metrics to localhost:8125.
Define metric types – Uses a MetricType enumeration to distinguish between COUNT and GAUGE metrics.
Convert tags – Implements a convert_tags function to format tags as key:value strings.
Send metrics to Datadog – The send_metrics_to_datadog function completes the following:
- Converts tags to the correct format.
- Constructs a metric name prefixed with optimizely..
- Sends metrics as COUNT (increments a value) or GAUGE (sets a value). Raises an error if you do not provide a value.
Configure decision notification listener – The on_decision function completes the following:
- Listens for flag decision events.
- Extracts relevant decision details (such as user_id, flag_key, and variation_key).
- Sends a metric to Datadog.
Register the listener with Optimizely – Attaches on_decision to Optimizely's notification center.
Trigger a test decision – Simulates a decision event to verify that the code sends metrics to Datadog. To verify, use Datadog's Metric Explorer to check for events with optimizely.* prefix.

Python SDK

from typing import Optional
from optimizely.helpers.enums import NotificationTypes
from optimizely.optimizely import Optimizely
from datadog import statsd, initialize
from enum import Enum

OPTIMIZELY_SDK_KEY = "<sdk_key>"

# Setup the Optimizely client
optimizely_client = Optimizely(sdk_key=OPTIMIZELY_SDK_KEY)

# Initialize datadog client
options = {
    "statsd_host": "localhost",
    "statsd_port": 8125,
}

initialize(**options)


class MetricType(Enum):
    COUNT = 'count'
    GAUGE = 'gauge'


def convert_tags(tags: dict[str, str]) -> list[str]:
    return [f"{k}:{v}" for (k, v) in tags.items()]


#######################################
# SEND METRICS TO DATADOG
#######################################
def send_metrics_to_datadog(
    metric_name: str,
    metric_type: MetricType,
    value: Optional[float] = None,
    tags: dict[str, str] = None,
):
    updated_tags = convert_tags(tags) if tags else []
    metric = f"optimizely.{metric_name}"

    print(tags)

    # Check the metric type and send datadog metrics
    if metric_type == MetricType.COUNT:
        statsd.increment(
            metric,
            1 if value is None else value,
            tags=updated_tags,
        )
    elif metric_type == MetricType.GAUGE:
        if value is None:
            raise ValueError("datadog gauge metrics require a value")
        statsd.gauge(metric, value, updated_tags)


#######################################
# SET UP DECISION NOTIFICATION LISTENER
#######################################
def on_decision(decision_type, user_id, attributes, decision_info):
    # Add a DECISION Notification Listener for type FLAG
    if decision_type != "flag":
        return
    flag_key = decision_info.get("flag_key")

    # Add some tags
    tags = {
        "user_id": user_id,
        "enabled": decision_info.get("enabled"),
        "flag_key": flag_key,
        "rule_key": decision_info.get("rule_key"),
        "variation_key": decision_info.get("variation_key"),
        **attributes,
    }

    # Send the metrics to datadog
    send_metrics_to_datadog(flag_key, MetricType.COUNT, 1, tags)


optimizely_client.notification_center.add_notification_listener(
    NotificationTypes.DECISION, on_decision
)


# Trigger the notification listener
decision_type = "flag"
user_id = "<user_id>"
attributes = {"attribute_key": "attribute_key"}

user = optimizely_client.create_user_context(user_id, attributes)
decision = user.decide("flag_key")

JavaScript (Node) SDK

const { StatsD } = require("hot-shots");
const { enums, createInstance } = require("@optimizely/optimizely-sdk");

const OPTIMIZELY_SDK_KEY = "<sdk_key>";

// Initialize Optimizely client
const optimizely = createInstance({
  sdkKey: `${OPTIMIZELY_SDK_KEY}`,
});

// Initialize hot-shots StatsD client
const statsd = new StatsD({
  host: "localhost",
  port: 8125,
});

const MetricType = {
  COUNT: "count",
  GAUGE: "gauge",
};

function convertTags(tags) {
  return Object.keys(tags).map((key) => `${key}:${tags[key]}`);
}

function sendMetricsToDatadog(metricName, metricType, value, tags) {
  const updatedTags = tags ? convertTags(tags) : [];
  const metric = `optimizely.${metricName}`;

  if (metricType === MetricType.COUNT) {
    statsd.increment(metric, value !== undefined ? value : 1, updatedTags);
  } else if (metricType === MetricType.GAUGE) {
    if (value === undefined) {
      throw new Error("datadog gauge metrics require a value");
    }
    statsd.gauge(metric, value, updatedTags);
  }
}

function onDecision({ type, userId, attributes, decisionInfo }) {
  // Add a DECISION Notification Listener for type FLAG
  if (type !== "flag") {
    return;
  }

  const flagKey = decisionInfo.flagKey;

  const tags = {
    user_id: userId,
    enabled: decisionInfo.enabled,
    flag_key: flagKey,
    rule_key: decisionInfo.ruleKey,
    variation_key: decisionInfo.variationKey,
    ...attributes,
  };

  sendMetricsToDatadog(flagKey, MetricType.COUNT, 1, tags);
}

optimizely.notificationCenter.addNotificationListener(
  enums.NOTIFICATION_TYPES.DECISION,
  onDecision,
);

// Trigger the notification listener
const userId = "<user_id>";
const attributes = { attribute_key: "attribute_key" };

optimizely.onReady().then(() => {
  const user = optimizely.createUserContext(userId, attributes);
  user.decide("flagKey");
});

Additional SDKs

To configure Datadog with other Feature Experimentation SDKs, see the following documentation on decision notification listeners:

Updated about 1 month ago

Configure Datadog

📘
Note

📘
Note

Use cases

Example code

Python SDK

JavaScript (Node) SDK

Additional SDKs

📘Note

📘Note

Use cases

Example code

Python SDK

JavaScript (Node) SDK

Additional SDKs

📘
Note

📘
Note