Run Contextual Multi-Armed Bandit optimizations

Contextual Multi-Armed Bandits (CMABs) are a machine learning strategy that personalizes user experiences by dynamically selecting the best-performing variation based on user-specific attributes.

Traditional multi-armed bandits (MABs) identify a single optimal variation for all users. CMABs, powered by Opal (Optimizely's embedded AI), tailor decisions to each user's context, such as device type, location, or behavioral history, to maximize performance on a defined metric.

CMABs address several challenges with traditional experimentation:

• Personalized experiments are resource-intensive – They can deliver higher uplift but are slow to run, especially for small segments.
• Traditional A/B testing is time-consuming – Teams often abandon tests due to the effort required.
• Manual personalization does not scale – Organizations need automated, scalable personalization to reduce setup and maintenance.

CMABs use reinforcement learning to optimize which variation Feature Experimentation shows to each user, based on their attributes and past responses. The model balances exploration (testing new variations) and exploitation (serving the best-known variation) to maximize uplift. Over time, the model l achieves one-to-one personalization as it learns from more data.

📘

Note

Optimizely does not support republishing CMABs that you have archived.

Configuration overview

To configure a CMAB, complete the following:

1. (Prerequisite) Create a flag in your Feature Experimentation project.

2. (Prerequisite) Handle user IDs.

📘

Note

If you are using a server-side SDK, you should configure a user profile service to ensure consistent user bucketing.

3. Define User Attributes. These are the characteristics you provide to CMAB for personalization.
4. Create and configure a CMAB rule.
5. If you have not already done so, implement the Feature Experimentation SDK decide method in your application's codebase through a flag.
6. Test your CMAB rule in a development environment. See Test and troubleshoot.

Configure a CMAB

1. Go to Flags, select your flag, and select your environment.

2. Click Add Rule and select Contextual Bandit.

   

3. Enter a Name for the rule.

4. The Key field auto-populates from the Name. Edit it if needed.

5. (Optional) Search for and add audiences. To create an audience, see Target audiences. Keep audiences broad to give the model enough data for effective personalization.

6. Set the Traffic Allocation percentage for the experiment.

7. Add metrics based on tracked user events.

   • For information on creating and tracking events, see Create events.
   • For information about selecting metrics, see Choose metrics.
   • For instructions on configuring metrics, see Create a metric in Optimizely using the metric builder.

📘

Note

CMAB optimizes exclusively for the primary metric. However, you can include secondary or monitoring metrics for informational purposes, even though they do not influence the optimization process.

8. Select a Distribution Goal. This determines how the model balances exploration (showing less-tested variations to gather data) and exploitation (serving the variation most likely to perform best for each user). Choose one of the following modes:

   1. Automated (recommended) – The model adjusts the exploration/exploitation ratio dynamically, learning from historical data to optimize for each user's attributes and past responses.
   2. Manual – Adjust the slider to set the exploration/exploitation ratio yourself. More exploration means more random testing across variations; more exploitation means the model relies more heavily on its current best predictions for each user. Use this if you want to fine-tune the balance beyond what the automated mode provides.
   3. Maximize Personalization – The slider automatically sets the ratio to 10% exploration / 90% exploitation. This lets the model occasionally test new variations while ensuring most users receive the variation predicted to perform best based on their attributes and past interactions.

9. Choose the Variations you want to optimize. Unlike A/B experiments, you do not need to compare against a baseline because Feature Experimentation does not calculate statistical significance for CMAB optimizations. The CMAB manages variation traffic distribution automatically.

10. Specify User Attributes for optimization. These are the characteristics you provide to the model for personalization—such as age, location, or interests. The model uses these attributes to identify user cohorts and learn which variation performs best for each group based on historical response data. As more data accumulates, the model segments users into increasingly narrow groups, enabling more granular personalization over time. To create attributes, see Define user attributes to build audiences.

    

11. (Optional) Add the CMAB to an Exclusion Group.

12. Click Save.

Configure CMAB in the SDK

Configure CMAB settings in your SDK, including caching behavior and prediction endpoint customization:

• Agent
• C#
• Go
• JavaScript
• Python
• Ruby

Key features

• Attribute-based personalization – The model learns which variations perform best for different user segments based on the attributes you provide.
• Automated traffic allocation – Traffic shifts toward better-performing variations per user segment without manual intervention.
• Configurable exploration/exploitation – Choose how aggressively the model favors known winners versus testing alternatives.
• Results page – Displays performance metrics, similar to Optimizely Web Experimentation and Personalization campaigns. (Currently in development.)

Why CMABs do not show statistical significance

A/B tests use fixed traffic splits to measure whether a variation is statistically significant compared to a control. CMABs work differently—they continuously shift traffic toward better-performing variations per user segment. This dynamic allocation optimizes for overall performance rather than producing a binary winner-or-loser verdict with statistical confidence.

Implement the experiment using the decide method

Flag is implemented in your code

If you have already implemented the flag with a Decide call, no further changes are needed. The SDKs let you reuse the same flag implementation across different flag rules.

Flag is not implemented in your code

If you have not implemented the flag yet, copy The following sample code into your application. Edit it so your feature code runs based on the decision returned by Decide.

Use the following example Decide method code to enable or disable the flag for a user.

// Decide if user sees a feature flag variation
user := optimizely.CreateUserContext("user123", map[string]interface{}{"logged_in": true})
decision := user.Decide("flag_1", nil)
enabled := decision.Enabled

// Decide if user sees a feature flag variation
var user = optimizely.CreateUserContext("user123", new UserAttributes { { "logged_in", true } });
var decision = user.Decide("flag_1");
var enabled = decision.Enabled;

// Decide if user sees a feature flag variation
var user = await flutterSDK.createUserContext(userId: "user123");
var decisionResponse = await user!.decide("flag_1");
var decision = decisionResponse.decision;
var enabled = decision!.enabled;

// Decide if user sees a feature flag variation
OptimizelyUserContext user = optimizely.createUserContext("user123", new HashMap<String, Object>() { { put("logged_in", true); } });
OptimizelyDecision decision = user.decide("flag_1");
Boolean enabled = decision.getEnabled();

// Decide if user sees a feature flag variation
const user = optimizely.createUserContext('user123', { logged_in: true });
const decision = user.decide('flag_1');
const enabled = decision.enabled;

// Decide if user sees a feature flag variation
$user = $optimizely->createUserContext('user123', ['logged_in' => true]);
$decision = $user->decide('flag_1');
$enabled = $decision->getEnabled();

# Decide if user sees a feature flag variation
user = optimizely.create_user_context("user123", {"logged_in": True})
decision = user.decide("flag_1")
enabled = decision.enabled

// Decide if user sees a feature flag variation
var [decision] = useDecision('flag_1', { overrideUserAttributes: { logged_in: true }});
var enabled = decision.enabled;

# Decide if user sees a feature flag variation
user = optimizely_client.create_user_context('user123', {'logged_in' => true})
decision = user.decide('flag_1')
decision.enabled

// Decide if user sees a feature flag variation
let user = optimizely.createUserContext(userId: "user123", attributes: ["logged_in":true])
let decision = user.decide(key: "flag_1")
let enabled = decision.enabled

For more detailed examples of each SDK, see:

• Android SDK example usage
• Go SDK example usage
• C# SDK example usage
• Flutter SDK example usage
• Java SDK example usage
• JavaScript SDK example usage – SDK versions 6.0.0 and later.
• JavaScript (Browser) SDK example usage – SDK versions 5.3.5 and earlier.
• JavaScript (Node) SDK example usage – SDK versions 5.3.5 and earlier.
• PHP SDK example usage
• Python SDK example usage
• React SDK example usage
• React Native SDK example usage
• Ruby SDK example usage
• Swift SDK example usage

Adapt this code in your application to show or hide functionality based on the boolean returned by Decide. The return value is determined by your flag rules—for example, false if the user is assigned to a control or "off" variation.

📘

Note

Users are evaluated against each rule in the flag's ruleset in order before being bucketed into a variation.

See Interactions between flag rules for information.