Dev GuideAPI Reference
Dev GuideAPI ReferenceUser GuideGitHubNuGetDev CommunitySumbit a ticketLog In
GitHubNuGetDev CommunitySumbit a ticket
These docs are for v2.0. Click to read the latest docs for v3.0.

Interactions between rollouts and feature tests

Feature tests take precedence over feature rollouts. This means that if your feature has both a feature test and rollout, the feature test is evaluated first. If a user does not qualify for the feature test, the rollout is evaluated.

The table lists the differences between feature tests and rollouts.

Feature testFeature rollout
PurposeLets you gather metrics data so you can compare multiple variations.Lets you deploy a feature that you've already tested or measured some other way, such as qualitative feedback.
User impressionsWhen a user is assigned to a feature test, Optimizely sends an impression so that information is recorded in your test results.When a user is assigned to a rollout, Optimizely does not send an impression. This means that extra network traffic and test results aren't generated.

Note: Rollouts do not count against your impressions limit.
VariationsTest multiple variations—on versus off, or a multivariate test of different configurationsOnly toggles the feature on or off.

Example scenario

To understand how this works in practice, imagine the following scenario. You've deployed a feature to your application and you've created both a feature test and rollout for that feature:

  • Feature test running with an audience and a traffic allocation
  • Rollout running with an audience and a traffic allocation

A series of users are handled by your application and are evaluated according to their audience attributes and your feature test and feature rollout rules. The table depicts all possible outcomes.

UserFeature Test AudienceFeature Test Traffic AllocationFeature Rollout AudienceFeature Rollout Traffic AllocationResult
user1passpassN/AN/AFeature test
user2passfailpasspassFeature rollout
user3failN/ApasspassFeature rollout
user4failN/ApassfailNo action
user5failN/AfailN/ANo action

Is Feature Enabled method

Feature tests and rollouts are both triggered using the Is Feature Enabled method.

Interactions in multiple feature tests

In each environment, you can only run one feature test on each feature at a time (meaning, you can't run two tests on the same feature simultaneously) unless your feature is assigned to a mutually exclusive group. If you try to launch a second feature test while another feature test is running, you’ll see a warning that only one feature test is allowed at a time—unless the tests are mutually exclusive.

Use mutually exclusive groups to run concurrent production feature tests. If feature tests are running in a mutually exclusive group, Optimizely evaluates a user’s mutually exclusive group first in the order of operations after calling the Is Feature Enabled method. After the mutually exclusive group is assigned, Optimizely evaluates the feature test first. If a user does not quality for the feature test, the feature rollout is evaluated.

Note that you can have as many draft, paused, and archived tests on a feature as you like.