The Full Stack Developer Guide Developer Hub

Welcome to the Full Stack Developer Guide developer hub. You'll find comprehensive guides and documentation to help you start working with the Full Stack Developer Guide as quickly as possible, as well as support if you get stuck. Let's jump right in!

Get Started    

Interactions between rollouts and feature tests

Feature tests take precedence over feature rollouts. This means that if your feature has both a feature test and rollout, the feature test is evaluated first. If a user does not qualify for the feature test, the rollout is evaluated.

The table lists the differences between feature tests and rollouts.

Feature test
Feature rollout

Purpose

Lets you gather metrics data so you can compare multiple variations.

Lets you deploy a feature that you've already tested or measured some other way, such as qualitative feedback.

User impressions

When a user is assigned to a feature test, Optimizely sends an impression so that information is recorded in your test results.

When a user is assigned to a rollout, Optimizely does not send an impression. This means that extra network traffic and test results aren't generated.

Note: Rollouts don't count against your impressions limit.

Variations

Test multiple variations—on versus off, or a multivariate test of different configurations

Only toggles the feature on or off.

Example scenario

To understand how this works in practice, imagine the following scenario. You've deployed a feature to your application and you've created both a feature test and rollout for that feature:

  • Feature test running with an audience and a traffic allocation
  • Rollout running with an audience and a traffic allocation

A series of users are handled by your application and are evaluated according to their audience attributes and your feature test and feature rollout rules. The table depicts all possible outcomes.

User
Feature Test Audience
Feature Test Traffic Allocation
Feature Rollout Audience
Feature Rollout Traffic Allocation
Result

user1

pass

pass

N/A

N/A

Feature test

user2

pass

fail

pass

pass

Feature rollout

user3

fail

N/A

pass

pass

Feature rollout

user4

fail

N/A

pass

fail

No action

user5

fail

N/A

fail

N/A

No action

Is Feature Enabled method

Feature tests and rollouts are both triggered using the Is Feature Enabled method.

Interactions in multiple feature tests

In each environment, you can only run one feature test on each feature at a time (meaning, you can't run two tests on the same feature simultaneously) unless your feature is assigned to a mutually exclusive group. If you try to launch a second feature test while another feature test is running, you’ll see a warning that only one feature test is allowed at a time—unless the tests are mutually exclusive.

Use mutually exclusive groups to run concurrent production feature tests. If feature tests are running in a mutually exclusive group, Optimizely evaluates a user’s mutually exclusive group first in the order of operations after calling the Is Feature Enabled method. After the mutually exclusive group is assigned, Optimizely evaluates the feature test first. If a user does not quality for the feature test, the feature rollout is evaluated.

Note that you can have as many draft, paused, and archived tests on a feature as you like.