Interactions between rollouts and feature tests

Feature tests take precedence over feature rollouts. This means that if your feature has both a feature test and rollout, the feature test is evaluated first. If a user does not qualify for the feature test, the rollout is evaluated.

The table lists the differences between feature tests and rollouts.

	Feature test	Feature rollout
Purpose	Lets you gather metrics data so you can compare multiple variations.	Lets you deploy a feature that you've already tested or measured some other way, such as qualitative feedback.
User impressions	When a user is assigned to a feature test, Optimizely sends an impression so that information is recorded in your test results.	When a user is assigned to a rollout, Optimizely does not send an impression. This means that extra network traffic and test results aren't generated. Note: Rollouts do not count against your impressions limit.
Variations	Test multiple variations—on versus off, or a multivariate test of different configurations	Only toggles the feature on or off.

Example scenario

To understand how this works in practice, imagine the following scenario. You've deployed a feature to your application and you've created both a feature test and rollout for that feature:

Feature test running with an audience and a traffic allocation
Rollout running with an audience and a traffic allocation

A series of users are handled by your application and are evaluated according to their audience attributes and your feature test and feature rollout rules. The table depicts all possible outcomes.

User	Feature Test Audience	Feature Test Traffic Allocation	Feature Rollout Audience	Feature Rollout Traffic Allocation	Result
user1	pass	pass	N/A	N/A	Feature test
user2	pass	fail	pass	pass	Feature rollout
user3	fail	N/A	pass	pass	Feature rollout
user4	fail	N/A	pass	fail	No action
user5	fail	N/A	fail	N/A	No action

Is Feature Enabled method

Feature tests and rollouts are both triggered using the Is Feature Enabled method.

Interactions in multiple feature tests

In each environment, you can only run one feature test on each feature at a time (meaning, you can't run two tests on the same feature simultaneously) unless your feature is assigned to a mutually exclusive group. If you try to launch a second feature test while another feature test is running, you’ll see a warning that only one feature test is allowed at a time—unless the tests are mutually exclusive.

Use mutually exclusive groups to run concurrent production feature tests. If feature tests are running in a mutually exclusive group, Optimizely evaluates a user’s mutually exclusive group first in the order of operations after calling the Is Feature Enabled method. After the mutually exclusive group is assigned, Optimizely evaluates the feature test first. If a user does not quality for the feature test, the feature rollout is evaluated.

Note that you can have as many draft, paused, and archived tests on a feature as you like.