HomeGuidesAPI Reference
Submit Documentation FeedbackJoin Developer CommunityLog In

Interactions between feature tests and rollouts

This topic describes the execution order of feature tests and rollouts and what will happen with your audiences.

Feature test configuration evaluates before feature rollouts configuration. This means that if your feature runs both a feature test and rollout simultaneously, Optimizely first evaluates if a given user qualifies for the experiment. If they do not, then Optimizely evaluates them for the rollout. Likewise, total traffic first evaluates for experiments, then for rollouts.

For example, if you configure 80% of total traffic to a test, and set 50% traffic for a simultaneously running rollout with the same audience, then 80% of traffic goes to the test, and half of the remaining 20%, or 10%, ends up in the rollout.

The table lists the differences between feature tests and rollouts.

Feature test

Feature rollout

Purpose

Lets you gather metrics data so you can compare multiple variations.

Lets you deploy a feature that you have already tested or measured some other way, such as qualitative feedback.

User impressions

When a user is assigned to a feature test, Optimizely sends an impression so that information is recorded in your test results.

When a user is assigned to a rollout, Optimizely does not send an impression. This means that extra network traffic and test results are not generated.

Variations

Test multiple variations—on versus off, or a multivariate test of different configurations

Only toggles the feature on or off.

Triggered by

Is Feature Enabled method

Is Feature Enabled method

Example scenario

To understand how this works in practice, imagine the following scenario. You have deployed a feature to your application and you have created both a feature test and rollout for that feature:

  • Feature test running with an audience and a traffic allocation
  • Rollout running with an audience and a traffic allocation

A series of users are handled by your application and are evaluated according to their audience attributes and your feature test and feature rollout rules. The table depicts all possible outcomes.

User

Feature Test Audience

Feature Test Traffic Allocation

Feature Rollout Audience

Feature Rollout Traffic Allocation

Result

user1

pass

pass

N/A

N/A

Feature test

user2

pass

fail

pass

pass

Feature rollout

user3

fail

N/A

pass

pass

Feature rollout

user4

fail

N/A

pass

fail

No action

user5

fail

N/A

fail

N/A

No action


Did this page help you?