Optimizely will be sunsetting Full Stack Experimentation on July 29, 2024. See the recommended Feature Experimentation migration timeline and documentation.

Dev GuideAPI Reference
Dev GuideAPI ReferenceUser GuideGitHubNuGetDev CommunitySubmit a ticketLog In
GitHubNuGetDev CommunitySubmit a ticket

Interactions between feature tests and rollouts

This topic describes the execution order of feature tests and rollouts and what will happen with your audiences.

Feature test configuration evaluates before feature rollouts configuration. This means that if your feature runs both a feature test and rollout simultaneously, Optimizely first evaluates if a given user qualifies for the experiment. If they do not, then Optimizely evaluates them for the rollout. Likewise, total traffic first evaluates for experiments, then for rollouts.

For example, if you configure 80% of total traffic to a test, and set 50% traffic for a simultaneously running rollout with the same audience, then 80% of traffic goes to the test, and half of the remaining 20%, or 10%, ends up in the rollout.

The table lists the differences between feature tests and rollouts.

Feature testFeature rollout
PurposeLets you gather metrics data so you can compare multiple variations.Lets you deploy a feature that you have already tested or measured some other way, such as qualitative feedback.
User impressionsWhen a user is assigned to a feature test, Optimizely sends an impression so that information is recorded in your test results.When a user is assigned to a rollout, Optimizely does not send an impression. This means that extra network traffic and test results are not generated.
VariationsTest multiple variations—on versus off, or a multivariate test of different configurationsOnly toggles the feature on or off.
Triggered byIs Feature Enabled methodIs Feature Enabled method

Example scenario

To understand how this works in practice, imagine the following scenario. You have deployed a feature to your application and you have created both a feature test and rollout for that feature:

  • Feature test running with an audience and a traffic allocation
  • Rollout running with an audience and a traffic allocation

A series of users are handled by your application and are evaluated according to their audience attributes and your feature test and feature rollout rules. The table depicts all possible outcomes.

UserFeature Test AudienceFeature Test Traffic AllocationFeature Rollout AudienceFeature Rollout Traffic AllocationResult
user1passpassN/AN/AFeature test
user2passfailpasspassFeature rollout
user3failN/ApasspassFeature rollout
user4failN/ApassfailNo action
user5failN/AfailN/ANo action