How bucketing works

Bucketing is assigning users to a variation according to the experiment's rules. Optimizely Web Experimentation evaluates user IDs and attributes to determine which variation they should see.

Bucketing is:

• Deterministic – A user sees the same variation across all devices they use every time they see your experiment, thanks to how Optimizely hashes user IDs. In other words, a returning user is not reassigned to a new variation.
• Sticky unless reconfigured – If you reconfigure a "live" (running) experiment, such as by decreasing and then increasing traffic, a user may get bucketed into a different variation. If you are using Stats Accelerator, Optimizely Web Experimentation automatically makes the user variation assignment sticky to keep a consistent user experience.

How users are bucketed

Here is how bucketing works in Web Experimentation:

• When a user visits a website that is running an Optimizely experiment, the bucketing algorithm determines which variation of the experiment the user sees. This is typically done using a hashing function, such as the MurmurHash, based on the user's unique identifier, such as a user ID or a cookie value.
• MurmurHash randomly assigns users to different variations so that the groups are statistically comparable.
• The bucketing is scalable, so you can run experiments efficiently across large numbers of users and multiple concurrent experiments.
• You can target specific user segments to tailor your experiments while maintaining randomization and consistency.

MurmurHash is deterministic, so a user ID always maps to the same variation to maintain the integrity of the experiment results as long as the experiment conditions do not change. However, if the traffic allocation percentage changes, the entire traffic allocation bucket shifts, which can rebucket users.

For example, with a traffic allocation of 40% (50/50 split), Visitor 1 sees variation A, but Visitors 2 and 3 are not in an experiment. When the traffic allocation changes to 65% (50/50 split), Visitor 1 sees variation A, Visitor 2 sees variation B, but Visitor 3 is not in an experiment. When the traffic allocation changes to 100% (60/40 split), Vistiors 1 and 2 see variation A, and Visitor 3 sees variation B.

Changing traffic can rebucket users

Increasing "live" traffic for an experiment is the most common way to change bucketing. You can increase the total live traffic allocation percentage without rebucketing users. However, if you change the traffic split percentage between variations, Web Experimentation may place your users in a different bucket than before.

If you decrease then increase traffic, your users are also rebucketed. You should avoid decreasing then increasing traffic in a running experiment because it can result in statistically invalid metrics. If you need to change "live" experiment traffic, do so at the beginning of the experiment.

End-to-end bucketing workflow

This is how Optimizely Web Experimentation evaluates a decision:

1. The snippet loads on the page with your experiment and evaluates the URL targeting.
2. When the page's activation trigger fires and targeting conditions are passed, the page activates.
3. Optimizely Web Experimentation evaluates the experiments that include that page.
4. Optimizely Web Experimentation evaluates whether a visitor is eligible to enter the experiment based on the experiment's traffic allocation and audience conditions.
5. If any audience conditions cannot be determined for a visitor based on known information, Optimizely Web Experimentation may wait a short time to gather this information before deciding whether the visitor qualifies for the audience. If both audience and traffic conditions are passed, the experiment activates.
6. Each visitor who is allocated to an experiment is assigned to a variation.
7. The snippet executes the code that applies those changes.
8. When visitors interact with the webpage, they may trigger a conversion event that Optimizely Web Experimentation tracks.
9. Optimizely Web Experimentation records the event and associates it with the variation.

See Snippet order of activation for more information.