How bucketing works

Bucketing assigns users to a variation according to the experiment's rules. Optimizely Web Experimentation evaluates user IDs and attributes to determine which variation they should see.

Bucketing is

• Deterministic – A user with the same ID always sees the same variation because Optimizely hashes user IDs consistently. Returning users are not reassigned.
• Sticky unless reconfigured – Reconfiguring a running experiment (such as decreasing then increasing traffic) can rebucket users into different variations. Stats Accelerator maintains sticky assignments to preserve consistent user experience.

How users are bucketed

When a user visits a page running an Optimizely experiment, the bucketing algorithm uses MurmurHash to hash the user's unique identifier (user ID or cookie) to determine which variation they see. MurmurHash distributes users across variations so that the groups are statistically comparable.

Because MurmurHash is deterministic, a user ID always maps to the same variation as long as experiment conditions do not change. However, if the traffic allocation percentage changes, the entire traffic allocation bucket shifts, which can rebucket users.

For example, with a traffic allocation of 40% (50/50 split), Visitor 1 sees variation A, but Visitors 2 and 3 are not in an experiment. When the traffic allocation changes to 65% (50/50 split), Visitor 1 sees variation A, Visitor 2 sees variation B, but Visitor 3 is not in an experiment. When the traffic allocation changes to 100% (60/40 split), Visitors 1 and 2 see variation A, and Visitor 3 sees variation B.

Changing traffic can rebucket users

You can increase total traffic allocation without rebucketing users. However, changing the traffic split between variations may reassign users to different variations.

Decreasing then increasing traffic also causes rebucketing, which can invalidate experiment metrics. If you need to adjust traffic on a running experiment, do so at the beginning before results accumulate.

End-to-end bucketing workflow

The following shows how Optimizely evaluates a decision:

1. The snippet loads on the page with your experiment and evaluates the URL targeting.
2. When the page's activation trigger fires and targeting conditions are passed, the page activates.
3. Optimizely evaluates the experiments that include that page.
4. Optimizely evaluates whether a visitor is eligible to enter the experiment based on the experiment's traffic allocation and audience conditions.
5. If any audience conditions cannot be determined for a visitor based on known information, Optimizely may wait a short time to gather this information before deciding whether the visitor qualifies for the audience. If both audience and traffic conditions are passed, the experiment activates.
6. Each visitor who is allocated to an experiment is assigned to a variation.
7. The snippet executes the code that applies those changes.
8. When visitors interact with the webpage, they may trigger a conversion event that Optimizely tracks.
9. Optimizely records the event and associates it with the variation.

See Snippet order of activation for more information.