After you start running an experiment, you can increase total traffic and rest assured that the same users get the same experience.
But things are a little more complicated if you decrease traffic or otherwise reconfigure an experiment after you start running it. If you intend to reconfigure an experiment while running, you must implement a user profile service to ensure sticky bucketing. Bucketing is the process of assigning users to different variations of an experiment. There are two scenarios in which you might need to implement sticky bucketing:
- Reconfiguring a running experiment to troubleshoot.
- Enabling Stats Accelerator.
The main reason you would decrease traffic is if something goes wrong or if you run into errors. Instead of reducing traffic, you can also turn off an experiment while investigating the failure.
Optimizely Feature Experimentation buckets your users into experiment variations using a deterministic hashing of the user ID and flag rule key. As long as your systems consistently share user IDs and user attributes, then this permits highly efficient bucketing across channels and multiple languages and experimentation without strong network connectivity.
However, if you add variations or change traffic allocation while an experiment is running, users without profiles often get rebucketed. For example, if you dialed traffic on an experiment down to 0% from 50% and then increased it back to 50%, Optimizely Feature Experimentation would reset the bucketing for that experiment, and some users would begin to see different variations than before.
For information, see How bucketing works.
To ensure sticky bucketing, implement a user profile service, which uses a caching layer to persist user IDs to variation assignments. For information, refer to your programming language's SDK topic:
Updated 3 days ago