User profile service in Optimizely Feature Experimentation

After starting an experiment, you can increase total traffic while ensuring that the same users experience consistent variations.

However, decreasing traffic allocation or reconfiguring an experiment after it has started is more complex. If you plan to make changes to an experiment while it is running, you need to implement a user profile service to maintain consistent bucketing. Bucketing refers to the process of assigning users to specific variations within an experiment. You may need to implement sticky bucketing in the following scenarios:

Reconfiguring a running experiment to troubleshoot.
Enabling Stats Accelerator.

The primary reason you would decrease traffic is to troubleshoot problems or address errors. Instead of reducing traffic to 0%, you can also pause the experiment rule while investigating the failure.

How users are bucketed

Optimizely Feature Experimentation uses a deterministic hashing method that combines the user ID and flag rule key to assign ("bucket") users to experiment variations. As long as your systems consistently share user IDs and attributes, you can efficiently bucket users across different channels, languages, and experiments, even in environments with poor network connectivity.

However, if you add variations or change traffic allocation while an experiment is running, users without user profiles may get rebucketed. For instance, if you reduce the traffic from 50% to 0% and then increase it back to 50%, Optimizely Feature Experimentation resets the bucketing for that experiment. As a result, some users might see different variations than before. Additionally, reducing and then increasing traffic allocation could lead to a sample ratio mismatch (SRM). For information, see Optimizely's automatic sample ratio mismatch detection.

Implement a user profile service for consistent bucket assignments

To ensure consistent bucketing, implement a user profile service. This service uses a caching layer to persist user ID to variation assignments. For instructions on configuring your user profile service, consult the SDK documentation for your platform:

Android
C#
Go
Java
JavaScript v6+
JavaScript (Browser) – SDK versions 5.3.5 and below
JavaScript (Node) – SDK versions 5.3.5 and below
PHP
React
React Native
Ruby
Python
Swift