The following topics describe the important core concepts of Feature Experimentation.
Optimizely Feature Experimentation SDKs provide a general interface for running experiments throughout your applications and services. Because you will likely have many teams running experiments in different parts of your stack with varying requirements, we have designed Feature Experimentation for maximum customization. You use projects, environments, flags, datafiles and clients to adapt a Feature Experimentation SDK to your developmental workflow.
Feature flags are more than simple on/off toggles for exposing features. You can add remote configuration variables to flag variations and build A/B tests on top of feature flags. You can access all the information about a flag decision for a particular user in the SDK using the Decide methods, including the flag's on/off status, its variable values, its variation key and whether the flag dispatched a decision event to trigger an impression in experiment results, and more.
Flag rules are how you enable and run flag deliveries and experiments.
|Targeted Delivery||Deliver your flag to visitors that match a specific audience. You can roll out your flag to a percentage of your general user base (or to a specific audience) or roll back if you hit bugs. Learn more about Flag deliveries.|
|A/B Test||Test multiple variations of your flag to find the best one. Learn more about A/B tests.|
|Multi-armed bandit optimizations||Use machine learning to allocate traffic to the best-performing variation dynamically. Learn more about Maximizing lift with multi-armed bandit optimizations.|
A Feature Experimentation project lets you isolate each of your teams or applications in their own separate shared workspace. Projects are where a team builds experiments, manages feature flags, defines target audiences and more. Each project has its own set of permissions.
Projects have a many-to-many relationship with your application stack. You can have many different projects to manage different parts of a single application or use a single project to manage omnichannel experiments across your website backend and mobile app.
Because your account already includes a project, creating another project is an optional step.
Use environments in Optimizely to develop and validate your experiment internally before displaying changes to your production website, whether you use a formal deployment environment or not. Each project can have one or more environments.
For example, it is common to set up both staging and production environments in Optimizely. You will often develop and QA your experiments in staging, and deploy to production when you are ready. You can set up different permissions for who can make changes in each of these environments.
Each environment in a project has a corresponding OptimizelyConfig, which is serialized as a datafile. This file captures the configuration data of all your experiments, such as feature flags and event tracking, in a JSON format. When you make changes in a project, each environment's datafile is automatically updated with the new configuration. By synchronizing a local copy of this datafile, the SDK can run experiments without making blocking network requests to an outside API, ensuring near-zero latency.
Datafile changes are also uploaded to the Optimizely content delivery network (CDN), and this process generally takes a few minutes.
When Optimizely actually fetches an updated datafile from the CDN depends on the specific SDK and version, the synchronization method, and the local cached copy version.
Optimizely generates a different version of the datafile for each environment you configure on your account. This allows you to toggle a feature flag in one environment while keeping it paused in another. You do not need to duplicate your code, your Optimizely project or experiments within your project.
A client is an instance of the Optimizely SDK running with a specific datafile and other configuration settings. To use the SDK, get the appropriate datafile and then instantiate a client with it. This client exposes all the methods you need to activate experiments, track events and perform other tasks.
The Android and Swift SDKs (also called the mobile SDKs) offer the additional abstraction of a client manager. The manager handles synchronizing datafiles and generating the associated clients for you, along with other optimizations around event dispatching and persisting user state on the device. Using the manager is optional, and you can instantiate a client directly if you prefer complete control.
Impressions & Decisions
An impression is a unit of measurement of flag usage. Optimizely Feature Experimentation counts an impression each time the SDK sends a decision event. See impressions for more information about:
- When we send impressions and when we do not.
- How we use impressions to calculate data for the Results page.
- Which method to use to generate or not generate impressions for certain use cases.
Our Support article What is an impression in Optimizely provides additional information about impressions, including the differences between how impressions are counted in Optimizely Feature Experimentation versus Optimizely Web Experimentation.
See the following links for explanations of core concepts and reference topics:
Updated about 1 month ago