This topic answers commonly asked questions.
- What can I do with Optimizely Full Stack?
- Can I run server-side experiments?
- How does Full Stack keep features consistent across servers and different SDKs?
- How does datafile management work in Full Stack?
- How do Full Stack SDKs handle bot traffic?
- Do I have to pass a user ID? What if I want anonymous users?
- What is the licensing model for paid plans?
- How can I try Optimizely in my tech stack?
- How is Rollouts different from the Rollouts Plus and Full Stack plans?
- How many collaborators can I add to my account?
- How does Full Stack affect my application speed?
- How long does it take for changes in Full Stack to be reflected in my app? How often do you poll the datafile?
- Does Full Stack make an API call to serve features to my users?
- What happens to my app if there is a service interruption?
- What is the overhead of requesting a feature or running a feature test?
- Can less-technical users work with Full Stack?
- Can I use Optimizely SDKs without a connection to the internet?
- Can I use Full Stack if my site is heavily cached to a CDN?
- Do I need to modify my firewall when using Full Stack?
- What if my stack is service-oriented or uses microservices?
- Does Full Stack need to connect to the cloud?
- Which languages does Full Stack work with?
- Can Full Stack work in a multi-tenant setup? Can I provision features among clients?
- Do I need to set up my feature in your UI after implementing it?
Integrations and Data
- How do I access raw events?
- What integrations does Full Stack offer?
- How do I track and manage flags to keep the code clean?
- What if I want to measure which feature people saw?
What can I do with Optimizely Full Stack?
Optimizely Full Stack is feature management and A/B testing for product teams.
- Feature flags and rollouts help you launch features safely and give you a kill switch if unanticipated issues arise.
- Feature tests empower you to experiment with feature configurations and iterate on features without deploying code.
- Optimizely's industry-leading Stats Engine calculates results so you can learn what is working and why.
- Full Stack A/B tests enable you to run experiments in any application. These require you to deploy code, so they are best used for one-off decisions such as comparing one algorithm against another.
Pair Optimizely's Full Stack and Web products to empower your entire organization, from product to marketing teams to experiment and build an optimized customer experience.
Can I run server-side experiments?
You can experiment anywhere in your technology stack with Optimizely's SDKs. If your plan includes Full Stack, you can run experiments in various languages. If you are interested in testing in a different language, contact us.
How does Full Stack keep flags and users consistent across servers and different SDKs?
You pass a user ID and feature flag key to Full Stack SDK, and this buckets people into the flag. So, if you are running multiple servers or programming languages with the same feature flag key, the SDK will always give the same response (Booleans and, if your plan includes them, flag variables). The SDK always gives the same answer based on the user information you've specified, because we use deterministic bucketing via MurmurHash3 to determine which experiments and variations should be active for a user.
How does datafile management work in Optimizely Full Stack?
The datafile is language-agnostic. You can use the same datafile from the same project across different SDKs and get consistent bucketing. This way, you can activate and track the exact same experiments across different SDKs and get consistent results.
How do Optimizely SDKs handle bot traffic?
Optimizely Full Stack SDKs support bot detection as of version 2.1. Enable bot filtering directly from your account’s project settings. Once the setting is enabled, events sent from web browsers have bot filtering applied automatically. Applying bot filtering to events sent from other environments requires configuration. For more context, read about bot filtering.
Do I have to pass a user ID? What if I want anonymous users?
You have to pass an ID, but it can be any hashed ID you’d like, as long as it is consistent with what you want to toggle. User IDs work best in most cases, but we have also seen customers use session IDs or even request-level IDs for experiments.
What is the licensing model for paid plans?
You are billed based on a monthly active user (MAU)-based model. For more information, see What are monthly active users.
How can I use Optimizely in my tech stack?
Optimizely Rollouts is free to all users and all companies - feel free to download Rollouts and start using the flags today. You will see the basic Optimziely UI and can run one experiment at a time. If you would like to add more seats or MAUs, upgrade to the full functionality of Full Stack, or if you are interested in our client-side solution, Optimizely Web Experimentation, contact us to discuss your requirements.
How is Rollouts different from the Rollouts Plus and Full Stack plans?
Rollouts is the free plan for Full Stack intended for startups and teams looking to get started with feature flags and A/B tests. Rollouts and Rollouts Plus both come with a subset of Full Stack features, including unlimited feature flags and controlled rollouts, and the ability to run one experiment at a time.
Teams ready to manage feature flags across more users and applications can upgrade to Rollouts Plus for paid access to additional collaborators and MAUs, tools for governance and security, technical email support, and unlimited Targeting Rules, Projects, and Change History.
Paid Full Stack plans, which removes the one concurrent experiment limit, offer a more robust feature set along with best-in-class support and services for organizations with more stakeholders invested in increasing experiment velocity and feature delivery.
How many collaborators can I add to my account?
In Full Stack, you can invite 20 collaborators, depending on your plan. We encourage you to invite the whole team to increase visibility, share results and insights and deepen experimentation across your working group. Optimizely offers permissions by role and additional integrated program management tools, so your team can collaborate more effectively.
How does Optimizely affect my application speed?
Full Stack SDKs are built so you can split traffic to experiments without making any network requests. Unlike some platforms, which call out to third-party servers for experiment decisions, all decisions are made in-memory using a cached copy of the datafile. The impact on latency is negligible.
In other words, Full Stack is faster because it does not make any blocking API requests to get decisions about which experiment variant to use; the device (your mobile phone or server) running the code makes that decision in under a millisecond. Full Stack will not slow down your end user's experience. However, there are still a few performance considerations to keep in mind as you scale your usage of Full Stack:
- When and how often to download the datafile that in-memory bucketing uses. Manage this by limiting the file size and download frequency.
- When and how often to send data about conversion events for tracking. Manage this by batching events and using asynchronous event dispatching (having the app send data out-of-band later).
We are always happy to discuss performance. Contact us if you are interested in seeing performance benchmarks for any of our SDKs.
How long does it take for changes in Full Stack to be reflected in my app? How often do you poll the datafile?
Changes in your app depend on how frequently you refresh, or poll, your datafile. Changes are typically reflected in the datafile within ~60 seconds.
Does Optimizely Full Stack make an API call to serve features to users?
No network requests are required to evaluate feature flags and experiments, so there's no added latency. Fetch the datafile in advance, the SDK only needs access to the datafile contents. Download the datafile out-of-band from serving an individual request from your users.
After the datafile is downloaded, no network requests are needed to serve features to users. However, the SDK will still send network requests to track events. You can control when and how this happens by implementing a custom event dispatcher. For more information on event dispatchers, see the event dispatcher configuration topic for your SDK.
What happens to my app if there is a service interruption?
An app.optimizely.com interruption will not affect your app’s operation. However, during the interruption, you will not be able to log into the site to update experiments or make other changes. For more information, see What happens when Optimizely goes down?.
Additionally, you can view the Optimizely status page.
What is the overhead of requesting a feature or running a feature test?
Optimizely does not have a concept of "requesting features." Optimizely does not make network calls to serve features or experiments to users, so there's zero latency. For more information, see Does Full Stack make an API call to serve features to my users?.
Can less technical users work with Optimizely Full Stack?
Full Stack is built for product teams. Developers work with our SDKs, but on many teams, product managers and analysts with less technical expertise configure features and experiments, manage rollouts, and analyze results in Optimizely Full Stack.
Can I use Optimizely SDKs without a connection to the internet?
Yes, you can use Full Stack using a local copy of your datafile and batch up events to send later when your app is back online.
Can I use Optimizely Full Stack if my site is heavily cached on a CDN?
Yes, it is possible to use Full Stack with a heavily cached site. This setup requires appropriate configuration at your CDN provider and some additional implementation in the backend. Contact your Customer Success Manager to discuss options. Read more about content delivery networks.
Do I need to modify my firewall when using Optimizely Full Stack?
You probably will not need to modify your firewall to use Full Stack. If your firewall has egress (outbound traffic) restrictions, you will need to allowlist
What if my stack is service-oriented or uses microservices?
You can use Optimizely as a service or include the Optimizely SDK in every service. For some best practices and special considerations when implementing Full Stack, see Microservices.
Does Full Stack need to connect to the cloud?
Yes. Our SDKs need to be able to download our JSON payload (the "datafile"). The datafile contains all the metadata about your features that the SDK needs to operate (toggle feature flags, apply remote configuration ("feature variables"), and segment users). The SDKs do not need two-way communication with our servers.
It is possible to run the open-source Optimizely Agent as a microservice or to download the datafile from an authenticated endpoint, which minimizes the security risk. Some customers even relay this information to their own CDN for added security and control.
Which languages does Full Stack work with?
Can Full Stack work in a multi-tenant setup? Can I provision features among clients?
Yes. Some Full Stack users set up a Full Stack environment for each of their customers. That way you can use the same flag key everywhere and deliver progressively.
Other Full Stack users use key-value pairs to create audiences targeting specific customers in order to provision flags.
Do I need to set up my feature in your UI after implementing it?
Yes, you need to log into the Optimizely app and set up the feature flag using exactly the same key. This also makes sure anyone logging into Optimizely knows what flags are where and why (you can label and describe them). You can also use our REST API to change features.
If you prefer, you can also build your own UI, or use it in combination with the Optimizely app. We have a microservice, REST API, and command-line interface you can talk to.
Integrations and Data
How do I access raw events?
With access to Optimizely export features, you can export the raw events from AWS S3.
What integrations does Full Stack offer?
We provide guides for connecting with analytics platforms. For more information, see Set up analytics platforms. For a full list of Optimizely partners, see Partners.
How do I track and manage flags to keep the code clean?
Most customers we speak to use our Jira integration with an internal workflow so that tickets track which feature flags are rolled out to what percent.
See plans to check if the Jira integration is included in your plan.
What if I want to measure which feature people saw?
You will want to run an experiment (as opposed to simply using a feature flag to deliver an on/off experience). To analyze your experiment results, you have the following options:
- You can integrate with your existing analytics tool. See Set up analytics platforms for guides for the major analytics platforms (for instance Google Analytics, Segment, or Mixpanel).
- If you would like to measure which state/feature works best, we suggest using the (paid) Optimizely product with our Stats Engine.
- If you have data scientists on your team, you can use Data Lab.
We suggest getting started with the free plan, implementing it, and seeing how you like it. If you are ready to start experimenting, you can use the same SDKs to upgrade to a paid plan.
Looking for more discussion on getting started with feature flags and experimentation? Check out Optimizely's Slack Developer Community to learn from other developers and share.
Updated about 2 months ago