Here's a QA checklist for launching an experiment:
- Start your test in a non-production environment.
- Force yourself into each experience using a QA tool. See Choose QA tests.
- Trigger any conversion events that are being tracked by the test AND are on the same page as a variation change. Testing pre-existing or downfunnel events may not be necessary, unless those events also change.
- If you're unable to see your testing activity affect the Results page, this may be due to how Optimizely counts conversions. Conversions are attributed to the first variation assignment.
It may be necessary to reset results in between testing your variations.
- If step 4 doesn’t produce the results you expected, troubleshoot by increasing the log level and consulting your error handler to debug.
- Repeat steps 2 through 5 for each variation of your test.
- Confirm your results. There is a 5 minute interval on results page updates, so you might have to wait up to 5 minutes for it to update. The time of the last update is shown on the results page.
- Reset the results of the experiment so that your QA activity doesn't skew the Results page.
- Launch in your production environment.
Updated about a year ago