When you are deciding what to test on your site, keep these guidelines in mind to help ensure the results you see in reporting are valid.
- Traffic. Check your analytics to ensure you have enough traffic to a location to enable a test to achieve significant results.
- Predictability. The goal of any test is to isolate (as much as possible) changes in the testing environment (web pages, application panels, etc.) so that only the tested content will affect user decisions.
- Time. Make sure tests include two business cycles to provide enough duration to ensure valid results.
- Segment. It is common that even though B might lose to A in overall results, B beat A in certain segments.