Optimization Glossary
To be able to learn and talk about testing, you have to know the language. Provided here is a glossary of common terms that I use and that are used around the industry. If you see a -More Info- link, it goes to a page with a more in-depth explanation of the term.
A
A/B test: (see Split test)
Array: (1) A group of factors and the levels associated with those factors. (2) A matrix with columns representing the factors in a run and rows representing the experiments in that run and the combination of factor levels for each experiment.
B
Baseline: (see Control)
Baseline test: Baseline tests are used to determine your current rate of traffic, conversion rate, and traffic sources, with the results being saved and used as a basis for future comparisons.
Bounce rate: The percentage of visitors who leave a Web page without visiting any pages linked from that page.
C
Challenger: A modified Web page that’s compared to the control version in a split test.
Champion: (see Control)
Content: (also known as an Element) The HTML markup and/or code snippets contained in a level. This is not the same as the traditional coding definition of “element.”
Control: (also known as the Baseline or Champion) A Web page that’s used as the basis for comparison with variations of that page.
Conversion action: An action performed by a landing page visitor. The goal of the landing page is to get visitors to perform the desired conversion action (clicking a button, filling out a form, subscribing to a newsletter, making a purchase, etc.).
Conversion rate: The percentage of visitors who perform the desired conversion action.
Conversions: Landing page visitors who perform the desired conversion action.
Cookie: A text file stored on a user’s PC that contains information about previous and/or current visits to a Web site.
D
Dependent variable: (see Response)
E
Element: (see Content)
Experiment: -More Info- (also known as a Recipe or Variation) A specific combination of factor levels (for example, the red headline combined with the small hero shot and the green button) used during a test.
F
Factor: (also known as a Variable or Independent variable) (1) An element of the Web page (headline, image, text) being tested. (2) A variable that is manipulated during an experiment by assigning different values (also known as levels) to that variable.
Factor interaction: (see Interaction)
Factor level: (see Level)
Factorial test: A test in which factors are manipulated simultaneously.
Fractional factorial test: (see also Full factorial test) Fractional factorial tests use a subset of the combinations used in full factorial tests. In a fractional factorial test, if you wanted to test 4 headlines and 4 hero shots, you would use fewer than the 16 possible combinations. Done properly, fractional factorial tests reduce the amount of time necessary to arrive at statistically significant test results.
Full factorial test: (see also Fractional factorial test) Full factorial tests use every possible combination of factor levels. If you wanted to test 4 headlines and 4 hero shots, you would need 16 different pages to test all the possible combinations.
H
Hero shot: A literal or symbolic picture or image of the product or service being marketed. Examples include product shots, photos of customers or seminar speakers, lifestyle photos and book jackets.
I
Independent variable: (see Factor)
Interaction: -More Info- When the effect of one factor on a response is dependent upon the effects of other factors on that same response.
K
Keywords: Search terms used by users of search engines and also the search terms targeted in SEM. The search results are displayed in Search Engine Results Pages (also known as SERPs).
L
Landing page: A Web page created for a specific marketing campaign. The goal of the landing page is to get visitors to perform the desired conversion action (clicking a button, filling out a form, subscribing to a newsletter, making a purchase, etc.).
Level: (also known as a Value) Content that is assigned to a specific factor – for example, one of a series of images to be tested.
LPO: Landing Page Optimization
M
Multivariable test: (see Multivariate test)
Multivariate test: (also known as a Multivariable test) A factorial test (either full or fractional) using multiple variations of a Web page, with multiple factors changing simultaneously.
MVT: MultiVariate Test
R
Recipe: (see Experiment)
Response: (also known as a Dependent variable) The measured output of an experiment. In Web page testing, the conversion rate is one of the responses that can be measured.
Run: (also known as a Treatment) A series of experiments sent to a Web page for display to visitors of a Web site.
S
Segment: A specific portion of the total traffic at a Web site or page. Breaking the traffic into segments provides greater opportunities for analysis.
SERP: Search Engine Results Page
Split test: (also known as an A/B test) A split test compares two or more Web pages. The most basic split test would compare 2 pages, identical in all aspects but one (for example, different headlines). During the test, visitors are shown one of the pages (“A” or “B”), asked to perform a conversion action, and the results from the two pages are compared.
Statistical Design of Experiments: (also known as DoE, DoX, and SDE) The theory of experimental design emphasizing full factorial and fractional factorial designs with the goal of efficient and thorough optimization of products and processes.
T
Test matrix: A table showing all the factors, all the factor levels, and all the content being tested. A test matrix is a useful tool for constructing and analyzing experiments.
Treatment: (see Run)
U
Uniform Resource Locator: (also known as a URL) The Web address of a site or page (for example, www.widemile.com).
URL: Uniform Resource Locator
V
Value: (see Level)
Variable: (see Factor)
Variation: (see Experiment)
