« Looking forward to 2008 | Home | 3 parts to picking a test page »
1 quick but powerful test design tip
By Billy | March 20, 2008
I was going over my testing plans with my boss, Frans Keylard, today and he reminded me of a very powerful rule.
Test if something works before you try variations of it.
In this case, I was testing out two testimonials. They were quite different in the messaging, however, do I even know if testimonials are read or impact visitors at all? If I test a testimonial and no testimonial, I will immediately know if I should continue trying testimonials. If testimonials win or compare favorably against having no testimonial, then I know to test additional testimonials.
Not that I have never tested factors on/off or tried totally different factors, e.g. a testimonial against a product shot. I had a strong feeling testimonials were going to work, so I assumed they would, although I know I shouldn’t assume anything. An honest mistake, but a good reminder.
Ideally I would be testing variations, along with showing nothing, or “off”, as a variation, however in this case the page didn’t get much traffic so I was limiting my testing to the most important variations and factors.
There might be some fringe cases where this isn’t necessarily true, but in most cases you should just save extra variations for future runs and first find out if your factor has any impact on the page. Maybe I need to read some of my old posts more often.
Related posts
Topics: Methodology |