Billy's Blog: Multivariate Testing & Optimization

I just received a link to an amazing resource from MarketingExperiments, it’s a compilation of great webinar summaries and case studies that they have done. They cover topics from landing page optimization to price testing to PPC and more. While not everything is about testing specifically, all their advice and ideas can be tested, which is why I think you all will find it valuable.

All testing should be carefully designed; it should be focused on best practices and tactics that are predicted to connect with the audience. You should take risks when testing, but they should be calculated risks.

Check it out and soak up some knowledge on optimization and get ideas to test on your site.

MarketingSherpa is giving us a lot of love recently with The Weather Channel case study and now one on Smartsheet. If you were too shy to sign up for our case study on the Widemile website, then please check out the one at MarketingSherpa.

I especially like the last 4 points in the story:

#1. Don’t sit pat on your conversion rates. Colacurcio didn’t know that her original 5%-7% was above the industry average going in and is glad she didn’t. “If I would have said, ‘My conversion rate is pretty good’ and done nothing, I would have totally missed the opportunity to double it.”

#2. Conducting just a few multivariate test and applying them to a greater number of landing pages works. Simply put, you don’t have to test each landing page individually. “Sometimes it seems overwhelming when you think of multivariate testing, but you can cut corners. There are a lot of things that are low-hanging fruit — things that can be applied across landing pages.”

#3. Redesigning shouldn’t stop with your team’s new ideas. “You really have to get your organization into the mindset: ‘We are testing. We are not just going to spit out the next five Web pages.’ ”

#4. Even if your higher-ups are impressed with the initial results, Colacurcio says, marketers should expect to face organizational barriers when they start their second round of testing.

Don’t forget to check out the before and after in the creative samples.

If none of that catches your eye, Janet Meiners at MarketingPilgrim wrote a great summary on the case study.  She also makes a great point that, “I’ve been in heated debates about the best course of action but testing works best – assuming you’re humble enough to go with the data over your ego.”

Trust your data and watch your numbers build your ego.

One of our good clients The Weather Channel were invited by MarketingSherpa to do a case study on a landing page optimization campaign we did with them.   This was a really interesting page with a lot of great learnings and obviously with that high of a lift, it was very successful. Check out the case study here and make sure you look at the creative samples.

One of the biggest things was that the flash banner out performed the static banner, which best practices typically argue against using flash banners. Not only did it out perform the static banner, but it was one of the most influential items on the page.

I try to keep this blog non-company specific but my boss Frans Keylard, Director of Optimization, helped write up an informative case study of a client he worked on, Smartsheet.com. The case study is 4 pages long, covering the process and analysis of a multivariate test we did on one of Smartsheet.com’s landing pages. You can download it from Widemile.com, it requires a name and an e-mail to get it (sorry!)

Here’s my favorite image from the whole thing…