NEW: Improve Your Sales — Not Just 10X faster, but 50X faster!
Dear Friend,
In my last post on this topic, I wen’t a little fast. This time I’ll slow down a little and work an example or two. There’s also a video of the new STA feature that allows you to “speed up Taguchi”.
You can read this post right off the screen, but it discusses a VERY IMPORTANT concept for testing, so you might do well to print it out and read it when you have some time, and can be comfortable and relaxed.
Let’s start with this fact:
When you want to improve your conversion rate (and thereby get more sales, optins, or whatever), you have to make some changes to your page.
So you test new ideas hoping to find something that will improve conversions.
Let’s say you test a new headline against your old one, and the new one is 20% better than the old one. You don’t know this at first, so you run some traffic to each version to see how each performs.
Now the question is: “How much traffic do you need in order to have a good chance of detecting a 20% improvement?”
And the answer is: “It depends on the conversion rate you start with.”
Here are some sample numbers.
Starting conversion rate: 0.5%
Ending conversion rate: 0.6%
Improvement: 20%
Visitors needed to confirm: About 60,000.
Starting conversion rate: 1.0%
Ending conversion rate: 1.2%
Improvement: 20%
Visitors needed to confirm: About 30,000.
Starting conversion rate: 10%
Ending conversion rate: 12%
Improvement: 20%
Visitors needed to confirm: About 2,700.
Starting conversion rate: 20%
Ending conversion rate: 24%
Improvement: 20%
Visitors needed to confirm: About 1,200.
Starting conversion rate: 50%
Ending conversion rate: 60%
Improvement: 20%
Visitors needed to confirm: About 250.
Starting conversion rate: 80%
Ending conversion rate: 96%
Improvement: 20%
Visitors needed to confirm: About 50.
So there’s a very important relationship between the starting conversion rate of your page, and the number of visitors it takes to reliably detect a 20% improvement.
Here’s the relationship boiled out in one spot:
0.5% — 60,000 visitors
1.0% — 30,000 visitors
10% — 2,700 visitors
20% — 1,200 visitors
50% — 250 visitors
80% — 50 visitors
We’ll call this “Principle #1“:
“The lower your starting conversion rate, the more traffic you need to detect improvements”
Have you ever noticed that you see split testing results for optin forms more often you see results for sales pages? Principle #1 explains why.
The typical optin form starts with a conversion rate of perhaps 10% to 20%. And the typical sale page starts with a conversion rate of perhaps 0.5% to 2%
It typically takes about 30,000 visitors to detect one 20% improvement on a sales page with a 1% conversion rate.
It typically takes about 2,700 visitors to detect one 20% improvement on an opt-in form with a 10% conversion rate.
Simply put . . .
All else equal . . . you can improve opt-in forms
ten times faster than sales pages.
However, . . .
There’s another principle at play here.
Ask a copywriter how easy it is to find a 20% improvement in each case, and you’ll get a trend diametrically opposed to the one noted above.
Going from 0.5% to 0.6% — Super Easy
Going from 1.0% to 1.2% — Easy
Going from 10% to 12% — A Little Difficult
Going from 20% to 24% — Moderately Difficult
Going from 50% to 60% — Very Difficult
Going from 80% to 96% — Practically Impossible
We’ll call this “Principle #2“:
The higher your starting conversion rate, the more difficult it is to find improvements of a given size (e.g., 20%)”
So here’s the deal. It’s super easy to go from 0.5% to 0.6%, but it will take you a very long time (60,000 visitors) to confirm the result.
On the other hand, it only takes 50 visitors to confirm an improvement from 80% to 96% — the only catch is that it’s pretty much impossible to do.
So the real action is somewhere in the middle.
There’s a “sweet spot” for testing where it’s moderatly easy to come up with improvements, and you don’t need a ton of traffic to detect them.
In my experience . . .
The real “sweet spot” is when a page has
an initial conversion rate between 10% and 50%.
And 20% to 30% is like a sweet spot within the sweet spot.
OK, so what does this mean?
Does it mean you shouldn’t run tests on your low converting sales pages?
No.
What it might mean, though, is that you shouldn’t optimize for “sales” on a sales page.
Instead, you should optimize for an outcome that’s correlated with sales, but which starts out in the “sweet spot”.
I’ve found two ways to move a low converting site into the “sweet spot”.
First,
you can break your sales page into multiple pages. Instead of having just a single page sales letter that goes to the payment gateway, you have, say, a 3 page sales letter that leads to the payment gateway. Then, on page one, instead of optimizing for sales, you optimize for the percentage of visitors clicking from page one to page two.
This “click-through” metric can be highly correlated with sales. And it often starts out in the “sweet spot”. It depends on where you make the break in the page, but “clicks from page one to page two” will probably start somewhere between 10% and 80%.
I think many marketers have already unwittingly take this approach by having a sales page and a separate intermediate order page. In other words, their sales page doesn’t lead directly to the payment gateway, but to an intermediate page that summarizes the offer before taking them to the payment gateway.
It’s not uncommon for someone using a sales page and an intermediate order page, with a 1% overall conversion rate, to see approximately 10% of people visiting the sales page click through to the order page, and about 10% of those viewing the order page ordering.
If your conversions break down like this, and you run a split test on both pages, you can essentially replace one super-slow test with two “sweet spot” tests.
And . . .
In 1/10 the time, you can improve
the “page” more than twice as much.
I used to see people debating the merits of having an intermediate order page. And for people who aren’t running tests on their pages, it’s probably a live question.
However, if you’re split testing, and your overall conversion rate isn’t intially in the “sweet spot”, having an intermediate order page is a “no brainer” when compared with having the sales page alone.
Here’s how it breaks down:
One Page Sales Process:
Measure percentage of visitors reaching the thankyou page.
Starting conversion rate: 1%
Traffic needed to detect a 20% improvement: 30,000 visitors
Two Page Sales Process:
Measure percentage of visitors reaching intermediate order page.
Starting conversion rate: 10%.
Traffic needed to detect a 20% improvement: 2,700 visitors.
Now, if “clicks to the intermediate sales page” correlates with “sales”, and it usually will to some degree (though you have to be careful to design your test for sales and not just clicks to the intermediate page), you could detect a 20% increase in sales with 1/10 the traffic.
You can also run a separate test from the intermediate sales page to the order page, though this will take longer since the intermediate page only sees 1/10 the traffic as the original landing page.
So, one way to move a low converting site into the “sweet spot” for testing, is to increase the number of steps taken by the visitor, and optimize for each step.
But there’s another way . . .
Second,
you can measure “time on page”.
It turns out that time on page is highly correlated with sales, and it’s usually in the “sweet spot”.
The percentage of visitors who stay 60 seconds or longer on the page (or take a desired action sooner), is often somewhere between 20% and 40%. It can be outside that range, but these are fairly common numbers in my experience.
And that’s definitely in the “sweet spot”.
So what you do is . . . instead of trying to take your sales from 0.5% to 0.6%, you can try to take the percentage who stay on your page at least 60 seconds (or take action before that) from 20% to 24%.
And what’s the benfit of doing this?
Here’s the big benefit in a nutshell:
Depending on how well time on page and sales are correlated, you could improve your sales the same amount with just 1,200 visitors instead of 60,000 visitors by optimizing for time on page instead of for sales.
Read that last sentence again. Please.
That’s why I’m taking time to explain this. Measuring time on page is not just a minor little feature. Depending on your starting conversion rate, how your page is set up, and the amount of traffic you get, it might be the ONLY sensible way to optimize your page.
And the time savings of “sweet spot” optimizing get compounded with the already incredible time savings you get with Taguchi testing.
Compared with ordinary split testing Taguchi testing can accelerate your testing 10X, and “sweet spot” testing can accelerate things 4-5 times (You might wonder why I don’t claim a 10X improvement for sweet spot testing . . . remember that, even though you can use less traffic, it can be a little harder to achieve 20% improvement when starting with a higher conversion rate. I’m giving a conservative estimate to take this into account).
So in combination . . .
we’re talking about perhaps 40-50X
faster testing on low converting sales pages.
Now, . . . is “time on page” perfectly correlated with sales?
No.
It’s pretty good, but it can deviate some.
And you have to be careful when you design your tests not to design FOR time on page. You still design your tests to improve sales. But you MEASURE time on page as a proxy for sales.
Also, keep in mind, that, on a long sales letter, only the stuff the visitor sees in the first 60 seconds will affect the time on page metric, so best results will be achieved by testing the “above the fold” and “near the fold” factors.
Anyway, I’m sure I’m missing some pros and cons (feel free to chime in :-D), but that’s the theory behind the new time on page feature in STA.
Now . . .
Let me show you how it works:
Click play to play the video.
http://www.splittestaccelerator.com/videos/timeonpage/timeonpage.html
. . . quick reminder . . .
STA will be up for sale again June 2nd and 3rd, 2008 (and in general the first Monday and Tuesday of each month).
Look for a new offer. I haven’t worked out all the details yet, but let’s just say that I’m going to be working hard to make sure my customers actually get a test set up as soon as possible after purchase.
It is my firm conviction (and a highly justified one) that no qualified customer who actually uses STA will fail to get their money’s worth.
So, once you make your purchase, I want to help you make sure you use it right away.
Stay tuned.
And, as always, please, feel free to comment on this post below.
Sincerely,
Jim
P.S. Quick clarification: Typically when you test a sales page with a low conversion rate, you’re not looking for any particular size of improvement, but you hope to find improvements of all kinds. I used 20% to simplify the discussion. With a low converting page, you’re not hoping to find 20% improvements, precisely because they take too long to find. You’re hoping to find 50% improvements for instance. And you don’t really run 60,000 visitors to your page. Instead, if you don’t find some 50% improvements by the time you’ve sent 10,000 visitors, you stop the test and start a new one. And it should be eaiser to find a 50% improvement on a site starting at 0.5% than one starting at 20%.
True enough.
However, due to the nature of the functions involved, there will still be a sweet spot between 10% and 50%. “Traffic needed” goes up REALLY FAST as the conversion rate goes below 10%. And the difficulty of improving things goes up REALLY FAST as the conversion rate goes over 50%. [feel free to clarify further, if you have questions about this, in the comments section at the end of the post]
May 19th, 2008 at 1:09 pm
Feel free to post comments or questions here
May 19th, 2008 at 8:04 pm
Awesome Jim. 10x-50x faster may just mean 2x-10x the sales!
May 29th, 2008 at 1:10 am
Hey. I’m new to all this. I’m just curious as to how taguchi testing can actually similate 4,000,000 tests with only 18 versions of a page, and a fraction of that traffic. I don’t really expect a super mathematical explanation, but just a general understanding would help me see how this is possible.
Time on page testing is a really good idea. The better your ‘hook’ is, the longer the time on page, so I guess we can focus on improving our site Hook.
Thanks for the tips.
May 29th, 2008 at 8:42 am
Hey Nate.
Actually it takes 36 versions of the page to test 4,000,000 combinations
The explanation takes a little while. I lay it out in the testing guide that’s included with my software.
Here’s an explanation that you might find useful:
http://controls.engin.umich.edu/wiki/index.php/Design_of_experiments_via_taguchi_methods:_orthogonal_arrays
September 2nd, 2008 at 12:29 am
Hello Jim,
I like your software - I noticed the time tracking utility first in a tracking program called Conversion Prophet and purchased it promptly.
It’s a solid program, but your utility is much more robust for a variety of reasons.
The only thing that I would switch is the ability to change the 3 time levels.
If I test a simple opt in page - 60 seconds doesn’t give me any usable data, 30 seconds is on the high end, and 10 seconds doesn’t parse down enough for me to get actionable results.
C.
September 2nd, 2008 at 10:36 am
Hey Christopher.
Thank you for your comment!
I do plan to allow user-defined time intervals.
The way I plan to implement it requires a change to the database, so I want to wait until the next major upgrade.
Jim