This message is relevant for anyone interested in getting better results from accelerated split testing methods. But it’s especially relevant if you’re thinking of purchasing STA during the May 5th/6th (or next) buying opportunity.
Dear Friend,
Please read this carefully. There’s a revolutionary new feature in STA. Existing customers will get it shortly — once I put the documentation together. New customers will get it immediately — but you’ll have to wait for the documentation.
Now, . . .
. . . do you know the difference between sales pages and opt-in forms? I mean from the testing standpoint?
Typically sales page conversion rates are between 0.5% and 3%, and opt-in form conversion rates are between 10% and 50%.
And that means it takes about 1/4 the time to test an opt-in page as it does a sales page.
If it takes 4,000 visitors to get good results for a couple factors in a multivariate test on a sales page, often you can get the same kinds of results with about 1000 visitors on an opt-in page.
It has to do with being in the informational “sweet spot”. When your conversion rate is really close to 0% (or 100% for that matter), it just takes longer for options to distinguish themselves. When the conversion rates are between 20% and 80%, you’re in a real sweet spot where differences show themselves more readily.
That’s why opt-in forms optimize more quickly than sales pages — typically.
Now, what if I told you . . .
There’s now a way to
speed up testing
on sales pages
We do it by measuring “time on page”.
STA now allows you to track the time a visitor spends on your page — and it integrates this data with all the multivariate features, so you can compare — factor by factor — which options cause visitors to linger longer on your pages.
The new time on page feature actually tracks the percentage of visitors who spend more than 10 seconds on your page. The percentage who spend more than 30 seconds, and the percentage who spend more than 60 seconds.
And here’s the thing. In the testing I’ve done with this so far, the percentage of visitors spending 60 seconds on the page has two important features:
- It correlates very highly with sales.
- The percentages are typically between 20% and 40%.
And that puts us in the informational sweet spot — even for sales pages that have conversion rates under 1%.
I plan to do a full special report on this, with a video demonstration. Look for that later this month.
For now just know that new installations of the Split Test Accelerator will have this feature, and the documentation is coming.
Existing customers, stay tuned. Some time in May you should be able to start using this feature.
A reminder for those who are thinking of purchasing STA sometime on May 5th or 6th 2008 (or the next opportunity relative to the time you’re reading this), you can get your copy here:
http://www.splittestaccelerator.com
Best wishes, as always.
Sincerely,
Jim.
P.S. As always, your comments or questions about this new feature are welcome. Just post your comment below.
May 5th, 2008 at 11:29 am
I don’t see how this speeds up testing, since if there is a correlation then wouldn’t we see the same critical factors with the current version looking at sales conversion?
You’re also mixing optin vs sales pages and how does that relate to visitor time vs sales?
May 5th, 2008 at 11:45 am
Hi Peter.
Great question.
I probably wrote a little quickly, so let me try to make the connections a little more clear here.
If your sales page gets only 0.5% conversion rate, then after 1000 visitors, you’ll probably have something in the neighborhood of 5 sales.
When you spread those sales across 3 options for some factor, you might have this:
A: 1 sale out of 333 impressions
B: 3 sales out of 334 impressions
C: 1 sale out of 333 impressions
That’s not going to be anywhere near statistically significant. You’ll have to wait a long time to make the call with your conversion rate so low.
But if you have time on page data as well, and it reads (for percentage staying 60 seconds or more):
A: 33% (say, 111/333)
B: 50% (say, 167/334)
C: 33% (say, 111/333)
You can be exremely confident in those relative numbers. It would be over 99% confidence.
So, you can conclude with confidence that option B causes more visitors to stay on the page at least 60 seconds than the other options.
Now, since time on page is highly correlated with sales, you can be fairly certain (less certain than about time on page, but still fairly certain), that, in the long run, option B will lead to more sales as well.
It kind of gives you a sneak peek at where the long term sales data is going.
It’s not foolproof, but it’s a tremendous new tool.
Jim
May 5th, 2008 at 1:52 pm
Sounds like a great new feature that I hadn’t thought of testing.
Have you thought of also measuring how far down the page the visitor scrolls as a factor in conversions?
May 8th, 2008 at 11:15 pm
[...] Speeding Up Taguchi?? — New STA Voodoo — time on page tracking [...]
May 18th, 2008 at 6:53 pm
Jim,
Great observation and great testing idea — my question is where you got the stat that time on page is correlated with increased sales? Do you have a link?
Roy
May 20th, 2008 at 1:00 am
Roy,
It was a conversation that started a few years ago… Dr. Kowalick was looking for a way to measure the attention value of an ad.
“You have to get attention first. Then know that, people give attention, which takes time on those things that interest them. Thus time = attention = interest.”
There is other thinking that will help with the analysis but that this the basis of the idea. Input design is still very important.
If you do analysis of ecommerce sites. The higher the bounce rate the lower the sales. As you reduce bounce rate, sales will increase. This is because people are staying on the site and getting the message.
You can not sell to people who are not there. They must be interested in your product or service.
The more time you have with them is a start to profit.
In the real world, this is called “facetime”. You will not close 100% of the calls you do not make. More facetime means the possibility of more sales.
- David Bullock
May 21st, 2008 at 11:03 am
Hey Chris, Roy, and David.
Chris: Yes, I looked into the “how far scrolled” metric a couple years ago. The implementation was a little funny on some platforms, so I set the idea aside. That metric should work pretty well for the same reasons that time on page works pretty well — perhpas even better, since the time on page metric is muddied a little by people who click to the page, get distracted by something else, and let the page sit for a while before “bouncing”.
Roy: So far my data is unpublished. And I’ve may have overstated the case a little. If I said T.O.P. “IS” highly correlated with sales [full stop], that’s an overgeneralization, of course. I’m trying to be careful, but some people will criticize me for putting too many qualifications in the text
Time on page CAN deviate from sales. And there is a real danger of this happening if you design FOR time on page, and not sales. But from what I’ve seen so far the two track well enough to allow enough confidence to speed up testing significantly.
David: Thanks for chiming in. So you could have told me about this years ago?? Why were you holding back?