There is quote I rather like that goes

“Give me six hours to chop down a tree and I will spend the first four sharpening the axe.”

There are a few different variations of the quote but the gist is do your homework, right?

For this case study its quite possible I only spent two minutes sharpening the axe.

More on that in a minute.

First, a quick word about both this case study and future case studies.

Everybody’s website and audience is a bit different.

Not only are the business types a bit different — print studio, photo lab, photos on canvas, artist, etc — but the audiences will always be different.

Point being that just because a test comes out a certain way in the case study does not mean it will come out the same way for you.

If there is any take away from this post I sincerely hope that it will be to show how easy it is to test and that you should always be testing — assuming you have decent traffic to start with.

In Case Study #01 – Testimonials Increase Conversion Rates we introduced our new #casestudy feature and how it would work going forward. That was a few weeks ago now though so let’s refresh the details on the test.

Profile:

Custom Canvas Prints is a Utah based print studio hosted on the Art Storefronts platform. They offer canvas and paper prints to users nation wide. They are a shop that prides itself on their customer service and they have the testimonials to back it up.

Idea to Test:

A testimonial placed on the upload to print page will increase overall conversions via the upload to print tool.

Test Setup:

First this tests assumes you have google analytics running on your site. If not read How to Install Google Analytics.

Version A
The current version of the upload to print page. As you will notice, no testimonial.

Version B

We have added a testimonial directly below the upload to print tool.

Per our last post we want the #casestudy feature of the blog to report on results but also to give you the step by step directions should you want to run your own test on your site. So the rest of the post will not only detail the results but also give you step by step directions on how to perform your own test.

To add the testimonial down there we started with a blockquote and gave it some custom css to standout. (many ways you can do this. What will follow is the EXACT code we used in the test). You can copy and use it verbatim should you see fit.

Starting with a blockquote

1. Navigate to your upload to print page
2. Locate the “Ending content – store page”

1. Paste the code below into the html editor. To do that you need to use the source code button (replace the testimonial text with your own testimonial)

   
   Exact source code below…

   <div class=”testimonial”>
   <h2 style=”text-align: center;”>&nbsp;</h2><blockquote>I never, ever write back to a company but I just wanted to say thank you so much! The canvas is amazing. It is exactly how I thought it would be. I can&#39;t wait to show it off at my shows. Thanks again!<br />
   <span style=”color:#c233aa;”><em>Dee</em></span></blockquote>
   </div>

2. Save your changes

Customize your CSS

1. Navigate to themes
2. Your theme
3. Edit theme
4. Locate CSS editor
5. Paste the exact code snippet into the editor

   

   here is the exact code snippet

   /* Custom code for testimonials on pricing page case study #1 */

   .testimonial {
   padding-top: 30px;
   }

   .testimonial h2 {margin-bottom: 40px;}

   blockquote {
   border-left: 1px solid #c233aa;
   }

6. Save your changes

What you now have is a nice and clean looking blockquote with your testimonial.

Measurement

So let’s go over how we are going to measure for this test.

If the idea is that the testimonial, placed below the upload to print tool is gonna increase visitors uploading then we are going to need our measurement parameters.

So what we want to measure is the # of unique visitors — if you come to the site once, twice, or even ten times you are only recorded as 1 visitor — (GA calls this users) that reach the upload to print page and then complete the upload. We then compare the version A vs version B and we see who wins.

With me so far?

Ok good.

I know what some of you are saying.

If you are running an A/B test arent you supposed to use a fancy software like Optimizely or Visual Website Optimizer or at least the Google’s Analyitcs Content Expirament feature?

Programs that will serve version A and version B to web visitors in equal measure thereby giving you a more statistically accurate report?

Short answer yes, longer answer no.

In future tests we are going to roll out those tools. For this one I wanted to keep it simple and go old fashioned style. Which, while perhaps not as scientific as using the tools above, still qualifies as a solid way to A/B test.

So lets get back to the setup.

To measure completion of this cropping we are going to need a goal that records when users hit the second page of the upload to print process.

Don’t have that goal setup? Its ok. We have a post detailing how to do it here.

With that goal is place we are going to measure how many unique visitors hit the upload to print page and continue on to the second step vs users that don’t. We then compare those two numbers to see who wins.

How do you know when the results are statiscally significant?

Good question.

If we used the fancy tools outlined above that tech would be built in. We are going the free and easy route so we are going to have to bolt on a tool. In this case Hubspots A/B test calculator.

It will allow us to input the results of A vs B and tell us when we have a clear winner.

Lets take a look at what this will all look like in Google Analytics (GA).

We are looking at users which are unique visitors to your website.

Users – remember are unique visitors – record page views as “unique page views”

So unique page views – in GA Behavior, Site Content, All Pages + the url of your upload to print page. In this case you can see the url is create-a-canvas as seen in the image below.

We are then going to measure how many goal completions we received for the upload to print goal.

Out of respect for the site owner I have blocked out some of the data on this page. What you do see is the # of cropping goals that were completed so you know what we are measuring.

Now lets get to the fun part…

Results

Remember the earlier quote about sharpening the axe? Here comes the part where I should of done my homework.

When I originally devised this test I did so based on past experiences in which testimonials really worked well when optimizing for conversions.

In this case I should of paid more attention to the data before I started ( I was eager to get our first case study going, it’s my bad)

Let me qualify that…

So I designed the test, setup a cropping goal on their site, and let the data gather. This was my WOW moment of this particular test.

The data

Version A – no testimonial
Unique Visitors that reached the cropping page: 722
Visitors that completed the cropping goal: 411
Conversion rate: 57%

You mean to tell me that nearly 6/10 people that hit that page uploaded an image?

Yes.

Which is absolutely insane. When is the last time you remember 6/10 people doing anything you wanted them to do. What a #!!!!

More importantly, in the world of internet marketing, when you see a # that high you quickly realize there are more important areas to focus on. If it ain’t broke don’t fix it right?

That being said we tested anyway.

So lets see how we did.

Version B – w/ testimonial
Unique Visitors that reached the cropping page: 752
Visitors that completed the cropping goal: 400
Conversion rate: 53%

Lets run the #’s through the A/B calculator

So with 92% certainity we can call the test complete. A testimonial placed on the upload to print page on the Custom Canvs Prints site decreased conversions in this case. The testimonial has since been taken down.

Conclusions

So this test was a loser. What a waste right?

Absolutely not.

By running this test we learned some amazing things that we would not of been aware of without running the test and diving into the numbers.

First, that in this particular case the upload to print tool on its own converts at an INSANE 57%.

When we look down the line further and take a look at how many of those visitors that ended up uploading an image and then either converting into a sale or not converting into a sale is when we find the amazing opportunity.

Let me explain it like this with #’s from the test.

So out of the 722 users that reached the upload tool 411 converted to the second page of the process. Ok so how many of those folks ended up ordering a print?

If we assume an average conversion rate of say 2.5% (pretty standard for an e-commerce site) then we get 11/411 bought prints or 2.5%.

That leaves us with 400 people during this particular window that did not end up ordering.

That is what we call an amazing opportunity!!!!

That’s 400 people we can now start thinking about how we are going to turn into customers going forward. In technical eCommerce terms this is called cart abandonment.

Immediately I thought of how to do that…

• What about Facebook ad campaign that attempts to bring them back with a coupon code
• What about an exit popup — a pop up that only appears when you are about to leave the page — that offers a coupon code if they order
• What about an exit pop up that asks for their email address ( read you can market to them in the future) and then send them a coupon code to get them to order

That’s just a few ideas and most importantly, they were generated as a direct result of running that first test and realizing what the numbers are.

That’s the whole key with testing in the first place and why this test is still a win overall.

You run a test and you win or lose. But you gain insights into your sites behavior and you get new ideas to test. Rinse and repeat.

In this case the minute I saw the data coming in I realized what an incredible opportunity there was here and set in motion the future tests already.

So in a future post I will report back on those results.

A parting thought and my two ¢…

57% is insane. When we saw this # internally here at ASF I would be lying if I said we didn’t go out that night for beers to celebrate. It’s just an incredible # and it lets us know we are on the right track with the upload tool.

The next day though we came back into the office laser focused on all those visitors that didn’t end up ordering.

How we are going to improve the entire process?

How are we going to incorporate new features and tools to convert those visitors?

What tests can we start running on the software to improve things?

Here at ASF we practice what we preach.

As a result of this case study we now have our own internal list of ideas to test. Ideas we have already started testing. Tests that are going to lead to improvements in the software. Improvements that are going to go live soon that are going to have your business operating at a higher clip than it was previously.

Rinse and repeat from there.

All of which really gets me fired up for these case study posts.

You get better, we get better, we all win.

Lets get testing…

Upcoming Case Studies

Interested in having a case study run on your site? Contact us to participate.