The right way to A/B test

A/B testing helps you test ideas on your retail eCommerce site. From the results you can decide if the idea has worked and how much extra revenue it has generated. If no more revenue was generated then you can keep on trying until you find one that works. Simple?

Not really, when you start looking at the data that is generated and examine the different shopper segmentation groups, you may find that result are lower or greater then expected. Why is this? It is because of who the shoppers are, where they have come from and which segmentation group they were put in for the A/B test. For example, you may have a simple A/B test that splits all your shoppers in half, so 50% go to test A and 50% go to test B. How do you ensure that the 50% that go to test A have not all come from affiliate feeds or a PPC advert, where they will naturally have a higher conversion rate and thus generate more revenue?

Now to solve this you could heavily tag the different tests and do deep drive analysis using either Coremetrics or Omniture.

Or, you take the simple route that I recently took, when trialling a product, of pre-segmentation before the A/B tests start. This is very similar to A/A testing but can work on a lot more tests, A/B/C/D/etc..
Before you start your test, segment your shoppers into their test groups. So if you are testing 2 different ideas, you would need 3 test groups, the last being the control group that just sees the normal site. These pre-segmentations should run for half the time of your test; of course as you are not actually running the test, each of your segment groups will just be seeing the normal site. If you can, and if your A/B testing suite allows it, make your shoppers sticky to their segment.
Now that your shopper are pre-segmented you can monitor the difference between your segments, this will be the noise that can get deducted from your final test results.
You can now run your test with the assurance that you have statistically relevant data.

BTW. If you are already doing an A/B test and have not done the above, you can always run it after the test.

- Posted usingt BlogPres from my iPhone

Facebook vs Google

Now that Facebook has over taken Google as the most visited site in the US, all UK retailers should be looking at different ways to Market their retail eCommerce site on Facebook. The reason for this is that the UK is never far behind the US with web trends.