Cursor_and_A_B_Testing__Compute_required_Sample_Size__A-Priori__-_Public_version_-_Google_Sheets

Compute required Sample Size for your A/B tests

Direct link to the Google Sheets Template Preview.

So if you want to get started with A/B testing, you should calculate up front if you have enough users and conversions to see if it is even possible to perform an A/B test on your site.

There are some general rules of thumb out there that say “don’t go A/B testing when you have < 1000 conversions”, which is fine, but probably not really helpful in your situation. It really depends on a lot of variables to know if you can run a specific test on your site (7 to be exact).

For simple, one-off calculations you can use Evan Millers website (http://www.evanmiller.org/ab-testing/sample-size.html) and it’ll quickly tell you the sample size for your website:

Sample_Size_Calculator__Evan_s_Awesome_A_B_Tools_

If your baseline is 5% conversion, you want to detect a 10% relative change, have 90% power with a 5% significance level, you need over 40K users to hit each variation in your test.

If you know the number of variants you want to test, the duration of the test and lookup the number of users you have in that period of time you know if it’s feasible to even run the test.

This is a great tool to do a couple of calculations, but what if you have multiple websites and maybe multiple segments within those websites? What if you want to know to check the testability not only for transaction, but also for other (micro)conversions. And numbers change over time so you might want to recalculate or do a separate calculation for peak season

That is A LOT of manual calculations. I’m way too lazy for that.

So I spend many hours over the last days couple of days (yes, irony…) building a spreadsheet that does this automatically. And I created a Google Sheets Template out of it to share with you so you can use it too!

Overview of the testability of your websites:

Cursor_and_A_B_Testing__Compute_required_Sample_Size__A-Priori__-_Public_version_-_Google_Sheets

Detailed view of the numbers of one website:

Cursor_and_A_B_Testing__Compute_required_Sample_Size__A-Priori__-_Public_version_-_Google_Sheets

Go to the Google Sheets Template Preview.

It’s as simple as entering the data (use Supermetrics to automate that too), selecting the variables and poof you know for all websites or segments if you can run a reliable test there! There is a detailed instruction in the first tab of the sheet.

I also created an explainer video to take you through the different settings. Let me know if you have any questions/remarks!

181H

Website Optimization Workflow Model: 5 elements to WOW your boss and customers.

Website optimization is not just about installing an A/B testing tool, creating some (random) tests and hope for the better. It’s one element of a testing process that builds on consistent gathering and testing knowledge about your customers and implementing all elements can seem a daunting task.

Last year I created a workflow to highlight the moving parts involved with website optimization to explain the process and resources needed and why it can take some time to implement properly. I used it in a public presentation (slide 29) as an illustration of the complexity of the testing process and how it was implemented. Read more

Future_of_Magento

The Future of the Magento Ecosystem

The Magento community is thriving and wants to push forward fast. This presentation is about feelings and ideas that live within the community and that I’ve discussed with many at Meet Magento conferences and on IM chats in the past months. I presented this at the Magento Snowcamp in Austria in January and this week at Meet Magento in Spain. Let me know what you think in the comments! Read more

Image by SimpleIllustrations

Checkout optimization 2/7: Layout

Image by SimpleIllustrationsCC by-nc-nd

This week I’ll be exploring the layout of the checkout process which is the second blog in this series. if you missed the first, checkout the Checkout Optimization Introduction post.

The way webpages are structured is of great importance to user experience. And on pages where getting to the end of a process like the checkout, it’s key to your shops succes. Here are some guidelines related to the layout and other visual elements on checkout pages.

Clear error messages

Make it crystal clear to your website visitors when they make a mistake and what can be done to fix the problem (without the use of technical jargon). The erroe message should be clear (high visual contrast) and placed near the error (and not only on top of the page). De notification should be clear and short.

Form error messages

Read more

checkout optimization

Checkout optimization 1/7: Introduction

Deploying an e-commerce platform is no easy task. Creating a logical checkout process that doesn’t confuse your users might be an even more daunting task. Checkouts can be overly complex thus requiring great focus and dedication from the user to complete it. This of course hurts your conversion and your turnover. Following some easy guidelines can boost your conversions greatly.

In this series of 7 blog posts I’ll show you how you can make your e-commerce checkout process easier for the user and a lot more profitable for you as a shop owner. And it’s not just me making things up, it’s actually based on research ;).

Part 1/7: Series Introduction

Amazon Shopping Cart

Most shops don’t lack visitors and many have beautiful designs. Usability of the default home-, category-, product- and searchpages are on an acceptable level and you’ve applied many of the online persuasion tips to get people to buy your product or service. Users visit your product pages and many of them add your awesome items in their online shopping basket.

SCORE! Another day with lot’s of products sold and customers added to your CRM!

Or not (yet)…? Read more

review_picture

Embarassing fact: Magento 2.0 was announced 3 years ago today

At the Magento Developers Paradise in 2010, then CTO Yoav Kutner announced Magento 2.0. Later that year, Magento announced that we could expect a stable release somewhere in Q4 2011.

Today, 3 years after that first announcement, we haven’t even seen any sign of a beta release.

At the time, the developers already realised that Magento needed a major code overhaul to support the growth of its customers and to keep up with the fast growing e-commerce world. So instead of building upon the 1.0 codebase, they decided to start from scratch to have maximum flexibility and to make development much faster.

Oh, the irony.

In the fast-paced e-commerce world that’s, that’s (very) embarrassing. If eBay doesn’t get its act together, Magento customers will fall behind the competition and switch platforms.

The original announcement can be viewed on YouTube.

Meet_Magento_DE_2013_-_Guido_Jansen_-_YouTube

Video: Online Persuasion @ Meet Magento Germany

Here’s the video from me at Meet Magento in Leipzig, Germany last June. I presented my Online Persuasion session and the video was published this week at nr-apps.com.

In addition to all the fancy persuasion techniques I also added some additional thoughts about how to optimize your website…

A) Prioritize and get a decent basis, or you’ll waste money with persuasion. Improve your website in the following order:

  1. Technology: it should work. No errors.
  2. Functionality: it should offer the functionality your customer wants. The most screen estate should go to the most important tasks for the customer.
  3. Usability: If it works and does whát the customer wants, make it as easy as you can. Remove the bumps in the road.
  4. Persuasion: Now that it’s technically and functionally possible and easy to do, motivate your customers to act.

Now I don’t say that step 1 to 3 should be perfect in order to start with persuasion. But there is no use in persuading customers to do something that has technical, functional or usability issues, that only leads to frustration.

B) Your gut feeling sucks. So does your HIPPO. We have customers that either test a lot or really everything they change on their site. Their optimization teams are really skilled and experienced in A/B testing and are awesome ‘customer intelligence teams’ that are invaluable to their business. This is because they fail as much as they possibly can. And even with all that experience, predicting the success of an A/B test is just above chance. But they don’t need to predict, after a few weeks testing, they KNOW what works best. So guess how good the gut feeling of your marketer or the HIPPO (Highest Paid Person’s Opinion) at your department is…

C) The ‘golden egg’ of persuasion exists. But it’s not what you think it is. People often look for a ‘golden egg’: something that will solve all their problems and sell their services like a madman. It does exist, but it’s different for every customer, every product, every brand and a plethora of external factors you have no control over. So the key is not to find one golden egg for every customer in every situation, they key is finding out what influences your customers behavior and adapt your communication accordingly. This means you should be testing what you do to see what works best for which customer segment and don’t forget to include external factors like weather conditions, competitor pricing or marketing actions or just the difference between a working day or the weekend.

In the end, you’ll end up with a basket full of eggs. The color depends on your team and testing processes ;) .

If you want to see my session live and/or talk to me about it: my next events are at TeCOMM in Romania (October 24th) and Meet Magento Poland (November 4/5th).