| Steps | Description |
| Step 1: Formation of a project team | When carrying out A / B testing, you can’t just rely on the A / B testing tool (we’ll go into that later). Someone who knows how to optimize conversions must also be involved in the test. So you need people for your team who understand the following topics: Excellent knowledge of data analysis Understanding of identifying conversion problems Understanding of user behavior |
| Step 2: Prioritize the tests | If you have discovered conversion problems, you have to rank them according to their priority. Keep a precise schedule of the testing and the schedule. You proceed according to the following criteria: Estimated benefit Traffic volume Easy to use |
| Step 3: running the tests | Depending on the type of test planned, there are different modes of operation and A / B test solutions. Sometimes a test can be very simple, but sometimes it can be very complex. If it is too complex and technically too difficult, you can outsource the A / B testing to a service provider. |
| Step 4: Evaluation of the tests | The purpose of evaluating the tests is to ensure that they are reliable, that statistical methods are used in the evaluation, and that best practices are used. |
| Step 5: Document the execution of the tests | In order to be able to benefit from the results of the tests later, you have to precisely document all tests carried out and also keep them for later measures. You have to record the following data in writing: What is the name of the test carried out? In what period did the test take place? Which hypothesis was tested? Which variants did you use for this? What are the results and conclusions of the test? What is the potential financial gain for a year? |
| Step 6: introducing the best version | The version that has proven to be the best version in A / B testing must be introduced. As a transitional solution, the user usually sees the best version until it can finally go online. After the introduction, you need to check whether the results from the test are still confirmed. |
| Step 7: Announce test results | Of course, depending on the size of your company, you have to make the results known to decision-makers and those responsible. There are results that are important for sales, marketing, etc. |
| Step 8: permanent tests | You can’t see A / B testing as a one-off test. This is an optimization process that you have to apply continuously. |
What types of A / B tests are there?
According to eshaoxing.info, there are different types of A / B tests for different sites.
- The classic A / B test: The classic A / B test is designed to show the user or visitor of your site two or more variants. But this always happens with the same URL. So you can measure certain elements for their success.
- Split test / redirect test: In the split test, your traffic is redirected to another or several other URLs. This test is especially interesting for you if you host new pages on your server.
- Multivariate test: This test is also called MVT for short. With this test you can impact the effects of multiple changed elements on the same page. If you want, you can change the design, change a banner or change the font color or font. The result of the MVT shows you which change is most popular.
How do you correctly evaluate the results of A / B testing?
The evaluation of the results is of course of subtle importance. At least one interface for reporting should be available for the AB testing tool used. Here you should display the following results:
- Conversions per version
- Conversion rate
- Improvement in percentage form compared to the original version
- Statistical reliability of each version tested
- It would also be ideal to illuminate the results in segments (e.g. traffic source; customer type, etc.)
Deriving goals
You have to derive goals from the results and evaluations of your AB test. There are two types of goals.
- Primary Goals: These goals are all that motivated you to take an Ab test. This includes orders, subscriptions or purchases.
- Secondary goals: These goals give you an idea of how user behavior is presented. For example, bounce rate, length of stay, etc. The secondary goals help you to optimize the conversion rate.
What tools can help you with AB testing?
There are many AB testing tools available on the market. At this point we would like to briefly introduce you to the three most important tools.
| Name of the tool | Description |
| AB Tasty | This tool is the best known and also the one that is best recommended. With AB Tasty you are able to do all kinds of tests. The tool consists of the modules testing, personalization and customer engagement. Each of these modules is available to you as a building block for native apps, websites and responsive pages. The entry-level version of AB Tasty is available from 199 euros. |
| Adobe Target | This tool is part of Adobe’s Marketing Cloud and represents an enterprise solution. You can get this tool as a self- or full-service solution. Setting up tests is no problem thanks to the step-by-step instructions. You can run tests for websites, apps, or mobile sites. The prices for the tool are only available on request. |
| Convert experiments | With this tool you can not only do AB testing, but also split url tests and multivariate tests. Here, too, there are step-by-step instructions that will help you set up your tests correctly even without technical support or IT. The basic version is available for you from 69 dollars. |
How can you find ideas for your A / B test?
To carry out an AB test, of course, you need ideas, you have to make a hypothesis. Only with additional information will you be able to understand the behavior of the users and identify the problems with the conversion. The AB tools listed and described above can help you to find the right idea or a strong hypothesis for your AB testing.
But you have to observe the following rules:
- Your idea / hypothesis must have to do with a problem that you clearly identify. You have to guess the cause of this problem.
- Your idea must indicate the expected result. It has to be directly related to the KPI you want to measure.
- Your idea / hypothesis should already contain a possible solution to the existing problem.
Example: You have identified a high abandonment rate on the registration form for your newsletter. This form seems to be too long for you. Your idea / hypothesis could therefore be: Shortening the form by removing optional fields and thus increasing the number of registrations.
