Conversion rate optimisation: find the leak, size the test, and know what to do at low traffic

Conversion rate optimisation is a loop, not a list of tips: find where people leave, form one specific hypothesis about why, work out how much traffic a test of that size needs before you build it, run it for whole weeks, then write down the result whichever way it went. Conversion rate is simply conversions divided by interactions, so a test either moves that fraction by enough to detect or it does not. The arithmetic of sample size is what decides your whole method: with a 3% baseline, detecting a 20% relative improvement at 95% confidence and 80% power needs roughly 14,000 visitors per variant, while detecting a 50% improvement needs roughly 2,500. If your traffic is small, test bigger changes.

This page covers the method: finding leaks, sizing tests, and what to do at low traffic. For the ad side of landing page performance, see a paid search guide.

Where the leaks are

Research comes before tests. Every hour spent here saves a month of testing the wrong thing. Six sources, in the order they usually pay:

• Funnel reports. Build the funnel in GA4 for your real path and find the step with the biggest drop between two steps that should be easy.
• Form field analytics. Which field is abandoned, which is corrected most, how long the form takes. Forms are where the most recoverable money sits.
• Session recordings and click maps. Watch twenty sessions that reached the page and left. You are looking for confusion, not for a pattern yet.
• Site search queries. What people type into your own search is a list of things your navigation failed to offer.
• Sales and support notes. The objections a salesperson answers fifty times a week belong on the page.
• Speed and stability per template. Measure your key templates separately against Core Web Vitals. A layout that shifts under a thumb loses orders before anybody reads the copy.

Add one exit question on the pages that matter: "what stopped you today?" with a free text box. It produces better hypotheses than any heatmap.

Write the hypothesis properly

A testable hypothesis has four parts, and writing it this way stops the common argument about what the result meant.

• Because we observed something specific in the research.
• We expect this change, described precisely enough to build.
• To cause a named metric to move, in a named direction.
• Measured by one primary metric, declared before the test starts, plus guardrail metrics such as revenue per visitor and refund rate.

One primary metric per test. Picking the winner after the fact from a list of twelve metrics is how teams accumulate wins that never show up in revenue.

Sizing a test, which decides everything else

Sample size depends on four inputs: your current conversion rate, the smallest improvement worth detecting, your confidence level and your power. For two variants at 95% confidence and 80% power, the standard calculation is:

n per variant = 7.85 * [ p1(1-p1) + p2(1-p2) ] / (p2 - p1)^2

p1 = current conversion rate
p2 = the rate you want to be able to detect
7.85 = (1.96 + 0.84)^2, for 95% confidence and 80% power

Run it before you build anything. The numbers are usually sobering, and they are the most useful thing in CRO:

Current rate	Improvement to detect	Approximate visitors per variant
3%	20% relative, to 3.6%	14,000
3%	50% relative, to 4.5%	2,500
10%	20% relative, to 12%	3,800
10%	50% relative, to 15%	700

Three rules follow from that table. Higher baseline rates need less traffic, so test further down the funnel where rates are higher. Small improvements are expensive to prove, so stop proposing button colours. And the honest way to use a calculator is to decide the minimum detectable effect from the traffic you have, rather than hoping for significance on a change nobody would notice.

Two operational rules go with it. Run in whole weeks, because Tuesday and Saturday behave differently, and fix the end date in advance. Checking a running test every morning and stopping it the day it looks good is the single most common way a flat result gets reported as a win.

What to do when traffic is small

Most sites do not have the traffic for classical testing on a checkout page, and that is fine. The method changes rather than stops.

• Test bigger changes. A whole new page concept against the old one needs far less traffic to resolve than a headline tweak, because the effect you are looking for is larger.
• Move the measurement up the funnel. Measure a higher volume step such as reaching the form or starting the checkout, with the final sale as a guardrail.
• Test site wide, not page wide. A change to a template that appears on two hundred pages collects traffic two hundred times faster than a change to one page.
• Just fix what is broken. A form that fails on mobile, a missing price, a 4 second hero image. These do not need a test, they need a developer.
• Use qualitative research instead. Five people attempting your main task while talking aloud will find more than a month of underpowered testing.
• Run before and after periods carefully, comparing equal length windows, with seasonality and traffic mix checked, and treat the result as a strong indication rather than a measurement.

Tooling, since Google Optimize closed

Google Optimize and Optimize 360 are no longer available as of 30 September 2023, and Google says it is investing in third party A/B testing integrations for Google Analytics, naming AB Tasty, Optimizely and VWO as collaborations and publishing the integration APIs for others.

• Client side testing tools such as VWO, Optimizely, AB Tasty, Convert and Kameleoon are quickest to start with and need no engineering for simple tests.
• Server side and feature flag platforms such as Statsig, GrowthBook and LaunchDarkly avoid the flicker of client side changes and suit product teams who already ship behind flags.
• Research tools for recordings, heatmaps and form analytics, which are what actually generate hypotheses.

Pricing in this market depends on traffic volume and tested sessions, so compare on each vendor's own pricing page. Whatever you pick, check two things in the trial: that a test can be targeted to a page template rather than a URL list, and that results can be read against your own conversion definitions in GA4.

Judging a CRO agency

• Ask for a test log from another client, with the losers in it. An agency with only winners is either very lucky or not showing you everything.
• Ask how they size tests, and listen for baseline rate, minimum detectable effect and a fixed end date.
• Ask what research they do before the first test, and how much of month one is research.
• Ask who implements and who checks the implementation on mobile.
• Prefer a fee for a research and testing programme over a fee tied to a reported uplift percentage, because whoever defines the uplift controls the invoice.

What people ask about conversion rate optimisation

What is a good conversion rate?

Your own last six months, by traffic source and device. Google defines conversion rate as conversions divided by interactions, and that fraction moves with your traffic mix, so a published industry average tells you almost nothing about your page. Segment before you judge: brand search and paid social convert very differently.

How long should an A/B test run?

Until it reaches the sample size you calculated, rounded up to whole weeks, with the end date set before launch. If the calculation says longer than about six weeks, test a bigger change instead, because the market and the page will both have moved by then.

What does statistical significance actually mean here?

It means that a difference this large is unlikely to come from random variation alone, at the confidence level you chose. It says nothing about how large the real effect is. That is why you read the confidence interval, not only the significance flag: a significant result with an interval from plus 1% to plus 30% is a signal to look again, not a number to put in a forecast.

Can I test with fewer than a thousand visitors a month?

Use the research and fix route instead: usability sessions, form analytics, speed work, and shipping obvious improvements without a test. Save formal testing for template level changes where traffic pools across many pages.

Where should I start on a page that converts badly?

Four things, in order: load speed and layout stability on a mid range phone, whether the page answers the question the visitor arrived with in the first screen, how many form fields you are asking for, and whether the price or the next step is visible without hunting.

When the sample is small, read the conversations

Low traffic limits which tests you can run, but it does not limit what you can learn. One source of insight that needs no sample size is the question a prospect asked an assistant before arriving: what they wanted compared, what worry they raised, what they were told.

Those questions point at the objections your page has to meet. Write down ten that fit your offer and read the answers an assistant gives. Wherever the answer is wrong or unflattering about you, that is a page to fix, and the fix does not need a split test. On AI Knows Us you can see those answers side by side for every assistant.