How to Choose a CRO Agency

Choosing a CRO agency is harder than choosing most marketing suppliers, because the quality of the work is hidden inside research notes, test settings and statistics that few buyers are asked to read. Two proposals can both promise more enquiries from the same traffic while describing completely different levels of rigour. The difference often only shows once months of tests have produced nothing you can trust. The questions below separate a capable partner from a confident pitch. They apply to a single audit as much as to ongoing conversion rate optimisation services for UK businesses.

Each section covers one test of an agency, starting with whether your website is ready for testing at all and finishing with the warning signs that should end a conversation early. Working through them in order tends to shorten the shortlist quickly, because weaker proposals rarely survive a direct question about sample sizes or who owns the data.

Know What a CRO Agency Should Do for Your Website

A CRO agency should find out why visitors who could convert are leaving, then prove which changes fix that through measured tests. Conversion rate optimisation covers the research that finds the problems, the design and development of alternatives and the experiments that show whether those alternatives perform better than the original page.

In practice the work splits into a few connected strands. An agency reviews analytics to see where people drop out of a journey, watches behaviour recordings to see why, forms a hypothesis about a better version and then tests that version against the current page with real visitors. The winning change is built permanently into the website and the cycle starts again on the next problem.

Some agencies sell only one of those strands, such as an audit or testing on its own, while others run the whole programme. Decide which of those you need before you request proposals, since an audit followed by changes your own team builds is a very different purchase from a monthly testing programme run entirely by the agency.

Check Your Website Has Enough Traffic to Test

An agency should check that your website has enough traffic and conversions to produce reliable test results before it proposes a testing programme. A test on a quiet page can run for months without reaching a trustworthy answer, which wastes the budget and often ends with a winner declared on too little evidence.

GOV.UK guidance on A/B testing as a comparative study lists the need for many users before the data is statistically significant as one of the method’s drawbacks. It also advises basing the length of a test on the number of monthly visitors and the size of the change in behaviour you expect to see.

Many B2B websites have plenty of visitors but relatively few enquiries, so a test measured on completed enquiries may need far longer than one measured on clicks. A capable agency will say so plainly and suggest alternatives, such as testing on the highest traffic templates, measuring an earlier step in the journey or making research led fixes without a formal test where the evidence is already strong.

Ask How the Research Comes Before Any Test

Research should come before testing, so ask each agency what it will study in the first weeks and how those findings turn into a list of tests. An agency that starts by suggesting new button colours or headline ideas before it has seen your data is guessing, however confident the proposal sounds.

Good research draws on several sources and looks for problems that more than one of them confirms. The usual sources are listed below. A proposal should explain which of them the agency will use for your website and why.

• Analytics data showing where visitors enter, which pages they leave from and how each traffic source converts
• Heatmaps and session recordings that show where people click, how far they scroll and where they hesitate
• Form analytics showing which fields cause people to abandon an enquiry or checkout
• Feedback from surveys, sales teams or customer service about what buyers ask before they commit
• Reviews of speed, mobile layouts and accessibility that can stop visitors from completing a task

Forms deserve particular attention on lead generation websites, because a single unnecessary field can cost enquiries on every page that carries it. Our guide to improving B2B enquiry forms covers what to look for in that part of the research.

Ask how recordings handle personal data as well. Microsoft’s documentation on masking content in Clarity explains that the tool masks sensitive content by default and that anything typed into input boxes is masked in every mode. An agency should be able to tell you how its chosen tools are configured to the same standard.

Judge the Testing Approach and Statistical Confidence

A sound testing approach fixes the hypothesis, the main measure of success, the sample size and the confidence threshold before a test starts, then leaves the test alone until it reaches them. Ask each agency to walk you through a recent test in that order, because the answer shows quickly whether its winners can be trusted.

The most common fault is stopping a test early because the variant looks ahead. Results swing widely in the first days of any test, so an agency that checks daily and calls a winner the moment the numbers look good will report improvements that disappear once the change goes live.

Ask too how the agency protects search visibility while tests run. Google’s guidance on A/B testing and Google Search recommends rel=”canonical” links on variant URLs and temporary 302 redirects rather than permanent 301 redirects. It also warns that showing Googlebot different content from visitors counts as cloaking and recommends removing test scripts once a test ends.

Our guide to A/B testing for B2B websites covers which page elements are worth testing first. An agency that can explain its own method at that level of detail is far more likely to run tests you can rely on.

Ask What the Reporting Will Show

Reporting should show every test the agency has run, including those that lost or proved inconclusive, with enough detail for you to judge each result yourself. A report that only lists winners and headline uplifts tells you what the agency wants you to see rather than what the programme has learned.

For each test, the report should state the hypothesis, the page and audience it ran on, the sample size reached, the confidence level and the effect on the main measure. Results tied to enquiries, qualified leads or revenue are more useful than results measured on clicks, because a change can raise clicks while lowering the quality of what follows.

Priority Pixels gives every CRO client access to a reporting platform that holds every test run on their website, including control versus variant performance, sample size, statistical significance and the hypothesis tested. Tests that did not beat the original are shown alongside the wins, because both inform the next round.

Ask how often results are reviewed with you and who attends. A regular conversation about what is running, what is queued and what the last round taught is worth more than a monthly document nobody reads, because it keeps the programme tied to commercial priorities.

Look at the Tools and Who Owns the Accounts

The tools matter less than how they are set up and who owns them, so ask which platforms the agency uses and whose name each account will sit under. Most CRO programmes rely on the same few categories of tool. The questions worth asking are similar for each one.

Accounts for every one of these tools should be registered to your organisation, with the agency added as a user. When the agency owns the accounts, the test history and the recordings leave with it at the end of the contract, which means the next supplier starts from nothing.

Consent affects testing more than many proposals admit. The ICO’s guidance on cookies and similar technologies says you must tell people cookies are there, explain what they do and get their consent, with an exception for cookies that are strictly necessary for a service the user has asked for. Our overview of the CRO tool stack for UK businesses sets out how the main categories fit together.

Find Out Who Builds the Winning Changes

Ask who designs the test variants, who builds them and who makes the winning version permanent, because many CRO programmes stall at that last step. An agency that only runs tests can hand you a list of winners that your own developers then have to rebuild, which often takes months and loses the momentum the tests created.

Variants built inside a testing platform are usually laid over the existing page with a script. That works for a test, but leaving a winning change running through the testing tool indefinitely slows the page and makes the website harder to maintain, so the change should be rebuilt properly in the website itself.

Where the website runs on WordPress, that means building the change into the theme through bespoke WordPress development rather than leaving it in a script that loads over the page. Ask whether the agency has developers who work on the platform your website uses. If not, find out whether that work passes to another company or back to your own team.

Every variant should also meet the same accessibility standard as the rest of the website. The Web Content Accessibility Guidelines published by W3C set that standard. A variant that wins on enquiries while breaking keyboard access or colour contrast has created a new problem rather than solving one.

Read the Contract Terms Before You Sign

The contract should set out what the agency delivers, how long you are committed and what you keep if the relationship ends. CRO programmes take time to produce results, so a minimum term is reasonable, but the terms around it decide how much risk you carry if the work disappoints.

Payment linked to results sounds attractive but deserves care. A fee tied to uplift encourages an agency to declare winners early and to choose easy tests over important ones, so any performance element should be measured on results that reach the confidence threshold agreed at the start.

Priority Pixels treats CRO as an ongoing discipline rather than a project with an end date. Whoever you choose, the contract should reflect that the work continues in rounds, with each round agreed on the evidence the previous one produced.

Red Flags That Should End the Conversation

The clearest red flag is a guaranteed uplift, because no agency can know how your visitors will respond to a change before it has been tested. Real testing programmes produce losing and inconclusive tests as well as winners, so a promise of a fixed improvement means the agency either does not understand testing or does not intend to report every result.

Testing without enough traffic is the second warning sign. An agency that proposes a busy schedule of tests for a website with few conversions will either run tests that never finish or call winners on evidence too thin to trust.

Other warning signs tend to appear in the first meeting or the proposal itself. The comparison below sets the answers a capable agency gives against the answers that should make you look elsewhere.

The right agency will be comfortable with every question here, because a well run programme has nothing to hide at the proposal stage. One that avoids talking about sample sizes, cannot show you a losing test or resists putting ownership of the data in writing is already telling you how the programme is likely to go.