Why Conversion Rate Alone is Misleading (And What to Measure Instead)

Simple messaging works. Single-minded messaging sticks. That’s why so many marketers bang the conversion drum… even when they know there’s far more to performance marketing.

Don’t get me wrong. Without conversions, there’s no business. But over-focusing on conversions can tank your business if you don’t factor in other essential performance metrics.

This came through loud and clear in the CXL Conversion Rate Optimization certification program that I finished earlier this year. CXL Institute is a recognized leader in conversion optimization, but they’re the first to admit: the term “CRO” oversimplifies what actually drives sustainable eCommerce growth.

“Conversion rate is one number — and it’s one number you can manipulate without making a cent more profit. It’s not the business goal; it’s just a proxy.” — CXL CRO for eCommerce Growth

Optimizing for that single metric (conversion rate) can lead you to celebrate supposed wins that actually harm revenue, margin, and lifetime customer value.

In this post, I’ll explain why overemphasizing conversion rates is a trap and what to do instead.

The CRO Trap

In the CXL course, CRO for eCommerce Growth, Carlos Trujillo, Senior Experimentation Strategist at Speero explains:

“If you set conversion rate as your North Star, you’ll inevitably run ‘successful’ tests that hurt the business. You can lift conversions by slashing prices, but the margin disappears. Or by removing high‑value upsells, but revenue per customer drops. The number went up — the business went down.”

CXL’s Testing Strategies course makes the same point from a testing perspective:

“Pick the wrong primary metric and you’ll get false positives — the test ‘works’ in analytics but fails in the bank account.”

And in CXL’s Statistics for A/B Testing, Georgi Georgiev warns:

“A test that increases conversion rate by 10% is meaningless if it increases return rates by 15%.”

And in A/B Testing Mastery, Ton Wesseling cautions against assuming that one winning experiment equals a lasting business benefit:

“One winning experiment does not prove the thing you were testing will work for the business. Metrics must tie directly to revenue, margin, or lifetime value.”

An example from e-commerce:

Imagine a fictional specialty coffee retailer, BeanCraft.

They test adding a complimentary 250g sample of a new single-origin roast to every order over $30. Conversions jump from 2.5% to 3.4%, and average order value edges up as customers add a little extra to hit the $30 threshold.

On paper, it’s a win. The plan was simple: the cost of the giveaway would be offset by the high lifetime value of new customers, who would come back again and again for BeanCraft’s signature blends.

But the offer drew in deal‑hunters and novelty‑seekers — customers who were never likely to reorder. Meanwhile, loyal specialty‑coffee drinkers, the ones who prize consistency and have strong preferences, disliked having “random” beans bundled into their carefully chosen order. Some skipped checkout entirely, opting to buy from a roaster that allows them to control every bean in the bag.

The result: a short‑term lift in conversions, but a drop in customer quality and loyalty. Worse, BeanCraft never recouped the cost of the promo because it didn’t lift LTV.

…

The takeaway? Conversion rate is useful—but on its own, it’s incomplete. Without tying experiments to metrics that reflect actual business health, you risk celebrating the wrong wins.

The Four‑Pillar Model for Growth

CXL explains that to achieve real, sustainable e-commerce growth, we need to pull all four levers:

CONVERSION RATE: The percentage of visitors who buy.
SPEND: How much each customer spends per transaction.
FREQUENCY: How often they return and purchase again.
MERCHANDISING: What you sell, how you present it, and the margins it generates.

Each lever influences the bottom line in different ways. The most effective optimization programs move all four in a focused, deliberate, and measured way.

1. Conversion rate

Conversion rate is often treated as the headline metric for e‑commerce, but CXL’s approach makes it clear: it’s just one part of the profitability equation, and it’s only meaningful when it’s measured correctly, interpreted in context, and improved in ways that align with your business model.

What is conversion rate optimization, and how is it done?

Conversion optimization is about increasing the ease and motivation for the right customers to say “yes” (opt-in or make a purchase).

Conversion optimization is not just removing every obstacle indiscriminately. As covered in CXL’s Good Practices and Digital Psychology & Behavioral Design, some friction is healthy if it filters for higher‑quality buyers. So, conversion optimization sometimes removes friction, but it always increases clarity, trust, and relevance—universal drivers of conversions.

Classic tactics to increase e-commerce conversions include:

Simplifying checkout flow and removing unnecessary form fields
Offering clear, benefit‑driven product descriptions and imagery
Differentiating from the competition with a clear USP, comparison features, etc.
Adding trust signals (reviews, guarantees, secure payment icons)
Using scarcity or urgency messaging to prompt immediate action
Introducing payment flexibility (BNPL, installments)

And remember, conversion rate without profitability is a vanity metric. The CXL framework always pairs conversion gains with companion metrics like average order value, margin, and payback period to ensure the lift is worth keeping.

2. Spend (Average Order Value)

There’s a cost to every transaction, especially the first purchase. Naturally, we ask, “How can we get each new customer to spend more in that same transaction?” How can you now lift average order value (AOV)?

“One of the fastest levers to pull is increasing spend per customer — bundling, cross-sells, and tiered discounts can outperform traffic growth for ROI.” — CRO for eCommerce Growth, Spend Lever Module

Classic tactics to increase spend include:

Creating threshold‑based incentives (free shipping, gift with purchase)
Using anchoring and tiered pricing to position higher‑tier options as value
Bundling complementary items to create complete sets or kits
Highlighting premium versions or add‑ons at checkout

There are countless tactics to try, but as per the coffee bean example above, some can backfire. The best way to “pre-test” an idea is to talk to your customers. (Get ideas from my Voice of Customer research post.)

3. Frequency (Retention)

How often do you need customers to return to recoup your acquisition costs and generate a profit?

It depends on your business model, of course. For brands selling higher-value products with strong margins, profitability often stems from fewer, high-ticket transactions. Brands built on lower‑ticket, consumable goods require higher purchase frequency to recover acquisition costs quickly and sustain revenue growth. Most e-commerce brands are in this latter category.

Take it from CXL’s People & Psychology course:

“Commitment and consistency are powerful for e-commerce — when customers commit to a subscription, even for a trial, the repeat rate climbs.”

Ok, but what metric do you use to gauge frequency?

Lifetime value (LTV) can tell you how much a customer might be worth over the long haul, but it’s slow to validate your strategy—you might need months or years of data to know for sure. That’s why CXL recommends tracking payback period alongside LTV.

Payback period focuses on how quickly each new customer earns back what you spent to acquire them.

“Payback period is a more actionable retention metric than LTV—it tells you how soon your acquisition spend is recovered.”
— CRO for eCommerce Growth

Classic tactics for increasing frequency include:

Launch subscription programs with flexible controls
Implement loyalty or rewards programs with achievable thresholds
Send targeted replenishment reminders based on usage cycles
Follow up first purchases with exclusive second‑order offers.

I’ll repeat what I said above: choose your strategies based on solid customer insight, an I’ll add: don’t stop listening. Collect customer feedback through multiple channels so you can discover how people are experiencing these nudges.

4. Merchandising

Finally, merchandising determines what you sell, how you position it, and how each product contributes to profit.

“Optimizing your catalog for margin—not just clicks—can mean demoting popular but low-profit SKUs in favor of high-margin alternatives.”— CRO for eCommerce Growth, Merchandising Lever Module

Merchandising can feel different from the other three levers. Conversion, Spend, and Frequency are outcomes you can measure directly. They answer the “how are we performing?” question.

Merchandising is less of a “performance metric” and more of a strategic mechanism for influencing the metrics. It’s about what you choose to sell and how you present it, which in turn influences those outcomes, particularly margin.

At its clearest, merchandising is about maximising profit per sale by steering demand toward products that are more appealing to customers and also more profitable for the business.

Classic tactics for profitable merchandising include:

Adjusting category sort order and homepage features to highlight profitable SKUs
Removing or demoting “toxic” products that erode trust or create returns
Focusing conversion optimization efforts on higher-margin products.

In the Four Pillar model, merchandising is the quiet multiplier—the lever that can raise margins, guide customers toward better choices, and subtly shape buying patterns without increasing traffic or discounting.

How to Apply the Four Levers in Testing

Once you understand the Four‑Pillar model, your testing strategy should shift from chasing generic “conversion lifts” to deliberately moving the lever (or levers) that matter most for your current business goals.

That starts with writing test hypotheses that are explicitly tied to a lever.

If you’re aiming to raise AOV, the hypothesis might be about bundling or price anchoring.
If you want to shorten the payback period, it might be about a subscription offer or a post‑purchase upsell that encourages a second order.
For merchandising, you might test promoting higher‑margin products or removing SKUs that erode trust.

From CXL’s Testing Strategies:

“Every change you make should support the same hypothesis tied to the business lever you want to move—conversions, AOV, repeat rate, or margin.”

This also means you can make multiple changes at once—as long as they’re all aligned with the same lever. In other words, you don’t need to test one tiny button color change if you’re confident about the strategic direction.

From A/B Testing Mastery:

“If you change 100 things at once, you can move the business fast—but only if all changes support the same lever.”

Example: A DTC apparel brand wants to grow margin without hurting conversion. They design a test that combines three changes: featuring higher‑margin products in the hero slot, adding comparison tables that highlight quality differences, and adjusting the default sort order to prioritize profitable SKUs. Because every change supports the same lever—merchandising—the test produces a clear, measurable lift in overall profit per visitor.

Caveat: While every experiment should be anchored to a primary lever, many tactics have the potential to influence multiple levers. A subscription program, for example, is a classic strategy to increase purchase frequency. Yet it can also nudge customers to spend a higher dollar value per delivery and introduce higher-margin products into the mix.

The key is to identify the primary lever you’re targeting so you can measure success meaningfully—and then track any secondary effects as added upside. This lever‑first approach ensures your testing stays focused without missing the broader gains a tactic can deliver.

Library of e-Commerce Growth Levers (Beyond Conversions)

Drawing from across the CXL Conversion Optimization program, I’ve created a master list of tactics you can use to pull these growth levers.

Conversion

Simplify checkout flow and remove unnecessary form fields (Spend)
Offer clear, benefit-driven product descriptions and high-quality imagery (Merchandising)
Add trust signals such as reviews, guarantees, and secure payment icons (Frequency)
Use scarcity or urgency messaging to prompt immediate action (Spend, Merchandising)
Provide payment flexibility such as buy now, pay later or installments (Spend)
Personalize on-site messaging and recommendations to match shopper intent (Spend, Frequency)

Spend (AOV)

Create threshold-based incentives such as free shipping or gift with purchase (Conversion)
Use anchoring and tiered pricing to position mid-tier options as best value (Merchandising)
Bundle complementary products to create complete sets or kits (Conversion, Merchandising)
Highlight premium versions or add-ons at checkout (Merchandising)
Offer post-purchase upsells to capture buying momentum (Conversion, Frequency)
Create time-limited bundles at a modest discount to drive urgency (Merchandising)

Frequency (Retention / Payback)

Launch subscription programs with flexible “pause or cancel anytime” controls (Spend)
Implement loyalty or rewards programs with achievable thresholds (Spend)
Send targeted replenishment reminders based on typical usage cycles (Conversion)
Follow up first purchases with exclusive second-order offers (Conversion)
Run seasonal or event-based campaigns to prompt off-cycle purchases (Merchandising)
Offer membership perks such as free shipping, VIP access, or early releases (Spend)
Recommend relevant products between purchases using browsing/order data (Spend, Merchandising)
Introduce product line extensions to give loyal customers new reasons to buy (Merchandising)
Add a low-cost, high-perceived-value gift to the customer’s next order (Spend)
Use gamification or progress tracking to encourage repeat purchases (Spend)
Offer time-limited discounts for early repeat purchases (Spend)

Merchandising (Profit per Sale)

Adjust category sort order and homepage features to highlight profitable SKUs (Spend)
Remove or demote “toxic” products that erode trust or cause high returns (Conversion)
Curate themed collections to simplify choice and inspire discovery (Conversion)
Rotate featured products to match seasonal demand or events (Frequency)
Use social proof-driven placement such as bestsellers or customer favourites (Conversion)
Cross-link related categories to promote complementary purchases (Spend)
Position premium products with benefit-driven copy, not just features (Spend)
Give prime placement to limited-edition or low-stock products (Conversion)
Test hero images, headlines, and CTAs for featured product slots (Conversion)
Use dynamic merchandising based on purchase history or browsing behaviour (Spend, Frequency)
Apply price anchoring in merchandising layouts (Spend)
Simplify category pages by reducing visual noise and filter complexity (Conversion)

Tactic	Primary Lever	Secondary Levers
Simplify checkout flow and remove unnecessary form fields	Conversion	Spend
Offer clear, benefit‑driven product descriptions and high‑quality imagery	Conversion	Merchandising
Add trust signals (reviews, guarantees, secure payment icons)	Conversion	Frequency
Use scarcity or urgency messaging to prompt immediate action	Conversion	Spend, Merchandising
Provide payment flexibility (buy now, pay later or installments)	Conversion	Spend
Personalize on‑site messaging and recommendations to match shopper intent	Conversion	Spend, Frequency
Create threshold‑based incentives (free shipping, gift with purchase)	Spend	Conversion
Use anchoring and tiered pricing to position mid‑tier options as best value	Spend	Merchandising
Bundle complementary products to create complete sets or kits	Spend	Conversion, Merchandising
Highlight premium versions or add‑ons at checkout	Spend	Merchandising
Offer post‑purchase upsells to capture buying momentum	Spend	Conversion, Frequency
Create time‑limited bundles at a modest discount to drive urgency	Spend	Merchandising
Launch subscription programs with flexible “pause or cancel anytime” controls	Frequency	Spend
Implement loyalty or rewards programs with achievable thresholds	Frequency	Spend
Send targeted replenishment reminders based on typical usage cycles	Frequency	Conversion
Follow up first purchases with exclusive second‑order offers	Frequency	Conversion
Run seasonal or event‑based campaigns to prompt off‑cycle purchases	Frequency	Merchandising
Offer membership perks (free shipping, VIP access, early releases)	Frequency	Spend
Recommend relevant products between purchases using browsing/order data	Frequency	Spend, Merchandising
Introduce product line extensions to give loyal customers new reasons to buy	Frequency	Merchandising
Add a low‑cost, high‑perceived‑value gift to the customer’s next order	Frequency	Spend
Use gamification or progress tracking to encourage repeat purchases	Frequency	Spend
Offer time‑limited discounts for early repeat purchases	Frequency	Spend
Adjust category sort order and homepage features to highlight profitable SKUs	Merchandising	Spend
Remove or demote “toxic” products that erode trust or cause high returns	Merchandising	Conversion
Curate themed collections to simplify choice and inspire discovery	Merchandising	Conversion
Rotate featured products to match seasonal demand or events	Merchandising	Frequency
Use social proof‑driven placement (bestsellers, customer favourites)	Merchandising	Conversion
Cross‑link related categories to promote complementary purchases	Merchandising	Spend
Position premium products with benefit‑driven copy, not just features	Merchandising	Spend
Give prime placement to limited‑edition or low‑stock products	Merchandising	Conversion
Test hero images, headlines, and CTAs for featured product slots	Merchandising	Conversion
Use dynamic merchandising based on purchase history or browsing behaviour	Merchandising	Spend, Frequency
Apply price anchoring in merchandising layouts	Merchandising	Spend
Simplify category pages by reducing visual noise and filter complexity	Merchandising	Conversion

Beyond Conversion Rate Optimization

Yes, conversion rate matters. But only if it also increases revenue and profitability over the long term.

It’s almost embarrassing to write because the most basic business course would make this clear. But in marketing, shiny objects can be a distraction from elemental truths.

Even more elemental is this: you must start with a deep, data-driven understanding of your buyers. When you have a mental map of their hopes, fears, desires, objections, and habits, you can make far smarter choices about how to move these levers.

If you’re ready to apply a data-driven approach to conversion optimization—backed by CXL Institute training—let’s talk. I help e‑commerce brands design and run tests that drive measurable, profitable growth. Reach out to book a free consultation.