Sites running an A/B testing tool are good on LCP just 58.5% of the time on mobile versus 68.7% without, a 10-point loading penalty and the steepest of any third-party category we measured.
Key Findings
- 1.A/B testing tools carry the largest loading penalty in our third-party data. Good-LCP falls from 68.7% to 58.5% on mobile, a 10.2-point drop.
- 2.The desktop hit is nearly as large: good-LCP 72.7% versus 80.8% (down 8.1 points), and pass rate 51.6% versus 55.9%.
- 3.Overall mobile pass is only slightly lower (41.5% versus 42.4%). The loading penalty is partly masked because A/B tools skew toward larger, better-resourced sites.
- 4.Responsiveness is barely affected (good-INP 85.8% versus 84.7%), and stability dips modestly. This is mostly a loading problem.
- 5.The mechanism is well understood. Many A/B tools inject a synchronous "anti-flicker" snippet that hides the page until the experiment loads, directly delaying the largest contentful paint.
- 6.A/B testing runs on about 3.4% of the mobile web (329,650 origins), a smaller footprint than chat or tag managers, but the steepest per-site cost.
Summary
Client-side A/B testing tools sit on roughly 330,000 mobile origins, about 3.4% of the web, and they inflict the harshest loading penalty of any third-party category in this research. Sites running an experimentation platform are good on Largest Contentful Paint just 58.5% of the time on mobile versus 68.7% without, a 10.2-point gap that beats live chat, tag managers, and cookie banners for sheer loading cost. The mechanism is not the platform itself but the anti-flicker snippet most teams paste in without a second thought.
The irony writes itself: the tool paid for to lift conversions is dragging the page speed that drives conversion in the first place. Google's field research puts most mobile abandonment on pages slower than three seconds to first paint, and slower pages convert worse across every category. A winning variant that arrives after the visitor has already bounced is not a win.
How this study was built
This study joins two Google-scale datasets. HTTP Archive's technology fingerprinting flags which of 9.58 million mobile origins are running an A/B testing or experimentation tool, and the Chrome User Experience Report contributes the Core Web Vitals those origins served to real Chrome users on real devices and connections. That is field data, the same input Google uses for search ranking, not a synthetic lab measurement.
The Core Web Vitals pass bar is Google's own: an origin only passes when the 75th percentile of its mobile traffic simultaneously clears Largest Contentful Paint under 2.5 seconds, Interaction to Next Paint under 200 milliseconds, and Cumulative Layout Shift under 0.1. Hitting all three at once is a demanding standard that a site usually meets only when loading, responsiveness, and visual stability are all in shape together.
What an A/B testing tool actually costs
Client-side experimentation platforms let teams test variations of a page against each other. To do that without visitors seeing the original page flash before the variant loads, most inject an anti-flicker snippet: a small piece of synchronous JavaScript that hides the page until the experiment framework is ready. That technique solves a visual problem and creates a performance one, and the field data shows exactly how large it is.
Sites running an A/B testing tool are good on Largest Contentful Paint just 58.5% of the time on mobile, versus 68.7% without, a 10.2-point loading penalty. That is the largest LCP gap of any third-party category in our research, ahead of live chat, tag managers, and cookie banners. When you deliberately hide the page until a script loads, you delay the exact moment Core Web Vitals measures.
Source: PageSpeed Matters analysis of 9.5 million origins with CrUX field data, mobile 75th percentile, mid-2026. Bars scaled to the largest penalty.
A/B testing sits at the top of the list. The anti-flicker snippet is the reason its loading cost outruns every other named tool.
The effect is almost purely on loading
Responsiveness, at 85.8% versus 84.7% good INP, is essentially unchanged, and stability dips modestly. The overall mobile pass rate barely moves either, 41.5% with A/B tools versus 42.4% without. Experimentation tools cluster on larger, better-resourced sites, e-commerce, SaaS, and media, that score above average on other dimensions, so the loading penalty eats into that head start rather than dragging an average site under the line. The clean, tool-attributable number is the LCP hit, and it is steep. A/B tools also have the smallest footprint of the categories we measured, on about 3.4% of the mobile web, roughly 330,000 origins, so this is a concentrated cost on a specific set of ambitious sites rather than a web-wide one.
Source: PageSpeed Matters, mobile 75th-percentile field data, mid-2026.
Overall pass barely moves because A/B tools live on stronger sites. The loading damage is real; the site quality masks it.
The anti-flicker snippet is the lever
The single most useful thing a team can do here is reconsider the anti-flicker snippet. It is often optional or tunable: many platforms let you shorten the hide timeout, load the experiment framework asynchronously, or run experiments server-side so nothing blocks rendering on the client at all. Server-side and edge experimentation move the test off the critical path entirely and remove the LCP penalty. If client-side testing is a requirement, a tight timeout and async loading limit the damage. It is worth weighing alongside the rest of your third-party JavaScript.
| Metric (mobile) | With A/B tool | Without |
|---|---|---|
| CWV pass | 41.5% | 42.4% |
| Good LCP | 58.5% | 68.7% |
| Good INP | 85.8% | 84.7% |
| Good CLS | 84.9% | 88.3% |
| Origins | 329,650 | 9,245,640 |
On desktop the loading hit is nearly as steep: good-LCP 72.7% versus 80.8%, and pass 51.6% versus 55.9%.
What to change if you run experiments
The number to act on is the 10.2-point loading gap, because one configurable thing causes it. Sites running an A/B tool are good on LCP just 58.5% of the time on mobile against 68.7% without, and the anti-flicker snippet that hides the page until the experiment loads is what opens that gap. It is the steepest loading cost of any third-party category we measured, ahead of live chat at 9.5 points and tag managers at 5.3. INP and CLS barely move, 85.8% good INP versus 84.7%, so this is a loading problem with a loading fix.
Move the experiment off the critical path. Running tests server-side or at the edge removes the anti-flicker snippet entirely, and if testing has to stay client-side, a short hide timeout plus async loading of the framework keeps the page from being hidden for long. The overall pass rate only slips from 42.4% to 41.5% because A/B tools cluster on larger e-commerce and SaaS sites that were already passing on their other metrics, so for those teams the LCP penalty is eating a head start they had paid for. Recover it and you keep the experiment and the fast page, which is the only version of a test that lifts the conversions it was built to raise.