Reconciling Lighthouse Scores with Field Data

“Lighthouse says 96 and our LCP is failing” is one of the most common performance conversations, and it is almost never a measurement error. The two numbers describe different things: one is a single scripted load on a simulated device, the other is the 75th percentile of every real load over 28 days. This guide sits under Synthetic vs Field Data Trade-offs and gives you the six structural differences that produce the gap, a checklist for closing the ones that are closable, and the residual size that indicates a genuine problem.

The goal is not to make the numbers match. It is to know how large the expected gap is, so that when it changes you know something real happened.

Prerequisites

• A Lighthouse run you can reproduce — same URL, same configuration, same number of runs.
• Field data for the same URL or template, aggregated at p75 over a 28-day window.
• The ability to segment your field data by device class and connection, as covered in Device & Network Segmentation.

How to reconcile them

Step 1 — Understand what each number is

The last row alone accounts for a large part of the gap. A median run and a 75th percentile of the same distribution are different by construction; comparing them and expecting agreement is a category error.

Step 2 — Close the closable differences, in order

Each of these makes the two numbers more comparable. Work down the list and re-check the gap after each.

1. Match the population. Filter field data to mobile only, since Lighthouse mobile simulates a phone. This is usually the largest single correction.
2. Match the cache state. Compare against first-visit sessions if you can identify them, since Lighthouse always loads cold.
3. Match the geography. A Lighthouse run from a CI runner in one region has a TTFB nothing like your global traffic. Run it from a region representative of your users, or compare against that region’s field data.
4. Match the statistic. Take the median of at least five Lighthouse runs — a single run has substantial variance — and compare that against your field median as well as the p75.
5. Match the URL. Lighthouse loads one URL; your field data covers a template. A product page for a cached, popular item behaves differently from a rarely visited one.

Step 3 — Account for what cannot be matched

Three differences remain after every correction, and they are the reason a residual gap is normal.

Real users interact. LCP stops growing at the first scroll or click. Lighthouse never interacts, so it always sees the full candidate stream. On pages where the largest element paints late, Lighthouse reports a larger LCP than users experience — the gap can run in either direction.

Real networks are not simulated throttling. Lighthouse’s simulated throttle applies a model to an unthrottled trace. It is remarkably good at approximating a slow connection and it is still a model: packet loss, radio wake-up latency and congestion do not appear in it.

Real devices are a distribution, not a point. The simulated mid-tier device sits somewhere in the middle of your traffic. Your p75 is drawn from the slower quarter of it, which by definition includes devices worse than the simulation.

Step 4 — Decide what the residual gap means

The third row is the one worth acting on, and there is a short list of causes. Personalised content that only real sessions receive, third-party scripts that a CI runner blocks or that fail to load from the test region, an A/B test the lab run never enters, and consent banners that only appear for real users are, in that order, the most common.

Step 5 — Use each tool for what it is good at

Once the relationship is understood, the argument disappears and the division of labour is obvious.

// lighthouserc.js — gate the build on a lab budget derived from field data,
// not on the Lighthouse score, which mixes weighted audits.
module.exports = {
  ci: {
    collect: { numberOfRuns: 5, url: ['https://staging.example.com/products/example'] },
    assert: {
      assertions: {
        // Budgets set at the lab-equivalent of the field target, using the
        // measured lab-to-field ratio for this template (0.72 here).
        'largest-contentful-paint': ['error', { maxNumericValue: 1800 }],
        'total-blocking-time': ['error', { maxNumericValue: 300 }],
        'cumulative-layout-shift': ['error', { maxNumericValue: 0.05 }],
        // The overall score is deliberately not asserted on.
        'categories:performance': 'off',
      },
    },
  },
};

Gating on the composite score is the mistake this configuration avoids. The score is a weighted blend that moves for reasons unrelated to what users experience; the individual metric budgets are what map onto the field.

Verifying your reconciliation

1. The matched comparison agrees within about 15%. If it does not after all five corrections, one of the structural causes in step four applies.
2. The lab-to-field ratio is stable week to week. A stable ratio is what makes the lab number useful as an early signal; a ratio that wanders means the lab configuration is drifting.
3. A deliberate regression moves both. Ship something knowingly heavy to a staging environment and confirm the lab catches it. If it does not, the lab run is not exercising the affected path.
4. The field number responds to a shipped fix within days, allowing for the cache-state mix, while CrUX takes four weeks — the reason is in CrUX & Public Field Data.

Writing the reconciliation down so it stops recurring

This conversation repeats every quarter unless the outcome is recorded somewhere the next person will find it. A short document per template, refreshed when the lab configuration changes, is enough.

What it should contain:

The measured lab-to-field ratio, per metric and per template, with the date it was measured. “Lab LCP runs at 0.72× the mobile field p75 on the product template” is the sentence that ends most future arguments.

The lab configuration it applies to — Lighthouse version, throttling profile, runner location and number of runs. Change any of those and the ratio is invalid.

The corrections applied to reach the comparison: device filter, statistic, cache state, geography. Someone will re-derive this from scratch otherwise, and they will get a different answer because they will apply different filters.

The residual gap and its accepted explanation. A documented 12% residual attributed to device mix is a settled question. An undocumented one is raised in every review.

The maintenance cost is roughly an hour a quarter, and it converts a recurring debate into a lookup. Where the ratio is used to derive CI budgets, as in the configuration above, keeping it current is not optional — a stale ratio produces budgets that either block every pull request or catch nothing.

Edge cases and gotchas

• Lighthouse variance is large. Single runs vary by 10–20% on the same page. Always take the median of five or more, and never compare two single runs.
• CI runners are noisy neighbours. Shared runners produce wildly variable results. A dedicated runner or a hosted measurement service is the difference between a usable gate and a flaky one.
• The performance score is not a metric. It is a weighted composite whose weights change between major versions, which means historical score comparisons across versions are meaningless.
• Blocked third parties. Corporate CI networks frequently block analytics and ad domains, so the lab page is materially lighter than the real one.
• Logged-in and personalised paths are usually untested. The lab tests the anonymous page; if most of your traffic is authenticated, the two are measuring different applications.
• Field data includes bots you did not filter. Automated traffic with unusual profiles can distort a p75 in either direction — filter it before reconciling.

FAQ

Why is my Lighthouse score 95 while my field LCP fails?

Because the score is a median-ish single run on a simulated device with a cold cache and no user interaction, and your field number is the 75th percentile across every real device and network. Match device, statistic and cache state before treating the difference as a defect.

Should I gate deploys on the Lighthouse score?

Gate on individual metric budgets instead. The composite score changes with audit weighting between versions and can move without any user-visible change.

How many Lighthouse runs are enough?

Five as a minimum, taking the median. Fewer than that and run-to-run variance dominates any change you are trying to detect.

The field data is better than the lab. Is something wrong?

Usually not. Real users interact, which stops LCP from growing, and many of them arrive with a warm cache. Both effects make the field number smaller than a cold, interaction-free lab run.

Related

• Synthetic vs Field Data Trade-offs — the wider framing of when to use each source.
• Building a Lab-to-Field Correlation Report — turning the ratio into a maintained artefact.
• CrUX & Public Field Data — the third dataset in the same argument.