Building a Lab-to-Field Correlation Report

A lab run is only useful as an early warning if you know how its numbers translate into the field. That translation is a ratio per template per metric, it drifts, and almost nobody measures it — which is why CI budgets are usually guesses. This guide sits under Synthetic vs Field Data Trade-offs and covers how to build a report that joins scheduled lab runs to field percentiles, computes the ratio, tracks its stability, and emits budgets you can gate on.

Prerequisites

Scheduled synthetic runs on a fixed configuration — same tool version, same throttling, same location, at least five runs per URL.
Field data aggregated at p75 per template, with device and cache-state segmentation available.
A place to store a daily row per template and metric: a warehouse table is ideal, a CSV in version control is enough to start.
The reconciliation work from Reconciling Lighthouse Scores with Field Data already done once, so you know which filters make the comparison valid.

How to build the report

Step 1 — Fix the lab configuration and record it

The ratio is only meaningful relative to a configuration. Pin it, and store the configuration hash next to every row so a change invalidates the history rather than silently corrupting it.

// lab-config.mjs — one place, versioned, hashed into every result row.
import { createHash } from 'node:crypto';

export const LAB_CONFIG = {
  tool: 'lighthouse',
  version: '12.x',
  formFactor: 'mobile',
  throttling: { cpuSlowdownMultiplier: 4, rttMs: 150, throughputKbps: 1638 },
  runs: 5,
  location: 'eu-west-1',
};

export const configHash = createHash('sha256')
  .update(JSON.stringify(LAB_CONFIG))
  .digest('hex')
  .slice(0, 12);

Step 2 — Collect the lab side on a schedule

Run one representative URL per template, not per page. Take the median of the runs — a single run varies by 10–20% and would make the ratio meaningless.

// collect-lab.mjs — median of N runs per template URL.
import { LAB_CONFIG, configHash } from './lab-config.mjs';

const median = (xs) => xs.slice().sort((a, b) => a - b)[Math.floor(xs.length / 2)];

export async function collectLab(templates, runLighthouse) {
  const rows = [];
  for (const { template, url } of templates) {
    const runs = [];
    for (let i = 0; i < LAB_CONFIG.runs; i++) runs.push(await runLighthouse(url));
    rows.push({
      day: new Date().toISOString().slice(0, 10),
      template,
      configHash,
      lab_lcp: median(runs.map((r) => r.lcp)),
      lab_tbt: median(runs.map((r) => r.tbt)),
      lab_cls: median(runs.map((r) => r.cls)),
      // Spread across runs: if this is wide, the ratio built on it is unstable.
      lab_lcp_spread: Math.max(...runs.map((r) => r.lcp)) - Math.min(...runs.map((r) => r.lcp)),
    });
  }
  return rows;
}

Storing the spread alongside the median is what tells you later whether a ratio change is a real shift or lab noise.

Step 3 — Collect the field side on the same schedule, with matched filters

-- Field side: mobile only, matching the lab form factor, per template.
SELECT
  toDate(ts)                                    AS day,
  route                                         AS template,
  quantileTDigestIf(0.75)(value, metric = 'LCP') AS field_lcp_p75,
  quantileTDigestIf(0.75)(value, metric = 'INP') AS field_inp_p75,
  quantileTDigestIf(0.75)(value, metric = 'CLS') AS field_cls_p75,
  countIf(metric = 'LCP')                        AS samples
FROM rum_events
WHERE ts >= today() - 1
  AND device_class IN ('mid', 'low')            -- the cohort the lab simulates
  AND nav_type = 'navigate'                     -- exclude bfcache restores
GROUP BY day, template
HAVING samples >= 1000;

The HAVING clause matters: a ratio computed on a template with 40 samples is noise, and it will produce a budget that blocks pull requests for no reason.

Step 4 — Compute and track the ratio

// ratio.mjs — the number the whole report exists to produce.
export function buildRatios(labRows, fieldRows) {
  const field = new Map(fieldRows.map((r) => [`${r.day}|${r.template}`, r]));
  return labRows.flatMap((lab) => {
    const f = field.get(`${lab.day}|${lab.template}`);
    if (!f) return [];
    return [{
      day: lab.day,
      template: lab.template,
      configHash: lab.configHash,
      // Ratio < 1 means the lab reads faster than the field p75, which is
      // the normal direction once you have matched the cohort.
      lcp_ratio: +(lab.lab_lcp / f.field_lcp_p75).toFixed(3),
      cls_ratio: +(lab.lab_cls / Math.max(f.field_cls_p75, 0.001)).toFixed(3),
      samples: f.samples,
      spread: lab.lab_lcp_spread,
    }];
  });
}

// A budget is the field target scaled back through the ratio.
export const labBudget = (fieldTarget, ratio) => Math.round(fieldTarget * ratio);

With a measured LCP ratio of 0.72 and a field target of 2,500 ms, the lab budget is 1,800 ms. That number is defensible in a way that “2,000 ms felt about right” is not.

Stable for nine weeks, then a sharp fall — the lab kept reporting the same number while the field got worse. That divergence is the report doing its job.

Step 5 — Alert on the ratio, not just the metrics

A falling ratio means the lab is no longer representative: the field is degrading in a way the lab cannot see. That is a distinct failure from either number moving, and it is the most valuable signal the report produces.

Ratio behaviour	Meaning	Action
Stable	Lab is representative	Keep gating on the derived budget
Falls	Field worse without lab noticing	Investigate a cohort, third party or personalised path
Rises	Lab worse without field noticing	Usually a runner or configuration problem
Jumps at a config change	Expected	Re-baseline; do not compare across the change

Step 6 — Publish the budgets automatically

The report should emit the CI configuration rather than have someone copy numbers into it. That is what keeps the two in sync.

// emit-budgets.mjs — write the assertions CI will run tomorrow.
import { writeFileSync } from 'node:fs';
import { labBudget } from './ratio.mjs';

const FIELD_TARGETS = { '/': 2500, '/products/[slug]': 2500, '/checkout': 3000 };

export function emitBudgets(latestRatios) {
  const assertions = {};
  for (const r of latestRatios) {
    const target = FIELD_TARGETS[r.template];
    if (!target || r.samples < 1000) continue;
    assertions[r.template] = {
      'largest-contentful-paint': ['error', { maxNumericValue: labBudget(target, r.lcp_ratio) }],
    };
  }
  writeFileSync('lighthouse-budgets.json', JSON.stringify(assertions, null, 2));
}

One global budget cannot serve a 0.85 template and a 0.61 one. The authenticated route has no lab counterpart at all, which is a finding rather than a gap to paper over.

What the report tells you after a quarter

Three patterns show up in nearly every long-running correlation report, and each one changes how much you trust the lab.

Ratios differ substantially between templates. A static marketing page may sit at 0.85 while a personalised dashboard sits at 0.45, because the lab never logs in. One global budget cannot serve both, which is the strongest argument for per-template budgets.

Ratios are more stable than either input. Both the lab and field numbers move with releases and traffic; their ratio usually does not. That stability is what makes it usable as a conversion factor.

A broken ratio predicts an incident. A ratio that falls without a config change is generally the earliest quantitative sign that some part of your real traffic is having an experience your testing does not reproduce — a new third party, a rolled-out experiment, or a cohort whose devices got slower relative to the simulation.

Verifying it works

Ratios are stable within about ±0.05 week to week on templates with adequate volume. Wider than that and the lab configuration is too noisy to derive budgets from.
Derived budgets fail the build on a deliberate regression and pass on an unchanged branch, tested at least once.
The config hash changes only when you intend it to. A silent tool upgrade in CI is the most common cause of a mysterious ratio jump.
Every row carries a sample count, and rows below the floor are excluded rather than quietly averaged in.

The last stage is what keeps the budgets current. A ratio that is read by a human and typed into a config file is stale within a quarter.

Edge cases and gotchas

Comparing across configuration changes. A Lighthouse major version can shift metrics by double digits. Treat the config hash as a partition key, never as metadata.
Templates the lab cannot reach. Authenticated, paywalled or checkout routes usually have no lab counterpart. Either build a fixture that reaches them or accept there is no ratio for those templates.
Field data contaminated by bots. Automated traffic distorts the denominator. Filter before joining, not after.
Seasonality in the field number. Traffic mix shifts on weekends and during campaigns, so a ratio computed on a single day is noisier than one computed on a rolling week.
Using the mean instead of the median for lab runs. One slow run pulls the mean far more than the median; with five runs the difference is routinely 10%.
Letting the report rot. A correlation report nobody looks at is worse than none, because its stale budgets are still gating pull requests.

Synthetic vs Field Data Trade-offs — why the two sources differ before you try to relate them.
Reconciling Lighthouse Scores with Field Data — the one-off reconciliation this report automates.
Performance Budgets & SLOs — where the derived budgets are actually enforced.