The Queue Has an Area

A queue is usually drawn as a line of jobs waiting for a server.

That picture is useful, but it hides the object that Little’s Law is really about. Draw the queue as a function of time instead:

\[N(t) = \text{number of jobs in the system at time } t.\]

Now the queue has an area.

Every job contributes one horizontal strip to that area, starting when it enters the system and ending when it leaves. If job \(i\) arrives at \(a_i\) and departs at \(d_i\), its strip has width \(d_i-a_i\). Stack all the strips and you get the area under the step function:

\[\int_0^T N(t)\,dt.\]

This is the small accounting fact that makes Little’s Law feel less like a queueing-theory spell:

\[\text{area under occupancy} = \text{sum of residence times}.\]

When the observation starts empty and ends empty, with \(n\) completed jobs in a window of length \(T\), the algebra is almost embarrassingly short:

\[L = \frac{1}{T}\int_0^T N(t)\,dt, \qquad \lambda = \frac{n}{T}, \qquad W = \frac{1}{n}\sum_{i=1}^n (d_i-a_i).\]

Because the integral equals the sum,

\[L = \frac{1}{T}\sum_{i=1}^n(d_i-a_i) = \frac{n}{T}\cdot\frac{1}{n}\sum_{i=1}^n(d_i-a_i) = \lambda W.\]

No exponential service times. No Poisson arrivals. No FIFO assumption. For this finite empty-to-empty experiment, the result is just a conserved area.

Little’s 1961 proof gave the stochastic stationary theorem under broad conditions.¹ Stidham later gave a direct sample-path proof using weaker, deterministic-looking hypotheses on limiting averages.² The later literature extends the relation in several directions; Whitt’s review is a useful map of those variants, and Little’s 2011 retrospective is a good guide to the theorem’s operational uses.³⁴ The practical lesson is less formal but very sharp: if your measurements do not cover the same area, rate, and residence-time ledger, the equation can look wrong for reasons you created.

The Lab: Cut the Log, Lose the Edge

The simulator below is a single-server queue with configurable arrival burstiness and service variability. The default view is intentionally a cropped production log slice. Some jobs were already in the system when the clock started. Some are still present when the clock stops.

The green step plot is \(N(t)\). The stacked bar underneath is the exact residence-time ledger inside the visible window:

\[\int_s^t N(u)\,du = \sum_i \left|[a_i,d_i)\cap[s,t]\right|.\]

The blue outline is the tempting shortcut: count only jobs that both arrive and depart inside the window, then compute \(\lambda W\) from those closed jobs. Switch the observation mode to “empty-to-empty drain” and the shortcut becomes exact. Switch back to a cropped slice and the missing edge terms appear.

Offered load: 82% Mean service: 52 ms Arrival CV: 100% Service CV: 100% Window length: 18 s Observation: Seed: 170

Occupancy area Closed jobs Left edge Right edge Spans window

Deterministic synthetic experiment. Interarrival and service times are lognormal with the selected means and coefficients of variation. The area under the step function is integrated exactly from event times; no sampling grid is used for the identity check.

The default settings usually show a positive edge debt. That is not a violation of Little’s Law. It is a bookkeeping error in the shortcut.

The exact finite-window statement is the intersection formula:

\[A(s,t) = \int_s^t N(u)\,du = \sum_i \left|[a_i,d_i)\cap[s,t]\right|.\]

The completed-only product is only this part:

\[\widehat L_\text{closed} = \frac{1}{t-s} \sum_{i: s\le a_i<d_i\le t}(d_i-a_i).\]

It drops jobs already present at \(s\) and jobs still present at \(t\). In a short window, a high-variance service distribution, or a system near saturation, those boundary rectangles can be a large fraction of the area.

Kim and Whitt make this point in a statistical setting: real applications of Little’s Law use finite-time measurements, not infinite limits or perfectly stationary expectations, so interval edge effects and confidence intervals have to be treated as part of the measurement problem.⁵

What the Equation Does Not Buy You

Little’s Law is a constraint, not a performance model.

If you know the long-run throughput and average residence time, it tells you the average number in the system. If you know average occupancy and throughput, it tells you average residence time. It does not tell you which service discipline created that residence time, whether the p99 is survivable, or what happens if the next burst arrives.

That distinction matters in systems work. A dashboard might say:

work in progress: 240 requests
throughput:        80 requests / second

If the measurements are consistent and the system is not drifting, the implied mean time in system is about three seconds. But the equation did not discover why. The cause could be slow service, queueing behind a saturated server, locks, retry amplification, rate limiting, downstream backpressure, or a measurement window that mixed several regimes.

Nor does the equation license offered load as \(\lambda\). In a lossy system, the arrival rate offered to the front door can differ from the effective rate that enters the measured system. If admission control rejects half the requests, using offered arrivals with the residence time of admitted requests crosses two different populations.

The word “system” is doing real work. You can apply Little’s Law to:

the waiting room only;
the server only;
queue plus service;
one shard;
one tenant;
one retry attempt;
all attempts belonging to one user-visible request.

Each boundary has its own \(N(t)\), \(\lambda\), and \(W\). Mixing boundaries is how a conservation law becomes a dashboard superstition.

A Useful Measurement Contract

For an operational service, I would rather see Little’s Law used as an audit than as a slogan. Pick a boundary and a time window, then ask whether the three measurements describe the same population.

Count occupancy by integrating queue depth or in-flight work over time.
Measure throughput over the same window and the same boundary.
Measure residence time for the same jobs, not a sampled subset with a different clock.
Report whether the window starts and ends empty, or account for boundary work explicitly.
Separate waiting time from service time when the remedy depends on the phase.
Use effective arrivals for admitted work; report rejected, timed out, canceled, and retried work separately.
Slice by route, tenant, shard, priority, or job class before averaging hides the queue that hurts.
Treat short windows as estimates with uncertainty, not as laws of nature.

The lab exports the same audit:

node - <<'NODE'
const lab = require("./assets/js/littles-law-lab.js");
const EXPECTED_GRID_SHAPE = [
  "modes=drain/slice",
  "loads=45/72/93",
  "arrivalCv=0/80/160",
  "serviceCv=0/90/150",
  "seeds=3/41"
];
const EXPECTED_MODE_SHAPE = [
  "drain:cases=54:passed=54:checks=702/702:meanEdgeDebt=0.000000:open=0/0:maxResidual=0.000000000000",
  "slice:cases=54:passed=54:checks=594/594:meanEdgeDebt=0.158744:open=191/249:maxResidual=0.000000000000"
];
const EXPECTED_LOAD_SHAPE = [
  "45:cases=36:passed=36:checks=432/432:meanEdgeDebt=0.004781:open=13/13:maxResidual=0.000000000000",
  "72:cases=36:passed=36:checks=432/432:meanEdgeDebt=0.020383:open=29/56:maxResidual=0.000000000000",
  "93:cases=36:passed=36:checks=432/432:meanEdgeDebt=0.212952:open=149/180:maxResidual=0.000000000000"
];
const EXPECTED_WORST_RESIDUAL_SHAPE = [
  "drain:93:160:0:3:residual=0.000000000000:edgeDebt=-0.000000:13/13:true",
  "slice:93:80:150:3:residual=0.000000000000:edgeDebt=1.119933:11/11:true",
  "slice:93:80:150:41:residual=0.000000000000:edgeDebt=0.489341:11/11:true",
  "slice:93:80:90:3:residual=0.000000000000:edgeDebt=0.355634:11/11:true",
  "drain:93:160:90:3:residual=0.000000000000:edgeDebt=0.000000:13/13:true",
  "slice:93:0:90:41:residual=0.000000000000:edgeDebt=0.178910:11/11:true"
];
const EXPECTED_CHECK_NAMES = [
  "area equals residence-time ledger",
  "area is finite and nonnegative",
  "closed lambda-W product matches closed rate times mean W",
  "drain closes all rectangles",
  "drain ends empty",
  "drain starts empty",
  "edge debt is nonnegative",
  "ledger components sum to total",
  "max queue length bounds endpoints",
  "queue length stays nonnegative",
  "slice edge debt equals boundary rectangles",
  "time-average L equals area divided by window",
  "time-average L splits into closed product plus edge debt",
  "window is finite and positive"
];
const EXPECTED_COMMON_CHECKS = [
  { name: "area is finite and nonnegative", ok: true },
  { name: "window is finite and positive", ok: true },
  { name: "area equals residence-time ledger", ok: true },
  { name: "ledger components sum to total", ok: true },
  { name: "time-average L splits into closed product plus edge debt", ok: true },
  { name: "closed lambda-W product matches closed rate times mean W", ok: true },
  { name: "time-average L equals area divided by window", ok: true },
  { name: "queue length stays nonnegative", ok: true },
  { name: "max queue length bounds endpoints", ok: true },
  { name: "edge debt is nonnegative", ok: true }
];
const EXPECTED_CHECKS_BY_MODE = {
  drain: EXPECTED_COMMON_CHECKS.concat([
    { name: "drain closes all rectangles", ok: true },
    { name: "drain starts empty", ok: true },
    { name: "drain ends empty", ok: true }
  ]),
  slice: EXPECTED_COMMON_CHECKS.concat([
    { name: "slice edge debt equals boundary rectangles", ok: true }
  ])
};
const EXPECTED_MODES = EXPECTED_MODE_SHAPE.length;
const EXPECTED_LOADS = EXPECTED_LOAD_SHAPE.length;
const EXPECTED_CASES = 2 * 3 * 3 * 3 * 2;
const EXPECTED_CHECKS = 1296;
const EXPECTED_MAX_RESIDUAL = "0.000000000000";

function sameList(actual, expected) {
  return actual.length === expected.length &&
    actual.every((value, index) => value === expected[index]);
}

function sameJson(actual, expected) {
  return JSON.stringify(actual) === JSON.stringify(expected);
}

function fixed(value, digits) {
  return Number(value).toFixed(digits);
}

function groupShape(row, key) {
  return `${row[key]}:cases=${row.cases}:passed=${row.passed}:` +
    `checks=${row.passedChecks}/${row.totalChecks}:` +
    `meanEdgeDebt=${fixed(row.meanEdgeDebt, 6)}:` +
    `open=${row.openAtStart}/${row.openAtEnd}:` +
    `maxResidual=${fixed(row.maxResidual, 12)}`;
}

function residualShape(row) {
  return `${row.mode}:${row.load}:${row.arrivalCv}:${row.serviceCv}:${row.seed}:` +
    `residual=${fixed(row.residual, 12)}:edgeDebt=${fixed(row.edgeDebt, 6)}:` +
    `${row.checks}:${row.passed}`;
}

function uniqueNumbers(audit, key) {
  return Array.from(new Set(audit.cases.map((row) => row[key]))).join("/");
}

const audit = lab.audit();
console.table(audit.byMode);
console.table(audit.byLoad);
console.table(audit.worstResiduals);
const shape = {
  byLoad: audit.byLoad.map((row) => groupShape(row, "load")),
  byMode: audit.byMode.map((row) => groupShape(row, "mode")),
  checkNames: Array.from(new Set(audit.cases.flatMap(
    (row) => row.checks.map((check) => check.name)
  ))).sort(),
  grid: [
    `modes=${uniqueNumbers(audit, "mode")}`,
    `loads=${uniqueNumbers(audit, "load")}`,
    `arrivalCv=${uniqueNumbers(audit, "arrivalCv")}`,
    `serviceCv=${uniqueNumbers(audit, "serviceCv")}`,
    `seeds=${uniqueNumbers(audit, "seed")}`
  ],
  maxResidual: fixed(audit.maxResidual, 12),
  worstResiduals: audit.worstResiduals.map(residualShape)
};
const checkRowDrifts = audit.cases.map((row) => ({
  key: `${row.mode}/${row.load}/${row.arrivalCv}/${row.serviceCv}/${row.seed}`,
  checks: row.checks.map((check) => ({
    name: check.name,
    ok: check.ok
  })),
  expected: EXPECTED_CHECKS_BY_MODE[row.mode]
})).filter((row) => !sameJson(row.checks, row.expected));
const shapeErrors = [];
if (!sameList(shape.grid, EXPECTED_GRID_SHAPE)) {
  shapeErrors.push({ name: "grid", actual: shape.grid, expected: EXPECTED_GRID_SHAPE });
}
if (!sameList(shape.byMode, EXPECTED_MODE_SHAPE)) {
  shapeErrors.push({ name: "byMode", actual: shape.byMode, expected: EXPECTED_MODE_SHAPE });
}
if (!sameList(shape.byLoad, EXPECTED_LOAD_SHAPE)) {
  shapeErrors.push({ name: "byLoad", actual: shape.byLoad, expected: EXPECTED_LOAD_SHAPE });
}
if (!sameList(shape.worstResiduals, EXPECTED_WORST_RESIDUAL_SHAPE)) {
  shapeErrors.push({
    name: "worstResiduals",
    actual: shape.worstResiduals,
    expected: EXPECTED_WORST_RESIDUAL_SHAPE
  });
}
if (!sameList(shape.checkNames, EXPECTED_CHECK_NAMES)) {
  shapeErrors.push({
    name: "checkNames",
    actual: shape.checkNames,
    expected: EXPECTED_CHECK_NAMES
  });
}
if (shape.maxResidual !== EXPECTED_MAX_RESIDUAL) {
  shapeErrors.push({
    name: "maxResidual",
    actual: shape.maxResidual,
    expected: EXPECTED_MAX_RESIDUAL
  });
}
if (checkRowDrifts.length) {
  shapeErrors.push({
    name: "checkRows",
    actual: checkRowDrifts.slice(0, 10),
    expected: EXPECTED_CHECKS_BY_MODE
  });
}

const failedModeGroups = audit.byMode.filter(
  (row) => row.passed !== row.cases || row.passedChecks !== row.totalChecks
);
const failedLoadGroups = audit.byLoad.filter(
  (row) => row.passed !== row.cases || row.passedChecks !== row.totalChecks
);
const failedCases = audit.cases.filter(
  (row) => !row.passed || row.passedChecks !== row.totalChecks
);
if (shapeErrors.length ||
    !audit.ok ||
    !audit.pass ||
    audit.byMode.length !== EXPECTED_MODES ||
    audit.byLoad.length !== EXPECTED_LOADS ||
    audit.cases.length !== EXPECTED_CASES ||
    audit.caseCount !== EXPECTED_CASES ||
    audit.total !== EXPECTED_CASES ||
    audit.totalChecks !== EXPECTED_CHECKS ||
    failedModeGroups.length ||
    failedLoadGroups.length ||
    failedCases.length ||
    audit.passed !== audit.total ||
    audit.passedChecks !== audit.totalChecks) {
  throw new Error(JSON.stringify({
    shapeErrors,
    failures: audit.failures,
    failedModeGroups,
    failedLoadGroups,
    failedCases: failedCases.slice(0, 10),
    totals: {
      caseCount: audit.caseCount,
      rows: audit.cases.length,
      passed: audit.passed,
      passedChecks: audit.passedChecks,
      total: audit.total,
      totalChecks: audit.totalChecks
    }
  }, null, 2));
}
console.log(
  `${audit.passed}/${audit.total} cases and ` +
  `${audit.passedChecks}/${audit.totalChecks} checks passed; ` +
  `max residual ${audit.maxResidual}`
);
NODE

The check is deliberately mundane. The current grid runs 108 deterministic simulations and passes 1296 named checks. It integrates the step function exactly, sums the residence-time intersections, verifies that the residual is numerical noise, checks that the ledger components sum back to the whole window, and confirms that \(L\) splits into completed-job \(\lambda W\) plus boundary debt. In empty-to-empty mode it also checks that the closed-job ledger equals the whole area and that the window starts and ends empty.

That is the right level of romance for a queue. The theorem is beautiful because the accounting is unforgiving. Jobs can wait in strange orders, arrive in bursts, and take variable service times. But once you draw the system boundary and the time window, the area has to go somewhere.

The Lab: Cut the Log, Lose the Edge

What the Equation Does Not Buy You

A Useful Measurement Contract

Further Reading

Related notes