The most misleading number on a fast chip is often the biggest one.

60 TFLOP/s
989 TFLOP/s
1.9 PFLOP/s

The digits look like speed. They are not quite speed.

They are a ceiling for a particular kind of arithmetic under a particular precision and instruction mix. To touch that ceiling, a kernel has to feed the math units quickly enough. If every useful operation needs a new cache line from memory, the arithmetic units spend most of their life waiting politely.

This is the small performance lie I want to unlearn:

low percent of peak FLOPs means the code is bad at arithmetic

Sometimes the code is bad. Sometimes the code is asking memory to deliver a miracle.

The Missing Denominator

A kernel does work (W). It also moves data (Q).

The roofline model calls their ratio operational intensity:

\[I = \frac{W}{Q}.\]

If (W) is measured in floating-point operations and (Q) is measured in bytes, then (I) has units:

FLOPs per byte

That unit is the clue. It asks how much useful arithmetic each byte buys after the byte has been fetched from the relevant memory level.

Williams, Waterman, and Patterson introduced the Roofline model as a visual bound-and-bottleneck model that ties together floating-point performance, operational intensity, and memory performance.1 In their paper, the central formula is:

\[P_{\text{attainable}} \le \min(P_{\text{peak}},\ B_{\text{mem}} I).\]

The horizontal roof is peak compute. The slanted roof is memory bandwidth times operational intensity. The kernel lives under the lower of the two.

The intersection is the ridge point:

\[I^* = \frac{P_{\text{peak}}}{B_{\text{mem}}}.\]

If a kernel’s intensity is left of the ridge, buying more peak FLOPs does little unless bandwidth or reuse changes. If it is right of the ridge, arithmetic throughput is the nearer ceiling.

The formula is simple enough to fit on a napkin.

It is also enough to explain a surprising amount of disappointment.

A Triad Is Mostly Traffic

Take the STREAM triad shape:

y[i] = a * x[i] + y[i]

Per element, in single precision, the useful arithmetic is roughly:

multiply + add = 2 FLOPs

The traffic is roughly:

read x[i]  = 4 bytes
read y[i]  = 4 bytes
write y[i] = 4 bytes

That gives:

\[I_{\text{triad}} = \frac{2}{12} \approx 0.167\ \text{FLOP/byte}.\]

Now put that kernel on an illustrative accelerator with 60 TFLOP/s of peak FP32-like compute and 3.35 TB/s of memory bandwidth. Those bandwidth and compute scales are in the neighborhood of modern data-center GPUs; NVIDIA’s H100 page, for example, lists 67 teraFLOPS FP32 and 3.35 TB/s memory bandwidth for H100 SXM, while tensor-core modes have much higher arithmetic ceilings.2

The ridge for the 60 TFLOP/s, 3.35 TB/s machine is:

\[I^* = \frac{60}{3.35} \approx 17.91\ \text{FLOP/byte}.\]

The triad’s 0.167 FLOP/byte is not close. The memory roof predicts:

\[3.35\ \text{TB/s}\times 0.167\ \text{FLOP/byte} \approx 0.558\ \text{TFLOP/s}.\]

That is less than one percent of 60 TFLOP/s.

Not because the multiply-add is slow. Because each two-FLOP morsel arrives with twelve bytes of billable traffic.

This is why STREAM exists. McCalpin’s STREAM benchmark was designed to measure sustained memory bandwidth for simple vector kernels, because user-code bandwidth is often the figure of merit for low-computational-density vector codes.3 A later LLNL summary describes STREAM as a synthetic benchmark for sustainable memory bandwidth using Copy, Scale, Add, and Triad kernels.4

The word “sustainable” matters. Vendor peak bandwidth, pin bandwidth, cache bandwidth, and application bandwidth are not the same promise. The original Roofline paper is careful about this too: it uses sustainable DRAM bandwidth behind the caches, not just the pin bandwidth of DRAM chips.5

Instrument Panel

The lab below is deterministic. It does not benchmark your browser. It computes the roofline bound and plots several kernel shapes using simple byte ledgers:

  • STREAM triad and dot product;
  • a sparse-matrix-vector style access pattern;
  • a stencil with adjustable neighbor reuse;
  • tiled matrix multiply;
  • an attention-score block with adjustable score-materialization traffic.

The default selected kernel is STREAM triad. Its point sits far left of the ridge. Change Selected kernel to tiled GEMM, raise Tile / block size, and watch the point move right as each loaded tile participates in more multiply-adds.

Bound model, not a benchmark. The byte ledgers are deliberately small enough to inspect; real kernels need measured traffic, cache effects, vectorization, occupancy, instruction mix, and launch overheads.

Two Ways to Move

On the roofline plot, there are only two honest directions.

You can move up by using more of the available compute ceiling:

vectorize
use tensor cores
increase occupancy
reduce instruction overhead

Or you can move right by making each byte do more work:

tile
fuse kernels
keep data in registers or shared memory
avoid materializing intermediates
compress or lower precision when numerically safe

The mistake is to apply the first list to a point that is trapped under the memory roof.

For the default triad, doubling peak compute changes nothing. The lab’s double peak bar is still about 558 GF/s, because the slanted roof has not moved. Halving the bytes, however, doubles the roof to about 1.12 TF/s.

For tiled GEMM at a large enough tile size, the story flips. Once the point passes the ridge, lowering traffic is still useful for energy and pressure on the memory system, but the roofline limit is now the flat compute roof. Better instructions, better scheduling, and higher arithmetic utilization become the natural questions.

This is why “optimize the kernel” is too vague to be an engineering plan.

The right verb depends on where the point is.

The Modern GPU Twist

Modern accelerators make the ridge move dramatically depending on which arithmetic roof you choose.

The same H100 product page lists 67 TFLOP/s FP32, 989 TFLOP/s TF32 Tensor Core throughput with sparsity, and 3.35 TB/s memory bandwidth for H100 SXM. Those produce very different ridge points:

FP32-ish roof:  67 / 3.35 ~= 20 FLOP/byte
TF32 tensor roof: 989 / 3.35 ~= 295 FLOP/byte

That does not mean tensor cores are pointless. It means the price of feeding them is higher. A kernel that is compute-bound under the FP32 roof may become memory-bound under the tensor-core roof unless it has enormous reuse.

This is one reason high-performance matrix multiply looks so unlike the mathematical expression (C=AB). The implementation is an operation in data reuse. Blocks are packed. Tiles live in shared memory or registers. Accumulators stay hot. The same values are multiplied many times before being evicted.

The algorithm is not merely doing arithmetic.

It is manufacturing operational intensity.

Percent of Peak Is Not a Diagnosis

Suppose a profile says:

kernel achieves 0.6 TFLOP/s on a 60 TFLOP/s chip

That sounds like 1% utilization.

But if the kernel has intensity 0.17 FLOP/byte and the machine has 3.35 TB/s of sustainable bandwidth, then the roof is already around 0.56 TFLOP/s. The kernel may be close to its bandwidth-limited ceiling.

The diagnostic question is not:

what percent of peak FLOPs did I get?

It is:

what percent of the relevant roof did I get?

That roof might be DRAM bandwidth. Or L2 bandwidth. Or shared-memory bandwidth. Or scalar instruction throughput. Or tensor-core throughput. Or PCIe. Or network all-reduce. The simple roofline picture has two roofs because two roofs are enough to teach the habit. Real performance work adds more ceilings.

The habit is the part that survives.

How I Would Use This in a Code Review

Before touching a performance kernel, I want a byte ledger:

  • how many useful operations are performed;
  • how many bytes are read and written at the memory level that matters;
  • whether the working set fits in cache;
  • whether intermediates are materialized and reread;
  • what precision is used for storage and arithmetic;
  • what the measured bandwidth roof is on the target machine.

Then I want the point on the roofline.

If it is far left of the ridge, I ask byte questions:

  • Can two passes become one?
  • Can an intermediate stay in registers?
  • Can a transpose or layout change make accesses coalesced?
  • Can a sparse format reduce index traffic?
  • Can a tile be reused more before eviction?

If it is far right of the ridge, I ask compute questions:

  • Is the vector or tensor instruction path actually used?
  • Is occupancy hiding latency?
  • Are dependent operations serializing the loop?
  • Is the compiler generating the expected instructions?
  • Is the launch overhead larger than the useful work?

If it is near the ridge, life gets interesting. Small changes to either traffic or arithmetic utilization can matter, and measurement error can change the story. That is where I stop trusting napkin arithmetic and start collecting counters.

The Useful Lesson

Peak FLOPs are not fake. They are just incomplete.

They answer:

how fast could arithmetic retire if data and instructions arrived perfectly?

The roofline asks a less glamorous question:

how much arithmetic does each byte buy before the next byte is needed?

That question changes the emotional temperature of performance work. A slow kernel is no longer automatically a failed attempt to use the math units. It may be a perfectly predictable consequence of a low denominator.

The chip did not promise your code 60 TFLOP/s.

It promised a roof.

You still have to climb to the right part of it.

  1. Samuel Williams, Andrew Waterman, and David Patterson, “Roofline: an insightful visual performance model for multicore architectures”, Communications of the ACM 52(4), 2009. OSTI record. ↩

  2. NVIDIA, “NVIDIA H100 GPU” product page, product specifications section. Accessed June 14, 2026. Page. ↩

  3. John D. McCalpin, “Sustainable Memory Bandwidth in Current High Performance Computers”, revised October 12, 1995. HTML. ↩

  4. LLNL, “STREAM Summary Version 1.0”, describing STREAM as a benchmark for sustainable memory bandwidth using simple vector kernels. PDF. ↩

  5. Samuel Williams, Andrew Waterman, and David Patterson, “Roofline: An Insightful Visual Performance Model for Floating-Point Programs and Multicore Architectures”, accepted manuscript. The paper states the bound as attainable GFLOP/s equals the minimum of peak floating-point performance and peak memory bandwidth times operational intensity. PDF. ↩