The D·A·M Taxonomy

A system that runs slowly rarely announces why. An idle accelerator can stem from a starved input pipeline (Data), an algorithm with too little arithmetic intensity (Algorithm), or a saturated memory bus (Machine). This appendix formalizes the book’s single-node diagnostic taxonomy, mapping profiler symptoms and measurements directly to the axis that actually binds execution before optimization begins. Readers should arrive with the three axes from the opening chapters and the iron law that quantifies them.

How to Use This Appendix

Use the D·A·M taxonomy when a system misses a performance, cost, or quality target and the cause is not yet clear. Work through the framework in four passes:

State the symptom and operating budget: Record the missed end-to-end target, workload, and deployment context.
Localize the candidate axis or boundary: Use table 1 for a first pass and table 2 when the evidence crosses axes.
Test the hypothesis: Map the evidence to the iron law and arithmetic intensity, then gather confirming measurements with table 6 and table 7.
Intervene and remeasure: Choose a lever from table 4 and repeat the end-to-end measurement. If the target does not improve, revise the hypothesis.

The symptom determines the most useful entry point. Low accelerator utilization or slow batch delivery should begin with section 1.5 and section 1.8. A missed service-level objective (SLO) should move from phase timing and queue depth in table 5 to the corresponding iron-law term. A quality plateau should be checked against the Data/Algorithm boundary and section 1.11 before allocating more compute. In every case, the axis label is a hypothesis, not a verdict: the axes overlap, and a useful diagnosis may name both a dominant axis and a boundary effect.

Diagnostic Summary

The taxonomy maps directly to the iron law of ML systems established in Iron Law of ML Systems. Table 1 summarizes the role, primary physical constraint, and core optimization pathway for each axis.

Table 1: D·A·M Axis Reference: Each axis maps to a constraint class and optimization strategy. Form a hypothesis, confirm it with measurements, then follow the chapter pointer.

Axis	Role	Physical Constraint	High-Leverage Optimization
Data (D)	Information (The Fuel)	Volume, quality, arrival rate	Data Selection (Data Selection)
Algorithm (A)	Logic (The Blueprint)	Operations and dependencies	Model Compression (Model Compression)
Machine (M)	Physics (The Engine)	Compute, memory, I/O	Hardware Acceleration (Hardware Acceleration)

This first-pass separation is useful, but production systems rarely suffer from a single axis-centered bottleneck. More often, the problem sits at the boundary between two axes—a data format choice that determines whether the GPU can be saturated, or a pruning strategy that changes the memory access pattern. Handling these cases requires mapping the intersections.

Intersection Landscape

The opening chapter uses The D·A·M Intersection Landscape to establish the conceptual landscape. Here, figure 1 turns that map into a diagnostic sequence. First identify the binding axis, then ask how an intervention will alter the other two.

\scalebox{0.85}{
\begin{tikzpicture}[line join=round,font=\sffamily\small]
\tikzset{
 Line/.style={line width=0.35pt,black!50,text=black},
 ALineA/.style={violet!80!black!50,line width=3pt,shorten <=2pt,shorten >=2pt,
  {Triangle[width=1.1*6pt,length=0.8*6pt]}-{Triangle[width=1.1*6pt,length=0.8*6pt]}},
LineD/.style={line width=0.75pt,black!50,text=black,dashed,dash pattern=on 5pt off 3pt},
Circle/.style={inner xsep=2pt,
  circle,
    draw=BrownLine,
    line width=0.75pt,
    fill=BrownL!40,
    minimum size=16mm
  },
 circles/.pic={
\pgfkeys{/channel/.cd, #1}
\node[circle,draw=\channelcolor,line width=\Linewidth,fill=\channelcolor!10,
minimum size=2.5mm](\picname){};
        }
}
\tikzset {
pics/cloud/.style = {
        code = {\colorlet{red}{RedLine}
\begin{scope}[local bounding box=CLO,scale=0.5, every node/.append style={transform shape}]
\draw[red,fill=white,line width=0.9pt](0.67,1.21)to[out=55,in=90,distance=13](1.5,0.96)
to[out=360,in=30,distance=9](1.68,0.42);
\draw[red,fill=white,line width=0.9pt](0,0)to[out=170,in=180,distance=11](0.1,0.61)
to[out=90,in=105,distance=17](1.07,0.71)
to[out=20,in=75,distance=7](1.48,0.36)
to[out=350,in=0,distance=7](1.48,0)--(0,0);
\draw[red,fill=white,line width=0.9pt](0.27,0.71)to[bend left=25](0.49,0.96);

\end{scope}
    }
  }
}
%streaming
\tikzset{%
 LineST/.style={-{Circle[\channelcolor,fill=RedLine,length=4pt]},draw=\channelcolor,line width=\Linewidth,rounded corners},
 ellipseST/.style={fill=\channelcolor,ellipse,minimum width = 2.5mm, inner sep=2pt, minimum height =1.5mm},
 BoxST/.style={line width=\Linewidth,fill=white,draw=\channelcolor,rectangle,minimum width=56,
 minimum height=16,rounded corners=1.2pt},
 pics/streaming/.style = {
        code = {\pgfkeys{/channel/.cd, #1}
\begin{scope}[local bounding box=STREAMING,scale=\scalefac, every node/.append style={transform shape}]
\node[BoxST,minimum width=44,minimum height=48](\picname-RE1){};
\foreach \i/\j in{1/north,2/center,3/south}{
\node[BoxST](\picname-GR\i)at(\picname-RE1.\j){};
\node[ellipseST]at($(\picname-GR\i.west)!0.2!(\picname-GR\i.east)$){};
\node[ellipseST]at($(\picname-GR\i.west)!0.4!(\picname-GR\i.east)$){};
}
\draw[LineST](\picname-GR3)--++(2,0)coordinate(\picname-C4);
\draw[LineST](\picname-GR3.320)--++(0,-0.7)--++(0.8,0)coordinate(\picname-C5);
\draw[LineST](\picname-GR3.220)--++(0,-0.7)--++(-0.8,0)coordinate(\picname-C6);
\draw[LineST](\picname-GR3)--++(-2,0)coordinate(\picname-C7);
 \end{scope}
     }
  }
}
%data
\tikzset{mycylinder/.style={cylinder, shape border rotate=90, aspect=1.3, draw, fill=white,
minimum width=25mm,minimum height=11mm,line width=\Linewidth,node distance=-0.15},
pics/data/.style = {
        code = {\pgfkeys{/channel/.cd, #1}
\begin{scope}[local bounding box=STREAMING,scale=\scalefac, every node/.append style={transform shape}]
\node[mycylinder,fill=\channelcolor!50] (A) {};
\node[mycylinder, above=of A,fill=\channelcolor!30] (B) {};
\node[mycylinder, above=of B,fill=\channelcolor!10] (C) {};
 \end{scope}
     }
  }
}
\pgfkeys{
  /channel/.cd,
  channelcolor/.store in=\channelcolor,
  drawchannelcolor/.store in=\drawchannelcolor,
  scalefac/.store in=\scalefac,
  Linewidth/.store in=\Linewidth,
  picname/.store in=\picname,
  channelcolor=BrownLine,
  drawchannelcolor=BrownLine,
  scalefac=1,
  Linewidth=0.5pt,
  picname=C
}
\node[Circle](MO){};
\node[Circle,below right=0.85 and 2.5 of MO,draw=GreenLine,fill=GreenL!40,](IN){};
\node[Circle,below left=0.85 and 2.5 of MO,draw=OrangeLine,fill=OrangeL!40,](DA){};
\draw[ALineA](MO)--(IN);
\draw[ALineA](MO)--(DA);
\draw[ALineA](DA)--(IN);
\node[below=2pt of MO]{Algorithm};
\node[below=2pt of IN]{Machine};
\node[below=2pt of DA]{Data};
%%
\begin{scope}[local bounding box=CIRCLE1,shift={($(MO)+(0.04,-0.24)$)},
scale=0.55, every node/.append style={transform shape}]
%1 column
\foreach \j in {1,2,3} {
  \pgfmathsetmacro{\y}{(1.5-\j)*0.43 + 0.7}
  \pic at (-0.8,\y) {circles={channelcolor=RedLine,picname=1CD\j}};
}
%2 column
\foreach \i in {1,...,4} {
  \pgfmathsetmacro{\y}{(2-\i)*0.43+0.7}
  \pic at (0,\y) {circles={channelcolor=RedLine, picname=2CD\i}};
}
%3 column
\foreach \j in {1,2} {
  \pgfmathsetmacro{\y}{(1-\j)*0.43 + 0.7}
  \pic at (0.8,\y) {circles={channelcolor=RedLine,picname=3CD\j}};
}
\foreach \i in {1,2,3}{
  \foreach \j in {1,2,3,4}{
\draw[Line](1CD\i)--(2CD\j);
}}
\foreach \i in {1,2,3,4}{
  \foreach \j in {1,2}{
\draw[Line](2CD\i)--(3CD\j);
}}
\end{scope}
%
\pic[shift={(-0.4,-0.08)}] at (IN) {cloud};
%
\pic[shift={(-0.05,-0.13)}] at  (IN){streaming={scalefac=0.25,picname=2,channelcolor=RedLine, Linewidth=0.65pt}};
%
\pic[shift={(0,-0.3)}] at  (DA){data={scalefac=0.3,picname=1,channelcolor=green!70!black, Linewidth=0.4pt}};
\end{tikzpicture}}

Figure 1: The D·A·M Axis Triad: A compact diagnostic view of Data, Algorithm, and Machine as three interdependent axes. The bidirectional links emphasize that an intervention on one axis changes the constraints on the other two.

With the first-pass axis identified, table 2 expands the pairwise and three-way intersections into concrete engineering techniques and their primary book chapters.

Table 2: D·A·M Intersection Reference: Each zone maps specific techniques to the axes they span and the chapters that cover them. The pairwise intersections require reasoning about two domains simultaneously; the center requires all three.

Zone	Name	Key Techniques	Book Coverage
Data	Information	Storage formats, data quality, distributions	Data Engineering
Algorithm	Logic	Loss functions, architectures, gradients	Neural Computation, Network Architectures
Data \(\cap\) Algorithm	What to Learn From	Data selection, curriculum learning, compute-optimal scaling	Data Selection, Model Training
Data \(\cap\) Machine	How to Move Information	I/O bandwidth, prefetching, data formats	Data Engineering, Hardware Acceleration
Algorithm \(\cap\) Machine	How to Execute Efficiently	Quantization, pruning, kernel fusion, mixed precision	ML Frameworks, Model Compression
Machine	Physics	Silicon, memory hierarchy, peak FLOP/s	Hardware Acceleration
Data \(\cap\) Algorithm \(\cap\) Machine	ML Systems Engineering	Iron law, Roofline, training loops, serving	Model Training, Model Serving, Benchmarking

In the D·A·M acronym, Data, Algorithm, and Machine are taxonomy axes. They are not mathematical variables; formal quantities still follow the notation chapter, where \(D\) denotes dataset size or training tokens.

The axis-centered zones contain concepts that belong primarily to one axis: storage formats and distributions emphasize Data, loss functions and gradients emphasize Algorithm, and silicon physics and peak FLOP/s emphasize Machine. Single-domain expertise can introduce them, but deployed behavior may still depend on the other axes.

The pairwise intersections are where systems thinking begins. \(\mathsf{Data} \cap \mathsf{Algorithm}\) (What to Learn From) encompasses data selection, curriculum learning, active learning, and scaling laws like Chinchilla (\(D \approx 20P\) for the dense autoregressive language-model regime studied) (Hoffmann et al. 2022)—all requiring joint reasoning about information content and algorithmic capacity. Adding data without considering whether the model can learn from it wastes compute; choosing architectures without considering data availability wastes engineering time. \(\mathsf{Data} \cap \mathsf{Machine}\) (How to Move Information) covers I/O bandwidth, prefetching strategies, data formats, and the energy-movement invariant. This intersection is where data gravity manifests: the physical cost of moving bytes through the memory hierarchy determines whether the machine can be fed fast enough. \(\mathsf{Algorithm} \cap \mathsf{Machine}\) (How to Execute Efficiently) spans quantization, pruning, kernel fusion, mixed precision, and computational graph optimization. A pruning strategy that reduces FLOPs but destroys memory access patterns can slow down execution on real hardware.

The center—\(\mathsf{Data} \cap \mathsf{Algorithm} \cap \mathsf{Machine}\)—is where all three axes converge. The iron law, the Roofline Model, end-to-end training loops, serving pipelines, and holistic benchmarking all require simultaneous reasoning about data flow, algorithmic complexity, and hardware utilization. This center is not a single technique; it is the discipline itself.

Understanding the landscape reveals where a technique lives. The next step is quantifying which axis dominates for a given workload—and that requires the iron law.

Iron Law Mapping

For serialized phases, the iron law maps work across the D·A·M axes to execution time. Here, \(T\) and \(L_{\text{lat}}\) are times in seconds; \(D_{\text{vol}}\) is bytes moved and \(\text{BW}\) is bytes per second; \(O\) is work in FLOPs, \(R_{\text{peak}}\) is FLOP/s, and \(\eta_{\text{hw}}\) is dimensionless hardware efficiency. These quantities map to the axes as follows: \[ T = \underbrace{ \frac{D_{\text{vol}}}{\text{BW}} }_{\text{Data/Machine}} + \underbrace{ \frac{O}{R_{\text{peak}} \cdot \eta_{\text{hw}}} }_{\text{Algorithm/Machine}} + \underbrace{ L_{\text{lat}} }_{\text{Cross-axis overhead}} \]

Algorithm and Machine share the compute term, separated by which variable the engineer controls. Reducing the total operations (\(O\)) is an Algorithm lever, while improving the hardware’s peak throughput (\(R_{\text{peak}}\)) or utilization (\(\eta_{\text{hw}}\)) is a Machine lever.

This equation transforms performance debugging from a qualitative guessing game into a quantitative engineering problem. The dominant measured cost appears in one of these terms. A slow system may move too much data (\(D_{\text{vol}}\)), lack bandwidth (\(\text{BW}\)), execute too many operations (\(O\)), fail to use the hardware’s peak capability (\(\eta_{\text{hw}}\)), or pay cross-axis overhead (\(L_{\text{lat}}\)). The levers below map specific optimizations to the variables they improve.

Component levers

Data Lever: Reducing moved bytes (\(D_{\text{vol}}\)) through deduplication, selection, or lower-precision representation, or increasing I/O bandwidth (\(\text{BW}\)).
Algorithm Lever: Reducing operations (\(O\)) through pruning or architectural refinement. Quantization narrows representation and may raise hardware throughput; it does not generally reduce operation count.
Machine Lever: Increasing the denominator of the compute term by improving peak throughput (\(R_{\text{peak}}\)) or increasing the utilization factor (\(\eta_{\text{hw}}\)) via kernel fusion.

D·A·M coordination: From sum to max

The additive iron law represents sequential execution. Let \(T_{\text{sequential}}\) denote time without overlap and \(T_{\text{pipelined}}\) denote measured time with overlap. Overlap can move the sum toward a max, but dependencies, contention, and incomplete overlap keep the measured time above this ideal lower bound: \[ T_{\text{sequential}} = \frac{D_{\text{vol}}}{\text{BW}} + \frac{O}{R_{\text{peak}} \cdot \eta_{\text{hw}}} + L_{\text{lat}} \quad \xrightarrow{\text{overlap}} \quad T_{\text{pipelined}} \geq \max\left(\frac{D_{\text{vol}}}{\text{BW}}, \frac{O}{R_{\text{peak}} \cdot \eta_{\text{hw}}}\right) + L_{\text{lat}} \]

The systems engineer’s job is to make these components run in parallel, not in series. Table 3 summarizes key D·A·M coordination techniques that overlap work across axes:

Table 3: D·A·M Overlap Techniques: Each technique overlaps work across axes, moving sequentially added Data/Machine and Algorithm/Machine time toward their ideal lower bound: the larger of the two terms. Dependencies and contention limit realized overlap.

Technique	D·A·M Axes Overlapped	Implementation
Prefetching	D overlaps M	DataLoader with `prefetch_factor`, `pin_memory=True`
CUDA Streams	D overlaps M	Separate streams for H2D transfer and compute
Async Gradient Sync	M (communication) overlaps A	Overlap bucketed AllReduce with remaining backward computation
Double Buffering	D overlaps M	Fill buffer N+1 while computing on buffer N

Overlaps provide the greatest relative speedup when the D·A·M time terms are reasonably balanced. If one term dominates (for example, severely memory bound), overlapping the smaller term with the larger yields negligible gain—the max is still dominated by the same bottleneck. This balance occurs near \(D_{\text{vol}}/\text{BW} \approx O/(R_{\text{peak}} \cdot \eta_{\text{hw}})\). The latency term requires separate treatment.

Systems Perspective 1.1: The overhead that cannot hide

The unhidden latency term \(L_{\text{lat}}\) (kernel launch, synchronization barriers, Python dispatch) contains serialization points that cannot be fully overlapped. Kernel fusion can reduce this term by combining operations, but it does not eliminate all launch, scheduling, or synchronization overhead.

The iron law tells the engineer how much time each term consumes. One critical question remains: does data movement or arithmetic throughput set the active ceiling? The answer lies in a single ratio.

Arithmetic Intensity Boundary

The boundary between the Data/Machine memory term and the Algorithm/Machine compute term is not arbitrary; it is defined mathematically by arithmetic intensity¹ (\(I\)) of the workload.

¹ Arithmetic intensity: This book uses the term for floating-point operations per byte transferred. Williams et al. (2009) calls the measured DRAM quantity operational intensity in the Roofline Model. Comparing the workload ratio with the hardware ridge point (\(R_{\text{peak}}/\text{BW}\)) gives an upper-bound diagnosis of memory- versus compute-bound execution.

Williams, Samuel, Andrew Waterman, and David Patterson. 2009. “Roofline: An Insightful Visual Performance Model for Multicore Architectures.” Communications of the ACM 52 (4): 65–76. https://doi.org/10.1145/1498765.1498785.

The Roofline Model provides an upper performance bound when traffic and achieved rates are measured consistently. In the middle of a production incident, faster screening heuristics can point to the measurements to collect next.

Rules of Thumb

In the heat of a production outage, there is rarely time to solve the full iron law equation. Veteran systems engineers instead rely on quantitative heuristics to quickly narrow the search space; the thresholds below are screening signals that should be checked against profiler traces and hardware counters.

Low accelerator utilization (\(<\) 80 percent): Screen for data, CPU, or launch starvation. Confirm with input-pipeline wait, host CPU saturation, memory-bandwidth counters, and trace gaps.
High accelerator utilization (\(>\) 95 percent): Treat this as machine bound only if compute units are busy and memory bandwidth is not saturated. If memory bandwidth is saturated, the bottleneck is still on the Data/Machine boundary.
If batch size is one: Treat this as a warning that latency or launch overhead may matter; confirm with traces before labeling the algorithm itself the bottleneck.
Low arithmetic intensity (below the hardware ridge point): The workload is likely memory bound (Data/Machine boundary). The precise boundary is \(R_{\text{peak}}/\text{BW}\) and depends on hardware and precision.
If the system works in dev but fails in prod: Treat data drift as one hypothesis alongside configuration, load, dependency, and runtime differences.

Common industry labels cross the axes. Memory-bound execution sits on the Data/Machine boundary because both bytes moved and available bandwidth matter. Compute-bound execution couples Algorithm work with Machine throughput. Latency-bound execution may reflect algorithmic serial depth, runtime dispatch, synchronization, or queueing.

Bottleneck diagnostic

Once profiling identifies the dominant term in the iron law, optimizations must target that specific axis. Table 4 contrasts the interventions that yield end-to-end speedups against nonbinding optimizations that waste engineering effort:

Table 4: What Works vs. What Is Wasted: Optimizing a nonbinding term yields little or no end-to-end improvement. A memory-bound large language model will not benefit from additional peak FLOP/s alone, though an accelerator with more memory bandwidth may help.

If the workload is…	Dominant Term	Optimization That Works	Optimization That is Wasted
Memory-Bound	\(D_{\text{vol}}/\text{BW}\)	Quantization; structured pruning; batching for weight reuse; fusion that removes intermediate traffic	Faster accelerator (more FLOP/s will not help)
Compute-Bound	\(O/(R_{\text{peak}} \cdot \eta_{\text{hw}})\)	Better kernels, Tensor Cores, faster accelerator, lower precision	More memory bandwidth (not the binding term)
Latency-Bound	\(L_{\text{lat}}\)	Reduce serial depth; fuse or dispatch asynchronously; batch within budget	More compute or bandwidth unless the trace shows it binding

Knowing what works also means recognizing what does not. In practice, teams under deadline pressure repeatedly fall into the same traps—optimizing the wrong axis with confidence. These failure modes are common enough to deserve their own names.

Anti-Patterns

Diagnosing systems is often a process of elimination. Before committing to complex kernel optimizations, watch for these common traps that waste engineering cycles.

The hardware crutch: Buying faster accelerators (Machine) to fix a slow Python data loader (Data). The new hardware will just idle faster.
The model twiddle: Changing neural architectures (Algorithm) when the bottleneck is actually network bandwidth or disk I/O.
The premature optimizer: Writing custom CUDA kernels (Machine) before verifying if the Algorithm is simply doing too many unnecessary operations.

Each anti-pattern follows the same root cause: acting before diagnosing. The following case studies show what proper diagnosis looks like—starting from a confusing symptom and systematically narrowing to the dominant D·A·M axis.

D·A·M Case Studies

Theoretical constraints often manifest as confusing symptoms in production. These representative scenarios illustrate how to apply the taxonomy. Each case follows the same three diagnostic moves: symptom, diagnosis, and fix.

Case 1: The starving accelerator (Data)

A team provisions a large A100 GPU instance to speed up training, but training time hardly improves and nvidia-smi shows GPU utilization fluctuating between 10 percent and 40 percent. The pattern suggests accelerator starvation but does not identify its cause. If traces show input-wait gaps with storage or CPU decoding saturation, the data path is binding. The useful intervention is then upstream of the model. Optimize the extract, transform, load (ETL) path by moving from raw JPEGs with heavy CPU decoding to sequential formats such as TFRecords or WebDataset, increasing data-loader parallelism, and prefetching batches into accelerator memory.

Case 2: The latency cliff (cross-axis)

A real-time recommendation service fails to meet a 20 ms latency service level agreement (SLA), while accelerator utilization is low and the batch size is one. That combination suggests launch overhead, memory traffic, or serial model depth rather than a saturated chip. If a trace confirms that the sequential layer path dominates, adding more hardware will not remove it, so pruning or knowledge distillation can reduce model work. If weight traffic dominates instead, quantization can reduce moved bytes by using INT8 where accuracy allows. The measurements select the remedy.

Case 3: The compute wall (Machine)

Accelerator utilization is pinned at 99 percent, memory bandwidth remains unsaturated, and training is stable but takes three weeks. If profiler counters also show high compute-pipeline occupancy and model FLOPs utilization (MFU), the workload is compute bound. The data path is keeping the accelerator fed, so the next step must change the compute term in the iron law. The team can scale up from an A100 to an H100-class accelerator, scale out through data parallelism, or lower precision from FP32/TF32 to BF16 where numerically safe. On NVIDIA Tensor Core paths, BF16 peak throughput is typically about 2\(\times\) TF32 peak, with realized speedup depending on kernels and bottlenecks (Choquette 2023). Taken together, the three cases show why the same utilization number can imply different next steps depending on loss behavior, batch size, and memory pressure.

Choquette, Jack. 2023. “NVIDIA Hopper H100 GPU: Scaling Performance.” IEEE Micro 43 (3): 9–17. https://doi.org/10.1109/mm.2023.3256796.

Checkpoint 1.1: D·A·M diagnosis check

A training job shows 95 percent accelerator utilization but loss has plateaued for two epochs. Which D·A·M axes could be responsible, and why is utilization alone insufficient to distinguish them?
Your colleague suggests adding more data loader workers to a job where nvidia-smi shows 98 percent GPU utilization. What trace evidence is needed before deciding whether this will help?
An inference server meets its latency SLO at batch size 1 but fails at batch size 16. Which iron-law terms and queueing effects may have changed, and which measurements identify the binding regime?

These three cases illustrate bottlenecks that can cross axes. Production incidents are rarely so tidy—symptoms often overlap, and the dominant axis can shift during debugging. The next section provides a systematic troubleshooting matrix for the messier scenarios encountered in practice.

Production Troubleshooting

Identifying the root cause of performance bottlenecks requires systematic elimination. Table 5 provides a diagnostic matrix for common failure modes observed in production deployments.

The diagnostic matrix indicates what to suspect. The next question is how to confirm that suspicion with evidence—which requires the right profiling tools.

Table 5: D·A·M Diagnostic Matrix: First hypotheses and measurements for common failures. Confirm the binding component before intervening.

Symptom	Hypothesis	Confirm With	Act After Confirmation
Low Accelerator Utilization	Data/host starvation	Trace gaps; loader, CPU, storage, launch counters	Prefetch, parallelize, or cut launches.
High Latency (P99)	Algorithm/runtime/queue	Phase p99 and queue depth	Optimize the dominant phase.
High Training Cost	Low useful-work efficiency	MFU, communication, idle time	Reduce the largest loss or resize.
High Inter-GPU Overhead	Communication bound	Device trace; collective gaps	Bucket AllReduce; tune data-parallel overlap.
Out-of-Memory (OOM)	Algorithm/Machine fit	Peak weights, states, activations	Checkpoint, shard, quantize, or shrink batch.

Tooling Map

Once a hypothesis exists (for example, “the workload appears Machine-bound”), evidence is needed to confirm it. Abstract concepts must be measured with concrete utilities. Table 6 connects the theoretical components to the specific Linux and Python profiling tools that confirm them.

Table 6: D·A·M Tooling Map: Profiling utilities for diagnosing bottlenecks along each D·A·M axis. Start with the primary tool for quick triage; use secondary tools for deep-dive analysis when the primary tool’s output is inconclusive.

Axis	Key Metric	Primary Tool	Secondary Tool
Data	Batch Load and Wait Time	Framework profiler trace	`iostat`, `pidstat` (I/O/CPU)
Algorithm	FLOPs, Model Depth	PyTorch Profiler	DeepSpeed Flops Profiler
Machine	Accelerator Utilization, SM Occupancy	Nsight Compute	`nvidia-smi`, Nsight Systems

Profiling tools generate raw numbers—utilization percentages, FLOP counts, and bandwidth measurements. They become actionable only in workload and hardware context. The D·A·M Scorecard provides screening signals rather than universal standards, narrowing which measurements and interventions deserve attention.

D·A·M Scorecard

To move beyond qualitative guessing, the efficiency indicators in table 7 help decide what to investigate next. This report-card view aids the first comparison, but no row provides a universal grade. MFU² is especially useful for large-model training.

² MFU (model FLOPs utilization): The ratio of achieved model FLOP/s to the hardware’s theoretical peak FLOP/s, introduced in the PaLM paper (Chowdhery et al. 2022). Unlike raw accelerator utilization (which counts any work the accelerator performs), MFU measures model computation rate relative to peak hardware throughput, so non-model work and system overhead are not counted as achieved model FLOPs. Identifying Bottlenecks develops MFU as a training diagnostic.

Chowdhery, Aakanksha, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, et al. 2022. “PaLM: Scaling Language Modeling with Pathways.” arXiv Preprint arXiv:2204.02311.

Table 7: D·A·M Screening Indicators: Context-dependent signals for investigation, not universal grades.

Axis	Metric	Definition	Investigate	Strong Signal
Data	I/O Overhead	\(\dfrac{\text{Data Wait Time}}{\text{Total Step Time}}\)	\(>\) 10% (screen)	\(<\) 1% (strong)
Algorithm	Avoidable Work	Profiler or ablation finds redundant operations	Redundant work found	End-to-end work falls
Machine	MFU	\(\dfrac{\text{Achieved model FLOP/s}}{\text{Peak FLOP/s}}\)	\(<\) 30% (large models)	\(>\) 50% (same regime)

The Scorecard and the Roofline Model both answer efficiency questions, but at different scales. The Scorecard screens the current system using context-dependent indicators. Scaling laws and the information-roofline metaphor ask what may happen as the system scales beyond its current size.

Scaling Laws vs. Roofline

Systems engineering requires distinguishing between growth trajectories and fundamental limits.

Scaling laws (the journey)

Scaling laws³ are empirical power laws that predict how held-out loss changes as resources increase within the fitted regime. Two landmark results are Kaplan scaling (Kaplan et al. 2020), which studied loss against parameter count (\(P\)), data (\(D\)), and total operations (\(O\)), and Chinchilla scaling (Hoffmann et al. 2022), which estimated a compute-optimal balance for the dense autoregressive language-model regime studied, often summarized as roughly 20 training tokens per parameter (\(D \approx 20P\)).

³ Scaling laws: Empirical relationships, typically power laws with a fitted negative exponent, that predict model loss as a function of dataset size, parameter count, or compute budget. Kaplan et al. (2020) studied these relationships for neural language models at OpenAI; Hoffmann et al. (2022) later estimated the compute-optimal training trade-off for its studied dense language-model regime. The compute-optimal frontier develops the compute-optimal frontier and its data-starvation diagnostic in detail.

Kaplan, J., S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei. 2020. “Scaling Laws for Neural Language Models.” ArXiv Preprint abs/2001.08361.

Hoffmann, Jordan, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, et al. 2022. “Training Compute-Optimal Large Language Models.” Advances in Neural Information Processing Systems (NeurIPS) 35: 30016–30. https://doi.org/10.52202/068431-2176.

As economic guides, these fits estimate how held-out loss changes with compute inside the observed regime. They do not guarantee a task error-rate reduction or extrapolate unchanged across architectures and data distributions.

Information roofline (the destination)

The information roofline is used here as a diagnostic metaphor rather than a standard quantitative law. Its destination image points to limits on what can be learned from the available data. For classification under a fixed data distribution and loss, the Bayes error rate⁴ is an irreducible error floor, not a ceiling. Noise, label ambiguity, coverage gaps, and distribution shift can create data-quality limits, but they do not define one universal slope or breakpoint.

⁴ Bayes error rate: The lowest achievable error rate under 0–1 loss for any classifier on a given data distribution, determined by class priors and the overlap between class-conditional distributions (Goodfellow et al. 2016). No amount of data, parameters, or compute can reduce error below this theoretical floor.

Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press.

The diagnostic lesson is narrower. Scaling laws describe a fitted improvement trajectory, while the information-roofline metaphor prompts a search for data-quality limits. If a loss curve flattens relative to the fitted trend, investigate optimization, capacity, evaluation noise, and data quality; flattening alone does not identify which cause dominates. The Bayes floor describes the best achievable classifier for the specified distribution. Adding accelerators or parameters is futile only after measurements establish that data quality is binding. Whether debugging a training step, evaluating hardware utilization, or planning a scaling campaign, measure which axis is binding before choosing a lever.

Summary

These tools form a single diagnostic workflow. D·A·M localizes candidate causes across Data, Algorithm, and Machine. The iron law and arithmetic intensity translate those causes into time, operations, and bytes, while the case studies, anti-patterns, tooling map, and scorecard show how to test them in production. Scaling laws and the information roofline then separate observed improvement trajectories from data-quality limits. Use the framework as a loop rather than as a label.

Key Takeaways: Where to look first

Begin with cross-axis hypotheses; a symptom rarely identifies its cause.
Translate each hypothesis into the relevant iron-law term and a measurement that could falsify it.
Use arithmetic intensity and the scorecard to identify the binding constraint, not to assign universal grades.
Apply the corresponding lever and remeasure end to end; a local gain that leaves the binding term unchanged is not a system improvement.