Warm-Up, Cache State, and Benchmark Design
A benchmark measures an experimental setup, not an abstract function. Warm-up, input shape, compiler behavior, cache state, and noise determine what the result means.
Planned model
Run the same operation with cold and warm code, cache, allocator, and pages. Show setup cost, steady state, optimization artifacts, and sample variance.
Questions
- Is the target workload cold, warm, or a known mixture?
- Has the compiler removed or transformed the measured work?
- Which environmental variables must be recorded or controlled?
Exercise
Write a benchmark protocol for one container operation that distinguishes typical cost from resize and cold-page slow paths.