Decoupled Vector Runahead for Prefetching Nested Memory-Access Chains

Abstract

Decoupled vector runahead (DVR) exploits massive amounts of memory-level parallelism to improve the performance of applications that feature indirect memory accesses by dynamically inferring loop bounds at runtime, recognizing striding loads, and speculatively vectorizing the subsequent instructions that are part of an indirect chain. DVR runs as an on-demand, speculative, in-order, lightweight hardware subthread alongside the main thread within the core. DVR incurs minimal hardware overhead while delivering a substantial performance boost.