Elider (µ-architecture)

Recently, technological CPU performance improvements have considerably slowed, and single-core performance improvements have effectively stopped altogether. Technological advancements, which allow more transistors per unit area, are now used to increase the on-chip buffer sizes, such as caches, TLBs, re-order buffers, and branch predictors. Increasing the buffer sizes enables greater tolerance for slow memory accesses, which gives some performance improvement, but does not reduce the total work done by the processor.  Given the diminishing returns in IPC improvement of this traditional approach, this proposal suggests hardware memoization as a radical alternative that can use additional transistors to achieve significant IPC improvements in future processors. The preliminary study in this proposal and prior work by the PI support the viability of this direction.

Prior attempts at hardware memoization demonstrated modest performance gains, and this work proposes two major approach improvements. First, a large opportunity exists in identifying which prior execution from a long track history is recurring, rather than memoizing only for the most-recent occurrence.  Secondly, much higher performance gains are expected by focusing on speculative memoization. This work will primarily target modern server software and is expected to demonstrate significant benefits of memoization for both desktop and server applications.

This project develops a practical hardware memoization technique for modern processors. The team will develop hardware techniques to track the
instructions being executed and identify which instructions are repeating a prior execution and precisely which prior execution is being repeated. Next, the mechanisms to speculatively utilize the result of the prior execution of the instructions will be developed, verifying the memoization when the current input
values are matched with the values of the prior execution. Finally, the team will study and work to mitigate the compiler and system software effects on the proposed memoization techniques.