Скачать с ютуб SC19 Awards Presentation: SC Test of Time Award - James Demmel, UC Berkeley в хорошем качестве

SC19 Awards Presentation: SC Test of Time Award - James Demmel, UC Berkeley

The SC Test of Time Award recognizes a paper from a past SC conference that has deeply influenced the HPC discipline. It is a mark of historical impact, and requires clear evidence that the paper has changed HPC trends. On Tuesday, November 19, at SC19 in Denver, SC Test of Time Award winner James Demmel presented to SC19 attendees (co-author Vasily Volkov was not in attendance): "From Dense Linear Algebra to GPU Occupancy (Published at SC08)" The CUDA programming model was introduced in 2007 and featured a number of new concepts, such as occupancy and shared memory. In this work, we considered performance implications of these concepts in application to dense matrix factorizations. Our findings were contrarian to the widely accepted recommendations of the day. (i) We found a performance optimization pattern that leads to lower occupancy, whereas it was recommended to maximize occupancy in order to hide memory latencies. (ii) We found that instruction-level parallelism contributes to latency hiding on GPUs, which was believed to be not the case. (iii) We found that performance can be improved by using massive register blocking, whereas it was recommended to minimize register use to maximize occupancy. (iv) We found that shared memory is slower than registers and the use of the latter should be favored over the former, if possible. These novel insights led us to a design of the matrix multiply routine that substantially outperformed the state-of-the-art vendor BLAS library. The optimization pattern we pointed out is found today in many high-performance GPU codes. — SC19: The International Conference for High Performance Computing, Networking, Storage, and Analysis https://sc19.supercomputing.org