high performance computing on graphics processing units: hgpu.org

From the standpoint of a computer engineer there are (at least) two ways to improve the execution time of an algorithm. First, one might build sequential processing units with increased speed (this is most common in CPUs, although those have also incorporated parallel processing paradigms), while the second alternative is to build a massive number of processing units into a single integrated circuit (this is most common in GPUs). Although this is hardly a new concept (supercomputers were employing thousands of processors for some time) to integrate them on a single chip with a unified global memory is. From a purely theoretical standpoint, as is taken in complexity theory, a multitude of processing units is not in any way superior to a single processor. However, in practical applications the constants do matter and thus it is a viable option to parallelize an algorithm to different processing units. Nevertheless, from a theoretical standpoint it is interesting to investigate to what extend we can accelerate an algorithm by distributing it to many processing units.