high performance computing on graphics processing units: hgpu.org

Simulation of biological spiking networks is becoming more relevant in understanding neuronal processes. An increasing proportion of these simulations focuses on large scale modeling efforts. Unfortunately the size of large networks is often limited by both computational power and memory. Computational power constrains both the maximum number of differential equations and the maximum number of spikes that can be processed per unit time. Memory size limits the maximum number of neurons and synapses that can be simulated. To solve for the computational bottleneck, a neuronal simulator is implemented on a CUDA-based General Purpose Graphic Processing Unit (GPGPU). CUDA provides a C-like environment to harness the computational power of specialized video cards from NVIDIA (these cards provide a computational peak power on single precision floats of 1TeraFLOPS, at least an order of magnitude higher than the fastest CPU). To solve for the memory bottleneck, a just-in-time synapse storing algorithm is implemented requiring only 4 bytes per synapse. Only the synaptic weight is stored, while both post-synaptic contact and delay are recomputed at run-time. This allows a resource shift from memory to computation which fits with the peculiar GPGPU architecture, where an abundance of compute nodes access the memory via a bandwidth-limited bus. Neurons are represented by a single compartment whose activity is modeled by the Izhikevich formalism. Excitatory synapses are plastic and follow both a spike-timing-dependent plasticity rule and a short term potentiation/depression rule. We are able to simulate networks with million of neurons and hundreds million of synapses on a single GPGPU card. For smaller networks the speedup obtained is at least of an order of magnitude compared to traditional CPU platform. In conclusion we present a novel simulator that is fast, synaptically optimized and anatomically accurate. At an additional cost to an available desktop PC of few hundred dollars the GPGPU is an efficient platform to simulate large spiking networks.