high performance computing on graphics processing units: hgpu.org

In "synchronous" PSO, positions and velocities of all particles are updated in turn in each "generation", after which each particle’s new fitness is evaluated. The value of the social attractor is only updated at the end of each generation, when the fitness values of all particles are known. The "asynchronous" version of PSO, instead, allows the social attractors to be updated immediately after evaluating each particle’s fitness, which causes the swarm to move more promptly towards newly-found optima. In asynchronous PSO, the velocity and position update equations can be applied to any particle at any time, in no specific order. The most common GPU implementations of PSO assign one thread per particle and do not take full advantage of the GPU power in evaluating the fitness function in parallel. Parallelization only occurs on the number of particles of a swarm and ignores the dimensions of the function. In our parallel implementations: (i) we designed the thread parallelization to be as fine-grained as possible, considering that, in PSO, velocity and position update occur independently over each dimension; (ii) we implemented an "asynchronous" PSO which, despite updating all particles in parallel, allows each of them to update the social attractor without waiting for all other particles’ fitness values to be evaluated.