Alpha-Beta Divergences Discover Micro and Macro Structures in Data

Karthik Narayan, Ali Punjani, Pieter Abbeel
University of California, Berkeley, CA, 94720, USA
The 32nd International Conference on Machine Learning, 2015


   title={Alpha-Beta Divergences Discover Micro and Macro Structures in Data},

   author={Narayan, Karthik and Punjani, Ali and Abbeel, Pieter},

   booktitle={Proceedings of The 32nd International Conference on Machine Learning},




Download Download (PDF)   View View   Source Source   



Although recent work in non-linear dimensionality reduction investigates multiple choices of divergence measure during optimization (Yang et al., 2013; Bunte et al., 2012), little work discusses the direct effects that divergence measures have on visualization. We study this relationship, theoretically and through an empirical analysis over 10 datasets. Our works shows how the alpha and beta parameters of the generalized alpha-beta divergence can be chosen to discover hidden macrostructures (categories, e.g. birds) or microstructures (fine-grained classes, e.g. toucans). Our method, which generalizes t-SNE (van der Maaten, 2008), allows us to discover such structure without extensive grid searches over (alpha, beta) due to our theoretical analysis: such structure is apparent with particular choices of (alpha, beta) that generalize across datasets. We also discuss efficient parallel CPU and GPU schemes which are non-trivial due to the tree-structures employed in optimization and the large datasets that do not fully fit into GPU memory. Our method runs 20x faster than the fastest published code (Vladymyrov & Carreira-Perpinan, 2014). We conclude with detailed case studies on the following very large datasets: ILSVRC 2012, a standard computer vision dataset with 1.2M images; SUSY, a particle physics dataset with 5M instances; and HIGGS, another particle physics dataset with 11M instances. This represents the largest published visualization attained by SNE methods. We have open-sourced our visualization code.
No votes yet.
Please wait...

* * *

* * *

HGPU group © 2010-2021 hgpu.org

All rights belong to the respective authors

Contact us: