Abstract

In this talk we will present new generic paradigms for conducting machine learning computations with structured linear projections. These paradigms can be applied to speed up several machine learning algorithms based on random linear projections such as: cross-polytope LSH techniques, kernel approximations via random feature maps, quantization methods using random projection trees, several variants of Johnson-Lindenstrauss transforms, and many more.

Their adaptive versions can be applied in neural network architectures to construct much more compact and faster yet still good quality neural network models. As a byproduct, we give the first theoretical guarantees regarding the fastest known cross-polytope LSH methods based on the Walsh-Hadamard Transform.

The proposed structured families of P-model and TripleSpin matrices cover as special cases all structured matrices used so far in this context, but also include new structured constructions not considered previously.

Bio: Krzysztof Choromanski is a member of the Google Brain Robotics Team in New York. He works on several problems regarding robotics and machine learning such as: reinforcement learning, control theory, state estimation, predictive state representation and Hilbert embeddings of dynamical systems. His research interests include also neural networks and the theory of structured nonlinear embeddings. The latter are used in several machine learning applications such as: compact neural networks with fast inference, kernel approximation techniques via random feature maps, and the fastest known cross-polytope LSH algorithms. Krzysztof plays piano and is an avid salsa dancer.