BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY: Lie Group Machine Learning and Natural Gradient from Information 
 Geometry - Dr Frederic Barbaresco\, THALES Land and Air Systems
DTSTART:20191204T140000Z
DTEND:20191204T150000Z
UID:TALK135115@talks.cam.ac.uk
CONTACT:Prof. Ramji Venkataramanan
DESCRIPTION:The classical simple gradient descent used in Deep Learning ha
 s two drawbacks: the use of the same non-adaptive learning rate for all pa
 rameter components\, and a non-invariance with respect to parameter re-enc
 oding inducing different learning rates. As the parameter space of multila
 yer networks forms a Riemannian space equipped with Fisher information met
 ric\, instead of the usual gradient descent method\, the natural gradient 
 or Riemannian gradient method\, which takes account of the geometric struc
 ture of the Riemannian space\, is more effective for learning. The natural
  gradient preserves this invariance to be insensitive to the characteristi
 c scale of each parameter direction. The Fisher metric defines a Riemannia
 n metric as the Hessian of two dual potential functions (the Entropy and t
 he Massieu Characteristic Function).\n\nIn Souriau’s Lie groups thermody
 namics\, the invariance by re-parameterization in information geometry has
  been replaced by invariance with respect to the action of the group. In S
 ouriau model\, under the action of the group\, the entropy and the Fisher 
 metric are invariant. Souriau defined a Gibbs density that is covariant un
 der the action of the group. The study of exponential densities invariant 
 by a group goes back to the work of Muriel Casalis in her 1990 thesis. The
  general problem was solved for Lie groups by Jean-Marie Souriau in Geomet
 ric Mechanics in 1969\, by defining a "Lie groups Thermodynamics" in Stati
 stical Mechanics. These new tools are bedrocks for Lie Group Machine Learn
 ing. Souriau introduced a Riemannian metric\, linked to a generalization o
 f the Fisher metric for homogeneous Symplectic manifolds. This model consi
 ders the KKS 2-form (Kostant-Kirillov-Souriau) defined on the coadjoint or
 bits of the Lie group in the non-null cohomology case\, with the introduct
 ion of a Symplectic cocycle\, called "Souriau's cocycle"\, characterizing 
 the non-equivariance of the coadjoint action (action of the Lie group on t
 he moment map).\n\nWe will introduce the link between Souriau "Lie Groups 
 Thermodynamics"\, Information Geometry and Kirillov representation theory 
 to define probability densities as Souriau covariant Gibbs densities (dens
 ity of Maximum of Entropy). We will illustrate this case for the matrix Li
 e group SU (1\,1) (case with null cohomology)\, and the one for the matrix
  Lie group SE(3) (case with non-null cohomology)\, through the computation
  of Souriau’s moment map\, and Kirillov's orbit method.\n\n\n*BIO*: F. B
 arbaresco received his State Engineering degree from the French Grand Ecol
 e CENTRALE-SUPELEC\, Paris\, France\, in 1991. Since then\, he has worked 
 for the THALES Group where he is now SENSING Segment Leader of Key Technol
 ogy Domain PCC (Processing\, Control & Cognition). He has been an Emeritus
  Member of SEE since 2011 and he was awarded the Aymé Poirson Prize (for 
 application of sciences to industry) by the French Academy of Sciences in 
 2014\, the SEE Ampere Medal in 2007\, the Thévenin Prize in 2014 and the 
 NATO SET Lecture Award in 2012. He is President of SEE Technical Club ISIC
  “Engineering of Information and Communications Systems” and a member 
 of the SEE administrative board. He is member of the administrative board 
 of SMAI and GRETSI. He was an invited lecturer for UNESCO on “Advanced S
 chool and Workshop on Matrix Geometries and Applications” in Trieste at 
 the ITCP in June 2013. He is the General Co-chairman of the new internatio
 nal conference GSI “Geometric Sciences of Information”. He was co-edit
 or of MDPI Entropy Books “Information\, Entropy and Their Geometric Stru
 ctures” and "Joseph Fourier 250th Birthday: Modern Fourier Analysis and 
 Fourier Heat Equation in Information Sciences for the XXIst century". He h
 as co-organized the CIRM seminar TGSI’17 “Topological and Geometrical 
 Structures of Information” and “FGSI’19 Cartan-Koszul-Souriau” in 
 2019. He was keynote speaker at SOURIAU’19 event.
LOCATION:LT6\, Baker Building\, CUED
END:VEVENT
END:VCALENDAR