Abstract

Information Measures are indisputably the main characters in Information Theory. Shannon completely characterised the problem of compression via Entropy and the problem of noisy communication via Mutual Information and Capacity. Over the years, numerous novel information measures have been defined, all sharing similar properties. However, some of these quantities have yet to be associated with practical applications. In this presentation, I will provide a perspective on these objects which enables a better understanding of their connection to practical problems. Moreover, I will demonstrate the practical application of these ideas by employing Information Measures in various scenarios. These include bounding the generalisation error in Learning Theory, establishing impossibility results in Estimation Theory, and addressing concentration phenomena for non-independent random variables.