Abstract

Speaker 1 – David Janz Title – Successor Uncertainties: Exploration and Uncertainty in Temporal Difference Learning Abstract – Probabilistic Q-learning is a promising approach balancing exploration and exploitation in reinforcement learning. However, existing implementations have significant limitations: they either fail to incorporate uncertainty about long-term consequences of actions or ignore fundamental dependencies in state-action values implied by the~Bellman equation. These problems result in sub-optimal exploration. As a solution, we develop Successor Uncertainties (SU), a probabilistic Q-learning method free of the aforementioned problems. SU outperforms existing baselines on tabular problems and on the Atari benchmark benchmark suite. Overall, SU is an improved and scalable probabilistic Q-learning method with better properties than its predecessors at no extra cost.

Speaker 2 – Jan Stuehmer Title – Independent Subspace Analysis for Unsupervised Learning of Disentangled Representations Abstract – Recently there has been an increased interest in unsupervised learning of disentangled representations using the Variational Autoencoder (VAE) framework. Most of the existing work has focused largely on modifying the variational cost function to achieve this goal. These modifications usually include a variable regularization strength parameter, which can be hard or impossible to choose in an unsupervised manner. We first show that methods like beta-VAE simplify the tendency of variational inference to underfit causing pathological over-pruning and over-orthogonalization of learned components. Second we propose a complementary approach: to modify the probabilistic model with a structured latent prior. This prior allows to discover latent variable representations that are structured into independent vector spaces. The proposed prior has three major advantages: First, in contrast to the standard VAE normal prior the proposed prior is not rotationally invariant. This resolves the problem of unidentifiability of the standard VAE normal prior. Second, extensive quantitative and qualitative experiments demonstrate that the prior encourages a disentangled latent representation which mitigates the need of carefully tuning the regularization strength parameter and therefore facilitates unsupervised learning of disentangled representations. Third, the experiments demonstrate that the prior significantly mitigates the trade-off introduced by modified cost functions like beta-VAE and TCVAE between reconstruction loss and disentanglement, which allows to improve these approaches with respect to both disentanglement and reconstruction quality significantly over the state of the art.