Abstract

Deep Neural Network (DNN) has been found to yield superior performance compared to the conventional Gaussian Mixture Model (GMM) based systems for automatic speech recognition. However, DNN has been used pretty much as a black box without much insights as to what the DNN has learned. This talk will present a novel approach for interpreting the DNN model, which is based on analysing the hidden activity pattern. This technique constructs a 2-dimensional hidden activity space where interpretable regions can be defined. This technique can be used to facilitate the understanding and comparison of the hidden activity patterns across different hidden layers, networks and time frames. Finally, a technique called “stimulated deep learning” will be presented, where phone stimuli are used to control the training process so that the hidden units of the resulting DNN yield interpretable activity patterns. Preliminary experimental results on TIMIT show that the proposed stimulated deep learning technique is able to learn DNNs with the hidden units showing phone-dependent activity regions without compromising the phone recognition performance.