Abstract

Deep neural networks achieve state-of-the-art performances on sensor data generated by a wide variety of mobile applications. However, the capability of deep learning models to perform lifelong learning (continual learning), that is, to learn new inputs (new tasks or classes) continuously, is often impaired by catastrophic forgetting, i.e., a model forgets previously learned knowledge when learning new tasks. This can be very detrimental in ubiquitous computing, where a deployed model needs to accommodate new sensor inputs and changing environments continuously. Several techniques have been proposed to solve catastrophic forgetting, but their performance has not been fully examined in mobile sensing applications. In this talk, for the first time, we systematically study the performance of three predominant lifelong learning schemes (i.e., regularization, replay and replay with examples) on mobile sensing applications of human activity recognition and gesture recognition. With scenarios consisting of different learning complexity, as encountered in practice, we investigate the generalizability, trade-offs between the performance, storage, and latency of different lifelong learning methods on mobile sensing applications. Finally, we summarize our results into a series of lessons that can guide practitioners in their use of lifelong deep learning for mobile applications.