Improving Interpretability and Regularization in Deep Learning
IEEE/ACM Transactions on Audio Speech and Language Processing
MetadataShow full item record
Wu, C., Gales, M., Ragni, A., Karanasou, P., & Sim, K. (2018). Improving Interpretability and Regularization in Deep Learning. IEEE/ACM Transactions on Audio Speech and Language Processing, 26 (2), 256-265. https://doi.org/10.1109/TASLP.2017.2774919
IEEE Deep learning approaches yield state-of-the-art performance in a range of tasks, including automatic speech recognition. However, the highly distributed representation in a deep neural network (DNN) or other network variations are difficult to analyse, making further parameter interpretation and regularisation challenging. This paper presents a regularisation scheme acting on the activation function output to improve the network interpretability and regularisation. The proposed approach, referred to as activation regularisation, encourages activation function outputs to satisfy a target pattern. By defining appropriate target patterns, different learning concepts can be imposed on the network. This method can aid network interpretability and also has the potential to reduce over-fitting. The scheme is evaluated on several continuous speech recognition tasks: the Wall Street Journal continuous speech recognition task, eight conversational telephone speech tasks from the IARPA Babel program and a U.S. English broadcast news task. On all the tasks, the activation regularisation achieved consistent performance gains over the standard DNN baselines.
Cambridge Assessment (unknown)
External DOI: https://doi.org/10.1109/TASLP.2017.2774919
This record's URL: https://www.repository.cam.ac.uk/handle/1810/274201