TALP talk: Human Language Technology and Machine Learning

Speaker: Pranava Swaroop Madhyastha

Date: Wednesday October 26th from 11:00 to 12:00
Place: A6203, A6 Building in Campus Nord at UPC

Abstract: I will talk about our recent paper on Mapping Unseen Words to Task-Trained Embedding Spaces ( https://aclweb.org/anthology/W/W16/W16-1612.pdf ) where we consider the setting in which we train a supervised model that learns task-specific word representations. We assume that we have access to some initial word representations (e.g., unsupervised embeddings), and that the supervised learning procedure updates them to task-specific representations for words contained in the training data. But what about words not contained in the supervised training data? When such unseen words are encountered at test time, they are typically represented by either their initial vectors or a single unknown vector, which often leads to errors. In this paper, we address this issue by learning to map from initial representations to task-specific ones. We present a general technique that uses a neural network mapper with a weighted multiple-loss criterion. This allows us to use the same learned model parameters at test time but now with appropriate task-specific representations for unseen words. We consider the task of dependency parsing and report improvements in performance (and reductions in out-of-vocabulary rates) across multiple domains such as news, Web, and speech. We also achieve downstream improvements on the task of parsing-based sentiment analysis.

To watch it: TALP_UPC Research Center youtube channel