Spring School on Cognitive Modeling - Prof. H. Baayen University of Tübingen

Title: Understanding what word embeddings understand

Abstract:

Word embeddings are numerical representations of word meanings that are widely used in natural language processing tasks. Word embeddings are calculated from text corpora, building on the 'distributional hypothesis', according to which words that are similar in meaning occur in similar textual contexts. Conceptualizing word meanings as vectors in a high-dimensional space offers linguistics new empirical ways forward for understanding how we think for speaking. In this presentation, two models formalizing 'thinking for inflected words' will be compared. However, understanding what word embeddings actually extract (or do not extract), out of text corpora is still an open question. Examples will be discussed where we are offered novel insights that are made possible thanks to word embeddings. But there are also linguistic data for which embeddings fail to capture important aspects of words' meanings, challenging the distributional hypothesis, and raising new questions of how to make the way in which we calculate embeddings cognitively more realistic.