Self supervised learning and the poverty of the stimulus

Abstract

Diathesis alternations are the possible expressions of the arguments of verbs in different, systematically related subcategorization frames. Semantically similar verbs such as spill and spray can behave differently with respect to the alternations they can participate in. For example one can “spill/spray water on the plant”, but while one can “spray the plant with water”, it is odd to say “spill the plant with water”. “Spray” is a verb which can alternate between syntactic frames while “spill” is not alternating. How human speakers learn the difference between such verbs is not clearly understood, because the primary linguistic data (PLD) they receive does not appear sufficient to infer the knowledge required for adult competence. More generally the poverty of the stimulus (POS) hypothesis states that the PLD is not sufficient for a learner to infer full adult competence of language. That is, learning relies on prior constraints introduced by the language faculty. We tested state-of-the-art machine learning models trained by self supervision, and found some evidence that they could in fact learn the correct pattern of acceptability judgement in the locative alternation. However, we argued that this was partially a result of fine-tuning which introduced negative evidence into the learning data, which facilitated shortcut learning. Large language models (LLMs) cannot learn some linguistic facts from normal language data, but they can compensate to some extent by learning spurious correlated features when negative feedback is introduced during the training cycle.