Typological Alignment of Stack-Based Language Models on Mildly Context-Sensitive Artificial Languages
What it is
This paper investigates whether stack-based language models (SLMs) exhibit learning biases that explain typological commonalities in natural languages, specifically regarding cross-serial dependencies. It evaluates SLMs on mildly context-sensitive artificial languages and finds that while they generally struggle with cross-serial dependencies, those with limited working memory generalize better, suggesting a basis for specific word order biases.
The problem it solves
Existing analyses of language model learning biases often fail to account for the specific computational properties of stack-based models or the upper limits of attested syntactic complexity like cross-serial dependencies.
What you could use it for
- Understanding the computational limits of stack-based architectures in learning hierarchical syntax
- Exploring the link between model inductive biases and natural language typology
- Benchmarking language models on mildly context-sensitive artificial grammars
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.