Zipf's law and random texts
Random-text models have been proposed as an explanation for the power law relationship between word frequency and rank, the so-called Zipf's law. They are generally regarded as null hypotheses rather than models in the strict sense. In this context, recent theories of language emergence and evo...
| Autores: | , |
|---|---|
| Tipo de recurso: | artículo |
| Fecha de publicación: | 2002 |
| País: | España |
| Institución: | Universitat Politècnica de Catalunya (UPC) |
| Repositorio: | UPCommons. Portal del coneixement obert de la UPC |
| Idioma: | inglés |
| OAI Identifier: | oai:upcommons.upc.edu:2117/180250 |
| Acceso en línea: | https://hdl.handle.net/2117/180250 https://dx.doi.org/10.1142/S0219525902000468 |
| Access Level: | acceso abierto |
| Palabra clave: | Zipf’s law Computational linguistics Human language Scaling Monkey languages Random texts Lingüística computacional Àrees temàtiques de la UPC::Informàtica::Intel·ligència artificial::Llenguatge natural |
| Sumario: | Random-text models have been proposed as an explanation for the power law relationship between word frequency and rank, the so-called Zipf's law. They are generally regarded as null hypotheses rather than models in the strict sense. In this context, recent theories of language emergence and evolution assume this law as a priori information with no need of explanation. Here, random texts and real texts are compared through (a) the so-called lexical spectrum and (b) the distribution of words having the same length. It is shown that real texts fill the lexical spectrum much more efficiently and regardless of the word length, suggesting that the meaningfulness of Zipf's law is high. |
|---|