Two regimes in the frequency of words and the origin of complex lexicons: Zipf's law revisited
Zipf's law states that the frequency of a word is a power function of its rank. The exponent of the power is usually accepted to be close to (-)1. Great deviations between the predicted and real number of different words of a text, disagreements between the predicted and real exponent of the pr...
| Autores: | , |
|---|---|
| Tipo de recurso: | artículo |
| Fecha de publicación: | 2001 |
| País: | España |
| Institución: | Universitat Politècnica de Catalunya (UPC) |
| Repositorio: | UPCommons. Portal del coneixement obert de la UPC |
| Idioma: | inglés |
| OAI Identifier: | oai:upcommons.upc.edu:2117/180381 |
| Acceso en línea: | https://hdl.handle.net/2117/180381 https://dx.doi.org/10.1076/jqul.8.3.165.4101 |
| Access Level: | acceso abierto |
| Palabra clave: | Computational linguistics Zipf’s law Power laws Word frequency Complex lexicons Language evolution Lingüística computacional Àrees temàtiques de la UPC::Informàtica::Intel·ligència artificial::Llenguatge natural |
| Sumario: | Zipf's law states that the frequency of a word is a power function of its rank. The exponent of the power is usually accepted to be close to (-)1. Great deviations between the predicted and real number of different words of a text, disagreements between the predicted and real exponent of the probability density function and statistics on a big corpus, make evident that word frequency as a function of the rank follows two different exponents, ~(-)1 for the first regime and ~(-)2 for the second. The implications of the change in exponents for the metrics of texts and for the origins of complex lexicons are analyzed. |
|---|