The brevity law as a scaling law, and a possible origin of zipfs law for word frequencies
An important body of quantitative linguistics is constituted by a series of statistical laws about language usage. Despite the importance of these linguistic laws, some of them are poorly formulated, and, more importantly, there is no unified framework that encompasses all them. This paper presents...
| Autores: | , |
|---|---|
| Tipo de recurso: | artículo |
| Estado: | Versión publicada |
| Fecha de publicación: | 2020 |
| País: | España |
| Institución: | Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya) |
| Repositorio: | Recercat. Dipósit de la Recerca de Catalunya |
| OAI Identifier: | oai:recercat.cat:2072/445809 |
| Acceso en línea: | http://hdl.handle.net/2072/445809 |
| Access Level: | acceso abierto |
| Palabra clave: | 51 |
| id |
ES_096e60adc0802fe2cd319a64fbf9e656 |
|---|---|
| oai_identifier_str |
oai:recercat.cat:2072/445809 |
| network_acronym_str |
ES |
| network_name_str |
España |
| repository_id_str |
|
| spelling |
The brevity law as a scaling law, and a possible origin of zipfs law for word frequenciesCorral, Á.Serra, I.51An important body of quantitative linguistics is constituted by a series of statistical laws about language usage. Despite the importance of these linguistic laws, some of them are poorly formulated, and, more importantly, there is no unified framework that encompasses all them. This paper presents a new perspective to establish a connection between different statistical linguistic laws. Characterizing each word type by two random variables-length (in number of characters) and absolute frequency-we show that the corresponding bivariate joint probability distribution shows a rich and precise phenomenology, with the type-length and the type-frequency distributions as its two marginals, and the conditional distribution of frequency at fixed length providing a clear formulation for the brevity-frequency phenomenon. The type-length distribution turns out to be well fitted by a gamma distribution (much better than with the previously proposed lognormal), and the conditional frequency distributions at fixed length display power-law-decay behavior with a fixed exponent α ≃ 1.4 and a characteristic-frequency crossover that scales as an inverse power δ ≃ 2.8 of length, which implies the fulfillment of a scaling law analogous to those found in the thermodynamics of critical phenomena. As a by-product, we find a possible model-free explanation for the origin of Zipfs law, which should arise as a mixture of conditional frequency distributions governed by the crossover length-dependent frequency. © 2020 by the authors.MDPI AG2020info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersion14 p.application/pdfhttp://hdl.handle.net/2072/445809RECERCAT (Dipòsit de la Recerca de Catalunya)reponame:Recercat. Dipósit de la Recerca de Catalunyainstname:Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)Inglésinfo:eu-repo/semantics/openAccessoai:recercat.cat:2072/4458092026-05-29T05:05:01Z |
| dc.title.none.fl_str_mv |
The brevity law as a scaling law, and a possible origin of zipfs law for word frequencies |
| title |
The brevity law as a scaling law, and a possible origin of zipfs law for word frequencies |
| spellingShingle |
The brevity law as a scaling law, and a possible origin of zipfs law for word frequencies Corral, Á. 51 |
| title_short |
The brevity law as a scaling law, and a possible origin of zipfs law for word frequencies |
| title_full |
The brevity law as a scaling law, and a possible origin of zipfs law for word frequencies |
| title_fullStr |
The brevity law as a scaling law, and a possible origin of zipfs law for word frequencies |
| title_full_unstemmed |
The brevity law as a scaling law, and a possible origin of zipfs law for word frequencies |
| title_sort |
The brevity law as a scaling law, and a possible origin of zipfs law for word frequencies |
| dc.creator.none.fl_str_mv |
Corral, Á. Serra, I. |
| author |
Corral, Á. |
| author_facet |
Corral, Á. Serra, I. |
| author_role |
author |
| author2 |
Serra, I. |
| author2_role |
author |
| dc.subject.none.fl_str_mv |
51 |
| topic |
51 |
| description |
An important body of quantitative linguistics is constituted by a series of statistical laws about language usage. Despite the importance of these linguistic laws, some of them are poorly formulated, and, more importantly, there is no unified framework that encompasses all them. This paper presents a new perspective to establish a connection between different statistical linguistic laws. Characterizing each word type by two random variables-length (in number of characters) and absolute frequency-we show that the corresponding bivariate joint probability distribution shows a rich and precise phenomenology, with the type-length and the type-frequency distributions as its two marginals, and the conditional distribution of frequency at fixed length providing a clear formulation for the brevity-frequency phenomenon. The type-length distribution turns out to be well fitted by a gamma distribution (much better than with the previously proposed lognormal), and the conditional frequency distributions at fixed length display power-law-decay behavior with a fixed exponent α ≃ 1.4 and a characteristic-frequency crossover that scales as an inverse power δ ≃ 2.8 of length, which implies the fulfillment of a scaling law analogous to those found in the thermodynamics of critical phenomena. As a by-product, we find a possible model-free explanation for the origin of Zipfs law, which should arise as a mixture of conditional frequency distributions governed by the crossover length-dependent frequency. © 2020 by the authors. |
| publishDate |
2020 |
| dc.date.none.fl_str_mv |
2020 |
| dc.type.none.fl_str_mv |
info:eu-repo/semantics/article info:eu-repo/semantics/publishedVersion |
| format |
article |
| status_str |
publishedVersion |
| dc.identifier.none.fl_str_mv |
http://hdl.handle.net/2072/445809 |
| url |
http://hdl.handle.net/2072/445809 |
| dc.language.none.fl_str_mv |
Inglés |
| language_invalid_str_mv |
Inglés |
| dc.rights.none.fl_str_mv |
info:eu-repo/semantics/openAccess |
| eu_rights_str_mv |
openAccess |
| dc.format.none.fl_str_mv |
14 p. application/pdf |
| dc.publisher.none.fl_str_mv |
MDPI AG |
| publisher.none.fl_str_mv |
MDPI AG |
| dc.source.none.fl_str_mv |
RECERCAT (Dipòsit de la Recerca de Catalunya) reponame:Recercat. Dipósit de la Recerca de Catalunya instname:Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya) |
| instname_str |
Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya) |
| reponame_str |
Recercat. Dipósit de la Recerca de Catalunya |
| collection |
Recercat. Dipósit de la Recerca de Catalunya |
| repository.name.fl_str_mv |
|
| repository.mail.fl_str_mv |
|
| _version_ |
1869403105436631040 |
| score |
15,812429 |