Online EM with weight-based forgetting
In the on-line version of the EM algorithm introduced by Sato and Ishii (2000), a time-dependent discount factor is introduced for forgetting the effect of the old posterior values obtained with an earlier, inaccurate estimator. In their approach, forgetting is uniformly applied to the estimators of...
| Autores: | , |
|---|---|
| Formato: | artículo |
| Fecha de publicación: | 2015 |
| País: | España |
| Recursos: | Universitat Politècnica de Catalunya (UPC) |
| Repositorio: | UPCommons. Portal del coneixement obert de la UPC |
| Idioma: | inglés |
| OAI Identifier: | oai:upcommons.upc.edu:2117/81127 |
| Acesso em linha: | https://hdl.handle.net/2117/81127 https://dx.doi.org/10.1162/NECO_a_00723 |
| Access Level: | acceso abierto |
| Palavra-chave: | learning (artificial intelligence) stochastic programming Classificació INSPEC::Cybernetics::Artificial intelligence Àrees temàtiques de la UPC::Informàtica::Intel·ligència artificial |
| Resumo: | In the on-line version of the EM algorithm introduced by Sato and Ishii (2000), a time-dependent discount factor is introduced for forgetting the effect of the old posterior values obtained with an earlier, inaccurate estimator. In their approach, forgetting is uniformly applied to the estimators of each mixture component depending exclusively on time, irrespective of the weight attributed to each unit for the observed sample. This causes an excessive forgetting in the less frequently sampled regions. To address this problem we propose a modification of the algorithm that involves a weight-dependent forgetting, different for each mixture component, in which old observations are forgotten according to the actual weight of the new samples used to replace older values. A comparison of the time-dependent versus the weight-dependent approach shows that the last one improves the accuracy of the approximation and exhibits a much greater stability. |
|---|