Computing the expected Markov reward rates with stationarity detection and relative error control
By combining in a novel way the randomization method with the stationary detection technique, we develop two new algorithms for the computation of the expected reward rates of finite, irreducible Markov reward models, with control of the relative error. The first algorithm computes the expected tran...
| Autor: | |
|---|---|
| Tipo de recurso: | artículo |
| Fecha de publicación: | 2016 |
| País: | España |
| Institución: | Universitat Politècnica de Catalunya (UPC) |
| Repositorio: | UPCommons. Portal del coneixement obert de la UPC |
| Idioma: | inglés |
| OAI Identifier: | oai:upcommons.upc.edu:2117/85787 |
| Acceso en línea: | https://hdl.handle.net/2117/85787 https://dx.doi.org/10.1007/s11009-016-9490-y |
| Access Level: | acceso abierto |
| Palabra clave: | Markov processes Random fields Error analysis (Mathematics) Markov reward model Markov chain Expected reward rate Relative error Randomization Stationarity detection Markov, Processos de Camps aleatoris Anàlisi d'error (Matemàtica) Àrees temàtiques de la UPC::Matemàtiques i estadística::Anàlisi matemàtica Àrees temàtiques de la UPC::Enginyeria electrònica |
| Sumario: | By combining in a novel way the randomization method with the stationary detection technique, we develop two new algorithms for the computation of the expected reward rates of finite, irreducible Markov reward models, with control of the relative error. The first algorithm computes the expected transient reward rate and the second one computes the expected averaged reward rate. The algorithms are numerically stable. Further, it is argued that, from the point of view of run-time computational cost, for medium-sized and large Markov reward models, we can expect the algorithms to be better than the only variant of the randomization method that allows to control the relative error and better than the approach that consists in employing iteratively the currently existing algorithms that use the randomization method with stationarity detection but allow to control the absolute error. The performance of the new algorithms is illustrated by means of examples, showing that the algorithms can be not only faster but also more efficient than the alternatives in terms of run-time computational cost in relation to accuracy. |
|---|