Decision tree statistical learning models. An application to new customer scoring
The aim of this thesis is to explore, understand and apply statistical learning methods based on decision trees, specifically individual decision trees and bagging, random forests and gradient boosting methods. In order to do this, aresearch has been done and the theory behind each one of these meth...
| Autor: | |
|---|---|
| Tipo de recurso: | tesis de maestría |
| Fecha de publicación: | 2020 |
| País: | España |
| Institución: | Universitat Politècnica de Catalunya (UPC) |
| Repositorio: | UPCommons. Portal del coneixement obert de la UPC |
| Idioma: | inglés |
| OAI Identifier: | oai:upcommons.upc.edu:2117/178053 |
| Acceso en línea: | https://hdl.handle.net/2117/178053 |
| Access Level: | acceso abierto |
| Palabra clave: | Decision trees Statistics--Study and teaching Arbres de decisió Estadistica -- Ensenyament Àrees temàtiques de la UPC::Matemàtiques i estadística::Estadística aplicada |
| Sumario: | The aim of this thesis is to explore, understand and apply statistical learning methods based on decision trees, specifically individual decision trees and bagging, random forests and gradient boosting methods. In order to do this, aresearch has been done and the theory behind each one of these methods understood.The main sources of information are thebooks “Introduction to Statistical Learning” and “The Elements of Statistical Learning” by T. Hastie, R. Tibshirani and J. Friedman.Afterwards this theory is put to practice using a real case data set and the R programming language to experiment with models of the mentioned methods. The data used comes from areal case project in which a business wishes to predict whether anew customer will be a good one based only in the information from its three first purchases.The tools used are also presented, consisting in the different R packages and functions used and its tuning parameters. The strategy used in order to obtain representative results that make possible to understand the concepts presented in the theory is explained. As well as how these results have been extracted.The sensitivity analysis has been done with the Minitab v18 software, provided by the Universitat Politècnica de Catalunya for research purposes.Finally the results are analysed. This analysis is divided in three sections.The first one is focused in a sensitivity analysis of parameters. The results show that, with the used dataset, for gradient boosting the tree depth allowed is critical to obtain a good quality of fit and prevent overfitting, andthe number of iterations allowed needs to be correctly alignedwith the learning parameter used. The results for bagging and random forests (merged as one is a particular case of the other) prove the lack of overfittingintrinsic of these modelsand discovers that if the number of variables is high and these are strongly correlated the recommended number of variables to choose at each tree node does not lead to optimum results. An initial hypothesis to guide the analysis of this fact is proposed but it is not inside the scope of the project to analyse and prove this hypothesis. The second section of the analysis consists in selecting the best performing method and apply it to the availabledataset. The gradient boosting method is chosen as the best one due to higher quality of fit obtained and a more consistent selection of variables among all scenarios. The third section compares the results obtained with gradient boosting versus the logistic regression model done by the student P. Casas in his bachelor thesis“New customers’ classifier”based on the same dataset. The results show that gradient boosting performs better in terms of prediction in two of the three models created, though the difference is small, and obtains the same quality of fitin the other case. Comparing variable relevancethe most important one is shared among both methods(the total value of the purchase). Other secondary variables are shared and some of them not. Therefore it can be said there is similarity in general terms but gradient boosting and logistic regression are nottotally close between them,as it happens with the decision tree methods used in the project. |
|---|