Optimal tree selection for classification and regression

Join Assistant Professor Zardad Khan from Abdul Wali Khan University Mardan, Pakistan for this week's seminar.

  • Fri 9 Mar 18

    15:00 - 16:00

  • Colchester Campus

    Room 6.300

  • Event speaker

    Assistant Professor Zardad Khan, Abdul Wali Khan University Mardan, Pakistan

  • Event type

    Lectures, talks and seminars
    Department of Mathematical Sciences Seminar Series

  • Event organiser

    Mathematical Sciences, Department of

  • Contact details

    Dr Andrew Harrison

Predictive performance of a random forest ensemble is highly associated with the strength of individual trees and their diversity. Ensemble of a small number of accurate and diverse trees, if prediction accuracy is not compromised, will also reduce computational burden. We investigate the idea of integrating trees that are accurate and diverse. For this purpose, we utilize out-of-bag observation as validation sample from the training bootstrap samples to choose the best trees based on their individual performance and then assess these trees for diversity using Brier score. Starting from the first best tree, a tree is selected for the final ensemble if its addition to the forest reduces error of the trees that have already been added. A total of 35 bench mark problems on classification and regression are used to assess the performance of the proposed method and compare it with kNN, tree, random forest, random projection ensemble, node harvest and support vector machine. We compute unexplained variances or classification error rates for all the methods on the corresponding data sets. Our experiments reveal that the size of the ensemble is reduced significantly and better results are obtained in most of the cases. For further verification, a simulation study is also given where four tree style scenarios are considered to generate data sets with several structures.

Zardad Khan is an Assistant Professor of Statistics, Abdul Wali Khan University Mardan, Pakistan and currently a Senior Research Officer and Visiting Fellow in the Department of Mathematical Sciences, University of Essex.

Related events