Title:\u00a0Cost-sensitive Boosting, Margin Maximization & Information Theory<\/p>\n
<\/p>\n
Abstract:\u00a0Boosting refers to a family of learning algorithms that additively combine multiple weak predictive models to construct a powerful one. The next model to be added is effectively fitting the residuals of the current model. In the first part of the talk, we will provide a unifying perspective for two decades of work on cost-sensitive Boosting algorithms for classification. We analyse the relevant literature using four theoretical frameworks: decision theory, functional gradient descent, margin theory, and probabilistic modelling. We find that only 3 of the \u2013more than 15\u2013 published variants are consistent with the rules of all the frameworks \u2014 and even they require their outputs to be calibrated to achieve this. The final recommendation \u2013 based on simplicity, flexibility and empirical performance \u2013 is to use the original AdaBoost algorithm with a shifted decision threshold and calibrated probability estimates.<\/p>\n
The reason for the poor calibration of the scores generated by Boosting models lies in the margin maximization property of Boosting, which forces the model to be overconfident in its predictions. In the second part of the talk, we focus on the more positive aspect of this property: constructing classifiers good at generalizing beyond the training sample. We then discuss ongoing work on interpreting margin maximization from an information-theoretic perspective and connecting it to a general language for expressing both the quality of fit and the effective complexity of a predictive model.<\/p>\n
<\/p>\n","protected":false},"excerpt":{"rendered":"
Title:\u00a0Cost-sensitive Boosting, Margin Maximization & Information Theory Abstract:\u00a0Boosting refers to a family of learning algorithms that additively combine multiple weak predictive models to construct … Read more<\/a><\/p>\n","protected":false},"author":57430,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[170526],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.kent.ac.uk\/pgrseminars\/wp-json\/wp\/v2\/posts\/326"}],"collection":[{"href":"https:\/\/blogs.kent.ac.uk\/pgrseminars\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.kent.ac.uk\/pgrseminars\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.kent.ac.uk\/pgrseminars\/wp-json\/wp\/v2\/users\/57430"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.kent.ac.uk\/pgrseminars\/wp-json\/wp\/v2\/comments?post=326"}],"version-history":[{"count":3,"href":"https:\/\/blogs.kent.ac.uk\/pgrseminars\/wp-json\/wp\/v2\/posts\/326\/revisions"}],"predecessor-version":[{"id":329,"href":"https:\/\/blogs.kent.ac.uk\/pgrseminars\/wp-json\/wp\/v2\/posts\/326\/revisions\/329"}],"wp:attachment":[{"href":"https:\/\/blogs.kent.ac.uk\/pgrseminars\/wp-json\/wp\/v2\/media?parent=326"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.kent.ac.uk\/pgrseminars\/wp-json\/wp\/v2\/categories?post=326"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.kent.ac.uk\/pgrseminars\/wp-json\/wp\/v2\/tags?post=326"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}