Boosted Decision Trees and Listshttp://mloss.orgUpdates and additions to Boosted Decision Trees and ListsenFri, 25 Jul 2014 23:08:32 -0000Boosted Decision Trees and Lists 1.0.4<html><p>Boosted Decision Trees and Lists (BDTL) </p> <p>The BDTL software package implements two main boosting algorithms with many variations. The first one is Turian and Melamed's extension of confidence-rated boosting (Schapire &amp; Singer, 1999). See Joseph Turian's thesis (NYU, 2007) for details. The second algorithm is Galron and Melamed's extension of the first one, to boost decision lists rather than decision trees. (Every decision tree ensemble is equivalent to some decision list ensemble, and vice versa.) </p> <p>Major features of the software include: </p> <ul> <li><p>Classification and regression </p> </li> <li><p>tested on 3M+ examples with 1M+ features, but limited only by your computer's RAM. </p> </li> <li><p>Easily customizable loss functions and regularization methods. Currently implemented loss functions are logistic and exponential loss for classification, and squared loss for regression. Currently implemented regularization methods are L1, L2, and ElasticNets. </p> </li> <li><p>Selection of weak learners to directly optimize the regularized training objective. </p> </li> <li><p>One-shot training following an entire regularization path, which can save a lot of time during hyperparameter optimization. </p> </li> <li><p>Continuous checkpointing, so that if a long learning cycle crashes, you can continue training where it left off. </p> </li> <li><p>Warm-start training. </p> </li> <li><p>Both binary and scalar feature types. </p> </li> </ul> <p>To get started, read the file README.1st in the top level directory. </p> <p>Questions, suggestions, and offers of collaboration are most welcome. </p></html>I. Dan Melamed, Daniel A. GalronFri, 25 Jul 2014 23:08:32 -0000 treesboostingdecision lists