Description:

Boosted Decision Trees and Lists (BDTL)

The BDTL software package implements two main boosting algorithms with many variations. The first one is Turian and Melamed's extension of confidence-rated boosting (Schapire & Singer, 1999). See Joseph Turian's thesis (NYU, 2007) for details. The second algorithm is Galron and Melamed's extension of the first one, to boost decision lists rather than decision trees. (Every decision tree ensemble is equivalent to some decision list ensemble, and vice versa.)

Major features of the software include:

• Classification and regression

• tested on 3M+ examples with 1M+ features, but limited only by your computer's RAM.

• Easily customizable loss functions and regularization methods. Currently implemented loss functions are logistic and exponential loss for classification, and squared loss for regression. Currently implemented regularization methods are L1, L2, and ElasticNets.

• Selection of weak learners to directly optimize the regularized training objective.

• One-shot training following an entire regularization path, which can save a lot of time during hyperparameter optimization.

• Continuous checkpointing, so that if a long learning cycle crashes, you can continue training where it left off.

• Warm-start training.

• Both binary and scalar feature types.

To get started, read the file README.1st in the top level directory.

Questions, suggestions, and offers of collaboration are most welcome.

Changes to previous version:

• added ElasticNets as a regularization option
• fixed some segfaults, memory leaks, and out-of-range errors, which were creeping in in some corner cases
• added a couple of I/O optimizations

BibTeX Entry: Download

Supported Operating Systems: Linux, Unix

Data Formats: Svmlight, Genpar

Tags: Classification, Regression, Decision Trees, Boosting, Decision Lists

Archive: download here

Other available revisons

Version	Changelog	Date
1.0.4	added ElasticNets as a regularization option fixed some segfaults, memory leaks, and out-of-range errors, which were creeping in in some corner cases added a couple of I/O optimizations	July 25, 2014, 23:08:32
1.0.3	faster warm-start made it easier to add more library paths to local makefile added scripts to remove rare features and to standardize features	May 1, 2014, 15:19:29
1.0.2	updated for gcc-4.8 added missing script used in sandbox evaluation	July 23, 2013, 22:40:34
1.0.1	Initial Announcement on mloss.org.	May 28, 2013, 05:29:07

Comments

No one has posted any comments yet. Perhaps you'd like to be the first?

Leave a comment

You must be logged in to post comments.