Description:

Functions compare two classifiers on one or multiple data sets. They compute three probabilities: the probability that the first classifier has higher scores than the second, the probability that differences are within the region of practical equivalence (rope), or that the second classifier has higher scores.

The region of practical equivalence is specified by the caller and should correspond to what is "equivalent" in practice; for instance, classification accuracies that differ by less than 0.5 may be called equivalent.

Similarly, whether higher scores are better or worse depends upon the type of the score.

The library can also plot the posterior distributions.

Changes to previous version:

Initial Announcement on mloss.org.

BibTeX Entry: Download

Supported Operating Systems: Any

Data Formats: Agnostic

Tags: Bayesian Tests, Classifier Comparison

Archive: download here

Comments

No one has posted any comments yet. Perhaps you'd like to be the first?

Leave a comment

You must be logged in to post comments.