An important problem in robotics is the empirical evaluation of classification algorithms that allow a robotic system to make accurate categorical predictions about its environment. Current algorithms are often assessed using sample statistics that can be difficult to interpret correctly and do not always provide a principled way of comparing competing algorithms. To offer a probabilistic alternative, we recently described a Bayesian framework for inferring on balanced accuracies. So far, however, this approach has itself been limited to binary classification. Here, we generalize the balanced accuracy to multiclass problems. Our approach uses Fourier transforms of class-specific performance distributions to obtain a posterior distribution that can be used to evaluate and compare competing classifiers. To facilitate its use, we provide an open-source MATLAB implementation.
The toolbox provides code for the assessment in a probabilistic manner of the accuracy and balanced accuracy (BAC) of a binary or multiclass classifier by computing their posterior distribution.
With the toolbox it is also possible to compute several statistics from the posterior distribution such as the mean, mode, median, alpha-CI, alpha-HDP and DBP.
- Changes to previous version:
Readme added. Explanation of the examples in the readme file.
No one has posted any comments yet. Perhaps you'd like to be the first?
Leave a comment
You must be logged in to post comments.