Project details for MIToolbox

Logo MIToolbox 2.1

by apocock - June 30, 2014, 01:05:57 CET [ Project Homepage BibTeX BibTeX for corresponding Paper Download ]

view (21 today), download ( 4 today ), 2 comments, 3 subscriptions


A mutual information library for C/C++ and Mex bindings for MATLAB.

This toolbox is aimed at people who wish to use mutual information for feature selection, and provides a range of information theoretic functions. All functions estimate the probabilities from the supplied data vectors. Some example implementations of common mutual information based feature selection algorithms are provided in both C and MATLAB, CMIM - (Fleuret 2004), mRMR - (Peng et al 2005), DISR - (Bontempi & Meyer 2006). The implementations contained in here are early versions of those in the FEAST library, and the implementations in FEAST should be used in preference to the ones in MIToolbox.

All functions discretise the inputs by rounding down to the nearest integer.

This toolbox was developed to support our work in feature selection, which resulted in the paper "Conditional Likelihood Maximisation: A Unifying Framework for Information Theoretic Feature Selection", G Brown, A Pocock, M-J Zhao, M Lujan. JMLR 2012 (link). Please cite this paper if you use our toolbox.

The feature selection algorithms developed for that paper form the FEAST toolbox, published at mloss here.

List of functions: Entropy, Conditional Entropy, Joint Entropy, Mutual Information, Conditional Mutual Information, Renyi's Entropy, Renyi's Mutual Information, Weighted Entropy, Weighted Conditional Entropy Creating a joint random variable

A Java implementation of the Shannon entropy functions is available from my GitHub page here.

Changes to previous version:

Added weighted entropy functions. Fixed a few memory handling bugs.

BibTeX Entry: Download
Corresponding Paper BibTeX Entry: Download
URL: Project Homepage
Supported Operating Systems: Linux, Macosx, Windows
Data Formats: Matlab
Tags: Conditional Entropy, Conditional Mutual Information, Entropy, Mutual Information, Renyi Entropy
Archive: download here

Other available revisons

Version Changelog Date

Major refactor of code and reorganised the repository so it's a little more sensible.

  • Refactored all C functions to expose a version which takes unsigned integer inputs.
  • Rearranged the repository to separate out headers from source, and MATLAB code from C library code.

Minor changes:

  • General code cleanup to reduce duplicated code.
  • Adding an COMPILE_R flag to go with the COMPILE_C flag, to make it easier to produce an R wrapper.
  • All code now compiles cleanly with "-std=c89 -Wall -Werror".
January 8, 2017, 00:43:22

Relicensed as BSD. Added checks to catch MATLAB inputs that aren't doubles.

January 10, 2016, 22:19:30

Added weighted entropy functions. Fixed a few memory handling bugs.

June 30, 2014, 01:05:57

Updated documentation & link to JMLR publication.

February 13, 2012, 19:45:07

Initial Announcement on

August 3, 2011, 20:58:19


Aida Brank (on September 22, 2016, 20:11:50)


I have following error:

Undefined function or variable 'FSToolboxMex'. Error in feast (line 70) selectedFeatures = FSToolboxMex(4,numToSelect,data,labels);

though I did as I run ComlileMIToolbox.m in MIToolbox directory.



Adam Pocock (on September 22, 2016, 22:09:08)

Hi Aida,

That error means you haven't compiled FEAST correctly. You need to compile both FEAST and MIToolbox separately.



Leave a comment

You must be logged in to post comments.