mloss.org MIToolbox

MIToolbox 3.0.1

Adam Pocock — Thu, 02 Mar 2017 00:38:52 -0000

A mutual information library for C/C++ and Mex bindings for MATLAB.

This toolbox is aimed at people who wish to use mutual information for feature selection, and provides a range of information theoretic functions. All functions estimate the probabilities from the supplied data vectors. Some example implementations of common mutual information based feature selection algorithms are provided in both C and MATLAB, CMIM - (Fleuret 2004), mRMR - (Peng et al 2005), DISR - (Bontempi & Meyer 2006). The implementations contained in here are early versions of those in the FEAST library, and the implementations in FEAST should be used in preference to the ones in MIToolbox.

All functions discretise the inputs by rounding down to the nearest integer.

This toolbox was developed to support our work in feature selection, which resulted in the paper "Conditional Likelihood Maximisation: A Unifying Framework for Information Theoretic Feature Selection", G Brown, A Pocock, M-J Zhao, M Lujan. JMLR 2012 (link). Please cite this paper if you use our toolbox.

The feature selection algorithms developed for that paper form the FEAST toolbox, published at mloss here.

List of functions: Entropy, Conditional Entropy, Joint Entropy, Mutual Information, Conditional Mutual Information, Renyi's Entropy, Renyi's Mutual Information, Weighted Entropy, Weighted Conditional Entropy Creating a joint random variable

A Java implementation of the Shannon entropy functions is available from my GitHub page here.

Comment by Aida Brank on 2016-09-22 20:11

Aida Brank — Thu, 22 Sep 2016 20:11:50 -0000

Hello,

I have following error:

Undefined function or variable 'FSToolboxMex'. Error in feast (line 70) selectedFeatures = FSToolboxMex(4,numToSelect,data,labels);

though I did as I run ComlileMIToolbox.m in MIToolbox directory.

Thanks!!!

Regards

Comment by Adam Pocock on 2016-09-22 22:09

Adam Pocock — Thu, 22 Sep 2016 22:09:08 -0000

Hi Aida,

That error means you haven't compiled FEAST correctly. You need to compile both FEAST and MIToolbox separately.

Thanks,

Adam

Comment by Fazeelat Mazhar on 2017-02-07 08:54

Fazeelat Mazhar — Tue, 07 Feb 2017 08:54:10 -0000

Hi Adam,

I am facing error in running CompileMIToolbox.m file on MATLAB. Error is:

Error using mex CalculateProbability.c C:\Users\Fazeelat\Downloads\MIToolbox-3.0.0\MIToolbox-3.0.0\src\CalculateProbability.c(271) : error C2275: 'ProbabilityState' : illegal use of this type as an expression ../include\MIToolbox/CalculateProbability.h(40) : see declaration of 'ProbabilityState' C:\Users\Fazeelat\Downloads\MIToolbox-3.0.0\MIToolbox-3.0.0\src\CalculateProbability.c(271) : error C2146: syntax error : missing ';' before identifier 'state' C:\Users\Fazeelat\Downloads\MIToolbox-3.0.0\MIToolbox-3.0.0\src\CalculateProbability.c(271) : error C2065: 'state' : undeclared identifier C:\Users\Fazeelat\Downloads\MIToolbox-3.0.0\MIToolbox-3.0.0\src\CalculateProbability.c(271) : error C2440: '=' : cannot convert from 'ProbabilityState' to 'int' C:\Users\Fazeelat\Downloads\MIToolbox-3.0.0\MIToolbox-3.0.0\src\CalculateProbability.c(276) : error C2065: 'state' : undeclared identifier C:\Users\Fazeelat\Downloads\MIToolbox-3.0.0\MIToolbox-3.0.0\src\CalculateProbability.c(276) : error C2440: 'return' : cannot convert from 'int' to 'ProbabilityState'

Comment by Adam Pocock on 2017-02-07 14:58

Adam Pocock — Tue, 07 Feb 2017 14:58:04 -0000

Hi Fazeelat,

What C compiler, Matlab version and Windows version are you using?

Thanks,

Adam

Comment by Fazeelat Mazhar on 2017-02-08 05:02

Fazeelat Mazhar — Wed, 08 Feb 2017 05:02:06 -0000

C compiler, Microsoft Visual studio 2010 Matlab version 8.4 Windows 8.1

Comment by Adam Pocock on 2017-02-09 02:41

Adam Pocock — Thu, 09 Feb 2017 02:41:55 -0000

Hi Fazeelat,

Turns out there was one place where I didn't strictly follow ANSI C. GCC wasn't throwing an error, but apparently Visual Studio does. I've fixed it now, can you try version 3.0.1 from my GitHub page.

Thanks,

Adam

Comment by Fazeelat Mazhar on 2017-02-09 06:20

Fazeelat Mazhar — Thu, 09 Feb 2017 06:20:30 -0000

Thanks Adam, that error has been fixed.

Another error of out of memory appeared while finding MI of two matrices. My dataset is a bit large dimension i.e.,146X16.

Can you help?

Comment by Adam Pocock on 2017-02-09 16:08

Adam Pocock — Thu, 09 Feb 2017 16:08:59 -0000

Hi Fazeelat,

MIToolbox isn't really suitable for that kind of mutual information estimation. You need many more samples to do discrete entropy estimation in a 16 dimensional space, and the mutual information will work in a 32 dimensional space (assuming both your matrices are the same size).

If you have a continuous space you might have more luck with a continuous MI estimator using a k-nn or minimum spanning tree approximation, but MIToolbox doesn't implement either of those. I think Zoltan Szabo's ITE toolkit might be more appropriate.

Thanks,

Adam

Comment by Fazeelat Mazhar on 2017-02-10 07:05

Fazeelat Mazhar — Fri, 10 Feb 2017 07:05:12 -0000

Thanks Adam.

Comment by Janek Janek on 2017-02-20 18:56

Janek Janek — Mon, 20 Feb 2017 18:56:54 -0000

Adam. Please Help me. This problem http://s1.fotowrzut.pl/YLV6QVY97K/1.jpg

Matlab R2016a Windows7 I try wersion 3.0.1

Help

Comment by Adam Pocock on 2017-02-20 19:14

Adam Pocock — Mon, 20 Feb 2017 19:14:21 -0000

Hi Janek,

Please check that CompileFEAST is pointing at v3.0.1 of MIToolbox. The version that is packaged in the MLOSS download is v3.0.0 which has that bug. You'll need to download MIToolbox v3.0.1 from Github and unpack it in the directory next to FEAST.

Thanks,

Adam

Comment by Janek Janek on 2017-02-20 19:51

Janek Janek — Mon, 20 Feb 2017 19:51:15 -0000

Adam,

Help Gee I do not understand You can check it's toolbox I use this

http://s1.fotowrzut.pl/O2V2AUIRNB/1.jpg http://s1.fotowrzut.pl/61GMCSFD4I/1.jpg

http://www82.zippyshare.com/v/BSJx3Xlk/file.html Please check whether it is good in the toolbox

Thanks,

Janek

Comment by Adam Pocock on 2017-02-20 20:04

Adam Pocock — Mon, 20 Feb 2017 20:04:32 -0000

Hi Janek,

You haven't unpacked the updated version of MIToolbox in the right directory. The error message from MATLAB shows it's looking in the directory above FEAST, not inside the FEAST directory.

The zip you sent compiles fine on my Linux box from within the FEAST/matlab directory. I don't have access to a copy of Matlab on Windows with Visual Studio to test it out further.

Thanks,

Adam

Comment by Janek Janek on 2017-02-20 20:36

Janek Janek — Mon, 20 Feb 2017 20:36:48 -0000

Hi Adam,

I fixed thanks for your help. http://s1.fotowrzut.pl/LKXJ3BQIDV/1.jpg

Thanks,

Janek