Description:

ITE can estimate

• entropy: Shannon entropy, Rényi entropy, Tsallis entropy (Havrda and Charvát entropy), complex entropy,

• mutual information: generalized variance, kernel canonical correlation analysis, kernel generalized variance, Hilbert-Schmidt independence criterion, Shannon mutual information, L2 mutual information, Rényi mutual information, Tsallis mutual information, copula-based kernel dependency, multivariate version of Hoeffding's Phi, complex mutual information, Cauchy-Schwartz quadratic mutual information, Euclidean distance based quadratic mutual information, distance covariance, distance correlation, approximate correntropy independence measure,

• divergence: Kullback-Leibler divergence (relative entropy, I directed divergence), L2 divergence, Rényi divergence, Tsallis divergence, Hellinger distance, Bhattacharyya distance, maximum mean discrepancy (kernel distance, an integral probability metric), J-distance (symmetrised Kullback-Leibler divergence, J divergence), Cauchy-Schwartz divergence, Euclidean distance based divergence, energy distance (specially the Cramer-Von Mises distance), Jensen-Shannon divergence, Jensen-Rényi divergence,

• association measures, including measures of concordance: multivariate extensions of Spearman's rho (Spearman's rank correlation coefficient, grade correlation coefficient), correntropy, centered correntropy, correntropy coefficient, correntropy induced metric, centered correntropy induced metric, multivariate extension of Blomqvist's beta (medial correlation coefficient), multivariate conditional version of Spearman's rho, lower/upper tail dependence via conditional Spearman's rho,

• cross quantities: cross-entropy.

ITE offers solution methods for

• Independent Subspace Analysis (ISA) and
• its extensions to different linear-, controlled-, post nonlinear-, complex valued-, partially observed models, as well as to systems with nonparametric source dynamics.

ITE is

• written in Matlab/Octave,
• multi-platform (tested extensively on Windows and Linux),
• free and open source (released under the GNU GPLv3(>=) license).

Changes to previous version:

• K divergence estimation: added,

• L divergence estimation: added,

• kNN squared distance computation: refined.

BibTeX Entry: Download

Corresponding Paper BibTeX Entry: Download

URL: Project Homepage

Supported Operating Systems: Linux, Windows

Data Formats: Matlab, Octave

Tags: Entropy, Mutual Information, Divergence, Independent Subspace Analysis, Separation Principles, Independent Process Analysis, Association Measure, Measure Of Concordance, Measure Of Independence, Nonparametric Estimation

Archive: download here

Other available revisons

Version	Changelog	Date
0.37	K divergence estimation: added, L divergence estimation: added, kNN squared distance computation: refined.	May 12, 2013, 15:35:39
0.36	Jensen-Rényi divergence estimation: added, Jensen-Shannon divergence estimation: added.	April 26, 2013, 18:45:27
0.35	An alternative Jacobi optimization based ICA solution with general entropy/mutual information estimators: added; The method extends the RADICAL ICA scheme to general objectives.	April 2, 2013, 10:37:51
0.34	Jacobi optimization based ICA solution with general entropy/mutual information estimators: added. The method extends the SWICA scheme to general objectives.	March 22, 2013, 11:41:05
0.33	Two one-dimensional Shannon entropy estimators based on the maximum entropy method: added.	March 6, 2013, 09:44:47
0.32	ICA and ISA structures: introduced for unified treatment of the estimators. It will also enable embedding of general ICA optimization algorithms such as the Jacobi method. 'stepwiseLS' mAR estimator: deleted. 'kdpee.c': MSVC does not provide log2. A more elegant solution: added.	February 25, 2013, 12:42:40
0.31	EASI (equivariant adaptive separation via independence) real/complex ICA method: added. Adaptive (k-d) partitioning based Shannon entropy estimation: added.	February 9, 2013, 11:31:39
0.30	Upper tail dependence via conditional Spearman's rho: added. Multivariate conditional version of Spearman's rho weighting the lower tail: added.	January 25, 2013, 15:11:25
0.29	Lower tail dependence via conditional Spearman's rho: added. Multivariate conditional version of Spearman's rho weighting the lower tail: added.	January 13, 2013, 11:38:17
0.28	Multivariate extension of Blomqvist's beta (medial correlation coefficient): added. Average pairwise Spearman's rho: added.	January 2, 2013, 22:51:22
0.27	Approximate correntropy independence measure estimator: added. Correntropy induced metric, centered correntropy induced metric estimators: added. Correntropy, centered correntropy, correntropy coefficient estimators: added. Handling of identically constant random variables in distance correlation computation: included.	December 28, 2012, 14:41:57
0.26	Distance covariance estimation via HSIC (Hilbert-Schmidt independence criterion): added. Energy distance estimation via MMD (maximum mean discrepancy): added. Energy distance estimation: added. We computed the square of distance correlation: sqrt included.	December 22, 2012, 14:12:33
0.25	Distance covariance, distance correlation estimation: added.	December 15, 2012, 10:20:06
0.24	MMD (maximum mean discrepancy) estimation based on U- and V-statistics: added.	December 12, 2012, 12:16:05
0.23	Three multivariate extensions of Spearman's rho (Spearman's rank correlation coefficient): added. Association (A) cost object type: added.	December 7, 2012, 18:33:26
0.22	Cauchy-Schwartz and Euclidean distance based divergence estimators: added. Cauchy-Schwartz and Euclidean distance based quadratic mutual information estimators: added.	December 1, 2012, 13:33:31
0.21	Kullback-Leibler divergence estimator based on cross-entropy and entropy: added. Cross-entropy estimation based on k-nearest neighbors: added. Cross cost object type: added.	November 25, 2012, 20:56:32
0.20	Two Shannon entropy estimators based on the distance (KL divergence) from the uniform/Gaussian distributions: added. Shannon entropy estimator based on Voronoi regions: added.	November 21, 2012, 13:55:09
0.19	Two k-nearest neighbor based Kullback-Leibler divergence estimators: added. compute_CDSS.cpp: 'sqrt(T)' -> 'sqrt(double(T))', to increase compatibility with compilers.	November 21, 2012, 13:47:05
0.18	8 sample spacing based 1d Shannon/Rényi entropy estimators: added.	November 10, 2012, 12:53:37
0.17	Edgeworth expansion based Shannon entropy estimator: accelerated (C++ alternative). 'Tsallis entropy <- Renyi entropy' meta estimator: added.	November 6, 2012, 22:18:43
0.16	Edgeworth expansion based Shannon entropy estimator: added. Lookup table for the underlying H/I/D estimation formulas: added (documentation).	November 2, 2012, 16:03:25
0.15	The Hellinger and Bhattacharyya distances are now available in ITE. They can be estimated via k-nearest neighbor methods. A '/'->'*' typo: corrected in 'DL2_kNN_k_estimation.m'.	October 29, 2012, 17:27:28
0.14	Monte-Carlo simulation to compute the additive constants in Rényi entropy estimation: added. some accelerations/compatibility enhancements: performed.	October 29, 2012, 12:31:34
0.13	Tsallis entropy is now available in ITE. It can be estimated via k-nearest neighbors. A '/'->'*' typo: corrected in 'HRenyi_kNN_k_estimation.m'.	October 27, 2012, 22:14:18
0.12	Schweizer-Wolff's sigma and kappa: added. Hoeffding's Phi computation: scaled-up.	October 27, 2012, 21:21:27
0.11	multivariate version of Hoeffding's Phi: added.	October 20, 2012, 23:30:17
0.1	Initial Announcement on mloss.org.	October 11, 2012, 07:47:43

Comments

No one has posted any comments yet. Perhaps you'd like to be the first?

Leave a comment

You must be logged in to post comments.