mloss.org The Infinite Hidden Markov Model

The Infinite Hidden Markov Model 0.5

jurgen van gael — Wed, 21 Jul 2010 23:41:24 -0000

A Matlab program which implements the beam sampler for an infinite hidden Markov model with multinomial output. Easy to extend to other output distributions. Also included is a collapsed Gibbs sampler for comparison.

Comment by Jonathan Laserson on 2009-11-12 06:19

Jonathan Laserson — Thu, 12 Nov 2009 06:19:02 -0000

Couldn't run the files. Missing implementation for the function randgamma and dirichlet_sample. What files am I missing?

Comment by Jurgen Van Gael on 2009-11-12 10:34

Jurgen Van Gael — Thu, 12 Nov 2009 10:34:08 -0000

The iHMM software depends on Tom Minka's lightspeed toolbox. The latest version of lightspeed has some issues, I will fix this this weekend.

Cheers, Jurgen

Comment by Jurgen Van Gael on 2010-07-21 23:42

Jurgen Van Gael — Wed, 21 Jul 2010 23:42:29 -0000

My weekend was a bit longer than expected. Hope things work out now ;)

Comment by Cao Thao on 2010-07-22 04:27

Cao Thao — Thu, 22 Jul 2010 04:27:40 -0000

Dear J. Van. Gael,

Does iHMM can be run in outModel is 'ar1', when I test this with generated data is [Y, STrue] = HmmGenerateData(1,T,pi,A,E,'ar1'); ... [S stats] = iHmmSampleBeam(Y, hypers, 500, 1, 1, ceil(rand(1,T) * 10));

There is error: ??? Attempted to access sample.Phi(1,0.0114177); index must be a positive integer or logical.

Error in ==> iHmmSampleBeam at 125 dynprog(k,1) = sample.Phi(k, Y(1)) * dynprog(k,1);

I think it may require the data is interger... Could you help me to solve this problem? Thank you very much, Cao Thao

Comment by Jurgen Van Gael on 2010-07-22 09:43

Jurgen Van Gael — Thu, 22 Jul 2010 09:43:16 -0000

Hi Cao,

No, the iHMMSampleBeam runs on discrete data and the iHMMNormalSampleBeam runs on normally distributed data. It's not too complicated to adapt the iHMMNormalSample beam to include the AR1 dependency though. It is essentially just changing the line which evaluates the likelihood in the dynamic programming section.

Cheers, Jurgen

Comment by kamel ait-mohand on 2010-09-20 13:14

kamel ait-mohand — Mon, 20 Sep 2010 13:14:26 -0000

Is there a simple way to extend the code so as to have: - mixture of Gausssians instead of one single Gaussian per state - learn from multiple sequences of data instead one one single sequence

Comment by Jurgen Van Gael on 2010-09-22 15:27

Jurgen Van Gael — Wed, 22 Sep 2010 15:27:33 -0000

Hi Kemal,

Yes this should be fairly easy. It would just require introducing an extra mixture model parameter sampling step at the end of the beam iteration loop. Hope this helps ...

Comment by kamel ait-mohand on 2010-09-28 11:44

kamel ait-mohand — Tue, 28 Sep 2010 11:44:09 -0000

Hi Jurgen, thank you for replying. So, I have to call the "SampleNormalMeans" function as many times as the number of Gaussians in the mixture. Is it correct?

What about making the Gaussian means multi-dimensional (for multi-dimensional data)? Is it possible and how? Last question : Is the algorithm efficient for large dimensionality data (~200-300 dimensions)?

Comment by Jurgen Van Gael on 2010-09-28 16:43

Jurgen Van Gael — Tue, 28 Sep 2010 16:43:21 -0000

Hi Kamel,

No, you'll have to replace SampleNormalMeans with your own function for resampling the parameters of a mixture model.

You could rewrite SampleNormalMeans for multidimensional data, that should not be too complicated and would retain the same structure as the current code. As for high dimensions, I think the biggest issue will be that you'll have to store a 300 dimensional covariance matrix and if you have a lot of states that might require a fair amount of RAM.

Comment by Sumithra Surendralal on 2014-02-11 17:32

Sumithra Surendralal — Tue, 11 Feb 2014 17:32:32 -0000

Hi Jurgen,

Could you give me a reference for the calcualation of the joint log-likelihood in 'iHMMJointLogLikelihood.m'? I'm trying to understand it.

Thanks, Sumithra

Comment by David Pfau on 2014-10-03 22:57

David Pfau — Fri, 03 Oct 2014 22:57:56 -0000

I'm not sure that I understand the resampling of the pi stick lengths at line 104 in iHmmSampleBeam.m

If I understand correctly, you have the stick for each pi that corresponds to the "unseen" state, and you need to break that in two once you've instantiated a new state. We can call this pi{k}{u}, corresponding to beta{u}, the remainder of the beta stick that also corresponds to all unseen states.

Having already broken beta{u} into beta{u} and beta{u+1}, you need to break pi{k}{u} as well for all states k. The fraction of pi{k}{u} that goes to the first stick is distributed as Beta(alpha * beta{u}, alpha * beta{u+1}), right? Then what's with the sum over all the other stick lengths going on in line 104?

Comment by Federica cloname on 2016-09-24 00:27

Federica cloname — Sat, 24 Sep 2016 00:27:27 -0000

Dear Jurgen,

I'm trying to implement iHMM with your code. My dataset is of about 100 strings of length varying from 1 to 130.

How can I modify the functions to make them run on all the dataset at the same time? (i.e. getting an estimate for the number of hidden states and transition and emission probabilities that would take in consideration all the strings)

Thank you, Federica