Hidden Markov Model

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Hidden Markov Model

Alex Williams
I coded up a simple HMM module in julia over the last few days. Mostly to teach myself the nuts and bolts of it, I don't expect this to be immediately useful to anyone.

https://github.com/ahwillia/ToyHMM.jl

That being said, I'd love to hear any comments/criticisms. I'm interested in pushing this a bit further if there is interest. Maybe following the rough outline here:

https://github.com/nfoti/hmm.jl


--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Hidden Markov Model

STAR0SS
Seems nice, I think the hard part if you want to do something more general is to be able to take advantage of transition probability that are separable, 
i.e. if x and y are two hidden variables and the transition probability goes like P(x,y) ~ P(x)P(y).

The separable case is much faster if you take advantage of it, you basically do two small matrix multiplications instead of a large one.

I did something of the sort at some point, where the user could provide the form of the transition probabilities with a formula, but I used a lot
of meta-programming so it was very ugly. 

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Hidden Markov Model

STAR0SS
By P(x,y) ~ P(x)P(y) I was meaning:


P(x,x',y,y') ~ P(x,x')P(y,y').

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Hidden Markov Model

Alex Williams
Thanks to the Distributions package, extending it to have arbitrary emission probability distributions wasn't too hard. I have a new repository here:

https://github.com/ahwillia/HMM.jl

I like the idea of extending it to have multiple (separable) hidden variables. Also allowing the emission distribution to be conditioned on other observed variables would be useful. E.g. each observation o ~ p(theta | c), where c is some independent variable that is observed alongside each o.

Is there an easy way of representing a conditional distribution in Julia? I.e. some Type that spits back a distribution if you supply it with a value for "c" (in my example).


On Monday, July 13, 2015 at 5:31:37 AM UTC-7, [hidden email] wrote:
By P(x,y) ~ P(x)P(y) I was meaning:


P(x,x',y,y') ~ P(x,x')P(y,y').

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Hidden Markov Model

John Myles White
I would make any conditional distributions you need be independent distribution types, then provide a function that given a joint distribution and values to condition produces an object of that conditional distribution type.

This essentially is what the conjugate prior/posterior functions do in Distributions already.

 -- John

On Jul 19, 2015, at 12:33 AM, Alex Williams <[hidden email]> wrote:

Thanks to the Distributions package, extending it to have arbitrary emission probability distributions wasn't too hard. I have a new repository here:


I like the idea of extending it to have multiple (separable) hidden variables. Also allowing the emission distribution to be conditioned on other observed variables would be useful. E.g. each observation o ~ p(theta | c), where c is some independent variable that is observed alongside each o.

Is there an easy way of representing a conditional distribution in Julia? I.e. some Type that spits back a distribution if you supply it with a value for "c" (in my example).


On Monday, July 13, 2015 at 5:31:37 AM UTC-7, yu...@altern.org wrote:
By P(x,y) ~ P(x)P(y) I was meaning:


P(x,x',y,y') ~ P(x,x')P(y,y').

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Hidden Markov Model

STAR0SS
I have my code here, but like I said it's quite ugly:

https://github.com/Staross/tests/blob/master/testHMM2.jl

I kind of like how you have to define things, it's relatively close to how you write things on paper:

https://gist.github.com/Staross/56ca6d7c546757bdb55a

Also if you want performance you'll need to use matrix multiplications instead of loops.

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Hidden Markov Model

Kevin Squire

On Fri, Jul 24, 2015 at 7:22 AM, <[hidden email]> wrote:
I have my code here, but like I said it's quite ugly:


I kind of like how you have to define things, it's relatively close to how you write things on paper:


Also if you want performance you'll need to use matrix multiplications instead of loops.

Have you tested this?  In Julia, this often (usually?) isn't true, although for large matrices, calling out to BLAS is probably better.

Cheers,
   Kevin

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Hidden Markov Model

STAR0SS
Yes I've tested it quite extensively, if you can write your problem in term of matrix multiplication it's almost always worth it to use them in my experience (if you care about performance that is).

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.