Multinomial Logistic Regression

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Multinomial Logistic Regression

Benjamin Deonovic
Can someone point me to the best place to do multinomial logistic regression in Julia? I've tried with https://github.com/lindahua/Regression.jl but I am not getting the right results according to my simulation, I'm not sure how to interpret the output from that package (documentation is rather lacking), and I am not getting any responses on their issues (https://github.com/lindahua/Regression.jl/issues/14).


--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Multinomial Logistic Regression

Andreas Noack
I think that Dahua doesn't have as much time for Julia package development as he used to have. I'm not aware of other implementations but there could easily be some that I don't know of. For now, it will probably be easiest to roll your own version or fork Dahua's package.

On Fri, Mar 25, 2016 at 12:51 PM, Benjamin Deonovic <[hidden email]> wrote:
Can someone point me to the best place to do multinomial logistic regression in Julia? I've tried with https://github.com/lindahua/Regression.jl but I am not getting the right results according to my simulation, I'm not sure how to interpret the output from that package (documentation is rather lacking), and I am not getting any responses on their issues (https://github.com/lindahua/Regression.jl/issues/14).


--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Multinomial Logistic Regression

Cedric St-Jean-2
Hi Benjamin,

You can use the scikit-learn version via ScikitLearn.jl if you don't mind the (moderate) cost of sending your data to Python.

using ScikitLearn
@sk_import linear_model: LogisticRegression

model = fit!(LogisticRegression(multi_class="multinomial"), X_train, y_train)
predict(model, X_test)

It was released recently, don't hesitate to file an issue if you get a problem with it.

Cédric

On Friday, March 25, 2016 at 12:58:43 PM UTC-4, Andreas Noack wrote:
I think that Dahua doesn't have as much time for Julia package development as he used to have. I'm not aware of other implementations but there could easily be some that I don't know of. For now, it will probably be easiest to roll your own version or fork Dahua's package.

On Fri, Mar 25, 2016 at 12:51 PM, Benjamin Deonovic <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="NNVPBi8PBgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">bdeo...@...> wrote:
Can someone point me to the best place to do multinomial logistic regression in Julia? I've tried with <a href="https://github.com/lindahua/Regression.jl" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNF1HsHOeGB05g3KM3Z7NsjQIO4f_A&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNF1HsHOeGB05g3KM3Z7NsjQIO4f_A&#39;;return true;">https://github.com/lindahua/Regression.jl but I am not getting the right results according to my simulation, I'm not sure how to interpret the output from that package (documentation is rather lacking), and I am not getting any responses on their issues (<a href="https://github.com/lindahua/Regression.jl/issues/14" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl%2Fissues%2F14\46sa\75D\46sntz\0751\46usg\75AFQjCNGHCWeQXTCSghZYiAhgiNJYwPMuXQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl%2Fissues%2F14\46sa\75D\46sntz\0751\46usg\75AFQjCNGHCWeQXTCSghZYiAhgiNJYwPMuXQ&#39;;return true;">https://github.com/lindahua/Regression.jl/issues/14).


--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="NNVPBi8PBgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">julia-stats...@googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Multinomial Logistic Regression

Benjamin Deonovic
I got the following error:

julia> model = ScikitLearn.fit!(LogisticRegression(multi_class="multinomial"), X, y)
ERROR: PyError (:PyObject_Call) <type 'exceptions.ValueError'>
ValueError('Solver liblinear does not support a multinomial backend.',)
  File "/home/benjamin/bin/miniconda/lib/python2.7/site-packages/sklearn/linear_model/logistic.py", line 1148, in fit
    self.dual, sample_weight)
  File "/home/benjamin/bin/miniconda/lib/python2.7/site-packages/sklearn/linear_model/logistic.py", line 412, in _check_solver_option
    "a multinomial backend." % solver)

 [inlined code] from /home/benjamin/.julia/v0.4/PyCall/src/exception.jl:81
 in pycall at /home/benjamin/.julia/v0.4/PyCall/src/PyCall.jl:344
 in call at /home/benjamin/.julia/v0.4/PyCall/src/PyCall.jl:372
 in fit! at /home/benjamin/.julia/v0.4/ScikitLearn/src/Skcore.jl:75




On Sunday, March 27, 2016 at 12:34:13 PM UTC-5, Cedric St-Jean wrote:
Hi Benjamin,

You can use the scikit-learn <a href="http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fscikit-learn.org%2Fstable%2Fmodules%2Fgenerated%2Fsklearn.linear_model.LogisticRegression.html\46sa\75D\46sntz\0751\46usg\75AFQjCNEoJuCyFJLOeHsJbtB6D0-tWFS6xw&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fscikit-learn.org%2Fstable%2Fmodules%2Fgenerated%2Fsklearn.linear_model.LogisticRegression.html\46sa\75D\46sntz\0751\46usg\75AFQjCNEoJuCyFJLOeHsJbtB6D0-tWFS6xw&#39;;return true;">version via <a href="https://github.com/cstjean/ScikitLearn.jl" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fcstjean%2FScikitLearn.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNFZrtpTOq_Jcx2ml0NydZWQie3plw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fcstjean%2FScikitLearn.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNFZrtpTOq_Jcx2ml0NydZWQie3plw&#39;;return true;">ScikitLearn.jl if you don't mind the (moderate) cost of sending your data to Python.

using ScikitLearn
@sk_import linear_model: LogisticRegression

model = fit!(LogisticRegression(multi_class="multinomial"), X_train, y_train)
predict(model, X_test)

It was released recently, don't hesitate to file an issue if you get a problem with it.

Cédric

On Friday, March 25, 2016 at 12:58:43 PM UTC-4, Andreas Noack wrote:
I think that Dahua doesn't have as much time for Julia package development as he used to have. I'm not aware of other implementations but there could easily be some that I don't know of. For now, it will probably be easiest to roll your own version or fork Dahua's package.

On Fri, Mar 25, 2016 at 12:51 PM, Benjamin Deonovic <[hidden email]> wrote:
Can someone point me to the best place to do multinomial logistic regression in Julia? I've tried with <a href="https://github.com/lindahua/Regression.jl" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNF1HsHOeGB05g3KM3Z7NsjQIO4f_A&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNF1HsHOeGB05g3KM3Z7NsjQIO4f_A&#39;;return true;">https://github.com/lindahua/Regression.jl but I am not getting the right results according to my simulation, I'm not sure how to interpret the output from that package (documentation is rather lacking), and I am not getting any responses on their issues (<a href="https://github.com/lindahua/Regression.jl/issues/14" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl%2Fissues%2F14\46sa\75D\46sntz\0751\46usg\75AFQjCNGHCWeQXTCSghZYiAhgiNJYwPMuXQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl%2Fissues%2F14\46sa\75D\46sntz\0751\46usg\75AFQjCNGHCWeQXTCSghZYiAhgiNJYwPMuXQ&#39;;return true;">https://github.com/lindahua/Regression.jl/issues/14).


--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to julia-stats...@googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Multinomial Logistic Regression

Benjamin Deonovic
I figured out I just need to change the solver from "liblinear" to "lbfgs"

On Wednesday, April 6, 2016 at 4:30:27 PM UTC-5, Benjamin Deonovic wrote:
I got the following error:

julia> model = ScikitLearn.fit!(LogisticRegression(multi_class="multinomial"), X, y)
ERROR: PyError (:PyObject_Call) <type 'exceptions.ValueError'>
ValueError('Solver liblinear does not support a multinomial backend.',)
  File "/home/benjamin/bin/miniconda/lib/python2.7/site-packages/sklearn/linear_model/logistic.py", line 1148, in fit
    self.dual, sample_weight)
  File "/home/benjamin/bin/miniconda/lib/python2.7/site-packages/sklearn/linear_model/logistic.py", line 412, in _check_solver_option
    "a multinomial backend." % solver)

 [inlined code] from /home/benjamin/.julia/v0.4/PyCall/src/exception.jl:81
 in pycall at /home/benjamin/.julia/v0.4/PyCall/src/PyCall.jl:344
 in call at /home/benjamin/.julia/v0.4/PyCall/src/PyCall.jl:372
 in fit! at /home/benjamin/.julia/v0.4/ScikitLearn/src/Skcore.jl:75




On Sunday, March 27, 2016 at 12:34:13 PM UTC-5, Cedric St-Jean wrote:
Hi Benjamin,

You can use the scikit-learn <a href="http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html" rel="nofollow" target="_blank" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fscikit-learn.org%2Fstable%2Fmodules%2Fgenerated%2Fsklearn.linear_model.LogisticRegression.html\46sa\75D\46sntz\0751\46usg\75AFQjCNEoJuCyFJLOeHsJbtB6D0-tWFS6xw&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fscikit-learn.org%2Fstable%2Fmodules%2Fgenerated%2Fsklearn.linear_model.LogisticRegression.html\46sa\75D\46sntz\0751\46usg\75AFQjCNEoJuCyFJLOeHsJbtB6D0-tWFS6xw&#39;;return true;">version via <a href="https://github.com/cstjean/ScikitLearn.jl" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fcstjean%2FScikitLearn.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNFZrtpTOq_Jcx2ml0NydZWQie3plw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fcstjean%2FScikitLearn.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNFZrtpTOq_Jcx2ml0NydZWQie3plw&#39;;return true;">ScikitLearn.jl if you don't mind the (moderate) cost of sending your data to Python.

using ScikitLearn
@sk_import linear_model: LogisticRegression

model = fit!(LogisticRegression(multi_class="multinomial"), X_train, y_train)
predict(model, X_test)

It was released recently, don't hesitate to file an issue if you get a problem with it.

Cédric

On Friday, March 25, 2016 at 12:58:43 PM UTC-4, Andreas Noack wrote:
I think that Dahua doesn't have as much time for Julia package development as he used to have. I'm not aware of other implementations but there could easily be some that I don't know of. For now, it will probably be easiest to roll your own version or fork Dahua's package.

On Fri, Mar 25, 2016 at 12:51 PM, Benjamin Deonovic <[hidden email]> wrote:
Can someone point me to the best place to do multinomial logistic regression in Julia? I've tried with <a href="https://github.com/lindahua/Regression.jl" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNF1HsHOeGB05g3KM3Z7NsjQIO4f_A&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNF1HsHOeGB05g3KM3Z7NsjQIO4f_A&#39;;return true;">https://github.com/lindahua/Regression.jl but I am not getting the right results according to my simulation, I'm not sure how to interpret the output from that package (documentation is rather lacking), and I am not getting any responses on their issues (<a href="https://github.com/lindahua/Regression.jl/issues/14" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl%2Fissues%2F14\46sa\75D\46sntz\0751\46usg\75AFQjCNGHCWeQXTCSghZYiAhgiNJYwPMuXQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl%2Fissues%2F14\46sa\75D\46sntz\0751\46usg\75AFQjCNGHCWeQXTCSghZYiAhgiNJYwPMuXQ&#39;;return true;">https://github.com/lindahua/Regression.jl/issues/14).


--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to julia-stats...@googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Multinomial Logistic Regression

Benjamin Deonovic
In reply to this post by Cedric St-Jean-2
How do I extract the coefficient values after I have fit the model? 

On Sunday, March 27, 2016 at 12:34:13 PM UTC-5, Cedric St-Jean wrote:
Hi Benjamin,

You can use the scikit-learn <a href="http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fscikit-learn.org%2Fstable%2Fmodules%2Fgenerated%2Fsklearn.linear_model.LogisticRegression.html\46sa\75D\46sntz\0751\46usg\75AFQjCNEoJuCyFJLOeHsJbtB6D0-tWFS6xw&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fscikit-learn.org%2Fstable%2Fmodules%2Fgenerated%2Fsklearn.linear_model.LogisticRegression.html\46sa\75D\46sntz\0751\46usg\75AFQjCNEoJuCyFJLOeHsJbtB6D0-tWFS6xw&#39;;return true;">version via <a href="https://github.com/cstjean/ScikitLearn.jl" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fcstjean%2FScikitLearn.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNFZrtpTOq_Jcx2ml0NydZWQie3plw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fcstjean%2FScikitLearn.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNFZrtpTOq_Jcx2ml0NydZWQie3plw&#39;;return true;">ScikitLearn.jl if you don't mind the (moderate) cost of sending your data to Python.

using ScikitLearn
@sk_import linear_model: LogisticRegression

model = fit!(LogisticRegression(multi_class="multinomial"), X_train, y_train)
predict(model, X_test)

It was released recently, don't hesitate to file an issue if you get a problem with it.

Cédric

On Friday, March 25, 2016 at 12:58:43 PM UTC-4, Andreas Noack wrote:
I think that Dahua doesn't have as much time for Julia package development as he used to have. I'm not aware of other implementations but there could easily be some that I don't know of. For now, it will probably be easiest to roll your own version or fork Dahua's package.

On Fri, Mar 25, 2016 at 12:51 PM, Benjamin Deonovic <[hidden email]> wrote:
Can someone point me to the best place to do multinomial logistic regression in Julia? I've tried with <a href="https://github.com/lindahua/Regression.jl" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNF1HsHOeGB05g3KM3Z7NsjQIO4f_A&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNF1HsHOeGB05g3KM3Z7NsjQIO4f_A&#39;;return true;">https://github.com/lindahua/Regression.jl but I am not getting the right results according to my simulation, I'm not sure how to interpret the output from that package (documentation is rather lacking), and I am not getting any responses on their issues (<a href="https://github.com/lindahua/Regression.jl/issues/14" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl%2Fissues%2F14\46sa\75D\46sntz\0751\46usg\75AFQjCNGHCWeQXTCSghZYiAhgiNJYwPMuXQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl%2Fissues%2F14\46sa\75D\46sntz\0751\46usg\75AFQjCNGHCWeQXTCSghZYiAhgiNJYwPMuXQ&#39;;return true;">https://github.com/lindahua/Regression.jl/issues/14).


--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to julia-stats...@googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Multinomial Logistic Regression

Cedric St-Jean-2
All Python attributes are accessible with [:attribute_name]. model[:coef_] and model[:intercept_] should work.

On Wednesday, April 6, 2016 at 5:40:31 PM UTC-4, Benjamin Deonovic wrote:
How do I extract the coefficient values after I have fit the model? 

On Sunday, March 27, 2016 at 12:34:13 PM UTC-5, Cedric St-Jean wrote:
Hi Benjamin,

You can use the scikit-learn <a href="http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html" rel="nofollow" target="_blank" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fscikit-learn.org%2Fstable%2Fmodules%2Fgenerated%2Fsklearn.linear_model.LogisticRegression.html\46sa\75D\46sntz\0751\46usg\75AFQjCNEoJuCyFJLOeHsJbtB6D0-tWFS6xw&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fscikit-learn.org%2Fstable%2Fmodules%2Fgenerated%2Fsklearn.linear_model.LogisticRegression.html\46sa\75D\46sntz\0751\46usg\75AFQjCNEoJuCyFJLOeHsJbtB6D0-tWFS6xw&#39;;return true;">version via <a href="https://github.com/cstjean/ScikitLearn.jl" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fcstjean%2FScikitLearn.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNFZrtpTOq_Jcx2ml0NydZWQie3plw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fcstjean%2FScikitLearn.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNFZrtpTOq_Jcx2ml0NydZWQie3plw&#39;;return true;">ScikitLearn.jl if you don't mind the (moderate) cost of sending your data to Python.

using ScikitLearn
@sk_import linear_model: LogisticRegression

model = fit!(LogisticRegression(multi_class="multinomial"), X_train, y_train)
predict(model, X_test)

It was released recently, don't hesitate to file an issue if you get a problem with it.

Cédric

On Friday, March 25, 2016 at 12:58:43 PM UTC-4, Andreas Noack wrote:
I think that Dahua doesn't have as much time for Julia package development as he used to have. I'm not aware of other implementations but there could easily be some that I don't know of. For now, it will probably be easiest to roll your own version or fork Dahua's package.

On Fri, Mar 25, 2016 at 12:51 PM, Benjamin Deonovic <[hidden email]> wrote:
Can someone point me to the best place to do multinomial logistic regression in Julia? I've tried with <a href="https://github.com/lindahua/Regression.jl" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNF1HsHOeGB05g3KM3Z7NsjQIO4f_A&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNF1HsHOeGB05g3KM3Z7NsjQIO4f_A&#39;;return true;">https://github.com/lindahua/Regression.jl but I am not getting the right results according to my simulation, I'm not sure how to interpret the output from that package (documentation is rather lacking), and I am not getting any responses on their issues (<a href="https://github.com/lindahua/Regression.jl/issues/14" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl%2Fissues%2F14\46sa\75D\46sntz\0751\46usg\75AFQjCNGHCWeQXTCSghZYiAhgiNJYwPMuXQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl%2Fissues%2F14\46sa\75D\46sntz\0751\46usg\75AFQjCNGHCWeQXTCSghZYiAhgiNJYwPMuXQ&#39;;return true;">https://github.com/lindahua/Regression.jl/issues/14).


--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to julia-stats...@googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Multinomial Logistic Regression

Benjamin Deonovic
Great thanks! I'm having trouble reproducing some R results for multinomial regression. Could you take a look if I am doing things right with the SciKitLearn package? http://stackoverflow.com/questions/36463072/julia-multinomial-regression-with-time-series-lagged-values

On Wednesday, April 6, 2016 at 5:06:09 PM UTC-5, Cedric St-Jean wrote:
All Python attributes are accessible with [:attribute_name]. model[:coef_] and model[:intercept_] should work.

On Wednesday, April 6, 2016 at 5:40:31 PM UTC-4, Benjamin Deonovic wrote:
How do I extract the coefficient values after I have fit the model? 

On Sunday, March 27, 2016 at 12:34:13 PM UTC-5, Cedric St-Jean wrote:
Hi Benjamin,

You can use the scikit-learn <a href="http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html" rel="nofollow" target="_blank" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fscikit-learn.org%2Fstable%2Fmodules%2Fgenerated%2Fsklearn.linear_model.LogisticRegression.html\46sa\75D\46sntz\0751\46usg\75AFQjCNEoJuCyFJLOeHsJbtB6D0-tWFS6xw&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fscikit-learn.org%2Fstable%2Fmodules%2Fgenerated%2Fsklearn.linear_model.LogisticRegression.html\46sa\75D\46sntz\0751\46usg\75AFQjCNEoJuCyFJLOeHsJbtB6D0-tWFS6xw&#39;;return true;">version via <a href="https://github.com/cstjean/ScikitLearn.jl" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fcstjean%2FScikitLearn.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNFZrtpTOq_Jcx2ml0NydZWQie3plw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fcstjean%2FScikitLearn.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNFZrtpTOq_Jcx2ml0NydZWQie3plw&#39;;return true;">ScikitLearn.jl if you don't mind the (moderate) cost of sending your data to Python.

using ScikitLearn
@sk_import linear_model: LogisticRegression

model = fit!(LogisticRegression(multi_class="multinomial"), X_train, y_train)
predict(model, X_test)

It was released recently, don't hesitate to file an issue if you get a problem with it.

Cédric

On Friday, March 25, 2016 at 12:58:43 PM UTC-4, Andreas Noack wrote:
I think that Dahua doesn't have as much time for Julia package development as he used to have. I'm not aware of other implementations but there could easily be some that I don't know of. For now, it will probably be easiest to roll your own version or fork Dahua's package.

On Fri, Mar 25, 2016 at 12:51 PM, Benjamin Deonovic <[hidden email]> wrote:
Can someone point me to the best place to do multinomial logistic regression in Julia? I've tried with <a href="https://github.com/lindahua/Regression.jl" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNF1HsHOeGB05g3KM3Z7NsjQIO4f_A&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNF1HsHOeGB05g3KM3Z7NsjQIO4f_A&#39;;return true;">https://github.com/lindahua/Regression.jl but I am not getting the right results according to my simulation, I'm not sure how to interpret the output from that package (documentation is rather lacking), and I am not getting any responses on their issues (<a href="https://github.com/lindahua/Regression.jl/issues/14" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl%2Fissues%2F14\46sa\75D\46sntz\0751\46usg\75AFQjCNGHCWeQXTCSghZYiAhgiNJYwPMuXQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl%2Fissues%2F14\46sa\75D\46sntz\0751\46usg\75AFQjCNGHCWeQXTCSghZYiAhgiNJYwPMuXQ&#39;;return true;">https://github.com/lindahua/Regression.jl/issues/14).


--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to julia-stats...@googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Multinomial Logistic Regression

Cedric St-Jean-2
The ScikitLearn.jl call looks OK. Some ideas:

- scikit-learn's regression is regularized by default. You can disable it with LogisticRegression(C=1.e9)
- you convert y to Vector{Int64}, might want to try Vector{Float64}
- isn't multinomial logistic regression non-identifiable under certain parametrizations? I would try computing the output (i.e. predict(model, X)) in both models and see if they differ

Otherwise, trying a third package might be useful in figuring out which one is wrong.

Cédric

On Wednesday, April 6, 2016 at 6:28:57 PM UTC-4, Benjamin Deonovic wrote:
Great thanks! I'm having trouble reproducing some R results for multinomial regression. Could you take a look if I am doing things right with the SciKitLearn package? <a href="http://stackoverflow.com/questions/36463072/julia-multinomial-regression-with-time-series-lagged-values" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fstackoverflow.com%2Fquestions%2F36463072%2Fjulia-multinomial-regression-with-time-series-lagged-values\46sa\75D\46sntz\0751\46usg\75AFQjCNF8y_YSJMNLIoKq0-g9bRfo74vlTA&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fstackoverflow.com%2Fquestions%2F36463072%2Fjulia-multinomial-regression-with-time-series-lagged-values\46sa\75D\46sntz\0751\46usg\75AFQjCNF8y_YSJMNLIoKq0-g9bRfo74vlTA&#39;;return true;">http://stackoverflow.com/questions/36463072/julia-multinomial-regression-with-time-series-lagged-values

On Wednesday, April 6, 2016 at 5:06:09 PM UTC-5, Cedric St-Jean wrote:
All Python attributes are accessible with [:attribute_name]. model[:coef_] and model[:intercept_] should work.

On Wednesday, April 6, 2016 at 5:40:31 PM UTC-4, Benjamin Deonovic wrote:
How do I extract the coefficient values after I have fit the model? 

On Sunday, March 27, 2016 at 12:34:13 PM UTC-5, Cedric St-Jean wrote:
Hi Benjamin,

You can use the scikit-learn <a href="http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html" rel="nofollow" target="_blank" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fscikit-learn.org%2Fstable%2Fmodules%2Fgenerated%2Fsklearn.linear_model.LogisticRegression.html\46sa\75D\46sntz\0751\46usg\75AFQjCNEoJuCyFJLOeHsJbtB6D0-tWFS6xw&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fscikit-learn.org%2Fstable%2Fmodules%2Fgenerated%2Fsklearn.linear_model.LogisticRegression.html\46sa\75D\46sntz\0751\46usg\75AFQjCNEoJuCyFJLOeHsJbtB6D0-tWFS6xw&#39;;return true;">version via <a href="https://github.com/cstjean/ScikitLearn.jl" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fcstjean%2FScikitLearn.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNFZrtpTOq_Jcx2ml0NydZWQie3plw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fcstjean%2FScikitLearn.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNFZrtpTOq_Jcx2ml0NydZWQie3plw&#39;;return true;">ScikitLearn.jl if you don't mind the (moderate) cost of sending your data to Python.

using ScikitLearn
@sk_import linear_model: LogisticRegression

model = fit!(LogisticRegression(multi_class="multinomial"), X_train, y_train)
predict(model, X_test)

It was released recently, don't hesitate to file an issue if you get a problem with it.

Cédric

On Friday, March 25, 2016 at 12:58:43 PM UTC-4, Andreas Noack wrote:
I think that Dahua doesn't have as much time for Julia package development as he used to have. I'm not aware of other implementations but there could easily be some that I don't know of. For now, it will probably be easiest to roll your own version or fork Dahua's package.

On Fri, Mar 25, 2016 at 12:51 PM, Benjamin Deonovic <[hidden email]> wrote:
Can someone point me to the best place to do multinomial logistic regression in Julia? I've tried with <a href="https://github.com/lindahua/Regression.jl" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNF1HsHOeGB05g3KM3Z7NsjQIO4f_A&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNF1HsHOeGB05g3KM3Z7NsjQIO4f_A&#39;;return true;">https://github.com/lindahua/Regression.jl but I am not getting the right results according to my simulation, I'm not sure how to interpret the output from that package (documentation is rather lacking), and I am not getting any responses on their issues (<a href="https://github.com/lindahua/Regression.jl/issues/14" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl%2Fissues%2F14\46sa\75D\46sntz\0751\46usg\75AFQjCNGHCWeQXTCSghZYiAhgiNJYwPMuXQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl%2Fissues%2F14\46sa\75D\46sntz\0751\46usg\75AFQjCNGHCWeQXTCSghZYiAhgiNJYwPMuXQ&#39;;return true;">https://github.com/lindahua/Regression.jl/issues/14).


--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to julia-stats...@googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Multinomial Logistic Regression

Cedric St-Jean-2
Note that not all of the \beta_k vectors of coefficients are uniquely identifiable. This is due to the fact that all probabilities must sum to 1, making one of them completely determined once all the rest are known. As a result there are only k-1 separately specifiable probabilities, and hence k-1 separately identifiable vectors of coefficients. One way to see this is to note that if we add a constant vector to all of the coefficient vectors, the equations are identical:

From Wikipedia. So it's expected that the coefficients are different, even if the predictions are the same. What do you want to use the coefficients for?


On Wednesday, April 6, 2016 at 7:09:09 PM UTC-4, Cedric St-Jean wrote:
The ScikitLearn.jl call looks OK. Some ideas:

- scikit-learn's regression is regularized by default. You can disable it with LogisticRegression(C=1.e9)
- you convert y to Vector{Int64}, might want to try Vector{Float64}
- isn't multinomial logistic regression non-identifiable under certain parametrizations? I would try computing the output (i.e. predict(model, X)) in both models and see if they differ

Otherwise, trying a third package might be useful in figuring out which one is wrong.

Cédric

On Wednesday, April 6, 2016 at 6:28:57 PM UTC-4, Benjamin Deonovic wrote:
Great thanks! I'm having trouble reproducing some R results for multinomial regression. Could you take a look if I am doing things right with the SciKitLearn package? <a href="http://stackoverflow.com/questions/36463072/julia-multinomial-regression-with-time-series-lagged-values" rel="nofollow" target="_blank" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fstackoverflow.com%2Fquestions%2F36463072%2Fjulia-multinomial-regression-with-time-series-lagged-values\46sa\75D\46sntz\0751\46usg\75AFQjCNF8y_YSJMNLIoKq0-g9bRfo74vlTA&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fstackoverflow.com%2Fquestions%2F36463072%2Fjulia-multinomial-regression-with-time-series-lagged-values\46sa\75D\46sntz\0751\46usg\75AFQjCNF8y_YSJMNLIoKq0-g9bRfo74vlTA&#39;;return true;">http://stackoverflow.com/questions/36463072/julia-multinomial-regression-with-time-series-lagged-values

On Wednesday, April 6, 2016 at 5:06:09 PM UTC-5, Cedric St-Jean wrote:
All Python attributes are accessible with [:attribute_name]. model[:coef_] and model[:intercept_] should work.

On Wednesday, April 6, 2016 at 5:40:31 PM UTC-4, Benjamin Deonovic wrote:
How do I extract the coefficient values after I have fit the model? 

On Sunday, March 27, 2016 at 12:34:13 PM UTC-5, Cedric St-Jean wrote:
Hi Benjamin,

You can use the scikit-learn <a href="http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html" rel="nofollow" target="_blank" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fscikit-learn.org%2Fstable%2Fmodules%2Fgenerated%2Fsklearn.linear_model.LogisticRegression.html\46sa\75D\46sntz\0751\46usg\75AFQjCNEoJuCyFJLOeHsJbtB6D0-tWFS6xw&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fscikit-learn.org%2Fstable%2Fmodules%2Fgenerated%2Fsklearn.linear_model.LogisticRegression.html\46sa\75D\46sntz\0751\46usg\75AFQjCNEoJuCyFJLOeHsJbtB6D0-tWFS6xw&#39;;return true;">version via <a href="https://github.com/cstjean/ScikitLearn.jl" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fcstjean%2FScikitLearn.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNFZrtpTOq_Jcx2ml0NydZWQie3plw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fcstjean%2FScikitLearn.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNFZrtpTOq_Jcx2ml0NydZWQie3plw&#39;;return true;">ScikitLearn.jl if you don't mind the (moderate) cost of sending your data to Python.

using ScikitLearn
@sk_import linear_model: LogisticRegression

model = fit!(LogisticRegression(multi_class="multinomial"), X_train, y_train)
predict(model, X_test)

It was released recently, don't hesitate to file an issue if you get a problem with it.

Cédric

On Friday, March 25, 2016 at 12:58:43 PM UTC-4, Andreas Noack wrote:
I think that Dahua doesn't have as much time for Julia package development as he used to have. I'm not aware of other implementations but there could easily be some that I don't know of. For now, it will probably be easiest to roll your own version or fork Dahua's package.

On Fri, Mar 25, 2016 at 12:51 PM, Benjamin Deonovic <[hidden email]> wrote:
Can someone point me to the best place to do multinomial logistic regression in Julia? I've tried with <a href="https://github.com/lindahua/Regression.jl" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNF1HsHOeGB05g3KM3Z7NsjQIO4f_A&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNF1HsHOeGB05g3KM3Z7NsjQIO4f_A&#39;;return true;">https://github.com/lindahua/Regression.jl but I am not getting the right results according to my simulation, I'm not sure how to interpret the output from that package (documentation is rather lacking), and I am not getting any responses on their issues (<a href="https://github.com/lindahua/Regression.jl/issues/14" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl%2Fissues%2F14\46sa\75D\46sntz\0751\46usg\75AFQjCNGHCWeQXTCSghZYiAhgiNJYwPMuXQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl%2Fissues%2F14\46sa\75D\46sntz\0751\46usg\75AFQjCNGHCWeQXTCSghZYiAhgiNJYwPMuXQ&#39;;return true;">https://github.com/lindahua/Regression.jl/issues/14).


--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to julia-stats...@googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Multinomial Logistic Regression

Benjamin Deonovic
Thanks, I don't need the coefficients but the estimated probabilityes, (in wikipedia's notation) $P(Y_i = k) = exp(X\beta_k)/(1 + \sum_u exp(X\beta_u))$. These should be identifiable, correct? 

On Wednesday, April 6, 2016 at 6:19:38 PM UTC-5, Cedric St-Jean wrote:
Note that not all of the \beta_k vectors of coefficients are uniquely <a href="https://en.wikipedia.org/wiki/Identifiability" title="Identifiability" style="color:rgb(11,0,128);background-image:none;font-family:sans-serif;font-size:14px" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FIdentifiability\46sa\75D\46sntz\0751\46usg\75AFQjCNF4ak2FENB0jSc5FwOigYFfESvCHw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FIdentifiability\46sa\75D\46sntz\0751\46usg\75AFQjCNF4ak2FENB0jSc5FwOigYFfESvCHw&#39;;return true;">identifiable. This is due to the fact that all probabilities must sum to 1, making one of them completely determined once all the rest are known. As a result there are only k-1 separately specifiable probabilities, and hence k-1 separately identifiable vectors of coefficients. One way to see this is to note that if we add a constant vector to all of the coefficient vectors, the equations are identical:

From Wikipedia. So it's expected that the coefficients are different, even if the predictions are the same. What do you want to use the coefficients for?


On Wednesday, April 6, 2016 at 7:09:09 PM UTC-4, Cedric St-Jean wrote:
The ScikitLearn.jl call looks OK. Some ideas:

- scikit-learn's regression is regularized by default. You can disable it with LogisticRegression(C=1.e9)
- you convert y to Vector{Int64}, might want to try Vector{Float64}
- isn't multinomial logistic regression non-identifiable under certain parametrizations? I would try computing the output (i.e. predict(model, X)) in both models and see if they differ

Otherwise, trying a third package might be useful in figuring out which one is wrong.

Cédric

On Wednesday, April 6, 2016 at 6:28:57 PM UTC-4, Benjamin Deonovic wrote:
Great thanks! I'm having trouble reproducing some R results for multinomial regression. Could you take a look if I am doing things right with the SciKitLearn package? <a href="http://stackoverflow.com/questions/36463072/julia-multinomial-regression-with-time-series-lagged-values" rel="nofollow" target="_blank" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fstackoverflow.com%2Fquestions%2F36463072%2Fjulia-multinomial-regression-with-time-series-lagged-values\46sa\75D\46sntz\0751\46usg\75AFQjCNF8y_YSJMNLIoKq0-g9bRfo74vlTA&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fstackoverflow.com%2Fquestions%2F36463072%2Fjulia-multinomial-regression-with-time-series-lagged-values\46sa\75D\46sntz\0751\46usg\75AFQjCNF8y_YSJMNLIoKq0-g9bRfo74vlTA&#39;;return true;">http://stackoverflow.com/questions/36463072/julia-multinomial-regression-with-time-series-lagged-values

On Wednesday, April 6, 2016 at 5:06:09 PM UTC-5, Cedric St-Jean wrote:
All Python attributes are accessible with [:attribute_name]. model[:coef_] and model[:intercept_] should work.

On Wednesday, April 6, 2016 at 5:40:31 PM UTC-4, Benjamin Deonovic wrote:
How do I extract the coefficient values after I have fit the model? 

On Sunday, March 27, 2016 at 12:34:13 PM UTC-5, Cedric St-Jean wrote:
Hi Benjamin,

You can use the scikit-learn <a href="http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html" rel="nofollow" target="_blank" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fscikit-learn.org%2Fstable%2Fmodules%2Fgenerated%2Fsklearn.linear_model.LogisticRegression.html\46sa\75D\46sntz\0751\46usg\75AFQjCNEoJuCyFJLOeHsJbtB6D0-tWFS6xw&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fscikit-learn.org%2Fstable%2Fmodules%2Fgenerated%2Fsklearn.linear_model.LogisticRegression.html\46sa\75D\46sntz\0751\46usg\75AFQjCNEoJuCyFJLOeHsJbtB6D0-tWFS6xw&#39;;return true;">version via <a href="https://github.com/cstjean/ScikitLearn.jl" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fcstjean%2FScikitLearn.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNFZrtpTOq_Jcx2ml0NydZWQie3plw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fcstjean%2FScikitLearn.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNFZrtpTOq_Jcx2ml0NydZWQie3plw&#39;;return true;">ScikitLearn.jl if you don't mind the (moderate) cost of sending your data to Python.

using ScikitLearn
@sk_import linear_model: LogisticRegression

model = fit!(LogisticRegression(multi_class="multinomial"), X_train, y_train)
predict(model, X_test)

It was released recently, don't hesitate to file an issue if you get a problem with it.

Cédric

On Friday, March 25, 2016 at 12:58:43 PM UTC-4, Andreas Noack wrote:
I think that Dahua doesn't have as much time for Julia package development as he used to have. I'm not aware of other implementations but there could easily be some that I don't know of. For now, it will probably be easiest to roll your own version or fork Dahua's package.

On Fri, Mar 25, 2016 at 12:51 PM, Benjamin Deonovic <[hidden email]> wrote:
Can someone point me to the best place to do multinomial logistic regression in Julia? I've tried with <a href="https://github.com/lindahua/Regression.jl" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNF1HsHOeGB05g3KM3Z7NsjQIO4f_A&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNF1HsHOeGB05g3KM3Z7NsjQIO4f_A&#39;;return true;">https://github.com/lindahua/Regression.jl but I am not getting the right results according to my simulation, I'm not sure how to interpret the output from that package (documentation is rather lacking), and I am not getting any responses on their issues (<a href="https://github.com/lindahua/Regression.jl/issues/14" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl%2Fissues%2F14\46sa\75D\46sntz\0751\46usg\75AFQjCNGHCWeQXTCSghZYiAhgiNJYwPMuXQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl%2Fissues%2F14\46sa\75D\46sntz\0751\46usg\75AFQjCNGHCWeQXTCSghZYiAhgiNJYwPMuXQ&#39;;return true;">https://github.com/lindahua/Regression.jl/issues/14).


--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to julia-stats...@googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Multinomial Logistic Regression

Cedric St-Jean-2
Yeah. Use `predict_proba(model, X)` to get them.

On Thu, Apr 7, 2016 at 3:00 PM, Benjamin Deonovic <[hidden email]> wrote:
Thanks, I don't need the coefficients but the estimated probabilityes, (in wikipedia's notation) $P(Y_i = k) = exp(X\beta_k)/(1 + \sum_u exp(X\beta_u))$. These should be identifiable, correct? 


On Wednesday, April 6, 2016 at 6:19:38 PM UTC-5, Cedric St-Jean wrote:
Note that not all of the \beta_k vectors of coefficients are uniquely identifiable. This is due to the fact that all probabilities must sum to 1, making one of them completely determined once all the rest are known. As a result there are only k-1 separately specifiable probabilities, and hence k-1 separately identifiable vectors of coefficients. One way to see this is to note that if we add a constant vector to all of the coefficient vectors, the equations are identical:

From Wikipedia. So it's expected that the coefficients are different, even if the predictions are the same. What do you want to use the coefficients for?


On Wednesday, April 6, 2016 at 7:09:09 PM UTC-4, Cedric St-Jean wrote:
The ScikitLearn.jl call looks OK. Some ideas:

- scikit-learn's regression is regularized by default. You can disable it with LogisticRegression(C=1.e9)
- you convert y to Vector{Int64}, might want to try Vector{Float64}
- isn't multinomial logistic regression non-identifiable under certain parametrizations? I would try computing the output (i.e. predict(model, X)) in both models and see if they differ

Otherwise, trying a third package might be useful in figuring out which one is wrong.

Cédric

On Wednesday, April 6, 2016 at 6:28:57 PM UTC-4, Benjamin Deonovic wrote:
Great thanks! I'm having trouble reproducing some R results for multinomial regression. Could you take a look if I am doing things right with the SciKitLearn package? http://stackoverflow.com/questions/36463072/julia-multinomial-regression-with-time-series-lagged-values

On Wednesday, April 6, 2016 at 5:06:09 PM UTC-5, Cedric St-Jean wrote:
All Python attributes are accessible with [:attribute_name]. model[:coef_] and model[:intercept_] should work.

On Wednesday, April 6, 2016 at 5:40:31 PM UTC-4, Benjamin Deonovic wrote:
How do I extract the coefficient values after I have fit the model? 

On Sunday, March 27, 2016 at 12:34:13 PM UTC-5, Cedric St-Jean wrote:
Hi Benjamin,

You can use the scikit-learn version via ScikitLearn.jl if you don't mind the (moderate) cost of sending your data to Python.

using ScikitLearn
@sk_import linear_model: LogisticRegression

model = fit!(LogisticRegression(multi_class="multinomial"), X_train, y_train)
predict(model, X_test)

It was released recently, don't hesitate to file an issue if you get a problem with it.

Cédric

On Friday, March 25, 2016 at 12:58:43 PM UTC-4, Andreas Noack wrote:
I think that Dahua doesn't have as much time for Julia package development as he used to have. I'm not aware of other implementations but there could easily be some that I don't know of. For now, it will probably be easiest to roll your own version or fork Dahua's package.

On Fri, Mar 25, 2016 at 12:51 PM, Benjamin Deonovic <[hidden email]> wrote:
Can someone point me to the best place to do multinomial logistic regression in Julia? I've tried with https://github.com/lindahua/Regression.jl but I am not getting the right results according to my simulation, I'm not sure how to interpret the output from that package (documentation is rather lacking), and I am not getting any responses on their issues (https://github.com/lindahua/Regression.jl/issues/14).


--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "julia-stats" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/julia-stats/2pHziogT9LI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Multinomial Logistic Regression

Benjamin Deonovic
Thanks Cedric, you've been a great help. It looks like all of the predicted probabilities are the same across all the different implementations (R, Regression.jl, and ScikitLearn) thanks for taking your time to help me with this. 

On Thursday, April 7, 2016 at 2:06:24 PM UTC-5, Cedric St-Jean wrote:
Yeah. Use `predict_proba(model, X)` to get them.

On Thu, Apr 7, 2016 at 3:00 PM, Benjamin Deonovic <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="a-sAsBLtGgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">bdeo...@...> wrote:
Thanks, I don't need the coefficients but the estimated probabilityes, (in wikipedia's notation) $P(Y_i = k) = exp(X\beta_k)/(1 + \sum_u exp(X\beta_u))$. These should be identifiable, correct? 


On Wednesday, April 6, 2016 at 6:19:38 PM UTC-5, Cedric St-Jean wrote:
Note that not all of the \beta_k vectors of coefficients are uniquely <a href="https://en.wikipedia.org/wiki/Identifiability" title="Identifiability" style="color:rgb(11,0,128);background-image:none;font-family:sans-serif;font-size:14px" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FIdentifiability\46sa\75D\46sntz\0751\46usg\75AFQjCNF4ak2FENB0jSc5FwOigYFfESvCHw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FIdentifiability\46sa\75D\46sntz\0751\46usg\75AFQjCNF4ak2FENB0jSc5FwOigYFfESvCHw&#39;;return true;">identifiable. This is due to the fact that all probabilities must sum to 1, making one of them completely determined once all the rest are known. As a result there are only k-1 separately specifiable probabilities, and hence k-1 separately identifiable vectors of coefficients. One way to see this is to note that if we add a constant vector to all of the coefficient vectors, the equations are identical:

From Wikipedia. So it's expected that the coefficients are different, even if the predictions are the same. What do you want to use the coefficients for?


On Wednesday, April 6, 2016 at 7:09:09 PM UTC-4, Cedric St-Jean wrote:
The ScikitLearn.jl call looks OK. Some ideas:

- scikit-learn's regression is regularized by default. You can disable it with LogisticRegression(C=1.e9)
- you convert y to Vector{Int64}, might want to try Vector{Float64}
- isn't multinomial logistic regression non-identifiable under certain parametrizations? I would try computing the output (i.e. predict(model, X)) in both models and see if they differ

Otherwise, trying a third package might be useful in figuring out which one is wrong.

Cédric

On Wednesday, April 6, 2016 at 6:28:57 PM UTC-4, Benjamin Deonovic wrote:
Great thanks! I'm having trouble reproducing some R results for multinomial regression. Could you take a look if I am doing things right with the SciKitLearn package? <a href="http://stackoverflow.com/questions/36463072/julia-multinomial-regression-with-time-series-lagged-values" rel="nofollow" target="_blank" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fstackoverflow.com%2Fquestions%2F36463072%2Fjulia-multinomial-regression-with-time-series-lagged-values\46sa\75D\46sntz\0751\46usg\75AFQjCNF8y_YSJMNLIoKq0-g9bRfo74vlTA&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fstackoverflow.com%2Fquestions%2F36463072%2Fjulia-multinomial-regression-with-time-series-lagged-values\46sa\75D\46sntz\0751\46usg\75AFQjCNF8y_YSJMNLIoKq0-g9bRfo74vlTA&#39;;return true;">http://stackoverflow.com/questions/36463072/julia-multinomial-regression-with-time-series-lagged-values

On Wednesday, April 6, 2016 at 5:06:09 PM UTC-5, Cedric St-Jean wrote:
All Python attributes are accessible with [:attribute_name]. model[:coef_] and model[:intercept_] should work.

On Wednesday, April 6, 2016 at 5:40:31 PM UTC-4, Benjamin Deonovic wrote:
How do I extract the coefficient values after I have fit the model? 

On Sunday, March 27, 2016 at 12:34:13 PM UTC-5, Cedric St-Jean wrote:
Hi Benjamin,

You can use the scikit-learn <a href="http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html" rel="nofollow" target="_blank" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fscikit-learn.org%2Fstable%2Fmodules%2Fgenerated%2Fsklearn.linear_model.LogisticRegression.html\46sa\75D\46sntz\0751\46usg\75AFQjCNEoJuCyFJLOeHsJbtB6D0-tWFS6xw&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fscikit-learn.org%2Fstable%2Fmodules%2Fgenerated%2Fsklearn.linear_model.LogisticRegression.html\46sa\75D\46sntz\0751\46usg\75AFQjCNEoJuCyFJLOeHsJbtB6D0-tWFS6xw&#39;;return true;">version via <a href="https://github.com/cstjean/ScikitLearn.jl" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fcstjean%2FScikitLearn.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNFZrtpTOq_Jcx2ml0NydZWQie3plw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fcstjean%2FScikitLearn.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNFZrtpTOq_Jcx2ml0NydZWQie3plw&#39;;return true;">ScikitLearn.jl if you don't mind the (moderate) cost of sending your data to Python.

using ScikitLearn
@sk_import linear_model: LogisticRegression

model = fit!(LogisticRegression(multi_class="multinomial"), X_train, y_train)
predict(model, X_test)

It was released recently, don't hesitate to file an issue if you get a problem with it.

Cédric

On Friday, March 25, 2016 at 12:58:43 PM UTC-4, Andreas Noack wrote:
I think that Dahua doesn't have as much time for Julia package development as he used to have. I'm not aware of other implementations but there could easily be some that I don't know of. For now, it will probably be easiest to roll your own version or fork Dahua's package.

On Fri, Mar 25, 2016 at 12:51 PM, Benjamin Deonovic <[hidden email]> wrote:
Can someone point me to the best place to do multinomial logistic regression in Julia? I've tried with <a href="https://github.com/lindahua/Regression.jl" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNF1HsHOeGB05g3KM3Z7NsjQIO4f_A&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNF1HsHOeGB05g3KM3Z7NsjQIO4f_A&#39;;return true;">https://github.com/lindahua/Regression.jl but I am not getting the right results according to my simulation, I'm not sure how to interpret the output from that package (documentation is rather lacking), and I am not getting any responses on their issues (<a href="https://github.com/lindahua/Regression.jl/issues/14" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl%2Fissues%2F14\46sa\75D\46sntz\0751\46usg\75AFQjCNGHCWeQXTCSghZYiAhgiNJYwPMuXQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Flindahua%2FRegression.jl%2Fissues%2F14\46sa\75D\46sntz\0751\46usg\75AFQjCNGHCWeQXTCSghZYiAhgiNJYwPMuXQ&#39;;return true;">https://github.com/lindahua/Regression.jl/issues/14).


--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to julia-stats...@googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "julia-stats" group.
To unsubscribe from this topic, visit <a href="https://groups.google.com/d/topic/julia-stats/2pHziogT9LI/unsubscribe" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/topic/julia-stats/2pHziogT9LI/unsubscribe&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/topic/julia-stats/2pHziogT9LI/unsubscribe&#39;;return true;">https://groups.google.com/d/topic/julia-stats/2pHziogT9LI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="a-sAsBLtGgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">julia-stats...@googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.