[ANN] StatsFuns & refactoring of StatsBase & Distributions

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

[ANN] StatsFuns & refactoring of StatsBase & Distributions

Dahua Lin
In the past several days, I did some refactoring of StatsBase & Distributions packages. The primary purpose is to decouple higher-level stuff and low-level computational routines. 

Specifically, I created a new package StatsFuns:

https://github.com/JuliaStats/StatsFuns.jl

This package provides a set of numerical routines for statistical computing -- the implementations of these functions are either migrated from StatsBase or Distributions, or delegated to Rmath. With this refactoring, Distributions is depending on StatsFuns, while the efforts of replacing Rmath functions will happen in StatsFuns. In this way, contributors of these two packages will focus on different aspects of the development.

Ultimately, the role of StatsFuns to JuliaStats would be similar to the role of Rmath to R. 

Also, if you have a package that requires certain stats functions (instead of full-fledged distributions), you can have the package depend on StatsFuns instead of Distributions.

For backward compatibility, several functions (e.g. logistic, logit, logsumexp, softmax, etc) are imported from StatsFuns to StatsBase, and re-exported therefrom. We will consider removing these functions from StatsBase in future. So if you are using these functions, please do `using StatsFuns`. 

Cheers,
Dahua




--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] StatsFuns & refactoring of StatsBase & Distributions

Dahua Lin
Note that these changes are completely backward compatible. It is internal restructuring, and all client codes should work as before.


On Sunday, July 26, 2015 at 11:47:59 PM UTC+8, Dahua Lin wrote:
In the past several days, I did some refactoring of StatsBase & Distributions packages. The primary purpose is to decouple higher-level stuff and low-level computational routines. 

Specifically, I created a new package StatsFuns:

<a href="https://github.com/JuliaStats/StatsFuns.jl" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2FJuliaStats%2FStatsFuns.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNH1_woCYNhlWFtIYgFrbFsW-h_urg&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2FJuliaStats%2FStatsFuns.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNH1_woCYNhlWFtIYgFrbFsW-h_urg&#39;;return true;">https://github.com/JuliaStats/StatsFuns.jl

This package provides a set of numerical routines for statistical computing -- the implementations of these functions are either migrated from StatsBase or Distributions, or delegated to Rmath. With this refactoring, Distributions is depending on StatsFuns, while the efforts of replacing Rmath functions will happen in StatsFuns. In this way, contributors of these two packages will focus on different aspects of the development.

Ultimately, the role of StatsFuns to JuliaStats would be similar to the role of Rmath to R. 

Also, if you have a package that requires certain stats functions (instead of full-fledged distributions), you can have the package depend on StatsFuns instead of Distributions.

For backward compatibility, several functions (e.g. logistic, logit, logsumexp, softmax, etc) are imported from StatsFuns to StatsBase, and re-exported therefrom. We will consider removing these functions from StatsBase in future. So if you are using these functions, please do `using StatsFuns`. 

Cheers,
Dahua




--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] StatsFuns & refactoring of StatsBase & Distributions

Andreas Noack
Great work. I think it makes perfect sense to have these factored out.

My only minor comment is that I'd preferred Functions to Funs.

On Sun, Jul 26, 2015 at 12:39 PM, Dahua Lin <[hidden email]> wrote:
Note that these changes are completely backward compatible. It is internal restructuring, and all client codes should work as before.


On Sunday, July 26, 2015 at 11:47:59 PM UTC+8, Dahua Lin wrote:
In the past several days, I did some refactoring of StatsBase & Distributions packages. The primary purpose is to decouple higher-level stuff and low-level computational routines. 

Specifically, I created a new package StatsFuns:


This package provides a set of numerical routines for statistical computing -- the implementations of these functions are either migrated from StatsBase or Distributions, or delegated to Rmath. With this refactoring, Distributions is depending on StatsFuns, while the efforts of replacing Rmath functions will happen in StatsFuns. In this way, contributors of these two packages will focus on different aspects of the development.

Ultimately, the role of StatsFuns to JuliaStats would be similar to the role of Rmath to R. 

Also, if you have a package that requires certain stats functions (instead of full-fledged distributions), you can have the package depend on StatsFuns instead of Distributions.

For backward compatibility, several functions (e.g. logistic, logit, logsumexp, softmax, etc) are imported from StatsFuns to StatsBase, and re-exported therefrom. We will consider removing these functions from StatsBase in future. So if you are using these functions, please do `using StatsFuns`. 

Cheers,
Dahua




--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] StatsFuns & refactoring of StatsBase & Distributions

Amelio Vazquez-Reina
Thank you Dahua -- This is great. 

Also +1 for StatsFunctions.

Amelio

On Mon, Jul 27, 2015 at 10:41 AM Andreas Noack <[hidden email]> wrote:
Great work. I think it makes perfect sense to have these factored out.

My only minor comment is that I'd preferred Functions to Funs.

On Sun, Jul 26, 2015 at 12:39 PM, Dahua Lin <[hidden email]> wrote:
Note that these changes are completely backward compatible. It is internal restructuring, and all client codes should work as before.


On Sunday, July 26, 2015 at 11:47:59 PM UTC+8, Dahua Lin wrote:
In the past several days, I did some refactoring of StatsBase & Distributions packages. The primary purpose is to decouple higher-level stuff and low-level computational routines. 

Specifically, I created a new package StatsFuns:


This package provides a set of numerical routines for statistical computing -- the implementations of these functions are either migrated from StatsBase or Distributions, or delegated to Rmath. With this refactoring, Distributions is depending on StatsFuns, while the efforts of replacing Rmath functions will happen in StatsFuns. In this way, contributors of these two packages will focus on different aspects of the development.

Ultimately, the role of StatsFuns to JuliaStats would be similar to the role of Rmath to R. 

Also, if you have a package that requires certain stats functions (instead of full-fledged distributions), you can have the package depend on StatsFuns instead of Distributions.

For backward compatibility, several functions (e.g. logistic, logit, logsumexp, softmax, etc) are imported from StatsFuns to StatsBase, and re-exported therefrom. We will consider removing these functions from StatsBase in future. So if you are using these functions, please do `using StatsFuns`. 

Cheers,
Dahua




--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] StatsFuns & refactoring of StatsBase & Distributions

Stefan Karpinski-2
This seems like a good change but it would be better to get some consensus on this kind of massive refactoring beforehand. Can you summarize what you changed here and what the issues with the old approaches were?

On Jul 27, 2015, at 12:53 PM, Amelio Vazquez-Reina <[hidden email]> wrote:

Thank you Dahua -- This is great. 

Also +1 for StatsFunctions.

Amelio

On Mon, Jul 27, 2015 at 10:41 AM Andreas Noack <[hidden email]> wrote:
Great work. I think it makes perfect sense to have these factored out.

My only minor comment is that I'd preferred Functions to Funs.

On Sun, Jul 26, 2015 at 12:39 PM, Dahua Lin <[hidden email]> wrote:
Note that these changes are completely backward compatible. It is internal restructuring, and all client codes should work as before.


On Sunday, July 26, 2015 at 11:47:59 PM UTC+8, Dahua Lin wrote:
In the past several days, I did some refactoring of StatsBase & Distributions packages. The primary purpose is to decouple higher-level stuff and low-level computational routines. 

Specifically, I created a new package StatsFuns:


This package provides a set of numerical routines for statistical computing -- the implementations of these functions are either migrated from StatsBase or Distributions, or delegated to Rmath. With this refactoring, Distributions is depending on StatsFuns, while the efforts of replacing Rmath functions will happen in StatsFuns. In this way, contributors of these two packages will focus on different aspects of the development.

Ultimately, the role of StatsFuns to JuliaStats would be similar to the role of Rmath to R. 

Also, if you have a package that requires certain stats functions (instead of full-fledged distributions), you can have the package depend on StatsFuns instead of Distributions.

For backward compatibility, several functions (e.g. logistic, logit, logsumexp, softmax, etc) are imported from StatsFuns to StatsBase, and re-exported therefrom. We will consider removing these functions from StatsBase in future. So if you are using these functions, please do `using StatsFuns`. 

Cheers,
Dahua




--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] StatsFuns & refactoring of StatsBase & Distributions

Dahua Lin
Actually, the refactoring is quite moderate. 

Basically, I moved some low-level computational routines and Rmath functions to StatsFuns (Distributions then import them from StatsFuns), and did some minor clean-up. The readme of StatsFuns provides the list of functions.

On Tuesday, July 28, 2015 at 2:17:29 AM UTC+8, Stefan Karpinski wrote:
This seems like a good change but it would be better to get some consensus on this kind of massive refactoring beforehand. Can you summarize what you changed here and what the issues with the old approaches were?

On Jul 27, 2015, at 12:53 PM, Amelio Vazquez-Reina <<a href="javascript:" target="_blank" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">amelio.vaz...@...> wrote:

Thank you Dahua -- This is great. 

Also +1 for StatsFunctions.

Amelio

On Mon, Jul 27, 2015 at 10:41 AM Andreas Noack <<a href="javascript:" target="_blank" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">andreasno...@...> wrote:
Great work. I think it makes perfect sense to have these factored out.

My only minor comment is that I'd preferred Functions to Funs.

On Sun, Jul 26, 2015 at 12:39 PM, Dahua Lin <<a href="javascript:" target="_blank" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">lind...@...> wrote:
Note that these changes are completely backward compatible. It is internal restructuring, and all client codes should work as before.


On Sunday, July 26, 2015 at 11:47:59 PM UTC+8, Dahua Lin wrote:
In the past several days, I did some refactoring of StatsBase & Distributions packages. The primary purpose is to decouple higher-level stuff and low-level computational routines. 

Specifically, I created a new package StatsFuns:

<a href="https://github.com/JuliaStats/StatsFuns.jl" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2FJuliaStats%2FStatsFuns.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNH1_woCYNhlWFtIYgFrbFsW-h_urg&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2FJuliaStats%2FStatsFuns.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNH1_woCYNhlWFtIYgFrbFsW-h_urg&#39;;return true;">https://github.com/JuliaStats/StatsFuns.jl

This package provides a set of numerical routines for statistical computing -- the implementations of these functions are either migrated from StatsBase or Distributions, or delegated to Rmath. With this refactoring, Distributions is depending on StatsFuns, while the efforts of replacing Rmath functions will happen in StatsFuns. In this way, contributors of these two packages will focus on different aspects of the development.

Ultimately, the role of StatsFuns to JuliaStats would be similar to the role of Rmath to R. 

Also, if you have a package that requires certain stats functions (instead of full-fledged distributions), you can have the package depend on StatsFuns instead of Distributions.

For backward compatibility, several functions (e.g. logistic, logit, logsumexp, softmax, etc) are imported from StatsFuns to StatsBase, and re-exported therefrom. We will consider removing these functions from StatsBase in future. So if you are using these functions, please do `using StatsFuns`. 

Cheers,
Dahua




--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">julia-stats...@....
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">julia-stats...@....
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">julia-stats...@....
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.