# How to create a covariance matrix of matrix that contains NA values

11 messages
Open this post in threaded view
|

## How to create a covariance matrix of matrix that contains NA values

 Hi all,Is there a way to create a covariance matrix of matrix that contains NA values, using "cov()" function from StatsBase? -- You received this message because you are subscribed to the Google Groups "julia-stats" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout.
Open this post in threaded view
|

## Re: How to create a covariance matrix of matrix that contains NA values

 I think you'd have to remove them first. E.g. something like julia> X = DataArray(randn(10,2));julia> X[2,1] = X[3,2] = NA;julia> cov(X[!vec(any(isna(X), 2)),:])2×2 DataArrays.DataArray{Float64,2}: 1.19373   0.236507 0.236507  0.524404On Tue, Jun 7, 2016 at 6:26 PM, Jessica Koh wrote:Hi all,Is there a way to create a covariance matrix of matrix that contains NA values, using "cov()" function from StatsBase? -- You received this message because you are subscribed to the Google Groups "julia-stats" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "julia-stats" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout.
Open this post in threaded view
|

## Re: How to create a covariance matrix of matrix that contains NA values

 Hello Andreas,Sorry I deleted the post before you commented on this. Thank you so much for your comment!Yes, I have already tried that, and that works great with 2 variables. However, I am dealing with multiple variables with missing values, and the location of missing values differ across different variables. I want the covariance function to handle missing values by pairwise deletion; all available observations should be used to calculate each pairwise covariance without regard to whether variables outside that pair are missing.I can technically write up the function from scratch to do this. But this seems like a basic problem, so I was guessing there might be some library already written that handle this. Do you suggest writing the function from scratch, or are you aware of the existing functions to solve this? On Tuesday, June 7, 2016 at 7:15:55 PM UTC-5, Andreas Noack wrote:I think you'd have to remove them first. E.g. something like julia> X = DataArray(randn(10,2));julia> X[2,1] = X[3,2] = NA;julia> cov(X[!vec(any(isna(X), 2)),:])2×2 DataArrays.DataArray{Float64,2}: 1.19373   0.236507 0.236507  0.524404On Tue, Jun 7, 2016 at 6:26 PM, Jessica Koh <jessica.y...@...> wrote:Hi all,Is there a way to create a covariance matrix of matrix that contains NA values, using "cov()" function from StatsBase? -- You received this message because you are subscribed to the Google Groups "julia-stats" group. To unsubscribe from this group and stop receiving emails from it, send an email to julia-stats...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "julia-stats" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout.
Open this post in threaded view
|

## Re: How to create a covariance matrix of matrix that contains NA values

Open this post in threaded view
|

## Re: How to create a covariance matrix of matrix that contains NA values

 In reply to this post by Jessica Koh Le mardi 07 juin 2016 à 17:23 -0700, Jessica Koh a écrit : > Hello Andreas, > > Sorry I deleted the post before you commented on this. Thank you so > much for your comment! > > Yes, I have already tried that, and that works great with 2 > variables. However, I am dealing with multiple variables with missing > values, and the location of missing values differ across different > variables. I want the covariance function to handle missing values by > pairwise deletion; all available observations should be used to > calculate each pairwise covariance without regard to whether > variables outside that pair are missing. > > I can technically write up the function from scratch to do this. But > this seems like a basic problem, so I was guessing there might be > some library already written that handle this. Do you suggest writing > the function from scratch, or are you aware of the existing functions > to solve this?  You're right that it's an essential function. I think we should write one based on the Nullable framework instead of on the NA/DataArrays one (which is on its way out). That function could either live in StatsBase.jl or in NullableArrays.jl. Regards > > I think you'd have to remove them first. E.g. something like  > > > > julia> X = DataArray(randn(10,2)); > > > > julia> X[2,1] = X[3,2] = NA; > > > > julia> cov(X[!vec(any(isna(X), 2)),:]) > > 2×2 DataArrays.DataArray{Float64,2}: > >  1.19373   0.236507 > >  0.236507  0.524404 > > > > > > On Tue, Jun 7, 2016 at 6:26 PM, Jessica Koh <[hidden email] > > > wrote: > > > Hi all, > > > > > > Is there a way to create a covariance matrix of matrix that > > > contains NA values, using "cov()" function from StatsBase? > > > -- You received this message because you are subscribed to the Google Groups "julia-stats" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout.
Open this post in threaded view
|

## Re: How to create a covariance matrix of matrix that contains NA values

Open this post in threaded view
|

## Re: How to create a covariance matrix of matrix that contains NA values

Open this post in threaded view
|

## Re: How to create a covariance matrix of matrix that contains NA values

Open this post in threaded view
|