error in assigning NA to Float64 in a DataFrame

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

error in assigning NA to Float64 in a DataFrame

Min-Woong Sohn
I am trying to work out the following example:

using DataFrames, GLM
N = 1000
data = DataFrame(X=randn(N),Y=randn(N))
beta_0 = 0
beta_1 = 1
index = (beta_0 + beta_1*data[:X])
probability = cdf(Normal(0,1),index)
D = zeros(N)
for i=1:N
    D[i]=rand(Bernoulli(probability[i]))
end
data[:D]=D
data2 = data
data2[data2[:D] .== 0.0, :Y] = NA
data2

data3 = DataFrame(rand(MvNormal([0,0.],[1 .5;.5 1]),N)')
names!(data3,[:X,:Y])
mean(convert(Array,data3),1)
cov(convert(Array,data3))
index = (beta_0 + beta_1*data3[:X])
probability = cdf(Normal(0,1),index)
D = zeros(N)
for i=1:N
    D[i]=rand(Bernoulli(probability[i]))
end
data3[:D]=D
data4 = data3
data4[data4[:D] .== 0.0, :Y] = NA

Lines 14 and 29 are exactly the same but the first one works fine and the second one gives an error: LoadError: MethodError: `convert` has no method matching convert(::Type{Float64}, ::DataArrays.NAtype)

Does anyone know what's happening here?


--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: error in assigning NA to Float64 in a DataFrame

Alex Williams
The problem is that data4[:Y] is an Array, not a DataArray... so you can't add NA values to it. I think it works if you add the following command right before the last line:

data4[:Y] = convert(DataArray,data4[:Y])



On Wed, Dec 30, 2015 at 1:39 PM, Min-Woong Sohn <[hidden email]> wrote:
I am trying to work out the following example:

using DataFrames, GLM
N = 1000
data = DataFrame(X=randn(N),Y=randn(N))
beta_0 = 0
beta_1 = 1
index = (beta_0 + beta_1*data[:X])
probability = cdf(Normal(0,1),index)
D = zeros(N)
for i=1:N
    D[i]=rand(Bernoulli(probability[i]))
end
data[:D]=D
data2 = data
data2[data2[:D] .== 0.0, :Y] = NA
data2

data3 = DataFrame(rand(MvNormal([0,0.],[1 .5;.5 1]),N)')
names!(data3,[:X,:Y])
mean(convert(Array,data3),1)
cov(convert(Array,data3))
index = (beta_0 + beta_1*data3[:X])
probability = cdf(Normal(0,1),index)
D = zeros(N)
for i=1:N
    D[i]=rand(Bernoulli(probability[i]))
end
data3[:D]=D
data4 = data3
data4[data4[:D] .== 0.0, :Y] = NA

Lines 14 and 29 are exactly the same but the first one works fine and the second one gives an error: LoadError: MethodError: `convert` has no method matching convert(::Type{Float64}, ::DataArrays.NAtype)

Does anyone know what's happening here?


--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: error in assigning NA to Float64 in a DataFrame

Min-Woong Sohn
Thanks a bunch. It works now.

On Wednesday, December 30, 2015 at 2:42:54 PM UTC-5, Alex Williams wrote:
The problem is that data4[:Y] is an Array, not a DataArray... so you can't add NA values to it. I think it works if you add the following command right before the last line:

data4[:Y] = convert(DataArray,data4[:Y])



On Wed, Dec 30, 2015 at 1:39 PM, Min-Woong Sohn <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="Un80DJiaCAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">mws...@...> wrote:
I am trying to work out the following example:

using DataFrames, GLM
N = 1000
data = DataFrame(X=randn(N),Y=randn(N))
beta_0 = 0
beta_1 = 1
index = (beta_0 + beta_1*data[:X])
probability = cdf(Normal(0,1),index)
D = zeros(N)
for i=1:N
    D[i]=rand(Bernoulli(probability[i]))
end
data[:D]=D
data2 = data
data2[data2[:D] .== 0.0, :Y] = NA
data2

data3 = DataFrame(rand(MvNormal([0,0.],[1 .5;.5 1]),N)')
names!(data3,[:X,:Y])
mean(convert(Array,data3),1)
cov(convert(Array,data3))
index = (beta_0 + beta_1*data3[:X])
probability = cdf(Normal(0,1),index)
D = zeros(N)
for i=1:N
    D[i]=rand(Bernoulli(probability[i]))
end
data3[:D]=D
data4 = data3
data4[data4[:D] .== 0.0, :Y] = NA

Lines 14 and 29 are exactly the same but the first one works fine and the second one gives an error: LoadError: MethodError: `convert` has no method matching convert(::Type{Float64}, ::DataArrays.NAtype)

Does anyone know what's happening here?


--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="Un80DJiaCAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">julia-stats...@googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.