Convert NullableArray w/o nulls to an Array

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Convert NullableArray w/o nulls to an Array

Douglas Bates
I remember someone (Milan?, Alex?) writing a short cal to convert a NullableArray to an Array but I can't seem to find it now.  Can someone refresh my memory, please?

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Convert NullableArray w/o nulls to an Array

Milan Bouchet-Valat
Le mercredi 14 septembre 2016 à 13:56 -0700, Douglas Bates a écrit :
> I remember someone (Milan?, Alex?) writing a short cal to convert a
> NullableArray to an Array but I can't seem to find it now.  Can
> someone refresh my memory, please?
What do you mean exactly? This works AFAICT:
julia> convert(Array, NullableArray(1:3))
3-element Array{Int64,1}:
 1
 2
 3


Regards

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Convert NullableArray w/o nulls to an Array

Douglas Bates
Thanks.  I didn't try that because I thought that a NullableArray was an Array so that conversion would be a no-op.  I should have tested it.

On Wednesday, September 14, 2016 at 4:02:04 PM UTC-5, Milan Bouchet-Valat wrote:
Le mercredi 14 septembre 2016 à 13:56 -0700, Douglas Bates a écrit :
> I remember someone (Milan?, Alex?) writing a short cal to convert a
> NullableArray to an Array but I can't seem to find it now.  Can
> someone refresh my memory, please?
What do you mean exactly? This works AFAICT:
julia> convert(Array, NullableArray(1:3))
3-element Array{Int64,1}:
 1
 2
 3


Regards

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Convert NullableArray w/o nulls to an Array

Douglas Bates
Of course, this brings me back to another problem, which is replacing a column of a dataframe with an Array without having the Array converted to a DataArray or a NullableArray.  I have a large CSV file in which over 99% of the columns do not contain nulls.  Because some of the columns contain nulls I can't use CSV.read(..., nullable = false) but I do want to convert those columns that do not contain nulls to Arrays.

If I try to replace the column as

mydf[1] = Array(mydf[1])

I get a DataArray in the release version of DataFrames.  (I'm not sure what happens in your nl/nullable branch).

The way I avoid this is right now is by working with mydf.columns directly but I always caution others not to work directly with fields of composite types and I should probably follow my own advice.

On Wednesday, September 14, 2016 at 4:07:21 PM UTC-5, Douglas Bates wrote:
Thanks.  I didn't try that because I thought that a NullableArray was an Array so that conversion would be a no-op.  I should have tested it.

On Wednesday, September 14, 2016 at 4:02:04 PM UTC-5, Milan Bouchet-Valat wrote:
Le mercredi 14 septembre 2016 à 13:56 -0700, Douglas Bates a écrit :
> I remember someone (Milan?, Alex?) writing a short cal to convert a
> NullableArray to an Array but I can't seem to find it now.  Can
> someone refresh my memory, please?
What do you mean exactly? This works AFAICT:
julia> convert(Array, NullableArray(1:3))
3-element Array{Int64,1}:
 1
 2
 3


Regards

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Convert NullableArray w/o nulls to an Array

Tom Short

For avoiding unwanted conversions, DataFramesMeta has a PassThrough type with an alias P. You should be able to do:

mydf[1] = P(Array(mydf[1]))


On Sep 14, 2016 5:17 PM, "Douglas Bates" <[hidden email]> wrote:
Of course, this brings me back to another problem, which is replacing a column of a dataframe with an Array without having the Array converted to a DataArray or a NullableArray.  I have a large CSV file in which over 99% of the columns do not contain nulls.  Because some of the columns contain nulls I can't use CSV.read(..., nullable = false) but I do want to convert those columns that do not contain nulls to Arrays.

If I try to replace the column as

mydf[1] = Array(mydf[1])

I get a DataArray in the release version of DataFrames.  (I'm not sure what happens in your nl/nullable branch).

The way I avoid this is right now is by working with mydf.columns directly but I always caution others not to work directly with fields of composite types and I should probably follow my own advice.

On Wednesday, September 14, 2016 at 4:07:21 PM UTC-5, Douglas Bates wrote:
Thanks.  I didn't try that because I thought that a NullableArray was an Array so that conversion would be a no-op.  I should have tested it.

On Wednesday, September 14, 2016 at 4:02:04 PM UTC-5, Milan Bouchet-Valat wrote:
Le mercredi 14 septembre 2016 à 13:56 -0700, Douglas Bates a écrit :
> I remember someone (Milan?, Alex?) writing a short cal to convert a
> NullableArray to an Array but I can't seem to find it now.  Can
> someone refresh my memory, please?
What do you mean exactly? This works AFAICT:
julia> convert(Array, NullableArray(1:3))
3-element Array{Int64,1}:
 1
 2
 3


Regards

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Convert NullableArray w/o nulls to an Array

Douglas Bates
On Wednesday, September 14, 2016 at 4:42:39 PM UTC-5, tshort wrote:

For avoiding unwanted conversions, DataFramesMeta has a PassThrough type with an alias P. You should be able to do:

mydf[1] = P(Array(mydf[1]))


True but the approach has a bit of a "burning the toast and then scraping it" feel to it.  (That was a favorite expression of Deming's re quality control.)  I think it would be better for setindex! in a DataFrame to skip the conversion to a DataArray or a NullableArray. 


On Sep 14, 2016 5:17 PM, "Douglas Bates" <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="2yjnr12tBgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">dmb...@...> wrote:
Of course, this brings me back to another problem, which is replacing a column of a dataframe with an Array without having the Array converted to a DataArray or a NullableArray.  I have a large CSV file in which over 99% of the columns do not contain nulls.  Because some of the columns contain nulls I can't use CSV.read(..., nullable = false) but I do want to convert those columns that do not contain nulls to Arrays.

If I try to replace the column as

mydf[1] = Array(mydf[1])

I get a DataArray in the release version of DataFrames.  (I'm not sure what happens in your nl/nullable branch).

The way I avoid this is right now is by working with mydf.columns directly but I always caution others not to work directly with fields of composite types and I should probably follow my own advice.

On Wednesday, September 14, 2016 at 4:07:21 PM UTC-5, Douglas Bates wrote:
Thanks.  I didn't try that because I thought that a NullableArray was an Array so that conversion would be a no-op.  I should have tested it.

On Wednesday, September 14, 2016 at 4:02:04 PM UTC-5, Milan Bouchet-Valat wrote:
Le mercredi 14 septembre 2016 à 13:56 -0700, Douglas Bates a écrit :
> I remember someone (Milan?, Alex?) writing a short cal to convert a
> NullableArray to an Array but I can't seem to find it now.  Can
> someone refresh my memory, please?
What do you mean exactly? This works AFAICT:
julia> convert(Array, NullableArray(1:3))
3-element Array{Int64,1}:
 1
 2
 3


Regards

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="2yjnr12tBgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">julia-stats...@googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Convert NullableArray w/o nulls to an Array

Tom Short

I agree.


On Sep 14, 2016 5:54 PM, "Douglas Bates" <[hidden email]> wrote:
On Wednesday, September 14, 2016 at 4:42:39 PM UTC-5, tshort wrote:

For avoiding unwanted conversions, DataFramesMeta has a PassThrough type with an alias P. You should be able to do:

mydf[1] = P(Array(mydf[1]))


True but the approach has a bit of a "burning the toast and then scraping it" feel to it.  (That was a favorite expression of Deming's re quality control.)  I think it would be better for setindex! in a DataFrame to skip the conversion to a DataArray or a NullableArray. 


On Sep 14, 2016 5:17 PM, "Douglas Bates" <[hidden email]> wrote:
Of course, this brings me back to another problem, which is replacing a column of a dataframe with an Array without having the Array converted to a DataArray or a NullableArray.  I have a large CSV file in which over 99% of the columns do not contain nulls.  Because some of the columns contain nulls I can't use CSV.read(..., nullable = false) but I do want to convert those columns that do not contain nulls to Arrays.

If I try to replace the column as

mydf[1] = Array(mydf[1])

I get a DataArray in the release version of DataFrames.  (I'm not sure what happens in your nl/nullable branch).

The way I avoid this is right now is by working with mydf.columns directly but I always caution others not to work directly with fields of composite types and I should probably follow my own advice.

On Wednesday, September 14, 2016 at 4:07:21 PM UTC-5, Douglas Bates wrote:
Thanks.  I didn't try that because I thought that a NullableArray was an Array so that conversion would be a no-op.  I should have tested it.

On Wednesday, September 14, 2016 at 4:02:04 PM UTC-5, Milan Bouchet-Valat wrote:
Le mercredi 14 septembre 2016 à 13:56 -0700, Douglas Bates a écrit :
> I remember someone (Milan?, Alex?) writing a short cal to convert a
> NullableArray to an Array but I can't seem to find it now.  Can
> someone refresh my memory, please?
What do you mean exactly? This works AFAICT:
julia> convert(Array, NullableArray(1:3))
3-element Array{Int64,1}:
 1
 2
 3


Regards

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to julia-stats...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Loading...