# equivalent of numpy newaxis?

10 messages
Open this post in threaded view
|

## equivalent of numpy newaxis?

 Some time ago I asked this question http://stackoverflow.com/questions/25486506/julia-broadcasting-equivalent-of-numpy-newaxisAs a more interesting example, here is some real python code I use: dist = mag_sqr (demod_out[:,np.newaxis] - const.map[np.newaxis,:]) where demod_out, const.map are each vectors, mag_sqr performs element-wise euclidean distance, and the result is a 2D array whose 1st axis matches the 1st axis of demod_out, and the 2nd axis matches the 2nd axis of const.map. From the answers I've seen, julia doesn't really have an equivalent functionality.  The idea here is, without allocating a new array, manipulate the strides to cause broadcasting. AFAICT, the best for Julia would be just forget the vectorized code, and explicitly write out loops to perform the computation.  OK, I guess, but maybe not as readable. Is there any news on this front?
Open this post in threaded view
|

## Re: equivalent of numpy newaxis?

 It's pretty close. In Julia 0.5, we have all the parts that are required to make this a possibility.  We have index types that specify both how many indices in the source array should be consumed (CartesianIndex{N} spans N dimensions) and types that determine what the dimensionality of the output should be (the dimensionality of the result is the sum of the dimensionalities of the indices).That means that we now know exactly what the Julia equivalent of `np.newaxis` should be: [CartesianIndex{0}()].  This is really cool, and something I never considered before.We're not quite there yet; views don't fully support CartesianIndices (fix in progress), and indexing by default creates a copy.On Monday, September 12, 2016 at 2:29:15 PM UTC-5, Neal Becker wrote:Some time ago I asked this question http://stackoverflow.com/questions/25486506/julia-broadcasting-equivalent-of-numpy-newaxis As a more interesting example, here is some real python code I use: dist = mag_sqr (demod_out[:,np.newaxis] - const.map[np.newaxis,:]) where demod_out, const.map are each vectors, mag_sqr performs element-wise euclidean distance, and the result is a 2D array whose 1st axis matches the 1st axis of demod_out, and the 2nd axis matches the 2nd axis of const.map. From the answers I've seen, julia doesn't really have an equivalent functionality.  The idea here is, without allocating a new array, manipulate the strides to cause broadcasting. AFAICT, the best for Julia would be just forget the vectorized code, and explicitly write out loops to perform the computation.  OK, I guess, but maybe not as readable. Is there any news on this front?
Open this post in threaded view
|

## Re: equivalent of numpy newaxis?

 In reply to this post by Neal Becker I'm not certain I understand what `np.newaxis` does, but doesn't `reshape` do the same thing? (newaxis does look like a convenient way to specify shape, though.) Best, --Tim On Monday, September 12, 2016 3:28:56 PM CDT Neal Becker wrote: > Some time ago I asked this question > http://stackoverflow.com/questions/25486506/julia-broadcasting-equivalent-of> -numpy-newaxis > > As a more interesting example, here is some real python code I use: > dist = mag_sqr (demod_out[:,np.newaxis] - const.map[np.newaxis,:]) > > where demod_out, const.map are each vectors, mag_sqr performs element-wise > euclidean distance, and the result is a 2D array whose 1st axis matches the > 1st axis of demod_out, and the 2nd axis matches the 2nd axis of const.map. > > > From the answers I've seen, julia doesn't really have an equivalent > functionality.  The idea here is, without allocating a new array, manipulate > the strides to cause broadcasting. > > AFAICT, the best for Julia would be just forget the vectorized code, and > explicitly write out loops to perform the computation.  OK, I guess, but > maybe not as readable. > > Is there any news on this front?
Open this post in threaded view
|

## Re: equivalent of numpy newaxis?

 I haven't studied it, but I guess that newaxis increases the dimensionality, while specifying 0 for the stride.  Can reshape do that? Tim Holy wrote: > I'm not certain I understand what `np.newaxis` does, but doesn't `reshape` > do the same thing? (newaxis does look like a convenient way to specify > shape, though.) > > Best, > --Tim > > On Monday, September 12, 2016 3:28:56 PM CDT Neal Becker wrote: >> Some time ago I asked this question >> http://stackoverflow.com/questions/25486506/julia-broadcasting-equivalent-of>> -numpy-newaxis >> >> As a more interesting example, here is some real python code I use: >> dist = mag_sqr (demod_out[:,np.newaxis] - const.map[np.newaxis,:]) >> >> where demod_out, const.map are each vectors, mag_sqr performs >> element-wise euclidean distance, and the result is a 2D array whose 1st >> axis matches the 1st axis of demod_out, and the 2nd axis matches the 2nd >> axis of const.map. >> >> >> From the answers I've seen, julia doesn't really have an equivalent >> functionality.  The idea here is, without allocating a new array, >> manipulate the strides to cause broadcasting. >> >> AFAICT, the best for Julia would be just forget the vectorized code, and >> explicitly write out loops to perform the computation.  OK, I guess, but >> maybe not as readable. >> >> Is there any news on this front?
Open this post in threaded view
|

## Re: equivalent of numpy newaxis?

 julia> a = rand(3) 3-element Array{Float64,1}: 0.47428  0.505429 0.198919 julia> reshape(a, (3,1)) 3×1 Array{Float64,2}: 0.47428  0.505429 0.198919 julia> reshape(a, (1,3)) 1×3 Array{Float64,2}: 0.47428  0.505429  0.198919 Is that what you want? (Note that for both of them, the result is 2-dimensional.)   --Tim   On Monday, September 12, 2016 6:47:04 PM CDT Neal Becker wrote: > I haven't studied it, but I guess that newaxis increases the dimensionality, > while specifying 0 for the stride. Can reshape do that? > > Tim Holy wrote: > > I'm not certain I understand what `np.newaxis` does, but doesn't `reshape` > > do the same thing? (newaxis does look like a convenient way to specify > > shape, though.) > > > > Best, > > --Tim > > > > On Monday, September 12, 2016 3:28:56 PM CDT Neal Becker wrote: > >> Some time ago I asked this question > >> http://stackoverflow.com/questions/25486506/julia-broadcasting-equivalent > >> -of -numpy-newaxis > >> > >> As a more interesting example, here is some real python code I use: > >> dist = mag_sqr (demod_out[:,np.newaxis] - const.map[np.newaxis,:]) > >> > >> where demod_out, const.map are each vectors, mag_sqr performs > >> element-wise euclidean distance, and the result is a 2D array whose 1st > >> axis matches the 1st axis of demod_out, and the 2nd axis matches the 2nd > >> axis of const.map. > >> > >> > >> From the answers I've seen, julia doesn't really have an equivalent > >> functionality. The idea here is, without allocating a new array, > >> manipulate the strides to cause broadcasting. > >> > >> AFAICT, the best for Julia would be just forget the vectorized code, and > >> explicitly write out loops to perform the computation. OK, I guess, but > >> maybe not as readable. > >> > >> Is there any news on this front?
Open this post in threaded view
|

## Re: equivalent of numpy newaxis?

 I use a simple function for this:function newdim(A::AbstractArray, d::Integer)    @assert 0 < d <= ndims(A)+1    dim = size(A)    reshape(A, dim[1:d-1]..., 1, dim[d:end]...)end  But having syntax for a newaxis would be great. See also:https://github.com/JuliaLang/julia/issues/5405https://github.com/JuliaLang/julia/issues/4774 (search for newaxis)BobOn Mon, Sep 12, 2016 at 5:00 PM, Tim Holy wrote: julia> a = rand(3) 3-element Array{Float64,1}: 0.47428  0.505429 0.198919 julia> reshape(a, (3,1)) 3×1 Array{Float64,2}: 0.47428  0.505429 0.198919 julia> reshape(a, (1,3)) 1×3 Array{Float64,2}: 0.47428  0.505429  0.198919 Is that what you want? (Note that for both of them, the result is 2-dimensional.)   --Tim   On Monday, September 12, 2016 6:47:04 PM CDT Neal Becker wrote: > I haven't studied it, but I guess that newaxis increases the dimensionality, > while specifying 0 for the stride. Can reshape do that? > > Tim Holy wrote: > > I'm not certain I understand what `np.newaxis` does, but doesn't `reshape` > > do the same thing? (newaxis does look like a convenient way to specify > > shape, though.) > > > > Best, > > --Tim > > > > On Monday, September 12, 2016 3:28:56 PM CDT Neal Becker wrote: > >> Some time ago I asked this question > >> -of -numpy-newaxis > >> > >> As a more interesting example, here is some real python code I use: > >> dist = mag_sqr (demod_out[:,np.newaxis] - const.map[np.newaxis,:]) > >> > >> where demod_out, const.map are each vectors, mag_sqr performs > >> element-wise euclidean distance, and the result is a 2D array whose 1st > >> axis matches the 1st axis of demod_out, and the 2nd axis matches the 2nd > >> axis of const.map. > >> > >> > >> From the answers I've seen, julia doesn't really have an equivalent > >> functionality. The idea here is, without allocating a new array, > >> manipulate the strides to cause broadcasting. > >> > >> AFAICT, the best for Julia would be just forget the vectorized code, and > >> explicitly write out loops to perform the computation. OK, I guess, but > >> maybe not as readable. > >> > >> Is there any news on this front?
Open this post in threaded view
|

## Re: equivalent of numpy newaxis?

 In reply to this post by Neal Becker For your particular example, it looks like what you want is (and I am just guessing what mag_sqr means):`dist = abs2.(x .- y.')`The performance should be the similar to a hand-written loop on version 0.5.You can read about it here: http://docs.julialang.org/en/release-0.5/manual/functions/#dot-syntax-for-vectorizing-functionsOn Monday, September 12, 2016 at 9:29:15 PM UTC+2, Neal Becker wrote:Some time ago I asked this question http://stackoverflow.com/questions/25486506/julia-broadcasting-equivalent-of-numpy-newaxis As a more interesting example, here is some real python code I use: dist = mag_sqr (demod_out[:,np.newaxis] - const.map[np.newaxis,:]) where demod_out, const.map are each vectors, mag_sqr performs element-wise euclidean distance, and the result is a 2D array whose 1st axis matches the 1st axis of demod_out, and the 2nd axis matches the 2nd axis of const.map. From the answers I've seen, julia doesn't really have an equivalent functionality.  The idea here is, without allocating a new array, manipulate the strides to cause broadcasting. AFAICT, the best for Julia would be just forget the vectorized code, and explicitly write out loops to perform the computation.  OK, I guess, but maybe not as readable. Is there any news on this front?
Open this post in threaded view
|

## Re: equivalent of numpy newaxis?

 So you're saying that abs2.(x .- y.') will not allocate a 2d array and then pass to abs2?  That's great!  But how would I know that? DNF wrote: > For your particular example, it looks like what you want is (and I am just > guessing what mag_sqr means): > dist = abs2.(x .- y.') > The performance should be the similar to a hand-written loop on version > 0.5. > > You can read about it here: > http://docs.julialang.org/en/release-0.5/manual/functions/#dot-syntax-for-vectorizing-functions> > > On Monday, September 12, 2016 at 9:29:15 PM UTC+2, Neal Becker wrote: >> >> Some time ago I asked this question >> >> http://stackoverflow.com/questions/25486506/julia-broadcasting-equivalent-of-numpy-newaxis>> >> As a more interesting example, here is some real python code I use: >> dist = mag_sqr (demod_out[:,np.newaxis] - const.map[np.newaxis,:]) >> >> where demod_out, const.map are each vectors, mag_sqr performs >> element-wise euclidean distance, and the result is a 2D array whose 1st >> axis matches the >> 1st axis of demod_out, and the 2nd axis matches the 2nd axis of >> const.map. >> >> >> From the answers I've seen, julia doesn't really have an equivalent >> functionality.  The idea here is, without allocating a new array, >> manipulate >> the strides to cause broadcasting. >> >> AFAICT, the best for Julia would be just forget the vectorized code, and >> explicitly write out loops to perform the computation.  OK, I guess, but >> maybe not as readable. >> >> Is there any news on this front? >> >>