

Some time ago I asked this question
http://stackoverflow.com/questions/25486506/juliabroadcastingequivalentofnumpynewaxisAs a more interesting example, here is some real python code I use:
dist = mag_sqr (demod_out[:,np.newaxis]  const.map[np.newaxis,:])
where demod_out, const.map are each vectors, mag_sqr performs elementwise
euclidean distance, and the result is a 2D array whose 1st axis matches the
1st axis of demod_out, and the 2nd axis matches the 2nd axis of const.map.
From the answers I've seen, julia doesn't really have an equivalent
functionality. The idea here is, without allocating a new array, manipulate
the strides to cause broadcasting.
AFAICT, the best for Julia would be just forget the vectorized code, and
explicitly write out loops to perform the computation. OK, I guess, but
maybe not as readable.
Is there any news on this front?


It's pretty close. In Julia 0.5, we have all the parts that are required to make this a possibility. We have index types that specify both how many indices in the source array should be consumed (CartesianIndex{N} spans N dimensions) and types that determine what the dimensionality of the output should be (the dimensionality of the result is the sum of the dimensionalities of the indices). That means that we now know exactly what the Julia equivalent of `np.newaxis` should be: [CartesianIndex{0}()]. This is really cool, and something I never considered before.
We're not quite there yet; views don't fully support CartesianIndices (fix in progress), and indexing by default creates a copy. On Monday, September 12, 2016 at 2:29:15 PM UTC5, Neal Becker wrote: Some time ago I asked this question
<a href="http://stackoverflow.com/questions/25486506/juliabroadcastingequivalentofnumpynewaxis" target="_blank" rel="nofollow" onmousedown="this.href='http://www.google.com/url?q\x3dhttp%3A%2F%2Fstackoverflow.com%2Fquestions%2F25486506%2Fjuliabroadcastingequivalentofnumpynewaxis\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHmFRKaZh8xfJ_rhiZvvCEqqe7MA';return true;" onclick="this.href='http://www.google.com/url?q\x3dhttp%3A%2F%2Fstackoverflow.com%2Fquestions%2F25486506%2Fjuliabroadcastingequivalentofnumpynewaxis\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHmFRKaZh8xfJ_rhiZvvCEqqe7MA';return true;">http://stackoverflow.com/questions/25486506/juliabroadcastingequivalentofnumpynewaxis
As a more interesting example, here is some real python code I use:
dist = mag_sqr (demod_out[:,np.newaxis]  const.map[np.newaxis,:])
where demod_out, const.map are each vectors, mag_sqr performs elementwise
euclidean distance, and the result is a 2D array whose 1st axis matches the
1st axis of demod_out, and the 2nd axis matches the 2nd axis of const.map.
From the answers I've seen, julia doesn't really have an equivalent
functionality. The idea here is, without allocating a new array, manipulate
the strides to cause broadcasting.
AFAICT, the best for Julia would be just forget the vectorized code, and
explicitly write out loops to perform the computation. OK, I guess, but
maybe not as readable.
Is there any news on this front?


I'm not certain I understand what `np.newaxis` does, but doesn't `reshape` do
the same thing? (newaxis does look like a convenient way to specify shape,
though.)
Best,
Tim
On Monday, September 12, 2016 3:28:56 PM CDT Neal Becker wrote:
> Some time ago I asked this question
> http://stackoverflow.com/questions/25486506/juliabroadcastingequivalentof> numpynewaxis
>
> As a more interesting example, here is some real python code I use:
> dist = mag_sqr (demod_out[:,np.newaxis]  const.map[np.newaxis,:])
>
> where demod_out, const.map are each vectors, mag_sqr performs elementwise
> euclidean distance, and the result is a 2D array whose 1st axis matches the
> 1st axis of demod_out, and the 2nd axis matches the 2nd axis of const.map.
>
>
> From the answers I've seen, julia doesn't really have an equivalent
> functionality. The idea here is, without allocating a new array, manipulate
> the strides to cause broadcasting.
>
> AFAICT, the best for Julia would be just forget the vectorized code, and
> explicitly write out loops to perform the computation. OK, I guess, but
> maybe not as readable.
>
> Is there any news on this front?


I haven't studied it, but I guess that newaxis increases the dimensionality,
while specifying 0 for the stride. Can reshape do that?
Tim Holy wrote:
> I'm not certain I understand what `np.newaxis` does, but doesn't `reshape`
> do the same thing? (newaxis does look like a convenient way to specify
> shape, though.)
>
> Best,
> Tim
>
> On Monday, September 12, 2016 3:28:56 PM CDT Neal Becker wrote:
>> Some time ago I asked this question
>> http://stackoverflow.com/questions/25486506/juliabroadcastingequivalentof>> numpynewaxis
>>
>> As a more interesting example, here is some real python code I use:
>> dist = mag_sqr (demod_out[:,np.newaxis]  const.map[np.newaxis,:])
>>
>> where demod_out, const.map are each vectors, mag_sqr performs
>> elementwise euclidean distance, and the result is a 2D array whose 1st
>> axis matches the 1st axis of demod_out, and the 2nd axis matches the 2nd
>> axis of const.map.
>>
>>
>> From the answers I've seen, julia doesn't really have an equivalent
>> functionality. The idea here is, without allocating a new array,
>> manipulate the strides to cause broadcasting.
>>
>> AFAICT, the best for Julia would be just forget the vectorized code, and
>> explicitly write out loops to perform the computation. OK, I guess, but
>> maybe not as readable.
>>
>> Is there any news on this front?


julia> a = rand(3) 3element Array{Float64,1}: 0.47428 0.505429 0.198919
julia> reshape(a, (3,1)) 3×1 Array{Float64,2}: 0.47428 0.505429 0.198919
julia> reshape(a, (1,3)) 1×3 Array{Float64,2}: 0.47428 0.505429 0.198919
Is that what you want? (Note that for both of them, the result is 2dimensional.)
Tim
On Monday, September 12, 2016 6:47:04 PM CDT Neal Becker wrote:
> I haven't studied it, but I guess that newaxis increases the dimensionality,
> while specifying 0 for the stride. Can reshape do that?
>
> Tim Holy wrote:
> > I'm not certain I understand what `np.newaxis` does, but doesn't `reshape`
> > do the same thing? (newaxis does look like a convenient way to specify
> > shape, though.)
> >
> > Best,
> > Tim
> >
> > On Monday, September 12, 2016 3:28:56 PM CDT Neal Becker wrote:
> >> Some time ago I asked this question
> >> http://stackoverflow.com/questions/25486506/juliabroadcastingequivalent
> >> of numpynewaxis
> >>
> >> As a more interesting example, here is some real python code I use:
> >> dist = mag_sqr (demod_out[:,np.newaxis]  const.map[np.newaxis,:])
> >>
> >> where demod_out, const.map are each vectors, mag_sqr performs
> >> elementwise euclidean distance, and the result is a 2D array whose 1st
> >> axis matches the 1st axis of demod_out, and the 2nd axis matches the 2nd
> >> axis of const.map.
> >>
> >>
> >> From the answers I've seen, julia doesn't really have an equivalent
> >> functionality. The idea here is, without allocating a new array,
> >> manipulate the strides to cause broadcasting.
> >>
> >> AFAICT, the best for Julia would be just forget the vectorized code, and
> >> explicitly write out loops to perform the computation. OK, I guess, but
> >> maybe not as readable.
> >>
> >> Is there any news on this front?


For your particular example, it looks like what you want is (and I am just guessing what mag_sqr means): The performance should be the similar to a handwritten loop on version 0.5.
You can read about it here: http://docs.julialang.org/en/release0.5/manual/functions/#dotsyntaxforvectorizingfunctionsOn Monday, September 12, 2016 at 9:29:15 PM UTC+2, Neal Becker wrote: Some time ago I asked this question
<a href="http://stackoverflow.com/questions/25486506/juliabroadcastingequivalentofnumpynewaxis" target="_blank" rel="nofollow" onmousedown="this.href='http://www.google.com/url?q\x3dhttp%3A%2F%2Fstackoverflow.com%2Fquestions%2F25486506%2Fjuliabroadcastingequivalentofnumpynewaxis\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHmFRKaZh8xfJ_rhiZvvCEqqe7MA';return true;" onclick="this.href='http://www.google.com/url?q\x3dhttp%3A%2F%2Fstackoverflow.com%2Fquestions%2F25486506%2Fjuliabroadcastingequivalentofnumpynewaxis\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHmFRKaZh8xfJ_rhiZvvCEqqe7MA';return true;">http://stackoverflow.com/questions/25486506/juliabroadcastingequivalentofnumpynewaxis
As a more interesting example, here is some real python code I use:
dist = mag_sqr (demod_out[:,np.newaxis]  const.map[np.newaxis,:])
where demod_out, const.map are each vectors, mag_sqr performs elementwise
euclidean distance, and the result is a 2D array whose 1st axis matches the
1st axis of demod_out, and the 2nd axis matches the 2nd axis of const.map.
From the answers I've seen, julia doesn't really have an equivalent
functionality. The idea here is, without allocating a new array, manipulate
the strides to cause broadcasting.
AFAICT, the best for Julia would be just forget the vectorized code, and
explicitly write out loops to perform the computation. OK, I guess, but
maybe not as readable.
Is there any news on this front?


So you're saying that abs2.(x . y.') will not allocate a 2d array and then
pass to abs2? That's great! But how would I know that?
DNF wrote:
> For your particular example, it looks like what you want is (and I am just
> guessing what mag_sqr means):
> dist = abs2.(x . y.')
> The performance should be the similar to a handwritten loop on version
> 0.5.
>
> You can read about it here:
> http://docs.julialang.org/en/release0.5/manual/functions/#dotsyntaxforvectorizingfunctions>
>
> On Monday, September 12, 2016 at 9:29:15 PM UTC+2, Neal Becker wrote:
>>
>> Some time ago I asked this question
>>
>> http://stackoverflow.com/questions/25486506/juliabroadcastingequivalentofnumpynewaxis>>
>> As a more interesting example, here is some real python code I use:
>> dist = mag_sqr (demod_out[:,np.newaxis]  const.map[np.newaxis,:])
>>
>> where demod_out, const.map are each vectors, mag_sqr performs
>> elementwise euclidean distance, and the result is a 2D array whose 1st
>> axis matches the
>> 1st axis of demod_out, and the 2nd axis matches the 2nd axis of
>> const.map.
>>
>>
>> From the answers I've seen, julia doesn't really have an equivalent
>> functionality. The idea here is, without allocating a new array,
>> manipulate
>> the strides to cause broadcasting.
>>
>> AFAICT, the best for Julia would be just forget the vectorized code, and
>> explicitly write out loops to perform the computation. OK, I guess, but
>> maybe not as readable.
>>
>> Is there any news on this front?
>>
>>


On Tue, 20160913 at 14:26, Neal Becker < [hidden email]> wrote:
> So you're saying that abs2.(x . y.') will not allocate a 2d array and then
> pass to abs2? That's great! But how would I know that?
The operators do not do the fusing yet, check right at the bottom of the
linked manual section. I think you can work around it by using their
functional form:
x .+ y # not fused
(+).(x,y) # fused
So:
out .= abs2.(().(x, y.'))
> DNF wrote:
>
>> For your particular example, it looks like what you want is (and I am just
>> guessing what mag_sqr means):
>> dist = abs2.(x . y.')
>> The performance should be the similar to a handwritten loop on version
>> 0.5.
>>
>> You can read about it here:
>> http://docs.julialang.org/en/release0.5/manual/functions/#dotsyntaxforvectorizingfunctions>>
>>
>> On Monday, September 12, 2016 at 9:29:15 PM UTC+2, Neal Becker wrote:
>>>
>>> Some time ago I asked this question
>>>
>>> http://stackoverflow.com/questions/25486506/juliabroadcastingequivalentofnumpynewaxis>>>
>>> As a more interesting example, here is some real python code I use:
>>> dist = mag_sqr (demod_out[:,np.newaxis]  const.map[np.newaxis,:])
>>>
>>> where demod_out, const.map are each vectors, mag_sqr performs
>>> elementwise euclidean distance, and the result is a 2D array whose 1st
>>> axis matches the
>>> 1st axis of demod_out, and the 2nd axis matches the 2nd axis of
>>> const.map.
>>>
>>>
>>> From the answers I've seen, julia doesn't really have an equivalent
>>> functionality. The idea here is, without allocating a new array,
>>> manipulate
>>> the strides to cause broadcasting.
>>>
>>> AFAICT, the best for Julia would be just forget the vectorized code, and
>>> explicitly write out loops to perform the computation. OK, I guess, but
>>> maybe not as readable.
>>>
>>> Is there any news on this front?
>>>
>>>


Oh, yeah. I forgot that .+, ., etc are not along for the ride yet. I understand it is pretty much ready for inclusion in 0.6.
There is also a one extra allocation, namely y.', which would not be necessary in a loop. But this is hardly worse than numpy, right?
On Tuesday, September 13, 2016 at 2:42:35 PM UTC+2, Mauro wrote: On Tue, 20160913 at 14:26, Neal Becker <<a href="javascript:" target="_blank" gdfobfuscatedmailto="dyESCWf5AQAJ" rel="nofollow" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">ndbe...@...> wrote:
> So you're saying that abs2.(x . y.') will not allocate a 2d array and then
> pass to abs2? That's great! But how would I know that?
The operators do not do the fusing yet, check right at the bottom of the
linked manual section. I think you can work around it by using their
functional form:
x .+ y # not fused
(+).(x,y) # fused
So:
out .= abs2.(().(x, y.'))
> DNF wrote:
>
>> For your particular example, it looks like what you want is (and I am just
>> guessing what mag_sqr means):
>> dist = abs2.(x . y.')
>> The performance should be the similar to a handwritten loop on version
>> 0.5.
>>
>> You can read about it here:
>> <a href="http://docs.julialang.org/en/release0.5/manual/functions/#dotsyntaxforvectorizingfunctions" target="_blank" rel="nofollow" onmousedown="this.href='http://www.google.com/url?q\x3dhttp%3A%2F%2Fdocs.julialang.org%2Fen%2Frelease0.5%2Fmanual%2Ffunctions%2F%23dotsyntaxforvectorizingfunctions\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHJPrAuqnWcEtB_tcn2b4A3wUspgg';return true;" onclick="this.href='http://www.google.com/url?q\x3dhttp%3A%2F%2Fdocs.julialang.org%2Fen%2Frelease0.5%2Fmanual%2Ffunctions%2F%23dotsyntaxforvectorizingfunctions\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHJPrAuqnWcEtB_tcn2b4A3wUspgg';return true;">http://docs.julialang.org/en/release0.5/manual/functions/#dotsyntaxforvectorizingfunctions
>>
>>
>> On Monday, September 12, 2016 at 9:29:15 PM UTC+2, Neal Becker wrote:
>>>
>>> Some time ago I asked this question
>>>
>>> <a href="http://stackoverflow.com/questions/25486506/juliabroadcastingequivalentofnumpynewaxis" target="_blank" rel="nofollow" onmousedown="this.href='http://www.google.com/url?q\x3dhttp%3A%2F%2Fstackoverflow.com%2Fquestions%2F25486506%2Fjuliabroadcastingequivalentofnumpynewaxis\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHmFRKaZh8xfJ_rhiZvvCEqqe7MA';return true;" onclick="this.href='http://www.google.com/url?q\x3dhttp%3A%2F%2Fstackoverflow.com%2Fquestions%2F25486506%2Fjuliabroadcastingequivalentofnumpynewaxis\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHmFRKaZh8xfJ_rhiZvvCEqqe7MA';return true;">http://stackoverflow.com/questions/25486506/juliabroadcastingequivalentofnumpynewaxis
>>>
>>> As a more interesting example, here is some real python code I use:
>>> dist = mag_sqr (demod_out[:,np.newaxis]  const.map[np.newaxis,:])
>>>
>>> where demod_out, const.map are each vectors, mag_sqr performs
>>> elementwise euclidean distance, and the result is a 2D array whose 1st
>>> axis matches the
>>> 1st axis of demod_out, and the 2nd axis matches the 2nd axis of
>>> const.map.
>>>
>>>
>>> From the answers I've seen, julia doesn't really have an equivalent
>>> functionality. The idea here is, without allocating a new array,
>>> manipulate
>>> the strides to cause broadcasting.
>>>
>>> AFAICT, the best for Julia would be just forget the vectorized code, and
>>> explicitly write out loops to perform the computation. OK, I guess, but
>>> maybe not as readable.
>>>
>>> Is there any news on this front?
>>>
>>>

