What's the status of SIMD instructions from a user's perspective in v0.5?

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

What's the status of SIMD instructions from a user's perspective in v0.5?

Florian Oswald
i see on the docs http://docs.julialang.org/en/release-0.5/stdlib/simd-types/?highlight=SIMD that there is a vecElement that is build for SIMD support. I don't understand if as a user I should construct vecElement arrays and hope for some SIMD optimization? thanks.

Reply | Threaded
Open this post in threaded view
|

Re: What's the status of SIMD instructions from a user's perspective in v0.5?

Valentin Churavy
If you want explicit simd the best way right now is the great SIMD.jl package https://github.com/eschnett/SIMD.jl  it is builds on top of VecElement.

In many cases we can perform automatic vectorisation, but you have to start Julia with -O3

On Thursday, 13 October 2016 22:15:00 UTC+9, Florian Oswald wrote:
i see on the docs <a href="http://docs.julialang.org/en/release-0.5/stdlib/simd-types/?highlight=SIMD" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fdocs.julialang.org%2Fen%2Frelease-0.5%2Fstdlib%2Fsimd-types%2F%3Fhighlight%3DSIMD\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFxPOLyQThP6XN9ku_FTrsLiVWpCg&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fdocs.julialang.org%2Fen%2Frelease-0.5%2Fstdlib%2Fsimd-types%2F%3Fhighlight%3DSIMD\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFxPOLyQThP6XN9ku_FTrsLiVWpCg&#39;;return true;">http://docs.julialang.org/en/release-0.5/stdlib/simd-types/?highlight=SIMD that there is a vecElement that is build for SIMD support. I don't understand if as a user I should construct vecElement arrays and hope for some SIMD optimization? thanks.

Reply | Threaded
Open this post in threaded view
|

Re: What's the status of SIMD instructions from a user's perspective in v0.5?

Florian Oswald
ok thanks! and so I should define my SIMD-able function like

function vadd!{N,T}(xs::Vector{T}, ys::Vector{T}, ::Type{Vec{N,T}})
    @assert length(ys) == length(xs)
    @assert length(xs) % N == 0
    @inbounds for i in 1:N:length(xs)
        xv = vload(Vec{N,T}, xs, i)
        yv = vload(Vec{N,T}, ys, i)
        xv += yv
        vstore(xv, xs, i)
    end
end
i.e. using vload() and vstore() methods?

On Thursday, 13 October 2016 15:29:50 UTC+2, Valentin Churavy wrote:
If you want explicit simd the best way right now is the great SIMD.jl package <a href="https://github.com/eschnett/SIMD.jl" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Feschnett%2FSIMD.jl\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEpXfPW96dGD44Z8GQlkr3IqUMrHA&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Feschnett%2FSIMD.jl\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEpXfPW96dGD44Z8GQlkr3IqUMrHA&#39;;return true;">https://github.com/eschnett/SIMD.jl  it is builds on top of VecElement.

In many cases we can perform automatic vectorisation, but you have to start Julia with -O3

On Thursday, 13 October 2016 22:15:00 UTC+9, Florian Oswald wrote:
i see on the docs <a href="http://docs.julialang.org/en/release-0.5/stdlib/simd-types/?highlight=SIMD" rel="nofollow" target="_blank" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fdocs.julialang.org%2Fen%2Frelease-0.5%2Fstdlib%2Fsimd-types%2F%3Fhighlight%3DSIMD\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFxPOLyQThP6XN9ku_FTrsLiVWpCg&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fdocs.julialang.org%2Fen%2Frelease-0.5%2Fstdlib%2Fsimd-types%2F%3Fhighlight%3DSIMD\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFxPOLyQThP6XN9ku_FTrsLiVWpCg&#39;;return true;">http://docs.julialang.org/en/release-0.5/stdlib/simd-types/?highlight=SIMD that there is a vecElement that is build for SIMD support. I don't understand if as a user I should construct vecElement arrays and hope for some SIMD optimization? thanks.

Reply | Threaded
Open this post in threaded view
|

Re: What's the status of SIMD instructions from a user's perspective in v0.5?

Erik Schnetter
If you want to use the SIMD package, then you need to manually vectorized the code. That is, all (most of) the local variables you're using will have a SIMD `Vec` type. For convenience, your input and output arrays will likely still hold scalar values, and the `vload` and vstore` functions access scalar arrays, reading/writing SIMD vectors. The function you quote above (from the SIMD examples) does just this.

What vector length `N` is best depends on the particular machine. Usually, you would look at the CPU instruction set and choose the largest SIMD vector size that the CPU supports, but sometimes twice that size or half that size might also work well. Note that using a larger SIMD vector size roughly corresponds to loop unrolling, which might be beneficial if the compiler isn't clever enough to do this automatically.

There's additional complication if the array size is not a multiple of the vector size. In this case, extending the array via dummy elements if often the easiest way to go.

Note that SIMD vectorization is purely a performance improvement. It does not make sense to make such changes without measuring performance before and after. Given the low-level nature if the changes, looking at the generated assembler code via `@code_native` is usually also insightful.

I'll be happy to help if you have a specific problem on which you're working.

-erik


On Thu, Oct 13, 2016 at 9:51 AM, Florian Oswald <[hidden email]> wrote:
ok thanks! and so I should define my SIMD-able function like

function vadd!{N,T}(xs::Vector{T}, ys::Vector{T}, ::Type{Vec{N,T}})
    @assert length(ys) == length(xs)
    @assert length(xs) % N == 0
    @inbounds for i in 1:N:length(xs)
        xv = vload(Vec{N,T}, xs, i)
        yv = vload(Vec{N,T}, ys, i)
        xv += yv
        vstore(xv, xs, i)
    end
end
i.e. using vload() and vstore() methods?

On Thursday, 13 October 2016 15:29:50 UTC+2, Valentin Churavy wrote:
If you want explicit simd the best way right now is the great SIMD.jl package https://github.com/eschnett/SIMD.jl  it is builds on top of VecElement.

In many cases we can perform automatic vectorisation, but you have to start Julia with -O3

On Thursday, 13 October 2016 22:15:00 UTC+9, Florian Oswald wrote:
i see on the docs http://docs.julialang.org/en/release-0.5/stdlib/simd-types/?highlight=SIMD that there is a vecElement that is build for SIMD support. I don't understand if as a user I should construct vecElement arrays and hope for some SIMD optimization? thanks.




--
Reply | Threaded
Open this post in threaded view
|

Re: What's the status of SIMD instructions from a user's perspective in v0.5?

Florian Oswald
Hi Erik,

that's great thanks. I may have a hot inner loop where this could be very helpful. I'll have a closer look and come back with any questions later on if that's ok. 

cheers
florian

On Thursday, 13 October 2016 16:24:03 UTC+2, Erik Schnetter wrote:
If you want to use the SIMD package, then you need to manually vectorized the code. That is, all (most of) the local variables you're using will have a SIMD `Vec` type. For convenience, your input and output arrays will likely still hold scalar values, and the `vload` and vstore` functions access scalar arrays, reading/writing SIMD vectors. The function you quote above (from the SIMD examples) does just this.

What vector length `N` is best depends on the particular machine. Usually, you would look at the CPU instruction set and choose the largest SIMD vector size that the CPU supports, but sometimes twice that size or half that size might also work well. Note that using a larger SIMD vector size roughly corresponds to loop unrolling, which might be beneficial if the compiler isn't clever enough to do this automatically.

There's additional complication if the array size is not a multiple of the vector size. In this case, extending the array via dummy elements if often the easiest way to go.

Note that SIMD vectorization is purely a performance improvement. It does not make sense to make such changes without measuring performance before and after. Given the low-level nature if the changes, looking at the generated assembler code via `@code_native` is usually also insightful.

I'll be happy to help if you have a specific problem on which you're working.

-erik


On Thu, Oct 13, 2016 at 9:51 AM, Florian Oswald <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="l0v-QQrJBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">florian...@...> wrote:
ok thanks! and so I should define my SIMD-able function like

function vadd!{N,T}(xs::Vector{T}, ys::Vector{T}, ::Type{Vec{N,T}})
    @assert length(ys) == length(xs)
    @assert length(xs) % N == 0
    @inbounds for i in 1:N:length(xs)
        xv = vload(Vec{N,T}, xs, i)
        yv = vload(Vec{N,T}, ys, i)
        xv += yv
        vstore(xv, xs, i)
    end
end
i.e. using vload() and vstore() methods?

On Thursday, 13 October 2016 15:29:50 UTC+2, Valentin Churavy wrote:
If you want explicit simd the best way right now is the great SIMD.jl package <a href="https://github.com/eschnett/SIMD.jl" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Feschnett%2FSIMD.jl\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEpXfPW96dGD44Z8GQlkr3IqUMrHA&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Feschnett%2FSIMD.jl\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEpXfPW96dGD44Z8GQlkr3IqUMrHA&#39;;return true;">https://github.com/eschnett/SIMD.jl  it is builds on top of VecElement.

In many cases we can perform automatic vectorisation, but you have to start Julia with -O3

On Thursday, 13 October 2016 22:15:00 UTC+9, Florian Oswald wrote:
i see on the docs <a href="http://docs.julialang.org/en/release-0.5/stdlib/simd-types/?highlight=SIMD" rel="nofollow" target="_blank" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fdocs.julialang.org%2Fen%2Frelease-0.5%2Fstdlib%2Fsimd-types%2F%3Fhighlight%3DSIMD\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFxPOLyQThP6XN9ku_FTrsLiVWpCg&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fdocs.julialang.org%2Fen%2Frelease-0.5%2Fstdlib%2Fsimd-types%2F%3Fhighlight%3DSIMD\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFxPOLyQThP6XN9ku_FTrsLiVWpCg&#39;;return true;">http://docs.julialang.org/en/release-0.5/stdlib/simd-types/?highlight=SIMD that there is a vecElement that is build for SIMD support. I don't understand if as a user I should construct vecElement arrays and hope for some SIMD optimization? thanks.




--
Erik Schnetter <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="l0v-QQrJBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">schn...@...> <a href="http://www.perimeterinstitute.ca/personal/eschnetter/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fwww.perimeterinstitute.ca%2Fpersonal%2Feschnetter%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGxlaNboZlt-tpAt8j3eV3SBzPUpg&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fwww.perimeterinstitute.ca%2Fpersonal%2Feschnetter%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGxlaNboZlt-tpAt8j3eV3SBzPUpg&#39;;return true;">http://www.perimeterinstitute.ca/personal/eschnetter/
Reply | Threaded
Open this post in threaded view
|

Re: What's the status of SIMD instructions from a user's perspective in v0.5?

Milan Bouchet-Valat
Le jeudi 13 octobre 2016 à 07:27 -0700, Florian Oswald a écrit :
>
> Hi Erik,
>
> that's great thanks. I may have a hot inner loop where this could be
> very helpful. I'll have a closer look and come back with any
> questions later on if that's ok. 
Maybe I'm stating the obvious, but you don't need to manually use SIMD
types to get SIMD instructions in simple/common cases. For example, the
following high-level generic code uses SIMD instructions on my machine
when passed standard vectors :

function add!(x::AbstractArray, y::AbstractArray)
    @inbounds for i in eachindex(x, y)
        x[i] += y[i]
    end
end


Regards

>
> cheers
> florian
>
> >
> > If you want to use the SIMD package, then you need to manually
> > vectorized the code. That is, all (most of) the local variables
> > you're using will have a SIMD `Vec` type. For convenience, your
> > input and output arrays will likely still hold scalar values, and
> > the `vload` and vstore` functions access scalar arrays,
> > reading/writing SIMD vectors. The function you quote above (from
> > the SIMD examples) does just this.
> >
> > What vector length `N` is best depends on the particular machine.
> > Usually, you would look at the CPU instruction set and choose the
> > largest SIMD vector size that the CPU supports, but sometimes twice
> > that size or half that size might also work well. Note that using a
> > larger SIMD vector size roughly corresponds to loop unrolling,
> > which might be beneficial if the compiler isn't clever enough to do
> > this automatically.
> >
> > There's additional complication if the array size is not a multiple
> > of the vector size. In this case, extending the array via dummy
> > elements if often the easiest way to go.
> >
> > Note that SIMD vectorization is purely a performance improvement.
> > It does not make sense to make such changes without measuring
> > performance before and after. Given the low-level nature if the
> > changes, looking at the generated assembler code via `@code_native`
> > is usually also insightful.
> >
> > I'll be happy to help if you have a specific problem on which
> > you're working.
> >
> > -erik
> >
> >
> > On Thu, Oct 13, 2016 at 9:51 AM, Florian Oswald <[hidden email]
> > om> wrote:
> > >
> > > ok thanks! and so I should define my SIMD-able function like
> > >
> > > function vadd!{N,T}(xs::Vector{T}, ys::Vector{T},
> > > ::Type{Vec{N,T}})
> > >     @assert length(ys) == length(xs)
> > >     @assert length(xs) % N == 0
> > >     @inbounds for i in 1:N:length(xs)
> > >         xv = vload(Vec{N,T}, xs, i)
> > >         yv = vload(Vec{N,T}, ys, i)
> > >         xv += yv
> > >         vstore(xv, xs, i)
> > >     end
> > > end
> > > i.e. using vload() and vstore() methods?
> > >
> > > >
> > > > If you want explicit simd the best way right now is the great
> > > > SIMD.jl package https://github.com/eschnett/SIMD.jl  it is
> > > > builds on top of VecElement.
> > > >
> > > > In many cases we can perform automatic vectorisation, but you
> > > > have to start Julia with -O3
> > > >
> > > > >
> > > > > i see on the docs http://docs.julialang.org/en/release-0.5/st
> > > > > dlib/simd-types/?highlight=SIMD that there is a vecElement
> > > > > that is build for SIMD support. I don't understand if as a
> > > > > user I should construct vecElement arrays and hope for some
> > > > > SIMD optimization? thanks.
> > > > >
> > > > >
> > > >
> > >
> >
> >
> >
> > -- 
> > Erik Schnetter <[hidden email]> http://www.perimeterinstitute.ca
> > /personal/eschnetter/