Quantcast

Passing N-D Julia arrays to C functions

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Passing N-D Julia arrays to C functions

Alexander Lyapin
There is a topic:
https://groups.google.com/d/msg/julia-users/EK9oNzzaoAk/kJqagPL0Ku0J

However could some one give an example how to pass 3-d or 4-d array to C function.

I have Array{Float64, 4} and for ccall I use Ptr{Ptr{Ptr{Ptr{Float64}}}} as Type of parameter. Is there some way to make it more common for Array{Float64, N}??? Thank you
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Passing N-D Julia arrays to C functions

Yichao Yu
On Fri, Nov 4, 2016 at 9:22 AM, Alexander Lyapin
<[hidden email]> wrote:
> There is a topic:
> https://groups.google.com/d/msg/julia-users/EK9oNzzaoAk/kJqagPL0Ku0J
>
> However could some one give an example how to pass 3-d or 4-d array to C
> function.
>
> I have Array{Float64, 4} and for ccall I use Ptr{Ptr{Ptr{Ptr{Float64}}}} as
> Type of parameter. Is there some way to make it more common for
> Array{Float64, N}??? Thank you

No matter what the dimension is,it's always a `Ptr{eltype(array)}` in
C (`Ptr{Float64}` in thiscase) and since it's just a normalpointer,
you always need topass the dimensions explicitlyto C in another
argumetns.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Passing N-D Julia arrays to C functions

Tim Holy
If you need the pointers to each lower-dimensional slice, you should be able to build it like so (warning: untested):

```
sz = size(A)[2:end]  # if you need type-stability, use Base.tail(size(A)) instead
Aptrs = Array{Ptr{eltype(A)}(sz)
for I in CartesianRange(sz)
    Aptrs[I] = pointer(A, sub2ind(size(A), 1, I.I...))
end
```
and then pass Aptrs to your C function.

Note that A must be an Array (not a generic AbstractArray) for this to work.

Best,
--Tim

On Fri, Nov 4, 2016 at 9:05 AM, Yichao Yu <[hidden email]> wrote:
On Fri, Nov 4, 2016 at 9:22 AM, Alexander Lyapin
<[hidden email]> wrote:
> There is a topic:
> https://groups.google.com/d/msg/julia-users/EK9oNzzaoAk/kJqagPL0Ku0J
>
> However could some one give an example how to pass 3-d or 4-d array to C
> function.
>
> I have Array{Float64, 4} and for ccall I use Ptr{Ptr{Ptr{Ptr{Float64}}}} as
> Type of parameter. Is there some way to make it more common for
> Array{Float64, N}??? Thank you

No matter what the dimension is,it's always a `Ptr{eltype(array)}` in
C (`Ptr{Float64}` in thiscase) and since it's just a normalpointer,
you always need topass the dimensions explicitlyto C in another
argumetns.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Passing N-D Julia arrays to C functions

Alexander Lyapin
In reply to this post by Yichao Yu
but in C i have **double, wich is pointer pointer to array. 
Also in link that I gave, there an example for 2D where it is Ptr{Ptr{Float64}}

But anyway, could you give an example for ccall in your case???

пятница, 4 ноября 2016 г., 17:05:57 UTC+3 пользователь Yichao Yu написал:
On Fri, Nov 4, 2016 at 9:22 AM, Alexander Lyapin
<<a href="javascript:" target="_blank" gdf-obfuscated-mailto="NfpQ35WBAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">lyapinaa...@...> wrote:
> There is a topic:
> <a href="https://groups.google.com/d/msg/julia-users/EK9oNzzaoAk/kJqagPL0Ku0J" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msg/julia-users/EK9oNzzaoAk/kJqagPL0Ku0J&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msg/julia-users/EK9oNzzaoAk/kJqagPL0Ku0J&#39;;return true;">https://groups.google.com/d/msg/julia-users/EK9oNzzaoAk/kJqagPL0Ku0J
>
> However could some one give an example how to pass 3-d or 4-d array to C
> function.
>
> I have Array{Float64, 4} and for ccall I use Ptr{Ptr{Ptr{Ptr{Float64}}}} as
> Type of parameter. Is there some way to make it more common for
> Array{Float64, N}??? Thank you

No matter what the dimension is,it's always a `Ptr{eltype(array)}` in
C (`Ptr{Float64}` in thiscase) and since it's just a normalpointer,
you always need topass the dimensions explicitlyto C in another
argumetns.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Passing N-D Julia arrays to C functions

Tim Holy
ccall(sym, Void, (Ptr{Ptr{T}},), Aptrs)

When you pass an array via `ccall`, it takes the pointer of the data, which is why this produces a Ptr{Ptr{T}} from an Array{Ptr{T}}.

Best,
--Tim

On Fri, Nov 4, 2016 at 9:14 AM, Alexander Lyapin <[hidden email]> wrote:
but in C i have **double, wich is pointer pointer to array. 
Also in link that I gave, there an example for 2D where it is Ptr{Ptr{Float64}}

But anyway, could you give an example for ccall in your case???

пятница, 4 ноября 2016 г., 17:05:57 UTC+3 пользователь Yichao Yu написал:
On Fri, Nov 4, 2016 at 9:22 AM, Alexander Lyapin
<[hidden email]> wrote:
> There is a topic:
> https://groups.google.com/d/msg/julia-users/EK9oNzzaoAk/kJqagPL0Ku0J
>
> However could some one give an example how to pass 3-d or 4-d array to C
> function.
>
> I have Array{Float64, 4} and for ccall I use Ptr{Ptr{Ptr{Ptr{Float64}}}} as
> Type of parameter. Is there some way to make it more common for
> Array{Float64, N}??? Thank you

No matter what the dimension is,it's always a `Ptr{eltype(array)}` in
C (`Ptr{Float64}` in thiscase) and since it's just a normalpointer,
you always need topass the dimensions explicitlyto C in another
argumetns.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Passing N-D Julia arrays to C functions

Alexander Lyapin
Tim, thank you for reply!

Everything was fine when I tried simple 2D array. Then I moved to real problem and got some errors... please check:

-----test.c-----
#include <stdio.h>
double test(double ****param) {
return param[1][1][1][1];
}


----Julia-----
# ar is Array{Float64,4}

sz = size(ar)[1:end] 
Aptrs = Array{Ptr{eltype(ar)}}(sz)
for I in CartesianRange(sz)
    Aptrs[I] = pointer(ar, sub2ind(size(ar), 1, I.I...))
end
println(ccall(("test","test"),Float64,(Ptr{Ptr{Ptr{Ptr{Float64}}}},), Aptrs))


something is wrong here...


пятница, 4 ноября 2016 г., 17:19:17 UTC+3 пользователь Tim Holy написал:
ccall(sym, Void, (Ptr{Ptr{T}},), Aptrs)

When you pass an array via `ccall`, it takes the pointer of the data, which is why this produces a Ptr{Ptr{T}} from an Array{Ptr{T}}.

Best,
--Tim

On Fri, Nov 4, 2016 at 9:14 AM, Alexander Lyapin <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="uDV2DVCCAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">lyapinaa...@...> wrote:
but in C i have **double, wich is pointer pointer to array. 
Also in link that I gave, there an example for 2D where it is Ptr{Ptr{Float64}}

But anyway, could you give an example for ccall in your case???

пятница, 4 ноября 2016 г., 17:05:57 UTC+3 пользователь Yichao Yu написал:
On Fri, Nov 4, 2016 at 9:22 AM, Alexander Lyapin
<[hidden email]> wrote:
> There is a topic:
> <a href="https://groups.google.com/d/msg/julia-users/EK9oNzzaoAk/kJqagPL0Ku0J" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/msg/julia-users/EK9oNzzaoAk/kJqagPL0Ku0J&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msg/julia-users/EK9oNzzaoAk/kJqagPL0Ku0J&#39;;return true;">https://groups.google.com/d/msg/julia-users/EK9oNzzaoAk/kJqagPL0Ku0J
>
> However could some one give an example how to pass 3-d or 4-d array to C
> function.
>
> I have Array{Float64, 4} and for ccall I use Ptr{Ptr{Ptr{Ptr{Float64}}}} as
> Type of parameter. Is there some way to make it more common for
> Array{Float64, N}??? Thank you

No matter what the dimension is,it's always a `Ptr{eltype(array)}` in
C (`Ptr{Float64}` in thiscase) and since it's just a normalpointer,
you always need topass the dimensions explicitlyto C in another
argumetns.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Passing N-D Julia arrays to C functions

Tim Holy
As I tried to explain before, Aptrs corresponds to a Ptr{Ptr{T}} *no matter how many dimensions you have*. But showing the C code you want to use makes your problem vastly clearer: now I know you want a `Ptr{Ptr{Ptr{Ptr{T}}}}` for 4 dimensions.

I think you want a recursive solution, something like
```
ptr1d(A::AbstractVector) = pointer(A)

function ptr1d(A::AbstractArray)
    colons = fill(:, ndims(A) - 1)
    ptr1 = ptr1d(view(A, colons..., 1)) # get 1 pointer so we know the type
    Aptr = Array{typeof(ptr1)}(size(A)[end])
    Aptr[1] = ptr1
    for i = 2:size(A)[end]
        Aptr[i] = ptr1d(view(A, colons..., i))
    end
    Aptr
end
```

Then just pass `ptr1d(A)` as the last argument of your ccall.

Best,
--Tim


On Fri, Nov 4, 2016 at 10:07 AM, Alexander Lyapin <[hidden email]> wrote:
Tim, thank you for reply!

Everything was fine when I tried simple 2D array. Then I moved to real problem and got some errors... please check:

-----test.c-----
#include <stdio.h>
double test(double ****param) {
return param[1][1][1][1];
}


----Julia-----
# ar is Array{Float64,4}

sz = size(ar)[1:end] 
Aptrs = Array{Ptr{eltype(ar)}}(sz)
for I in CartesianRange(sz)
    Aptrs[I] = pointer(ar, sub2ind(size(ar), 1, I.I...))
end
println(ccall(("test","test"),Float64,(Ptr{Ptr{Ptr{Ptr{Float64}}}},), Aptrs))


something is wrong here...


пятница, 4 ноября 2016 г., 17:19:17 UTC+3 пользователь Tim Holy написал:
ccall(sym, Void, (Ptr{Ptr{T}},), Aptrs)

When you pass an array via `ccall`, it takes the pointer of the data, which is why this produces a Ptr{Ptr{T}} from an Array{Ptr{T}}.

Best,
--Tim

On Fri, Nov 4, 2016 at 9:14 AM, Alexander Lyapin <[hidden email]> wrote:
but in C i have **double, wich is pointer pointer to array. 
Also in link that I gave, there an example for 2D where it is Ptr{Ptr{Float64}}

But anyway, could you give an example for ccall in your case???

пятница, 4 ноября 2016 г., 17:05:57 UTC+3 пользователь Yichao Yu написал:
On Fri, Nov 4, 2016 at 9:22 AM, Alexander Lyapin
<[hidden email]> wrote:
> There is a topic:
> https://groups.google.com/d/msg/julia-users/EK9oNzzaoAk/kJqagPL0Ku0J
>
> However could some one give an example how to pass 3-d or 4-d array to C
> function.
>
> I have Array{Float64, 4} and for ccall I use Ptr{Ptr{Ptr{Ptr{Float64}}}} as
> Type of parameter. Is there some way to make it more common for
> Array{Float64, N}??? Thank you

No matter what the dimension is,it's always a `Ptr{eltype(array)}` in
C (`Ptr{Float64}` in thiscase) and since it's just a normalpointer,
you always need topass the dimensions explicitlyto C in another
argumetns.


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Passing N-D Julia arrays to C functions

Alexander Lyapin
Tim... I just can't say how thank I am to you. Really. Your code is so advanced. Works perfect.

By the way, the numeration of array after ccall is inversed. Interesting.

Thank you a lot

пятница, 4 ноября 2016 г., 19:23:25 UTC+3 пользователь Tim Holy написал:
As I tried to explain before, Aptrs corresponds to a Ptr{Ptr{T}} *no matter how many dimensions you have*. But showing the C code you want to use makes your problem vastly clearer: now I know you want a `Ptr{Ptr{Ptr{Ptr{T}}}}` for 4 dimensions.

I think you want a recursive solution, something like
```
ptr1d(A::AbstractVector) = pointer(A)

function ptr1d(A::AbstractArray)
    colons = fill(:, ndims(A) - 1)
    ptr1 = ptr1d(view(A, colons..., 1)) # get 1 pointer so we know the type
    Aptr = Array{typeof(ptr1)}(size(A)[end])
    Aptr[1] = ptr1
    for i = 2:size(A)[end]
        Aptr[i] = ptr1d(view(A, colons..., i))
    end
    Aptr
end
```

Then just pass `ptr1d(A)` as the last argument of your ccall.

Best,
--Tim


On Fri, Nov 4, 2016 at 10:07 AM, Alexander Lyapin <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="Edp1MxaJAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">lyapinaa...@...> wrote:
Tim, thank you for reply!

Everything was fine when I tried simple 2D array. Then I moved to real problem and got some errors... please check:

-----test.c-----
#include <stdio.h>
double test(double ****param) {
return param[1][1][1][1];
}


----Julia-----
# ar is Array{Float64,4}

sz = size(ar)[1:end] 
Aptrs = Array{Ptr{eltype(ar)}}(sz)
for I in CartesianRange(sz)
    Aptrs[I] = pointer(ar, sub2ind(size(ar), 1, I.I...))
end
println(ccall(("test","test"),Float64,(Ptr{Ptr{Ptr{Ptr{Float64}}}},), Aptrs))


something is wrong here...


пятница, 4 ноября 2016 г., 17:19:17 UTC+3 пользователь Tim Holy написал:
ccall(sym, Void, (Ptr{Ptr{T}},), Aptrs)

When you pass an array via `ccall`, it takes the pointer of the data, which is why this produces a Ptr{Ptr{T}} from an Array{Ptr{T}}.

Best,
--Tim

On Fri, Nov 4, 2016 at 9:14 AM, Alexander Lyapin <[hidden email]> wrote:
but in C i have **double, wich is pointer pointer to array. 
Also in link that I gave, there an example for 2D where it is Ptr{Ptr{Float64}}

But anyway, could you give an example for ccall in your case???

пятница, 4 ноября 2016 г., 17:05:57 UTC+3 пользователь Yichao Yu написал:
On Fri, Nov 4, 2016 at 9:22 AM, Alexander Lyapin
<[hidden email]> wrote:
> There is a topic:
> <a href="https://groups.google.com/d/msg/julia-users/EK9oNzzaoAk/kJqagPL0Ku0J" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/msg/julia-users/EK9oNzzaoAk/kJqagPL0Ku0J&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msg/julia-users/EK9oNzzaoAk/kJqagPL0Ku0J&#39;;return true;">https://groups.google.com/d/msg/julia-users/EK9oNzzaoAk/kJqagPL0Ku0J
>
> However could some one give an example how to pass 3-d or 4-d array to C
> function.
>
> I have Array{Float64, 4} and for ccall I use Ptr{Ptr{Ptr{Ptr{Float64}}}} as
> Type of parameter. Is there some way to make it more common for
> Array{Float64, N}??? Thank you

No matter what the dimension is,it's always a `Ptr{eltype(array)}` in
C (`Ptr{Float64}` in thiscase) and since it's just a normalpointer,
you always need topass the dimensions explicitlyto C in another
argumetns.


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Passing N-D Julia arrays to C functions

Steven G. Johnson
Note that using **double is not actually that great a way to do multidimensional arrays in C -- way harder to do the memory management, and also slower.   Most high-performance code uses just double* and then accesses it in row-major or column-major order.

See, for example, the discussion here: http://www.fftw.org/doc/Multi_002ddimensional-Array-Format.html ... what you are doing is discussed under "dynamic arrays - the wrong way".

(Julia's arrays are stored in column-major order, so they can be passed directly to a C program expecting a double* pointer to column-major data.)
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Passing N-D Julia arrays to C functions

Alexander Lyapin
Thank you, for this info. I will definitely need this!

пятница, 4 ноября 2016 г., 22:26:50 UTC+3 пользователь Steven G. Johnson написал:
Note that using **double is not actually that great a way to do multidimensional arrays in C -- way harder to do the memory management, and also slower.   Most high-performance code uses just double* and then accesses it in row-major or column-major order.

See, for example, the discussion here: <a href="http://www.fftw.org/doc/Multi_002ddimensional-Array-Format.html" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fwww.fftw.org%2Fdoc%2FMulti_002ddimensional-Array-Format.html\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFgzm-YXLd0D3yBGeUMdftDbw74vA&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fwww.fftw.org%2Fdoc%2FMulti_002ddimensional-Array-Format.html\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFgzm-YXLd0D3yBGeUMdftDbw74vA&#39;;return true;">http://www.fftw.org/doc/Multi_002ddimensional-Array-Format.html ... what you are doing is discussed under "dynamic arrays - the wrong way".

(Julia's arrays are stored in column-major order, so they can be passed directly to a C program expecting a double* pointer to column-major data.)
Loading...