Best approach for serializing subset of AST for FHE

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Best approach for serializing subset of AST for FHE

Jose Calderon
Hello Julia-devs,

I’m working on a grant for developing tools that allow data scientist
to utilize fully homomorphic encryption (FHE).

One of the steps required is serializing the AST and sending it to a
data-warehouse where the data is encrypted. We want to serialize
Julia’s AST and not llvm because we’d like to be able to optimize at
different levels; FHE is slow we might want to do some rewriting of
the code to get more parallelism, etc.

The Julia compiler has some serialization capabilities but when I
looked around it seems that it does not support sending code to other
julia sessions, is this still true?

Because of that our current plan is to serialize a subset of Julia’s
lowered AST by hand and modify the interpreter to recognize when we
want to compute using FHE and send the serialized code to the data
warehouse. Since we’re new to the Julia codebase and ecosystem I
thought it was worth pinging this group to see if there’s some more
elegant way of doing it that we’ve not seen, perhaps utilizing Julia’s
reflection features.

Any insight on this sort of thing would be greatly appreciated.

Cheers,

Jose
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Best approach for serializing subset of AST for FHE

Valentin Churavy
You should be able to serialise lambdas

julia> io = IOBuffer()
IOBuffer(data=UInt8[...], readable=true, writable=true, seekable=true, append=false, size=0, maxsize=Inf, ptr=1, mark=-1)

julia> serialize(io, (x)->x^2)

julia> seekstart(io)
IOBuffer(data=UInt8[...], readable=true, writable=true, seekable=true, append=false, size=262, maxsize=Inf, ptr=1, mark=-1)

julia> f = deserialize(io)
(::#1) (generic function with 1 method)

julia> f(2)
4

you could also take a look at base/multi.jl where Julia's distributed parallelism is defined, which seems similar to your problem (code has to be send to workers and executed there)

On Tuesday, 1 November 2016 23:43:59 UTC+9, Jose Calderon wrote:
Hello Julia-devs,

I’m working on a grant for developing tools that allow data scientist
to utilize fully homomorphic encryption (FHE).

One of the steps required is serializing the AST and sending it to a
data-warehouse where the data is encrypted. We want to serialize
Julia’s AST and not llvm because we’d like to be able to optimize at
different levels; FHE is slow we might want to do some rewriting of
the code to get more parallelism, etc.

The Julia compiler has some serialization capabilities but when I
looked around it seems that it does not support sending code to other
julia sessions, is this still true?

Because of that our current plan is to serialize a subset of Julia’s
lowered AST by hand and modify the interpreter to recognize when we
want to compute using FHE and send the serialized code to the data
warehouse. Since we’re new to the Julia codebase and ecosystem I
thought it was worth pinging this group to see if there’s some more
elegant way of doing it that we’ve not seen, perhaps utilizing Julia’s
reflection features.

Any insight on this sort of thing would be greatly appreciated.

Cheers,

Jose
Loading...