Re: NeuralNetworks.jl

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Re: NeuralNetworks.jl

Alan Chan
Hi Wade - any updates on releasing the naive Julia Theano-port? Thx

On Wednesday, December 11, 2013 10:43:24 PM UTC-5, S Wade wrote:
Yes, that's the plan. Theano does expression-level fusion which generally produces very optimized gpu code. We don't quite have a direct alternative yet for Julia.

Btw we're going to release our code in the next month.

thanks,
wade

> On Dec 11, 2013, at 10:13 PM, John Myles White <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="9ZedjLPrN8sJ" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">johnmyl...@...> wrote:
>
> Can you use some of the growing CUDA integration libraries to push things from Julia to the GPU?
>
> — John
>
>> On Dec 11, 2013, at 7:10 PM, S Wade <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="9ZedjLPrN8sJ" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">swade...@...> wrote:
>>
>> We're in the process of doing a port of our deep network implementation from theano/c.  The good news is that the basics (bunched forward, cd training, backprop, etc.) are close to c performance if we optimize carefully.  Gpu implementations are needed for any real problems. So the bad news is that our theano implementation with the cuda back end is easily an order of magnitude faster.  
>>
>> thanks,
>> wade
>>
>>> On Dec 11, 2013, at 9:56 PM, John Myles White <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="9ZedjLPrN8sJ" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">johnmyl...@...> wrote:
>>>
>>> I agree that this would be a huge gain for us.
>>>
>>> If you’d like to implement a serious neural network library, I’d try to find a template with an acceptable license and translate it. There’s so much black magic in effective NN code that any de novo implementation is likely to do much worse than the state of the art.
>>>
>>> Some examples:
>>>
>>> * <a href="https://code.google.com/p/matrbm/" target="_blank" onmousedown="this.href='https://code.google.com/p/matrbm/';return true;" onclick="this.href='https://code.google.com/p/matrbm/';return true;">https://code.google.com/p/matrbm/
>>> * <a href="https://code.google.com/p/cuda-convnet/" target="_blank" onmousedown="this.href='https://code.google.com/p/cuda-convnet/';return true;" onclick="this.href='https://code.google.com/p/cuda-convnet/';return true;">https://code.google.com/p/cuda-convnet/ (This is almost certainly where you want to start from. Friends in Hinton’s lab say this is their preferred implementation.)
>>>
>>> Some examples that need proper licenses to be distributable:
>>>
>>> * <a href="https://github.com/rasmusbergpalm/DeepLearnToolbox" target="_blank" onmousedown="this.href='https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Frasmusbergpalm%2FDeepLearnToolbox\46sa\75D\46sntz\0751\46usg\75AFQjCNHMTpj7v-di2IKnMh0vHFbX633oKQ';return true;" onclick="this.href='https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Frasmusbergpalm%2FDeepLearnToolbox\46sa\75D\46sntz\0751\46usg\75AFQjCNHMTpj7v-di2IKnMh0vHFbX633oKQ';return true;">https://github.com/rasmusbergpalm/DeepLearnToolbox
>>> * <a href="http://www.cs.toronto.edu/%7Ehinton/MatlabForSciencePaper.html" target="_blank" onmousedown="this.href='http://www.google.com/url?q\75http%3A%2F%2Fwww.cs.toronto.edu%2F%257Ehinton%2FMatlabForSciencePaper.html\46sa\75D\46sntz\0751\46usg\75AFQjCNHV6F0PcS6YkaQPRolvisiKNu9V8A';return true;" onclick="this.href='http://www.google.com/url?q\75http%3A%2F%2Fwww.cs.toronto.edu%2F%257Ehinton%2FMatlabForSciencePaper.html\46sa\75D\46sntz\0751\46usg\75AFQjCNHV6F0PcS6YkaQPRolvisiKNu9V8A';return true;">http://www.cs.toronto.edu/%7Ehinton/MatlabForSciencePaper.html
>>>
>>> Or you might start from some tutorials:
>>>
>>> * <a href="http://deeplearning.net/software/theano/" target="_blank" onmousedown="this.href='http://www.google.com/url?q\75http%3A%2F%2Fdeeplearning.net%2Fsoftware%2Ftheano%2F\46sa\75D\46sntz\0751\46usg\75AFQjCNFJoBcoA4NcY2Q-YzLHUW1Ho-Wexw';return true;" onclick="this.href='http://www.google.com/url?q\75http%3A%2F%2Fdeeplearning.net%2Fsoftware%2Ftheano%2F\46sa\75D\46sntz\0751\46usg\75AFQjCNFJoBcoA4NcY2Q-YzLHUW1Ho-Wexw';return true;">http://deeplearning.net/software/theano/
>>> * <a href="http://deeplearning.net/software/pylearn2/" target="_blank" onmousedown="this.href='http://www.google.com/url?q\75http%3A%2F%2Fdeeplearning.net%2Fsoftware%2Fpylearn2%2F\46sa\75D\46sntz\0751\46usg\75AFQjCNGBMnzNn_OpNA3DwbMvS-yIlfaGKQ';return true;" onclick="this.href='http://www.google.com/url?q\75http%3A%2F%2Fdeeplearning.net%2Fsoftware%2Fpylearn2%2F\46sa\75D\46sntz\0751\46usg\75AFQjCNGBMnzNn_OpNA3DwbMvS-yIlfaGKQ';return true;">http://deeplearning.net/software/pylearn2/
>>>
>>> — John
>>>
>>>> On Dec 11, 2013, at 4:45 PM, Avik Sengupta <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="9ZedjLPrN8sJ" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">avik.s...@...> wrote:
>>>>
>>>> If you want a toy implementation, you can start with <a href="https://github.com/aviks/SimpleML.jl" target="_blank" onmousedown="this.href='https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Faviks%2FSimpleML.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNGQ630yykZV5yPjmzGTEg4V6hw9jg';return true;" onclick="this.href='https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Faviks%2FSimpleML.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNGQ630yykZV5yPjmzGTEg4V6hw9jg';return true;">https://github.com/aviks/SimpleML.jl . This is not in METADATA for obvious reasons.
>>>>
>>>>
>>>> On Wednesday, 11 December 2013 17:47:31 UTC, Milktrader wrote:
>>>> I've got too much on my plate to do this probably, but I *hope* someone gets motivated. I think Julia is particularly well-suited to deal with neural network algorithms. I'm going to hack something together in an off-line repo and name it something obtuse, like NeuralNetworkForHakers.jl or something so nobody gets confused it's a scholarly-sound implementation.
>>>>
>>>> Cheers.
>>>>
>>>> On Wednesday, December 11, 2013 12:06:12 PM UTC-5, Milktrader wrote:
>>>> Well maybe I should've github searched first. There is a neuralnetwork.jl repo. Pinging @tomaskrehlik
>>>>
>>>>
>>>>
>>>> On Wednesday, December 11, 2013 12:04:28 PM UTC-5, Milktrader wrote:
>>>> Haven't noticed anyone taking a stab at implementing this. I don't have expertise, but can hack out some methods and start designing a framework. Unless I've missed it and it's being done somewhere else.
>>>>
>>>> I sorta need one so I'm going to begin hacking something together, but I don't want to take the name if someone else is working on it. No plans to publish it until it gets robust of course.
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google Groups "julia-stats" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="9ZedjLPrN8sJ" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">julia-stats...@googlegroups.com.
>>>> For more options, visit <a href="https://groups.google.com/groups/opt_out" target="_blank" onmousedown="this.href='https://groups.google.com/groups/opt_out';return true;" onclick="this.href='https://groups.google.com/groups/opt_out';return true;">https://groups.google.com/groups/opt_out.
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups "julia-stats" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="9ZedjLPrN8sJ" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">julia-stats...@googlegroups.com.
>>> For more options, visit <a href="https://groups.google.com/groups/opt_out" target="_blank" onmousedown="this.href='https://groups.google.com/groups/opt_out';return true;" onclick="this.href='https://groups.google.com/groups/opt_out';return true;">https://groups.google.com/groups/opt_out.
>>
>> --
>> You received this message because you are subscribed to the Google Groups "julia-stats" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="9ZedjLPrN8sJ" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">julia-stats...@googlegroups.com.
>> For more options, visit <a href="https://groups.google.com/groups/opt_out" target="_blank" onmousedown="this.href='https://groups.google.com/groups/opt_out';return true;" onclick="this.href='https://groups.google.com/groups/opt_out';return true;">https://groups.google.com/groups/opt_out.
>
> --
> You received this message because you are subscribed to the Google Groups "julia-stats" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="9ZedjLPrN8sJ" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">julia-stats...@googlegroups.com.
> For more options, visit <a href="https://groups.google.com/groups/opt_out" target="_blank" onmousedown="this.href='https://groups.google.com/groups/opt_out';return true;" onclick="this.href='https://groups.google.com/groups/opt_out';return true;">https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.