Recurrent Support

[chinux23]

I love your toolkit, it's wonderful and super easy to use. I am wondering if you plan to add support for recurrent neural network?

[alexjc]

Yes, we're thinking about it :-) Would you want the API to be different (better suited to temporal data) or the same from what scikit-learn supports?

[chinux23]

Glad to hear that 👍 I am not sure what API would be like for temporal data. But it seems intuitive at least for me if it's like scikit-learn.

[chinux23]

I found a python2 recurrent project here: http://pyneurgen.sourceforge.net/recurrent.html
I am not sure if this could be helpful in design the API in any way.

[alexjc]

A more temporal API would include a step() function that can take infinite sequences of data. With scikit-learn you limit the API to something finite and fixed length.

[ssamot]

@alexjc
My two cents. Real time recurrent neural networks are hard to get working in python - I am not aware of a theano implementation that actually works. A temporal API would accept some 3D structure (e.g. `n_timestep, n_samples, n_inputs' and once trained the network would be able to accept a similar structure that accepts the whole temporal sequence - this is how most packages do it.

Not sure why this is not flagged as a greater problem - it will look much deeper into it in the following months

[alexjc]

Should be possible now, here's how it would work:
http://lasagne.readthedocs.org/en/latest/modules/layers/recurrent.html

We'll probably just add a Recurrent layer type like Convolution and have it take rnn | gru | lstm as a type, then connect everything up via Lasagne automatically :-)

[alexjc]

#195 lets you add any native Lasagne layer to sknn, so you can use this to inject LSTM layers for example. If you try this, let me know so I can look at the code and see how best to abstract it ;-)

[pGit1]

#195 lets you add any native Lasagne layer to sknn, so you can use this to inject LSTM layers for example. If you try this, let me know so I can look at the code and see how best to abstract it ;-)

Has anyone tried this? Having difficulty with lasagne Recurrent Layer myself.