Starting from a previous training

[ngoel17]

I think there is a need to allow iteration over previous training, and also to allow multiple threads during training. Overall the process is very slow. What are your thoughts about adapting http://www.rnnlm.org/ to this task. I think the LSTM cell is missing there.

[ottokart]

Thanks for the feedback!
Do you mean the ability to continue training from the last saved state in case the program crashes?

Numpy with MKL or ATLAS/BLAS/LAPACK should be able to utilise multiple threads (I see 66% utilisation of my 4 cores with MKL).
The rnnlm toolkit is single-threaded and does not support mini-batches, so I don't expect it to be much faster.

The example runs about 6000 wps in 1st phase on my laptop. How slow is it for you?
Reducing the BPTT steps (from 5 to 3-4) should also speed things up and not hurt performance too much.