Nice summary. I'm playing with RNNs now to generate text, and man, the 'remembering' part is a double-edged sword. It's fascinating when it works, but a pain to troubleshoot. Drives me nuts sometimes, but I love it.