About the Dictionary Learning

[TDBaker2016]

Hi, minhnhat93. I tested your implementation of the LISTA. During the dictionary learning part, I find some bugs. The dictionary couldn't be learned when I tried to fix them. Could you please help me with it?

Besides, according to the original paper of LISTA in parameter setting, the learning rate for LISTA training decays as 1/n (n-iter number), but the thing is when I did the exact same thing. The gradient is exploded to NaN. I guess it is the single-sample training strategy instead of mini-batch training causes this, making the gradient unstable. I notice you set extremely small learning rate, so there is no such problem. My question is if it is possible to deploy batch gradient descends to address this problem?

Looking forword to hearing from you!

[minhnhat93]

Hi TDBaker2016, I think batch training could work and would work way better. This Github repository is what I did almost 2 years before when I knew almost nothing about training neural networks and I did the gradient descent from scratch so there might be a lot of bugs in it. It would be the best for you to consider using one of the other frameworks like Tensorflow with nicer interfaces and automatic gradient computing. The repository is no longer maintained and I moved to new projects. Sorry. *Minh Nhat Nguyen* Grad Student, Department of Computing Science, University of Alberta Email: minhnhat93@gmail.com / nmnguyen@ualberta.ca

…

On 22 March 2017 at 03:22, TDBaker2016 ***@***.***> wrote: Hi, minhnhat93. I tested your implementation of the LISTA. During the dictionary learning part, I find some bugs. The dictionary couldn't be learned when I tried to fix them. Could you please help me with it? Besides, according to the original paper of LISTA in parameter setting, the learning rate for LISTA training decays as 1/n (n-iter number), but the thing is when I did the exact same thing. The gradient is exploded to NaN. I guess it is the single-sample training strategy instead of mini-batch training causes this, making the gradient unstable. I notice you set extremely small learning rate, so there is no such problem. My question is if it is possible to deploy batch gradient descends to address this problem? Looking forword to hearing from you! — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#2>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AIVxydnvK8BOQL6E0f_g6rQA_E8kNBPoks5roOg-gaJpZM4Mk62v> .