Plot memory interactions with multiple heads

[opocaj92]

I need to plot the weights and the interactions with the memory for models with multiple read and write heads, but the current implementation of the Dashboard just shows it for a single head (there is a single white square for each time step in the memory column-like plot). Also, for some models (expecially when using the LSTMController) I got weird memory plots, like an almost entirely white read column (that should mean that the controller is interacting with each memory row with a degree of 1, that is not possible or correct)

[tristandeleu]

Indeed that dashboard functionality was intended to be very simple, and does not generalize to multi-head read and write. For anything more complex, I usually had to write code for my specific use-cases.

Visualizing all the heads at once seems to be challenging. The best option might be to plot each head individually on individual axes, which might not be ideal. I'm sorry I can't be of much help here, but I suggest you have a look at the code for Dashboard if you want to extend it for your own visualization.

As for the almost all white columns, this is most likely a problem of normalization. I didn't explicitly normalize the plots in [0, 1], which means that if the weights are almost all the same (~1/128), then the normalization will display them all as white. I will fix this, than you noticing it!

[opocaj92]

Ok, got it! Can you please point me out at the code in the Dashboard that get the location for the heads, so that I can try and make it multi-head myself (probably sweeping through the HeadCollection)?
Thanks for your help, I'm still using this library to do some work on NTM, even if the model itself didn't get all that much attention.

[tristandeleu]

From memory, this is the relevant part for plotting the read and write heads.
I'm glad to hear that you are still using NTMs and this library on your work. Looking forward to seeing what you come up with!

[opocaj92]

Thanks @tristandeleu, I'll surely have a look a the code you pointed me to. In case I can make it work for multiple heads I'll surely create a pull request.
Anyway, I'm more playing around with NTMs than actually working, let's see where it goes.