Causal connection weight update with MSTDPET

[aomafarag]

Hello 👋

I'm using BindsNET to train an SNN to solve the CartPole problem. Since I want a finer control of the encoding and rewards, I've opted out of using the pipeline.

Here's the structure of my network:

inpt = Input(n=81, shape=(1, 81), traces=True)
hidden = LIFNodes(n=100, traces=True)
out = LIFNodes(n=2, refrac=0, traces=True)

inpt_hidden = Connection(source=inpt, target=hidden, wmin=0, wmax=1)
hidden_out = Connection(source=hidden, target=out, wmin=0, wmax=1, update_rule=MSTDPET, nu=1e-1)

I noted that whenever I do

network.run(inputs={"Input Layer": data}, time=time, reward=reward)

the reward is applied indiscriminately to all the weights of hidden_out (all increase/decrease), not taking into account which output neuron caused that reward.

How can I specify which connection weights to update? Do I have to break the output layer into two separate ones? Bonus: How can I propagate this causal update through a deep network?

[Hananel-Hazan]

Using reward and MSTDPET will modify the wight for the relevant neuron. Please read the docs on MSTDPET and the Florian 2007 paper.