This is the code for the work presented in the paper "Synthesizing Images from Spatio-Temporal Representations using Spike-based Backpropagation" https://www.frontiersin.org/articles/10.3389/fnins.2019.00621/full
For audio to image conversion, 2 different datasets are used :
Dataset A: one image per class
Dataset B: one image per audio
The files for dataset A are
prepare_imdb.m
ac_test.m
ac_train.m
The corresponding files for dataset B are
prepare_imdb_v2.m
ac_test_v2.m
ac_train_v2.m
The preprocessed multimodal datasets can be found here:
Dataset A:https://purdue0-my.sharepoint.com/:u:/g/personal/roy77_purdue_edu/EfXH0MneHTJOo_WySaIXyZABL3GiEVhU6UITmeZckBsXAg?e=3vDciX