We investigate the emergence of factual knowledge in pretrained MLLMs. More concretely, we conduct a study to explore factual knowledge sharing and symbolic reasoning in a zero-shot cross-lingual setting. For this we investigate (i) how much these models depend on a shared representation when being probed for factual knowledge and (ii) the ability to use symbolic reasoning across languages to infer factual knowledge not seen explicitly during pretraining.
To run the experiments we provide several bash scripts with already pre-selected hyperparameters.