Relation Matching for MetaQA

[XLR81999]

Hi,
I've been following this approach for a while, I've noticed that Relation Matching module has been added, however I couldn't find it for MetaQA/LSTM approach, will it be added later or is it not yet implemented?

[apoorvumang]

Hi, I will try too add ASAP

[namadjidku]

Hi, in the paper you mentioned that relation matching was performed for the larger dataset to boost its performance and typically for relatively smaller KGs like MetaQA the answer is selected based on the highest score. Using only the highest score, it is not possible to achieve the reported results for MetaQA, as mentioned in #31 the best result for 3-hop questions is around 70. Did you use relation matching also for MetaQA/LSTM? If yes, it will be highly appreciated if you upload the code. Thank you.

[apoorvumang]

@namadjidku Relation matching was needed for only 3-hop full setting in MetaQA since the paths are too long for the base model to handle properly. For 3-hop half, relation matching was not used (since it doesn't work well on incomplete graphs).

The code requires a bit of cleaning up before upload. Will be uploading it ASAP.

[okanvk]

Hello, is there any updates for relation matching, to generate 3-hop model as state-of-the-art classifier.

[okanvk]

Hi, is there any updates for Relation Matching code?

[apoorvumang]

Not yet unfortunately. I will pin this issue until it gets added. However, based on more recent work in this area/MetaQA dataset, I would recommend against using EmbedKGQA in full KG setting - semantic parsing models should be used in such a setting IMO.

If needed, you can use the 0.728 number mentioned in #31 for reporting.