MAP score report on which split?

[aneesh-joshi]

Hi @shuohangwang
I ran your Lua code and found that
MAP on dev split is 0.74
MAP on test split is 0.69

for WikiQA

Is the score in the paper(0.74) on the dev set or the test set?

[shuohangwang]

Thank you for the interest in this work!

It's on the test. Could you try to run the code under the docker I provided?

Thanks,
Shuohang

[aneesh-joshi]

@shuohangwang
Thank you for the reply.
I will try again soon.