top_k in psuedo_label_generator

[sinchanabhat]

Hi, I am currently playing around with GPL negative mining step. I am just getting one negative doc per positive doc even after setting top_k value to any number like 10 or 50. Am I setting the variable in some weird way? Any inputs would be appreciated. I have total of 278 positive passages.

[vblagoje]

Hey @sinchanabhat, can you share a colab notebook? I'll take a look.

[vblagoje]

Hey @sinchanabhat no reason to think about how many negatives you need. In the colab, I saw you were trying to get 50 negatives - why? Each training sample generated from the mining/score negatives stage is a dict with a question, pos_doc, and neg_doc keys - each key having a corresponding value. So if you need to examine the generated training data - you can. No, you can't use DPR - only embedding retrievers. HTH, cheers!

[sinchanabhat]

Hey @sinchanabhat no reason to think about how many negatives you need. In the colab, I saw you were trying to get 50 negatives - why? Each training sample generated from the mining/score negatives stage is a dict with a question, pos_doc, and neg_doc keys - each key having a corresponding value. So if you need to examine the generated training data - you can. No, you can't use DPR - only embedding retrievers. HTH, cheers!

Thanks alot for swift reply @vblagoje !
Oh my bad! I misunderstood top_k value as to yield top_k negative passages for a single positive passage (how DPR train data looks). I guess, this solves my problem. I can now just train the retriever for few epochs and evaluate how good or bad the retriever is if I am not wrong?

[vblagoje]

You are welcome. It should work, but your number of documents is limited - you just used 100. One needs tens of thousands of documents to produce enough training data for adaptation :-)

[sinchanabhat]

Got it! Thanks again @vblagoje !

[vblagoje]

You are welcome @sinchanabhat , anytime!