re-add GritLM-7B-noinstruct

[Muennighoff]

It seems like the GritLM__GritLM-7B-noinstruct results present here https://github.com/embeddings-benchmark/results/tree/319e81d64298f3bdc5f09c8b6ff91f5015b3664c/results/GritLM-7B-noinstruct are no longer present here https://github.com/embeddings-benchmark/results/tree/9a79f7e07542ad2f5cb47490fa1e5ac2ba57d7a8/results; probably they were accidentally deleted so this PR adds them back in & also merges the separate GritLM folders; Currently the regular GritLM-7B is missing from the RAR-b leaderboard - this PR should fix it.

[Samoed]

We've discussed why it was removed #25 (comment)

[Muennighoff]

I see; I guess we should add it back in but maybe make sure model_meta is different then?

[KennethEnevoldsen]

With the name fix, I think we can add it in. Tests also still fail

[Muennighoff]

Test is failing; I think it is because it is looking for where exists = PosixPath('/home/runner/work/results/results/results/GritLM__GritLM-7B/no_revision_available/model_meta.json').exists? Is the model_meta required even for no_revision_available folders? I think I can just move the results from that folder to the 13f00a0e36500c80ce12870ea513846a066004af folder as I doubt the model was different

[Samoed]

I think that you can change fixture for test, because no_revision_available this is old result and GritLM versions are already presented here

results/tests/test_correct_folder_structure.py

Line 36 in b7f3700

folders_without_meta = [ # please do not add to this list it is only intended for backwards compatibility. Future results should have a model_meta.json file

or merge folders if results are the same or have different datasets

[Muennighoff]

Makes sense, thanks!

It seems like there is some difference in FollowIR results e.g. https://github.com/embeddings-benchmark/results/blob/standardizegrit/results/GritLM__GritLM-7B/no_revision_available/Robust04InstructionRetrieval.json vs https://github.com/embeddings-benchmark/results/blob/standardizegrit/results/GritLM__GritLM-7B/13f00a0e36500c80ce12870ea513846a066004af/Robust04InstructionRetrieval.json & same for others. I think it is because I reran all MTEB tasks including them with GritLM.
Should we update to the results from rerunning (in the folder with a commit id) @orionw? It seems like the FollowIR scores are higher after rerunning for all three tasks; Is it because of a difference in prompting?

[orionw]

It could be, perhaps I did something wrong to the formatting. As long as the instructions were correctly passed in and it looks like it would be unless there was a prompt conflict then it should be fine.

I'm going to take a closer look at the instruction setup next week so I'm good with keeping the newer results!