Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add test dataset for trec tip of the tongue dataset #249

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

mam10eks
Copy link
Contributor

The test queries for the trec tot task are now available, and this pull request adds them in a similar way to how they are added for dev and train (no qrels are available yet, we can also wait to merge this until the qrels are available).

@seanmacavaney
Copy link
Collaborator

Thanks! A few points:

  • Can we rename it to trec-tot/2023/test? This is more aligned with the typical naming convention.
  • Are the sentence_annotations of a standard structure? If so, they should be represented as NamedTuples instead of dicts.
  • Can you add a metadata record and documentation for this dataset? (This is why the tests failed.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants