Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add odinson and gilda grounding for spine #145

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

svempati21
Copy link

@svempati21 svempati21 commented Oct 19, 2023

@cthoyt
Copy link
Member

cthoyt commented Oct 24, 2023

Next steps:

  1. Create a 4 column spreadsheet (e.g., with annotations from SciWheel) with example sentences, subject, predicate, and object that will serve as a "benchmark" corpus. This probably needs 20-30 examples to start
  2. We're going to run the workflow on these sentences and make a comparison of how often they get the right answer (based on the human curated benchmark). We can then calculate the accuracy (and other metrics) based on that.
  3. (optional) If we want to use metrics that incorporate false positives, we can also include 20-30 random strings from the same corpus but that don't have any relations that should get extracted

@bgyori
Copy link
Member

bgyori commented Nov 15, 2023

I just noticed - this work doesn't really belong in this repository. It should be in the https://github.com/gyorilab/indra_spine repository since that's where this line of work has been developed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants