Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve model score #119

Open
connorjoleary opened this issue Jan 28, 2022 · 0 comments
Open

Improve model score #119

connorjoleary opened this issue Jan 28, 2022 · 0 comments
Assignees

Comments

@connorjoleary
Copy link
Owner

Is your feature request related to a problem? Please describe.
Currently the model takes the similarity of each block of text to the parent. Then after every block is computed, it multiplies this score by it's similarity to the original claim. This algorithm leads to two problems:

  • The claims which are more dissimilar, but further down in the tree don't get scored highly. We should take into account how far down the source is an if it is very far down increase the score.
  • The algorithm can have runaway branches where it doesn't realize how far off of the claim it is going, because it only looks at the similarity to the parent.

Describe the solution you'd like
A model which calculates the score for each layer and takes into account the similarity to the original claim (not just at the end) and weights deeper nodes as better.

Describe alternatives you've considered
Potentially this is where we could introduce a simple ml model which creates these scores based on the data we have labeled, but this seems like overkill at this point.

Additional context
Using a Jupyter notebook to rerun data would be a good way to test. Then you can see if the labeled data is getting higher or lower scores.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant