You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
For this, we need to institute more metrics (tracked in #18 )
Explicit feedback
Thumbs up/ thumbs down. But this is rare.
Implicit feedback
Inferred from user behavior, such as response time, follow-up questions, or even actions like copying and pasting responses. Implicit feedback is far more abundant and can provide valuable insights into how well the LLM is performing
how would you want the feedback to be stored, do you want a feeder pipeline, that feeds back into the evaluation dataset and then based on that each subsequent prompt answer is influenced by these metrics
do you also want the thumbs up and thumbs down feature to be implemented, I believe the functionality would be similar to how LLMs like chatGPT take the feedback, like once about 100 prompts maybe?
If those are the requirements can I work on this issue?
Think about an automatic mechanism in which we can use the feedback we get from the users back in our evaluation dataset.
Use it as part of our dataset ground truth (good or bad )
Tasks
For this, we need to institute more metrics (tracked in #18 )
Explicit feedback
Implicit feedback
The text was updated successfully, but these errors were encountered: