Lower default max_labeled_demos
in BootstrapFewShot
#358
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hey team, I think it would be good to lower the default
max_labeled_demos
from 16 to 8. I have been playing with answering FAQs with 16 and it quickly results in compiling a massive prompt such that I exceed the max token limit. There is probably another way around this -- like ofc setting the argument to 8 or 4 or whatever, but I think this might slightly improve the UX for beginners starting with QA RAG andGPT-3.5-Turbo
/ ~4K input length LLMs.