Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Evaluating needle in a haystack the responses are abnormal #194

Open
hshc123 opened this issue Jan 19, 2025 · 0 comments
Open

Evaluating needle in a haystack the responses are abnormal #194

hshc123 opened this issue Jan 19, 2025 · 0 comments

Comments

@hshc123
Copy link

hshc123 commented Jan 19, 2025

Hello, Author. When I changed the question in the "searching for a needle in the haystack" evaluation from one about the needle to a different question (for example, "please describe this video"), the LLM (Large Language Model) outputs were all garbled characters. Only when the query pertains to the needle does the output come out as expected. What could be causing this issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant