Improve the accuracy of generator/prompt node/agent #5603
-
|
I want to create an project prototype that can let me ask the system question and the system will give me the answer based on the document i write into document store. The generator/prompt node generate quite incorrect answer while the retriever retrieved totally correct document. So as the agent. How can i improve the accuracy of generator? I have tried several method such as choose different method. Is fine tuning help this things？ how should i fine tunning this? I use google flan large so the main task is text2text generation right? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 5 replies
-
|
Hi @zh0613! Have you tried using other models? We have made much better experiences with OpenAI's models like gpt-3.5-turbo and gpt-4. If you want to use an open source model, I'd suggest trying out llama v2, this should also give better results than the flan-t5 models. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks! @bogdankostic . For this simple chatbot, should i use agent or just generative pipeline enough? As i follow the tutorial in hugging face, the result generated by the agent seems like its own though instead on genereted from the document |
Beta Was this translation helpful? Give feedback.
-
|
It depends a bit if the user can ask follow up questions or if the conversation is ended after the first question. For the first case, you probably need some memory to store the conversation state and it makes sense to use an Agent. For this, the tutorial on building a conversational chat app and customizing agents to chat with your agents might be interesting for you. |
Beta Was this translation helpful? Give feedback.
-
|
Ow! Okay. If possible can i ask the prompt node, can retrieve only single answer only? like what is black box testing. It can retrieve black box testing is the testing method.... But when i ask the rules of testing. The document contain 10 rules of testing, but it can only retrieve one rules. So is this possible to retrieve 10 rules or not? Thanks! |
Beta Was this translation helpful? Give feedback.
-
|
This depends on the format of your documents, if the 10 rules are listed in a single Document that can fit inside the input limits of a PromptNode, it should be feasible to get the 10 rules back as an asnwer. |
Beta Was this translation helpful? Give feedback.
-
|
Understandable. Thank you for your helping @bogdankostic |
Beta Was this translation helpful? Give feedback.
This depends on the format of your documents, if the 10 rules are listed in a single Document that can fit inside the input limits of a PromptNode, it should be feasible to get the 10 rules back as an asnwer.