feat: websearch, tool use, user intent, dynamic retrieval, multiple questions

[jacopo-chevallard]

Description

This PR includes far too many new features:

• detection of user intent (closes CORE-211)
• treating multiple questions in parallel (closes CORE-212)
• using the chat history when answering a question (closes CORE-213)
• filtering of retrieved chunks by relevance threshold (closes CORE-217)
• dynamic retrieval of chunks (closes CORE-218)
• enabling web search via Tavily (closes CORE-220)
• enabling agent / assistant to activate tools when relevant to complete the user task (closes CORE-224)

Also closes CORE-205

Checklist before requesting a review

Please delete options that are not relevant.

• My code follows the style guidelines of this project
• I have performed a self-review of my code
• I have commented hard-to-understand areas
• I have ideally added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged

Screenshots (if appropriate):

[linear]

CORE-211 Detect user intent to modify the system prompt

CORE-212 Treat multiple questions in a single user input

CORE-213 Using chat history when answering user questions in RAG mode

CORE-217 Enable filtering reranked chunks by relevance threshold

CORE-218 Enable retrieval of a dynamic number of chunks

CORE-220 Enabling web search via tavily

CORE-224 Enable system to activate / deactivate nodes (and tools)

CORE-205 Simplify the code - remove sync functions

[jacopo-chevallard]

@AmineDiro @StanGirard all tests are passing by now

[AmineDiro]

Nice work on the PR 💯 !

Here are some feedback on the design and current architecture of the classes:

• I think that the QuivrQARAGLangGraph should get leaner: it has method that don't directly operate on self. It should offload to some node and only hold a the graph itself: how to build the graph, how to run the graph , how to print the graph ....
• The state inside the graph should be typed. Coming from the outside It is very difficult to debug the state inside a node. Currently the state is passed as dict between nodes.
  One approach would be to use the TypeState pattern : The Object state can be generic over the some transitionStatte Type State<T> where T. Then on each transition we could have a new object State than is correctly typed and holds a list of predetermined attributes that are clear to the IDE, debugger etc ...
• The tools interface should be cleanup a bit. IMO the registry pattern is the correct abstraction here but I think that some abstraction like ToolWrapper, Tools, create_tool_... are not necessary and could be simplified. We should have a common interface over Tools as some abstract class and then build each tool class internal to quiver as its own object that can be added to the registry if its key matches the config key.
• The prompts interface should probably adhere to the registry pattern.