-
Notifications
You must be signed in to change notification settings - Fork 383
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Backend: Check tool parameters generated by model #849
Merged
EugeneLightsOn
merged 13 commits into
main
from
eugene/tlk-2091-add-way-to-check-if-model-returned-parameters-correspond-to
Nov 26, 2024
Merged
Backend: Check tool parameters generated by model #849
EugeneLightsOn
merged 13 commits into
main
from
eugene/tlk-2091-add-way-to-check-if-model-returned-parameters-correspond-to
Nov 26, 2024
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
EugeneLightsOn
requested review from
malexw,
scottmx81,
tianjing-li and
a team
as code owners
November 19, 2024 13:24
EugeneLightsOn
force-pushed
the
eugene/tlk-2091-add-way-to-check-if-model-returned-parameters-correspond-to
branch
from
November 19, 2024 13:27
12b3eec
to
0ac0ac4
Compare
EugeneLightsOn
changed the title
Backend: Check tool parameters generated by model
WIP: DON'T MERGE - Backend: Check tool parameters generated by model
Nov 19, 2024
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #849 +/- ##
==========================================
+ Coverage 79.56% 79.61% +0.04%
==========================================
Files 251 251
Lines 10632 10785 +153
==========================================
+ Hits 8459 8586 +127
- Misses 2173 2199 +26 ☔ View full report in Codecov by Sentry. |
EugeneLightsOn
changed the title
WIP: DON'T MERGE - Backend: Check tool parameters generated by model
Backend: Check tool parameters generated by model
Nov 19, 2024
EugeneLightsOn
changed the title
Backend: Check tool parameters generated by model
WIP: Backend: Check tool parameters generated by model
Nov 19, 2024
…if-model-returned-parameters-correspond-to
EugeneLightsOn
temporarily deployed
to
development
November 20, 2024 12:00 — with
GitHub Actions
Inactive
EugeneLightsOn
changed the title
WIP: Backend: Check tool parameters generated by model
Backend: Check tool parameters generated by model
Nov 20, 2024
…if-model-returned-parameters-correspond-to
EugeneLightsOn
temporarily deployed
to
development
November 22, 2024 14:30 — with
GitHub Actions
Inactive
…if-model-returned-parameters-correspond-to # Conflicts: # src/backend/tests/unit/chat/test_tool_calls.py # src/backend/tools/base.py
EugeneLightsOn
temporarily deployed
to
development
November 22, 2024 18:47 — with
GitHub Actions
Inactive
tianjing-li
reviewed
Nov 26, 2024
tianjing-li
reviewed
Nov 26, 2024
tianjing-li
reviewed
Nov 26, 2024
tianjing-li
reviewed
Nov 26, 2024
tianjing-li
approved these changes
Nov 26, 2024
EugeneLightsOn
temporarily deployed
to
development
November 26, 2024 21:13 — with
GitHub Actions
Inactive
EugeneLightsOn
deleted the
eugene/tlk-2091-add-way-to-check-if-model-returned-parameters-correspond-to
branch
November 26, 2024 21:24
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The metaclass of the tool's base class decorates the call method of the tools to validate the passed parameters.
AI Description
This pull request introduces a new
ParametersCheckingMeta
metaclass to theBaseTool
class, which automatically decorates thecall
method with a parameter checker. This checker validates the parameters passed to thecall
method against the tool's parameter definitions, ensuring that required parameters are provided and that the types of the parameters match the expected types.The parameter checker is implemented using two new functions:
check_type
andcheck_tool_parameters
. Thecheck_type
function recursively checks the type of a value against a given type description, while thecheck_tool_parameters
function decorates thecall
method and performs the actual parameter validation.The
ParametersCheckingMeta
metaclass is used as the metaclass for theBaseTool
class, ensuring that all subclasses ofBaseTool
have theircall
method decorated with the parameter checker.check_type
function is added to check the type of a value against a given type description.check_tool_parameters
function is added to decorate thecall
method and perform parameter validation.ParametersCheckingMeta
metaclass is introduced to automatically decorate thecall
method with the parameter checker.BaseTool
class is modified to use theParametersCheckingMeta
metaclass.ToolDefinition
class is updated to include aget_tool_definition
method, which returns aToolDefinition
object.mock_get_available_tools
function is modified to use theget_tool_definition
method to get the tool definition for a given tool.test_async_call_tools_success
,test_async_call_tools_failure
,test_async_call_tools_timeout
, andtest_async_call_tools_failure_and_success
functions are updated to use theget_tool_definition
method to get the tool definition for theMockCalculator
andMockWebScrape
tools.test_tools_params_checker_invalid_param
,test_tools_params_checker_invalid_param_type
, andtest_tools_params_checker_required_param_empty
. These functions test the parameter checker by passing invalid parameters to thecall
method and asserting that the expected errors are raised.