[AGENT-6788] Prompt could be in the form of list #1285
Conversation
For chat api, the prompt could be in the form of list. Make sure to serialize it before reporting for monitoring
|
The Needs Review labels were added based on the following file changes. Team @datarobot/core-modeling (#core-modeling) was assigned because of changes in files:custom_model_runner/datarobot_drum/drum/language_predictors/base_language_predictor.py Team @datarobot/custom-models (#custom-models) was assigned because of changes in files:custom_model_runner/datarobot_drum/drum/language_predictors/base_language_predictor.py tests/unit/datarobot_drum/drum/language_predictors/test_base_language_predictor.py If you think that there are some issues with ownership, please discuss with C&A domain at #sdtk slack channel and create PR to update DRCODEOWNERS\CODEOWNERS file. |
| concatenated_prompt.append(message) | ||
| latest_message = "\n".join(concatenated_prompt) | ||
| else: | ||
| logger.warning(f"Unhandled prompt type: {type(prompt_content)}") |
There was a problem hiding this comment.
Should we also consider sending an MLOps event because this message will only be visible in the model log. This probably requires more thought (as would we spam our platform with events for a poorly configured model?)
There was a problem hiding this comment.
Yeah, it needs more thought.
I think sending event will be little too expensive - because in case of series of wrong prompts - it will fill up events table. This is model monitoring not able to understand the type and not the model. So, if model is misconfigured, user would know the error right away. In case of monitoring - we can try to deduplicate the events and send only 1 event for 1 unhandled type. I will create an improvement ticket but not sure if we will be able to work on it by 11.0
|
Label Needs Review: Custom Models was removed because @yakov-g is part of Custom Models domain. |
For chat api, the prompt could be in the form of list. Make sure to serialize it before reporting for monitoring
This repository is public. Do not put here any private DataRobot or customer's data: code, datasets, model artifacts, .etc.
Summary
Rationale