Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Multiple models support for LLM TGI #835

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

Multiple models support for LLM TGI #835

wants to merge 7 commits into from
+95 −25

Commits on Oct 29, 2024

  1. Update gateway and docarray from mega and proto services to have mode… 

    …l field for ChatQnAGateway and LLMParams respectively
    @sgurunat
    sgurunat committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    11f378f View commit details
    Browse the repository at this point in the history

  2. Add load_model_configs method in utils.py to validate and load the mo… 

    …del_configs
    @sgurunat
    sgurunat committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    3cd528c View commit details
    Browse the repository at this point in the history

  3. Update llms text-generation tgi file (llm.py) to support multiple mod… 

    …els. Uses load_model_configs method from utils
    @sgurunat
    sgurunat committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    d9e5a32 View commit details
    Browse the repository at this point in the history

  4. Update llms text-generation tgi template to add different templates f… 

    …or different models
    @sgurunat
    sgurunat committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    c3bc176 View commit details
    Browse the repository at this point in the history

Commits on Oct 30, 2024

  1. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Oct 30, 2024
    Configuration menu
    Copy the full SHA
    310201a View commit details
    Browse the repository at this point in the history

  2. fixed llm_endpoint empty string issue on error scenario 

    Signed-off-by: sgurunat <[email protected]>
    @sgurunat
    sgurunat committed Oct 30, 2024
    Configuration menu
    Copy the full SHA
    15aafbc View commit details
    Browse the repository at this point in the history

Commits on Nov 4, 2024

  1. Function to get llm_endpoint and keep the code clean 

    Signed-off-by: sgurunat <[email protected]>
    @sgurunat
    sgurunat committed Nov 4, 2024
    Configuration menu
    Copy the full SHA
    9b7deaf View commit details
    Browse the repository at this point in the history