Allow custom OpenAI ENDPOINT & Model Names - e.g. Sambanova API Cloud

[soufianechami]

I am trying to use Sambanova API which is compatible with OpenAI SDK. Would love to see how fast inference can benefit here and also use open source LLMs.

Below is an example of Sambanova API call:

import os
import openai

client = openai.OpenAI(
    api_key=os.environ.get("SAMBANOVA_API_KEY"),
    base_url="https://api.sambanova.ai/v1",
)

response = client.chat.completions.create(
    model="Meta-Llama-3.1-8B-Instruct",
    messages=[{"role":"system","content":"You are a helpful assistant"},{"role":"user","content":"Hello"}],
    temperature=0.1,
    top_p=0.1
)

print(response.choices[0].message.content)

[Chenglong-MS]

@soufianechami This should be supported now, using

provider="openai"
api_key="xxxxxxx"
api_base="https://api.sambanova.ai/v1"
model="Meta-Llama-3.1-8B-Instruct"

Thanks!

[soufianechami]

Thanks @Chenglong-MS - But user still can't do this via the GUI. I am unable to edit the ednpoint link. Check below:

[Chenglong-MS]

You should update data-formulator with pip install —upgrade data-formulator! :)

…

________________________________ From: Soufiane Chami ***@***.***> Sent: Monday, February 24, 2025 4:30:13 AM To: microsoft/data-formulator ***@***.***> Cc: Mention ***@***.***>; Comment ***@***.***>; Subscribed ***@***.***> Subject: Re: [microsoft/data-formulator] Allow custom OpenAI ENDPOINT & Model Names - e.g. Sambanova API Cloud (Issue #63) Thanks @Chenglong-MS<https://github.com/Chenglong-MS> - But user still can't do this via the GUI. I am unable to edit the ednpoint link. Check below: Screenshot.2025-02-24.at.3.29.07.PM.png (view on web)<https://github.com/user-attachments/assets/f7df05a7-7b58-4075-a06b-255ced24570a> — Reply to this email directly, view it on GitHub<#63 (comment)> or unsubscribe<https://github.com/notifications/unsubscribe-auth/AWJXEPBT7NQDIKESSDBMEZ32RL7ELBFKMF2HI4TJMJ2XIZLTSOBKK5TBNR2WLJDUOJ2WLJDOMFWWLO3UNBZGKYLEL5YGC4TUNFRWS4DBNZ2F6YLDORUXM2LUPGBKK5TBNR2WLJDUOJ2WLJDOMFWWLLTXMF2GG2C7MFRXI2LWNF2HTAVFOZQWY5LFUVUXG43VMWSG4YLNMWVXI2DSMVQWIX3UPFYGLLDTOVRGUZLDORPXI6LQMWWES43TOVSUG33NNVSW45FGORXXA2LDOOJIFJDUPFYGLKTSMVYG643JORXXE6NFOZQWY5LFVE4DCMRQG43TMMRRQKSHI6LQMWSWS43TOVS2K5TBNR2WLKRSHA2DGOBVGEZTANNHORZGSZ3HMVZKMY3SMVQXIZI>. You are receiving this email because you were mentioned. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.