Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

how to call the remote api #62

Open
jackqdldd opened this issue Sep 18, 2024 · 0 comments
Open

how to call the remote api #62

jackqdldd opened this issue Sep 18, 2024 · 0 comments

Comments

@jackqdldd
Copy link

jackqdldd commented Sep 18, 2024

The remote server directory:
7a3f1c5610cc98b06ecae5ebdad460c8

The request url is like: http://10.10.10.10:40105/v1/chat/completions

Then I user the command:guidellm \ --target "http://10.10.10.10:40035/v1" \ --model "MiniCPM3-4B" \ --data-type emulated \ --data "prompt_tokens=512,generated_tokens=128" \ --rate-type sweep --rate 2 --max-requests 2

I got the error: Max retries exceeded with url: /MiniCPM3-4B/resolve/main/config.json (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f1e0daaad70>, 'Connection to huggingface.co timed out. (connect timeout=10)'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant