Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Invalid model_id for J-350M and T5P-220M #857

Closed
praktiskt opened this issue Nov 21, 2023 · 2 comments
Closed

Invalid model_id for J-350M and T5P-220M #857

praktiskt opened this issue Nov 21, 2023 · 2 comments
Labels
question Further information is requested

Comments

@praktiskt
Copy link
Contributor

praktiskt commented Nov 21, 2023

Describe the bug

Unable to run some of the models available on TabbyML huggingface.

Reproducible examples:

# TabbyML/J-350M fails
docker run tabbyml/tabby:0.5.5 serve --model TabbyML/J-350M

# TabbyML/T5P-220M fails
docker run tabbyml/tabby:0.5.5 serve --model TabbyML/T5P-220M

# TabbyML/StarCoder-1B works just fine
docker run tabbyml/tabby:0.5.5 serve --model TabbyML/StarCoder-1B

Information about your version

Tabby 0.5.5, using the docker-container tabbyml/tabby:0.5.5.

Information about your GPU

No GPU, just CPU.

Additional context

Setting RUST_BACKTRACE=1 yields the following stacktrace for T5P-220M. J-350M yields an identical error.

thread 'main' panicked at crates/tabby-common/src/registry.rs:72:32:
Invalid model_id <TabbyML/T5P-220M>
stack backtrace:
   0: rust_begin_unwind
             at ./rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:595:5
   1: core::panicking::panic_fmt
             at ./rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/core/src/panicking.rs:67:14
   2: tabby_common::registry::ModelRegistry::get_model_info
   3: tabby_download::download_model::{{closure}}
   4: tabby::serve::main::{{closure}}
   5: tokio::runtime::park::CachedParkThread::block_on
   6: tokio::runtime::context::runtime::enter_runtime
   7: tokio::runtime::runtime::Runtime::block_on
   8: tabby::main
@wsxiaoys
Copy link
Member

Hey - since 0.4.0, we removed T5P-220M / J-350M from supported model list: https://tabby.tabbyml.com/docs/models/

If you're interested in using a customized model in tabby, please refer the How can I convert my own model for use with Tabby? in https://tabby.tabbyml.com/docs/faq

@wsxiaoys wsxiaoys added question Further information is requested and removed bug-unconfirmed labels Nov 22, 2023
@praktiskt
Copy link
Contributor Author

Gotcha. Might make sense to remove the models from Huggingface if they are no longer supported (or flag them as such in their respective READMEs).

Closing since there's nothing else to be done. Thanks for the swift response 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants