Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[EAGLE-3887] model uploadv2 cli #269

Merged
merged 9 commits into from
Jan 23, 2024

Conversation

phatvo9
Copy link
Contributor

@phatvo9 phatvo9 commented Jan 12, 2024

New cli interface

$ clarifai -h
 upload              Upload Clarifai model
 create              Create component of Clarifai platform
 login               Login to Clarifai and save PAT locally
 example             Download/List examples of model upload

Usage:

  1. Login: Persist PAT locally at CLARIFAI_HOME/pat
$ clarifai login
Get your PAT from https://clarifai.com/settings/security and pass it here: <your pat>
  1. Create model
# from scratch
$ clarifai create model --type <model-type> --working-dir <dir> ...
# from an example, then pick it using terminal selection
$ clarifai create model --from-example
? Select an example:
❯ multimodal_embedder/clip
  text_classifier/xlm-roberta
  text_embedder/instructor-xl
  ...

NOTE: if working-dir exists, need to set --overwrite flag otherwise an error arises
3. See available examples

$ clarifai example list 
Found 11 examples
 * multimodal_embedder/clip
 * text_classifier/xlm-roberta
 * text_embedder/instructor-xl
  ....
  1. Upload.

By default It will run test before uploading, looking for test.py in current directory or you can specify the test file with test-path arg. To upload without test, set no-test flag.

upload without config file

$ clarifai upload model --url <s3 or http> --type <model-type>  --id <model-id> --user-app <user-id>/<app-id> ...

upload with config file (WIP)

$ clarifai upload model --url ... --config <path-to-config.yaml>

@phatvo9 phatvo9 requested review from zeiler and ackizilkale January 12, 2024 07:46
@phatvo9 phatvo9 marked this pull request as ready for review January 22, 2024 04:24
"--infer-param",
required=False,
default="",
help="Path to json file contains inference parameters")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are we going to use json or yaml?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh this one is json I guess.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll use .yaml, but this one is for inference_paramters used when user input args one by one (like previous). User also can use only .yaml with all args provided in it.

@phatvo9 phatvo9 merged commit 89ae3ed into model_upload_v2 Jan 23, 2024
1 of 16 checks passed
@phatvo9 phatvo9 deleted the EAGLE-3887-model-uploadv2-cli branch January 23, 2024 04:15
@phatvo9 phatvo9 mentioned this pull request Feb 5, 2024
phatvo9 added a commit that referenced this pull request Feb 7, 2024
* [EAGLE-3887] model uploadv2 cli (#269)

* update upload/init cli

* add login,example + update create

* implement upload cli

* minor fix: download when list example

* update help of login

* force to test before uploading

* upload cli: upload -> upload model

* add info after model created

* addressed comments

---------

Co-authored-by: phtvo <[email protected]>

* [EAGLE-3915]- refactor model uploadv2 repository (#276)

* refactor inference.py + config

* add init repo

* fix init repo

* update type check msg

* fix triton list input + load user cfg

* resolve merge

* update init cli

* format triton output cfg, add build module, build repo

* fix missing model.py

* always check image_shape

* Update triton_config docstring

* exclude pycache

* typos

* check in and out len

* check all cfg before gen files

* fix docstring

* fix typos

---------

Co-authored-by: phtvo <[email protected]>

* fix typos

* [EAGLE-3916] build cli (#278)

* add build cli

* checking clarifai config

* add desc attr

* update upload cli

* typos

* rm unused

* check in and out len

* check all cfg before gen files

* fix docstring

* fix typos

* typo

---------

Co-authored-by: phtvo <[email protected]>

* [EAGLE-3917] - model uploadv2 user test (#279)

* add default tests + fix parse user cfg

* force using np in output

* fix input type of multimodal emb

* fix default test for multimodal

* fix parse local cfg

* enable test when building

* typo

---------

Co-authored-by: phtvo <[email protected]>

* [EAGLE-3918]-module tests (#285)

* update tests + fix import inference in test

* typos

* not set image_shape for non-image input

* module tests

* constant for tensor names

* correct type,value of infer params

---------

Co-authored-by: phtvo <[email protected]>

* fix upload

* disallow using test_path

* postprocess det

* fix load examples

* rm unused args

* ensure using list or tuple for labels

* fix force download example

* [EAGLE-3932] model uploadv2 doc (#287)

* update doc

* minor update

* update rm unused upload args

---------

Co-authored-by: phtvo <[email protected]>
@sanjaychelliah sanjaychelliah mentioned this pull request Feb 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants