diff --git a/examples/Huggingface_Transformers/README.md b/examples/Huggingface_Transformers/README.md index 0c0679d62c..31d688e058 100644 --- a/examples/Huggingface_Transformers/README.md +++ b/examples/Huggingface_Transformers/README.md @@ -1,6 +1,6 @@ ## Serving Huggingface Transformers using TorchServe -In this example, we show how to serve a fine tuned or off the shelf Transformer model from [huggingface](https://huggingface.co/transformers/index.html) using TorchServe. +In this example, we show how to serve a fine tuned or off the shelf Transformer model from [huggingface](https://huggingface.co/docs/transformers/index) using TorchServe. We use a custom handler, [Transformer_handler.py](https://github.com/pytorch/serve/blob/master/examples/Huggingface_Transformers/Transformer_handler_generalized.py). diff --git a/examples/Huggingface_Transformers/torchscript.md b/examples/Huggingface_Transformers/torchscript.md index 74a01f8c00..42a9e3ba79 100644 --- a/examples/Huggingface_Transformers/torchscript.md +++ b/examples/Huggingface_Transformers/torchscript.md @@ -4,7 +4,7 @@ Torchscript exposes two APIs, script and trace, using any of these APIs, on the regular Pytorch model developed in python, compiles it to Torchscript. The resulted Torchscript can be loaded in a process where there is no Python dependency. The important difference between trace and script APIs, is that trace does not capture parts of the model which has data dependency such as control flow, this is where script is a better choice. -To create Torchscript from Huggingface Transformers, torch.jit.trace() will be used that returns an executable or [`ScriptFunction`](https://pytorch.org/docs/stable/jit.html#torch.jit.ScriptFunction) that will be optimized using just-in-time compilation. We need to provide example inputs, torch.jit.trace, will record the operations performed on all the tensors when running the inputs through the transformer models. This option can be chosen through the setup_config.json by setting *save_mode* : "torchscript". We need to keep this in mind, as torch.jit.trace() record operations on tensors, the size of inputs should be the same both in tracing and when using it for inference, otherwise it will raise an error. Also, there is torchscript flag that needs to be set when setting the configs to load the pretrained models, you can read more about it in this [Huggingface's doc](https://huggingface.co/transformers/torchscript.html). +To create Torchscript from Huggingface Transformers, torch.jit.trace() will be used that returns an executable or [`ScriptFunction`](https://pytorch.org/docs/stable/jit.html#torch.jit.ScriptFunction) that will be optimized using just-in-time compilation. We need to provide example inputs, torch.jit.trace, will record the operations performed on all the tensors when running the inputs through the transformer models. This option can be chosen through the setup_config.json by setting *save_mode* : "torchscript". We need to keep this in mind, as torch.jit.trace() record operations on tensors, the size of inputs should be the same both in tracing and when using it for inference, otherwise it will raise an error. Also, there is torchscript flag that needs to be set when setting the configs to load the pretrained models, you can read more about it in this [Huggingface's doc](https://huggingface.co/docs/transformers/torchscript). Here is how Huggingface transfomers can be converted to Torchscript using the trace API, this has been shown in download_Transformer_models.py as well: @@ -31,4 +31,4 @@ torch-model-archiver --model-name BERTSeqClassification_Torchscript --version 1. ``` -And then follow the remaining instructions in [README.md](README.md) \ No newline at end of file +And then follow the remaining instructions in [README.md](README.md)