GPT-J support

[JonathanLehner]

How can we use GPT-J for inference?

[pommedeterresautee]

Work to support of GPT-2/T5 architectures just started (right now it's focused on encoder only models). GPT-J follows GPT-2 architecture so hopefully it should be ready in the next few weeks.

[oborchers]

@pommedeterresautee: I caution that this is probably more difficult as expected due to (currently trying to figure that out):
onnx/onnx-tensorrt#818 and maybe even also https://forums.developer.nvidia.com/t/how-to-convert-a-large-pytorch-model-to-trt-model/182782/4

[pommedeterresautee]

@oborchers Indeed, master branch of ORT has currently a bug with external data. Issue opened, someone from Msft is on it.

[YarrDOpanas]

Hi, everyone! @pommedeterresautee , is there any updates? Can't convert large models like gpt-j, GPT Neo 2.7B , while gpt-2 and openai-gpt works correctly.

[pommedeterresautee]

Right now we focus more on Kernl / Triton, and plan (not now) to merge the 2 libs.