Question about generative model notebook

[hyunwoongko]

https://github.com/ELS-RD/transformer-deploy/blob/main/demo/generative-model/gpt2.ipynb

In this notebook, when you tested cache feature, I think you should use generate function rather than forward function because the advantages of cache can be obtained with saving past generated tensors. but if we use forward function there are no past tensors. Please let me know if I was wrong.

[pommedeterresautee]

I would check the T5 notebook for the caching feature with generative models.
GPT2 one needs some rework on that topic.