PromptNode with LLM from EC2 instance

[siljuovix]

Is it possible to use PromptNode with an open source LLM from HF (TheBloke/Mistral-7B-Instruct-v0.1-AWQ) running on an EC2 instance using an API ?
Currently the workflow for querying the model looks something like this :

prompt = [INST] Why do airplanes leave contrails in the sky? [/INST]
generated_response = model_response(prompt, temperature=0.1, top_p=0.95, top_k=60, max_new_tokens=512)

model_response does a POST request to the API and gets the text back.

if it is, how can one initialize the PromptNode?

Thank you