Question regarding an inference server

[michaelmohamed]

Hi, is there an inference server available that implements sahi?

Regards,
M

[fcakyon]

Hello @mm-e1, thanks for your question. What do you mean by inference server? You mean dockerfile/docker image/web app? Can you be more specific?

[michaelmohamed]

Hi, yes.

I have a use case where I would like to inference using a sahi service in:

1. a batch mode - where I would like to process a lot of images in a folder. it would be ideal if i could use multiprocessing to bring up multiple instances of the model to inference the batch faster.
2. transactional mode from a web app. it would be ideal if i could interface my web app over rest or grpc. similar to NVIDIA Triton Inference Server or Tensorflow Serving

Is there a sahi inference service that exists that can do either?

[devrimcavusoglu]

Hi @mm-e1, there is no such web inference service for sahi officialy (do not know any third party). We have HF Spaces for demonstration purposes and you can upload your image by drag & drop there and see the result; however, I don't think it can process your potentially batch inference (usually the request/min is limited). As you may've presumed such services are costly (both in terms of pricing and also computationally).

Having said that, you probably need to create your own REST like API or integrate sahi's inference pipeline to your worker/project on some resource (which would probably be a bit costly), and since you develop it, you can either have (1) or (2). Apart from integrating sahi's pipeline, you can directly use the CLI documented here.