deploying a Keras ResNet model with Flask and Docker for subsequent monitoring in production
serve a Keras model as a Flask API containerize a Flask model
- Expose a ResNet image predictor app built with Keras and Flask with
python run_keras_server.py - Make requests to the app with
python simple_requests.py - Build a Docker container containing the app with
docker build -t flask-sample-one:latest . - Run the Docker container with
docker run -d -p 5000:5000 flask-sample-one
- don't forget to expose the Flask App by setting
host = '0.0.0.0'
if __name__ == '__main__': app.run(debug=True,host='0.0.0.0')
- Although the Dockerfile uses Ubuntu as a base image, for production purposes using a WSGI image might be a better choice.
- The Dockerfile uses
pip install -r requirements.txtto install the needed Python packages. There seems to be issues around usingconda installwith Anaconda images atm.
The next thing to do after containerizing our app would be to monitor it in production. While software developers have many ways of checking that their app functions well (monitoring latency, number of requests per minute etc.), machine learning engineers are still experimenting with tooling and paradigms. Some good projects to look at include:
These cover topics like building Runtime Graphs, A/B testing, Multi-Arm Bandits, Feature Normalisation, Concept Drift and Low-latency serving.
this information is up-to-date as of 10th July 2018