What is Heroku? Why Heroku?
Heroku is a platform as a service (PaaS) that enables developers to build, run, and operate applications entirely in the cloud. I am going to use Heroku because it’s free to use for development and testing, easy to deploy (does not require much work), supports many languages by default,
(As I could not find many resources on deploying TensorFlow models to Heroku, please leave comments below if you have trouble and I will help troubleshoot)
Signup & Install heroku-cli
After you have signed up to install heroku-cil, run the following commands to confirm the installation.
> heroku --version
heroku/7.18.3 darwin-x64 node-v10.12.0
# make sure you have logged in to your heroku account
> heroku login
# Output should have:
Logged in as [email protected]
Nice work — now we are going to deploy TensorFlow Serving docker image to Heroku; however, since the default TensorFlow Serving docker image is not optimized to be deployed to Heroku, I have created a Dockerfile:
Run the following code to build the docker image, making sure you are in the same folder as Dockerfile live.
> docker build -t tf-serving-heroku-1.11 .
Let’s run it locally to test it
> docker run -p 8501:8501 -e PORT=8501 -t tf-serving-heroku-1.11
2018-10-27 21:17:47.515120: I tensorflow_serving/model_servers/server.cc:301] Exporting HTTP/REST API at:localhost:8501 ...
Call the model
Now we are ready to call the production-ready model.
Before calling the model, let’s understand how TensorFlow Serving RESTful API actually works:
In our case the request will look like:
Second, the JSON data that is sent to TensorFlow Model Server has to be structured in a very particular way
// (Optional) Serving signature to use.
// If unspecified default serving signature is used.
// Input Tensors in row ("instances") or columnar ("inputs") format.
// A request can have either of them but NOT both.
In our case:
You can see the full documentation about RESTful API Here.
Now, I will show an example how to call our model from python
Image before preprocess:
Image after preprocess:
It will print T-shirt/top