Everything is ready, and your model is shining, but now you want to share it.
In this article you’re going to learn how to:
- create an API using the FastAPI framework for your model
- containerize your API using Docker
- deploy your containerized API on a Cloud provider or a VPS
Let’s dive in!
BTW: if you do not have any deep learning model you want to share, you can read the PART 1 article :
How to build a simple API
Warning: This is a simple API for demonstrating purpose, do NOT use this in a production environment! You should add some security walls (https, set up an access token…)
This is an API for a text-generative model, it may not be suitable for other types of models. You must put exactly what you need to run predictions on a local setup. For a classification model, you would need a csv/json with the labels for example.
Build our container
To build a docker container, you will need a folder with these files :
- Your API
- Your model file (.pt, .pth…)
- a .env file
- a Dockerfile (without extension)
- a requirements.txt file with ALL of the modules you need
Dockerfile
Here is the Dockerfile for the project:
FROM python:3.8-slim
: This line indicates that the base image for the Docker container will be Python version 3.8, specifically the slim version of this image which includes the minimum needed to run Python (i.e., the Python runtime and some basic dependencies).WORKDIR /app
: This command sets the working directory in the Docker container to/app
. Any commands that follow in the Dockerfile (that aren't prefaced with an absolute path) will be run in this directory.COPY . .
: This command copies the contents of the current directory on the host machine (i.e., the machine building the Docker image) into the current directory in the Docker image (which, as mentioned above, has been set to/app
).RUN pip3 --no-cache-dir install torch==1.13.1
: This is a command that will be run during the build of the Docker image. In this case, it's using pip (Python's package installer) to install the PyTorch library at a specific version. The--no-cache-dir
option prevents pip from caching installed packages, which can help to reduce the size of the image. This is because the torch module is heavy and can make crash the image buildRUN pip3 install -r requirements.txt
: This line also runs a command during the build of the Docker image, again using pip to install Python packages. However, this time, it's installing all the packages listed in arequirements.txt
file (which was presumably copied over in the previousCOPY . .
command).ENV MODEL_FILE=model.pth
: This line sets an environment variable namedMODEL_FILE
within the Docker container and assigns it the valuemodel.pth
. This could be a model file that your application needs to operate and presumably is in the directory copied over earlier.EXPOSE 8000:8000
: This tells Docker that the application inside the Docker container will be listening on port 8000 and that port should be mapped to port 8000 on the host machine. It's a way of enabling communication with the outside world.ENTRYPOINT ["uvicorn"]
: This sets the default application that Docker will run when the container starts up. In this case, it's the Uvicorn server, a lightning-fast ASGI server, which is often used for Python web applications (like those built with the FastAPI or Starlette frameworks).CMD ["api:app", "--host", "0.0.0.0"]
: This line sets the default arguments that will be passed to the application specified in theENTRYPOINT
. In this case, it's telling Uvicorn to run the application found inapi:app
(which means theapp
object in theapi
module) and to listen on all IP addresses (0.0.0.0
). The CMD can be overridden by providing arguments after the image name when starting a container.
.env file
For every env line you have on your Dockerfile, you will need to create a line in your .env file. For example, here we only need to provide our model path:
MODEL_FILE=path_to_finetuned_model.pt
requirements.txt file
All the modules needed in your Python file :
We do not provide the torch module as we’ve installed it previously
And… that’s it! We are ready to build our Docker image!
Build the docker image
This is it. If you’re using a Windows device, good luck. Docker desktop installation on Windows is a MESS (Yes I still have nightmares) and it crashes a LOT. A pleasure :)
Push on a VPS
I took the lowest configuration I could find (6$ per month) :
- Debian 10 (because docker was pre-installed on it)
- 1 vCore
- 2 Go
- 20 Go SSD SATA
Then, after the initial setup, you have to send your files via STP or SSH and you’re good to build!
After building your image and running your container, you should be able to access your API by making a call to http://your-vps-ip/8000/your_endpoint like this :
import requests
def query_api():
# The API endpoint URL
url = "http://your-vps-ip/8000/your-endpoint"
# Send a GET request to the API
response = requests.get(url)
# Check if the request was successful
if response.status_code == 200:
# Parse the JSON response
data = response.json()
print(data)
else:
print(f"Failed to query API. Status code: {response.status_code}")
if __name__ == "__main__":
query_api()
Perfect! You are now able to query your API from everywhere in the world! Do not forget to secure it!
Cloud Provider
Scaleway is a cloud provider (Like AWS or GCP). It provides a lot of interesting services at affordable cost. The servers are mainly located in France, so it’s a good solution if you are located in the EU.
For this tutorial, we are going to use the serverless services :
https://www.scaleway.com/en/pricing/?tags=serverless
Serverless means that you are able to run functions, and apps without having to manage servers.
There are two distinct services :
- Serverless Functions: this makes you able to run functions in the cloud. For example, you can run a Python function that triggers your smart lamp every morning without having your computer on
- Serverless containers: to run your Docker Containers in the cloud
This time, it’s easier, you just have to push your docker image into Scaleway. To do this:
- Create a namespace into “Container Registry” from the “Containers” section
- Use the namespace endpoint to push your container from docker Hub to Scaleway
BTW: do not forget to add all of your env variables into the SECRET Scaleway section!
Finally, you can run your container in the Cloud using the serverless container service, and voilà!
Thank you so much for reading! Now you should be able to deploy your first projects, this is a first step in the wide MLops world!
Do not hesitate to ask questions, I hope this article will help you in your Deep Learning journey!