How to deploy your Pytorch model: Scaleway Cloud VS VPS

Photo by imgix on Unsplash

Everything is ready, and your model is shining, but now you want to share it.

In this article you’re going to learn how to:

• create an API using the FastAPI framework for your model
• containerize your API using Docker
• deploy your containerized API on a Cloud provider or a VPS

Let’s dive in!

BTW: if you do not have any deep learning model you want to share, you can read the PART 1 article :

PART 1: How to enhance your fine-tuning journey using GPT-4 and BARThez for summarization?

How to build a simple API

Warning: This is a simple API for demonstrating purpose, do NOT use this in a production environment! You should add some security walls (https, set up an access token…)

This is an API for a text-generative model, it may not be suitable for other types of models. You must put exactly what you need to run predictions on a local setup. For a classification model, you would need a csv/json with the labels for example.

Build our container

To build a docker container, you will need a folder with these files :

• Your API
• Your model file (.pt, .pth…)
• a .env file
• a Dockerfile (without extension)
• a requirements.txt file with ALL of the modules you need

Dockerfile

Here is the Dockerfile for the project:

• FROM python:3.8-slim: This line indicates that the base image for the Docker container will be Python version 3.8, specifically the slim version of this image which includes the minimum needed to run Python (i.e., the Python runtime and some basic dependencies).
• WORKDIR /app: This command sets the working directory in the Docker container to /app. Any commands that follow in the Dockerfile (that aren't prefaced with an absolute path) will be run in this directory.
• COPY . .: This command copies the contents of the current directory on the host machine (i.e., the machine building the Docker image) into the current directory in the Docker image (which, as mentioned above, has been set to /app).
• RUN pip3 --no-cache-dir install torch==1.13.1: This is a command that will be run during the build of the Docker image. In this case, it's using pip (Python's package installer) to install the PyTorch library at a specific version. The --no-cache-dir option prevents pip from caching installed packages, which can help to reduce the size of the image. This is because the torch module is heavy and can make crash the image build
• RUN pip3 install -r requirements.txt: This line also runs a command during the build of the Docker image, again using pip to install Python packages. However, this time, it's installing all the packages listed in a requirements.txt file (which was presumably copied over in the previous COPY . . command).
• ENV MODEL_FILE=model.pth: This line sets an environment variable named MODEL_FILE within the Docker container and assigns it the value model.pth. This could be a model file that your application needs to operate and presumably is in the directory copied over earlier.
• EXPOSE 8000:8000: This tells Docker that the application inside the Docker container will be listening on port 8000 and that port should be mapped to port 8000 on the host machine. It's a way of enabling communication with the outside world.
• ENTRYPOINT ["uvicorn"]: This sets the default application that Docker will run when the container starts up. In this case, it's the Uvicorn server, a lightning-fast ASGI server, which is often used for Python web applications (like those built with the FastAPI or Starlette frameworks).
• CMD ["api:app", "--host", "0.0.0.0"]: This line sets the default arguments that will be passed to the application specified in the ENTRYPOINT. In this case, it's telling Uvicorn to run the application found in api:app (which means the app object in the api module) and to listen on all IP addresses (0.0.0.0). The CMD can be overridden by providing arguments after the image name when starting a container.

.env file

For every env line you have on your Dockerfile, you will need to create a line in your .env file. For example, here we only need to provide our model path:

MODEL_FILE=path_to_finetuned_model.pt

requirements.txt file

All the modules needed in your Python file :

We do not provide the torch module as we’ve installed it previously

And… that’s it! We are ready to build our Docker image!

Build the docker image

This is it. If you’re using a Windows device, good luck. Docker desktop installation on Windows is a MESS (Yes I still have nightmares) and it crashes a LOT. A pleasure :)

Push on a VPS

I took the lowest configuration I could find (6$ per month) :

• Debian 10 (because docker was pre-installed on it)
• 1 vCore
• 2 Go
• 20 Go SSD SATA

Then, after the initial setup, you have to send your files via STP or SSH and you’re good to build!

After building your image and running your container, you should be able to access your API by making a call to http://your-vps-ip/8000/your_endpoint like this :

import requests

def query_api():
    # The API endpoint URL
    url = "http://your-vps-ip/8000/your-endpoint"

    # Send a GET request to the API
    response = requests.get(url)

    # Check if the request was successful
    if response.status_code == 200:
        # Parse the JSON response
        data = response.json()
        print(data)
    else:
        print(f"Failed to query API. Status code: {response.status_code}")

if __name__ == "__main__":
    query_api()

Perfect! You are now able to query your API from everywhere in the world! Do not forget to secure it!

Cloud Provider

Scaleway is a cloud provider (Like AWS or GCP). It provides a lot of interesting services at affordable cost. The servers are mainly located in France, so it’s a good solution if you are located in the EU.

For this tutorial, we are going to use the serverless services :

https://www.scaleway.com/en/pricing/?tags=serverless

Serverless means that you are able to run functions, and apps without having to manage servers.

There are two distinct services :

• Serverless Functions: this makes you able to run functions in the cloud. For example, you can run a Python function that triggers your smart lamp every morning without having your computer on
• Serverless containers: to run your Docker Containers in the cloud

This time, it’s easier, you just have to push your docker image into Scaleway. To do this:

• Create a namespace into “Container Registry” from the “Containers” section
• Use the namespace endpoint to push your container from docker Hub to Scaleway

BTW: do not forget to add all of your env variables into the SECRET Scaleway section!

Finally, you can run your container in the Cloud using the serverless container service, and voilà!

Thank you so much for reading! Now you should be able to deploy your first projects, this is a first step in the wide MLops world!

Do not hesitate to ask questions, I hope this article will help you in your Deep Learning journey!