Deploying ML Models using Flask and Heroku
Most people who consider themselves self-taught Data Scientists have probably started their journey by completing some of the countless online courses that are out there. These courses usually start with basic programming skills in R or Python, proceed with data cleaning and exploration, and usually end with basic model training and evaluation using some specific framework like caret (R) or scikit-learn (Python). However, stopping at this point does not enable practitioners to create business value by applying their newly learned skills.
Often, Machine Learning and Deep Learning models end up in Jupyter Notebooks. These are great for exploration or model training, but usually do not aid in data driven decision making. Hence, in order to make use of the predictive analysis, models often need to be taken into production.
In this notebook I describe the often neglected process of model deployment, which comes after training and building the model. There are many possibilities and frameworks for deploying models. In this post, I create a small App using Flask and basic HTML and deploy it using Heroku since it can be used for free and is perfect for small projects.
In the last post I built a model which classifies songs into one of three genres based on their lyrics. This model will be used in this example.
At first, we import the packages needed, which are Flask, and since we trained the model using fastai, fastai and sentencepiece as the tokenizer used in the model.
from flask import Flask, render_template, url_for, redirect, request
from fastai.text.all import *
import sentencepiece
We exported the model and saved it to disk, so that we can now use it without needing to retrain. We can simply import it:
learn_inf = load_learner("genreModel.pkl")
1. Building the App
We first initialize an object of the Flask class called app. Then, we create our first route, which will be the landing page of the app.
app = Flask(__name__)
On this page, there should be a short introductory text and a field in which the user can paste song lyrics. Moreover, there should be a clickable button which starts the prediction. This can be achieved by only a few lines of HTML which looks like the following:
<html>
<body>
<h1> Genre Classifier</h1>
Willkommen beim Genre-Classifier für deutsche Musik. Füge im Feld unten einfach die Lyrics eines deutschen Songs ein, um zu erfahren, welchem Genre der Text am ehesten entspricht.
<p>Derzeit unterstützte Genres:</p>
<nav>
<ul>
<li>Hip-Hop</li>
<li>Pop</li>
<li>Schlager</li>
</ul>
</nav>
<form action="{{ url_for('lyrics') }}" method="post" class="lyrics" id="textzeilen">
Lyrics:<br><br>
<textarea id="textzeilen" rows="30" cols="60" name="lyrics"></textarea>
<br><br><br>
<input class="button" type="submit" value="Go!">
</form>
</body>
</html>
In the root directory of the project, we create a folder called “templates”, paste the above code into an empty file and save it as “index.html”. We now tell Flask to render this HTML file when the base URL is accessed.
@app.route("/", methods=["GET"])
def index():
return render_template("index.html")
We further need a function which takes the lyrics input from the user as an argument, passes it to the model and returns the prediction. To achieve this, Flask’s request.form is used. It returns the content from a HTML post form, which is exactly what the lyrics textarea on the index.html page is. The lyrics are then passed as input to fastai’s predict function and the prediction is saved.
@app.route("/lyrics", methods=["POST"])
def lyrics():
lyrics = request.form["lyrics"]
pred = learn_inf.predict(lyrics)[0]
The results page should show an image that somehow relates to one of the genres and the predicted genre. We choose one image per genre and save them as “hiphop.jpg”, “schlager.jpg”, and “pop.jpg” in a folder named static. Lastly, the results page should have a clickable button which redirects back to the landing page saying “Do it again!”. We define three HTML pages, one for each genre, which look as follows:
<html>
<body>
<div align="center" style="padding-top:20px;">
<img src="/static/genre.jpg">
<h3>{{ pred }}</h3>
<br />
<form action="{{ url_for('index') }}">
<input type="submit" value="Do it again!" />
</form>
</div>
</body>
</html>
Some string editing is added on the prediction and then the route renders the corresponding genre page:
@app.route("/lyrics", methods=["POST"])
def lyrics():
lyrics = request.form["lyrics"]
pred = learn_inf.predict(lyrics)[0]
# edit string output
if pred == "hiphop":
pred = "Hip-Hop"
if pred in ["pop", "schlager"]:
pred = pred.capitalize()
# render corresponding html file
if pred == "Hip-Hop":
return render_template("hiphop.html", pred=pred)
elif pred == "Pop":
return render_template("pop.html", pred=pred)
elif pred == "Schlager":
return render_template("schlager.html", pred=pred)
The app can be run and tested locally using the run() function.
app.run()
2. Deploying the App to Heroku
After building our App, we will now use Heroku to deploy it. The first step is to combine all of the above code snippets into a single app.py file.
if __name__ == "__main__":
app.run(host='0.0.0.0', port=5000)
The above statement is included at the end of the script. This ensures that app.run() is run when the script is executed, e.g. from the command line. We then create a file named “Procfile” which contains “web: python app.py”. Under linux this can be done in the terminal:
!echo "web: python app.py" > Procfile
We further need a requirements.txt file which contains all the dependencies and python packages for the project. By providing this file, we make sure that all of these get installed when the app is deployed on Heroku. The following command line does this:
!pip freeze > requirements.txt
Next, we need to create a repository on GitHub, initialize our project folder as a git repository and push it to the just created GitHub repo. Since GitHub has a size limit of 100 MB for files, we use git lfs to track our model file, which is around 125 MB.
The next step is to create a Heroku account. There are two ways to deploy an app, either using Heroku CLI or using the website. We use the latter approach here. After logging in, choose “Create new app”, add a name for your app and create the app. Click on “Deploy”, choose Github as the deployment method, connect to your Github and choose the repository. To start the deployment, click on “Deploy Branch” on the bottom of the page.
Since we used git lfs in our project, we need to adjust some settings since Heroku cannot handle git lfs files by default. To solve this issue, navigate to “Settings” and click on “Add buildpack”. Paste “https://github.com/raxod502/heroku-buildpack-git-lfs.git" as the URL. Lastly, we need to add config vars for this buildpack. Choose “Reveal Config Vars” and use “HEROKU_BUILDPACK_GIT_LFS_REPO” for the key field. The value needs to be a personal access token which can be generated via GitHub. To do so, go to your GitHub > Settings > Developer Settings > Personal access tokens. Here you can generate a new SSH key. Copy this key. The value field requires your token in the following form:
“https://
@github.com/username/repository.git”
In addition to the git lfs buildpack we also need to add the python buildpack. To do this, simply click on “Add buildpack” and choose Python from the list.
You can now navigate back to “Deploy” and click on “Deploy Branch”. The app is now being built and deployed, which might take a few minutes. After that, you can access your website.
3. The final Github Repo and the App
The complete Github Repository for this project can be seen here. My app can be accessed via https://whichgenreisthis.herokuapp.com/.