Create and deploy a Python Azure Function

Case
I want to create an Azure Function with Python code. How do I create and deploy one in Azure?

Write Python in Visual Studio Code to create an Azure Function

Solution
In this blogpost we will create and deploy a very simple 'hello world' Azure Function with an HTTP trigger which you can extend to your own needs. After that we can use the Azure Data Factory pipeline with an Azure Function activity to execute it.

As an alternative you could also create an Azure Function with a Blob Storage trigger that executes when a new file arrives, but we rather want to use that same trigger type to start an Azure Data Factory pipeline that then starts this Function followed by other pipeline activities. This way we have one place that does the triggering/orchestration.

1) Create new Azure Function project
Please first follow the steps of our previous post on how to prepare Visual Studio Code for creating Azure Functions with Python. After that open Visual Studio code and perform the steps below to create your first hello world Function.

• In Visual Studio code click on the Azure icon in the left menu.
• In the newly opened pane click on the folder with the lightning icon on it to create a new project. (An additional function can later-on be added to the project with the Lightning-plus icon)
• Select the folder of the new project (or use the Browse... option)
• Next select Python as coding language
• Select the Python interpreter. In our example it is py 3.8.3 (if it's not in the list you need to browse to python.exe on your device)
• Select HTTP trigger as the template for this example
• Enter the Function name. This is the name of the function within the project (that can contain multiple functions)
• For this test example use anonymous as Authorization level
• The project has been created, but continue with the next step below with some additional actions before you can start coding

Create new Azure Function project

2) Select interpreter and install Linter pylint
To finish the creation of the project we need to select the location of the interpreter and install a linter.

• Click on the popup in the bottom left corner to select the interpreter. 
• On the top in the middle you can now select the interpreter. Select the one in the .venv folder which is a subfolder of your project.
• Next step is to install Linter pylint by clicking on the install button on the new popup in the bottom left corner.
• Now wait a few seconds for the installation to finish

Select interpreter and install Linter pylint

3) Code in __init__.py
The file '__init__.py' contains your Python code. Below you see the standard / generated code with some extra comment lines for if you are new to Python. For this example we do not extend the code.

# Import module for logging purposes
import logging

# Import module for Azure Functions and give it an alias
import azure.functions as func

# Main function and entry point of this Azure Function
def main(req: func.HttpRequest) -> func.HttpResponse:
    # Log information
    logging.info('Python HTTP trigger function processed a request.')

    # Retrieve parameter 'name' from querystring
    name = req.params.get('name')
    # If not found try to retrieve it from the request body
    if not name:
        try:
            # Check if there is a request body
            req_body = req.get_json()
        except ValueError:
            # On failure do nothing
            pass
        else:
            # On success try to retrieve name from request body
            name = req_body.get('name')

    # If a name was found then response with 'Hello [name]'
    if name:
        return func.HttpResponse(f"Hello {name}!")
    else:
        # If a name was not found response with an error message
        return func.HttpResponse(
             "Please pass a name on the query string or in the request body",
             status_code=400
        )

4) Debug locally
Now we are going to test the Azure Function locally on our Windows device. There are multiple ways to start debugging. Pressing F5 is probably the easiest. See animated gif for more options.

• In the Run menu on the top of the screen you will find the Start Debugging option. 
• The terminal pane on the bottom will show a lot of details. Wait a few seconds for it to finish and click (while also pressing CTRL) on the green URL.
• A new browser window will open and it shows the error output that it cannot find the name parameter.
• In the browser add a querystring after the URL: ?name=Joost (or your own name of course). Now it will respond with a greeting
• Close the browser and then hit the disconnect icon on top to stop debugging

Debugging your function locally

5) Create Azure Function in Azure Portal
Before you can deploy your newly created function you first need to create an Azure Function in the Azure portal.

• Go to the Azure Portal and click on Create a resource
• Search for function and select Function App
• Click on the Create button
• On the Basics tab you find the most important settings
• Select your Subscription and Resource Group
• Enter an unique Function App name
• Select Python as Runtime stack
• Select the Python Version (3.8 in our example)
• Select the Region (probably the same as your Resource Group)
• Optionally go to the Hosting tab for extra settings
• Choose a new or existing Storage account
• Change the Plan type (default: Serverless)
• Optionally go to the Monitoring tab for extra settings
• Disable or enable Application insights and change its name
• Click the Review + create button
• Review the settings and click on the Create button

Create new Azure Function (app) on Azure portal

Note 1: you cannot create an Azure Function with a linux worker (python) if there is already a Windows worker (C#) in that same resource group and with the same region. You will then get an error: LinuxWorkersNotAllowedInResourceGroup - Linux workers are not available in resource group bitools. Use this link to learn more https://go.microsoft.com/fwlink/?linkid=831180. Also see the Azure Function documentation. Summary: don't mix C# and Python functions within the same resource group.

Note 2: you could also perform these steps within Visual Studio Code during deployment.

6) Deploy Azure Function to Azure Portal
Now that we have an (empty) Azure Functions app in the Azure portal we can deploy our newly created Azure Function to this resource.

• In Visual Studio code click on the Azure icon in the left menu.
• In the newly opened pane click on the blue arrow (deploy) icon
• In the drop down select your Azure Functions App from the previous step

Deploy Azure Functions from Visual Studio Code

7) Testing in portal
Now that we have deployed our project to Azure Functions we can test it in the Azure Portal. For this example we will use the post method.

• Go to the Azure Portal and then open you Azure Functions App
• In the left menu click on Functions
• In the list of functions click on your function (only one in this example)
• In the left menu click on Code + Test
• Click on the test button (top center)
• Change the HTTP method to post
• Select one of the keys
• Enter a JSON message in the body: {"name":"Joost"} (name=case-sensitive)
• Click on the Run button and see the result

Testing in the Azure Portal

7) Executing from Azure Data Factory
Now if you want to execute this new Azure Function in Azure Data Factory with the Azure Function Activity you can follow the steps in this previous post. However without code changes it will return an error stating that the response is invalid: 3603 - Response Content is not a valid JObject

3603 - Response Content is not a valid JObject

At the moment it is returning a so called JArray, but it is expecting a JObject (J = JSON). Any other return types than JObject will throw the error above. To overcome this we need two code changes. First we need to import the JSON module by adding: import json at the top of the code with the other imports. Then we need to adjust the return of the Hello world message. See changes after lines 4 and 30.

# Import module for logging purposes
import logging

# import json module to return a json message
import json

# Import module for Azure Functions and give it an alias
import azure.functions as func

# Main function and entry point of this Azure Function
def main(req: func.HttpRequest) -> func.HttpResponse:
    # Log information
    logging.info('Python HTTP trigger function processed a request.')

    # Retrieve parameter 'name' from querystring
    name = req.params.get('name')
    # If not found try to retrieve it from the request body
    if not name:
        try:
            # Check if there is a request body
            req_body = req.get_json()
        except ValueError:
            # On failure do nothing
            pass
        else:
            # On success try to retrieve name from request body
            name = req_body.get('name')

    # If a name was found then response with 'Hello [name]'
    if name:
        #return func.HttpResponse(f"Hello {name}!")
        message = {'message': f"Hello {name}!"}
        return json.dumps(message)
    else:
        # If a name was not found response with an error message
        return func.HttpResponse(
             "Please pass a name on the query string or in the request body",
             status_code=400
        )

Below a couple of screenshots on how to configure and test this in Azure Data Factory. You might want to store the Function key in Azure Key Vault to avoid keys in your ETL/ELT code.

Set up the Azure Function Activity in Azure Data Factory

After configuring the Azure Function activity you can hit the debug button and see the result. This output could then be used as input for successive pipeline activities

Successfully executing and getting the response

Conclusion
First a big thank you to colleague Jasper Diefenbach for helping me out with the Python stuff. In this blog post you learned how to create, test and deploy your first (very basic) Azure Function App with Python code. Then we also showed you how to execute this from Azure Data Factory. In a couple of follow up posts we will show you how to build some useful functions for DWH projects and show you some technical stuff like adding Azure Key Vault to the game. Also check out the C# version of this blogpost.