Monday 18 November 2019

Runbook snack: read parameters from Data Factory

Case
I want to pass parameters from an Azure Data Factory pipeline web(hook) activity to my Azure Automation Runbook. How do I do that?
Supplying parameters via the Body property in Web(hook) activity























Solution
You cannot pass through a single parameters and then read that particularly parameter with PowerShell. Instead you have to read all the data coming from the webhook as one big Object parameter. That object also contains the Body property which on its turn contains a JSON message with a parameter name and value.

1) JSON Message
The parameter 'ContainerName'' will be provided in Azure Data Factory (ADF) via a JSON message that looks like
{"ContainerName":"OfficeData"}
2) Coding
In an Azure Automation Runbook you can start your code with the following code snippit that retrieves the value from the JSON messages from ADF. For this example it gets the name of a Blob Storage Container.
# PowerShell code

# Parameters
Param
(
    # Get webhook data via webhook
    [Parameter(Mandatory=$False,Position=1)]
    [object] $WebhookData
)

# Get all parameters from body (passed from Data Factory Web Activity)
$Parameters = (ConvertFrom-Json -InputObject $WebhookData.RequestBody)

# Get single parameter from the set of parameters
$ContainerName = $Parameters.ContainerName

# Write the value of the parameter
Write-Output "The value of the container parameter is $($ContainerName)"

3) Adding Webhook
Now that the code is ready, you have to create a Webhook for ADF. Make sure to choose a correct expire date. By default it is only valid for one year. Also make sure to copy the URL, because you can only see and copy it once. You need this URL in ADF for the Web(hook) activity. Don't share the URL because it is a URL and Password in one.
Adding a Webhook to a Runbook
















4) Azure Data Factory
In the pipeline of ADF you can use the Web(hook) activity to call the Webhook and use a JSON message in the Body parameter to provide a value for a parameter. The URL is the Webhook URL from the previous step that you copied. For Method you need to select "POST" and in the Body you can enter the JSON message from step 1.
Supplying parameters via the Body property in Web(hook) activity























Conclusion
Perhaps a bit strange, but this seems to be the only way when using webhooks for your runbook and calling them in ADF. As an alternative you could provide the value of the parameters in the Webhook when you create the Webhook instead of supplying them via ADF, but that is not very flexible if you expect a lot of different values. The code for retrieving parameters from the Webhook itself looks like this:
# PowerShell code

# Parameters
Param
(
    # Get parameter via webhook
    [Parameter(Mandatory=$False,Position=1)]
    [string] $ContainerName
)

# Write the value of the parameter
Write-Output "The value of the container parameter is $($ContainerName)"

No comments:

Post a Comment

All comments will be verified first to avoid URL spammers. यूआरएल स्पैमर से बचने के लिए सभी टिप्पणियों को पहले सत्यापित किया जाएगा।