Q:

How to configure Webhook activity for runbooks execution in Azure Data Factory v2?

I am working on running runbooks (powershell and graphical) from ADF. One of the ways I found to accomplish this task is to use webhooks. I will have runbooks running in parallel and in series (if dependency exists on previous runbook).
Overall,

If a flat file is dropped in Azure Blob storage then it triggers the pipeline that contains respective runbook(s). This part is working.
The webhook of runbook(s) are used in ADF webhook activity. This is where I am facing the problem. I am unsure about what should be in the body of webhook activity?

After some research I was able to find something about Callback uri that needs to be added (or somehow generated) in the body of the webhook. How can I get this Callback uri? If I don't add proper callback uri then the activity runs till timeout. I believe the functioning should be webhook activity completes when the runbook it's running is executed successfully so we can move on to next webhook activity in a pipeline. I have tried web activity as well but it's the same issue.
The body I am using right now is just below json.
{"body":{"myMessage":"Sample"}}
I have referenced:
https://vanishedgradient.com/2019/04/25/webhooks-with-azure-data-factory/
https://mrpaulandrew.com/2019/06/18/azure-data-factory-web-hook-vs-web-activity/
https://social.msdn.microsoft.com/Forums/en-US/2effcefb-e65b-4d5c-8b01-138c95126b79/in-azure-data-factory-v2-how-to-process-azure-analysis-service-cube?forum=AzureDataFactory

A:

Thanks for the links, they are useful sources. I've managed to get this working for a pipeline that calls a runbook to resize azure analysis services.  Having the runbook return failure and success information was not well documented.
Here's some code to assist a little, which i've taken from several places but a lot from the open issue (https://github.com/MicrosoftDocs/azure-docs/issues/43897) on this Microsoft page: https://docs.microsoft.com/en-us/azure/data-factory/control-flow-webhook-activity
The datafactory Webhook activity passes in some "Headers", SourceHost which is @pipeline().DataFactory and SourceProcess which is @pipeline().Pipeline. This was so we can do some checking to confirm that the runbook is being run by acceptable processes.
The Body of the call is then other variables we required:
@json(concat('{"AnalysisServer":"', pipeline().parameters.AASName, '", "MinimumSKU":"', pipeline().parameters.SKU,'"}') )

Your runbook needs the WebhookData parameter
param
(
    [Parameter (Mandatory=$false)]
    [object] $WebhookData
)

You can then grab all the bits you need, including checking if a callbackuri was provided:
if ($WebhookData)
{

    # Split apart the WebhookData
    $WebhookName     =     $WebhookData.WebhookName
    $WebhookHeaders  =     $WebhookData.RequestHeader
    $WebhookBody     =     $WebhookData.RequestBody | Convertfrom-Json
    $WebhookADF      =     $WebhookHeaders.SourceHost
    $WebhookPipeline =     $WebhookHeaders.SourceProcess

Write-Output -InputObject ('Runbook started through webhook {0} called by {1} on {2}.' -f $WebhookName, $WebhookPipeline, $WebhookADF)

# if there's a callBackURI then we've been called by something that is waiting for a response
If ($WebhookBody.callBackUri)
{
    $WebhookCallbackURI =  $WebhookBody.callBackUri
}

...
}

The variable $WebHookHeaders: @{Connection=Keep-Alive; Expect=100-continue; Host=sXXevents.azure-automation.net; SourceHost=**MYDATAFACTORYNAME**; SourceProcess=**MYPIPELINENAME**; x-ms-request-id=**UNIQUEIDENTIFIER**}
You can then grab information out of your json body: $AzureAnalysisServerName = $WebHookBody.AnalysisServer
Passing an error/failure back to your runbook is relatively easy, note that I put my success/update message in to $Message and only have content in $ErrorMessage if there's been an error:
$ErrorMessage = "Failed to do stuff I wanted" 

if ($ErrorMessage) 
{

    $Output = [ordered]@{ output= @{
        AzureAnalysisServerResize = "Failed" }
        error = @{
            ErrorCode = "ResizeError"
            Message = $ErrorMessage
        }
        statusCode = "500"
    }
} else {
    $Output = [ordered]@{ 
        output= @{ 
            "AzureAnalysisServerResize" = "Success"
            "message" = $Outputmessage 
        }
        statusCode = "200"
    }
}

$OutputJson = $Output | ConvertTo-Json -Depth 10

# if we have a callbackuri let the ADF Webhook activity know that the script is complete 
# Otherwise it waits until its timeout
If ($WebhookCallBackURI)
{
    $WebhookCallbackHeaders = @{
        "Content-Type"="application/json"
    }
    Invoke-WebRequest -UseBasicParsing -Uri $WebhookCallBackURI -Method Post -Body $OutputJson -Header $WebhookCallbackHeaders
}

I then end the if ($WebhookData) { call with an else to say the runbook shouldn't be running if not called from webhook:
} else {
    Write-Error -Message 'Runbook was not started from Webhook' -ErrorAction stop
}

Passing back an error message was quiet easy, passing back a success message has been traumatic, but the above seems to work, and in my datafactory pipeline i can access the results.
Output
{
    "message": "Analysis Server MYSERVERNAME which is SKU XX is already at or above required SKU XX.",
    "AzureAnalysisServerResize": "Success"
}

Note that with the Invoke-WebRequest, some examples online don't specify -UseBasicParsing but we had to as the runbook complained: Invoke-WebRequest : The response content cannot be parsed because the Internet Explorer engine is not available, or Internet Explorer's first-launch configuration is not complete. 

A:

Apologies for delay. I had found the complete solution few months back. Thanks to Nick and Sara for adding the pieces. I used similar code as return code. We were using graphical runbooks with limited changes allowed so I just added return code (Powershell) at the end of the runbook to have little to no impact. I plugged in below code:
if ($WebhookData)
{
    Write-Output $WebhookData
    $parameters = (ConvertFrom-Json -InputObject $WebhookData.RequestBody)
    if ($parameters.callBackUri)
    {
        $callbackuri = $parameters.callBackUri
    }
}

if ($callbackuri)
{
    Invoke-WebRequest -Uri $callbackuri -UseBasicParsing -Method POST
}

Write-Output $callbackuri

After this I added an input parameter using "Input and Output" button available in the runbook. I named the input parameter as "WebhookData" and type as "Object". The name of input parameter is case-sensitive and should match the parameter used in Powershell code.
This resolved my issue. The runbook started when called from ADF pipeline and moved to next pipeline only when the underlying runbook called by the webhook was completed.

