Azure Durable Functions Monitoring and Diagnostics

The post shows some of the possibilities to monitor Azure Durable Functions and how diagnostic APIs could be implemented.

Code: https://github.com/damienbod/AzureDurableFunctions

Posts in this series

Diagnostic APIs

Azure Functions could be used to add APIs which can request the status of the different orchestration instances or even complete lists of flows using the Azure Durable Function APIs.

The IDurableOrchestrationClient provides different APIs for displaying the state of the different orchestrations. The GetStatusAsync method can be used to return the actual status of a single orchestration instance. The status can be returned with the history if required.

[FunctionName(Constants.Diagnostics)]
public async Task<IActionResult> Diagnostics(
 [HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = null)] HttpRequest req,
 [DurableClient] IDurableOrchestrationClient starter,
 ILogger log)
{
	string instanceId = req.Query["instanceId"];
	log.LogInformation($"Started DiagnosticsApi with ID = '{instanceId}'.");

	var data = await starter.GetStatusAsync(instanceId, true);
	return new OkObjectResult(data);
}

This can be viewed in the browser using a HTTP Get request with the instanceId query parameter.

An Azure Function GetCompletedFlows can be implemented to return a list of orchestration as required. The following example returns all completed instances which ran in the last N days read from the query parameter days. If nothing is defined, the last day is returned.

[FunctionName(Constants.GetCompletedFlows)]
public async Task<IActionResult> GetCompletedFlows(
[HttpTrigger(AuthorizationLevel.Anonymous, "get")] HttpRequest req,
[DurableClient] IDurableOrchestrationClient client,
ILogger log)
{
	var runtimeStatus = new List<OrchestrationRuntimeStatus> {
		OrchestrationRuntimeStatus.Completed
	};

	return await FindOrchestrations(req, client, runtimeStatus,
		DateTime.UtcNow.AddDays(GetDays(req)),
		DateTime.UtcNow, true);
}

private static int GetDays(HttpRequest req)
{
	string daysString = req.Query["days"];
	if (!string.IsNullOrEmpty(daysString))
	{
		var ok = int.TryParse(daysString, out int days);
		if (!ok)
		{
			return -1;
		}
		return -days;
	}

	return -1;
}

The FindOrchestrations implements the search using the ListInstancesAsync method from the Azure Durable Functions IDurableOrchestrationClient interface. This takes a OrchestrationStatusQueryCondition parameter which can be used as required.

private async Task<IActionResult> FindOrchestrations(
	HttpRequest req,  
	IDurableOrchestrationClient client,
	IEnumerable<OrchestrationRuntimeStatus> runtimeStatus,
	DateTime from,
	DateTime to,
	bool showInput = false)
{
	// Define the cancellation token.
	CancellationTokenSource source = new CancellationTokenSource();
	CancellationToken token = source.Token;

	var instances = await client.ListInstancesAsync(
		new OrchestrationStatusQueryCondition
		{
			CreatedTimeFrom = from,
			CreatedTimeTo = to,
			RuntimeStatus = runtimeStatus,
			ShowInput = showInput
		},
		token
	);

	return new OkObjectResult(instances);
}

When the functions are running, the data can be viewed directly in the browser using a simple GET request. If processing senstive data, these APIs would have to be secured.

DurableFunctionsMonitor

DurableFunctionsMonitor provides another way of monitoring your Azure Durable Functions. This is an excellent OSS project which can be added to Visual Studio Code as an extension or used directly.

https://github.com/scale-tone/DurableFunctionsMonitor

I added this to Visual Studio Code as an extension. Once installed, you need to configure this. In the Visual Studio Code explorer, you should have a Durable Function menu. Click the plug and add your connection string to your storage. This can be found using the Azure Storage Explorer and when you click the Storage Account, which you want to view, the properties will display your connection string.

Then you need to select the Durable Function Hub. This can be found in the Storage Explorer in the Tables. You will have one for the History and one for the Instances. Use the name before this.

Now the Functions can be viewed and active requests can be used like purge, terminate and so on.

Azure Durable Functions are built using Azure Functions which is an Azure App Service. This also provides further monitoring and diagnostic APIs for further standard monitoring. The persisted Azure Durable Function data can also be directly viewed in the Azure Storage Explorer.

Links:

https://github.com/scale-tone/DurableFunctionsMonitor

https://www.npmjs.com/package/azure-functions-core-tools

https://damienbod.com/2018/12/23/using-azure-key-vault-with-asp-net-core-and-azure-app-services/

https://docs.microsoft.com/en-us/azure/azure-functions/functions-how-to-use-azure-function-app-settings

https://docs.microsoft.com/en-us/azure/azure-functions/durable/

https://github.com/Azure/azure-functions-durable-extension

https://damienbod.com/2019/03/14/running-local-azure-functions-in-visual-studio-with-https/

Microsoft Azure Storage Explorer

Microsoft Azure Storage Emulator

Install the Azure Functions Core Tools

NodeJS