implement-durable-functions

Implement Durable Functions

In this post we are going to describe the following objects:

• App patterns typically used in durable functions
• Dhe four durable function types
• The function Task Hubs perform in durable functions
• The use of durable orchestrations, timers, and events

Durable Functions app patterns

the durable functions extension lets you define stateful workflows by writing orchestrator functions and stateful entities by writing entity functions using the Azure Functions programming model. Behind the scenes, the extension manages state, checkpoints, and restarts for you, allowing you to focus on your business logic.

Application patterns

The primary use case for Durable Functions is simplifying complex, stateful coordination requirements in serverless applications. The following sections describe typical application patterns that can benefit from Durable Functions:

• Function chaining
• Fan-out/fan-in
• Async HTTP APIs
• Monitor
• Human interaction

Function chaining

In the function chaining pattern, a sequence of functions executes in a specific order. In this pattern, the output of one function is applied to the input of another function.

function is applied to the input of another function.

In the code example below, the values F1, F2, F3, and F4 are the names of other functions in the function app. You can implement control flow by using normal imperative coding constructs. Code executes from the top down. The code can involve existing language control flow semantics, like conditionals and loops. You can include error handling logic in try/catch/finally blocks.

[FunctionName("Chaining")]
 public static async Task<object> Run( [OrchestrationTrigger] IDurableOrchestrationContext context) 
{ 
try 
{
 var x = await context.CallActivityAsync<object>("F1", null);
 var y = await context.CallActivityAsync<object>("F2", x); 
var z = await context.CallActivityAsync<object>("F3", y); 
return await context.CallActivityAsync<object>("F4", z); 
} 
catch (Exception) 
{ 
// Error handling or compensation goes here. 
} 
}

In the fan out/fan in pattern, you execute multiple functions in parallel and then wait for all functions to finish. Often, some aggregation work is done on the results that are returned from the functions.

With normal functions, you can fan out by having the function send multiple messages to a queue. To fan in you write code to track when the queue-triggered functions end, and then store function outputs.

In the code example below, the fan-out work is distributed to multiple instances of the F2 function. The work is tracked by using a dynamic list of tasks. The .NET Task.WhenAll API or JavaScript context.df.Task.all API is called, to wait for all the called functions to finish. Then, the F2 function outputs are aggregated from the dynamic task list and passed to the F3 function.

[FunctionName("FanOutFanIn")]
public static async Task Run( [OrchestrationTrigger] IDurableOrchestrationContext context)
{
    var parallelTasks = new List<Task<int>>();
   // Get a list of N work items to process in parallel.
    object[] workBatch = await context.CallActivityAsync<object[]>("F1",
    null);
    for (int i = 0; i < workBatch.Length; i++)
    {
      Task<int> task = context.CallActivityAsync<int>("F2", workBatch[i]);
     parallelTasks.Add(task);
   }
     await Task.WhenAll(parallelTasks);
     // Aggregate all N outputs and send the result to F3.
      int sum = parallelTasks.Sum(t => t.Result);
     await context.CallActivityAsync("F3", sum);
 }

The automatic checkpointing that happens at the await or yield call on Task.WhenAll or context.df.Task.all ensures that a potential midway crash or reboot doesn’t require restarting an already completed task.

Async HTTP APIs

The async HTTP API pattern addresses the problem of coordinating the state of long-running operations with external clients. A common way to implement this pattern is by having an HTTP endpoint trigger the long-running action. Then, redirect the client to a status endpoint that the client polls to learn when the operation is finished.

Durable Functions provides built-in support for this pattern, simplifying or even removing the code you need to write to interact with long-running function executions. After an instance starts, the extension exposes webhook HTTP APIs that query the orchestrator function status.

The following example shows REST commands that start an orchestrator and query its status. For clarity, some protocol details are omitted from the example. using curl in command line

> curl -X POST https://myfunc.azurewebsites.net/orchestrators/DoWork -H "Content-Length: 0" -i HTTP/1.1 202 Accepted Content-Type:
application/json Location: https://myfunc.azurewebsites.net/runtime/webhooks/durabletask/b79baf67f717453ca9e86c5da21e03ec
{"id":"b79baf67f717453ca9e86c5da21e03ec", ...}
 > curl https://myfunc.azurewebsites.net/runtime/webhooks   /durabletask/b79baf67f717453ca9e86c5da21e03ec -i
HTTP/1.1 202 Accepted
Content-Type: application/json
Location: https://myfunc.azurewebsites.net/runtime/webhooks/durabletask /b79baf67f717453ca9e86c5da21e03ec
{"runtimeStatus":"Running","lastUpdatedTime":"2019-03-16T21:20:47Z", ...}
> curl https://myfunc.azurewebsites.net/runtime/webhooks/durabletask /b79baf67f717453ca9e86c5da21e03ec -i
 HTTP/1.1 200 OK
Content-Length: 175
Content-Type: application/json
 {"runtimeStatus":"Completed","lastUpdatedTime":"2019-03-16T21:20:57Z", ...}

The Durable Functions extension exposes built-in HTTP APIs that manage long-running orchestrations. You can alternatively implement this pattern yourself by using your own function triggers (such as HTTP, a queue, or Azure Event Hubs) and the orchestration client binding.

You can use the HttpStart triggered function to start instances of an orchestrator function using a client function.

public static class HttpStart
 {
   [FunctionName("HttpStart")] public static async
      Task<HttpResponseMessage>  Run(      [HttpTrigger(AuthorizationLevel.Function, methods: "post",Route = "orchestrators/{functionName}")]
   HttpRequestMessage req, [DurableClient] IDurableClient starter,
    string functionName, ILogger log)
  {
      // Function input comes from the request content.
       object eventData = await req.Content.ReadAsAsync<object>();
      string instanceId = await starter.StartNewAsync(functionName,  eventData);
     log.LogInformation($"Started orchestration with ID = '{instanceId}'.");
    return starter.CreateCheckStatusResponse(req, instanceId);
   }
}

To interact with orchestrators, the function must include a DurableClient input binding. You use the client to start an orchestration. It can also help you return an HTTP response containing URLs for checking the status of the new orchestration.

Monitor

The monitor pattern refers to a flexible, recurring process in a workflow. An example is polling until specific conditions are met. You can use a regular timer trigger to address a basic scenario, such as a periodic cleanup job, but its interval is static and managing instance lifetimes becomes complex. You can use Durable Functions to create flexible recurrence intervals, manage task lifetimes, and create multiple monitor processes from a single orchestration.

In a few lines of code, you can use Durable Functions to create multiple monitors that observe arbitrary endpoints. The monitors can end execution when a condition is met, or another function can use the durable orchestration client to terminate the monitors. You can change a monitor’s wait interval based on a specific condition (for example, exponential backoff).

The following code implements a basic monitor:

[FunctionName("MonitorJobStatus")]
 public static async Task Run( [OrchestrationTrigger] IDurableOrchestrationContext context) 
{
 int jobId = context.GetInput<int>();
 int pollingInterval = GetPollingInterval();
 DateTime expiryTime = GetExpiryTime();
 while (context.CurrentUtcDateTime < expiryTime) 
{
 var jobStatus = await context.CallActivityAsync<string>("GetJobStatus", jobId);
 if (jobStatus == "Completed")
 {
 // Perform an action when a condition is met.
 await context.CallActivityAsync("SendAlert", machineId); 
break; 
} 
// Orchestration sleeps until this time.
 var nextCheck = context.CurrentUtcDateTime.AddSeconds(pollingInterval); await context.CreateTimer(nextCheck, CancellationToken.None);
 }
 // Perform more work here, or let the orchestration end.

Human interaction

Many automated processes involve some kind of human interaction. Involving humans in an automated process is tricky because people aren’t as highly available and as responsive as cloud services. An automated process might allow for this interaction by using timeouts and compensation logic.

interaction by using timeouts and compensation logic.

You can implement the pattern in this example by using an orchestrator function. The orchestrator uses a durable timer to request approval. The orchestrator escalates if timeout occurs. The orchestrator waits for an external event, such as a notification that’s generated by a human interaction.

Additional resources

• To learn about the differences between Durable Functions 1.x and 2.x visit:
  • Durable Functions versions overview

There are currently four durable function types in Azure Functions: orchestrator, activity, entity, and client. The rest of this section goes into more details about the types of functions involved in an orchestration.

Orchestrator functions

Orchestrator functions describe how actions are executed and the order in which actions are executed. Orchestrator functions describe the orchestration in code (C# or JavaScript) as shown in the previous unit. An orchestration can have many different types of actions, including activity functions, sub-orchestrations, waiting for external events, HTTP, and timers. Orchestrator functions can also interact with entity functions.

Orchestrator functions are written using ordinary code, but there are strict requirements on how to write the code. Specifically, orchestrator function code must be deterministic. Failing to follow these determinism requirements can cause orchestrator functions to fail to run correctly.

Activity functions

Activity functions are the basic unit of work in a durable function orchestration. For example, you might create an orchestrator function to process an order. The tasks involve checking the inventory, charging the customer, and creating a shipment. Each task would be a separate activity function. These activity functions may be executed serially, in parallel, or some combination of both.

Unlike orchestrator functions, activity functions aren’t restricted in the type of work you can do in them. Activity functions are frequently used to make network calls or run CPU intensive operations. An activity function can also return data back to the orchestrator function.

An activity trigger is used to define an activity function. .NET functions receive a DurableActivityContext as a parameter. You can also bind the trigger to any other JSON-serializeable object to pass in inputs to the function. In JavaScript, you can access an input via the <activity trigger binding name> property on the context.bindings object. Activity functions can only have a single value passed to them. To pass multiple values, you must use tuples, arrays, or complex types.

Entity functions

Entity functions define operations for reading and updating small pieces of state. We often refer to these stateful entities as durable entities. Like orchestrator functions, entity functions are functions with a special trigger type, entity trigger. They can also be invoked from client functions or from orchestrator functions. Unlike orchestrator functions, entity functions do not have any specific code constraints. Entity functions also manage state explicitly rather than implicitly representing state via control flow. Some things to note:

• Entities are accessed via a unique identifier, the entity ID. An entity ID is simply a pair of strings that uniquely identifies an entity instance.
• Operations on entities require that you specify the Entity ID of the target entity, and the Operation name, which is a string that specifies the operation to perform.

Client functions

Orchestrator and entity functions are triggered by their bindings and both of these triggers work by reacting to messages that are enqueued in a task hub. The primary way to deliver these messages is by using an orchestrator client binding, or an entity client binding, from within a client function. Any non-orchestrator function can be a client function. For example, You can trigger the orchestrator from an HTTP-triggered function, an Azure Event Hub triggered function, etc. What makes a function a client function is its use of the durable client output binding.

Unlike other function types, orchestrator and entity functions cannot be triggered directly using the buttons in the Azure portal. If you want to test an orchestrator or entity function in the Azure portal, you must instead run a client function that starts an orchestrator or entity function as part of its implementation. For the simplest testing experience, a manual trigger function is recommended.

Explore task hubs

A task hub in Durable Functions is a logical container for durable storage resources that are used for orchestrations and entities. Orchestrator, activity, and entity functions can only directly interact with each other when they belong to the same task hub.

If multiple function apps share a storage account, each function app must be configured with a separate task hub name. A storage account can contain multiple task hubs. This restriction generally applies to other storage providers as well.

Azure Storage resources

A task hub in Azure Storage consists of the following resources:

• One or more control queues.
• One work-item queue.
• One history table.
• One instances table.
• One storage container containing one or more lease blobs.
• A storage container containing large message payloads, if applicable.

All of these resources are created automatically in the configured Azure Storage account when orchestrator, entity, or activity functions run or are scheduled to run.

Task hub names

Task hubs in Azure Storage are identified by a name that conforms to these rules:

• Contains only alphanumeric characters
• Starts with a letter
• Has a minimum length of 3 characters, maximum length of 45 characters

The task hub name is declared in the host.json file, as shown in the following example:

{
"version": "2.0", 
"extensions": 
{ 
"durableTask":
 { 
"hubName": "MyTaskHub"
 }
 } 
}

[FunctionName("BillingIssuer")] 
public static async Task Run( [OrchestrationTrigger] IDurableOrchestrationContext context)
 {
 for (int i = 0; i < 10; i++) 
{
 DateTime deadline = context.CurrentUtcDateTime.Add(TimeSpan.FromDays(i + 1));
 await context.CreateTimer(deadline, CancellationToken.None); 
await context.CallActivityAsync("SendBillingEvent");
 }
 }

[FunctionName("TryGetQuote")] 
public static async Task<bool> Run( [OrchestrationTrigger] IDurableOrchestrationContext context) 
{
 TimeSpan timeout = TimeSpan.FromSeconds(30);
 DateTime deadline = context.CurrentUtcDateTime.Add(timeout); 
using (var cts = new CancellationTokenSource())
 {
 Task activityTask = context.CallActivityAsync("GetQuote");
 Task timeoutTask = context.CreateTimer(deadline, cts.Token); 
Task winner = await Task.WhenAny(activityTask, timeoutTask);
 if (winner == activityTask) 
{
 // case success 
 cts.Cancel();
 return true;
 }
 else
 {
 // case timeout
 return false;
 }
 }
 }

[FunctionName("BudgetApproval")] 
public static async Task Run( [OrchestrationTrigger] IDurableOrchestrationContext context)
 {
 bool approved = await context.WaitForExternalEvent<bool>("Approval");
 if (approved)
 {
 // approval granted - do the approved action
 }
 else
 {
 // approval denied - send a notification
 }
 }

[FunctionName("ApprovalQueueProcessor")] 
public static async Task Run( [QueueTrigger("approval-queue")] string instanceId, [DurableClient] IDurableOrchestrationClient client) 
{
 await client.RaiseEventAsync(instanceId, "Approval", true);
 }

Conclusion

In this post I have described  the app patterns typically used in Durable Functions, the four durable function types,  have explain the function Task Hubs perform in Durable  Functions and describe the use of durable orchestrations, timers, and events.

In my next post I will explain Develop Azure Functions

This post is part of “Azure step by step 

Back to home page