Telemetry Correlation in Azure Functions
0/5 (0 vote)
ASP.NET automatically parses distributed tracing headers, in azure functions however a bit of custom code is required.
In a distributed system, distributed tracing (log/trace/telemetry correlation) can be nice to have or absolutely necessary, depending on the complexity of the system. With the rise of Microservices it's becoming more and more the latter.
When using the HttpClient in .NET, the correlation information is automatically passed to other web application using headers. Those web applications can than parse the headers and include the correlation data in log entries. When the receiving web application is based on ASP.NET (Core or Full) the most you have to do is install a nuget package and the header parsing and initialization is done automatically. In Azure functions however, this feature is missing (see github issues #1, #2)
Workarounds for the missing correlation feature in Azure Functions usually point to manual logging via TelemetryClient. This is however rather cumbersome and requires a lot of boilerplate code (e.g. for exception tracking alone you need to basically wrap every function...). On top of that, correlation data is also not passed to other system when using HttpClient.
IMHO the better option is to implement the correlation with custom code. Fortunately ASP.NET is open source, so several implementations are available as blueprints, e.g. in AspNet.TelemetryCorrelation there is an implementation of the header parsing.
After parsing the data, it still needs to be made available, so it will be picked up the HttpClient, the application insights logger, or other logging/tracing code. This can be done by creating a new `Activity` instance, setting current request id as the activities parentId and calling the `Start()` method then on this instance. This will internally set the static (AsyncLocal) Activity.Current property which is accessed by correlation data consumers.
A very basic implementation could look like this:
public static class CorrelationExtensions
{
public static CorrelationData InitializeLogCorrelationFromHeaders(
this ExecutionContext context, HttpRequestMessage request)
{
CorrelationData correlationData = ExtractCorrelationDataFromHeaders(request.Headers);
if (correlationData != null)
{
context.InitializeLogCorrelation(correlationData);
}
return correlationData;
}
public static void InitializeLogCorrelation(this ExecutionContext context, CorrelationData correlationData)
{
Activity activity = new Activity(context.FunctionName);
activity.SetParentId(correlationData.ExternalOperationParentId);
activity.Start();
}
private static CorrelationData ExtractCorrelationDataFromHeaders(HttpRequestHeaders requestHeaders)
{
const string RequestIdHeaderName = "Request-Id";
if (!requestHeaders.TryGetValues(RequestIdHeaderName, out IEnumerable<string> requestIDs)
|| string.IsNullOrEmpty(requestIDs?.FirstOrDefault()))
{
return null;
}
return new CorrelationData { ExternalOperationParentId = requestIDs.First() };
}
}
public class CorrelationData
{
public string ExternalOperationParentId { get; set; }
}
This obviously only works for azure functions using http triggers. When using a different trigger, you need to pass the correlation data in different way. Especially with Durable Functions, I think correlation is absolutely essential. You can, use a small wrapper class when passing information between functions, which includes the correlation data as well as the actual payload, e.g.
[FunctionName("httpFunction")]
public static async Task<HttpResponseMessage> HttpStartImport(
[HttpTrigger(AuthorizationLevel.Function, "post")] HttpRequestMessage req,
[OrchestrationClient]DurableOrchestrationClient orchestrationClient,
ExecutionContext context)
{
CorrelationData correlationData = context.InitializeLogCorrelationFromHeaders(req);
string content = await req.Content.ReadAsStringAsync();
Payload payload = JsonConvert.DeserializeObject<Payload>(content);
var dataContainer = new WorkflowDataContainer<ImportData>(correlationData, payload);
string instanceId = await orchestrationClient.StartNewAsync("orchestrateFunction", dataContainer);
return orchestrationClient.CreateCheckStatusResponse(req, instanceId);
}
[FunctionName(orchestrateFunction)]
public static async Task Orchestrator(
[OrchestrationTrigger] DurableOrchestrationContext orchestrationContext,
ExecutionContext context)
{
var dataContainer = orchestrationContext.GetInput<WorkflowDataContainer<ImportData>>();
context.InitializeLogCorrelation(dataContainer.CorrelationData);
...
}
public class WorkflowDataContainer<T>
{
public CorrelationData CorrelationData { get; set; }
public T Content { get; set; }
public WorkflowDataContainer() { }
public WorkflowDataContainer(CorrelationData correlationData, T content)
{
this.CorrelationData = correlationData;
this.Content = content;
}
}
Just be aware that you need to this for every function!
Links
Telemetry correlation in Application Insights