Structured Logging
Here’s a method we had in production. It downloads the raw MIME content of an email from Microsoft Graph:
public async Task<Stream> GetMime(string azureAccessToken, string id)
{
var graphServiceClient = CreateGraphServiceClient(azureAccessToken);
var stream = await graphServiceClient.Me.Messages[id].Content.Request().GetAsync();
return stream;
}
It’s short and readable, and when it failed it told us almost nothing. The log had an exception and a stack trace, and that was it. We couldn’t see which message ID had been requested, whether the failure came from building the client or from the Graph call, or what Graph actually returned. Troubleshooting meant reproducing the problem by hand.
This post covers the C# logging and exception handling practices we now design into methods like this from the start. They include structured log messages, log levels that mean something, context that follows the request, exceptions that are logged once instead of five times, and retries you can see in the logs. At the end, we rewrite GetMime using all of them.
Write log messages as templates, not strings
Structured logging stores each value in a log message as its own field instead of baking it into a sentence. ILogger supports this natively, and Serilog, NLog and OpenTelemetry all pass the fields through to your log store. Then you can run a query like “every failure for MessageId X” or “count of errors by StatusCode” in Application Insights, Seq or Elasticsearch without writing a regex.
The easiest way to lose that structure is string interpolation:
// The log store receives one opaque string
logger.LogError(ex, $"Failed to get MIME content for {messageId}");
// MessageId is kept as a field you can filter and group on
logger.LogError(ex, "Failed to get MIME content for {MessageId}", messageId);
The analyzer rule CA2254 warns about the first form. Enable it and treat it as an error. Name placeholders consistently across the codebase as well. If one class logs {MessageId} and another logs {MsgId}, no single query will find both.
For code that logs a lot, the LoggerMessage source generator (.NET 6 and later) generates the logging method at compile time. It skips string formatting and boxing when the level is disabled, and it gives every call site the same template:
public static partial class MailLog
{
[LoggerMessage(Level = LogLevel.Debug, Message = "Requesting MIME content for {MessageId}")]
public static partial void RequestingMime(this ILogger logger, string messageId);
}
Choosing log levels
Log levels decide what you pay to store and what wakes someone up. These are the definitions we use:
- Trace: very detailed data, such as payloads and intermediate values. Never enable it in production.
- Debug: the steps inside a method, such as “Requesting MIME content” or “Received response”. It’s off in production by default, and you turn it on for one category when you’re investigating.
- Information: events worth counting, such as an exported message, a completed job or an application start. “Entering GetMime” isn’t one of them. In a service that processes thousands of messages an hour, method-entry logs at Information level add up to a large ingestion bill.
- Warning: something unexpected happened, but the code handled it. Examples are a retry that succeeded, a fallback to cached data, or a call to a deprecated API.
- Error: the operation failed and someone may need to look at it.
- Critical: the application can’t keep running, for example when the database is unreachable at startup.
Set levels per category in appsettings.json. A common setup is Information as the default and Warning for Microsoft and System, so the framework’s own messages don’t drown out yours.
Put context on every line with scopes and trace IDs
A log line without context leaves you guessing. You want the message ID, the user or tenant, the service name, and anything else that tells you which request failed. Instead of adding those to every template, open a logging scope once at the start of the operation:
using var scope = logger.BeginScope(new Dictionary<string, object>
{
["MessageId"] = messageId,
["TenantId"] = tenantId
});
Every log written inside that block, including logs from the methods it calls, now carries both fields. The console provider needs IncludeScopes turned on to display them. Serilog needs Enrich.FromLogContext(), and the OpenTelemetry logger needs IncludeScopes = true.
Correlation IDs are mostly handled for you in modern .NET. ASP.NET Core and HttpClient use Activity and the W3C traceparent header, so one trace ID follows a request from the incoming call through every outgoing HTTP call. The default host already adds TraceId and SpanId to log scopes, and with OpenTelemetry you can follow a request across services in a single view. Where you do have to handle it yourself is anything that isn’t HTTP. When you put work on a queue, copy the trace context into the message properties. Most Azure SDKs, including Service Bus, do that automatically.
Log an exception once
The most common logging mistake we find in code reviews is catch, log, rethrow, repeated at every layer. A single failure then shows up as five error entries with five stack traces, and only one of them, the deepest, has the context you need. A better rule is this: log an exception where you handle it, and below that point, add context and let it go.
- Catch only the exceptions you can do something useful with, such as translating them, adding context or recovering. Leave everything else alone. A broad
catch (Exception)belongs at the boundary: ASP.NET Core’s exception handler (which already logs unhandled exceptions), the loop in a background worker, or a message handler. - Rethrow with
throw;, neverthrow ex;. The second form resets the stack trace to the current line, so you lose where the error started. - When you wrap an exception, pass the original as the inner exception and put the identifiers in the message.
logger.LogError(ex, ...)writes the whole chain, inner exceptions included, so you don’t have to log the inner exception separately. - Never swallow an exception silently. If an empty catch block is truly correct, add a comment explaining why.
- Cancellation isn’t a failure. When an
OperationCanceledExceptionhappens because the caller’s token was cancelled, let it propagate, or log it at Debug level. Don’t log it as an error.
Logging from an exception filter
C# exception filters (catch ... when) run before the stack unwinds, which allows a trick Stephen Cleary describes. The filter logs the exception and returns false, so the catch block never runs and the exception keeps propagating. Because the stack hasn’t unwound yet, every logging scope opened deeper in the call stack is still active when the log is written. The MessageId scope from inside GetMime appears on an error logged two layers up:
try
{
await ProcessMailboxAsync(mailboxId, ct);
}
catch (Exception ex) when (LogAndPropagate(ex))
{
throw; // never runs: the filter returns false
}
bool LogAndPropagate(Exception ex)
{
logger.LogError(ex, "Processing mailbox {MailboxId} failed", mailboxId);
return false;
}
This works best at the boundary, where you want one log entry with all the context.
Log calls to external services
When a call to an external service fails, you need to know whether it failed before the request left, while it was in flight, or while you were handling the response. You don’t need to write most of those log lines yourself:
- Clients created by
IHttpClientFactorylog the start and end of each request, with the elapsed time, under theSystem.Net.Http.HttpClient.{name}categories. OpenTelemetry’s HTTP instrumentation records each call as a span with its URL, status code and duration. - Log the identifiers the other side will ask for. Every Microsoft Graph response includes a
request-idheader, and the SDK sends aclient-request-idwith each request. Microsoft support asks for both, plus a timestamp. - Log the status code, the service’s error code and the endpoint. Never log tokens, passwords or message content. The original
GetMimetakes a raw access token as a string parameter, so one careless{@Args}in a log template would put it in your logs. TheMicrosoft.Extensions.Compliance.Redactionpackage (.NET 8 and later) can mask fields you’ve classified as sensitive.
Retries for transient failures
Network blips, timeouts and 429 or 503 responses are worth retrying with exponential backoff and jitter. Polly v8 does this, and Microsoft.Extensions.Http.Resilience builds on it for HTTP clients with AddStandardResilienceHandler(). Whatever you use, log each retry, or you’ll never find out that every call to a service quietly takes three attempts:
var pipeline = new ResiliencePipelineBuilder()
.AddRetry(new RetryStrategyOptions
{
ShouldHandle = new PredicateBuilder().Handle<HttpRequestException>(),
MaxRetryAttempts = 3,
BackoffType = DelayBackoffType.Exponential,
UseJitter = true,
OnRetry = args =>
{
logger.LogWarning(args.Outcome.Exception,
"Retry {Attempt} after {Delay}", args.AttemptNumber + 1, args.RetryDelay);
return default;
}
})
.Build();
Check what your SDK already does before you add a retry policy. The Graph SDK includes a retry handler that respects Retry-After on 429, 503 and 504 responses. If you wrap your own three retries around its three, a single outage turns into up to sixteen requests.
What the caller sees
Detailed errors belong in your logs, not in the response. If GetMime sits behind an API, the client should get a clear Problem Details response with a status code, a short message and the trace ID, never the Graph error text or a stack trace. Support staff can paste that trace ID into the log search and find everything this post told you to log.
The refactored GetMime
Here’s the method again, with these practices applied. We moved it to Graph SDK v5, which replaced .Request().GetAsync() with GetAsync() and replaced ServiceException with ODataError. The structure is the same on v4.
public async Task<Stream> GetMimeAsync(string azureAccessToken, string messageId, CancellationToken ct = default)
{
using var scope = logger.BeginScope(new Dictionary<string, object> { ["MessageId"] = messageId });
var graphClient = CreateGraphServiceClient(azureAccessToken);
logger.LogDebug("Requesting MIME content from Microsoft Graph");
try
{
var stream = await graphClient.Me.Messages[messageId].Content.GetAsync(cancellationToken: ct);
logger.LogDebug("Received MIME content");
return stream ?? throw new InvalidOperationException($"Graph returned no content for message {messageId}.");
}
catch (ODataError ex) when (ex.ResponseStatusCode == (int)HttpStatusCode.NotFound)
{
throw new MessageNotFoundException(messageId, ex);
}
catch (ODataError ex)
{
throw new GraphRequestException(
$"Getting MIME content for message {messageId} failed with {ex.ResponseStatusCode} ({ex.Error?.Code}).",
ex.ResponseStatusCode,
ex.Error?.Code,
ex.Error?.InnerError?.RequestId,
ex);
}
}
The custom exception keeps the Graph details as properties, so whatever logs it at the boundary can write them out as fields:
public sealed class GraphRequestException(
string message, int statusCode, string? errorCode, string? requestId, Exception inner)
: Exception(message, inner)
{
public int StatusCode { get; } = statusCode;
public string? ErrorCode { get; } = errorCode;
public string? RequestId { get; } = requestId;
}
Compared with the first version of this refactor, which logged “Starting GetMime” and “GraphServiceClient created successfully” at Information level and logged every exception as an error before rethrowing it, this version is quieter and tells you more:
- The step-by-step messages are at Debug level, so they cost nothing in production until you need them.
- The message ID is in a scope, so it’s attached to every log line, including lines from
CreateGraphServiceClient. - A missing message becomes a domain exception the caller can handle, perhaps by skipping the message with a warning, instead of an error-level log entry.
- Every other Graph failure is wrapped once, with the status code, Graph’s error code and the request ID support will ask for. The method logs nothing at error level itself. The boundary does that, once.
There’s one more step we’d take that this example doesn’t show. GraphServiceClient is designed to be created once and reused. Registering it with dependency injection using a TokenCredential removes the token parameter completely. That makes the method easier to test, and there’s no token left to leak into a log.
Test the failure paths
Logging and error handling are code, and untested code drifts. Since GraphServiceClient accepts an HttpClient, you can give it a stub HttpMessageHandler that returns a 404 or a 503 and check what the method does:
[Fact]
public async Task Missing_message_becomes_MessageNotFoundException()
{
var logger = new FakeLogger<MailService>();
var service = new MailService(logger, GraphReturning(HttpStatusCode.NotFound));
var ex = await Assert.ThrowsAsync<MessageNotFoundException>(
() => service.GetMimeAsync("token", "msg-1"));
Assert.IsType<ODataError>(ex.InnerException);
Assert.DoesNotContain(logger.Collector.GetSnapshot(), r => r.Level >= LogLevel.Error);
}
FakeLogger comes from the Microsoft.Extensions.Diagnostics.Testing package (.NET 8 and later). It records every log entry with its level, message, structured state and scopes. That lets you assert the rules from this post directly, such as no error-level log at this layer and the message ID present in scope. In the test above, GraphReturning is a small helper you write that builds a GraphServiceClient around the stub handler.
Further reading
-
- Logging in C# and .NET | Microsoft Learn: configuration, providers, log levels, scopes and structured logging with
ILogger. - Compile-time logging source generation | Microsoft Learn
- Exceptions and exception handling in C# | Microsoft Learn:
try,catch,finally,throwand exception filters. - Best practices for exceptions | Microsoft Learn
- A New Pattern for Exception Logging | Stephen Cleary: the exception filter technique shown above, explained in full.
- C# Exception Handling Best Practices | Hamid Mosalla
- Logging in C# and .NET | Microsoft Learn: configuration, providers, log levels, scopes and structured logging with
- Polly documentation: retry, timeout and circuit breaker strategies.
- The Twelve-Factor App: Logs: why an app should write logs as an event stream and let the platform route them.
