C# Logging Best Practices to Improve Troubleshooting

Designing methods with a focus on logging and exception handling is crucial for building maintainable and debuggable software. In this post, we’ll discuss some C# logging best practices about designing method in a way that gives us the most level of details. A lot of time we see that an error happens in our software, but we don’t have enough information about what the root cause it and at what stage the error is happening. What if we could follow some best practices that gives that level of information for the beginning? How should we think about logging and exception handling in our design to improve debugging and troubleshooting in the future ? Here are some best practices you can follow to achieve a design that provides maximum detail and clarity when issues arise.

Structured Logging

Log Contextual Information: Include relevant contextual details that can help identify the source of the problem, such as method parameters (e.g., Message ID, User ID), execution context (e.g., current tenant or service name), and any other relevant state. For instance, in the GetMime method, logging the message ID provides essential context to track down which request failed.
Use Log Levels Appropriately:
- Information: For general operational logs that indicate the flow of the application (e.g., “Starting GetMime method”).
- Warning: When something is off but not necessarily a failure (e.g., a deprecated API usage).
- Error: For exceptions or failures that prevent the method from completing successfully.
- Debug/Trace: For detailed debugging information, especially during development. This can include more granular logs for each step inside the method.
Use Structured Logging: If you are using a logging framework like Serilog or NLog, take advantage of structured logging (logging key-value pairs instead of just plain text). This allows logs to be easily queried, filtered, and analyzed. For example:
```
_logger.LogError("Failed to get MIME content. Status Code: {StatusCode}, Message ID: {MessageId}", ex.StatusCode, messageId);
```
This approach provides better querying capabilities in log aggregation tools.

Use Try-Catch for Exception Handling Strategically

Catch Specific Exceptions Where Possible: If you know specific exceptions can be thrown, catch them explicitly to provide more detailed error handling. For example, catch ServiceException separately from a general Exception to handle Graph-specific issues differently from unexpected errors.
Log the Exception Details: Include information such as the stack trace, error message, and inner exception (if any). This provides a complete picture of what went wrong.
Avoid Swallowing Exceptions Silently: If an exception is caught, it should be logged appropriately, even if it’s re-thrown or converted into a different exception.
Use Exception Filters When Necessary: In C#, exception filters allow you to log or perform other actions based on the exception type or condition without actually catching it:
```
catch (ServiceException ex) when (ex.StatusCode == HttpStatusCode.NotFound)
{
    _logger.LogWarning("Resource not found. Message ID: {MessageId}", messageId);
    throw;
}
```

Design for Clarity and Separation of Concerns

Separate Concerns in Your Methods: If a method does multiple things (e.g., creating a client and making a request), consider separating them into smaller methods. This makes it easier to log and handle exceptions at more granular levels.
Avoid Overly Long Methods: Break down long methods into smaller, more manageable pieces. For example, separate the code that sets up the GraphServiceClient from the code that makes the request. This helps you focus on each part separately and improves the ability to add detailed logging.

Use Correlation IDs for Tracing Across Logs

Generate a Correlation ID: Pass a correlation ID through the method calls, which can be used to trace a single request across multiple components or services. If you are working with microservices or distributed systems, this is especially valuable for tracing requests across services.
Log the Correlation ID: Include it in your logs to ensure that all logs related to a single request can be easily identified and grouped together.

Add Logging Around External Dependencies

Log Before and After Calling External Services: When making calls to external dependencies like the Microsoft Graph API, log before and after the call. This helps to understand if the problem is occurring before reaching the external service, during the call, or in the handling of the response.
Log Details About the Request: For external service calls, log relevant information about the request, such as:
- The endpoint being called
- Any parameters or identifiers
- Authentication-related information (without sensitive data)
Log Response Details (Without Sensitive Data): If the response from an external service contains useful information for debugging (e.g., HTTP status code, response headers), log these details while avoiding logging sensitive information like authentication tokens or user data.

Fail Gracefully and Provide Meaningful Error Messages

Wrap Exceptions with Custom Messages When Re-throwing: If you catch an exception and need to add more context, wrap it with a custom exception or add a meaningful error message. For example:
```
catch (ServiceException ex)
{
    throw new CustomApplicationException($"Failed to retrieve MIME content for Message ID: {messageId}", ex);
}
```
Return User-Friendly Error Messages: If this code is part of an API, ensure that the error messages returned to the client are user-friendly and do not expose sensitive internal details. You can still log the detailed error message internally for debugging.

Implement Retry Logic for Transient Failures

Use a Library like Polly: To automatically retry transient errors (e.g., network issues, timeouts), use a library such as Polly. This allows you to implement retry policies (with exponential backoff) for specific types of exceptions or status codes.
Log Each Retry Attempt: When a retry is performed, log the attempt count and the reason for retrying.

Test Your Logging and Error Handling

Unit Tests for Error Handling: Write unit tests to ensure that your logging and error handling behave as expected. For example, verify that specific exceptions are logged at the correct log level.
Simulate Failure Scenarios: Test how your method behaves when exceptions are thrown. Make sure it logs the necessary information and handles the exception as expected.

Example Refactored Code Based on Best Practices

Imagine we have a method called GetMime that looks like the following and we have problem troubleshooting:

public async Task<Stream> GetMime(string azureAccessToken, string id)
{
    var graphServiceClient = CreateGraphServiceClient(azureAccessToken);
    var stream = await graphServiceClient.Me.Messages[id].Content.Request().GetAsync();

    return stream;
}

Here’s how we might refactor the original GetMime method to follow these best practices :

public async Task<Stream> GetMime(string azureAccessToken, string id)
{
    logger.LogInformation("Starting GetMime method. Message ID: {MessageId}", id);

    try
    {
        var graphServiceClient = CreateGraphServiceClient(azureAccessToken);

        logger.LogInformation("GraphServiceClient created successfully. Fetching MIME content for message ID: {MessageId}", id);

        // Make the request to get the MIME content
        var stream = await graphServiceClient.Me.Messages[id].Content.Request().GetAsync();

        logger.LogInformation("Successfully retrieved MIME content for message ID: {MessageId}", id);

        return stream;
    }
    catch (ServiceException ex)
    {
        logger.LogError(ex, "ServiceException occurred in GetMime method. Status Code: {StatusCode}, Message: {ErrorMessage}, Request ID: {RequestId}",
            ex.StatusCode,
            ex.Error?.Message,
            ex.Error?.ClientRequestId);

        if (ex.InnerException != null)
        {
            logger.LogError(ex.InnerException, "Inner exception: {InnerExceptionMessage}", ex.InnerException.Message);
        }

        throw;
    }
    catch (Exception ex)
    {
        logger.LogError(ex, "An unexpected error occurred in GetMime method. Message ID: {MessageId}", id);

        throw;
    }
}

Summary

Log contextual details to provide more information about the request.
Use appropriate log levels for different types of messages.
Catch and handle exceptions thoughtfully, logging relevant details without exposing sensitive information.
Fail gracefully and provide meaningful error messages.
Use correlation IDs and structured logging for better traceability.
Implement retry logic for transient errors and log each attempt.
Test your logging and error handling to ensure they work as expected.

By following these best practices, you can design methods that offer comprehensive insight into the application’s behavior, making debugging and troubleshooting more efficient.

Hamid Mosalla

Programming Adventures