Creating your own Pattern Layout Converter for Log4net

Pattern Layout Converter is the way you tell log4net how to log something that it doesn’t know yet.

You first create your class that will get the information, for instance, this class will is to log the machine name:

using System;
using System.IO;
using log4net.Layout.Pattern;

namespace MyApplication.Logging
{
    public class MachinePatternConverter : PatternLayoutConverter
    {
        protected override void Convert(TextWriter writer, log4net.Core.LoggingEvent loggingEvent)
        {
            writer.Write(Environment.MachineName);            
        }
    }
}

Then, you edit your config file adding the converter you just created:

  <log4net>
    <appender name="Application" type="log4net.Appender.FileAppender">
      <file value="log\application.log" />
      <appendToFile value="true" />
      <maximumFileSize value="1024KB" />
      <layout type="log4net.Layout.PatternLayout">        
        <converter>
          <name value="machine" />
          <type value="MyApplication.Logging..MachinePatternConverter" />
        </converter>        
        <conversionPattern value="%date [%thread] %level %logger %machine - %message e:%exception%newline %newline" />
      </layout>
    </appender>
<!-- lines removed for brevity -->

In the code above, we added our new converter and named it “machine”. Then, In the conversionPattern line, we can use it wherever we want by adding the %machine variable.

Here are others converters you may want to use:

    public class IPPatternConverter : PatternLayoutConverter
    {
        protected override void Convert(TextWriter writer, log4net.Core.LoggingEvent loggingEvent)
        {
            if (HttpContext.Current != null)
            {
                writer.Write(HttpContext.Current.Request.UserHostAddress);
            }
        }
    }
    public class UrlPatternConverter : PatternLayoutConverter
    {
        protected override void Convert(TextWriter writer, log4net.Core.LoggingEvent loggingEvent)
        {
            if (HttpContext.Current != null)            
                writer.Write(HttpContext.Current.Request.Url.AbsoluteUri);
        }
    }
    public class UserAgentPatternConverter : PatternLayoutConverter
    {
        protected override void Convert(TextWriter writer, log4net.Core.LoggingEvent loggingEvent)
        {
            if (HttpContext.Current != null)
            {
                writer.Write(HttpContext.Current.Request.UserAgent);
            }
        }
    }

Now, applying all converter in our config file:

  <log4net>
    <appender name="Application" type="log4net.Appender.FileAppender">
      <file value="log\application.log" />
      <appendToFile value="true" />
      <maximumFileSize value="1024KB" />
      <layout type="log4net.Layout.PatternLayout">
        <converter>
          <name value="url" />
          <type value="MyApplication.Logging.UrlPatternConverter" />
        </converter>
        <converter>
          <name value="ip" />
          <type value="MyApplication.Logging.IPPatternConverter" />
        </converter>
        <converter>
          <name value="machine" />
          <type value="MyApplication.Logging.MachinePatternConverter" />
        </converter>
        <converter>
          <name value="userAgent" />
          <type value="MyApplication.Logging.UserAgentPatternConverter" />
        </converter>
        <conversionPattern value="%date [%thread] %level %logger %url %ip %machine %userAgent - %message e:%exception%newline %newline" />
      </layout>
    </appender>

Do you have any interest Pattern Layout Converter you use in your application?

Logging to multiple files – Log4Net

I am going to explain how to log using Log4Net and how to separate the different types of logs from your application into different output files. The main focus of this post is not explain how log4net works in detail, however if your are interesting about learning more about it I recommend this article http://www.codeproject.com/KB/dotnet/Log4net_Tutorial.aspx.

The purpose here is to separate the log by source, for instance, caching log in one file, queries in another file, application log in another one, and also a single file for logging errors from all sources, this way we can find errors easier. The errors will still be logged in the library’s log though, because we may want to focus on that library specifically.

Take a look at this configuration file and will explain where the separation comes up later:

<?xml version="1.0" encoding="utf-8" ?>
<log4net>
  <appender name="Application" type="log4net.Appender.FileAppender">
    <file value="log\application.log" />
    <appendToFile value="true" />
    <maximumFileSize value="1024KB" />
    <layout type="log4net.Layout.PatternLayout">
      <conversionPattern value="%date [%thread] %-5level %logger [%property{NDC}] - %message%newline" />
    </layout>
  </appender>

  <appender name="ErrorLog" type="log4net.Appender.RollingFileAppender">
    <file value="log\error.log" />
    <rollingStyle value="Date" />
    <appendToFile value="true" />
    <datePattern value="ddMMyyyy" />
    <maxSizeRollBackups value="10" />
    <filter type="log4net.Filter.LevelRangeFilter">
      <levelMin value="ERROR"/>
    </filter>
    <layout type="log4net.Layout.PatternLayout">
      <conversionPattern value="%date [%thread] %-5level %logger [%property{NDC}] - %message%newline" />
    </layout>

  </appender>

  <appender name="AdoNetAppender" type="log4net.Appender.AdoNetAppender">
    <bufferSize value="1" />
    <connectionType value="System.Data.SqlClient.SqlConnection, System.Data, Version=1.0.3300.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" />
    <connectionString value="data source=127.0.0.1;initial catalog=Application_Log_Database;integrated security=false;persist security info=True;User ID=abc123;Password=abc123" />
    <!-- AppId parameter comes from the table Applications -->
    <commandText value="INSERT INTO application_log ([date],[thread],[level],[logger], [message], [exception]) VALUES (@log_date,@thread,@log_level,@logger, @message,@exception)" />

    <filter type="log4net.Filter.LevelRangeFilter">
      <levelMin value="ERROR"/>
      <levelMax value="FATAL"/>
    </filter>

    <parameter>
      <parameterName value="@log_date" />
      <dbType value="DateTime" />
      <layout type="log4net.Layout.RawTimeStampLayout" />
    </parameter>

    <parameter>
      <parameterName value="@thread" />
      <dbType value="String" />
      <size value="255" />
      <layout type="log4net.Layout.PatternLayout">
        <conversionPattern value="%thread" />
      </layout>
    </parameter>

    <parameter>
      <parameterName value="@log_level" />
      <dbType value="String" />
      <size value="50" />
      <layout type="log4net.Layout.PatternLayout">
        <conversionPattern value="%level" />
      </layout>
    </parameter>

    <parameter>
      <parameterName value="@logger" />
      <dbType value="String" />
      <size value="255" />
      <layout type="log4net.Layout.PatternLayout">
        <conversionPattern value="%logger" />
      </layout>
    </parameter>

    <parameter>
      <parameterName value="@message" />
      <dbType value="String" />
      <size value="4000" />
      <layout type="log4net.Layout.PatternLayout">
        <conversionPattern value="%message" />
      </layout>
    </parameter>

    <parameter>
      <parameterName value="@exception" />
      <dbType value="String" />
      <size value="4000" />
      <layout type="log4net.Layout.ExceptionLayout" />
    </parameter>    

  </appender>

  <appender name="OpenRasta" type="log4net.Appender.FileAppender">
    <file value="log\openrasta.log" />
    <appendToFile value="true" />
    <maximumFileSize value="1024KB" />
    <layout type="log4net.Layout.PatternLayout">
      <conversionPattern value="%date [%thread] %-5level %logger [%property{NDC}] - %message%newline" />
    </layout>
  </appender>

  <appender name="Messaging" type="log4net.Appender.FileAppender">
    <file value="log\messaging.log" />
    <appendToFile value="true" />
    <maximumFileSize value="1024KB" />
    <layout type="log4net.Layout.PatternLayout">
      <conversionPattern value="%date [%thread] %-5level %logger [%property{NDC}] - %message%newline" />
    </layout>
  </appender>

  <appender name="Queries" type="log4net.Appender.FileAppender">
    <file value="log\queries.log" />
    <appendToFile value="true" />
    <maximumFileSize value="1024KB" />
    <layout type="log4net.Layout.PatternLayout">
      <conversionPattern value="%date [%thread] %-5level [%property{NDC}] - %message%newline" />
    </layout>
  </appender>

  <appender name="Memcached" type="log4net.Appender.FileAppender">
    <file value="log\memcached.log" />
    <appendToFile value="true" />
    <maximumFileSize value="1024KB" />
    <layout type="log4net.Layout.PatternLayout">
      <conversionPattern value="%date [%thread] %-5level [%property{NDC}] - %message%newline" />
    </layout>
  </appender>

  <root>
    <level value="All" />
    <appender-ref ref="ErrorLog" />
    <appender-ref ref="AdoNetAppender" />
  </root>

  <logger name="Log4NetExample">
    <level value="INFO" />
    <appender-ref ref="Application" />
  </logger>

  <logger name="OpenRasta">
    <level value="INFO" />
    <appender-ref ref="OpenRasta" />
  </logger>

  <logger name="Spring">
    <level value="WARN" />
    <appender-ref ref="Messaging" />
  </logger>
  <logger name="NServiceBus">
    <level value="WARN" />
    <appender-ref ref="Messaging" />
  </logger>

  <logger name="System.Data.Linq">
    <level value="DEBUG" />
    <appender-ref ref="Queries" />
  </logger>

  <logger name="Enyim.Caching.Memcached">
    <level value="WARN" />
    <appender-ref ref="Memcached" />
  </logger>

</log4net>

Don’t be scared! It may look big, but that is because I’m showing a configuration with a lot of libraries and the AdoNetAppender is a little big too. What this configuration is doing is creating the Appenders, setting the root section, and then setting the level things will be logged with the loggers section.

The Appenders are most of them logging into text files, and one is logging into the database. In order to have the errors from all sources, notice we are adding the ErrorLog appender and the AdoNetAppender in the root section. This is because all the loggers must inherit those two. All the loggers will log their errors in a text file and in the database:

<root>
	<level value="All" />
	<appender-ref ref="ErrorLog" />
	<appender-ref ref="AdoNetAppender" />
</root>

The root’s log level is set to “All“, so if you don’t specify any level to a logger, it will inherit from the root and its value will be All as well.

If you look at those two appenders we are filtering their minimum level types as ERROR in the appender itself.

<appender name="ErrorLog" type="log4net.Appender.RollingFileAppender">    
	<filter type="log4net.Filter.LevelRangeFilter">
	  <levelMin value="ERROR"/>
	</filter>
...

This is how we carry the error loggers along with the other loggers but logging only the errors. The libraries are filtered by their namespace, for instance Enyim.Caching.Memcached, and each library has its own logger set with a specific level of logging.

Log4net is a well known library and most of other libraries support it by default. For those libraries that doesn’t, for example, LinqToSql, a quick search on Google can give you the code to make it supported.