Logging to multiple files – Log4Net

I am going to explain how to log using Log4Net and how to separate the different types of logs from your application into different output files. The main focus of this post is not explain how log4net works in detail, however if your are interesting about learning more about it I recommend this article http://www.codeproject.com/KB/dotnet/Log4net_Tutorial.aspx.

The purpose here is to separate the log by source, for instance, caching log in one file, queries in another file, application log in another one, and also a single file for logging errors from all sources, this way we can find errors easier. The errors will still be logged in the library’s log though, because we may want to focus on that library specifically.

Take a look at this configuration file and will explain where the separation comes up later:

<?xml version="1.0" encoding="utf-8" ?>
<log4net>
  <appender name="Application" type="log4net.Appender.FileAppender">
    <file value="log\application.log" />
    <appendToFile value="true" />
    <maximumFileSize value="1024KB" />
    <layout type="log4net.Layout.PatternLayout">
      <conversionPattern value="%date [%thread] %-5level %logger [%property{NDC}] - %message%newline" />
    </layout>
  </appender>

  <appender name="ErrorLog" type="log4net.Appender.RollingFileAppender">
    <file value="log\error.log" />
    <rollingStyle value="Date" />
    <appendToFile value="true" />
    <datePattern value="ddMMyyyy" />
    <maxSizeRollBackups value="10" />
    <filter type="log4net.Filter.LevelRangeFilter">
      <levelMin value="ERROR"/>
    </filter>
    <layout type="log4net.Layout.PatternLayout">
      <conversionPattern value="%date [%thread] %-5level %logger [%property{NDC}] - %message%newline" />
    </layout>

  </appender>

  <appender name="AdoNetAppender" type="log4net.Appender.AdoNetAppender">
    <bufferSize value="1" />
    <connectionType value="System.Data.SqlClient.SqlConnection, System.Data, Version=1.0.3300.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" />
    <connectionString value="data source=127.0.0.1;initial catalog=Application_Log_Database;integrated security=false;persist security info=True;User ID=abc123;Password=abc123" />
    <!-- AppId parameter comes from the table Applications -->
    <commandText value="INSERT INTO application_log ([date],[thread],[level],[logger], [message], [exception]) VALUES (@log_date,@thread,@log_level,@logger, @message,@exception)" />

    <filter type="log4net.Filter.LevelRangeFilter">
      <levelMin value="ERROR"/>
      <levelMax value="FATAL"/>
    </filter>

    <parameter>
      <parameterName value="@log_date" />
      <dbType value="DateTime" />
      <layout type="log4net.Layout.RawTimeStampLayout" />
    </parameter>

    <parameter>
      <parameterName value="@thread" />
      <dbType value="String" />
      <size value="255" />
      <layout type="log4net.Layout.PatternLayout">
        <conversionPattern value="%thread" />
      </layout>
    </parameter>

    <parameter>
      <parameterName value="@log_level" />
      <dbType value="String" />
      <size value="50" />
      <layout type="log4net.Layout.PatternLayout">
        <conversionPattern value="%level" />
      </layout>
    </parameter>

    <parameter>
      <parameterName value="@logger" />
      <dbType value="String" />
      <size value="255" />
      <layout type="log4net.Layout.PatternLayout">
        <conversionPattern value="%logger" />
      </layout>
    </parameter>

    <parameter>
      <parameterName value="@message" />
      <dbType value="String" />
      <size value="4000" />
      <layout type="log4net.Layout.PatternLayout">
        <conversionPattern value="%message" />
      </layout>
    </parameter>

    <parameter>
      <parameterName value="@exception" />
      <dbType value="String" />
      <size value="4000" />
      <layout type="log4net.Layout.ExceptionLayout" />
    </parameter>    

  </appender>

  <appender name="OpenRasta" type="log4net.Appender.FileAppender">
    <file value="log\openrasta.log" />
    <appendToFile value="true" />
    <maximumFileSize value="1024KB" />
    <layout type="log4net.Layout.PatternLayout">
      <conversionPattern value="%date [%thread] %-5level %logger [%property{NDC}] - %message%newline" />
    </layout>
  </appender>

  <appender name="Messaging" type="log4net.Appender.FileAppender">
    <file value="log\messaging.log" />
    <appendToFile value="true" />
    <maximumFileSize value="1024KB" />
    <layout type="log4net.Layout.PatternLayout">
      <conversionPattern value="%date [%thread] %-5level %logger [%property{NDC}] - %message%newline" />
    </layout>
  </appender>

  <appender name="Queries" type="log4net.Appender.FileAppender">
    <file value="log\queries.log" />
    <appendToFile value="true" />
    <maximumFileSize value="1024KB" />
    <layout type="log4net.Layout.PatternLayout">
      <conversionPattern value="%date [%thread] %-5level [%property{NDC}] - %message%newline" />
    </layout>
  </appender>

  <appender name="Memcached" type="log4net.Appender.FileAppender">
    <file value="log\memcached.log" />
    <appendToFile value="true" />
    <maximumFileSize value="1024KB" />
    <layout type="log4net.Layout.PatternLayout">
      <conversionPattern value="%date [%thread] %-5level [%property{NDC}] - %message%newline" />
    </layout>
  </appender>

  <root>
    <level value="All" />
    <appender-ref ref="ErrorLog" />
    <appender-ref ref="AdoNetAppender" />
  </root>

  <logger name="Log4NetExample">
    <level value="INFO" />
    <appender-ref ref="Application" />
  </logger>

  <logger name="OpenRasta">
    <level value="INFO" />
    <appender-ref ref="OpenRasta" />
  </logger>

  <logger name="Spring">
    <level value="WARN" />
    <appender-ref ref="Messaging" />
  </logger>
  <logger name="NServiceBus">
    <level value="WARN" />
    <appender-ref ref="Messaging" />
  </logger>

  <logger name="System.Data.Linq">
    <level value="DEBUG" />
    <appender-ref ref="Queries" />
  </logger>

  <logger name="Enyim.Caching.Memcached">
    <level value="WARN" />
    <appender-ref ref="Memcached" />
  </logger>

</log4net>

Don’t be scared! It may look big, but that is because I’m showing a configuration with a lot of libraries and the AdoNetAppender is a little big too. What this configuration is doing is creating the Appenders, setting the root section, and then setting the level things will be logged with the loggers section.

The Appenders are most of them logging into text files, and one is logging into the database. In order to have the errors from all sources, notice we are adding the ErrorLog appender and the AdoNetAppender in the root section. This is because all the loggers must inherit those two. All the loggers will log their errors in a text file and in the database:

<root>
	<level value="All" />
	<appender-ref ref="ErrorLog" />
	<appender-ref ref="AdoNetAppender" />
</root>

The root’s log level is set to “All“, so if you don’t specify any level to a logger, it will inherit from the root and its value will be All as well.

If you look at those two appenders we are filtering their minimum level types as ERROR in the appender itself.

<appender name="ErrorLog" type="log4net.Appender.RollingFileAppender">    
	<filter type="log4net.Filter.LevelRangeFilter">
	  <levelMin value="ERROR"/>
	</filter>
...

This is how we carry the error loggers along with the other loggers but logging only the errors. The libraries are filtered by their namespace, for instance Enyim.Caching.Memcached, and each library has its own logger set with a specific level of logging.

Log4net is a well known library and most of other libraries support it by default. For those libraries that doesn’t, for example, LinqToSql, a quick search on Google can give you the code to make it supported.

Advertisements

2 thoughts on “Logging to multiple files – Log4Net

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s