Integration Of Log4net And Nunit

Published Date: 02 Nov 2017

Abstract— This paper describes the Log4NUnit Library and points its issues which are identified during its integration. NUnit is very similar to JUnit in all that cases are built directly to the code of the project. Log4NUnit provides the capability of logging the test results along with testing classes. It also provides the capability to log the messages from within the testing methods. It supplies extension to NUnit’s Test class. It has built-in logging functionality, based on Log4net library, to document the test settings which is done at the time of testing, test case execution and it’s results. Attractive feature of Log4Nunit library is that it redirects the log messages to console if Log4net library is not present in the classpath at runtime. This makes it easy for users who do not want to download Log4net and put it in their classpath. Log4NUnit instantiates a Log4net logger instance and implements utility logging methods such as info, debug, etc. Developer just needs to call the method in order to log messages according to required log levels. This work was prompted due to lack of published results concerning the implementation of Log4NUnit. This paper portrays our experiences and lists benefits & limitations of Log4NUnit & several recommendations are outlined for future work on it.

Keywords— Document test results, NUnit, Log4NUnit, logging, log4net.

Introduction

Log4NUnit is an OpenFuture Project [1]. As described by Bill Hamiltion [2], Log4NUnit is a NUnit extension combining NUnit [3] with Log4net [4] as shown in figure 1. The purpose behind integrating such an extension is the necessity to create test protocols for NUnit. NUnit requires only testcase and one more test fixtures. Successful execution of a test case is not further documented. To keep track between the testing processes and to monitor the performance of test cases, successful execution of a test case, results of successful testing and bugs which are identified during the testing should be further documented. So the basic intention behind building Log4NUnit is the necessity to create test protocols in order to document the initial test settings, the test case execution and its results. Along with the testing framework(NUnit testing) a logging component(Log4net) is required to document the results of test case executions. One of the greatest logging libraries out there for .net is log4net.In addition to the core feature available in log4net some advance features are also there like ADO.net, LINQ queries and .net Trace and debug API. The rest of the paper is organized as follows. In section 2, the paper describes the background knowledge required to understand the Log4NUnit structure. Section 3 will demonstrates how to make use of it in different programs (for code and detailed steps please contact corresponding author of the paper). In Section 4, emphasis is given to the limitations & in section 5 various recommendations are outlined for future work on it. In the end section 6 illustrates the conclusions of this paper and section 7 consists of acknowledgments.

Figure 1: Log4NUnit as a Integration of Log4net and NUnit

BACKGROUND WORK

According to Michael Stal, Log4NUnit is a ready-made solution for logging messages from within the test cases [5]. It provides us with an extension of NUnit’s Test class. At the time of production, you cannot debug your application and sometimes errors that appear in the production server don't appear in the development or in preProd server. So if you try to get the error to be solved, you need to find a way that will help you in getting the error and tracing your application at production time without stopping or using debug mode in the application. Log4net is a tool to help the programmer output log statements to a variety of output targets in production, PreProd or development server. Here is the explanation how to install and configure this tool to work with ASP.NET 2.0, 3.0, 3.5 using C#.

Using log4net will help you to collect errors in many ways:

FileAppender (collect messages in file)

SMTPAppender (collect messages in email)

ADONetAppender (collect messages in database)

EventLogAppender (collect messages in event viewer)

There are several log levels supported by log4net library which are

OFF - nothing gets logged (cannot be called)

FATAL

ERROR

WARN

INFO

DEBUG

ALL - everything gets logged (cannot be called)

Above logging methods are provided in the Log4net library for various log levels. Here the log method itself is not provided; instead each log level method need to be used directly. It can be said that the design pattern known as Adapter [6] is used to design this class. It uses a concept where Object Adapter relies on object composition as shown in Figure 2. Setup and Teardown methods are also provided.

Figure 2: Object Adapter design pattern diagram from [6]

In NUnit, preparation for testing is called setup. It’s independent of the assertions, so the same scaffolding can apply to several individual tests. The setup is run before each test. When a test ends, the scaffolding might require some cleanup actions.

INTEGRATION OF LOG4NET AND NUNIT

Lots of people just run headlong into an automation project and immediately start writing scripts. I experience, that is a bit rash as there are some infrastructure bits that need to take care of first.

The first thing is to figure out where the scripts will be stored in version control. Everything should be in version control for automation. (If anyone has recommended solutions for automation consultants who don’t always have access to client repos but write code.) This should be the first step.

The next step is to identify logging strategy. The reason for this is two-fold. First, it gives you a means of debugging the script in a clean manner and second, it allows you to create the record of test execution, result and test failure details for latter reference.

Log4Net is a widely used framework for logging and an application has to configure it in order for your instructions, such as where to log your message, what to ignore, etc, are carried out. However, there are several quirks that one has to be aware about, particularly when we are using it with NUnit. I am currently using NUnit version 2.5.7 and log4net version 1.2.10 and hence the configuratoin below are relevant to this version combination.

However, when used in NUnit, report has been filed which does not generate any log file. In other word, the combination fails to find the required configuration file. The report accused NUnit being the culprit but I can assure that it is not entirely. This triggers a number of unnecessary works around.

Typically, an assembly can include any custom assembly-level attribute like Log4Net, XmlConfiguratorAttribute, for specifying the configuration information. In a situation where we need to load up the configuration file, log4net internal logic will look for an instance of this custom attribute in a given assembly for the configuration file, etc.

This class is located in src/Config/XmlConfiguratorAttribute.cs.

In theory, each assembly can be used to specify different configuration file. log4net does not process or configure its system until the first reference to create a logger or an appender. But in practice the first logger created determines the configuration file used and subsequent calls to create different loggers will not load the config file specified in that assembly. When NUnit is launched with log4net and before running your test assemblies, code inside NUnit causes NUnit.Core.Log4NetCapture.StartCapture() to execute. This is an override method of its parent abstract class TextCapture. The result is that will causes log4net.Appender.TextWriteAppender to be created.When this happens, somehow it triggers a call to log4net.Config.BasicConfigurator() for this appender. This in turn causes the creation of Logger Repository for the assembly nunit.core, which in turns starts looking for custom attributes for repository.

To use Log4NUnit follow the guide lines given below.

Include the assembly attribute for log4net located in src/Config/XmlConfigurationAttribute.cs

Add log4net reference to your vs project

Add log4net bits to the script

Next is to configure the log4net for that you need to add new configuration file to your projectlog4net.config

As things stand right though, log4net will never find this file as it expects it by default in an app.config or web.config. So we need to explicitly add the log4net where it’s config can be found for that we add the given assembly to our Assembly info.cs file.

[assembly: log4net.Config.XmlConfigurator(ConfigFile = "log4net.config", Watch = true)]

Now you have integrated log4net into your project and you can send log messages to it by using log.Debug, log.Info, log.Error, etc. NUnit will be default capture for log4net messages as pass they along to the user if they are Error or higher — regardless of what log4net says. So in this case we need another Application Configuration File. But if you run your NUnit scripts from the NUnit GUI you can specify the config file you want.

If Everything goes right with Integrating the Log4net and Nunit testing we have created the Log4NUnit Library which gives all the logging facility like logging Initial test setting, test results and error result can be logged in a text format.

LIMITATIONS OF LOG4NUNIT

A. Log4NUnit requires user Setuptest to be extended from LoggingTestCase. Many users may feel that they are forced into extending a class which is not very importance in their software product. If users want to extend their base test class from other class to provide some other capability (along with the capability to Log the Test case results) then they cannot do so since they would have already extended the LoggingTestCase class.

B. Log4NUnit supports only Log4net library for logging. Although Log4net is a very flexible and powerful logging framework, sometimes users may need to use other logging implementations like SmartInscept, Nlog, ObjectGayFramework or commons logging library as per the requirements.

C. Log4NUnit was designed to work with NUnit version 2.5.7, and so isn't compatible with the new syntax of NUnit. Since in new syntax of NUnit programmers do not need to extend the Setuptest class.

D. Log4NUnit is tightly coupled with its underlying logging framework. (the defacto standard – Log4net)

RECOMMENDATIONS FOR FUTURE WORK ON

A. Log4Unit is an open source and small library, so developers can customize it to support the logging implementation of their choice i.e. If any programmer wants to use any other logging framework, for example take SmartInspect [9], than they can change the Log4NUnit source and create their own library providing customized logging backend. Programmers can abstract the logging implementation using a different bridge such as Nlog Logging framework.

B. If developers plan to write more tests, it is recommend that they extend a base Setuptest of their own from LoggingTestCase and extend all your other tests from that base Setuptest. This way if Log4NUnit is removed or replaced, they only have to change the base setuptest and not each and every TestCase classes.

C. Another characteristic of Log4NUnit is that should be looked into is that, it becomes tightly coupled with the underlying logging framework (the default Log4net or the customized one, whichever the user implements). To overcome these inflexible situation programmers can use a logging framework like ObjectGay Framework [10]. ObjectGay will serve as a simple logging framework or abstraction for various logging framework, allowing the end user to plug in the desired logging framework at deployment time. This can provide a loose coupling of Log4NUnit with the underlying logging framework.

D. Scarcity of documentation on Log4NUnit makes it difficult for programmers and research students to understand its structure and take a look at the Software engineering concepts used in its structure design. So a lot of work needs to be done for creating the class diagrams and other documentation on this topic so that it can be used for further research.

Conclusions

This paper presented the Log4NUnit library whose focus is to create test protocols for NUnit testing framework. It is being observed that the library is asymmetrical in the sense that it focuses on the documentation of test failures and errors. It is also tightly coupled with the underlying logging framework log4net. This library is very much useful in small scale projects that need to startup immediately with a Testing mechanism together with a logging framework. It also reduces the efforts of programmer by maintaining a separate logging framework. This library is already been used by some developers and had proved useful to small scale projects. Since the Library is open source and it can also be customized to support the business requirements of individual organizations. Programmers can abstract its logging implementation by using a SmartInject or Nlog so as to decouple it from the logging library, thus making it independent of underlying logging framework.

Acknowledgment

Authors of this paper appreciate the original author of the Log4NUnit library, for providing his invaluable time to review the ideas and also answer the queries promptly. Although this library is open source, this research work began after obtaining prior consent of its original author. Authors of this paper would also thank their colleagues, friends, classmates, teachers and other guides for providing their immense support and helpful comments to improve this paper.