Improving Nunit Testing By Adding Log4net Library

Published Date: 02 Nov 2017

Abstract— This paper describes the Log4NUnit Library and points its issues which are identified during its integration. NUnit is very similar to JUnit in all that cases are built directly to the code of the project. Log4NUnit provides the capability of logging the test results along with testing classes. It also provides the capability to log the messages from within the testing methods. It supplies extension to NUnit’s Test class. It has built-in logging functionality, based on Log4net library, to document the test settings which is done at the time of testing, test case execution and it’s results. Attractive feature of Log4Nunit library is that it redirects the log messages to console if Log4net library is not present in the classpath at runtime. This makes it easy for users who do not want to download Log4net and put it in their classpath. Log4NUnit instantiates a Log4net logger instance and implements utility logging methods such as info, debug, etc. Developer just needs to call the method in order to log messages according to required log levels. This work was prompted due to lack of published results concerning the implementation of Log4NUnit. This paper portrays our experiences and lists benefits & limitations of Log4NUnit & several recommendations are outlined for future work on it.

Keywords— Document test results, NUnit, Log4NUnit, logging, log4net.

Introduction

Log4NUnit is an OpenFuture Project [1]. As described by Bill Hamiltion [2], Log4NUnit is a NUnit extension combining NUnit [3] with Log4net [4] as shown in figure 1. The purpose behind integrating such an extension is the necessity to create test protocols for NUnit. NUnit requires only testcase and one more test fixtures. In the entire testing tool successfully executed test cases are not documented. To keep track between the testing processes and to monitor the performance of test cases, successful execution of a test case, results of successful testing and bugs which are identified during the testing should be further documented. So the basic intention behind building Log4NUnit is the necessity to create test protocols in order to document the entire initial test setting, test case execution and results of successful test case. Along with the testing framework(NUnit testing) a logging component(Log4net) is required to document the results of test case executions. One of the greatest logging libraries out there for .net is log4net.In addition to the core feature available in log4net some advance features are also there like ADO.net, LINQ queries and .net Trace and debug API. The rest of the paper is organized as follows. In section 2, the paper describes the background knowledge required to understand the Log4NUnit structure. Section 3 will demonstrates how to make use of it in different programs (for code and detailed steps please contact corresponding author of the paper). In Section 4, emphasis is given to the limitations & in section 5 various recommendations are outlined for future work on it. In the end section 6 illustrates the conclusions of this paper and section 7 consists of acknowledgments.

Figure 1: Log4NUnit as a Integration of Log4net and NUnit

BACKGROUND WORK

According to Michael Stal, Log4NUnit is a ready-made solution for logging messages from within the test cases [5]. It provides us with an extension of NUnit’s Test class. At the time of production, you cannot debug your complete application and it may be possible that some errors which appear in production server don't appear in development or in preprod server. So if you are tiring to slove the error, you need to find a proper way which will help you in identifying the error and tracing your application at the time production without stopping or using debug mode in the application. Log4net is a ready made tool for the programmer to output log statements to variety of output targets in production, preprod or to the development server. There are many ways For Installing and configuring the tool. This tool is able to work with ASP.NET 2.0, 3.0, 3.5 using C#.

log4net will help you for collecting errors in following ways:

Using FileAppender (collect messages in file)

Using SMTPAppender (collect messages in email)

Using ADONetAppender (collect messages in database)

Using EventLogAppender (collect messages in event viewer)

There are several kinds of logging levels supported by log4net library which are given below

Off() - nothing gets logged (cannot be called)

Fatal()

Error()

Warn()

Info()

Debug()

All - everything gets logged (cannot be called)

Above logging methods are provided in the Log4net library for various log levels. Here the log method itself is not provided; instead each log level method need to be used directly. It can be said that the design pattern known as Adapter [6] is used to design this class. It uses a concept where Object Adapter relies on object composition as shown in Figure 2. Setup and Teardown methods are also provided.

Figure 2: Object Adapter design pattern diagram from [6]

In NUnit, preparation for testing is called setup. It’s independent of the assertions, so the same scaffolding can apply to several individual tests. The setup is run before each test. When a test ends, the scaffolding might require some cleanup actions.

INTEGRATION OF LOG4NET AND NUNIT

Lots of people head the project into automation and eventually start writing different scripts. I have experienced, that is a bit rash as there are some infrastructure bits which needs to be take care.

The first thing that we need to keep in mind is to identify where the scripts will be stored in version control. Everything should be there in version control for providing automation. (If anyone has proposed the solutions consultants who don’t have the access to client content but write code.) This should be the first step.

The next step is to identify the logging strategy. The reason behind doing this is two fold. First, it will give you means of debugging the script and second it will allow you to create the record for test execution, result and test failure details for latter reference.

There are verity of logging framework available for .net project but among those most widely used framework for logging is log4net and an application need to configure it in order for your instructions, such as where to log the message, what to ignore with it, etc, are to be carried out. However, there are several characteristics that one has to be aware about, particularly when we are using it with NUnit testing tool. Currently i am using NUnit version 2.5.7 for testing framework and log4net version 1.2.10 as a logging framework and hence the configuration below is relevant to the combination of this version only.

However, when we are using it in NUnit, a report has been filed which will not generate any log file. In other word, we can say that the combination fails to find the required configuration file. The report charges NUnit being the convict but I can assure that it is not entirely. This triggers lots of unnecessary works around.

For specifying the configuration information, an assembly can able to include any custom assembly-level attribute like Log4Net, XmlConfiguratorAttribute. For the situation where we need to load the configuration file, the internal logic of log4net will look for an instance of this custom attribute in a given assembly for the configuration file.

This class is located in src/Config/XmlConfiguratorAttribute.cs file.

In theory, each assembly can be used to specify different configuration file. log4net does not process or configure its system until the first reference to create a logger or an appender. But in practice the first logger created determines the configuration file used and subsequent calls to create different loggers will not load the config file specified in that assembly. When NUnit is launched with log4net and before running your test assemblies, code inside NUnit causes NUnit.Core.Log4NetCapture.StartCapture() to execute. This is an override method of its parent abstract class TextCapture. The result is that will causes log4net.Appender.TextWriteAppender to be created.When this happens, somehow it triggers a call to log4net.Config.BasicConfigurator() for this appender. This in turn causes the creation of Logger Repository for the assembly nunit.core, which in turns starts looking for custom attributes for repository.

To use Log4NUnit follow the guide lines given below.

Include the assembly attribute for log4net located in src/Config/XmlConfigurationAttribute.cs

Add log4net reference to your vs project

Add log4net bits to the script

Next is to configure the log4net for that you need to add new configuration file to your projectlog4net.config

As things stand right though, log4net will never find this file as it expects it by default in an app.config or web.config. So we need to explicitly add the log4net where it’s config can be found for that we add the given assembly to our Assembly info.cs file.

[assembly: log4net.Config.XmlConfigurator(ConfigFile = "log4net.config", Watch = true)]

Now you have integrated log4net into your project and you can send log messages to it by using log.Debug, log.Info, log.Error, etc. NUnit will be default capture for log4net messages as pass they along to the user if they are Error or higher — regardless of what log4net says. So in this case we need another Application Configuration File. But if you run your NUnit scripts from the NUnit GUI you can specify the config file you want.

If Everything goes right with Integrating the Log4net and Nunit testing we have created the Log4NUnit Library which gives all the logging facility like logging Initial test setting, test results and error result can be logged in a text format.

LIMITATIONS OF LOG4NUNIT

A. Log4NUnit requires user Setuptest to be extended from LoggingTestCase. Many users may feel that they are forced into extending a class which is not very importance in their software product. If users want to extend their base test class from other class to provide some other capability (along with the capability to Log the Test case results) then they cannot do so since they would have already extended the LoggingTestCase class.

B. Log4NUnit supports only Log4net library for logging. Although Log4net is a very flexible and powerful logging framework, sometimes users may need to use other logging implementations like SmartInscept, Nlog, ObjectGayFramework or commons logging library as per the requirements.

C. Log4NUnit was designed to work with NUnit version 2.5.7, and so isn't compatible with the new syntax of NUnit. Since in new syntax of NUnit programmers do not need to extend the Setuptest class.

D. Log4NUnit is tightly coupled with its underlying logging framework. (the defacto standard – Log4net)

RECOMMENDATIONS FOR FUTURE WORK ON

A. Log4Unit is an open source and small library, so developers can customize it to support the logging implementation of their choice i.e. If any programmer wants to use any other logging framework, for example take SmartInspect [9], than they can change the Log4NUnit source and create their own library providing customized logging backend. Programmers can abstract the logging implementation using a different bridge such as Nlog Logging framework.

B. If developers plan to write more tests, it is recommend that they extend a base Setuptest of their own from LoggingTestCase and extend all your other tests from that base Setuptest. This way if Log4NUnit is removed or replaced, they only have to change the base setuptest and not each and every TestCase classes.

C. Another characteristic of Log4NUnit is that should be looked into is that, it becomes tightly coupled with the underlying logging framework (the default Log4net or the customized one, whichever the user implements). To overcome these inflexible situation programmers can use a logging framework like ObjectGay Framework [10]. ObjectGay will serve as a simple logging framework or abstraction for various logging framework, allowing the end user to plug in the desired logging framework at deployment time. This can provide a loose coupling of Log4NUnit with the underlying logging framework.

D. Scarcity of documentation on Log4NUnit makes it difficult for programmers and research students to understand its structure and take a look at the Software engineering concepts used in its structure design. So a lot of work needs to be done for creating the class diagrams and other documentation on this topic so that it can be used for further research.

Conclusions

This paper presented the Log4NUnit library whose focus is to create test protocols for NUnit testing framework. It is being observed that the library is asymmetrical in the sense that it focuses on the documentation of test failures and errors. It is also tightly coupled with the underlying logging framework log4net. This library is very much useful in small scale projects that need to startup immediately with a Testing mechanism together with a logging framework. It also reduces the efforts of programmer by maintaining a separate logging framework. This library is already been used by some developers and had proved useful to small scale projects. Since the Library is open source and it can also be customized to support the business requirements of individual organizations. Programmers can abstract its logging implementation by using a SmartInject or Nlog so as to decouple it from the logging library, thus making it independent of underlying logging framework.

Acknowledgment

Authors of this paper appreciate the original author of the Log4NUnit library, for providing his invaluable time to review the ideas and also answer the queries promptly. Although this library is open source, this research work began after obtaining prior consent of its original author. Authors of this paper would also thank their colleagues, friends, classmates, teachers and other guides for providing their immense support and helpful comments to improve this paper.