Cloud Based Performance Testing Web Application

Published Date: 02 Nov 2017

INTRODUCTION

Performance Testing is the process of determining the speed or effectiveness of a computer, software program or device. It is usually done towards the end of the testing phase. If the performance tests are executed on cloud, we will have the ability to deploy an environment which will be required to test necessary performance, with minimal cost and effort; through reduced capital expenses, minimize support and administrative costs, and still retaining the confidence in expected performance [1, 4]. Even a completely bug-free web application is doomed to fail if it can only serve an average amount of traffic, but is not able to handle the significant peak loads in a real life situation. A large number of failures of web applications in the industry have been attributed to performance flaws in the system. Each of these failures cost the businesses money and a hit on the brand. Thus effective testing of websites for performance, ability to scale up the web application when necessary is critical for the success of the channel. Cloud computing means that users are connecting to applications that run on a set of shared or pooled servers, rather than running on a single dedicated server. It provides high availability, reliability, scalability, flexibility, reduced time-to-market, and many other cloud-oriented benefits [5, 6, 7].

Performance Testing Process

The success of performance testing is highly dependent on the methodical approach followed to plan, execute and analyze tests. A structured approach is depicted in the diagram below and explained.

Figure 1. Performance Testing process flow

There are mainly six steps for performance testing process is as follows:

Step 1: Plan: Define the focus of the test and create a detailed test plan.

Step 2: Record: Most tools provide the capability to record test cases.

Step 3: Modify: Modify load test scripts defined by recorder.

Step 4: Execute: Execute the test script to achieve the test goal.

Step 5: Monitor: Monitoring the scenarios.

Step 6: Analyze: Analyzing test results.

Parameters of Performance Testing

Performance testing through automated testing tools simulates a variety of normal and abnormal peak load conditions of the system of performance indicators/ parameters for testing. Performance model provides measurable standards of performance, while the standard is constituted by a series of performance parameters. Typical performance parameters are Response Time, System Throughput and System Resource Utilization [2, 3, 8].

Scenario 1. Performance Testing of web application for on-premises

Performance testing through automated testing tools simulates a variety of normal and abnormal peak load conditions of the system of performance indicators for testing. Following diagram shows the details of testing on-premises with physical resources which are available in premise (i.e. Application server, web server and database server).

Figure 2. Physical structure - Performance testing of web application for on-premise

In this scenario, web application and database are loaded on physical machine and we can test the performance of web application by using testing tool. Here we are using LoadUI testing tool for performance testing [9]. But we need all physical machines for this testing.

PERFORMANCE TESTING IN CLOUD ENVIRONMENT

At its simplest, cloud computing means that users are connecting to applications that run on a set of shared or pooled servers, rather than running on a single dedicated server. There are two scenarios to test the performance in cloud environment are [1, 5]:

Database in cloud

Web application and database in cloud

Scenario 2. Database in cloud

In this test scenario, we planning to moved our database in cloud. For this test, we executed our web application from local machine and linked our database with this web application. We need one micro instance for this test scenario. Following diagram will show the details.

Figure 3. Physical structure – Database in cloud

Scenario 3. Web application and database in cloud

In this scenario, the entire application was moved on the cloud.

Figure 4. Physical Structure – web application and database in cloud

RESULTS

Below are the charts comparing response times across multiple tests above. The chosen scenarios are 3 requests / sec and 10 requests / sec for analysis purposes.

Home Page

3 Requests / Sec 10 Requests / Sec

Figure 5. Comparison of scenarios for Home page

Observations: As the stress increases (tests are run for longer times), the performacne of scenario 2 degrades significantly. The other scenarios don’t show the change in performance or degradation which is significant.

Discussion: Firstly, the home page is a simple page which when hit just ensures database connections are up, and shows a page with options to login as user or admin or register new user. So, data flow between client and server is minimal. But dataflow between the application server and database is significant – as the check for database running is done. This takes a hit in scenario 2. Hence the degradation.

It is also important to note that the difference beween performacne scenario 1 and 3 is due to the Network Latency which seeps in with infrastructure being on the cloud.

Final Proceedings

3 Requests / sec 10 requests / sec

Figure 6. Comparison of scenarios for Final Proceedings

Observation: As load and stress increase (higher number of requests / second; and tests run for a longer time) we noticed that the impact of network latency on performance is insiginificant.

Discussion: Notice that for 10 request / sec results of scenario 3 are similar for on premise and on cloud. But when we look at the failures – the number of transactions which have not gone thru, we will notice that Scenario 3 outperforms every other scenario. To understand further we looked at the failures of transactions for these scenarios. Below is the table that provides the same.

TABLE 1.FAILURE RATE - 3 REQUESTS / SEC AND TIME 30 SECONDS

Scenario 1

Scenario 2

Scenario 3

Total requests

667

674

849

Assertion failure

59

59

57

Failure rate

8.8%

8.8%

6.7%

TABLE 2. FAILURE RATE - 10 REQUESTS / SEC AND TIME 90 SECONDS

Scenario 1

Scenario 2

Scenario 3

Total requests

7264

7270

21675

Assertion failure

663

4834

1408

Failure Rate

9%

66%

6.5%

So, from throughput standpoint – Scenario 3 works best. It handled over 21K requests in a matter of 90 Seconds with a failure rate of 6.5%; Where as the other scenarios could handle less than 8K Requests in the same timeframe. Also the failure rate was higher than scenario 3. Which indicates that even the reliability of scenario 3 is far higher than the other scenarios. This is further supported by failure rates with lower stress as well. Notice the table above for 3 req / sec. The failure rate for scenario 1 and 2 is 8.8% - where as for scenario 3 is lower – 6.6%.

CONCLUSION

If the application will always be used on premise by a limited number of users (e.g. Intranet, college schedules), performance will be better with on premise. But these days, every enterprise and college will allow access to intranet application types over the internet from home. Hence these types of applications will reduce day by day. Another benefit is, when the number of users increases, cloud performs better, and is more reliable. Cloud (scenario 3) has least number of failure percentage with increasing load and requests. This is a clear indicator that for scale cloud is the best option.

ACKNOWLEDGMENTS

We thank our institute which has always supported and motivated us for research and learning activities.