Performance Testing

Performance Testing also known as “perf testing” is a sub-set of Performance engineering. It is a non-functional testing technique intended to determine the speed, response time, throughput, stability, reliability, scalability and resource usage of a software application under a particular workload. Perf testing helps identify and fix the performance bottlenecks in the software application.

Key Attributes of Perf Testing:

  • Speed – how quickly the application responds
  • Scalability – how much user load / requests the application can scale up to
  • Stability – how application behaves under varying load
  • Reliability – how reliable the application is during long / extended hours

In this article, we’ll go through –

Why perf testing is necessary?

Perf testing is critical to customer satisfaction. if the application goes to market with poor performance then it’s likely to damaged brand credibility and reputation. Customers will move on to competitors which will translate into loss of revenue. Perf Testing reveals if the product is ready to go to market or what needs to be improved before the product goes to market. Apart from that Perf testing can also be used to validate if the system meets specifications mentioned by the manufacturer / product owner.

A performance tester needs to have skillsets required to follow perf testing process. The skill sets include good understanding of architecture, understanding of operating systems, understanding user behavior, scripting, interpreting performance results, basic knowledge of databases.

Performance testing metrics

Here is a list of few important Performance metrics also known as “Key Performance Indicators” to lookout for while analyzing performance of an application.

  • Throughput: units of information a system or network processes over a specified time
  • Memory: the physical memory available to a processor or workload.
  • Response time: elapsed time between request sent and first byte of response received.
  • Error rate – percentage of problems compared to all requests
  • Bandwidth:  bits per second that can be used by network.
  • Processor Usage – an amount of time processor spends executing non-idle threads.
  • Disk time – amount of time disk is busy executing a read or write request.
  • Database locks – locking of tables and databases needs to be monitored and carefully tuned.
  • Garbage collection – It has to do with returning unused memory back to the system. Garbage collection needs to be monitored for efficiency.

Above are a few of key metrics used to identify issues / bottlenecks in the application under test. There are some common performance issues for any web / mobile application. For example – most web applications have a good performance as long as only a few users are active but when large number of users start working on / accessing the application simultaneously, performance issues tend to appear.

Types of performance testing

  • Load testing – Determines the application’s ability to perform under user loads or the throughput required to support the anticipated peak production load. It helps detect concurrency, connection pool and code issues under load.
  • Stress testing – Determines the breaking point of an application or estimate of how far beyond the target load an application can go before causing performance degradations.
  • Endurance testing – Determines the software / application can handle the expected load over a long period of time. Memory leak is a key metric to look out during this test. It helps understand if the application metrics are performing consistently throughout the test.
  • Spike testing – Determines software’s performance during sudden / significant spikes in user load or throughput. A large event like Black Friday sale is an example where traffic is likely to be higher compared to normal days.
  • Volume testing – Determines software application’s performance under varying database volumes. For example, a batch processing huge volume of data during business / non-business hours.
  • Scalability testing – Determines whether an application is able to scale up to increased user load / demand. If done properly, it will help in capacity planning for the application.

Performance testing tools

There is a variety of performance testing tools available in the market. Based on the requirements like protocol support, hardwire requirements, platform support, license cost and after sales support, one can choose the right tool. Below is a list of some popular perf testing tools used industry wide. The list contains licensed as well as open-source tools.

LoadRunner by Micro FocusLoadRunner, by Micro Focus, tests and measures the performance of applications under load. It is the most popular Perf testing tool in the market. LoadRunner can simulate thousands of end users, as well as record and analyze load tests. LoadRunner has a free version with 50 users for personal & learning purpose and Licensed enterprise version for organizations. This tool supports and helps test performance of almost all products including web, thick clients, mobile.

Apache JmeterJmeter, an Apache perf testing tool, can generate load tests on web and application services. Jmeter supports an integrated development environment (IDE) for test recording for browsers or web applications, as well as a command-line mode for load testing Java-based operating systems. This is a community based free tool for use. It becomes a cost-effective solution for small organizations with less budget and web / api oriented applications.

Neoload by NeotysNeoLoad, by Neotys, provides load and stress tests for web and mobile applications, and is specifically designed to test apps before release for DevOps and continuous delivery pipelines. This tool can simulate millions of users, and it performs tests in-house or via the cloud. This tool has a licensed version for industrial use.

IBM RPTRational Performance Tester by IBM is designed to create and run performance tests. It is also a powerful analysis tool. The tool allows you to verify the stability and scalability of web-based applications before they are deployed. This is an automated performance testing tool that develops a demo of the transaction process between the web service and the user.

LoadNinjaLoadNinja by smartbear reduces testing time by creating scriptless sophisticated load tests. It also replaces load emulators with real browsers and gets actionable, browser-based metrics at ninja speed. LoadNinja empowers teams to increase their test coverage by removing the tedious efforts of dynamic correlation, script translation, and script scrubbing.

Leave a Reply