An Introduction to Performance Testing

Following this year's Testing SIG held in conjunction with SAP and Experior, we take a look at performance testing and some of main questions and challenges users face.

All dimensions of your business need to experience fast and simple interaction with your systems. It needs to look good, work seamlessly and perform without compromise. Poor system performance on go-live is as likely to colour end users perception of a system as faulty functionality. Through careful planning and execution of performance testing, the risks can be reduced and a users’ experience can be improved, resulting in a potential positive impact on bottom line.

What makes good performance testing?

As essential as performance testing is, it's important it's done efficiently and effectively for your business. You should be aiming for a better understanding of how systems behave and interact with one another, and in turn how users interact with these systems. Setting clear objectives is vital, think about targets like how many users are you expecting before, during and after?

But don't just consider the setting up stage. Arguably the hardest, but certainly most important work comes from comprehensively monitoring these tests and results.

The value of good performance testing

Good performance testing provides invaluable insight to the overall health of your systems. In order to ensure that tests are repeatable, an automated testing tool such as LoadRunner could be employed. This can be used to optimise performance through researched and recommended coding and configuration adjustments.

Performance testing can reduce the risk of negative user experience such as slow response times. Testing can also serve to validate and verify other quality attributes of the system, such as scalability, reliability and resource usage.

The methods available

• LoadRunner

• Performance Center

Identifying what to test

Before any testing takes place you need to identify and assess potential risks. This will enable you to identify the scope and prioritise the execution of performance testing.

Are we testing or should we be tuning?

Unlike functional testing, performance testing is not as simple as pass or fail. Defects are not always obvious and it should be seen as an iterative process rather than a box ticking exercise. This is all about mitigating risks and providing information regarding areas that may benefit from tuning.

We don't have the luxury of full-scale test systems, so why should we bother?

The planning phase of performance testing should include a comparison of the test and production systems. It’s not an exact science, but the skilled performance test analyst can draw conclusions from this.

If the test environment is 25% of the size of production, it would be accurate to run performance tests with virtual users that are 25% of the number of actual users we have in production.

What to look for in results

This will vary depending on the purpose of the test, but typically, response times for actions are captured by the automated testing tool in use. Data should be captured at a number of layers, most notably application, database and infrastructure. This will allow data such as logs of memory and CPU usage during execution to be stored.

Can we continue to test or measure performance in BAU/production?

A number of tools are available for BAU monitoring. Transaction response times can be viewed whilst in use and Early Watch reporting can give an indication of system health.

Why use LoadRunner/Performance Centre?

HP's LoadRunner tool stimulates a large number of users which generate load in parallel for a system under test. This tests the performance of a diverse number application types and is one of the world's most popular performance testing methods. Perhaps the biggest advantage of LoadRunner is the tools’ reporting engine that provide a variety of reporting formats and in-depth insight.

LoadRunner is the market leader in terms of performance testing, with SAP themselves even using LoadRunner for testing their own developments. For small scale usage, there is a Freemium community edition that can but used to simulate 50 virtual users.

We’re on the Cloud, so surely I don’t need to worry about performance?

Performance testing of Cloud based apps does present an issue and you may need to seek agreement from the Cloud provider before undertaking any execution. Developer tools are available to execute single user performance tests which may provide information regarding transactions that would benefit from optimisation

If you’d like to know more about User Testing, take a look at the resources from this year’s Testing SIG. To keep up to date with all our upcoming events, keep an eye on our Twitter, Facebook and LinkedIn pages.