The objective of performance testing is to create a baseline where,
• bottlenecks have been removed,
• systems have been optimised.
Once this baseline has been established, we have a starting point from where other non-functional testing such as load and stress testing can be carried out. This leads to far greater accuracy in the test results produced.
Performance testing must be viewed as paramount in establishing a healthy, working and usable thin client based system.
It would be a pointless exercise to run load and stress tests in a 'dirty environment' where bottlenecks such as those induced by network bandwidth limitations, database inefficiencies and even inadequate Citrix server optimisation, taint the quality of result data produced.
Through performance testing we can create a controlled environment where the unknowns become known and if need be, are eliminated.
Performance testing is about creating a usable environment. One where users will be actually able to use the applications in an acceptable manner.
Performance testing is NOT about,
(i) finding bugs in applications.
(ii) finding out whether a server can handle the number of required users.
(iii) finding out the maximum number of users a Citrix server can adequately handle before degrading.
Finding bugs in applications, is the role of functional testing and is a completely separate process to performance testing. It's important to appreciate that functional testing must be satisfactorily completed before any performance testing is initiated.
As the environment used for performance testing must be stable. There's little point in trying to find performance bottlenecks when the applications don't work as they should or cause the servers to freeze due to inadequate configuration.
Finding out whether a server can handle the number of required users is the responsibility of load testing and this can only be accurately determined once performance testing has been completed.
Finding out the maximum number of users a Citrix server can adequately handle before degrading is the responsibility of stress testing and again can only be determined once performance testing has been completed.
When trying to establish the acceleration time for a car from 0 to 60 mph, without first eliminating any bottlenecks or tuning, will lead to inaccurate acceleration times.
However by running a series of performance tests, the engine and any other components can be tuned and any bottlenecks removed. Only then the time taken to accelerate from 0 to 60 mph will be representative of the car's acceleration capabilities.
The bottlenecks aren't just reflected by the backend systems or the network. Many involve poor Citrix server installations and on many occasions I have bore witness to Citrix systems that have been installed simply out of the box without any regard for optimisation.
The danger of memory leaks is intensified in Citrix environments, which if not determined can cause major problems.
Unlike running an application in a desktop environment such as one with Windows XP, where the likelihood of memory leaks causing problems is minimal, as this computer will generally be switched of on a daily basis. In a Citrix environment, servers may be rebooted once a week, causing any memory leaks to intensify.
It can get worse as the application may be run many times by multiple users on a Citrix server, accelerating the memory leak. However performance testing can help find these memory leaks. I found one application which after a weeks continuous use, was so damaged, it would cause the Citrix server to run out of memory within a few minutes of powering on.
The interesting thing was I only discovered how bad this application was when I ran a week long test using automated test tools. Thereby simulating months of usage in seven days.
Now if this application had gone 'live' into production then within months all the Citrix servers with this application installed would have turned to 'vegetables' that is become unresponsive and would have needed to be rebuilt. The worse case scenario would have been the whole Citrix farm dying as servers died pushing load on less and less available servers in the farm.
In conclusion the system must be clean and optimised, which can only be done using Citrix performance testing.