Today's discussion I would like to explain in detail what non functional testing is and how we can measure some of the quality attributes related to non functional testing.

Difference between functional and non functional testing:
Functional testing Non functional testing
Tests what SUT does or specific behavior of the system. Deals with how or how well the SUT does something.
Requirements may or may not be measurable. Requirements are always measurable
Can be done manually or automated way. Cannot be done manually, automation is must.

Quality attributes related to non functional testing:
  1. Efficiency
  2. Reliability
  3. Maintainability
  4. Portability
Efficiency or performance testing: 
As the name implies here we are measuring the performance of the sut against permit able range. Mostly we consider response time, throughput and resource utilization factor of the sut here. Following are few performance testing types used widely:

  • Load testing - Here we test the sut with increasing load. Example of load can be number of concurrent users or sessions or transactions etc. Load testing focus is on to figure our system response on realistic load. We can measure response time and throughput.
  • Stress testing - Here we test the sut at or beyond the limit of specified load with reduced availability of resources like memory or servers etc. The goal here is to ensure that response times, reliability and functionality degrade slowly and predictably along with increasing load.
  • Scalability testing - This will let you determine how your application scales with increasing workload. Scalability testing is performed as a series of load tests with different hardware or software settings,keeping other testing environment unchanged. As per your requirements,you can vary the CPU speed, number and type of servers, the amount of available RAM etc.
  • Resource utilization testing - Here goal is to evaluate the usage of various resources like cpu, memory, disk etc under a given load.
  • Endurance testing - Here we run the sut on high load for a period of time. This helps to uncover errors that occurs only after keeping the system running for some time for example memory leak errors.
When we should do efficiency or performance testing:
  • We should start as early as during unit testing.Any synchronization code, use of locks, semaphores and threading must be tested thoroughly to avoid performance bottleneck.
  • During integration testing, performance testing of integrated units should be done. Any code that transfers data between modules should be tested. All interfaces should be scrutinized for deadlock problems.
  • During system testing, performance testing of the whole system should be done. All the performance requirements should be validated carefully.

Performance issues:
  • Slow response time with permit able load
  • Slow response time with pick load
  • Slow response time after a period of time
  • Error/exceptions with permitted/pick load
Open source performance test tools:

17 Command Line Tools to Monitor Linux Performance