The Technology Blog Latest trends and news surrounding software

The Four Categories of Performance Testing Metrics

0 Comments
231
25 Sep 2017

Over the years, a set of four performance metric categories have become very valuable to me.

Without the categories of 1) Workload, 2) Work, 3) Throughput, and 4) Experience, all the metrics that are available become a jumble. For example, somebody throws Average Response Time, Bytes per Second, Memory Pressure, Connection Pool Utilization at you, and you’re like “Yeah? So?”.  But, if we have some meaningful categories to bring order to them, well then, now we’re talking. 

AT THE GROCERY STORE 

To get to know how these four categories fit together, let’s consider how they would make sense in a very familiar context, the grocery store.  We’ll focus particularly on the checkout process. (This analogy hits close to home since my youngest daughter works as a cashier and bagger at our local market.)

Workload

How many people are on line at the registers?
How many items do they each have in their carts? (If it’s the express line, do they have more than 10 items? A challenging situation for cashiers managing other customers’ expectations) 

Work

Cashiers scan the items or type the codes.
The bagger bags your items (Hopefully separating them properly. We don’t want our veggies and cleaning products in the same bag. Ick!) 

Throughput

 How many items are scanned per minute?
How many customers complete paying for their purchases per hour?
How many customers are extreme couponers that will hold up the line? 

Experience

How long did each customer have to wait?
What kinds of errors/issues did they have? 

With all this in mind, can you see the cause and effect relationship as we moved through the checkout process? I figured you would. 

Workload → Work → Throughput → Experience 

IN COMPUTER SYSTEMS

Now that we can see the categories and how they fit together, let’s apply this to any computer system or application. Below are just a few classic metrics per category for software applications. 

Workload 

Users present
Requests per Second 

Work

CPU Utilization
Database Concurrent Connections 

Throughput

Transactions per second
Bytes per second 

Experience 

Average Response Time
Errors per second 

Once these categories make sense, you can use the following questions to help you write and read performance test reports: 

What category does a metric fit in?
Do I have information from each of the four categories?
Does this data tell the whole story of application performance?
Are there reliable cause and effect associations shown by this data?
Did we articulate our performance requirements properly and completely?   

So when you’re in line at the market during the Thanksgiving rush, or online on Cyber Monday, this blog will give you something to think about. You’re welcome. ;) 

P.S. If you would like to take these four categories out for a test drive, check out Andreas Grabner’s posts  here and here. Use the questions above with each of the metrics he discusses and consider how they work together to provide clearer insights into systems. 

NOTE WELL: We can use the four categories for each subsystem as well as the entire system.

Tom Horsfield Author:
Tom Horsfield
Sr. Technical Engineer
Checkpoint Technologies

Last Modified: Friday 29 September 2017 07:05
Related Articles: Getting Into the Head of a Performance Tester Atlassian and Software Testing in a DevOps World Four Score and Mobile Center 2.50 update ago Checkpoint Technologies and Mobile Labs to Present Upcoming Webinar on Mobile Testing Strategies Test is a Four-Letter Word What Performance Requirements? Can automation completely replace manual testing? Embracing the Changing Landscape of Testing
QA performance testing Software Testing

No Comments Yet...

Leave a reply

Your email address will not be published.