Welcome!

Java Authors: Maureen O'Gara, Bruce Armstrong, Liz McMillan, Walter H. Pinson, III, Yakov Werde

Related Topics: Java

Java: Article

JUnit, Assertions, and Management

JUnit, Assertions, and Management
By Bill Dudney
Article Rating:
August 30, 2004 12:00 AM EDT
Reads:
 15,521
I've recently been having a discussion about the quality of JUnit tests and how manager types might be able to judge the quality of the tests.

In my mind this is not so easy to resolve. I know managers (especially on very big projects) need a way to judge the quality of what is being delivered, but often they take their statistics far too seriously.

As an example, take code coverage calculated by Clover or the like. As a developer I find code coverage results to be a fantastic tool to determine if my developer tests are covering all the code under test that they should. And to a certain extent the code coverage numbers are good to give to management, so they can see the progress of the tests. If the tests are good then - as this number goes up - you should be able to assume better quality in the code base. All too often though I see tests that have 100 lines of code and a single println at the end, not a single assert. Tests like this will generate a lot of coverage but don't really tell you anything about the quality of your code.

In my experience developer tests are too often on the poorer end of quality. So, how do we measure what good is? There are the simple statements link "Each test should be tightly focused and have one assertion." The problem is that this statement is too broad to measure the quality of your tests. This statement could possibly be refined with a statement like this "The ratio of non-comment lines of code to assertions should be low." In other words you should have a few lines of code in a test and an assertion to make sure that what was expected actually happens.

What are your thoughts on measuring test quality? Should management even be concerned with this measurement?

Published August 30, 2004 – Reads 15,521
Copyright © 2004 SYS-CON Media, Inc. — All Rights Reserved.
Syndicated stories and blog feeds, all rights reserved by the author.

More Stories By Bill Dudney

Bill Dudney is Editor-in-Chief of Eclipse Developer's Journal and serves too as JDJ's Eclipse editor. He is a Practice Leader with Virtuas Solutions and has been doing Java development since late 1996 after he downloaded his first copy of the JDK. Prior to Virtuas, Bill worked for InLine Software on the UML bridge that tied UML Models in Rational Rose and later XMI to the InLine suite of tools. Prior to getting hooked on Java he built software on NeXTStep (precursor to Apple's OSX). He has roughly 15 years of distributed software development experience starting at NASA building software to manage the mass properties of the Space Shuttle.

Comments (5) View Comments

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.

Most Recent Comments
Geoffrey Wiseman 08/31/04 04:58:25 PM EDT

I''m not convinced that there''s an automated tool that will help with this sort of thing. As you say, coverage will take you part-way there, but if you really want to verify the quality of tests, that comes down to code review.

One practice that can increase the amount of ''code review'' going on is pair programming. You could argue that a pair is more likely to write higher-quality tests.
Bill Dudney 08/31/04 11:15:04 AM EDT

Hi Jon,

Depends on how sure you want to be of your code. If you want to be ''sampling'' sure that your code is good then you can sample.

Think about ''is it there'' like this. If you write a check do you want your bank to make sure that your account is debited by the amount of the check or simply that a withdraw was made. I for one hope the amount is verified at many levels, developer testing, system testing, user testing etc.

The higher quality your unit tests the more confidence you can have in your code. The less likely it is that your users will find problems in the code (requirement changes are another story of course). Delivering quality is always good, for the team and for the users.

Ok I''m off my soap box now.
Jon Allen 08/31/04 10:16:17 AM EDT

How exhaustive should unit tests be?
I.e., if a method has a range of possible parameter values, do you test the entire range, some sampling along the range, or is a simple "is the method there" test sufficient?
Ted Beckett 08/31/04 07:36:42 AM EDT

The open source test tool Jester, measures the quality of your unit tests. It will catch bogus unit tests that aren''t comparing actual results to expected values. See http://jester.sourceforge.net/.
Jez Nicholson 08/31/04 05:40:36 AM EDT

JUnit is an excellent opportunity for communication between developers and managers. Any way in which information on the quality of a system can be presented to managers (or clients) in a format that they can understand and do something with is great, especially if it can be generated as a by-product of normal activity.
It is quick to see how JUnit works, but difficult to know what you are actually supposed to do with it. I would welcome further measures that encourage best practice.