Welcome!

Java IoT Authors: Pat Romanski, Liz McMillan, Elizabeth White, William Schmarzo, Stackify Blog

Related Topics: Java IoT

Java IoT: Article

How Good Is Good Enough?

Code quality and the point of diminishing returns

Intellectually everyone understands that improving code quality is a good thing. After all, we know bad quality when we see it. (Anyone old enough can cast his or her mind back to the late '80s and Microsoft Word for Windows 1.0.) But we also know that there comes a point where there's a diminishing return on our investment in code quality. How much work would you put into tracking down a bug that's only ever been reported once by a user running on OS/2 Warp?

The problem with code quality initiatives is that we really don't know how much effort to put into them. We have never truly answered the question: how much quality is enough?

Why Code Quality Is Important
The Standish Group famously reports on the software industry's inability to deliver successful projects on a regular basis. In 2004, it reported that just 29% of software projects were considered a "success."

Projects fail for a number of reasons, and as far as the Standish survey is concerned, that means either the projects were completely canceled, or that they were "challenged" because they significantly overran time and/or cost.

There are many reasons that projects are canceled or overrun, and almost certainly the most common is that the software just didn't do what it was supposed to - it didn't conform to requirements. That's a hard problem to fix - and it's an even harder one to measure.

Code quality, on the other hand, which is probably the second largest contributor to project failure, is much easier to measure, and therefore easier to manage. Yet, many development teams waste time on rework cycles fixing bugs that crop up in QA or production that could have been caught and fixed upstream, with relatively little effort.

A big part of the cause of that problem is that during the coding phase, oftentimes the only measurement we have available to track progress is whether we hit the deadline or not. Even if we monitor defect rates (it may shock you to learn that not all development teams do), we just don't have any meaningful metrics by which to manage the process until it's too late.

Agile methods help. Test-driven development, where unit tests are written with, or even before the code, and continuous integration and test cycles flush out many bugs in development before they get to QA.

But fundamentally, managing only to deadlines causes an accumulation of "technical debt" - a term coined by Ward Cunningham to describe the effect of all those decisions to use a quick and dirty approach to solving a problem, with the idea that you'd come back and fix it later. Just like financial debt, technical debt isn't necessarily a bad thing, especially when an important deadline is looming. But like financial debt, if you have too much of it, it can get out of control and becomes very hard to pay back.

How Much Quality Is "Enough"?
Quality is a tradeoff between cost and risk. If you ask most (inter-nal or external) customers what level of quality they'd like to see in their software applications, they'll likely tell you "perfect" - i.e., zero defects. Of course, that's something of a trick question, because you didn't tell them how much "perfect" would cost.

After a 1996 Fast Company article on the Lockheed Martin group that builds and maintains software for the space shuttle program, that software is often cited as the most expensive code on the planet, line for line. I'm not sure anyone really knows the cost per line of the space shuttle software (it's been estimated at $1,000 per line), but we do know that (as of 1996) it took roughly 260 developers to maintain 420,000 lines of code, which comes out at about 1,600 lines per person. That's expensive - but the approach seems to work: according to the article, the previous three versions of the software had only a single defect detected per release.

Of course, a typical business application isn't controlling a space shuttle (which, even in 1996 dollars, cost $4 billion a pop, not to mention the lives of a half-dozen astronauts being at stake), and many development organizations struggle with the cost/risk equation. How much quality is enough for your organization? For your application? How can we even start to answer that question?

Ideally, we should be able to come up with a mathematical solution. In probability terms, the expected cost of failure would be expressed as the cost of something bad happening multiplied by the probability that the bad thing will happen. So, as you spend more on increasing the quality of the software, the probability of failure comes down, reducing the expected cost of failure.

This approach becomes problematic in practice, because it's hard to measure the incremental cost of increasing quality and the probability of failure. But, unless we (as an industry) start to measure these things, we'll remain in the Dark Ages.

What Gets Measured Gets Managed!
So, what to measure? A good place to start is the cost of fixing a defect. Once you know the cost of fixing a defect, and you start to measure defect rates, you can assess the return on the investment spent on quality measures (which should be driving defect rates down).

The cost to fix a defect depends on many factors. Conventional wisdom states that the earlier in the software lifecycle a defect is found and fixed, the lower the cost. Of course that's true, although depending on the application, it probably costs less to fix an application in production now than it did even a few years ago. (How many times have you opened up an application only to have it ask if you want to automatically download and install the latest and greatest version?)

There are other quality-related metrics that can help get a handle on how well code is being written. Most development organizations have a set of coding standards. How many organizations measure how well the standards are being applied? Coding standards don't just tell you where to put your curly braces. If applied consistently, they can have a real, measurable impact on code quality and maintainability.

There are a number of static code analyzers available that help development teams implement their coding standards. And static code analysis can go further than just checking adherence to standards - it can find bugs that would otherwise slip through the compiler undetected (consider the old chestnut of incorrect use of the "=" operator in a conditional statement).

Unit test activity is another early indicator of software quality. Although it's not perfect, code coverage (the percentage of executable lines of code exercised by your unit test suite) is a good measure of the effectiveness of your unit tests. You have to keep in mind of course that code coverage isn't the same as path coverage (in other words, 100% code coverage doesn't mean that your code is bug free!). But it's still fair to say that, up to a point at least, the higher your code coverage percentage, the better chance you have of squeezing those bugs out.

This is a great example of a place to look closely at return on investment. You should set a realistic goal for code coverage. A development team could spend many hours trying to get the coverage rate from 98% to 100% - with very little return in terms of the number of defects trapped in development. Many of the better-known open source projects show coverage rates in the 50% to 70% range, and I know for a fact that many commercial projects sit at about half that number.

The important point here, to roll out a well-worn adage, is that you can't manage what you don't measure. If you're not measuring defect rates at different stages of the lifecycle, start now. Take a look at some of the tools available for static analysis and code coverage and, more importantly, start to track those quality metrics over time. Only when you have sufficient data to analyze can you start to make some decisions on how far to go with your code quality initiatives, and finally be in a position to answer the question: how good is good enough?

More Stories By Nigel Cheshire

Nigel Cheshire is CEO of Enerjy Software, a division of Teamstudio Inc. He oversees product strategy and has been driving the company's growth since he founded it in 1996. Prior to founding Teamstudio, Inc., Nigel was co-founder and principal of Ives & Company, a CRM solutions consultancy. He holds a Bachelor of Science degree in computer science from the University of Teesside, England.

Comments (1) View Comments

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


Most Recent Comments
JohnnyCantCode 12/15/06 08:32:24 AM EST

These snippet's have several issues that your product does not catch or comment on.
http://www.enerjy.com/for-developers/index.html
(goto overview for developers and then click to enlarge the image)

(1) First access of public variables within a method, ie without context. No parameters for values used within the method body. Use of underscore within a variable name, ambiguous meanings applied to special characters. Finally regardless of "integer division in floating point context" as your tool notes, it isn't clear at all that the number of bounced checks divided by a transaction count(of any nature) would provide a bounced percentage that is meaningful in any context, so basically naming conventions are thrown out the window.

public float getBouncedPercentage()
{
return bouncedChecks_ / transactionCount_;

}

(2) One, never ever place the value testing for first in a statement. First it's simply incorrect to test if null is equal to an object, the test is whether the object is equal to null. I see this in young so-called programmers you haven't yet formed proper styles in any language. Second, never bracket a one statement if, this is simply confusing to the reader and requires wasted effort to decipher. For example, if (1 == javaCoderLevel) should be if (javaCoderLevel == 1), the first is just nonsense.

if (null == ci)
{
return false;
}

The proper code is(note the false extends slightly beyond the null, although may not be present in this text editor). It's immediately readable and understood without wasting space and brain cycles.

if (ci == null)
return false;

(3) However, this is the worst code snippet. What exactly is being returned? Excessive use of parenthesis to break a simple comparison. Again, the obvious lack of variable context. Avoiding an if statement for expediency will later on cause the poor sap who has to work with this code to re-write it.

return (ci.customerNum_ == customerNum) && (ci.balance_ == balance);

The proper statement (assuming proper logic in the first place and these are all primitives) is the following. Note the intent is immediate, the proper use of white space makes it easily readable and the BUSINESS purpose is immediately understood. I tend to break down multi-level and/or's into tiered stmts so as not to extend beyond the IDE view pane and for readability. In this case for a simple 2 point comparison it makes no difference.

if ( ci.customerNum_ == customerNum && ci.balance_ == balance )
return true;
else
return false;

I think fools that actually believe that you can legislate quality simply have never created a "successful" product. The example of code on your website is extremely poor coding practices period. I think the IT ranks should be decimated and returned back to real technologists not Java or .Net weenies who can only code, albeit code very poorly. Your tool ain't going to change any of that, and projects will continue to fail regardless because quality cannot be manufactured, it takes real talent, not reams of documentation and management. Note to be fair to you I'm not attacking you personally, I'm just tired of all these people who simply want to make a lot of money in IT forcing their ideas onto idiot managers who purchase tools and force "best Practices" on us. This has been going on for many years, all the so-called "innovative" ideas and IT "visionaries". Well guess what, the products of today don't manage businesses any better than they did on mainframes 30 years ago and that's what it's all about, not the latest rad fad BS. The only thing they do is take more time to build, more time to re-engineer, more time and money spent on server farms and wasted CPU cycles and more "engineers" that can't code and definitely cannot convert a design document into a working product.

@ThingsExpo Stories
SYS-CON Events announced today that Enroute Lab will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Enroute Lab is an industrial design, research and development company of unmanned robotic vehicle system. For more information, please visit http://elab.co.jp/.
SYS-CON Events announced today that Mobile Create USA will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Mobile Create USA Inc. is an MVNO-based business model that uses portable communication devices and cellular-based infrastructure in the development, sales, operation and mobile communications systems incorporating GPS capabi...
SYS-CON Events announced today that Suzuki Inc. will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Suzuki Inc. is a semiconductor-related business, including sales of consuming parts, parts repair, and maintenance for semiconductor manufacturing machines, etc. It is also a health care business providing experimental research for...
Real IoT production deployments running at scale are collecting sensor data from hundreds / thousands / millions of devices. The goal is to take business-critical actions on the real-time data and find insights from stored datasets. In his session at @ThingsExpo, John Walicki, Watson IoT Developer Advocate at IBM Cloud, will provide a fast-paced developer journey that follows the IoT sensor data from generation, to edge gateway, to edge analytics, to encryption, to the IBM Bluemix cloud, to Wa...
SYS-CON Events announced today that Massive Networks, that helps your business operate seamlessly with fast, reliable, and secure internet and network solutions, has been named "Exhibitor" of SYS-CON's 21st International Cloud Expo ®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. As a premier telecommunications provider, Massive Networks is headquartered out of Louisville, Colorado. With years of experience under their belt, their team of...
SYS-CON Events announced today that Nihon Micron will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Nihon Micron Co., Ltd. strives for technological innovation to establish high-density, high-precision processing technology for providing printed circuit board and metal mount RFID tags used for communication devices. For more inf...
SYS-CON Events announced today that mruby Forum will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. mruby is the lightweight implementation of the Ruby language. We introduce mruby and the mruby IoT framework that enhances development productivity. For more information, visit http://forum.mruby.org/.
In his session at @ThingsExpo, Greg Gorman is the Director, IoT Developer Ecosystem, Watson IoT, will provide a short tutorial on Node-RED, a Node.js-based programming tool for wiring together hardware devices, APIs and online services in new and interesting ways. It provides a browser-based editor that makes it easy to wire together flows using a wide range of nodes in the palette that can be deployed to its runtime in a single-click. There is a large library of contributed nodes that help so...
SYS-CON Events announced today that Ryobi Systems will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Ryobi Systems Co., Ltd., as an information service company, specialized in business support for local governments and medical industry. We are challenging to achive the precision farming with AI. For more information, visit http:...
SYS-CON Events announced today that SIGMA Corporation will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. uLaser flow inspection device from the Japanese top share to Global Standard! Then, make the best use of data to flip to next page. For more information, visit http://www.sigma-k.co.jp/en/.
SYS-CON Events announced today that Daiya Industry will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Daiya Industry specializes in orthotic support systems and assistive devices with pneumatic artificial muscles in order to contribute to an extended healthy life expectancy. For more information, please visit https://www.daiyak...
SYS-CON Events announced today that B2Cloud will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. B2Cloud specializes in IoT devices for preventive and predictive maintenance in any kind of equipment retrieving data like Energy consumption, working time, temperature, humidity, pressure, etc.
SYS-CON Events announced today that NetApp has been named “Bronze Sponsor” of SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. NetApp is the data authority for hybrid cloud. NetApp provides a full range of hybrid cloud data services that simplify management of applications and data across cloud and on-premises environments to accelerate digital transformation. Together with their partners, NetApp em...
What is the best strategy for selecting the right offshore company for your business? In his session at 21st Cloud Expo, Alan Winters, U.S. Head of Business Development at MobiDev, will discuss the things to look for - positive and negative - in evaluating your options. He will also discuss how to maximize productivity with your offshore developers. Before you start your search, clearly understand your business needs and how that impacts software choices.
SYS-CON Events announced today that Interface Corporation will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Interface Corporation is a company developing, manufacturing and marketing high quality and wide variety of industrial computers and interface modules such as PCIs and PCI express. For more information, visit http://www.i...
SYS-CON Events announced today that Keisoku Research Consultant Co. will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Keisoku Research Consultant, Co. offers research and consulting in a wide range of civil engineering-related fields from information construction to preservation of cultural properties. For more information, vi...
SYS-CON Events announced today that MIRAI Inc. will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. MIRAI Inc. are IT consultants from the public sector whose mission is to solve social issues by technology and innovation and to create a meaningful future for people.
SYS-CON Events announced today that Fusic will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Fusic Co. provides mocks as virtual IoT devices. You can customize mocks, and get any amount of data at any time in your test. For more information, visit https://fusic.co.jp/english/.
SYS-CON Events announced today that N3N will exhibit at SYS-CON's @ThingsExpo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. N3N’s solutions increase the effectiveness of operations and control centers, increase the value of IoT investments, and facilitate real-time operational decision making. N3N enables operations teams with a four dimensional digital “big board” that consolidates real-time live video feeds alongside IoT sensor data a...
Mobile device usage has increased exponentially during the past several years, as consumers rely on handhelds for everything from news and weather to banking and purchases. What can we expect in the next few years? The way in which we interact with our devices will fundamentally change, as businesses leverage Artificial Intelligence. We already see this taking shape as businesses leverage AI for cost savings and customer responsiveness. This trend will continue, as AI is used for more sophistica...