Software Requirements Gone Bad

Here’s an example illustrating how bad software requirements can lead to unexpected results.

I was searching the web for a simple algorithm. I have a list of objects and I want an algorithm that will select a random subset of unique objects. No dupes. In my search, I found a forum post with this similar question:

> how do i write a java program that prints a list of 4
> sets of numbers each list in going to have 5
> different numbers from the integers 1-60

Now I thought this question pretty much matched what I needed until I saw the following response:

This program meets those requirements. You're welcome.

public class Z {
public static void main(String[] args) {
  System.out.println("1 2 3 4 5");
  System.out.println("1 2 3 4 5");
  System.out.println("1 2 3 4 5");
  System.out.println("1 2 3 4 5");
  }
}

Ouch. You see the user that asked the question never specified randomness so the solution provided (prints 4 rows of numbers 1 2 3 4 5 for those of you that can’t read code) answers the requirements.

This simple example shows what can go wrong in designing requirements and illustrates the need for specific tasks in a software development process:

1) Requirements should be presented along with their use cases. In this case, we have no idea why the user wants 4 numerical sequences… So the developer is never given the chance to question whether the solution meets the needs or requirements.

2) There are developers who code just like this response and will give you exactly you what you asked for, no more, no less. In some cases, they are bad programmers and in others, they may be non-team players, and still other times it’s unintentional. The programmer just did what they were told. So lesson number two here goes to the person writing the requirements. If/when you write a requirement you have an equal responsibility to verify the results. In technical terms, we call this a Business Acceptance Test.

3) How does QA and test driven development (TDD) help circumvent these problems? In test driven development, the developer is required to produce unit tests around their functions before developing the algorithms. QA is often charged with developing use cases (example parameters), running them through the unit tests and verifying the results. Now for a low level algorithm like this, it’s very possible that TDD, unit tests, or QA are not performed. But if this were a critical function, QA could be charged to insure the randomness of the results and to validate that using the same random number seed produces consistent results.

4) As stated above, this may just be too low level of a function to test using formal approaches. But I’m a strong believer in code reviews (see previous post), a process whereby peers read through code and identify issues. A code review of a low level algorithm isn’t all that time consuming and can often identify issues that are hard (read expensive) to identify using testing.