Testing software is an integral part of building a system. However, no matter how well the code has been written, if the software is based on inaccurate requirements then the outcome will be unsatisfactory. Many defects can be traced back to wrong, missing, vague or incomplete requirements. My first blog post will explain how DEPT® test requirements before a software solution is built, to help ensure a satisfactory outcome.
A set of ‘tests’ can be used against requirements that cover relevance, coherency, traceability and completeness. However the starting criteria is that each requirement should have at least one quality measure. This measure is used to test whether any given solution satisfies or does not satisfy the requirement.
The aim is to trap requirements-related defects as early as possible, to prevent incorrect requirements being incorporated into the design and implementation, where they are more difficult and expensive to find and resolve.
The requirements tests
We’ve developed a checklist to ‘test’ the requirements that we are reviewing. You can download the checklist from here. Below is an explanation about the tests.
It’s important to note that each of the following tests should be applied to each requirement in unison.
1. Is the requirement measurable?
When analysing a requirement, you need to identify the quality measure. So for example, a requirement that simply states “the system must respond quickly to customer enquiries” can not easily be measured, because the term “quickly” is too ambiguous. A unit of time would need to be agreed (e.g. minutes) and then the stakeholders would need to consider how many minutes delay would constitute a failure (e.g. longer than 3 minutes).
Therefore your recommendation would be to amend the requirement to state “the system must respond to customer enquiries within three minutes”.
You will come across requirements that do not appear to have an obvious means of measurement, for example “the new POS user interface must be easy to learn”. Again, the term “easy to learn” is ambiguous and not specific enough to measure. Therefore consider how this could be measured and what would constitute a failure. In this instance, you could think about a novice user learning the system and the timescales you would expect them to be able to use it productively (e.g. 30 minutes). Therefore your recommendation could be to amend the requirement to state “a novice user must be able to learn to successfully complete a customer order transaction within 30 minutes of first using the system”.
2. Is the requirement coherent/consistent?
You may find many requirements are open to interpretation. Requirements should be understood in the same way by each person who reads them and they should be specified as such.
When you come across a common term used in various parts of a requirements specification, you need to check that the meanings of that term are properly defined. Then check that every use of that term is consistent with the meaning defined.
3. Are the requirements complete?
When reviewing requirements specifications, it is important to ask the right questions to ensure all the relevant requirements have actually been captured and specified. It is accepted that requirements evolve and new requirements are added, however it is important that new requirements are indeed ‘new’ and not requirements that should have been captured originally.
When looking at the context of the requirements specification, consider anything that needs to be built, changed or removed. Any interfaces or effects on other systems, people or processes.
Question whether parts of the system external to the software have been considered. Are any of the interfaces around the boundary of the context being changed?
Another test for completeness is whether all the requirements currently known are captured. The types of requirements to search for are:
- Conscious requirements : problems that the new system must solve.
- These are easier to discover because topmost in stakeholders minds.
- Unconscious requirements : already solved by the current system.
- These are more difficult to discover because if a problem is already solved by a current system, then it is less likely to be thought of and mentioned as a requirement for a new system.
- Unthought-of requirements : would be a requirement if it could be imagined or known to be possible.
- These are the most difficult to discover and usually only surface once the new system is in use (“I didn’t know that would have been possible otherwise I would have asked for it”)
If you happen to be reviewing requirements for a project similar to one you have experienced before, it can be useful to compare the requirements with those from the previous project, in order to help identify any missing requirements. This can help discover unconscious and unthought-of requirements.
4. Are the requirements relevant?
Irrelevant requirements can sneak in to requirements specifications as a result of stakeholders not properly understanding the project goals, or because of previous bad experiences with another system, and so including requirements ‘just in case they need it’.
To test for relevance, check the requirement against the stated goals for the project or system.
Does this requirement contribute to those goals? If this requirement is excluded then will it prevent those goals being met? Are there any other requirements that are dependent on this requirement?
Some irrelevant requirements are not really requirements, but solutions. When solutions are mistaken for requirements, then the real requirement is often missed. Also, the eventual solution may not be as good as it could because the designer is not free to consider all the possible ways of meeting that requirement.
For each requirement ask “Why is this a requirement?” Is it there because of a genuine constraint? Is it there because it is needed? Or is it the solution to a perceived problem? If the “requirement” names a piece of technology that could be implemented by another technology, then unless the specified technology is a genuine constraint, the “requirement” is really a solution.
5. Are the requirements traceable?
The progress of each requirement identified needs to be traceable through the detailed analysis, design and eventual implementation phases. You need to be able to map the requirements to the solution for testing purposes.
Each requirement must have a unique identifier (usually a number). Where requirements are connected to each other, the connections should be shown (may be grouped or developed into Use Cases). This is to enable you to keep track of these (i.e. using a requirements tool or a spreadsheet). In the event that one of the requirements changes, you should be able to identify all the parts of the system that are affected.
