Software Quality

What is Software Quality?

Conformance to requirements:

• Lack of bugs
  • Low defect rate (# of defects/size unit)
• High reliability (number of failures per $n$ hours of operations)
  • Measured as mean time to failure (MMTF), i.e., the probability of failure-free operation in a specified time

Software quality:

1. Relates to verification “Are we building the product right?”
   • Verification is the process of checking that the software meets its stated functional and non-functional requirements
2. Relates to validation “Are we building the right product?”
   • Validation is a more general process with the aim of ensuring that the customer will be satisfied with the outcome

Software Quality Assurance:

Importance of Software Quality

• Software is a major component of computer systems (about 80% of the cost) used for
  • Communication (e.g., phone system, email system)
  • Health monitoring
  • Transportation (e.g., automobile, aeronautics)
  • Economic exchanges (e.g., e-commerce)
  • Entertainment
• Software defects may be extremely costly in terms of
  • Money
  • Reputation
  • Loss of life

Software Quality Factors

• Product operation → the extent to which the software fulfils its specifications (i.e., requirements)
• Product revision → Ability to change the software
• Product transition → Adaptability of the software to new environments
• Correctness → The functionality matches the spec
  • Accuracy and completeness of required output
  • Up-to-dateness and availability
• Reliability → The extent to which the system does not fail
  • Maximum failure rate
• Efficiency → System resource usage (CPU, disk, Memory, network)
  • Hardware resources needed to perform software function (processing capabilities, data storage, bandwidth, power usage)
• Integrity → Protection from unauthorized access
  • Software system security, access rights
• Usability
  • Training required, ability to learn and perform required task
• Maintainability
  • Effort to identify and fix software failures (modularity, documentation etc.)
• Flexibility
  • Degree of adaptability (to new customers, tasks, etc)
• Testability
  • Support for testing (e.g., log files, automatic diagnostics, etc), traceability
• Portability
  • Adaptation to other environments (hardware, software)
• Reusability
  • Use of software components for other projects
• Interoperability - Ability to interface with other components/systems

From PD10

In the context of Software Engineering, software quality refers to two related but distinct notions:

• Software’s functional quality reflects how well it complies with or conforms to a given design, based on functional requirements or specifications. That attribute can also be described as the fitness for purpose of a piece of software or how it compares to competitors in the marketplace as a worthwhile product. It is the degree to which the correct software was produced.
• Software structural quality refers to how it meets non-functional requirements that support the delivery of the functional requirements, such as robustness or maintainability. It has a lot more to do with the degree to which the software works as needed.

Approaches to software quality:

1. Verification of the product
2. Certification of the process used to build the product
3. Certification of the people who build the product

Verification of The Product

An analyzer checks compliance of these coding standards and the analyzer runs every night. In fact, at NASA, there is considerable emphasis on automated code analyses. Theses tools report problems that exist outside of the executing of the code (e.g. errors like memory leaks or impossible to execute fragments of code).

Just as the analysis tools examine the code looking for specific types of errors, developers will manually read the code to look for errors. Code inspections are severely underrated. They are very cheap to perform, and highly effective - more effective at finding errors than testing is.

There is a strong emphasis on using assertions as a form of self-testing of the code. Each assertion is effectively a small sanity check embedded in the program that executes whenever the surrounding code executes.

Certify The Process

When public safety is at stake, assessing safety and product quality does not rest solely with the product’s developer. Independent regulators are also responsible to certify that the product is safe.

Surprisingly, the certification team does not independently re-verify the product. Instead, they assess the development process. Specifically, they check that the process was thorough and that it complies with any process standards that exist for the particular application domain.

Therefore, the developer is still solely responsible for using product-focused techniques to verify their software. In addition, though, they must document the software development process that they used. Documenting would include such things as:

• Requirements followed and validation of requirements
• Design used and design rationale
• Verification of the code

The certification team bases their assessment of the development process on this documentation.

Hire Certified People

A third approach to improving software quality is to hire qualified developers.

Hiring qualified people means either assessing their education during the interview process or hiring personnel whose qualifications a professional organization has already assessed and approved (e.g. personnel certified by the Professional Engineers Ontario - PEO).

IN ONTARIO, (AND IN MOST OTHER PROVINCES IN CANADA), ANY DEVELOPER WHO CONTRIBUTES TECHNICALLY TO THE DEVELOPMENT OF A SAFETY-CRITICAL SYSTEM IS REQUIRED, BY LAW, TO BE LICENSED AS A PROFESSIONAL ENGINEER.