Functional qualification: a technical brief

By Mark Hampton, Certess Inc.
edadesignline.com (March 02, 2009)

Introduction

If there were a bug in your design, could the verification environment find it? Functional qualification is the first technology to provide an objective answer to this fundamental question. It is an important addition to the solutions available for the increasingly challenging task of delivering functionally correct silicon on time and on budget. Functional qualification enables the rapid improvement and cost reduction of verification. The core technology underlying functional qualification is mutation analysis [1]. Mutation analysis has been actively researched for over 30 years in the software testing community (with, among others, PIMS [2,3], Mothra [4,5,6], Proteum [7], Jester [8], MuJava [9], Jumble [10], etc.) but Certess provides a commercial tool (Certitude) that uses this technology within the electronic design automation (EDA) space.

Functional qualification overview

To be effective, verification must ensure that designs are shipped without critical bugs. To find a design bug, three things must occur during the execution of the verification environment:

The bug must be activated; i.e. the code containing the bug is exercised.
The bug must be propagated to an observable point; e.g. the outputs of the design.
The bug must be detected; i.e. behavior is checked and a failure indicated.

Traditional EDA technologies have focused on item 1, activating the bug. Techniques such as code coverage and functional coverage can help ensure that design code is well-activated, but they cannot guarantee that design bugs will be propagated. Nor can they guarantee that the bugs will be detected by the checkers, assertions or comparison against a reference model.

Functional qualification automatically inserts artificial bugs into the design and determines if the verification environment can detect these bugs. A known artificial bug that cannot be detected points to a verification weakness. If an artificial bug cannot be detected, there is evidence that actual design bugs would also not be detected by the verification environment. A functional qualification tool, such as Certitude from Certess, helps the user understand the nature of these verification weaknesses. Functional qualification is able to provide new information to the verification engineer (verifier). For the first time, verifiers can measure the ability of their verification environments to propagate and check potential design bugs. Put more bluntly, Certitude is the first tool to measure the quality of their work comprehensively.

A functional qualification run (or "a qualification run") is performed in two possible modes:

Metric mode: In this mode, Certitude provides an objective and comprehensive score of verification quality. This score can be used to benchmark different verification environments.
Verification Improvement mode: In this mode, Certitude identifies specific faults that cannot be detected so that the verification environment can be improved, and to ensure there are no real design bugs coupled to this fault.

For most engineers, functional qualification is a new concept, which naturally raises questions. The following sections are intended to provide answers to the most common concerns.

Click here to read more ...