VSI Alliance Quality IP Metric

by Kathy Werner, Mentor Graphics
Austin, TX  USA
    
Abstract :

Virtual Components (VC), also known as Intellectual Property (IP), have long been a part of the engineering reality.  Business drivers, such as improved time to market and better resource utilization are factoring ever more into the make versus buy decision process.  Maximizing in-house design resources and purchasing commodity or standard IP has become the de facto business model.  Unfortunately, with the increasing number of IP vendors competing in the marketplace, the decision making process is not obvious.  Simplistically, functionality needs to be the first criteria, but when two or more similar IPs are available, the selection criterion quickly becomes more difficult.  The quality of the chosen IP is key and will have the largest impact on the final product’s time to market.  This paper addresses the process of measuring IP quality, presents a summary of the VSIA Quality IP (QIP) Metric, and reports the on-going work.

Introduction

Several years ago, the VSIA assembled a Quality Study Group to address quality issues.  At the time, many companies reported that IP quality, or the lack thereof, was becoming a significant problem in the IC industry.  Representatives from many of the top companies in the world --representing IP providers, IP customers, as well as companies interested in in-house design reuse  --worked together to first define quality as it related to IP, identified the key areas that impact IP quality, and finally began the process to define a means to objectively and quantitatively measure the quality.  The Study Group subsequently  evolved into a VSIA Development Working Group (DWG) with even more industry representation.  

The resulting QIP Quality Metric, which builds on the Study Group groundwork -- and subsequent technology donations -- is currently in VSIA Member Review, and is concurrently undergoing a rigorous beta testing process.

 Quality

Quality can be defined as a degree or grade of excellence; more concretely, it can be defined as conformance to specifications.  While the statements are intuitive, the application to IP is not.  Historically, the single most important criteria for selecting IP, beyond required functionality, was cost.  The actual experience of using the IP varied greatly between IP providers, end application consumers, and even individual integrators.  There has been no consistent mechanism to communicate an IP’s suitability to purpose between vendors and end customers.  OpenMORE, an early metric based on the “Reuse Methodology Manual”[1] was developed in a joint venture between Mentor Graphics and Synopsys.  Publicly available and an admirable first step, actual usage showed that much of the metric was open to interpretation.  The spreadsheet focused on soft IP and while many valuable criteria were raised, the spreadsheet tended to blur the lines between deliverables, documentation, methodology, code requirements, scripting, and tool issues.  It also did not completely address enough quantifiable checkpoints to ensure robust, synthesizable code that was reusable across tools and platforms.  An unbiased, complete, flexible, and fully quantifiable metric was needed.
 
QIP Metric Overview

The VSIA Quality Study Group and DWG drew on industry expertise, and incorporated donations from companies that pioneered attempts to measure quality.  OpenMORE was eventually donated to VSIA, and many of its concepts were carried forward.  In addition, there were donations of the Quality Attributes Checklist (QAC) from STMicroelectronics, and ChartReuse from Agere and Cadence.  ChartReuse built on the OpenMORE spreadsheet concept by expanding the line items to encompass requirements due to newer tools and methodologies.  It also expanded the scope of OpenMORE by beginning to address IP types other than just Soft IP.  The QAC donation resulted in the Quality Axes concept discussed below.

Quality Axes

As the Study Group progressed in their work to define quality, it became apparent that quality is multi-dimensional.  Many seemingly unrelated factors all influence the end quality of IP.  The Study Group determined that there are four factors that influence the overall quality of IP: authoring, verification, the maturity of the IP, and the capabilities of the provider.  These four major focus areas are referred to as the Quality Axes of the Quality Metric.
 
Quality Scoring

The Quality DWG determined that all quality factors are not created equal.  Every identified line item in the Quality Metric was evaluated for its impact on the end user’s reuse experience and three levels of priorities were assigned.  A default priority level is assigned that provides the baseline by which all IP providers are measured; however these levels may be consciously evaluated or modified by the IP consumers for their own applications and internal systems.

The first priority level, Imperative, is defined as an attribute that must be met, otherwise it may be impossible to use the IP within a user project.  The second priority level, Rule, is defined as attributes that should be met. Failure to meet them may significantly impact the cost of using the IP within a user project. The final level, Guideline, concern attributes that if met, will result in a general improvement in usability and maintainability of the IP, or provide evidence that good practice has been used in the development of the IP.

IP Suitability

The measurement of IP quality is divided into separate sections tailored to the IP provider as well as the end user.  The self-assessment paradigm for the IP providers is intended to measure the quality of attributes that may not be visible to the IP integrator, but impact the reusability of the IP.  These areas contribute to the design robustness along with the ease of integration and reuse by a third party.

A separate section is available for the IP integrators that measures and evaluates the ease of reuse and integration.  Not only does this section provide an independent review of the IP and supporting collateral from the providers, it serves as a continual process improvement mechanism to the IP provider.  The communication of information around IP in a standardized format will lead to more partnering of providers and consumers and pave the way for better overall standardization.

While these areas are important in a global sense, the needs of individual end users may vary with respect to specific IP.  The scoring illustrates the compliance of the IP to a common quality measurement baseline, and is key in providing consistent evaluations for IP.  However, the relative importance of the individual metrics may differ by application.  For this reason, a mechanism is in place to allow the IP consumers to modify the importance level of the individual line items to accurately reflect their needs.  In effect, this allows the end users to customize the rules for their environment, improving the chances of finding an IP suitable for their needs.
 
Status and What’s Next

The QIP is a Quality Metric that will continue to evolve with the industry and technology.  The Metric for digital soft IP is currently in VSIA Member Review with expected VSIA release in fall 2003.  In addition, a formal beta test program is currently in process.  Preliminary work has also been performed by the DWG in the areas of digital verification and also for software and analog IP. Hard IP efforts are just now starting.  Each type of IP has it’s own particular attributes that will ultimately determine it’s quality, but the identified axes of authoring, verification, maturity, and vendor capability are applicable to all.  Persons interested in participating in the development and refinement of metrics in these areas are encouraged to contact VSIA or the Quality DWG chair.
 
Summary

IP quality has long been subjective, if not impossible, to ascertain.  The QIP Quality Metric quantitatively provides the information needed for the end users to quickly compare and select suitable IP using a common baseline.  QIP addresses the needs of the marketplace by providing an objective means to measure IP, reducing the integration time of quality IP, and providing an measurable method to guide IP vendors in the development of and to showcase the quality of their IP.

References

[1]    M. Keating, P. Bricaud, “Reuse Methodology Manual for System-on-a-Chip Designs”, Kluwer Academic Publishers, 1999