IP verification panel advocates standards

Panelists said adequate verification requires, among other things, verification IP.

"Doing good IP verification requires verification IP," said John Goodenough, director of design technology at ARM Ltd.

Sean Smith, chief verification architect at Denali Software, echoed Goodenough's comment and chided the industry for a lack of focus on verification IP. While designers are saying that verification accounts for up to 70 percent of design time, he said, the industry's focus remains on design IP rather than verification IP.

"IP and verification IP are so synergistic," he said. "I don't understand how you buy an IP core and not get verification IP. IP vendors need to be selling solutions to IP problems, not point tools for IP problems."

Smith argued that verification IP has come a long way in the past 10 years, providing much greater customer value than in the past with things like assertion libraries, functional coverage and a greater means to measure performance.

"If you haven't looked at modern verification IP versus what was out there 5-10 years ago, I think you are in for a surprise," Smith said.

Goodenough went on to say that all designs have bugs and that there is no such thing as "correct by design."

"As a user of IP, you need to understand that that's the way it is," Goodenough said. "You need to ask IP vendors what they verified and how they verified it. The most important thing is not what is verified, but what is not verified."

Harry Foster, chief methodologist at Jasper Design Automation, cited several examples of IP customers devoting excessive money and engineering resources to the verification of IP they had purchased from a third party vendor. He knew of one case, he said, in which a networking provider bought third-party IP and ended up paying another vendor 10 times the original cost of the IP to have it verified.

Satu Lummevuo, verification coordinator at Nokia Corp., said the industry needs a standard for measuring IP quality and deliverables and that IP vendors must comply with them.

"Let's find some way of getting better interoperability between these things," Goodenough said. "Let's set some standards."

Smith applauded the efforts of movements such as PCI-Express and Serial ATA for trying to set some standards for the industry. But, he cautioned, compliance with these standards only provides quality assurance for the scenarios that the standards cover.

"Just because it is compliant does not mean it's verified," Smith said.

Panelists differed about who should pick up the tab for more comprehensive, or "exhaustive" verification.

Lummevuo said third-party IP should comprehensively verify IP cores prior to the sale at their expense. "We should not have to pay extra for that," she said.

Goodenough argued that unless IP vendors could pass along the costs of more comprehensive pre-sale verification and more robust design kits, they would not have the resources to provide them.

"Unless customers can value those features [pay for them], it's very difficult for an IP provider to monetize producing them," Goodenough said.

IBM distinguished engineer Bruce Wile, ASIC chief engineer for the company's Systems and Technology group, described his company's methodical approach to exhaustive verification. He advocated a hierarchical approach to verification, moving up one level at a time and not advancing to the next level until the bug rate has dropped off.

"If you didn't create the scenario that is going to cause that assertion to go off before you get to silicon, then you've missed the bug," Wile said."Exhaustive verification is much easier to reach at the lower levels," Wile said.

Speaking to the enormity of the verification task, Wile cautioned that catching bugs is dependent on creating as many scenarios as possible and placing checkers appropriately.

Doron Stein of Cisco Systems Inc. urged IP providers to offer as much documentation as possible to make it as easy as possible for customers to use.

He said documentation is valuable for many reasons, including psychological factors like easing customer/vendor tension and reflecting positively on the quality of the IP.

"A customer feels that a vendor has nothing to hide," when they provide adequate documentation, he said, noting that it is ultimately more efficient to provide adequate documentation ahead of time than to supply answers to customer questions later.

Lummevuo used the panel to deliver somewhat tongue-in-cheek messages to both EDA vendors and IP providers: make better tools and make better IPs, respectively. She argued that EDA tools should be subjected to a quality standard prior to their release.

"Customers should not have to do tool debugging on production versions of tools," she said.