Simulators to run benchmarking suit on soft cores

Simulators to run benchmarking suit on soft cores
By Stephan Ohr, EE Times
October 9, 2000 (3:55 p.m. EST)
URL: http://www.eetimes.com/story/OEG20001009S0054

SAN JOSE, Calif. — The EDN Embedded Benchmarking Consortium (EEMBC) will announced this week that its benchmarking suit can be run on simulators as well as on production silicon. The development, which will be announced at this week's Microprocessor Forum, will allow softcore and intellectual property suppliers to run, certify and publish EEMBC benchmarks, said Marcus Levy, editor of MicroDesign Resources and chairman of EEMBC.

About 30 percent of the EEMBC member companies are softcore vendors with products used in system-on-chip designs. The group includes processor vendors like NEC, Hitachi, ARM, Infineon and Philips, toolmakers like Green Hills and Wind River, and EDA tool vendors like Cadence Design. While some member companies such as ARM Ltd. or MIPS Technologies Inc. can point to existing silicon that incorporates their products, younger companies like ARC Cores and Tensilica do not yet have production silicon on which to run the EEMBC benchmarks. The use of cycle-accurate simulators will allow potential users to evaluate the performance of these companies' cores against competitive units, before going to actual silicon.

EEMBC (pronounced "embassy") was formed in a effort to come up with a series of application-based performance metrics that would allow users to compare processors for embedded applications. The Mips rate, for example, is tied to clock speeds, and does not accurately reflect the work performed by processors with variable bus widths and instruction words. The EEMBC benchmark kernels were formulated for specific real-world applications like communications, automotive and consumer electronics.

The new EEMBC regulations allow core makers to use simulators to compare the performance of their products. Simulation benchmarking, however, requires that the simulator accurately reflect real-world characteristics with cycle accuracy. While hardware and software simulations are allowed, vendors must provide an assessment of al l resources (for example, the peripherals, memory and I/O structures) utilized to run the timed portions of the benchmark suite, not just the processor core. Manufacturers must verify the performance of their cores with the appropriate register transfer language simulator, such as Verilog or VHDL. EEMBC's Certification Labs will recreate the simulation environment, re-execute the benchmarks on that simulator, and verify the disclosure reports in the same manner as for hardware-based certification.

"We can't have people making up stuff — '2000 Mips!' — and just get EEMBC to endorse it," Levy said. EEMBC certification requires that the performance of the softcore with its peripherals be verified with cycle-accurate RTL simulators.

As with benchmarks run on production silicon, scores will be reported in cycles rather than in time.