Metrix Driven Hardware Software System Level Verification

Aniruddha Baljekar, Srinivas Puttur NXP Semiconductors Pvt. Ltd., INDIA

Abstract

Industrial data shows that verification takes about 70 to 80 % of the total project development time. With the increasing complexity of the SoC and with increasing software content in the sub-system, System Level Hardware Software verification of the sub-system is one of the challenges for the verification teams. In addition to HW SW coverification, coverage information of the verification goals is extremely important to check the progress of the verification and conclude when the verification cycle can end. One of the other challenges is to be able to run the same system level tests with:

• RTL
• Netlist with SDF timing annotation

To reduce the verification process, re-use of the verification from the IP level to the System Level Verification is also extremely important to improve TTM.

The scope of the paper is to explain the constraint randomized OVM [1] based CDV methodology [4] adopted for HW SW co-verification at System Level Verification by making use of IP level verification resources and enable reuse of the verification framework across product lines within our business Lines.

Our paper details the following:

• Virtual sequencer to co-ordinate the sequences executing “C-based” test cases/SW running on the processor via generic software adapter and generate responses from the OVC’s in the SoC.
• Generic software adapter (SW eVC [ISX]) to map the SW API to call the required function implemented in “C” from Virtual sequencer.
• Scalable Verification environment for scalable architectures
• Constrained Random generation of test parameters/arguments for SW/Firmware running on Processor from OVM
• Multi-language support (“e” to SV and vice-versa) with OVM.
• OVM based sequences/scenarios for IP’s used in the System.
• Re-use of the sequences/scenarios from IP level to system level.
• Re-use of coverage details, Checkers, scoreboard etc. from IP level to system level.
• Re-use of IP level assertions at System level during dynamic simulation
• Transaction recording for SV and “e” sequences for ease in debugging

The environment uses following tools/vIP’s:

• ISX (SW eVC) from Cadence
• OVC’s developed using OVM methodology
• eVC’s

1. Introduction

The paper details out CDV methodology for SoC Verification and advantages of OVM based methodology over other conventional approaches.

• It captures the methodology of controlling the execution of testcases/API on the processor from multi-channel sequencer (virtual sequencer).
• It captures the architecture of verification environment to build a scalable verification environment re-usable across different architectures.
• It explains the methodology to enable mixed language integration between “SV” and “e” languages.
• It also details out why re-use of unit level verification environment is important for SoC verification and how unit level verification environment can be re-used for SoC verification.

2. Motivation

Following are the motivations for adopting OVM based methodology for SoC verification:

1. Unified methodology for SoC verification is the need of the hour to enable re-usability across different architectures.
2. Verification environment has a consistent/uniform structure
3. The methodology should also enable leveraging the unit level verification environment, verification components of the IP’s to SoC verification.
4. The verification environment should be capable of generating traffic to enable architecture, performance testing besides interconnection etc kind of testing.
5. It should enable generation of traffic which is dynamic in nature and mimics the behavior one normally sees in a real chip.
6. Coverage information of SW variables + HW functional paths
7. Verification environment should be re-usable at different levels of abstraction

3. Technical Details

3.1. Verification Environment Overview:

Figure:1 Block Diagram: Verification Environment

3.2. Details of the verification components:

The verification environment is built using OVM methodology [1]. OVM enables the changing of the environment for a specific scenario on the fly and enables generation of constrained random sequences and configure responses from the peripherals/slaves using OVC’s. The responses can be configured to generate good (error free responses) and error kind of responses to check the robustness of the SoC. The environment allows generation of dynamic traffic which is the key requirement for performance testing apart from the functional checks. The environment also allows a specific master/slave to be replaced by the OVC easily, enabling generation of different scenarios otherwise not possible. Since OVM is a great enabler for layered approach, the environment is designed in a layered manner to be scalable. New features to be verified and/or new requirements of the design are extended with minimum effort.

(1) Multi Channel Sequencer

Testcase stimulus is implemented using sequences [4]. Co-ordination between various sequences is controlled using virtual sequences. The sequences contain test scenarios to verify the specific feature IP-SOC 2010 Conference – Nov. 30 - Dec. 1, 2010 3 of the IP’s and virtual sequences for implementing test scenarios. Virtual sequences are used to combine sequences and to control the order of execution of different sequences. This is required in an environment that needs different kinds of scenarios to be executed from processor and to coordinate responses from various peripheral OVC’s. Using a real processor in the various environment enables the testing of the firmware/software closed to the one used for the product.

(2) SW eVC

ISX/Generic Software Adapter (GSA) [2] is a library based e language functions. One can use these functions to write an adapter which enables the verification environment to communicate with the embedded software. GSA functions provide a mailbox memory to implement communication ports between the verification environment and the embedded software. GSA defines the communication protocol to initialize and access this mailbox. The firmware API’s executed on processor can be controlled from the SV virtual sequences using GSA. The methodology is based on a common mailbox implemented in the RAM area which is accessible to both the verification environment and processor. The arguments to be passed from the “e”/SV environment can be randomized and passed to the API’s executed on the processor. The function called from the virtual sequences is posted in the mailbox as the next index to be executed. Based on this index the desired use-case/test sequence is executed.

For integrating the SW eVC to application code stub files need to be generated for the SW eVC. This GSA “c” language stub is compiled with the user application code/embedded software to provide the additional functions to enable integration of GSA with the application code. The generation of the stub is automatically handled by the tool (Specman) in this case.

Figure 2: Mailbox communication structure

The steps below list the flow of execution after executing a sequence from the SV/verification environment

• SV virtual sequences call the function to be executed on ARM. This request is passed from the virtual sequencer to the driver of SV2e adapter. This internally makes a call to the out method port which is mapped to the function to executed from ARM. The out method port is mapped to the specific API/function implemented in the user code.
• During initialization the mailbox status is set by a predefined function on the software side sn_gsa_init(), to the READY state
• The mailbox can take the request to initiate the calls only when it is in READY state. The SV sequence request using the method port mechanism is written to the mailbox. Each of the API/functions are mapped to a unique index. Based on the index posted in the mailbox the application code, reads the mailbox and gets the index requested to execute.
• When there is a pending function call, the status of the mail box is changed to PENDING
• The application code periodically checks the mailbox, retrieves the details of the function to be executed. The application code gets the handle of the function to be executed (based on the function id). The mailbox status changes to EXEC when the function is executed. The SV sequence will not exit till “C” function is exited.
• After the execution of the function the mailbox status changes to READY which indicates that the new commands can be posted in the mailbox.

(3) Peripheral/Master/Slave OVC’s

The peripheral responses are generated using the peripheral OVC’s. These OVC’s can generate and drive traffic on the bus and/or respond to the traffic as a slave. Protocol compliance check on the interface of the peripheral coverage of the bus traffic is built-in feature of OVC’s. The traffic can be easily configured as a fully random/constrained random response. Monitor part of the OVC’s provide the necessary hooks to check data integrity e.g. PCI OVC, USB OVC, OCP OVC, CAN OVC etc

The environment consists of OVC’s which are configured as PASSIVE/ACTIVE agents based on the test scenario requirement. Some of the master/s e.g. Processor can be easily changed to BUS OVC with ACTIVE agent to generate the kind of regression traffic otherwise not possible from the processor.

This configurable nature of OVC defined by OVM methodology enables the re-use of the verification components from unit level to system level verification. This is an important aspect from the context of robustness of the verification environment and improving TTM.

The protocol compliance tests provided as part of the OVC are used at unit level to check the IP level protocol compliance of the Design. This reduces time taken to write tests for protocol compliance checking by the verification engineer. As these compliance tests are quite exhaustive, protocol noncompliance of the Design are found early during the verification cycle and there is very less probability of protocol non-compliance issues found at SOC verification. The issues which might surface during SOC verification would be mostly due to interconnection problems incorrect IP integration.

(4) Protocol checkers

The protocol checkers which are part of the OVC’s are used for checking protocol compliance for all simulation runs. E.g. PCI, USB, I2C protocol checkers.

(5) Scoreboard/Data checkers

Scoreboard is implemented to check the data integrity of the incoming data and outgoing data. Data on both sides may have different forms, which is the function of the IP. The scoreboard interprets the data and makes it suitable for comparison. Incase the data from both sides is not equal, the specific datapath is not correct and the execution of the test cases can be stopped with an ERROR message. During an end of a simulation all the scoreboard lists will be checked to ensure that the list are empty to ensure that all the data is accounted for. ERROR message is issued in case the scoreboard lists are not empty at the end of simulation runs.

The IP level scoreboards are re-used at subsystem/ SOC level by mapping them to correct interface signals. The functionality of the Scoreboard does not change.

(6) “C” testcase/Firmware running on Processor

The “C” testcase/API’s of HAL are mapped to the GSA. This enables to call the required “C” function/API from the SV sequences. The execution of these functions is controlled from the virtual sequences. This enables the executing of the SW/Firmware closest to the one required for the product which can be controlled from the verification environment. By defining proper coverage matrix it is possible to know the coverage details of “C” API/function/variables. Since these API’s/firmware is the most important part of the overall product, the firmware API’s used for verification of individual IP’s are put together as library and used at sub-system/SOC level to verify the sub-system/SOC functionality. With suitable modifications the API’s/Firmware are re-usable for newer version of the product as ISX makes it easy to re-use.

(7) Coverage

The critical global software variables and functional features important from the SoC context can be covered. The coverage items/vPlan from the unit level verification environment can be easily integrated to the System Level coverage items and cover only the important features from the SoC context.

(8) System verilog to “e” adapter for GSA

This verification component is responsible to translate the SV sequences to “e” sequences [3] using the multi-language interoperability feature of the OVM.

(9) Using SimVision to trace embedded S/W with ISX

ISX provides a SimVision plug-in to help debug embedded software in a simulation environment. The plug-in is a multi-instance window type called Embedded Software Trace. This allows tracing the execution of embedded software along with the hardware design, directly correlate the hardware and software behavior across simulation time. The plug-in is most useful when there is no other way to debug embedded software interactively with an external source-level debugger e.g. gdb. With this its possible to do a post simulation debugging.

Following are the requirements to do a S/W tracing:

• The embedded software executable file must be available in executable and linkable format (ELF)
• Processor program counter (PC) trace is required to correlate the execution to the embedded software source code.
• Utilities required for the compiler and target processor. For arm core the Embedded Software Trace window uses a utility from the ARM compiler tools, fromelf. This tool helps correlate the PC to the software source code

Figure: 3 Embedded Software Trace shows ease of the use of debugging software C code and its corresponding disassembly with respect to instruction execution on the CPU.

Figure: 3 Embedded Software Trace

(10) Transaction recording for “e”/SV transactions

The OVC monitors are extended to provide transaction recording of the transactions. This allows the responses/initiator sequences generated by the OVC to be debugged simultaneously with the hardware i.e. directly correlating hardware and captured transactions. Transaction recording allows the details of the transactions to be captured at a high level of abstraction instead of the details of the pin level transactions. This helps during the debugging of the hardware.

Figure: 4 Transaction recording

3.3. What the verification environment offers?

The verification environment offers/enables following:

• Check interconnection of all IP’s
• Check Interoperability between IP’s
• Check the address map/memory map of IP’s in the SoC
• Check the interrupt configuration and its priority
• Generate dynamic traffic to the SoC for functional verification
• Check the exception handling for the SoC
• Enables checking of parallel paths/dynamic traffic (e.g. traffic concurrently on different peripheral interfaces)
• Enables checking of key performance related parameters critical to SoC functionality eg. low power features, clock gating implemented by the Bus etc.
• Enables the scheduling of regression runs using tools like vManager
• Enables the execution of regression test cases at RTL and Netlist with SDF back annotation.
• Enables the usage of golden reference models e.g. TLM models to check the functionality of the RTL IP/Scoreboard functionality
• Generate coverage reports for Software/Firmware execution
• Generate code coverage information for the Soc/Design.
• Transaction recording of traffic generated on various SOC internal/external bus’s for ease of debug.
• Embedded software trace mechanism to get a trace on the execution of the embedded s/w along with HW simulation

4. Next Steps

• Upgrade the environment to run the actual production firmware/software.
• Tweak the verification environment for reduction in simulation run-time.
• Upgrade the verification environment for automatic document and report generations.
• Performance measurement with production SW
• Improving the quality of scenario libraries/environment based on the historical taped out chip data
• Create metrix for Power measurement, performance parameters of the chip with production SW (simulation v/s silicon)

5. Summary

• SW eVC mapped to a set of API is re-usable across architectures and has no dependency on the processor used in the SoC
• Virtual sequencer enables control over the execution of the application SW and vIP’s
• OVM based verification environment enables to create environment re-usable across different architectures
• The verification environment generation of random and dynamic traffic mimicking the real time traffic enabling the environment to be used for performance measurement
• The verification environment enables coverage of functionality of the block for SoC requirement. It also enables coverage of SW functions and SW variables.
• Environment creation does take more effort than the normal environment but the scalability the platform offers more than compensates this effort.

6. Limitations

Due to multiple vIP’s used in the environment the simulation run time is quite slow. This would prove to be an issue while executing the full product software and debug failures.

7. References

[1] OVM reference manual from OVM world (www.ovmworld.org)

[2] Incisive Software eXtensions user guide

[3] “e” Language Reference from Cadence

[4] IPCM documentation from Cadence

Item	Description
CDV	Coverage/Metrix Driven Verification
eVC	“e” Verification Component
GSA	Generic Software Adapter
IP	Intellectual Property
ISX	Incisive SW eXtensions (from Cadence)
OVC	Open Verification Component
UVC	Universal Verification Component
UVM	Universal Verification Module
vIP	Verification IP
HAL	Hardware abstraction layer
SoC	System on Chip
TTM	Time to Market