FPGA configures DSP core in imaging app

FPGA configures DSP core in imaging app
By David Main, EE Times
November 24, 2003 (12:13 p.m. EST)
URL: http://www.eetimes.com/story/OEG20031121S0040

When Intevac developed the embedded electronic systems for NightVista, a compact, high-performance, ultralow-light camera, the first approach was built around a Texas Instruments DSP. But after a few months of hardware development, including a board respin, it became clear that this solution wouldn't hit the performance targets and that it was too heavy and used too much power. It was also clear there would be significant thermal management problems due to the high power dissipation in the resulting high-density stacked pc board configuration. So the DSP-based design was scrapped in favor of a hybrid systolic logic and soft-core microprocessor implementation based on a field-programmable gate array (FPGA).

Though Intevac had no experience with using microprocessors integrated into programmable logic, the evaluation using a demonstration kit looked very promising since it was apparent that the latest low-cost FPGAs had the capacity to integrate a co mplete 32-bit RISC processor along with memory blocks, phased-locked loops (PLLs) and plenty of logic resources for implementing video-specific signal-processing functions. The use of PLLs integrated into the FPGA promised to solve numerous problems associated with board-level multiclock systems. Based on a number of criteria it was decided to pursue a solution based upon Altera's Nios processor implemented in a Cyclone FPGA.

After making this decision, Intevac had to determine how much of its existing DSP software could be ported over to the Nios processor. Eighteen man-months of software development with the prior DSP attempt had brought the team to the point of passing image data through the processor to the output, but with no video processing. The role of the Nios processor in the FPGA-based camera design was so dramatically different that only the RS-232 serial communication protocol used to communicate with the host PC and the video sensor configuration protocol could be reused. Fortunately, softw are development for the new processor was straightforward; by using a Nios development board Intevac established communication between the processor and the host PC within hours.

It would be a month before the new FPGA-based design's circuit boards would be finished; in the interim, Intevac continued to create and exercise its code via a Nios processor development board. With the DSP, Intevac had expected to use the provided real-time operating system (RTOS) to manage the complex timing of its video processing algorithms. The new processor did not include an RTOS out of the box, and the software team encountered many cases where it was unsure whether it would be able to meet all of its timing requirements. After consulting with the hardware team, the software people soon discovered that the configurable nature of this new processor allowed for a high degree of control over the timing of signals, and often only minor changes to the FPGA design were needed to reach timing goals requirements. The highly in tegrated nature of the hardware and firmware processing within the same FPGA environment allowed quick and easy optimization of control and video processing tasks.

Further exploration of these possibilities led Intevac to develop custom functions and peripherals to meet its exact needs. The hardware and software teams met frequently, discussing how to partition tasks between the FPGA and the processor on a white board. If there was a bottleneck in the software, it was an easy task for the hardware team to develop a coprocessor to speed things up; often, it had the result up and running within an hour.

The hardware team designed a custom video encoder, FIFOs to buffer video data and specialized DMA controllers to keep the encoder fed with a constant video data stream, eliminating the need for an external encoder and FIFOs. They also built a custom SDRAM controller to improve performance by allowing all video, attribute and Nios processor command fetch and data storage needs to share the same memory de vice. Several of these functions required their own clocks, so the FPGA's onboard PLLs were employed to generate three different clocks from a master clock: one for the video encoder, another for SDRAM timing and a third for the external pixel sensor.

In addition to implementing functions that had previously required external devices, Intevac added functionality that wasn't possible with the DSP implementation. Video test pattern generators were added to simulate camera operation, enabling the software team to perfect the various algorithms for video processing and to assist in system alignment. Another addition was a statistics generator, which was used to analyze the nature of the video data in order to make decisions about image enhancement and image intensification. The image statistics generator required several mathematical operations that would have been too slow if implemented in software, so Intevac used logic resources in the FPGA to build the generator from logic instead and set it up to pass its results to the processor.

After receiving finished circuit boards, it took a only few hours to transfer the code from the development board and get it up and running on the new board. For the next few months, software and hardware development continued in parallel while Intevac further refined and debugged the design. Both the processor and the rest of the FPGA design were modified many times, without any impact on the board layout.

In the end, using the Cyclone device and soft-core processor to achieve a system-on-a-programmable chip reduced five boards of components to a single board. This integration lightened the camera, reduced the required number of supported voltages from four to two and achieved a nearly 80 percent reduction in power consumption. It also enabled Intevac to efficiently produce multiple products using the same board set.

In addition, the simplification of the design greatly reduced component and manufacturing costs and increased NightVista quality and reliability. This solution also brought increased functionality beyond the initial product specification, and by leaving additional resources in the FPGA Intevac can develop further upgrades that can be added to the camera while it is in the field. Finally, this solution enabled Intevac to explore and refine a more rapid and efficient design development flow, which will save a great deal of time and resources in future product development.

A partial list of the functions of NightVista's electronic systems includes: camera power-on test and initialization; video sensor calibration and characterization; automatic gain control for image intensification management; on-screen display of graphics, text and watermarks; real-time adaptive contrast adjustment; gamma correction, video freeze frame capture and storage to flash memory; real-time clock; programmable user-defined preset configurations, communication with the host PC via RS-232-C; remote update of camera features and parameters and transferring of video data from the c amera to the host PC.

David Main is engineering group manager and Vladimir Adam is a software engineer at Intevac (Santa Clara, Calif.); Martin S. Won is senior member of the technical staff at Altera Corp. (Santa Clara).

See related chart