Hitting the 10-Gbit/s Backplane Mark

Hitting the 10-Gbit/s Backplane Mark

Leo Wong, Rambus
Apr 08, 2004 (3:00 AM)
URL: http://www.commsdesign.com/showArticle.jhtml?articleID=18900285

As the demand for more bandwidth continues to grow, rather than investing in a forklift upgrade, IT managers are demanding more performance and longer product life from their existing equipment. This leaves system designers no choice but to find creative ways to amortize every penny invested in existing backplane technologies.

The seemingly simple solution is to extend the life of the copper backplane by decreasing the unit interval each time the data bandwidth requirements increase. Unfortunately, the predominance of loss, reflections, crosstalk, and skew encountered at higher speeds present an entirely different priority of challenges to OEMs attempting to stretch the performance envelope of their last-generation systems.

To solve today's backplane dilemma, one must address the signal integrity issues that pose the most serious problems at data rates in the 3- to 10-Gbit/s range: skin effect, dielectric loss, reflections, crosstalk, inter-symbol interference (ISI), and intra-pair skew. Increasing the speed of legacy two-level I/O or utilizing common copper cable equalizer are insufficient because those techniques were developed to overcome the nature and extent of the channel impairments typical to lower-speed backplanes.

Today's engineers must adopt the appropriate techniques and technologies that will enable them to achieve acceptable bit error rates (BER) for data rates approaching 10Gbps. Among the most effective of these are the use of a multi-level signaling technique called pulse amplitude modulation (PAM), and the use of an adaptive equalization technology known as decision feedback equalization (DFE).

Another issue facing OEMs is determining the best method to deploy these enhancement technologies in their legacy backplanes. Is it best to create a custom ASIC or will an off-the-shelf ASSP suffice? The answer depends on the economies of scale involved and the nature and specifications of the system.

Channel Impairments
The backplane is a complex environment consisting of many components and represents a serious challenge to signaling rates above 5 Gbit/s. As shown in Figure 1, the signal path includes over eleven different components, each of which has its own impedance variations. In addition there are up to 10 vias in the signal path, each having both a through and stub component, and thus each presenting an additional potential impedance discontinuity and the commensurate resonant pole. As a result, the transfer functions of channels in this environment vary significantly.

Figure 1: In a typical backplane, as shown here, each active and passive component presents different signaling challenges.
At Nyquist frequencies below 2 GHz there are some channel differences but the presence of vias and impedance discontinuities (reflections) is not significant. Above 2GHz, channels vary significantly depending on the signaling layer (and thus the through/stub ratio of the via), trace length, and dielectric material (and thus the skin and dielectric loss). Achieving high data rates across this variance of channel behaviors presents a significant challenge for high-speed serial links.
Two of the more destructive channel impairments encountered in high-frequency backplanes are inter-symbol interference (ISI) and reflections. Each has unique sources and effects, however an innovative application of adaptive equalization techniques can overcome the effects of both.
ISI Challenges
One of the significant effects of channel dispersion is the "spreading" of a single-bit response, which causes ISI to adjacent symbols. Consider ISI in the frequency domain. The backplane channel behaves like a lowpass filter, where high frequency components are attenuated and low frequency signals travel largely unaffected (Figure 2).

Figure 2: (a) Backplane S21 curve; behaves like a low-pass filter. (b) Inverse frequency equalizer S21 curve; behaves like a high-pass filter. (c) Combined S21 curve; a flat and desirable transfer function across targeted frequency range.
Designers can also view ISI in the time domain by analyzing the single-bit response of the channel. Figure 3 demonstrates the destructive effect of ISI on a simple 101 pattern transmitted down a lossy channel to a receiver. The resulting error is induced by "pre-cursor" ISI from the blue waveform added with "post-cursor" ISI from the green waveform, the sum of which produces a voltage for the "0" bit significantly above the 0/1 voltage threshold.

Figure 3: A simple 101 pattern without equalization at the input to channel (black) and at the output of the channel (red).
The most common approach to cancel ISI is inverse frequency equalization. In the backplane link environment, the challenge is performing effective equalization at very high performance with very low cost in area and power. Transmit equalization (often called pre-emphasis or de-emphasis) is a simple and often effective way of coping with dispersion-induced ISI. In transmit equalization, low-frequencies are attenuated relative to the signaling Nyquist frequency, thus flattening the overall system response and removing ISI (Figure 4).

Figure 4: Single-bit response without equalization and a 5-tap equalizing transmitter showing the reduction of ISI with transmit equalization.
Note that in Figure 4 the output swing has not been increased for the equalized case and the system has maintained a consistent peak-power constraint in order to make a fair comparison. Despite a smaller single-bit height, the removal of ISI through transmit equalization effectively improves signal-to-noise ratio (SNR).
Reflections
Virtually all high-speed backplane enhancements must overcome substantial increases in reflections. Reflections due to impedance mismatches occur for a variety of reasons. To understand the causes of reflections, one must thoroughly analyze various elements in the backplane.
As shown in Figure 1 above, chips are mounted in packages that are soldered to line cards that plug into the backplane. The channel is the complete path from one die to the other die. The signal has to traverse a number of traces to get from source to destination. Line attenuation due to skin effect and dielectric loss occurs along the long horizontal traces.
However, the most troubling effect is not derived from the long traces but the short vertical traces that connect the components of the system. These vertical traces, also known as vias, connect the package to the line card, and from the line card into the connector and the backplane.
The vias have strict size and spacing requirements set by PCB and connector manufacturing constraints, which are sometimes in direct conflict with requirements for good electrical performance. The connectors themselves frequently have internal impedance discontinuities, or can have discontinuities when combined with line-card and backplane vias in a real system. Time domain reflection (TDR) analysis can uncover such impedance discontinuities (Figure 5).

Figure 5: Reflections reduce the signal amplitude delivered to the receiver and cause resonance dips in the channel transfer function.
The DFE Answer
Decision feedback-based receive equalization (DFE) can be effective when dealing with loss and dispersion ISI, but it also helps minimize configuration-dependent reflections as well. This technique uses both transmit and receive equalizers combined to make a range-restricted DFE with effective ranges as shown in Figure 6.1

Figure 6: (a) Equalization architecture with combined transmit and receive to make a DFE. (b) Equalization tap ranges overlaid on channel single-bit response.
Since dispersion varies as a function of many properties in backplanes, flexibility in the transmit equalizer, both in the number of taps and in tap settings, is highly desirable. Similarly, as the receive equalizer is predominantly used for minimizing reflections, flexibility in tap assignments and weights is critical for dealing with the varying reflections present in different high-performance backplane configurations.
One of the challenges of any equalization architecture is setting the tap weights or equalization coefficients. In a typical backplane environment with substantial channel-to-channel variations, there is no simple set of coefficient settings that will work for all channels. By using adaptation, one can simultaneously determine the optimum solution for each of the equalization coefficients.
The two basic adaptation methods are "set and forget" and "continuous". In the "set and forget" method, the adaptation loop runs during power-up to establish the coefficient settings, after which the adaptation loop shuts down and the link runs with the coefficients fixed.
In the "continuous" method, coefficients continuously, automatically adapt during live data transmission. Thermal and humidity variations are the two most common effects requiring continuous adaptation in the backplane. They in turn cause changes in the channel transfer function. It has been shown that, humidity variations, in combination with temperature 60 deg. C or higher cause variations up to 10dB in channel performance at 6GHz.² The dynamic nature of the transfer function is previously ignored, and more investigation is required. Since the channel itself is changing, device must employ some sort of continuous, adaptive equalization to compensate. Programmable or "set and forget" method will not be able to achieve and maintain acceptable BER over the variances.
Multi-Level Signaling
An alternate method of dealing with the increase in losses when running faster frequencies in the backplane is to simply use voltage to increase the data rate (i.e., multi-level signaling) instead of time. In traditional binary signaling, only a single bit is transmitted and received during each symbol time. Using multi-level signaling methods such as PAM, multiple bits are transmitted during each symbol time, allowing the symbol to run at a lower Nyquist frequency to achieve the same data rate. The technique called 4-PAM uses 4 such levels to encode 2 bits per symbol, as shown in Figure 7.

Figure 7: Two ways of signaling two bits (a) in time with binary signaling and (b) in voltage with multi-level signaling (4-PAM).
Any system which has greater than 10 dB of loss difference between the 2-PAM and 4-PAM Nyquist fundamental frequencies would likely benefit from 4-PAM signaling. This becomes clear from a simple first-order analysis of the relative eye sizes. The transfer functions of two example backplane channels and their resultant 2-PAM and 4-PAM eyes running at 6.4 Gbit/s are shown in Figure 8.

Figure 8: Comparison of 2-PAM vs. 4-PAM for two different backplane channels.
It is interesting to note that both channels are from the same backplane with equal trace length and equal total via length. The only difference in these channels is the backplane signaling layer, and thus, the ratio of through-via to stub-via.
In the top panels of Figure 8 (blue S21), the transfer function is not very steep between the 4-PAM Nyquist frequency of 1.6 GHz and the 2-PAM frequency of 3.2 GHz. The 2-PAM eye has superior voltage margin in this case.
In the bottom panels of Figure 8 (red S21), the channel characteristics show a difference in the transfer function at 1.6 GHz and 3.2 GHz of almost 30 dB, and as expected, the 4-PAM eye shows superior voltage margin in this case.
As these two channels are almost identical physically, but so different electrically, this clearly demonstrates that there is no definitive answer to the question" "Which is better — 2-PAM or 4-PAM?" One must conclude that each channel's characteristics will determine the appropriate choice for that channel. Thus, it is imperative that engineers perform careful worst-case analyses, but it is just as important that they deploy a flexible equalization solution.
Deployment
System designers re-vamping their backplane to accommodate higher data rates have two choices—buy an ASSP off the shelf or design an ASIC to do the job. Two primary factors—commercial and technical—determine the proper deployment vehicle to extend their backplane performance.
If the target system ships in low volume, an ASSP is typically the most cost-effective choice. For example, in the case of a 500-Gbit/s core router that ships less than 500 units per year, an ASSP might cost approximately $200 in modest quantity, making the total cost around $100,000 per year. On the other hand, the average non-recurring engineering (NRE) costs for a sophisticated ASIC could exceed $2 Million. Simply put, a modest quantity volume may not justify the high up-front NRE costs associated with an ASIC solution.
There are cases, however, where it is simply not feasible to implement an ASSP, regardless of cost. Consider a switch fabric that must integrate 128 channels. One simply cannot install 128 single channel discrete serializer/deserializer (serdes) or 36 quad-channel discrete serdes on a board. The number of PCB layers and the complexity of the signal routing would be a nightmare. In such a case, an ASIC is the clear choice.
References

Zerbe et al., "Equalization and Clock Recovery for a 2.5 to 10 Git//s 2-PAM/4-PAM Backplane Transceiver Cell," ISSCC 2003 Digest of Technical Papers.

G. Sheets et al., "The Impact of Environmental Conditions on Channel Performance" DesignCon 2004.

Wrap Up
Leo Wong runs 10 Gb and backplane product planning and market development at Rambus. Previously, he held senior management positions with BitBlitz and Altera Corp. Leo holds degrees in electrical engineering and computer science from UC Berkeley and can be reached at lwong@rambus.com.

Copyright © 2003 CMP Media, LLC | Privacy Statement