Multi-Channel Multi-Rate (MCMR) Forward Error Correction (FEC) - IP for High Speed Networking Applications

By Devendra Godbole, Kalpesh Sanghvi (Open-Silicon)

Before we move into the discussion on MCMR FEC, let’s look at high speed SerDes and how it is being adopted. The need for higher and higher bandwidth is growing, which is pushing the SerDes speed to 56G and beyond. The next generation of networking devices are already pushing for 400G bandwidth and above. Both the OIF CEI-56G and IEEE working groups have ratified the 56G specifications, and for higher speed SerDes, PAM4 signaling is the way forward.

The idea behind PAM4 is to map every two bits to a voltage level. These two bits can represent four distinct symbols. Thus, for the same data rate, the symbol is twice that of a bit duration in NRZ. In frequency domain, this implies that the PAM4 requires half the bandwidth of that of NRZ. The Nyquist frequency for PAM4 is 56/4 (i.e. 14Ghz). Hence, we will see more and more adoption of PAM4 signaling for high speed SerDes. However, this also results in a higher bit error rate in the 10^-6 range due to the fact that the margins and swing levels have been reduced. This higher bit error rate will impact the overall bandwidth, which results in the requirement for additional bandwidth. Forward error correction is the technique used to achieve a bit error rate less than 10^-15 with a higher input bit error rate.

The tables below represent details extracted from a web search regarding the different standards for high speed interfaces. The data highlighted in blue is the PAM4 signaling format. The table to the left shows the optical interface standards. It is clear that almost all of them are moving towards the PAM4 signaling format for the high speed SerDes interface. The PAM4 is heavily adopted for the 400G, 200G and 50G, and some of the 100G applications. The 400G BASE-DR4 and 100G BASE-DR are for the 100G PAM4 SerDes channel, and the rest are for the 50G PAM4 SerDes channel. The same is true in the table to the right for the electrical interface.

There are several issues and challenges associated with high speed serial links in a system. As an example, this article will discuss some details on the backplane channel. The diagram below shows a typical backplane with line cards.

For the data to be transmitted from the high speed TX to the high speed RX, there are a variety of different components that are involved in the channel, such as the package to board via, the line card trace, the line card via, the backplane via, the backplane trace, the backplane connector, the chip and the package. All of these form the source for signal degradation.

The signal integrity and quality suffers along the path due to many impairments, like signal distortion caused by channel variation from multiple sources (trace routing, temperature, humidity), inter-symbol interference, reflections due to impedance mismatches at connectors, stubs via, device parasitics, package etc. Additionally, co-channel interference causes crosstalk issues due to jitter, noise, skew and several other factors. All of these factors indirectly or directly impact the overall throughput of the device.

As we have seen, the adoption of PAM4 signaling for higher SerDes speeds is taking off, but due to the SNR loss, insertion loss and other factors cause the bit error rate to escalate (i.e. in the range of 10^-4 and 10^-6). Hence, it is important to have an engine, like forward error correction, that can improve the bit error rate without adding a lot of overhead.

Open-Silicon’s MCMR FEC Overview

• Open Silicon’s multi-channel and multi rate forward error correction engine is fully configurable and programmable to support different channels, like ethernet and Interlaken, and a variety of different data rates up to 400G in a variety of different permutation and combinations.
• The IP is fully compliant to the Interlaken FEC IP extension, and is also compliant with the ethernet standard specification 802.3 clauses 91, 119, 134.
• The same FEC IP can be instantiated in the customer design to work with different protocols, like ethernet and Interlaken, with run time configuration options.
• The IP supports both KP4 [ RS(544, 514)] and KR4 [RS(528, 514)] forward error correction algorithms.
• The IP is designed in such a way that it can seamlessly integrate with Open-Silicon’s PCS, FlexE and Interlaken IP, thus providing a complete networking IP subsystem.
• Supports pre-FEC distribution and FEC symbol interleaving as required by 802.3bs
• FEC degrade functionality
• Per-port FEC bypass to support 100/50/25/ 10GE ports w/o FEC
• Alarms and monitoring counters
• Support for by pass mode
  • Supports bypass at FEC boundary to support 802.3bs
  • Supports bypass at ‘Alignment Marker + FEC’ boundary to support standards

MCMR FEC Features

MCMR FEC Transmitter

• Alignment Marker (AM) insertion. Programmable common and unique AM for each lane
• Supports both KP4 RS(544,514) and KR4 RS(528,514) FEC, and parity calculation can be run time configurable or define based
• Symbol insertion is 2x interleaved in the case of 200G and 400G, and non-interleaved in the case of 100G and 50G
• Symbol distribution on a maximum of eight lanes in the case of 200G and 16 lanes for 400G
• PRBS generator for test pattern generation. Supports PRBS31, PRBS9 for NRZ, PRBS 31Q, PRBS13Q and SSPRQ for PAM4
• Statistics block per FEC instance
• Configuration and status port
• Exception detection and interrupt reporting
• Run time configurable FEC bypass operation

MCMR FEC Receiver

• Configurable alignment lock and lane de-skew from 2 to 16 lanes
• Programmable common and unique AM for each lane
• Lane reordering support
• Supports both KP4(RS544,514) and KR4(528,514) FEC and parity calculation. Can be run time configurable or define based
• Symbol insertion is 2x interleaved in the case of 200G and 400G, and non-interleaved in the case of 100G and 50G
• Alignment marker removal
• Test pattern checker
• Error detection and interrupt reporting
• Run time FEC bypass operation
• FEC statistics per instance.
  • FEC corrected code word count
  • FEC uncorrected code word count
  • FEC symbol error counter
• PCS control and status
• Lane mapping for each of physical lanes 0-7 / 0-15
• Dynamic skew measurement for each lane
• PCS Status – link up/down
• BER counter

Multi-lane AM status (locked and aligned/not locked and aligned

Ease of Integration

• FEC can be seamlessly integrated into Interlaken, Ethernet PCS and FlexE
• It can be segmented by input corresponding to one SerDes lane
• Multiple segments are bound together, depending on PCS bandwidth
• Bypass available for ‘AM module + FEC’ or FEC alone to support AUI
• FEC supports lane remapping
• It supports pipelining at interface to assist physical placement and floor planning

Applications

The multi-channel multi-rate FEC engine can be used in several applications that require the need to improve the bit error rate on the interface channels. Some of the applications captured here are packet processing or networking processing unit, traffic management for quality of service, switch fabric interface, framers and mappers to interface with optics and ethernet, ethernet PHYs, ethernet network adaptors, switching and routing applications, data center applications, TCAM memory used for lookup interfaced with NPU, serial memory interface used in data center and HPC and FPGA devices.

What follows is a backplane system application requiring multi-channel multi-rate forward error correction engines.

The block diagram below shows a typical chassis layout, with several line cards and switch fabric/cards connecting to the backplane. On top of it all sits the control and management plane. Now, let’s expand this further and look only at the line card. To the right, is the high level overview of all the components involved in the line card. Starting from the left, the media interface is used for providing the required interfaces on the line side. Next comes the PHY, used for performing encoding/decoding operations and driving and receiving the data on the line. Next is the MAC, used for several operations, like framing, addressing, error control and flow control. What follows next is the network processor, used for packet look-ups, classification and forward functions. The network processor connects to the TCAM and DRAM for packet lookup memory and packet buffer memory. Last in the line is the fabric interface chip, which connects to the switch fabric.

The upcoming generation of line cards will transfer data at a very high speed between all these components, which requires the use of the high speed SerDes. As we have learned thus far with high speed SerDes, we have the penalty of bit error rate going up. Therefore, the use of forward error correction becomes a must-have requirement for applications using high speed SerDes.

Open-Silicon’s MCMR FEC IP core is a single solution to meet the requirements of different protocols like Interlaken, Flex Ethernet and Ethernet 802.3x to significantly improve bandwidth by enabling high speed SerDes integration. The FEC can easily achieve a BER (Bit Error Rate) of 10^-15, which is required by most electrical interface standards using high speed SerDes. Built upon a flexible and robust architecture, Open-Silicon’s MCMR FEC IP core is compatible with various SerDes supporting different widths.

Key Takeaways

• A single core solution to meet the requirements of different protocols
• Supports Interlaken, Flex Ethernet and Ethernet 802.3x protocols
• Significantly improves bandwidth by enabling high speed SerDes integration
• Supports bandwidths up to 400G with the ability to connect 16 SerDes lanes
• Achieves a BER (Bit Error Rate) of <10^-15 with an input BER of >10^-6, required by most standards
• Flexible and robust architecture enables compatibility with various SerDes supporting different widths
• Designed and tested to be easily synthesizable into many SoC technologies, making it a viable solution for high performance networking and data center applications