A Redbox on an "All Programmable SOC"

Rainer Fehr, Stefan Leuenberger, NetModule AG Niederwangen, Switzerland

Abstract:

Two redundancy protocols have recently been standardized by the International Electrotechnical Commission as IEC 62439-3. Both protocols allow systems to overcome any single network failure without affecting the data transmission. A node with only one Ethernet port can be attached through a RedBox, a device that provides an interface to the redundant network.

The "All Programmable SOC" has been introduced by Xilinx Inc. under the product name "ZYNQ". The Zynq-7000 products incorporate a dual-core ARM® Cortex™-A9 core processor with programmable logic on a 28-nm-basis.

A PRP/HSR Redbox has been implemented on ZYNQ, using NetModule's modular PRP/HSR IP-core deployed on the proprietary "Zynq4Ethernet" platform which provides 5 Ethernet ports.

Most of the Link Redundancy Entity has been implemented in the Programmable Logic section but management and supervision functions which are not time critical are executed in software.

This solution has sufficient performance to operate not only at 100 Mb but also at Gigabit speed. It consumes about 25% of the programmable logic resources of an XC7Z020 chip.

1. INTRODUCTION OF NETMODULE

NetModule provides you with access to state of the art technologies for integration in your Embedded Systems. The offering comprises a wide range of products and services for the embedded systems domain.

Applications of the Ethernet and IP protocols have always been a focus of NetModule's activities, first for Voice-over-IP applications and later for industrial applications like Ethernet/IP (Ertec), Profinet and Ethernet Protocols for redundant communication.

Embedded System Design has always meant Hardware/Software Co-Design. Many systems incorporate a "Processor System" (e.g. PowerPC or ARM) and "Programmable Logic" (FPGA).

Therefore NetModule has not only a large team for embedded software development which is focused on Wind River VxWorks and Embedded Linux but also an experienced team for FPGA design and implementation and expert hardware developers. This is the ideal precondition for using the upcoming "All-programmable SOCs" which are single-chip implementations of a dual ARM core and a powerful FPGA fabric.

2. REDUNDANT ETHERNET ACCORDING TO IEC62439-3 (PRP/HSR)

Two redundancy protocols have recently been standardized by the International Electrotechnical Commission as IEC 62439-3. Both protocols allow systems to overcome any single network failure without affecting the data transmission [1].

IEC 62439-3 Clause 4 describes the Parallel Redundancy Protocol (PRP). Each network node has two Ethernet ports attached to two completely separated local area networks. A "doubly attached" source node always sends two copies of the same frame, one over each port. The destination node accepts the first frame of a pair and discards the second. If one LAN fails, the second LAN can still provide full connectivity and no frame is lost.

Figure 1: PRP Stack

A node with only one Ethernet port can be attached directly to any of the two LAN's. In this case there is no redundancy of course. Otherwise it can be attached through a RedBox (redundancy box), a device that provides an interface to both LAN's and behaves like a doubly attached node.

A PRP node can be implemented on any processor system where two Ethernet ports are available. The protocol may be programmed in software. Implementation in an FPGA fabric is preferred for high-speed applications and for the integration of functionality for the IEC 61588 (PTP) precision clock synchronization protocol.

IEC 62439-3 Clause 5 describes the High-availability Seamless Redundancy protocol (HSR). HSR typically uses a ring topology. Compared to PRP there is less cabling and switches required for setting up the network. In a HSR ring, each node in the network must quickly analyze the header of an incoming frame and forward it to the next node in the ring unless the frame has to be terminated in this node. In order to keep ring propagation delay low, the frames are forwarded in "cut-through" mode, i.e. forwarding begins immediately after the decision has been taken (based on the MAC fields and the HSR tag) without intermediate storage of the frame.

Short cut-through delays can only be achieved when the link redundancy layer is implemented in application specific logic (in this case FPGA).

Figure 2: HSR Ring

Applications for PRP and HSR protocols areas where the recovery time of commonly used protocols such as Rapid Spanning Tree or the proprietary Hiper-RING is not acceptable, such as protection for electrical substation automation and controllers for machines with synchronized drives.

3. ALL PROGRAMMABLE SOC

The "All Programmable SOC" has recently been introduced by Xilinx Inc. under the product name "ZYNQ". The Xilinx Zynq-7000 product family offers developers new levels of performance, durability, flexibility, and scalability for embedded systems. The Zynq-7000 products incorporate a dual-core ARM® Cortex™-A9 core processor with programmable logic on a 28-nm-basis. The resulting "All Programmable SOC" provides customers with multiple benefits: savings in time and cost, faster time to market, more time to focus on own core competencies, and therefore better products in the end.

Figure 3: Zynq™ All Programmable SoC Architecture (Source: Xilinx Inc.)

A comprehensive ecosystem for the Zynq platform provides development tools, peripherals in the form of IP cores and development platforms.

4. THE ZYNQ4ETHERNET PLATFORM

For Ethernet centric applications, a Zynq™ platform is required which has several Ethernet ports. The Zynq4Ethernet platform has been developed by NetModule for this purpose and is most suitable for rapid prototyping and small volume production. It provides 5 Ethernet ports which can be either operated in fast Ethernet or in Gigabit Ethernet mode. Four of them are directly connected to the programmable logic section.

Figure 4: NetModule's Zynq4Ethernet Platform

The ZYNQ™ platform is very powerful for applications where Ethernet frames must be handled faster than possible in a conventional programmable system. Hence it is perfect for the implementation of a RedBox according to IEC 62439-3 where the time critical circuits of the link redundancy layer can be implemented in the programmable logic part.

5. IMPLEMENTATION OF THE REDBOX

A PRP/HSR Redbox has been implemented on the Zynq4Ethernet platform. The characteristics of the architecture are:

• The non-redundant port is connected directly to the processor system (Cortex A-9). The processor handles all non-redundant traffic and maintains the node proxy table, if needed.
• The processor system has an internal link to the programmable logic part (FPGA fabric) where the link redundancy entity (LRE) is implemented. The redundant ports are driven directly from the programmable logic.
• Supervision frames are sent by the processor system.
• Duplicate discard process and the node table are managed in the programmable logic part.
• A diagnostics Ethernet port has been added for test purposes.

Functions which are not time critical are implemented in software since implementing and maintaining software is less expensive and more flexible than FPGA development.

The functions deployed on the processor system are running on an Embedded Linux operating system (currently based on Kernel 3.5) and are programmed in C/C++. The wide choice of packages available for Linux allows extending the RedBox with additional functions and protocols for different use cases.

The functionality executed in software can be grouped into the following parts:

• System initialization and configuration (e.g. setup the Ethernet controllers in the program-mable logic part)
• Handling of supervision frames
• Management and monitoring of the RedBox (e.g. SNMP, CLI, GUI, etc.)

For the functions deployed on the programmable logic part, a modular IP-Core has been developed. In a first step, its architecture was modeled in UML (on Enterprise Architect, Sparx Systems Ltd.).

Some of its basic classes are shown in figure 5. From the basic classes, the main components can be built. One example shown in figure 6 is the REDM_port component which is instantiated for each port of the RedBox system.

Figure 5: Class Model of NetModule's PRP/HSR IP-core

Figure 6: Composite Structure Diagram of REDM_port

The IP-core has been coded in VHDL. UML data types have been mapped to VHDL type definitions. UML attributes have been mapped to VHDL generics.

This portable IP-core can be used for many applications, e.g. for setting up a doubly attached node or for building a RedBox or Quadbox for the PRP and/or the HSR protocol.

Figure 7 shows the application of a RedBox which includes a 4th test-port. The test-port allows to observe network traffic and to perform detailed network monitoring.

The verification test-benches have been setup in VHDL and are simulated by Modelsim. Transac-tion-based simulation principles are applied, using a NetModule proprietary test-bench frame-work.

Figure 7: Block diagram of RedBox

The IP-core uses internally a simple synchronous processor bus. An AXI bus bridge is used for interfacing to the Zynq processor system. Other processor interfaces may easily be adapted in big- or little-endian mode.

Synthesis and place-and-route has been carried out in Xilinx ISE Version 14.3. The target is XC7Z020.

Chip utilization of PL section:

Item	Available	Used
Slice LUTs	53,200	18%
Block RAMs (18kb)	280	28%

Timing characteristics:

Item	Value	Unit
Maximum system clock frequency	> 100	MHz

Only a fraction of the chip resources are used for this application and the performance is sufficient for a future extension for Gigabit Ethernet traffic.

The single-chip integration of the processor system with the programmable logic provides several benefits:

• Large bandwidth interconnects are available between the programmable logic part and the processor system. This may be used for automatic synchronization of the node table (which is implemented in the programmable logic part) with the management software database maintained in the main memory of the processor system.
• The system is scalable with respect to the programmable logic resources, providing a migration path to more complex implementations.

6. CONCLUSION

The upcoming technology of single-chip ARM+FPGA devices has been used for the implementation of a "RedBox" for IEC62439-3 redundant Ethernet networks. The system has been successfully integrated on the Zynq4Ethernet platform, proving the huge potential of this new technology for Ethernet-centric applications.

REFERENCES

[1] Prof. Dr. Hubert Kirrmann (ABB), Prof. Thomas Müller (ZHAW), Oliver Kleineberg (Hirschmann), Clemens Hoga (Siemens), CIGRE 2010, Paris | Use of Highly Available Networks for IEC 61850: Parallel Redundancy Protocol - PRP