Rapid IO fits interconnect requirements for embedded systems

Rapid IO fits interconnect requirements for embedded systems
By Dan Bouvier, PowerPC Architecture Manager, Motorola SPS, Austin, Texas, Sam Fuller, President, RapidIO Trade Association, Austin, Texas, EE Times
October 28, 2002 (10:18 a.m. EST)
URL: http://www.eetimes.com/story/OEG20021023S0013

High-performance embedded systems are typically made up of a variety of processing elements operating in a distributed computing fashion. The processing elements move data between each other using direct peer-to-peer communications. This is different from the typical personal computer where there is a grouping of peripherals that communicate with a common host-processor through a shared memory.

As the industry moves away from shared broadcast buses such as PCI, to point-to-point links like RapidIO, processing elements should be able to communicate directly with each other without the need for intervening bridges or host controllers. The most effective way to accomplish this is to use source directed addressing.

Source addressing means that a transaction contains the explicit address or device ID of the destination device. A transaction is routed to the destination through the use of routing tables in switches. This method allows traffic to be easily rerouted to an alternate path by simply updating an entry in a routing table. In a memory-mapped system, the device ID is just a portion of the overall system address. Explicitly defining the device ID limits the number of entries that a switch must compare when doing routing. RapidIO is the only next generation intra-system interconnect that uses source routing. PCI full address decode, on the other hand, forces a bridge device to do a match over an entire address width.

Technologies such as Infiniband have a major impact on legacy runtime software and are consequently disadvantaged in the embedded market. Technologies such as RapidIO, PCI Express and Hypertransport transparently support memory-mapped operations including support for PCI bridging. Thus, they do not have a major impact on existing software bases.

Embedded systems typically carry transient data, which moves from one processing element to another for further processing. Moving transient data using D MA is not practical since this requires full memory space visibility between the sending and receiving devices. A messaging protocol is better suited for this task. Ethernet or InfiniBand are examples of messaging protocols.

The disadvantage of Ethernet and InfiniBand is the large transaction overhead required to accommodate the large networks and data security that are required for inter-system communications. This overhead becomes less burdensome for large data payload messages moving across large networks, but can severely impact performance for short payload inter-processor communications.

An example of the sorts of small payloads seen in embedded systems are UDP packets (less than 64 bytes) used to carry voice traffic between DSPs for an IP Telephony application or the use of AAL2 based ATM cells (less than 53 bytes) for carrying packet voice in a 3G basestation. RapidIO is the only next generation intra-system interconnect with a low overhead messaging protocol.

An i nterconnect protocol for embedded systems must be designed for reliable transactions. Because these systems are typically working on real time data flows, error recovery should be managed in hardware to reduce recovery latency and simplify software design. RapidIO is the only interconnect with link-by-link error detection and recovery in hardware. The interconnect provides the means for a system to precisely detect, correct, recover from and report a dropped packet for any transaction in the system.

Unlike previous industry-standard broadcast buses, RapidIO uses point-to-point links allowing processing elements to communicate directly to one another. The technique uses source-directed addressing where a transaction contains the explicit address of the desitnation device. Data is route d to its destination using routing tables embedded in switches.
Source: RapidIO Trade Association

The embedded OEM may want to connect many processors in a system. It is possible that in a large system thousands of device IDs may be necessary. Additionally, a system may require redundant transaction paths and multiple hosts in order to managed failed links.

This means that the topology of a system may be more complex than the simple tree or daisy chain found in the typical PC system. For example, there may be more than one path between two endpoints. System discovery software must have the ability to walk through a system and enumerate devices in a complex topology. Further, for reasons of redundancy, it is required that more than one host be able to discover the system concurrently.

RapidIO allows a small or large transport address supporting 256 or 64k devices respectively. Each RapidIO device contains maintenance registers that are used by a sy stem host to discover, enumerate and configure a device in any arbitrary topology.

Because embedded systems are becoming increasingly modular, parallel interfaces must be capable of communication between mezzanine boards and serial links must be capable of communicating over backplanes and possibly between chassis over cable.

The RapidIO parallel physical layer uses LVDS, which can cross mezzanine board connectors. Serial RapidIO can sustain longer transmission distances and greater inter-signal skew for backplane and cabled applications.

An embedded protocol must be able to withstand both graceful and ungraceful hot swap. A RapidIO link carries a constant stream of state information allowing a device to immediately detect when its link partner is no longer present or operating correctly as might occur when a board is suddenly removed. For a more graceful hotswap, software can quiesce a device through RapidIO using defined maintenance transactions.