VMIPCI-5588 High-Speed Fiber-Optic Reflective Memory with Interrupts • High-speed, easy-to-use fiber-optic network (1.2 Gbaud serially) • Data written to memory in one node is also written to memory in all nodes on the network • Up to 256 nodes • Connection with multimode fiber up to 1,000 ft; single-mode fiber up to 10 km or twinax cable up to 30 m • Data transferred at 29.5 Mbyte/s without redundant transfer • Data transferred at 14.8 Mbyte/s with redundant transfer • Any node on the network can generate an interrupt in any other node on the network or in all network nodes with a single command • Error detection • Redundant transmission mode for suppressing errors • No processor overhead • No processor involvement in the operation of the network • Up to 16 Mbyte of Reflective Memory Each node on the system has a unique identification • D32:D16:D8 memory access number between 0 and 255. The node number is established • Single PCI 5 V slot, standard long card during hardware system integration by placement of jumpers • PCI target data bursts supported with 44 Mbyte/s transfer rates on the board. This node number can be read by software by • PCI master DMA controller (optional). DMA transfer rates of 132 Mbyte/s maximum and 33 Mbyte/s sustained. accessing an on-board register. In some applications, this • Configurable endian conversions for multiple CPU architectures on the node number would be useful in establishing the function of network the node. • Communication link compatible with VMIVME-5588 and VMIPMC-5588 • Software-addressable digital output bit available at output connector for use with optical switch board or any user-defined purpose In order to achieve an aggregate throughput of • Windows NT® OS driver available which operates the VMIPCI-5588 as a 29.5 Mbyte/s, nodes capable of writing to the Reflective network interface board. Allows TCP/IP and other protocol connectivity on a network of VMIPCI-5588 or VMIVME-5588 nodes. Memory network at an aggregate rate of 29.5 Mbyte/s must be present. INTRODUCTION — VMIPCI-5588 is a high-performance, daisy-chained PCI-to-PCI network. Data LINK ARBITRATION — The VMIPCI-5588 system is is transferred by writing to on-board global RAM. The data is a fiber-optic or twinax daisy chain ring as shown in Figure 1. automatically sent to the location in memory on all Reflective Each transfer is passed from node-to-node until it has gone all Memory boards on the network. The Reflective Memory the way around the ring and reaches the originating node. network can include both PCI and VME systems. Data can be Each node retransmits all transfers that it receives except shared in VMEbus systems with the network-compatible those that it originated. Nodes are allowed to insert transfers VMIVME-5588. between transfers passing through. PRODUCT OVERVIEW — The Reflective Memory INTERRUPT TRANSFERS — In addition to concept provides a very fast and efficient way of sharing data transferring data between nodes, the VMIPCI-5588 will allow across distributed computer systems. any processor in any node to generate an interrupt on any other node. These interrupts would generally be used to VMIC’s VMIPCI-5588 Reflective Memory interface indicate to the receiving node that new data has been sent and allows data to be shared between up to 256 independent is ready for processing. These interrupts are also used to systems (nodes) at rates up to 29.5 Mbyte/s. Each Reflective indicate that processing of old data is completed and the Memory board may be configured with 256 Kbyte to receiving node is ready for new data. 16 Mbyte of on-board SRAM. The local SRAM provides fast Read access times to stored data. Writes are stored in local Three interrupts are available. The user may define the SRAM and broadcast over a high-speed fiber-optic data path function for each interrupt. Any processor can generate an to other Reflective Memory nodes. The transfer of data interrupt on any other node on the network. In addition, any between nodes is software transparent, so no I/O overhead is processor on the network can generate an interrupt on all required. Transmit and Receive FIFOs buffer data during nodes on the network. Interrupts are generated by simply peak data rates to optimize CPU and bus performance to writing to a VMIPCI-5588 register. maintain high data throughput. All data and interrupt command transfers contain the The Reflective Memory also allows interrupts to one or node number of the node that originated the transfer. This more nodes by writing to a byte register. These interrupt information is used primarily so the originating node can (three levels, each user definable) signals may be used to remove the transfer from the network after the transfer has synchronize a system process, or used to follow any data. The traversed the ring. The node identification is also used by interrupt always follows the data to ensure the reception of nodes receiving interrupt commands. When a node receives the data before the interrupt is acknowledged. an interrupt command for itself, it places the identification number of the originating node in a FIFO. Up to 512 The VMIPCI-5588 requires no initialization unless the interrupts can be stacked in the FIFO. During the interrupt user desires to use interrupts, endian byte swapping, or an service routine, the identification of the interrupting node can external mode bypass switch such as the VMIVME-5591. be read from the FIFO. VMIC • 12090 South Memorial Parkway • Huntsville, Alabama 35803-3308 1 VME Microsystems Intl. Corp. 12090 South Memorial Parkway, Huntsville, AL 35803-3308 USA VMIPCI-5588 PCI INITIATOR/DMA CAPABILITIES — The DMA capability is initiated by a PCI host. After the DMA engine is Ordering Options initialized by a host, the VMIPCI-5588 will request the PCI January 14, 1998 800-855588-000 E ABC – D E F bus as appropriate and move up to 16 Mbyte as a PCI initiator. VMIPCI-5588 – – This capability removes the CPU from the responsibility of requesting the PCI bus and moving the data itself. This feature A = Memory Options 0 = 256 Kbyte is very useful for moving large blocks of data. The PCI 1 = 512 Kbyte architecture ensures that the VMIPCI-5588 does not 2 = 1 Mbyte monopolize the PCI bus during this process. Large DMA 3 = 2 Mbyte 4 = 4 Mbyte blocks will automatically be split into smaller bursts on the 5 = 8 Mbyte PCI bus by the DMA engine. The VMIPCI-5588 can be 6 = 16 Mbyte programmed to issue a PCI interrupt at the conclusion of a B = FIFO Option 0 = 512 Transfer FIFO DMA transfer or the host can poll the status of the DMA 1 = 4 Kbyte Transfer FIFO process. Although the DMA engine can do both DMA reads C = Transmission Mode and DMA writes, they cannot occur simultaneously. The Transmit Receive Mode VMIPCI-5588 can burst data onto the PCI bus at a maximum 0 = Reserved 1 = mm Fiber mm Fiber Multimode DMA rate of 132 Mbyte/s and a sustained rate of 33 Mbyte/s. If the 2 = sm Fiber sm Fiber Single-Mode DMA target operates at a slower rate, the PCI handshaking 3 = mm Fiber sm Fiber Mixed-Mode DMA capabilities will throttle the data rate. 4 = sm Fiber mm Fiber Mixed-Mode DMA 5 = Twinax Twinax Cable DMA 6 = Twinax mm Fiber Mixed DMA ERROR MANAGEMENT — Errors are detected by the 7 = Twinax sm Fiber Mixed DMA VMIPCI-5588 with the use of the error detection facilities of 8 = mm Fiber Twinax Mixed DMA 9 = sm Fiber Twinax Mixed DMA the Fiber Channel encoder/decoder and additional interlaced parity encoding and checking. When a node detects an error, Multimode Fiber-Optic Cable Assemblies the erroneous transfer is removed from the system and a PCI ABC – D E F bus interrupt is generated, if enabled. The error rate of the VMICBL-000-F3 –0 – VMIPCI-5588 is a function of the rate of errors produced in A = Fiber-Optic Connector Type the optical portion of the system. This optical error rate 0 = Ceramic Ferrule St Connector depends on the length and type of fiber-optic cable. Assuming 1 = Stainless Steel Ferrule St Connector -12 BC = Cable Lengths an optical error rate of 10 , the error rate of the 00 = Not Used -10 VMIPCI-5588 is 1.6 x 10 transfers/transfer. 01 = 5 ft (1.5 m) 02 = 25 ft (7.6 m) 03 = 50 ft (15.2 m) The VMIPCI-5588 can be operated in a redundant 04 = 100 ft (30.4 m) transfer mode in which each transfer is transmitted twice. In 05 = 150 ft (45.7 m) this mode of operation, the first of the two transfers is used 06 = 200 ft (60.9 m) 07 = 350 ft (106.7 m) unless an error is detected in which case the second transfer is 08 = 500 ft (152.4 m) used. In the event that an error is detected in both transfers, the 09 = 1,000 ft (304.8 m) node removes the transfer from the system. The probability of Twinax Cable Assemblies -20 both transfers containing an error is 2.56 x 10 , or about one ABC – D E F error every 330,300 years at maximum data rate. VMICBL-001-33 –0 – ENDIAN CONVERSIONS — Data lane steering can be A = 0 (Option reserved for future use) BC = Cable Lengths configured in a Control Register to allow CPUs of different 00 = Not used architectures to communicate. Byte swap, Word swap, and 01 = 0.5 m Byte-Word swap options are available. 02 = 1 m 03 = 3 m 04 = 5 m PROTECTION AGAINST LOST DATA — Data 05 = 7 m received by the node from the cable is error checked and 06 = 10 m 07 = 15 m placed in a receive FIFO. Arbitration with accesses from the 08 = 20 m PCI bus then takes place, and the data is written to the node’s 09 = 25 m SRAM and to the node’s transmit FIFO. Data written to the 10 = 30 m board from the PCI bus is placed directly into SRAM and into For Ordering Information, Call: the transmit FIFO. Data in the transmit FIFO is transmitted by 1-800-322-3616 or 1-256-880-0444 • FAX (256) 882-0859 E-mail: info@vmic.com Web Address: www.vmic.com the node over the fiber-optic cable to the next node. Data could Copyright © April 1995 by VMIC be lost if either FIFO were allowed to become full. Specifications subject to change without notice. The product is designed to prevent either FIFO from becoming full and overflowing. It is important to note the only way that data can start to accumulate in FIFOs is for data to enter the node at a rate greater than 29.5 Mbyte/s or 14.8 Mbyte/s in redundant mode. Since data can enter from 2 For Ordering Information, Call: 1-800-322-3616 or 1-256-880-0444 • FAX (256) 882-0859 VMIPCI-5588 the cable and from the PCI bus, it is possible to exceed these Reflective Memory and the VMIPMC-5588 PMC rates. If the transmit FIFO becomes half-full, a bit in the Status bus-based boards. The 5588 Series of products has Register is set. This is an indication to the node’s software that auxiliary products which include the VMIVME-5591 subsequent WRITEs to the Reflective Memory should be Active Fiber-Optic Bypass Switch which is utilized to suspended until the FIFO is less than half-full. If the half-full bypass nodes in a system with effectively 0.0 dB indication is ignored and the transmit FIFO becomes full, then insertion loss, and the VMIVME-5592 which is utilized writes to the Reflective Memory will be acknowledged with a to allow the 1,000 ft fiber-optic cable length limit to be STOP *. No data will be lost. extended to 10 km. If the receive FIFO is allowed to become over half-full, Memory: The 256 Kbyte, 512 Kbyte, and 1 Mbyte there is a danger the receiver FIFO may overflow resulting in memory options of the VMIPCI-5588 all use 1 Mbyte of data loss. In order to prevent this situation, all PCI writes will address space to locate data in the local node. Since the be acknowledged by a STOP* until the receive FIFO is less PCI bus is an automapped bus, all of these memory than half-full. options must occupy 1 Mbyte of memory space on the PCI bus. This mapping structure allows these options to NETWORK MONITOR — There is a bit in a Status be compatible with VMIVME-5588 Reflective Register that can be used to verify that data is traversing the Memories of size 1 Mbyte and smaller. ring (that is, the ring is not broken). This can also be used to measure network latency. The 2 and 4 Mbyte memory options of the VMIPCI-5588 both use 4 Mbyte of address space to locate data in the SPECIFICATIONS local node. Both of these memory options must occupy 4 Mbyte of memory space on the PCI bus. This mapping Memory Size: 256 Kbyte, 512 Kbyte, 1 Mbyte, structure allows these options to be compatible with 2 Mbyte, 4 Mbyte, 8 Mbyte, or 16 Mbyte VMIVME-5588 Reflective Memories of size 2 and 4 Mbyte. PCI Transfer Rate: 44 Mbyte/s as a Target. As DMA initiator: 132 Mbyte/s maximum, 33 Mbyte/s sustained. The 8 and 16 Mbyte memory options of the VMIPCI-5588 both use 16 Mbyte of address space to TRANSFER SPECIFICATION locate data in the local node. Both of these memory options must occupy 16 Mbyte of memory space on the Transfer Rate: 29.5 Mbyte/s (longword accesses) PCI bus. This mapping structure allows these options to without redundant transfer be compatible with VMIVME-5588 Reflective 14.7 Mbyte/s (longword accesses) with redundant Memories of size 8 and 16 Mbyte. transfer SOFTWARE DRIVER CABLES A Windows NT OS driver for the VMIPCI-5588 is VMIC offers the following cable assemblies that are available. The driver operates the VMIPCI-5588 as a compatible with the VMIPCI-5588. network interface board. A portion of the Reflective Memory is reserved by the driver for network use, and the Multimode Fiber Cable: remainder of memory is available as simple user-shared ST connectors memory. When used as a network interface board, the 1,000 feet maximum VMIPCI-5588 has a dual role; the board provides both 62.5 micron core network connectivity as well as fast shared memory for 10 dB maximum attenuation between nodes distributed applications. Wavelength = 860 nm The driver is NDIS compliant and supports the protocols Twinax Cable: configured into the Windows NT OS system including 9-pin D-type connector TCP/IP. For more information, see the Equalized, 30 m maximum VMIVME/SW-5588 specification. The VMIPCI-5588 is compatible with single-mode fiber PHYSICAL/ENVIRONMENTAL with SC connectors and a maximum of 10 km. VMIC does not supply single-mode fiber cable. Temperature Range: 0 to 65 °C, operating with forced air cooling. -40 to 85 °C, storage. COMPATIBILITY Relative Humidity: 20 to 80 percent, noncondensing Reflective Memory Products: The VMIPCI-5588 is compatible with VMIC’s line of Reflective Memory Power Requirements: 5.0 A maximum at +5 VDC products which include VMIVME-5588 VMEbus-based VMIC • 12090 South Memorial Parkway • Huntsville, Alabama 35803-3308 3 VME Microsystems Intl. Corp. 12090 South Memorial Parkway, Huntsville, AL 35803-3308 USA VMIPCI-5588 DATA TRANSFERS TRADEMARKS Data written into the Reflective Memory is broadcast to Windows NT is a registered trademark of Microsoft all nodes on the network without further involvement of Corporation. Other registered trademarks are the property of the sending or receiving nodes. Data is transferred from their respective owners. memory locations on the sending nodes to corresponding memory locations on the receiving nodes. A functional block diagram of the VMIPCI-5588 is shown in Figure 2. CABLE CABLE C 5 P 5 VMEbus CHASSIS U 8 CHANNEL CHANNEL NODE 0 WITH CPU AND 8 RCV TX VMIPMC-5588 RECEIVER TRANSMITTER INTERRUPT LOGIC ENDEC C 5 P 5 U 8 VMEbus CHASSIS NODE 1 8 WITH CPU AND 4 Kbyte/ 4 Kbyte/ VMIVME-5588 CONTROL 512 DEEP 512 DEEP LOGIC IN FIFO OUT FIFO CTRL CTRL VME COMPUTER NODES CAN BE VMEbus-BASED COMPUTERS OR COMPUTERS 5 WITH VMEbus I/O CHANNELS CS 5 FIFO/RCV CTRL SUCH AS SUN, HARRIS NIGHT NODE 2 RAM CTRL 17 8 WR 8 HAWK, CONCURRENT, SILICON GRAPHICS, DATA GENERAL, RAM MOTOROLA DELTA SERIES, ADDRESS ENCORE 91 SERIES, ANY CONTROL VMEbus CHASSIS, ETC. PCI PCI bus I/F 16 Mbyte RAM MAXIMUM 22 LINES DATA ENDIAN 32 LINES SWAP NODE 255 BUFFER PCI bus PCI bus CHASSIS WORKSTATION WITH VMIPCI-5588 UP TO 1,000 ft BETWEEN NODES Figure 2. VMIPCI-5588 Functional Figure 1. Network Example Using Reflective Block Diagram Memory System 4 For Ordering Information, Call: 1-800-322-3616 or 1-256-880-0444 • FAX (256) 882-0859