The TorusNX in Network-on-chips:

New Structure for interconnection networks

Teamour Esmaeili
Dep.of Computer Engineering
DareShahr Branch,
Islamic Azad University, Iran
Ghazal Lak
Dep.of Computer Engineering
DareShahr Branch,
Islamic Azad University, Iran
Akram Noori Rad
Dep.of Computer Engineering
DareShahr Branch,
Islamic Azad University, Iran
AbstractXthe overall performance of modern computing systems is increasingly determined by the characteristics of the
interconnection network used to provide communication links between on-chip cores and off-chip memory. Photonic technology
has been proposed as an alternative to traditional electronic interconnects because of its advantages in bandwidth density,
latency, and power efficiency. Circuit-switched photonic interconnect topologies take advantage of the optical spectrumto create
high-bandwidth transmission links through the transmission of data channels on multiple parallel Wavelengths. The silicon
microring resonator electro-optic modulator is a ubiquitous device within many proposed photonic NoCs. One such network
leverages the TorusNX topology. It is designed to mimic the connectivity of a folded torus for transmitting wavelength-parallel
messages. Each wavelength channel of the wavelength-parallel message can be independently generated by a uniquely-tuned
modulator.
Index TermsX Network on Chip (NOC), TorusNX, NS-2 .
~~~~~~~~~~ ~~~~~~~~~~
1 INTRODUCTION
ecent progress in silicon photonic technology has
enabled the prospect of high-performance photonic
networks-on-chip (NoCs), which have become very
attractive solutions to the growing bandwidth and power
consumption challenges of future high-performance chip
multiprocessors [1-4]. The design of the high-
performance photonic NoC commences at the individual
silicon photonic device and produces a full-scale on-chip
optical interconnection network.
Our design track for integrating silicon photonic de-
vices in a photonic NoC architecture is carried out in four
phases: first, a silicon photonic device is proposed based
on a target utility in the network (e.g., power-efficient
electro-optic modulation, broadband electro-optic switch-
ing, wavelength-selective filtering, etc.), and is then de-
signed based on fundamental physical principles; the de-
vice is then fabricated and verified to follow the predicted
physical characteristics (e.g., insertion loss, switching
speed, resonance response, etc.); next, the device is evalu-
ated in a high-performance system-level environment to
demonstrate its viability within the target network archi-
tecture, where valuable system-level performance metrics
are extracted (e.g., power penalty, data rate capability,
aggregate bandwidth capacity, etc.); lastly, these per-
formance metrics are inputted into PhoenixSim [5], a
physically-accurate network-level simulation environ-
ment, where we evaluate the scalability of this device in a
full-scale network architecture.
2 CURRENT NETWORKS
This section describes the current network topologies that
are included in the PhoenixSim distribution. They can be
used immediately as sample networks, or can be used as
templates for creating your own networks. The included
networks are roughly categorized into three types:
1) Electronic networks which networks implemented us-
ing standard router pipeline modeling
2) a class of photonic networks that have been extensively
studied in the LIGHTWAVE RESEARCH LABORATORY
that uses circuit-switching for arbitration
3) and miscellaneous photonic networks that have been
proposed by groups outside of the LIGHTWAVE RE-
SEARCH LABORATORY.
Why Photonics?
Photonics changes the rules for Bandwidth-per-Watt.
PHOTONICS:
ra-high bandwidth data stream
once: jusl p}/lil.nol p}/lil/nn
entire multiwavelength stream.
-chip BW = on-chip BW for nearly same power.
ELECTRONICS:
-transmit at every router.
-chip BW requires much more power than on-chip
R
JOURNAL OF COMPUTING, VOLUME 4, ISSUE 6, JUNE 2012, ISSN (Online) 2151-9617
https://sites.google.com/site/journalofcomputing
WWW.JOURNALOFCOMPUTING.ORG 139
BW.
2.1 Electronic Networks
Electronic NoCs are the near-term solution to intercon-
necting many cores in a CMP because they are already
established in the CMOS production line. Aside from
their simplicity, many of these electronic topologies are
used for applications that have traffic patterns that can be
well mapped, allowing better utilization of network re-
sources. All topologies assume a regularly spaced grid
configuration. For the electronic networks illustrated be-
low, red circles represent the communicating nodes,
while the blue squares represent network routers.
2.1.1 Electronic Mesh
The electronic mesh topology connects the nearest neigh-
bor nodes along each of the four cardinal directions (N, S,
E, and W). This formation is well suited for a 2-D topol-
ogy since it contains now wire crossings, thereby reduc-
ing the complexity of the chip.
2.1.2 Electronic Mesh - Circuit Switched
Conventionally, meshes are used as packet-switched
networks, however the design can be adapted to use a
circuit-switching protocol. A circuit-switched electronic
mesh effectively requires two separate mesh networks,
one to transmit control and arbitration packets in a packet
switched manner. This packet-switched plane is used to
configure a secondary data plane that is configured to be
purely circuit switched for transmitting large data pay-
loads across the network without buffering. This can pro-
duce a significant reduction in power dissipation since
only the control message need to be buffered, while the
payload message can propagate through the network
without any buffering. This can also possibly reduce la-
tency since once a path is setup, no clock cycles are ex-
pended for data to be buffered along the circuit path.
2.1.3 Electronic Torus
In a mesh topology, the nodes at the edges of the network
will have one or two fewer connections than a node in the
niddIe. In nany ciicunslances, each liIe of lhe nelvoik
is designed to be exactly the same (e.g. a five-port router
that connects N, S, E, W, and the node). For the nodes at
the end this can leave one or more dangling connections
that are unused and wasted. The torus topology can take
advantage of these by taking every pair of edge nodes
along a single row or column and connecting their un-
connected ports. This produces a ring along each row and
column, and increasing the diversity of the network. The
one disadvantage to this approach is that the connections
that connect the edges together can be very long.
Figure 2 shows a further optimization that can be imple-
mented by altering the positions of the routers, referred to
as a folded torus. This eliminates the long wires by con-
necting the second nearest neighbors and enables a net-
work that can be traversed in fewer hops as well.
Fig.1. Silicon Photonic Switch
Fig.2. Electronic mesh topology
Fig.3. Electronic torus topology
JOURNAL OF COMPUTING, VOLUME 4, ISSUE 6, JUNE 2012, ISSN (Online) 2151-9617
https://sites.google.com/site/journalofcomputing
WWW.JOURNALOFCOMPUTING.ORG 140
2.1.4 CMesh
Similar to the torus topology, CMesh takes advantage of
unconnected ports by connecting them to
other routers along the same edge [6]. In contrast with the
torus, no additional wire crossings are
introduced since the added connections remain around
the edge of the chip. A version of a 4*4
CMesh is illustrated in Figure 4.
2.1.5 Flattened Butterfly
The Flattened Butterfly attempts to decrease transmission
latency in the network by reducing the number of hops
required to travel from any source-destination pair. This
form of network takes advantage of high-radix routers
(routers with many ports) so that a packet can travel an-
ywhere along the length of a row or column in a single
hop [7]. In effect, this allows any packet to move from
source to destination in two hops (assuming dimension
ordered routing).
2.2. Hybrid Circuit-Switched Photonic Networks
The remainder of this section is left to describing several
hybrid photonic networks that have been devised and
published by the LIGHTWAVE RESEARCH LABORA-
TORY group.
2.2.1 Photonic Torus
The photonic torus was the first proposed optical on-chip
network to use the concept of spatial switching [8]. This
topology, shown in Figure 6 is meant to mimic the layout
of a folded-torus network. The illustration shows the to-
rus as the bold black lines, with the switching nodes indi-
cated by lIocks naiked vilh an X. The svilching nodes
were only designed as 4*4 photonic switches, therefore
cannot be easily modified to have a fifth connection to a
communication end-point.
Instead, this design uses an complimentary access net-
work (shown as red lines and red blocks) to allow mes-
sages lo enlei and exil lhe loius. C naiks lhe end-point
vheie nessages enlei and exil lhe lopoIogy, vhiIe I and
L naik lhe oplicaI svilches used lo injecl and ejecl fion
the torus, respectively.
2.2.1 Types of fault
Fig.4. Electronic cmESH topology
Fig.6. Photonic Torus topology
Fig.5. Electronic Flattened-Butterfly topology
JOURNAL OF COMPUTING, VOLUME 4, ISSUE 6, JUNE 2012, ISSN (Online) 2151-9617
https://sites.google.com/site/journalofcomputing
WWW.JOURNALOFCOMPUTING.ORG 141
2.2.2 Nonblocking Torus
The non-blocking version of the photonic torus is illus-
trated in Figure 7, first designed in [9]. This version of the
network attempts to increase performance by creating a
strictly non-blocking network, which allows a source to
be able to allocate a path on the network to any destina-
tion as long as the destination is not busy with another
transmission. This is not true of the original photonic to-
rus since paths in the network can be allocated in a way
that will block other transmissions from occurring.
2.2.3 Photonic Mesh
The photonic mesh is characteristically similar to an elec-
tronic mesh, which has been designed to use 5*5 photonic
switches. See [10] for more details. While this design
might decrease the path diversity that can be offered to a
packet, it makes up for it by lowering the insertion loss
costs considerably.
2.2.4 TorusNX
TorusNX is a newly developed network that simplifies
the photonic torus which results in more efficient layout,
better physical-layer performance, and better system-level
performance.
Pairs of high-bandwidth waveguides are used to form
bidirectional transmission lines to connect gateways (Fig.
9a) and 44 non-blocking switches (Fig. 9b) throughout
the network.
2.2.5 Square Root
Square Root takes a non-traditional approach to the pho-
tonic topology by considering a hierarchical layout
(shown in Figure 10).
Fig.7. Photonic Non-blocking Torus topology
Fig.8. Photonic 4*4 TorusNX topology
Fig.9. (a) Layout of the gateway. (b) Layout of the 44 non-blocking
switch.4.1 Fat-Pyramid
Fig.10. Photonic Square Root topology.
JOURNAL OF COMPUTING, VOLUME 4, ISSUE 6, JUNE 2012, ISSN (Online) 2151-9617
https://sites.google.com/site/journalofcomputing
WWW.JOURNALOFCOMPUTING.ORG 142
REFERENCES
[1] M. Petracca et al., IEEE Micro 29 (4), 74-85 (2009).
[2] C. Batten et al., HOTI 2008, 21-30 (2008).
[3] A. V. Krishnamoorthy et al., Proc. SPIE 7220, 72200V
(2009).
[4] R. G. Beausoleil et al., HOTI 2008, 182-189 (2008).
[5] J. Chan et al. DATE 2010, (2010).
[6] J. Balfour and W. J. Dally. Design tradeoffs for tiled
cmp on-chip networks. In ICS 06: Proceedings of the
20th annual international conference on Supercom-
puting, pages 187-198, New York, NY, USA,2006.
ACM.
[7] J. Kim, J. Balfour, andW. Dally. Flattened buttery
topology for on-chip networks. Microarchitecture,
IEEE/ACM International Symposium on, 0:172-182,
2007.
[8] A. Shacham, K. Bergman, and L. Carloni. On the de-
sign of a photonic network-on-chip. In First Interna-
tional Symposium on Networks-on-Chip, 2007.
[9] M. Petracca, B. G. Lee, K. Bergman, and L. Carloni.
Design exploration of optical interconnection net-
works for chip multiprocessors. In 16th IEEE Sympo-
sium on High Performance Interconnects, Aug 2008.
[10] G. Hendry, J. Chan, D. Brunina, L. P. Carloni, and K.
Bergman. Circuit-switched silicon nanophotonic
networks-on-chip for ultra power-efhcienl off-chip
memory access. In ACM/IEEE International Sympo-
sium Computer Architecture, 2010. in review.
JOURNAL OF COMPUTING, VOLUME 4, ISSUE 6, JUNE 2012, ISSN (Online) 2151-9617
https://sites.google.com/site/journalofcomputing
WWW.JOURNALOFCOMPUTING.ORG 143