I/O Consolidation in the Data Center

by Silvano Gai, Claudio DeSanti

Overview

Using Fibre Channel over Ethernet (FCoE) and related technologies, data centers can consolidate data traffic onto a single network switch, simplifying their environments, promoting virtualization, and substantially reducing power and cooling costs. This emerging technology is drawing immense excitement, but few enterprise IT decision-makers and implementers truly understand it. I/O Consolidation in the Data Center is the only complete, up-to-date guide to FCoE. FCoE innovators Silvano Gai and Claudio DeSanti (chair of the T11 FCoE standards working group) systematically explain the technology: its benefits, tradeoffs, and what it will take to implement it successfully in production environments. Unlike most other discussions of FCoE, this book fully reflects the final, recently-approved industry standard. The authors also present five detailed case studies illustrating typical FCoE adoption scenarios, as well as an extensive Q and A section addressing the issues enterprise IT professionals raise most often. This is a fully updated version of Silvano Gai's privately-published book on FCoE, written for leading FCoE pioneer Nuova Systems before the company was acquired by Cisco. Nearly 12,000 copies of that book have already been distributed, demonstrating the immense interest in FCoE technology, and the scarcity of reliable information that has existed about it.

Product Details

ISBN-13:	9781587141225
Publisher:	Pearson Education
Publication date:	09/08/2009
Series:	Networking Technology
Sold by:	Barnes & Noble
Format:	eBook
Pages:	168
File size:	6 MB
Age Range:	18 Years

About the Author

Silvano Gai, who grew up in a small village near Asti, Italy, has more than 27 years of experience in computer engineering and computer networks. He is the author of several books and technical publications on computer networking and multiple Internet Drafts and RFCs. He is responsible for 30 issued patents and 50 patent applications. His background includes 7 years as a full professor of computer engineering, tenure track, at Politecnico di Torino, Italy, and seven years as a researcher at the CNR (Italian National Council for Scientific Research). For the past 12 years, he has been in Silicon Valley where, in the position of Cisco Fellow, he was an architect of the Cisco Catalyst family of network switches, of the Cisco MDS family of storage networking switches, and of the Nexus family of data center switches. Silvano teaches a course on I/O Consolidation, Data Center Ethernet, and Fibre Channel over Ethernet at Stanford University (see http://scpd.stanford.edu/certifi cates/fcoe).

 

Claudio DeSanti is a Distinguished Engineer in the Advanced Architectures & Research organization at Cisco. He represents Cisco in several National and International Standards Bodies, such as INCITS Technical Committee T11, IEEE 802.1, IETF, and in industry associations. He is vice chairman of the INCITS T11 Technical Committee, chairperson of various working groups, including FC-BB-5, where FCoE has been developed, and technical editor of different standards, including IEEE 802.1Qbb, where Priority-based Flow Control is defi ned. He is author of several patents and international publications, including ten RFCs in IETF and other standards in the American National Standard Institute. He received many honors and awards, including the INCITS 2008 Technical Excellence Award, the INCITS 2007 Team Award, and the INCITS 2006 Gene Milligan Award for Effective Committee Management. Claudio’s research interests include network protocols, storage networking, routing, and security. He holds a Ph.D. in computer engineering from the Scuola Superiore Sant’Anna in Pisa, Italy.

Table of Contents

I/O Consolidation 1

    Introduction 1

    What Is I/O Consolidation 2

    Merging the Requirements 3

    Why I/O Consolidation Has Not Yet Been Successful 4

    Fundamental Technologies 5

    PCI-Express 5

    10 Gigabit Ethernet 5

    Additional Requirements 8

    Buffering Requirements 8

    Layer 2 Only 9

    Switch Architecture 9

    Low Latency 10

    Native Support for Storage Traffi c 11

    RDMA Support 11

Enabling Technologies 15

    Introduction 15

    Lossless Ethernet 15

    PAUSE 15

    Credits Versus PAUSE 17

    PAUSE Propagation 18

    Is Lossless Better? 19

    Why PAUSE Is Not Widely Deployed 20

    Priority-based Flow Control (PFC) 20

    Additional Components 22

    DCBX: Data Center Bridging eXchange 22

    Bandwidth Management 23

    Congestion Management 25

    Delayed Drop 26

    Going Beyond Spanning Tree 28

    Active-Active Connectivity 32

    Etherchannel 32

    Virtual Switching System (VSS) 32

    virtual Port Channel (vPC) 34

    Ethernet Host Virtualizer 36

    Layer 2 Multipath (L2MP) 38

    Basic Mechanisms in L2MP 40

    Cisco DBridges 47

    IETF RBridges and the TRILL Project 51

    VEB: Virtual Ethernet Bridging 52

    Server Virtualization 53

    SR-IOV 54

    The IEEE Standard Effort 54

    VEB in the Adapter 55

    VEB in the Switch 56

    VNTag 57

    Fabric Extenders 59

    VN-Link 60

    Questions and Answers 63

    Does FCoE Uses Credits? 63

    High Availability of PAUSE and Credits 63

    Queue Size 63

    Long-Haul 63

    FECN/BECN 64

    Confi guration 64

    Bandwidth Prioritization 64

    Storage Bandwidth 64

    Cisco DCB/FCoE Support 65

    10GE NICs 65

    IP Routing 65

    Lossless Ethernet Versus Infi niband 66

    Nomenclature 66

Fibre Channel over Ethernet 67

    Introduction 67

    Fibre Channel 69

    Fibre Channel Architectural Models 71

    FCoE Mapping 73

    FCoE Architectural Models 74

    FCoE Benefi ts 79

    FCoE Data Plane 80

    FCoE Topologies 82

    FCoE Addressing 85

    FCoE Forwarding 87

    FPMAs and SPMAs 90

    FIP: FCoE Initialization Protocol 92

    FIP Messages 93

    FIP VLAN Discovery 97

    FIP Discovery 98

    FIP Virtual Link Instantiation 103

    FIP Virtual Link Maintenance 108

    Converged Network Adapters 110

    FCoE Open Software 112

    Network Tools 113

    FCoE and Virtualization 114

    Fibre Channel Block I/O 115

    iSCSI Block I/O 116

    Moving a VM 117

    FCoE and Block I/O 117

    FCoE FAQ 118

    Is FCoE Routable? 118

    iSCSI Versus FCoE? 120

    Does FCoE Require Gateways? 123

Case Studies 125

    Introduction 125

    I/O Consolidation with Discrete Servers 126

    Top-of-Rack Consolidated I/O 129

    Example with Blade Servers 131

    Updating the Distribution Layer 133

    Unifi ed Computing System 136

Bibliography 139

    PCI Express 139

    IEEE 802.3 139

    IEEE 802.1 139

    Ethernet Improvements 139

    Fibre Channel 139

    FCoE 140

    TRILL 140

    Virtualization 140

Glossary 141

Figures 145

Tables 149

Index 151