OCP 2014 Eng Workshop MG Bryan Kelly

OCP 2014 Eng Workshop MG Bryan Kelly
January 28–29, 2014San Jose
Engineering Workshop
Microsoft cloud server specification
Blade overview – Compute and Storage
Martin Goldstein,
Principal Systems Architect
Microsoft Cloud Server Infrastructure Engineering
Engineering Workshop
Chassis
Engineering Workshop
Microsoft cloud server features
EIA 19” Standard Rack Compatibility
Chassis 12U



Highly efficient design with shared power, cooling, and management
Cable-free architecture enables simplified installation and repair
High density: 24 blades / chassis, 96 blades / rack
Flexible Blade Support


Compute blades – Dual socket, 4 HDD, 2 SSD
JBOD Blade – scales from 10 to 80 HDDs
Scale-Optimized Chassis Management


Secure REST API for out-of-band controls
Hard-wired interfaces to OOB blade management
Engineering Workshop
Chassis components
Chassis
Fan door
6 PSU
12 tray slots
12U tall
manager
12 trays
Engineering Workshop
Compute Blade
Engineering Workshop
Blade design: guiding principles
Simplicity
Flexibility

No hot plug or redundant
components
 Low cost, simplified management

Serviceability
Total Cost of Ownership
Blind-mate connectors simplify
server insertion and removal
 Cable-free design minimizes
cable-based NTF issues


Three IO card options (LAN, SAS, PCIe)
 LFF SATA HDD and SFF SATA SSD
Density optimized for IT-PAC (container)
deployments
 Shared chassis infrastructure amortized
across 24 servers
Engineering Workshop
Key compute blade specifications
Processor
CPU
Core QTY
TDP Wattage
Memory
Memory Busses and DIMM
Slots
DIMM Type / Speed
Max Capacity
On-Board Devices
Storage Controller
SATA (HDD)
SATA (SSD)
Server Management
Chipset
Interface
System Firmware
Version
Security
Blade I/O
PCI-Express Slots
Networking
SAS
Dual Intel® Xeon® E5-2400 v2 family
Up to 10 cores / CPU, 20 / Blade
Up to 95W
3X memory bus / CPU, 6 / Blade
6 DIMM slots
16GB, 2Rx4, 1333MHz, 1.35V
192 GB / Blade
Intel® C602 PCH
4 ports @ 3.0 Gb/s
2 ports @ 6.0 Gb/s
BMC-Lite serial thru Chassis Manager
REST API, CLI thru Chassis Manager
UEFI 2.3.1
TPM 1.2, Secure Boot
One Gen3 X16 Riser
Single or Dual 10G Mezzanine Card
Dual 4X SAS @ 6G Mezzanine Card
Engineering Workshop
Compute blade highlights
Optional
SAS Mezz
Intel® Xeon® E5-2400
v2 series CPU
DIMMs
SSD
Connection
BMC-Lite
Exhaust
Thermal
sensor
HDD
Connection
LAN Mezz
[reference designs shown]
DIMMs
Intel® Xeon® E5-2400
v2 series CPU
Intel® C600
series chipset
Optional
PCIe x16
Engineering Workshop
Compute blade BMC-Lite
 IPMI basic mode over Serial
 I2C Master (SDR)
 UART I/O
 System Event Log
 Power Control




KVM
Ethernet, Network Stack or SOL
USB
Full IPMI Command Set
Engineering Workshop
Compute blade I/O
Flexible options to meet networking and storage requirements
1200W
Power
Mgmt
NWK 2
0 1 2 3
NC
NC
NC
PCI-Express
Gen3 x8
44 channel
channel
Retimer
Retimer
300W 12V
Airmax
Network Mezzanine
NWK 1
PCIe
0 1 2 3
NC
NC
NC
Tray
Backplane
SFP+ #2
10G
10G
SFP+ #1
• Enabled for 40 GbE
• Specified for 1x or 2x 10GbE
Airmax
CPU
PCI-Express
Gen3 x8
x4
MiniSAS x4
MiniSAS x4
0 1 2 3
4 5 6 7
SAS
SAS Mezzanine
PCIe
Motherboard
Tray
Engineering Workshop
LAN mezzanine
•
•
PCI-Express Gen3 x8
Single or Dual 10Gbit Ethernet
Direct Attach Copper Cables, up to 5 meters
Heatsink
NIC Mezzanine
3.5mm
Device
SEAM
62 mm (2.44”)
•
Backside Keepout
10mm
6.5mm
SEAF
Motherboard
83 mm (3.27”)
[reference designs shown]
Engineering Workshop
SAS mezzanine
•
•
PCI-Express Gen3 x8
Eight lanes of 6Gb/s SAS
Direct Attach Copper Cables, 0.5 meters
Heatsink
SAS Mezzanine
3.5mm
Device
SEAM
83 mm (3.55”)
•
Backside Keepout
10mm
6.5mm
SEAF
Motherboard
105 mm (4.49”)
[reference designs shown]
Engineering Workshop
Safety and compliance
Tray gaskets seal
to bottom of the
tray above
Ready for data centers world-wide



Microsoft requires full compliance
Containment at blade and tray
Chassis is contained for use in EIA racks
Safety is Microsoft top priority

UL, IEC, CSA standards among others
EMI Compliance is important

CISPR, ANSI, IEC standards to start with
Blade gaskets seal
to the bottom of
the tray above
Engineering Workshop
Additional features
Status LEDs


Health LED in the front
Attention LED in the front and back
Blade Insertion and Removal



Front access, tool-less blade extraction
Rotate latch before engaging release lever
Two-phase release enables in-rack shipments
Green Health:
Blade OK
Amber Health:
Blade Fault
Red Attention:
Identify Blade
Engineering Workshop
Manufacturing blade build-out
Bare metal
W/ motherboard
W/ HDD backplane
W/ 4x3.5” HDDs
Engineering Workshop
Expansion JBOD
Engineering Workshop
JBOD design: guiding principles
Simplicity
Flexibility

Scale in 10 HDD blocks
 Direct attach cabling

Serviceability
Total Cost of Ownership
Blind-mate connectors simplify
JBOD insertion and removal
 Cable-free design minimizes
cable-based NTF issues


Support 14 to 84 HDD per server
 Eight SAS lanes in or Four in / Four out
Density optimized, up to 800 HDDs / rack
 Short cables save cost and weight
 Shared chassis infrastructure is
amortized across 24 blades
Engineering Workshop
Key JBOD specifications
Storage
HDDs
Capacity
Connectivity
Signals
Expander
LFF 7200 RPM 3.5” SATA
20 TB, 10 x 2TB HDD
30 TB, 10 x 3TB HDD
40 TB, 10 x 4TB HDD
50 TB, 10 x 5TB HDD
60 TB, 10 x 6TB HDD
8 Channel SAS @ 6G
LSI 20-port LSISAS2X20
Storage Server Configuration Capacities
Storage Controller : Single Compute with 4 HDDs plus
+ 1 JBOD
14 HDDs
+ 2 JBOD
24 HDDs
+ 4 JBOD
44 HDDs
+ 6 JBOD
64 HDDs
+ 8 JBOD
84 HDDs
Engineering Workshop
Expansion JBOD reference design
10 HDDs
Expander
8 Lanes SAS
20-lane SAS expander


10 internal lanes connect to LFF SATA HDDs
8 external lanes connect to tray backplane
Expander connects to chassis manager via RS-232 port

Managed with the same command set as the compute blade
Engineering Workshop
Expander board details
Storage expander board
Storage HDD backplane



Blind-mate to tray backplane (SAS, management)
Direct connect to two 3.5” SATA HDDs
Cable connect to two storage HDD backplanes
Cable-free attach simplifies
drive replacement and eliminates NTFs
caused by cable connection issues
[reference designs shown]
Engineering Workshop
Modular design for storage flexibility
Compute (4 HDD)
JBOD (10 HDD)
14 HDD per server
…
…
Compute (4 HDD)
JBOD (10 HDD)
JBOD (10 HDD)
JBOD (10 HDD)
JBOD (10 HDD)
JBOD (10 HDD)
JBOD (10 HDD)
JBOD (10 HDD)
84 HDD per server
JBOD (10 HDD)
Engineering Workshop
Manufacturing JBOD build-out
Bare metal
W/ expander backplane
W/ 10x3.5” HDDs
Engineering Workshop
More Information: Technical breakouts
Technical Workshop
Presenter
Hardware Overview
Mark Shaw, Director
Management Software Overview
Badriddine Khessib, Director
Chassis Manager Hardware Overview
Bryan Kelly, Senior Platform SW Engineer
Engineering Workshop
Microsoft cloud server spec: OCP contribution
Source Code
Specifications
Chassis management
source code through
Open Source
Mechanical CAD Models
Chassis, Blade, Chassis Manager, Chassis, Blade, Chassis Manager,
Mezzanines, Management APIs
Mezzanines
/// <summary>
/// Gets Fan speed in RPM
/// </summary>
/// <param name=“fanId">target fan Id</param>
/// <returns>Fan speed in RPM</returns>
internal FanSpeedResponse GetFanSpeed(byte fanId)
{
Board Files & Gerbers
Chassis Manager, Tray Backplane,
Power Distribution Backplane
Engineering Workshop
Microsoft datacenter resources
Microsoft Datacenters
Web Site & Team Blogs
▪
www.microsoft.com/datacenters
Windows Azure
▪
http://www.windowsazure.com
Office 365
▪
http://www.office365.com
Engineering Workshop
Q&A
Engineering Workshop
© 2014 Microsoft Corporation. All rights reserved. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because
Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the
date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
Engineering Workshop
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertising