High Ambient Temperature with Intel Xeon

Data Center Efficiency
& Power Solutions
David Jenkins
Jay Kyathsandra
DCG Technology Marketing
Sept 2011
Legal Disclaimer
•
Information in this document is provided in connection with Intel® products. No license, express or implied, by estoppels or otherwise, to any
intellectual property rights is granted by this document. Except as provided in Intel's Terms and Conditions of Sale for such products, Intel
assumes no liability whatsoever, and Intel disclaims any express or implied warranty, relating to sale and/or use of Intel® products including
liability or warranties relating to fitness for a particular purpose, merchantability, or infringement of any patent, copyright or other intellectual
property right. Intel products are not intended for use in medical, life saving, or life sustaining applications.
Intel may make changes to specifications and product descriptions at any time, without notice.
•
Designers must not rely on the absence or characteristics of any features or instructions marked "reserved" or "undefined." Intel reserves
these for future definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them.
•
Intel processors and chipsets may contain design defects or errors known as errata which may cause the product to deviate from published
specifications. Current characterized errata are available on request.
•
This document contains information on products in the design phase of development. The information here is subject to change without notice.
Do not finalize a design with this information.
•
All products, computer systems, dates, and figures specified are preliminary based on current expectations, and are subject to change without
notice.
•
Intel does not control or audit the design or implementation of third party benchmarks or Web sites referenced in this document. Intel
encourages all of its customers to visit the referenced Web sites or others where similar performance benchmarks are reported and confirm
whether the referenced benchmarks are accurate and reflect performance of systems available for purchase.
•
Intel Intelligent Power Node Manager technology requires a computer system with an Intel® Xeon® processor, supported Intel® Enterprise
chipset, BIOS, and other requirements documented in the applicable Platform Design Guideline documentation and applications enabled for
virtualization technology. Functionality, performance or other power capping technology benefits will vary depending on hardware and software
configurations.
•
Intel®, Intel® Xeon® processors, the Intel® logo, Intel® Intelligent Power Node Manager, Intel® Data Center Manager (Intel® DCM)
trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.
•
Intel processor numbers are not a measure of performance. Processor numbers differentiate features within each processor family, not across
different processor families
Intel Confidential
Copyright © 2011, Intel Corporation
*Other brands and names may be claimed as the property of others.
2
INTEL CONFIDENTIAL
Agenda
• Data Center Efficiency
Challenges
• Efficiency Priorities
• Intel Approach To Efficiency
• Solutions Overview
− Server Refresh w/ Intel® Xeon®
Processors
− Intel® 10GbEthernet
− Intel® Intelligent Power Node Manager
− Intel® Data Center Manager
− Solutions in development
− Power Thermal Aware scheduling
− High Ambient Temperature Operation
− Intel® Server Battery Backup Unit
− Data Center Design
• Summary
3
INTEL CONFIDENTIAL
Data Center Efficiency Challenges
 WW Datacenter construction
spending is increasing –
60% growth (~30$B) over
10 years.. 2011 –MS (C
Belady)
 Total power consumed by
data centers ..2-3% of all
electricity generated by
2014..EPA
 $27 B/yr spent on server
energy costs..IDC 2009
Intel® technologies and data center optimization solutions will
help achieve higher efficiency (lower PUE) and lower total cost
of computing
4
INTEL CONFIDENTIAL
Data Center Efficiency Priorities
Achieve efficiency and reliability by maximizing available
capacity and modular build out for growth
2
3
Manage
Maximize
Measurement and
Accountability
1
Modernize
Efficient IT
Infrastructure
Operational Efficiency
•
•
•
•
Power Utilization -(PUE)
Operational costs
Capital Costs
Server deployment time
A holistic approach – systems, rack, design and monitoring
5
INTEL CONFIDENTIAL
Intel Approach To Data Center Efficiency…
 Deliver leading products, technologies and datacenter efficiency
solutions that lower Capital Expenditures and Operational Costs
Platform
Innovation
System
Design
Data center
Optimization
Industry/
End user
Adoption
Efficiency Across – Platform, Infrastructure, & Operations
6
INTEL CONFIDENTIAL
Improve Your Data Center Efficiency
Modernize, Manage & Maximize with Intel®
Modernize
Efficient Infrastructure
Manage
Measurement &
Accountability
Maximize
Operational Efficiency
•
•
•
•
•
Server Refresh with Intel® Xeon® Processors
• Simplify
Update
and Refresh
Servers
® 10GbE
Network
with Intel
• High
Virtualized
& Consolidate
Workload
Ambient Temperature
optimized
designs
• Battery
SelectBackup
HTA capable
systems
Unit solutions
Rack technologies for shared power & cooling
® Intelligent
•• Intel
Measure
3P’s (Power,
and
PowerProductivity,
Node Manager
PUE)Data Center and container design best
• Adopt
• practices
Understand Datacenter Costs
• Assign Organizational Accountability
• • Adopt
efficient
powerPower
deliver
schemes
Manage
data center
and
Thermal profile
®
•
Dynamic
management
of
power,
using Intel Intelligent Powerthermal
Node and
workload
andCenter
resources
Managerplacement
Intel® Data
Manager
• • Raise
DataITCenter
Temperature
via best systems
Integrate
and Facilities
management
practices
with Power Thermal Assessment Scheduling
Technology solutions for ongoing efficiency improvements
7
INTEL CONFIDENTIAL
Server Refresh: Single Core  Xeon® 5600
2005
2010
95% Annual Energy
Efficiency
Refresh
15:1
Cost Reduction
(estimated)
As Low as
1 rack of Intel® Xeon®
5600 Based Servers
5 Month
Payback
– OR –
15 racks of
Intel® Xeon®
Single Core
Servers
Up to 15x
Performance
Performance
Refresh
1:1
8% Annual Energy
15 racks of Intel®
Xeon® 5600 Based
Servers
Costs Estimated
Reduction (estimated)
Source: Intel measurements as of Feb 2010. Performance comparison using server side java bops (business operations per second). Results have been estimated based on
internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual
performance. For detailed calculations, configurations and assumptions refer to the legal information slide in backup.
8
INTEL CONFIDENTIAL
The Cost of Waiting in 2010
Monthly costs for NOT refreshing 500 Single-core
Servers with 30 Intel® Xeon® 5600 Servers
Software support:
Utility costs:
Warranty costs:
up to
$50,920
$18,380
$31,250
per month
per month
per month
$100,000
per month1
(estimated)
Refresh: If not NOW, when?
1 Monthly Savings in utility and SW support costs determined by comparing the incremental costs associated with not refreshing 50 older single-core servers purchased back in 2005 vs. refreshing on an approximate 15:1 ratio with a Xeon
5680-based servers in Year 1. Warranty cost assumes $750/year per server if purchased after the initial OEM 3-year warranty period has expired. Actual total cost is $100,750. Source: Consolidation ratio calculated using the Xeon Server
Refresh Savings Estimator (www.intel.com/go/xeonestimator) and uses publicly available RHEL OS support costs, default utility settings, and SPECint*_rate_base2006 performance and power data for the Xeon® X5680 as found in the backup.
See the speaker notes or backup foils (if needed) for system configurations.
9
INTEL CONFIDENTIAL
A Unified Network with
Intel® 10G Ethernet Solutions
GbE Server Connections
estimated
45%
Reduction in
Power per Rack
Intel® 10 Gigabit X520
10GbE Dual Port
(x/k Mezzanine Card)
estimated
80%
Reduction in
Cables and
Switch ports
10GbE Server Connections
estimated
estimated
Reduction in
Infrastructure
Costs
Improved
Bandwidth
per Server
2x
15%
Intel® Ethernet Server
Adapter X520-DA2
(with optional 10GbE/Gbe SR optics)
Intel® 10 Gigabit X520T2 Server Adapter
(10GBASE-T)
Realize the full potential for Xeon® 5600 platforms
Source: Intel 10GbE ROI calculator. See backup for details
Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of
those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products.
10
INTEL CONFIDENTIAL
Intel Intelligent Power Node Manager
and Data Center Manager
Managing Data Center Power & Energy
Intel® Intelligent Power Node
Manager
Monitor and Control Server Power
1
• Report system level energy use
• Limit individual SERVER power
consumption
Intel® Data Center Manager
Manage and Coordinate at the
Data Center Level
Limit aggregated
3 ROW power draw
2 • Limit total RACK power draw
• More productivity per rack
Aggregated, Policy-Based Power Management for the Data
Center
11
INTEL CONFIDENTIAL
Node Manager & Data Center Manager
Results
Power and
Thermal
Monitoring
Replace IP power strips and
serial concentrators, saving
~$400 per rack
Maximize
Increased
Rack Density
Up to 40% more servers and
performance per rack
Maximize
Workload
Power
Optimization
Up to 30% power
optimization without
performance impact
Maximize
Business
Continuity
Continued compute
availability through power or
thermal event
Manage
Delivering Results End Users Value
INTEL CONFIDENTIAL
Solution Choices For Directed Power
Management
Node Manager Servers
Data Center
Solutions
DCM Enabled Consoles
PowerEdge C
Growing Choices For Solutions Using Intel® Directed Power
Management
13
INTEL CONFIDENTIAL
.. Solutions in Development
Maximize
• Power Thermal Aware Scheduling –
PTAS - Concept
Modernize
• Server Battery Backup Unit - Pilot
Modernize
14
• High Ambient Temperature Operation Pilot
INTEL CONFIDENTIAL
Power Thermal Aware Scheduling (PTAS)
•
Achieve efficiency and lower PUE thru a holistic and integrated approach to Data
center infrastructure management
Integrate IT and
Facilities Management
•Lower operational costs
~20% +
•Recovery up to 50% of
un used cooling
capacity+
•Reduce DC monitoring
instrumentation costs
Server level – Power, Thermal and
Workload Data
IT Infrastructure
Power Thermal Aware Scheduling
Dynamic Provisioning - Workload placement,
Cooling systems automation
Maximize
Operational Efficiency
Facility Systems
Making Integration of IT and Data center Infrastructure a
reality
15
+Intel
internal estimate
INTEL CONFIDENTIAL
Intel® Battery Backup Solution
Server BBU – growing trend, leading lower capital spending and increase power efficiency
Intel solution - Licensing Intel® Server BBU solution architecture to OEM’s, battery and
power supply manufacturers
Reduce Data Center
Capital Costs
• Reduce UPS related
capital expenditure
costs ~ 5X1
• Recover UPS related
power efficiency loss –
~30-40%2
• Easy deployment and
time to market solution
..Integrates with most systems
…enhanced with Intel® Intelligent Power Node Manager
PDB
•
•
Built for
Power Resiliency
1Intel
2
internal estimate
APC whitepaper # 108
Simplifies infrastructure and increases flexibility for modular build out
16
Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are
measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other
information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products.
INTEL CONFIDENTIAL
High Temperature Operation (HTA)
•
•
Operate at optimal datacenter temperature set point. Typically at ~21C, trending to ~30 C
Benefit - Decrease chiller energy costs by operating at higher temperature
Google: Raise Your Data
Center Temperature..80 F
> 40°C
Containers and modular
designs
2
Facebook Saves ~229K/yr in
energy bill by retooling its
Cooling..to 81 F 1
35 – 40°C
Hot/Cold aisle isolation,
economizers
25–27°C  30–35°C
Hot/Cold aisle isolation,
economizers- retrofit
18–21°C  25–27°C
Hot/Cold aisle airflow
management
• Sun – 4% savings in chiller energy
costs for 1 0C in upward change 2
• Microsoft saved 250K/yr energy costs
by raising 2-4 0c 2
• Intel IT raised to 92 0F, for 10 mo
(Proof of concept) 2
1) http://www.datacenterknowledge.com/archives/2010/10/14/facebook-saves-big-by-retooling-its-cooling/
2) http://www.datacenterknowledge.com/archives/2008/10/14/google-raise-your-data-center-temperature
3) http://www.datacenterknowledge.com/archives/2010/04/26/yahoo-computing-coop-the-shape-of-things-to-come/
4) Intel internal estimate and based on market data analysis
INTEL CONFIDENTIAL
Intel Solution For HTA
Intel Solution - Integrated solution tools and methodology to achieve optimal set
point – Component selection, system design & datacenter infrastructure
System Layout
Reduce Cooling Costs
•~ 4-5% cooling related
reduction /1 0c increase
in temp1
•Optimize for IT &
Facilities infrastructure
The Right
Temperature
Set Point
Varies
Platform design guide
Component
Selection
Optimized for
performance/
power and
Infrastructure
• Choice of Platform
• + Intel® Intelligent
Power Node Manager
Data Center
Infrastructure
Data center design
guide
Leading OEM’s delivering HTA capable systems on
Intel® Xeon® processor based platform
1 Source: Sun http://www.datacenterknowledge.com/archives/2008/10/14/google-raise-your-data-center-temperatur
18
INTEL CONFIDENTIAL
Summary
• Achieving significant improvements in
data center efficiency requires IT to
modernize, manage and maximize IT
and cooling infrastructure
• Intel delivers leading edge products &
technologies that address customer
power and cooling needs
• Develop innovative solutions that
improve Data center efficiency lower
Capital costs and operational costs
• Engage actively with Industry to
accelerate adoption and support end
users to test, validate and
incorporate solutions in their data
centers
INTEL CONFIDENTIAL
Back Up
20
INTEL CONFIDENTIAL
Intel® Intelligent Power Node Manager
•Second generation power reporting and
capping technology from Intel®
•Provides directed power management
features that compliment a BMC or
manageability controller functionality
Policy
Intel
Chipset
BMC
Power
NM
PSU
Console Manager
•Monitors and reports system
level and processor and memory
power consumption
•Enforces power caps by adjusting
processor or memory power or
dynamically allocating processor
cores
Delivers Platform Level Instrumentation Required To Improve
Data Center Power Efficiency
INTEL CONFIDENTIAL
Intel® Intelligent Power Node Manager
Use Case Evolution
Power and
Thermal
Monitoring
Static Power
Capping
Dynamic
Power
Capping
Integration
Workload
Power
Characterization
Increased Rack
Density
Business
Continuity
Chiller Event
Power & Thermal
Aware Scheduling
Facilities Planning
Workload Power
Optimization
Business
Continuity
Power Event
Power & Thermal
Aware Balancing
Chargeback Models
Boot Power Spike
Management
IP Power Strip &
Concentrator
Replacement
Power Supply /
System RAS
Basics
Close Coupled
Cooling
Advanced
Start With Basics – Move To More Advanced Cases with
Time & Experience
INTEL CONFIDENTIAL
More Compute/Fixed Power Infrastructure
Maximize ROI for Available Rack Power and Capital
Expenditure
Set rack level power
cap policy 14A
(3,080W)
Increased
Rack/Compute
Density
9 units
server/rack
10 units
server/rack
8 units
server/rack
Additional 40% rack
density captured through
power capping, minimal
performance impact+
Increased rack density
10-30%, saving $500/
server over 4 years with
no major performance
impact+
+Publicly
23
available white papers. See backup for details
INTEL CONFIDENTIAL
Baidu current
rack density
unites server per
rack, 385W power
consumption
budget for per
server.
Stable rack
density increase
policy 8 to 9,
342W power
consumption
budget for per
server.
More efficiency rack
density increase
policy 9 to 10, 308W
power consumption
budget for per
server.
End-User Proof Points and Results
WORKLOADS
HPC
HPC/FSI
Database
Search
IO Optimized Search
CPU Intensive
Workload Power
Optimization
0
10
20
30
40
WATTS SAVED/NODE
FSI
30W per server
savings with no
performance
impact1
~19% per server
savings without
performance
tradeoff1
~35% power
savings
expected1
100W savings for
4 servers for
Microsoft* SQL
Server* workload1
Power Optimization results are dependent on the system configuration and the workload – results will vary
accordingly 1Public whitepapers available for download
24
INTEL CONFIDENTIAL
22W savings
per server on
Monte Carlo
ZCS workload
Intel® Data Center Manager
An SDK Datacenter Power and Thermal Management; Easy to
Integrate in the Management Console
MONITORING
• Real-time monitoring of actual power and
inlet temp data* aggregated to rack, row,
room and user-defined logical groups
TRENDING
• Log power and thermal
data, query trend data
using filters
• Saves 1 year of history
data for capacity
planning
CONTROL
• Supports multiple active
power policy types,
schedule by time of day
• Co-existing policies at
multiple hierarchy levels
e.g., row, rack, group.
Policy accepts SLA
priority as directive
Enables Management Consoles To Scale Node Manager
Functionality To The Group Level
INTEL CONFIDENTIAL
5 Month Single Core Refresh ROI Claim
• 5 month ROI claim estimated based on comparison between 2S Single Core Intel®
Xeon® 3.80 with 2M L2 Cache and 2S Intel® Xeon® X5680 based servers. Calculation
includes analysis based on performance, power, cooling, electricity rates, operating
system annual license costs and estimated server costs. This assumes 8kW racks, $0.10
per kWh, cooling costs are 2x the server power consumption costs, operating system
license cost of $900/year per server, per server cost of $7200 based on estimated list
prices and estimated server utilization rates. All dollar figures are approximate.
Performance and power comparisons are based on measured server side java benchmark
results (Intel Corporation Feb 2010). Platform power was measured during the steady
state window of the benchmark run and at idle. Performance gain compared to baseline
was 15x.
− Baseline platform: Intel server platform with two 64-bit Intel Xeon Processor 3.80Ghz with
2M L2 Cache, 800 FSB, 8x1GB DDR2-400 memory, 1 hard drive, 1 power supply, Microsoft*
Windows* Server 2003 Ent. SP1, Oracle* JRockit* build P27.4.0-windows-x86_64 run with 2
JVM instances
− New platform: Intel server platform with two Intel® Xeon® Processor X5680 (12M Cache,
3.33 GHz, 6.40 GT/s Intel® QPI), 24 GB memory (6x4GB DDR3-1333), 1 SATA 10krpm
150GB hard drive, 1 800w power supply, Microsoft Windows Server 2008 64 bit SP2, Oracle*
JRockit* build P28.0.0-29 run with 4 JVM instances
• Performance tests and ratings are measured using specific computer systems and/or
components and reflect the approximate performance of Intel products as measured by
those tests. Any difference in system hardware or software design or configuration may
affect actual performance. Buyers should consult other sources of information to
evaluate the performance of systems or components they are considering purchasing.
For more information on performance tests and on the performance of Intel products,
visit Intel Performance Benchmark Limitations.
26
Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such
as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of
those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your
INTEL
CONFIDENTIAL
contemplated
purchases, including the performance of that product when combined with other products.
Move to 10GbE and Save
10GbE ROI Calculation
GbE Network (Using 20 Servers in a Rack)
Cisco Catalyst 4948
Intel® Gigabit ET Quad Port Server
Adapter
1GbE RJ45 Patch Cable CAT6
No of units
Total Cost [$ US]
21
3.696
220.395
200
1.680
89.800
1000
-
5.000
5.656
315.195
Total
10GbE network (Using 20 Servers in a Rack) No of units
Cisco Nexus 5020
Power [W]
Power [W]
Total Cost [$ US]
5
2.400
181.500
Intel® Ethernet Server Adapter X520-DA2
100
790
79.900
10GbE Direct Attach SFP+ Twinax Copper
Cable
200
-
9.000
3.190
270.400
Total
Savings
Infrastructure
Cost
15%
Cables
Reduction
80%
Power
Savings
45%
Assumptions:
This ROI calculator is a cost comparison for a highly virtualized solution, using multiple 1GbE
connections versus a dual port 10GbE implementation. In the 1GbE solution we use in each server two
Intel® Gigabit ET Quad Port Server Adapters, in addition to two LOM connections with a total system
bandwidth of 10 Gb. This is then compared with a 10GbE solution, using one Intel® 10 Gigabit AF DA
Dual Port Server Adapter and a total system bandwidth of 20Gb. These adapters are connected to a topof-rack 10GbE switch using passive direct attach twinax coax cables
http://www.event-management-online.de/LAD/calculator.aspx
•
Source: Intel 10GbE ROI Calculator. This ROI calculator is a cost comparison for a highly virtualized solution, using multiple 1GbE connections versus a dual port 10GbE
implementation
INTEL CONFIDENTIAL
Node Manager Claims Back Up
Extreme Efficiency: Power Management
•Rack density statement based on Baidu proof of concept results documented in Intel legally approved whitepaper
posted at http://communities.intel.com/docs/DOC-4212.
•Power Optimization claims based on BMW proof of concept results documented in Intel legally approved whitepaper
posted at http://communities.intel.com/docs/DOC-4040
Increasing Rack Density Proof Points
•Baidu statement based on proof of concept results documented in Intel legally approved whitepaper posted at
http://communities.intel.com/docs/DOC-4212.
•Oracle statement based on proof of concept results documented in Intel legally approved whitepaper posted at
http://communities.intel.com/docs/DOC-3977
Increasing Rack Density Proof Points
•Baidu statement based on proof of concept results documented in Intel legally approved whitepaper posted at
http://communities.intel.com/docs/DOC-4212.
•BMW statement based on proof of concept results documented in Intel legally approved whitepaper posted at
http://communities.intel.com/docs/DOC-4040
•Oracle statement based on proof of concept results documented in Intel legally approved whitepaper posted at
http://communities.intel.com/docs/DOC-3977
•Intel IT and FSI results based on Intel internal testing of Intel Xeon Processor 5500 series whiteboxes in an NDA
environment.
Increasing Rack Density Model Baidu Proof Point
Baidu statement based on proof of concept results documented in Intel legally approved whitepaper posted at
http://communities.intel.com/docs/DOC-4212.
Power Optimization Model Oracle Proof Point
Oracle statement based on proof of concept results documented in Intel legally approved whitepaper posted at
http://communities.intel.com/docs/DOC-3977
INTEL CONFIDENTIAL