35 HP ProLiant DL385 servers onto 5 Dell PowerEdge M610 blade

TEST REPORT
OCTOBER 2009
Real-world server consolidation: 35 HP ProLiant
DL385 servers onto 5 Dell PowerEdge M610 blade
servers running Hyper-V
Executive summary
Dell Inc. (Dell) commissioned Principled Technologies (PT)
to consolidate 35 4-year-old HP ProLiant DL385 server and
storage solutions onto 5 Dell™ PowerEdge™ M610 blade
server and storage solutions using Microsoft Windows
Server 2008 R2 with Hyper-V. Unlike studies that estimate
the potential consolidation factor based on a small sample,
we consolidated the full set of server and storage solutions,
allowing us to confirm the consolidation factor.
Our testing involved the following two solutions:
• 35 AMD Opteron 254-based HP ProLiant DL385 G1
servers with 4 GB of memory each and 18 HP
StorageWorks MSA30 storage (HP ProLiant DL385
solution)
• 5 Intel® Xeon® Processor X5570-based Dell
PowerEdge M610 blade servers with 72 GB of
memory, each using Hyper-V, and 5 arrays of Dell™
EqualLogic™ PS5000XV storage (Dell PowerEdge
M610 solution)
KEY FINDINGS
z
Five Dell PowerEdge M610 solutions
with Hyper-V replaced 35 legacy HP
ProLiant DL385 G1 solutions, yielding a
payback in as few as 10 months.1 (See
Figures 1 and 2.)
z
The 5 Dell PowerEdge M610 solutions
with Hyper-V used around 1/4th of the
power, 1/6th of the software costs, and
1/5th the data center rack space of the
35 legacy HP ProLiant DL385 G1
solutions.1 (See Figures 1, 3, and 6.)
z
The 5 Dell PowerEdge M610 solutions
with Hyper-V achieved over 70 percent
more throughput, delivering an average
OPM for each VM of 15,734 compared
to 9,005 average for each legacy HP
ProLiant DL385 G1 solution.1 (See
Figure 4.)
Figure 1: Five Intel Xeon Processor X5570-based Dell PowerEdge M610 solutions with Hyper-V allow you to consolidate 35 AMD
Opteron 254-based HP ProLiant DL385 solutions, with an initial investment payback period of just under 10 months and delivering
over 70 percent more throughput. We base this estimation on our specific database workload.
Our test case modeled a typical enterprise datacenter with 35 legacy HP ProLiant DL385 servers and 18 HP
StorageWorks MSA30 storage arrays—each approximately four years old—running high-demand database
workloads. The legacy servers each used 4 GB of memory. The enterprise in this test case seeks to consolidate
1
Source: Principled Technologies, Inc., “Real-world server consolidation: 35 HP ProLiant DL385 servers onto 5 Dell PowerEdge M610 blade
servers running Hyper-V” an October 2009 report commissioned by Dell.
these legacy workloads onto five Dell PowerEdge M610 blade servers and five Dell EqualLogic storage solutions
using Hyper-V, configured with sufficient processors, memory, and storage to handle these workloads. We
assume the enterprise is running SQL Server 2005 Enterprise Edition on Windows Server 2003 R2 Enterprise
Edition x64 on each of the HP ProLiant DL385 solution servers. We kept those same versions in the Hyper-V
guests on the Dell PowerEdge M610 solution. Our test case is a specific migration scenario: For our calculations,
we assume the use of Microsoft System Center Virtual Machine Manager (SCVMM) to perform our physical to
virtual (P2V) migration in order to minimize migration time, and also to keep the legacy software, including the
operating system, as is. This models an organization that wishes to reap the benefits of virtualization without
incurring the maintenance and redesign costs that sometimes accompany an operating system or application
upgrade; the P2V process allows for this. The organization can then upgrade individual virtual machines as time
and resources allow.
We used benchmark results from the Dell DVD Store Version 2.0 (DS2) performance benchmark to determine the
number of older servers with accompanying storage that a Dell PowerEdge M610 solution could replace for our
test scenario. To define the replacement factor, we measured the number of orders per minute (OPM) that the HP
ProLiant DL385 solution could perform when running a demanding DS2 workload. We then ran DS2 in virtual
machines (VMs) on the Dell PowerEdge M610 solution and measured how many VMs this newer solution could
run, while obtaining OPM in each VM greater than that of each HP ProLiant DL385 solution, with both solutions
demonstrating a heavy load, at least over 70 percent CPU utilization. The total number of VMs the Dell
PowerEdge M610 solution could handle determined the replacement factor. Based on our specific workload, the
replacement factor was seven.
Figure 1 shows the conclusions of our consolidation testing. We replaced 35 HP ProLiant DL385 servers and 18
HP StorageWorks MSA30 storage arrays with 5 Dell PowerEdge M610 blade servers and 5 Dell EqualLogic
PS5000XV storage arrays. The Dell PowerEdge M610 solution delivered over 70 percent more throughput than
did the HP ProLiant DL385 solution. Addtionally, we calculated an investment return of just under 10 months.
Figure 2 graphs the payback period and the cost savings of the five Dell PowerEdge M610 solutions. Savings
continue after the initial investment payback period. By the end of year one, we project a savings of $49,061. The
line representing the Dell PowerEdge M610 solution accumulates the initial investment cost and the monthly costs
of the solution. The initial investment cost includes the list price of the servers and three-quarters of the cost of the
shared storage2, as well as the costs of migrating from the 35 HP ProLiant DL385 solutions to the 5 newer Dell
PowerEdge M610 solutions. The line for the 35 HP ProLiant DL385 solutions shows the accumulated monthly
costs of 35 HP ProLiant DL385 solutions. The lines cross at the end of the payback period. Appendix C describes
the payback period calculation.
2
We include five arrays of Dell EqualLogic PS5000XV storage in the next generation solution. Our test databases do not use all of the
storage. We therefore do not assign all storage costs for the five arrays to this solution. We estimate at least 25 percent of this storage is
available for other uses, and assign 75 percent of the acquisition and on-going costs of the storage arrays to this solution. For the legacy
solution, we use 35 of the available 36 server connections from the 18 storage arrays so we include costs related to 17 ½ arrays.
Principled Technologies, Inc.: Real-world server consolidation: 35 HP ProLiant DL385 servers
onto 5 Dell PowerEdge M610 blade servers running Hyper-V
2
$1,200,000
Dell PowerEdge M610 solution pays back the initial investment in around
10 months
$1,000,000
$800,000
$600,000
Breakeven at
$283,255
$229,559 in 9.9 months
$400,000 initial investment
$327,681
savings at 2
years
$606,301
savings at 3
years
$49,061
savings at 1 year
$200,000
$0
1
3
5
7
9
11 13 15 17 19 21 23 25 27 29 31 33 35
HP ProLiant DL385 solutions
Dell PowerEdge M610 solution
Payback period
Initial investment
2nd year savings
3rd year savings
1st year savings
3
Figure 2: The payback period and cumulative estimated costs for the 35 HP ProLiant DL385 solutions and the 5 Dell
PowerEdge M610 solutions with Hyper-V. Lower costs and higher savings are better.
Costs for the Dell PowerEdge M610 solutions include the initial investment of $229,559 plus annual costs of
$65,172, for a total year-one cost of $294,731. These solutions could save approximately $49,061 in year one
over the $343,792 one-year costs for the 35 HP ProLiant DL385 servers and 18 HP StorageWorks MSA30
storage arrays that these 5 solutions can replace. Figure 3 shows the specific areas of savings. It shows that, for
this workload and cost scenario, the investment costs of buying the server and storage hardware and of migrating
from the older to the newer solution are offset in just under 10 months due to savings for the Dell PowerEdge
M610 solutions in hardware support costs, software licenses, software support agreements, facility costs
(including switch costs), energy costs, and management costs.
For this workload and test scenario, the 5 Dell PowerEdge M610 solutions not only replaced 35 HP Proliant
DL385 solutions, but also provided over 70 percent better throughput for users. Thus, the Dell PowerEdge M610
solution delivers not only data center savings, but also improved performance for end users.
3
We divide annual costs by 12 and do not adjust for inflation.
Principled Technologies, Inc.: Real-world server consolidation: 35 HP ProLiant DL385 servers
onto 5 Dell PowerEdge M610 blade servers running Hyper-V
3
One year costs of 35 HP
Proliant DL385 solutions and 5
Dell PowerEdge M610 solutions
with Hyper-V
Cost
$400,000
$350,000
$300,000
$250,000
$200,000
$150,000
$100,000
$50,000
$0
$343,792
$294,731
HP ProLiant
DL385 solutions
Dell PowerEdge
M610 solution
with Hyper-V
Server and storage
hardware
$0
$228,892
Migration costs
$0
$667
Sof tware support
agreements
$157,255
$27,445
Hardware support
agreements
$21,315
$8,496
Facility costs
$7,232
$1,396
Energy costs
$26,740
$5,960
Management costs
$131,250
$21,875
Total
$343,792
$294,731
HP ProLiant DL385 solutionsDell PowerEdge M610
solution with Hyper-V
Figure 3: Estimated year-one costs for the 35 HP ProLiant DL385 solutions and the 5 Dell
PowerEdge M610 solutions with Hyper-V. Lower costs are better.
Four benefits of the Dell PowerEdge M610 solution contribute significantly to the cost savings for this solution:
•
•
•
Replaces 35 HP ProLiant DL385 servers and 18 HP StorageWorks MSA30 storage arrays. The
increased I/O capacity, database performance, and memory efficient design of the Dell PowerEdge M610
solutions enables consolidation of multiple older systems for the specific workload we tested.
Uses around one-fourth of the power of the 35 HP ProLiant DL385 solutions they replace. The
consolidated Dell PowerEdge M610 solutions provide considerable energy savings because they require
on average a little less than 27 percent of the power of the older HP ProLiant DL385 solutions that they
replace. (See Figure 7.) This calculation includes the cost of the power usage for the solutions measured
during our testing.
Requires around one-sixth the software license costs. Consolidation provides software savings if the
organization maintains software assurance agreements or can reuse elsewhere the licenses that
consolidation frees. We do not include license costs for either group of solutions, as the enterprise would
already have paid for the licenses used on the older solutions and would transfer them to the new
Principled Technologies, Inc.: Real-world server consolidation: 35 HP ProLiant DL385 servers
onto 5 Dell PowerEdge M610 blade servers running Hyper-V
4
•
solutions. For the calculations in this paper, we assume that the target organization maintains software
assurance agreements for OS and database software and that it can choose to renew, transfer, or cancel
them at the time of consolidation. We move from having 35 servers licensed per-server for Windows
Server 2003 Enterprise Edition to 5 servers licensed per-processor for Windows Server 2008 R2
Datacenter Edition. For SQL Server 2005, we move from having 35 servers licensed per-processor for
SQL Server 2005 Enterprise Edition, to 5 servers licensed per-processor for SQL Server 2005 Enterprise
Edition.4
Requires one-fifth of the data center rack space. The consolidated Dell PowerEdge M610 solutions
require a blade chassis that consumes 10u of rack space for the servers and 3u for each of the five
storage arrays, for a total of 25u for our five server and storage solutions. The older HP ProLiant DL385
solutions require 2u for each of the 35 servers and 3u for each of the 18 drive arrays plus 1u for a KVM,
for a total of 125u; which requires almost three full racks. By contrast, the five Dell M610 solutions
consume only 25u or just over one ½ rack.
Workload
We conducted our testing using DVD Store Version 2, an open-source application with a back-end database
component, a front-end Web application layer, and a driver layer that actually executes the workload. DS2 models
an online DVD store. Simulated customers log in; browse movies by actor, title, or category; and purchase
movies. The workload also creates new customers. Browsing movies involves select operations, some of which
use full-text search and some of which do not. The purchase, login, and new customer stored procedures involve
update and insert statements, as well as select statements. The workload’s main reporting metric is orders per
minute.
Because our goal was to isolate and test database server and storage, we did not use the Web application layer.
Instead, we ran the driver application on client machines directly via its command-line interface. To simulate a
heavily loaded environment, the load-generating systems ran with no think time, blasting requests as quickly as
the server could handle them. We used the default DS2 parameters and setup configuration, with the exceptions
we note in the DVD Store setup section in the Test methodology section of this report.
For client machines, we used desktop systems running Windows Server 2003 R2 with SP2. Each of these client
machines ran a single instance of DS2, which spawned 12 threads against the HP ProLiant servers and 32
threads against the PowerEdge M610 server virtual machines. This simulated a heavily loaded environment
targeting at least 70 percent CPU utilization on both server solutions. Our goal was to represent the consolidation
of older servers facing a demanding workload, so we adjusted the DVD Store thread count to 12 on the older
server in order to get the server to the target processor utilization throughout the test.
On the HP ProLiant servers, we used four internal hard drives that we configured in two RAID1 volumes on all
servers, installing the OS on one volume and using the other volume for SQL Server logs, which we stored on a
15GB partition. We configured the external storage as one large RAID 5 LUN with a small 50GB partition for SQL
Server data.
On each Dell PowerEdge M610 server, we used two internal hard drives in a RAID 1 configuration, installing the
host operating system on that internal volume. We configured each tray of external Dell EqualLogic storage as
having 1 large 400GB LUN for VM OS virtual hard drive (VHD) files, 21 smaller LUNs – 2 25GB LUNs per VM for
SQL Server data, and 1 15GB LUN per VM for SQL Server logs for each of our seven VMs. We configured the
two SQL Server data volumes and one SQL Server log volume in a pass-through configuration to each VM, while
4
Our calculations include the costs for Windows Server 2003 R2 Enterprise Edition software assurance for the older solution and for the
upgraded Windows Server 2008 Datacenter Edition for the Dell PowerEdge M610 solution, as the organization upgrades some of the licenses
because it uses the Windows Server 2008 Datacenter Edition as the host OS. With Datacenter Edition per-processor licensing, the
organization may run any number of virtual operating system environments on the server. We assume the five upgraded licenses transfer
between the solutions, and we base the upgrade of only five licenses on the replacement factor of seven, as we discuss in the Executive
Summary section. The organization saves on SQL Server Enterprise licenses because each of those licenses allows any number of SQL
Server software instances in one physical server when licensed per-processor. We do not include the costs of CAL (Client Access Licenses) in
this model because the number of these would not change between solutions; we do assume the enterprise maintains Software Assurance
Agreements for the CALs and is therefore able to upgrade them at no cost.
Principled Technologies, Inc.: Real-world server consolidation: 35 HP ProLiant DL385 servers
onto 5 Dell PowerEdge M610 blade servers running Hyper-V
5
the OS VHD was on the virtual IDE controller for each VM. We performed this storage configuration on each of
our five trays of Dell EqualLogic PS5000XV storage.
The DS2 driver application creates an orders-per-minute performance counter on each client machine. While the
DVD Store client application visually outputs OPM at 10-second intervals, we chose to collect this OPM metric via
a performance monitor counter on each client machine at 1-second intervals.
For more details about the DS2 tool, see http://www.delltechcenter.com/page/DVD+Store.
We ran this workload on the Dell PowerEdge M610 blade servers and the HP ProLiant DL385 servers for 30
minutes.
Our test case modeled a typical enterprise datacenter with 35 legacy HP ProLiant DL385 solutions running highdemand database workloads. The enterprise in this test case seeks to consolidate the workloads of 35 of these
legacy systems onto 5 Dell PowerEdge M610 solutions that run Hyper-V, and wants to yield a payback around 10
months. The enterprise seeks to balance performance and cost.
We ran the DS2 benchmark to verify that, using our test workload, the newer solutions would outperform the
legacy solutions they would replace. On all machines, physical or virtual, we used a 10GB database. Each VM on
the newer system delivered better performance than did a single legacy system with the same workload. We then
estimated the monthly costs for maintaining the 35 legacy and 5 new systems and estimated the payback period
for the investment in the newer solutions.
To model this test case, we did not compare identical solutions but instead configured the newer solutions with
sufficient processors, memory, and storage to not only match or improve on the performance that the legacy
solutions deliver for the test workload, but to also yield a payback in just under 10 months.
We selected 4GB RAM for the legacy solutions as it is a realistic amount of memory for a four-year-old solution.
We configured the newer solutions as we expect an IT manager would when purchasing a Intel Xeon Processor
5500 series-based database solution intended to replace 35 HP ProLiant DL385 systems running the highdemand databases modeled by our workload. We selected and priced five Intel Xeon Processor X5570-based
PowerEdge M610 servers with two processors, 72GB RAM, and modern storage. As we expect an IT manager
would, we balanced performance and cost to define the newer configuration.
We used nearly the maximum available memory per VM that the new server could support with 72 GB of memory,
allocating 9 GB of memory per VM for a total allocation of 63 GB for the seven VMs. We expected that the IT
administrator in our test case would take advantage of the memory capacity of modern servers in order to
improve performance and consolidation and would select 9 GB of memory per VM, enough memory that most of
the 10GB database would be likely fit into memory.
We tested the 35 legacy solutions with 18 HP StorageWorks MSA30 storage arrays, which had 14 146GB 10,000
RPM U320 SCSI disks each. We configured the five newer solutions with modern storage, selecting five Dell
EqualLogic PS5000XV storage arrays with 16 146GB 15,000 RPM SAS disks each. The list price for that array is
$43,001. We assumed that the solution we modeled would require 75 percent of that storage and included that
percentage of the costs of the storage array, including $32,251 of the investment costs, in the payback analysis.
We tested the legacy solutions with 4-year old software versions and the newer solution with current software
versions at the host level, assuming that the IT department in the test case would take advantage of the
performance benefits of newer software. Inside the guest, however, to show ease of migration to a virtualized
environment with no application upgrades, we kept the older software versions, expecting that this migration
would be less volatile for IT engineers. We tested the Dell PowerEdge M610 solution running Microsoft Windows
Server 2008 R2 Datacenter Edition with Hyper-V and the HP ProLiant DL385 solution running Windows Server
2003 x64 R2 Enterprise Edition. The guests on the Hyper-V host ran Windows Server 2003 x64 R2 Enterprise
Edition to match the HP ProLiant systems. The HP ProLiant DL385 solution used Microsoft SQL Server 2005 x64.
We also installed Microsoft SQL Server 2005 x64 in each VM on each Dell PowerEdge M610 server.
Principled Technologies, Inc.: Real-world server consolidation: 35 HP ProLiant DL385 servers
onto 5 Dell PowerEdge M610 blade servers running Hyper-V
6
Our results show the advantage in performance, cost, and workload consolidation that modern hardware,
operating systems and virtualization technologies can offer over legacy hardware and software solutions for this
database workload.
Test results
We tested the 35 legacy solutions with a single database instance running a 10 GB DS2 database, which alone
nearly fully utilized the system’s CPU; additional instances yielded unacceptable results. To achieve maximum
DS2 results on the five Dell PowerEdge M610 solutions, we installed one instance of SQL Server 2005 x64 on
each of seven VMs on the five servers and ran one 10GB DS2 database on each virtual machine, resulting in 35
simultaneous database tests. We show the number of OPM each solution achieved during our test. We calculate
OPM as the number of overall orders divided by the elapsed seconds multiplied by 60. We report the last OPM
score the benchmark reported while all VMs and servers were simultaneously running. Each of the 35 VMs on the
5 Dell PowerEdge M610 solutions delivered performance better than the performance of the 35 HP ProLiant
DL385 solutions.
Figure 4 shows the specific performance results, in both OPM and CPU percentage, for the 5 Dell PowerEdge
M610 blade servers with the Dell EqualLogic PS5000XV storage solution and for the 35 HP ProLiant DL385
servers with the HP StorageWorks MSA30 storage solution while running the DVD Store tests. Results are from
the test run that produced the median of three OPM results. Higher OPM numbers are better.
Performance
System
5 Dell PowerEdge M610 solutions (35 VMs)
35 HP ProLiant DL385 solutions
Average throughput improvement for the
Dell PowerEdge M610 solutions
Median run
Average OPM
15,734
9,005
CPU percentage
83.3
72.2
75%
Figure 4: Average per database OPM and CPU performance results per server or virtual server for the 5 Dell PowerEdge M610
solutions and the 35 HP ProLiant DL385 solutions while running the DVD Store workload. Higher OPM numbers are better.
Figure 5 shows the individual VM OPM and server OPM scores from the median run for the 5 Dell PowerEdge
M610 solutions and the 35 HP ProLiant DL385 solutions. Higher OPM numbers are better.
Principled Technologies, Inc.: Real-world server consolidation: 35 HP ProLiant DL385 servers
onto 5 Dell PowerEdge M610 blade servers running Hyper-V
7
System
VM/Server 1 OPM
VM/Server 2 OPM
VM/Server 3 OPM
VM/Server 4 OPM
VM/Server 5 OPM
VM/Server 6 OPM
VM/Server 7 OPM
VM/Server 8 OPM
VM/Server 9 OPM
VM/Server 10 OPM
VM/Server 11 OPM
VM/Server 12 OPM
VM/Server 13 OPM
VM/Server 14 OPM
VM/Server 15 OPM
VM/Server 16 OPM
VM/Server 17 OPM
VM/Server 18 OPM
VM/Server 19 OPM
VM/Server 20 OPM
VM/Server 21 OPM
VM/Server 22 OPM
VM/Server 23 OPM
VM/Server 24 OPM
VM/Server 25 OPM
VM/Server 26 OPM
VM/Server 27 OPM
VM/Server 28 OPM
VM/Server 29 OPM
VM/Server 30 OPM
VM/Server 31 OPM
VM/Server 32 OPM
VM/Server 33 OPM
VM/Server 34 OPM
VM/Server 35 OPM
Total OPM
5 Dell PowerEdge M610 solutions
16,966
13,864
15,248
17,881
15,429
17,377
13,824
17,086
14,005
15,218
17,518
15,477
17,086
13,994
13,799
17,009
14,842
17,320
15,096
17,143
13,763
16,732
14,014
15,380
17,006
15,295
17,077
13,899
16,967
14,230
15,599
17,539
15,621
17,228
14,153
550,685
35 HP ProLiant DL385 solutions
8,998
8,672
9,465
9,321
9,875
8,893
8,500
8,166
8,683
9,007
8,662
8,976
8,703
8,954
9,208
8,905
10,388
8,526
8,845
8,922
8,872
8,845
8,998
9,379
9,722
8,784
9,552
9,093
9,616
8,515
9,055
8,891
8,559
8,863
8,754
315,167
Figure 5: Individual VM scores and the total VM score for the 5 Dell PowerEdge M610 solutions and the 35 HP ProLiant DL385
solutions. Higher OPM numbers are better.
Figure 6 shows the specific power results during the median run of the 5 Dell PowerEdge M610 blade servers
with the 5 Dell EqualLogic PS5000XV storage arrays and the 35 HP ProLiant DL385 servers with the 18 HP
StorageWorks MSA30 storage arrays. Lower numbers are better.
To calculate the power, we average the power we recorded during our peak stable workload period, minutes 24
through 29 of the test, for both server and storage on the HP ProLiant solution, and for both blade chassis and
storage on the Dell PowerEdge M610 solution. For the total solution power, we averaged the server and storage
power and then took the sum as total. For the idle power, we allowed the system to sit for 10 minutes after
booting to the desktop and then started 2 minutes of idle power capture. As we note in Appendix C, we calculate
the average of the active and idle power from the benchmark runs, then we use that for our payback analysis.
Principled Technologies, Inc.: Real-world server consolidation: 35 HP ProLiant DL385 servers
onto 5 Dell PowerEdge M610 blade servers running Hyper-V
8
Power in Watts (median results)
Workload
Idle
5 Dell PowerEdge M610
solutions
35 HP ProLiant DL385
solutions
Usage of 5 Dell
PowerEdge M610
solutions relative to 35 HP
ProLiant DL385 solutions
Average
4,265
3,472
3,869
14,796
14,328
14,562
29%
24%
27%
Figure 6: Median power results, in watts, for the 5 Dell PowerEdge M610 solutions and the 35 HP ProLiant DL385 solutions while
running the DVD Store workload and while idle. These results are from the runs that produced the OPM scores in Figure 5. Lower
numbers are better.
Test methodology
Adjusting BIOS settings
We used the default BIOS settings on the HP ProLiant DL385 server and enabled Virtualization Technology on
the Dell PowerEdge M610 blade server. We used the latest released BIOS updates on both solutions.
Setting up and configuring the Dell PowerEdge M610 blade server and Dell EqualLogic
PS5000XV storage
Dell PowerEdge M610 and Dell EqualLogic PS5000XV storage configuration
We installed two internal drives in each Dell PowerEdge M610 blade server. We attached each server to a Dell
EqualLogic PS5000XV containing 16 drives. We cabled each Dell EqualLogic PS5000XV to a Dell™
PowerConnect™ 6248 switch via its three available ports, and we cabled the server to the switch using three of
the four server NICs. We split the Dell PowerConnect 6248 switch into five virtual LANS (VLANs) to separate out
the five server/storage solutions’ iSCSI traffic. Each Dell PowerEdge M610 had two on-board NICs, and we
installed a mezzanine card with an additional two NICs, for a total of four NICs. We installed the latest Dell
EqualLogic Host Integration Toolkit (HIT) on each server for storage management and for multipath I/O support at
the host level. We enabled jumbo frames on each server’s three iSCSI-dedicated NICs, the Dell PowerConnect
6248 switch, and each NIC on the Dell EqualLogic storage. We left the default maximum transmission unit (MTU)
for the client/server NIC. Figure 7 shows the storage layout we used on the Dell PowerEdge M610 blade server
and the Dell EqualLogic PS5000XV storage.
Principled Technologies, Inc.: Real-world server consolidation: 35 HP ProLiant DL385 servers
onto 5 Dell PowerEdge M610 blade servers running Hyper-V
9
Figure 7: The sto
F
orage layout we
e used on the Dell
D PowerEdge M610 server an
nd Dell EqualLog
gic PS5000XV
s
storage.
We configured
W
d the internal drives on eacch server in one
o RAID 1 vo
olume. Each Dell EqualLog
gic PS5000X
XV
c
contained
16 146GB 15,00
00RPM drivess. We configured each Dell EqualLogic PS5000XV in
n RAID 10 mo
ode, and
c
created
one la
arger 400GB LUN for VM operating
o
sysstem virtual ha
ard drive (VHD) files, which
h we presente
ed to the
h
host
OS. We also
a
created 14 25GB LUN
Ns to be used
d for SQL Serrver data passs-through diskks (2 for each
h of our 7
V
VMs),
and 7 15GB
1
LUNs to
o be used for SQL Server log pass-thro
ough disks (1 for each of ou
ur 7 VMs). We
e
r
repeated
this procedure on
n each of our five trays of Dell
D EqualLog
gic storage.
On each Dell PowerEdge M610,
O
M
we insttalled Window
ws Server 200
08 R2 Datace
enter Edition with
w Hyper-V on the
m
mirrored
intern
nal volume, and
a placed the
e virtual mach
hine operating
g system VHD
Ds and SQL Server
S
databa
ase data
a logs on itss associated Dell EqualLogic PS5000X
and
XV volume.
Setting up th
S
he internal storage for ho
ost operating
g system insttallation
1. Enter the RAID con
ntroller BIOS by pressing Ctrl+R
C
at the prompt.
2. Highliight Controller 0, and presss F2.
3. Selecct Create New
w VD.
4. Selecct the two driv
ves, select RA
AID level 1, tab to the OK button,
b
and prress Enter. Acccept the warrning
regard
ding initializattion.
5. Selecct the new virttual drive, pre
ess F2, and se
elect Initializa
ation, Start Init.
6. Wait for
f the initializ
zation operation to complete.
7. Presss Escape, and
d choose to Save
S
and Exit to return to th
he boot seque
ence.
8. Repeat steps 1 thrrough 7 for the
e four remain
ning blade serrvers.
Setting up th
S
he external sttorage
1. Using
g the comman
nd line, reset the
t Dell Equa
alLogic PS500
00XV, and se
etup a storage
e group, with the
t
follow
wing IP addres
ss scheme:
a. Group IP address:
a
192.168.#.10 (# being
b
the trayy number 1 tho
ough 5)
b. Primary NIIC address: 192.168.#.11 (# being the tray
t
number 1 through 5)
Principled Tecchnologies, In
P
nc.: Real-world server conssolidation: 35 HP ProLiant DL385 serve
ers
o
onto
5 Dell Po
owerEdge M610 blade servvers running Hyper-V
H
10
2.
3.
4.
5.
6.
7.
8.
9.
c. Subnet mask: 255.255.255.0
d. Groupname: testgroup# (# being the tray number 1 through 5)
Using the EqualLogic web interface to enable all network connections on the Dell EqualLogic PS5000XV
using the following IP address scheme:
a. IP Address: 192.168.#.11, 12, 13 (# being the tray number between 1 through 5)
b. Subnet Mask: 255.255.255.0
Create one 400GB volume in the default storage pool with no snapshot reserve, and name it dvd-store
Create 14 25GB volumes in the storage pool with no snapshot reserve, and name each of them
dbdataVM#@ (where # is the number of the VM, and @ is the database volume – e.g., dbdata1A).
Create seven 15GB volumes in the storage pool with no snapshot reserve, and name each of them
dblogVM# (where # is the number of the VM – e.g., dblog1).
Enable shared access to the iSCSI target from multiple initiators on the volume.
Create an access control record for the volume without specifying any limitations.
Create a read-only SNMP community name to use for group monitoring.
Repeat steps 1 through 8 for the remaining five trays of Dell EqualLogic PS5000XV storage.
Configuring the Dell PowerConnect 6248 switch with jumbo frames
1. Obtain the IP address to the switch from your system administrator and connect a workstation to the
switch via an Ethernet cable.
2. Using Internet Explorer, navigate to the switch IP address and login.
3. Click the first port you will use for iSCSI traffic.
4. Modify the port settings as follows: Max Frame size 9216, which is the maximum.
5. Click Switching, Ports, Port Configuration table.
6. Choose Copy Parameters from unit 1, port g1.
7. Select all other ports you will use for iSCSI traffic.
8. Click Apply.
Configuring VLANs on the Dell PowerConnect 6248 switch
1. Obtain the IP address to the switch from your system administrator, and connect a workstation to the
switch via an Ethernet cable.
2. Using Internet Explorer, navigate to the switch IP address, and login.
3. Click Switching, VLAN, VLAN membership.
4. Click Add.
5. Enter a VLAN id and name, and click Apply Changes. Note the minimum VLAN id is 2, as the switch
reserves VLAN number 1.
6. Repeat steps 4 through 5 to add four additional VLANs.
7. Click VLAN membership.
8. Select the relevant VLAN id.
9. In the VLAN membership table, toggle the value to U to indicate that this port is assigned to this VLAN.
10. Repeat for other applicable ports. Note, in our configuration, we configured three ports for the Dell
EqualLogic PS5000XV storage, three ports for the server, and left two ports for management of the
VLAN, for a total of eight ports per VLAN.
11. Click Apply Changes.
12. Repeat steps 7 through 11 for the remaining four VLANs.
Setting up the Dell PowerEdge M610 blade servers
We installed a fresh copy of Windows Server 2008 R2 Datacenter Edition on each Dell PowerEdge M610.
Installing Windows Server 2008 R2 Datacenter Edition on the hosts
1. Boot the server, and insert the Windows Server 2008 R2 installation DVD in the DVD-ROM drive.
2. At the Language Selection Screen, click Next.
3. Click Install Now.
4. Select Windows Server 2008 R2 Datacenter (Full Installation), and click Next.
5. Click the I accept the license terms check box, and click Next.
6. Click Custom.
7. Click Drive options (advanced).
Principled Technologies, Inc.: Real-world server consolidation: 35 HP ProLiant DL385 servers
onto 5 Dell PowerEdge M610 blade servers running Hyper-V
11
8.
9.
10.
11.
12.
13.
14.
Ensure you select the proper drive, and click New.
Click Apply.
Click Next.
At the User’s password must be changed before logging on warning screen, click OK.
Type Password1 as the new password in both fields, and click the arrow to continue.
At the Your password has been changed screen, click OK.
Repeat steps 1 through 13 on each Dell PowerEdge M610.
Setting up network configuration on the server
1. Click StartÆControl PanelÆNetwork and InternetÆNetwork Connections, and double-click the Local
Area Connection assigned to client/server network traffic.
2. Select Internet Protocol Version 4 (TCP/IPv4), and click Properties.
3. In the Internet Protocol Version 4 (TCP/IPv4) Properties screen, select the Use the following IP address
radio button.
4. Enter a valid static IP address, subnet mask, and default gateway.
5. Click OK, and click Close to exit.
6. Repeat steps 1 through 6 for the remaining three Network Connections you will use for iSCSI, modifying
the subnet mask to match the storage subnet mask, and the IP address to match the iSCSI network
addressing scheme.
7. On the three Network Connections you will use for iSCSI, perform the following tasks:
a) Right-click the connection, and choose properties.
b) Click Configure.
c) Click the Advanced tab.
d) Click Jumbo MTU, and set the value to 9000.
8. Repeat steps 1 through 7 on each Dell PowerEdge M610.
Installing system updates in Windows Server 2008 R2
Because of the recent release of Windows Server 2008 R2, there were no updates available.
Installing driver updates and BIOS updates
On each server, we installed the latest network drivers and BIOS updates available via support.dell.com.
Installing the Host Integration toolkit on the Hyper-V server
1. Download the host integration toolkit (Setup64.exe).
2. Double click Setup64.exe.
3. Accept the security warnings, and choose to run the EXE.
4. Click Next.
5. Click Accept, then Next.
6. Click Typical, then Next.
7. Proceed through the installation, accepting the defaults.
8. Click Install.
9. Click Yes to install multipath software automatically.
10. Choose to restart the server when the application prompts you to do so.
11. Repeat steps 1 through 10 on each Dell PowerEdge M610 blade server.
Connecting the storage to the host
We attached the volumes that we used for VHD storage on each tray of Dell EqualLogic storage to their
respective Dell PowerEdge M610 blade server by connecting all three NICs available on each Dell EqualLogic
PS5000XV array to the Dell PowerConnect 6248 switch. We describe these steps above in the Dell PowerEdge
M610 and Dell EqualLogic PS5000XV storage configuration section.
Adding the additional drives for VHD and pass-through usage in the parent partition via the iSCSI initiator
1. Click Start, Control Panel, and double-click iSCSI Initiator.
2. Add the IP address of the Dell EqualLogic storage group in the Quick Connect box.
3. Find the relevant 400GB volume for VM OS VHDs, and select it. Click Log On…
Principled Technologies, Inc.: Real-world server consolidation: 35 HP ProLiant DL385 servers
onto 5 Dell PowerEdge M610 blade servers running Hyper-V
12
4. Check the checkbox to allow automatic logon on reboot and to enable multipath.
5. Click OK.
6. Repeat steps 3 through 5 for the 14 SQL Server data LUNs. Later, we configure the VMs with passthrough disks that we assigned to these LUNs.
7. Repeat steps 3 through 5 for the 7 SQL Server log LUNs. Later, we configure the VMs with pass-through
disks that we assigned to these LUNs.
8. Repeat steps 1 through 7 on each Dell PowerEdge M610 blade server.
Configuring the large external LUN in Disk Management and additional SQL Server data and log LUNs
1. On the host server, click Start, Administrative Tools, Computer Management.
2. Click Disk Management.
3. Right-click the uninitialized external volume, and choose Initialize Disk.
4. Choose MBR.
5. Right-click the volume, and choose New Simple Volume. Click Next.
6. Keep the default drive space, and click Next.
7. Keep the default drive letter assignment, and click Next.
8. Choose NTFS, and check the Perform a quick format checkbox. Click Next.
9. Click Finish.
10. Create folders necessary for the VM VHD storage.
11. Right-click the uninitialized smaller LUNs assigned for SQL Server data and log use, and choose Initialize
Disk.
12. Choose MBR.
13. Right-click the volume, and choose New Simple Volume. Click Next.
14. Keep the default drive space, and click Next.
15. Choose to not assign a drive letter, and click Next.
16. Choose NTFS, assign a 64KB allocation unit size, and check the Perform a quick format checkbox. Click
Next.
17. Click Finish.
18. Right-click the drive, and choose offline to make it available to VMs for pass-through usage.
19. Repeat steps 11 through 18 for the remaining SQL Server data and log LUNs.
20. Repeat steps 1 through 19 on each Dell PowerEdge M610 blade server.
Installing the Hyper-V role in Windows Server 2008 R2
1. Click StartÆAdministrative ToolsÆServer Manager.
2. Click Roles.
3. Click Add Roles.
4. Click Next.
5. Select the Hyper-V role by checking the box beside it, and click Next.
6. Check the box beside the Ethernet Card you wish to use for client/server traffic, and click Next.
7. Click Install to begin the Hyper-V installation.
8. When the installation finishes, click Close.
9. When the installer prompts you to restart now, click Yes.
10. Follow the prompt to log onto the system.
11. Enter the password, and click the arrow to continue. The Resuming Configuration Wizard should start
automatically.
12. Once the Resuming Configuration Wizard completes, the installation results should appear with a
message that the installation succeeded.
13. Click Close.
14. Repeat steps 1 through 13 on each Dell PowerEdge M610 blade server.
Creating the Hyper-V virtual machine
1. Click Start.
2. Expand Administrative Tools, and click Hyper-V Manager.
3. From the Action menu, select NewÆVirtual Hard Disk.
4. Choose Fixed Size, assign 30GB for size, and place the VHD in the appropriate VM folder on the 400GB
external volume.
Principled Technologies, Inc.: Real-world server consolidation: 35 HP ProLiant DL385 servers
onto 5 Dell PowerEdge M610 blade servers running Hyper-V
13
5. From the Action menu, select NewÆVirtual Machine.
6. On the Before You Begin page, click Next.
7. On the Specify Name and Location page, enter the name for your new virtual machine. Place the virtual
machine configuration in the relevant folder on the 400GB external volume, and click Next.
8. On the Assign Memory page, enter 9,216 MB of RAM.
9. On the Configure Networking, choose the network assigned to client/server traffic, and click Next.
10. Choose to add a virtual hard disk later.
11. On the installation Options page, accept the default of Install an operating system later, and click Next.
12. On the Completing the New Virtual Machine Wizard page, click Finish.
13. Right-click the virtual machine, and choose Settings.
14. Click Processors, and choose 2 virtual processors.
15. Click the virtual IDE controller 0.
16. Click Add, and click Hard Drive.
17. Browse to the VHD you created in Step 4, and choose that VHD for the guest OS VHD.
18. Click Apply.
19. Click the SCSI Controller, and click the Physical hard disk radio button to configure your SQL Server data
and SQL Server log pass-through disks. You may wish to do this step after cloning VMs.
20. Choose the appropriate LUN.
21. Repeat steps 19 and 20 for the remaining SQL Server data and SQL Server log LUNs assigned to this
VM.
22. Repeat steps 5 through 21 for your remaining VMs.
23. Repeat steps 1 through 22 on each Dell PowerEdge M610 blade server.
Installing the guest operating system
1. Insert the installation DVD for Windows Server 2003 x64 R2 with SP2 into the DVD drive.
2. In Hyper-V Manager, right-click the virtual machine, and choose connect.
3. On the media menu, choose DVD DriveÆCapture D:
4. On the Action menu, choose Start.
5. At the Welcome to Set Up screen, press Enter to begin the installation.
6. At the Windows Licensing Agreement screen, press F8 to agree to the terms.
7. Press Enter to install in the default location.
8. Select Format the partition using the NTFS file system (Quick), and press Enter to continue.
9. At the Regional and Language Options screen, click Next.
10. Enter Name and Organization, and click Next. Note: Organization is an optional field, while username is
required.
11. At the Your Product Key screen, enter your product key, and click Next.
12. At the Licensing Modes screen, click Next.
13. At the Computer Name and Administrator Password screen, type Password1 as the new password in
both fields, and click Next.
14. At the Date and Time Settings screen, change the Date & Time and Time Zone if appropriate, and click
Next.
15. After the system restarts automatically, follow the prompt to log into the server.
16. At the Windows Setup screen, the installer will prompt you to insert the second installation CD. Insert the
appropriate CD, and click OK.
17. At the Windows Server 2003 R2 Setup Wizard Welcome screen, click Next.
18. At the Windows Server 2003 R2 Setup Wizard Summary screen, click Next.
19. When the installer prompts you, click Finish to end the installation.
20. After the guest installation is complete, choose Action, Insert Integration Services Setup disk.
21. Proceed through the Integration Services installation. Restart the VM.
Configuring networking in the VM
1. Power on the VM, and open the console.
2. Click Start, Control Panel, right-click Network Connections, and choose Open.
3. Right-click the NIC, and choose properties.
4. Select TCP/IP (v4), and choose properties.
Principled Technologies, Inc.: Real-world server consolidation: 35 HP ProLiant DL385 servers
onto 5 Dell PowerEdge M610 blade servers running Hyper-V
14
5. Set the IP address, subnet, gateway, and DNS server for this NIC, which will handle outgoing server
traffic (e.g., the public IP). Click OK, and click Close.
6. Note that you must reconfigure all IP addresses after cloning VMs.
Installing SQL Server 2005 x64 on the VM
1. Insert the installation DVD for SQL Server 2005 x64 into the DVD drive.
2. In Hyper-V Manager, right-click the virtual machine, and choose connect.
3. On the media menu, choose DVD DriveÆCapture D:
4. On the Action menu, choose Start.
5. Navigate to the SQL Server 2005 DVD, and double-click. Choose the appropriate edition if the installer
prompts you with a menu.
6. Click the Server components link under the Install section to begin the installation.
7. At the End User License Agreement, accept the licensing terms, and click Next.
8. After installing the prerequisites, click Next.
9. At the Welcome screen, click Next.
10. At the System Configuration Check, click Next. For this installation, you may ignore the possible warning
regarding IIS, as it is unnecessary for this configuration.
11. At the Registration Screen, enter the appropriate name and company.
12. At the Components to Install screen, select SQL Server Database Services and Workstation components.
13. Click Advanced.
14. Uncheck Business Intelligent Development Studio, uncheck Replication, and uncheck Books Online &
Sample Databases.
15. Click Next to start the installation.
16. At the Instance Name screen, click the Default instance radio button, and click Next.
17. At the Service Account screen, select the Use the built-in System account radio button.
18. Select Local system, and click Next.
19. At the Authentication Mode screen, select Mixed Mode.
20. Specify the “sa” logon password by entering a password in both fields, and click Next.
21. At the Collation Settings screen, click Next.
22. Accept the default Error and Usage Report Settings, and click Next.
23. At the Ready to Install screen, click Install to begin the installation.
24. At the end of the installation in the Setup Progress screen, click Next.
25. Click Finish to complete the installation.
26. Click Start, All Programs, Microsoft SQL Server, SQL Server Management Studio. Connect to the SQL
Server instance.
27. Create a SQL Server login for the ds2user (see DVD Store setup section below for the specific script to
use).
28. Click Start, Administrative Tools, and Local Security Policy.
29. Double-click Perform Volume Maintenance Tasks, and add the System account.
30. Double-click Lock Pages in Memory, and add the System account.
31. Reboot the VM before installing SQL Server SP3.
Installing Service Pack 3 for Microsoft SQL Server 2005 in the VM
Service Pack 3 is available for download from
http://www.microsoft.com/downloads/details.aspx?FamilyID=ae7387c3-348c-4faa-8ae5949fdfbe59c4&displaylang=en.
1. In Hyper-V Manager, right-click the virtual machine, and choose connect.
2. On the media menu, choose DVD DriveÆCapture D:
3. On the Action menu, choose Start.
1. Run SQLServer2005SP3-KB955706-x64-ENU.exe.
2. At the Welcome screen, click Next.
3. At Accept the License Agreement, click Next.
4. At the Feature Selection screen, keep the defaults, and click Next.
5. At the Authentication screen, click Next.
6. At the Error and Usage Reporting Settings screen, click Next.
7. At the Running Processes screen, click Next.
Principled Technologies, Inc.: Real-world server consolidation: 35 HP ProLiant DL385 servers
onto 5 Dell PowerEdge M610 blade servers running Hyper-V
15
8.
9.
10.
11.
12.
13.
Click Install.
If a compatibility error occurs during the installation, click Run program.
Click Next.
At the Installation Complete screen, click Next.
Click Finish.
Reboot the VM.
Installing system updates in Windows Server 2003 x64 in the VM
We installed the following updates on the VM using the Microsoft Windows Update feature:
• Update to .NET Framework 3.5 Service Pack 1 for the .NET Framework Assistant 1.0 x64 (KB963707)
• Update for Windows Server 2003 x64 Edition (KB961118)
• Microsoft .NET Framework 3.5 Service Pack 1 and .NET Framework 3.5 Family Update for .NET versions
2.0 through 3.5 (KB951847) x64
• Microsoft .NET Framework 3.5 Service Pack 1 and .NET Framework 3.5 Family Update for .NET versions
2.0 through 3.5 (KB951847) x64
• Microsoft .NET Framework 3.5 Service Pack 1 and .NET Framework 3.5 Family Update for .NET versions
2.0 through 3.5 (KB951847) x64
• Security Update for Microsoft XML Core Services 6.0 Service Pack 2 for x64-based Systems (KB954459)
• Update for Windows Server 2003 x64 Edition (KB970653)
• Security Update for Windows Server 2003 x64 Edition (KB973869)
• Security Update for Windows Server 2003 x64 Edition (KB971657)
• Security Update for Windows Server 2003 x64 Edition (KB973815)
• Security Update for Windows Server 2003 x64 Edition (KB960859)
• Security Update for Windows Server 2003 x64 Edition (KB973507)
• Security Update for Windows Server 2003 x64 Edition (KB973354)
• Security Update for Windows Server 2003 x64 Edition (KB958469)
• Security Update for Windows Server 2003 x64 Edition (KB973540)
• Security Update for Windows Server 2003 x64 Edition (KB971557)
• Security Update for Windows Server 2003 x64 Edition (KB971032)
• Update for Windows Server 2003 x64 Edition (KB968389)
• Security Update for Windows Server 2003 x64 Edition (KB961371)
• Windows Malicious Software Removal Tool x64 - August 2009 (KB890830)
• Cumulative Security Update for Internet Explorer 6 for Windows Server 2003 x64 Edition (KB972260)
• Cumulative Security Update for ActiveX Killbits for Windows Server 2003 x64 Edition (KB973346)
• Security Update for Windows Server 2003 x64 Edition (KB971633)
• Security Update for Windows Server 2003 x64 Edition (KB970238)
• Security Update for Windows Server 2003 x64 Edition (KB968537)
• Security Update for Windows Server 2003 x64 Edition (KB961501)
• Security Update for Windows Server 2003 x64 Edition (KB959426)
• Security Update for Windows Server 2003 x64 Edition (KB960803)
• Security Update for Windows Server 2003 x64 Edition (KB952004)
• Security Update for Windows Server 2003 x64 Edition (KB956572)
• Security Update for Windows Server 2003 x64 Edition (KB923561)
• Update for Windows Server 2003 x64 Edition (KB967715)
• Security Update for Windows Server 2003 for x64 Edition (KB938464)
• Security Update for Windows Server 2003 x64 Edition (KB960225)
• Security Update for Windows Server 2003 x64 Edition (KB958687)
• Security Update for Windows Server 2003 x64 Edition (KB956802)
• Security Update for Windows Server 2003 x64 Edition (KB954600)
• Security Update for Windows Server 2003 x64 Edition (KB952069)
• Security Update for Windows Server 2003 x64 Edition (KB957097)
• Security Update for Windows Server 2003 x64 Edition (KB955069)
• Security Update for Windows Server 2003 x64 Edition (KB958644)
Principled Technologies, Inc.: Real-world server consolidation: 35 HP ProLiant DL385 servers
onto 5 Dell PowerEdge M610 blade servers running Hyper-V
16
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Security Update for Windows Server 2003 x64 Edition (KB956803)
Security Update for Windows Server 2003 x64 Edition (KB950974)
Security Update for Windows Server 2003 x64 Edition (KB952954)
Security Update for Outlook Express for Windows Server 2003 x64 Edition (KB951066)
Security Update for Windows Server 2003 x64 Edition (KB944338)
Security Update for Windows Server 2003 x64 Edition (KB951748)
Security Update for Windows Server 2003 x64 Edition (KB950762)
Security Update for Windows Server 2003 x64 Edition (KB945553)
Update for Windows Server 2003 x64 Edition (KB948496)
Security Update for Windows Server 2003 x64 Edition (KB943055)
Security Update for Windows Server 2003 x64 Edition (KB946026)
Security Update for Windows Server 2003 x64 Edition (KB944653)
Security Update for Windows Server 2003 x64 Edition (KB941569)
Security Update for Windows Server 2003 x64 Edition (KB943460)
Security Update for Windows Server 2003 x64 Edition (KB938127)
Security Update for Windows Server 2003 x64 Edition (KB926122)
Security Update for Windows Media Player 6.4 (KB925398)
Cumulative Security Update for Outlook Express for Windows Server 2003 x64 Edition (KB929123)
Security Update for Windows Server 2003 x64 Edition (KB924667)
Update for Windows Server 2003 x64 Edition (KB927891)
Security Update for Windows Server 2003 x64 Edition (KB932168)
Security Update for Windows Server 2003 x64 Edition (KB930178)
Security Update for Windows Server 2003 x64 Edition (KB925902)
Configuring additional drives in Disk Management
1. In Hyper-V Manager, right-click the virtual machine, and choose connect.
2. On the Action menu, choose Start.
3. Log into the VM.
4. Click Start, Administrative Tools, Computer Management.
5. Click Disk Management.
6. Right-click the disks assigned for SQL Server data and log use, and choose online.
7. Create the folders necessary for the database. We used <driveletter>:\SQLData for the SQL Server
database volume and <driveletter>:\SQLLog for the SQL Server transaction log volume.
8. Repeat steps 3 and 4 for each volume on the VM.
9. Repeat steps 1 through 8 for each VM.
10. Repeat steps 1 through 9 on each Dell PowerEdge M610 blade server.
Creating the additional VMs on each host server
After installing and configuring the initial VM, we disconnected the primary VHD file, and then copied that VHD to
all other VMs to use for their operating system VHD. We reconfigured all IP addresses and hostnames on new
VMs. Using Hyper-V Manager, we also connected each VM’s associated SQL Server data LUNs and SQL Server
log LUNs. We repeated this process on each host server.
Configuring the additional VMs after cloning
1. In Hyper-V Manager, right-click the virtual machine, and choose connect.
2. On the Action menu, choose Start.
3. Click StartÆControl PanelÆ right-click Network Connections, and double-click the Local Area
Connection.
4. Select Internet Protocol Version 4 (TCP/IPv4), and click Properties.
5. In the Internet Protocol Version 4 (TCP/IPv4) Properties screen, select the Use the following IP address
radio button.
6. Enter a valid static IP address for this new VM, subnet mask, and default gateway.
7. Click OK, and click Close to exit.
8. Click StartÆControl PanelÆSystem.
Principled Technologies, Inc.: Real-world server consolidation: 35 HP ProLiant DL385 servers
onto 5 Dell PowerEdge M610 blade servers running Hyper-V
17
9.
10.
11.
12.
13.
14.
15.
Click Change Settings to change the computer name.
Click the Change button.
Enter the new computer name.
Click OK, and click OK again.
Restart the VM.
Repeat steps 2 through 13 for each VM.
Repeat steps 1 through 14 on each Dell PowerEdge M610 blade server.
Setting up and configuring the HP DL385 server and HP Modular Smart Array 30 storage
We setup 35 HP ProLiant DL385 servers and 18 Modular Smart Array 30 (MSA30) storage arrays in identical
fashion, as we describe below.
We installed a fresh copy of Windows Server 2003 R2 x64 Enterprise Edition with Service Pack 2 on the HP
ProLiant DL385 servers.
We installed four internal drives in each HP DL385. We attached each server to a Modular Smart Array 30
(MSA30). We cabled the MSA30 to the Smart Array 6400 storage adapter in the server via one SCSI cable. The
I/O module on the MSA30 was a dual port I/O module. Each array contained the drives relevant to two servers,
with each server getting 7 of the 14 drives in the server.
We configured the internal drives in two RAID 1 volumes, with two disks per volume. The MSA30 contained 14
146GB 10,000 RPM U320 SCSI drives. As we note above, using the dual port I/O module, we split each MSA30
in half, with seven drives going to each server. We configured the MSA30 volumes in a RAID 5 array for each
seven drive set, resulting in approximately an 800GB volume at the operating system level, which we then
partitioned, using a 50GB partition for SQL Server data on the external storage. We used a 15GB partition for
SQL Server logs on the second RAID 1 set internally.
We installed Windows Server 2003 R2 x64 Enterprise Edition and SQL Server 2005 x64 software on the first
mirrored internal volume, configured the SQL Server transaction logs on the second internal volume, and placed
the SQL Server database data on the MSA30 volume.
Setting up the internal storage for operating system installation
1. Enter the RAID controller BIOS by pressing F8 at the HP Smart Array 6i Controller prompt.
2. Select Create Logical Drive.
3. Select the two drives to use for this RAID 1 volume.
4. Press F8 to save this configuration.
5. Press Enter to continue.
6. Press Escape to return to the boot sequence.
7. Repeat steps 1 through 6 on each HP ProLiant Server.
Setting up the external storage
We configured each MSA30 storage array via the HP Array Configuration Utility after installing the operating
system. See the steps in Adding additional drives to the server via the HP Smart Array Configuration Utility,
DISKPART utility, and Disk Management Utility for details.
Installing Windows Server 2003 R2 x64 Enterprise Edition with Service Pack 2
1. Boot the server, and insert the Windows Server 2003 R2 x64 installation CD in the DVD-ROM drive.
2. At the Welcome to Set up screen, press Enter to begin the installation.
3. At the Windows Licensing Agreement screen, press F8 to agree to the terms.
4. Press Enter to install in the default location.
5. Select Format the partition using the NTFS file system (Quick), and press Enter to continue.
6. At the Regional and Language Options screen, click Next.
7. Enter Name and Organization, and click Next. Note: Organization is an optional field, while username is
required.
8. At the Your Product Key screen, enter your product key, and click Next.
Principled Technologies, Inc.: Real-world server consolidation: 35 HP ProLiant DL385 servers
onto 5 Dell PowerEdge M610 blade servers running Hyper-V
18
9. At the Licensing Modes screen, click Next.
10. At the Computer Name and Administrator Password screen, type Password1 as the new password in
both fields, and click Next.
11. At the Date and Time Settings screen, change the Date & Time and Time Zone if appropriate, and click
Next.
12. After the system restarts automatically, follow the prompt to log into the server.
13. At the Windows Setup screen, the installer will prompt you to insert the second installation CD. Insert the
appropriate CD, and click OK.
14. At the Windows Server 2003 R2 Setup Wizard Welcome screen, click Next.
15. At the Windows Server 2003 R2 Setup Wizard Summary screen, click Next.
16. When the installer prompts you, click Finish to end the installation.
17. Repeat steps 1 through 16 on each HP ProLiant Server.
Setting up network configuration on the server
1. Click StartÆControl PanelÆNetwork ConnectionsÆLocal Area Connection.
2. Click Properties.
3. Select Internet Protocol (TCP/IP), and click Properties.
4. In the Internet Protocol (TCP/IP) Properties screen, select the Use the following IP address radio button.
5. Enter a valid static IP address, subnet mask, and default gateway.
6. Select the Use the following DNS server addresses radio button, and enter at least one valid DNS server
address.
7. Click OK, and click OK to exit.
8. Repeat steps 1 through 7 on each HP ProLiant Server.
Installing system updates in Windows Server 2003 R2 x64 Enterprise Edition
We installed the following updates on the machines using the Microsoft Windows Update feature:
• Update to .NET Framework 3.5 Service Pack 1 for the .NET Framework Assistant 1.0 x64 (KB963707)
• Update for Windows Server 2003 x64 Edition (KB961118)
• Microsoft .NET Framework 3.5 Service Pack 1 and .NET Framework 3.5 Family Update for .NET versions
2.0 through 3.5 (KB951847) x64
• Microsoft .NET Framework 3.5 Service Pack 1 and .NET Framework 3.5 Family Update for .NET versions
2.0 through 3.5 (KB951847) x64
• Microsoft .NET Framework 3.5 Service Pack 1 and .NET Framework 3.5 Family Update for .NET versions
2.0 through 3.5 (KB951847) x64
• Security Update for Microsoft XML Core Services 6.0 Service Pack 2 for x64-based Systems (KB954459)
• Update for Windows Server 2003 x64 Edition (KB970653)
• Security Update for Windows Server 2003 x64 Edition (KB973869)
• Security Update for Windows Server 2003 x64 Edition (KB971657)
• Security Update for Windows Server 2003 x64 Edition (KB973815)
• Security Update for Windows Server 2003 x64 Edition (KB960859)
• Security Update for Windows Server 2003 x64 Edition (KB973507)
• Security Update for Windows Server 2003 x64 Edition (KB973354)
• Security Update for Windows Server 2003 x64 Edition (KB958469)
• Security Update for Windows Server 2003 x64 Edition (KB973540)
• Security Update for Windows Server 2003 x64 Edition (KB971557)
• Security Update for Windows Server 2003 x64 Edition (KB971032)
• Update for Windows Server 2003 x64 Edition (KB968389)
• Security Update for Windows Server 2003 x64 Edition (KB961371)
• Windows Malicious Software Removal Tool x64 - August 2009 (KB890830)
• Cumulative Security Update for Internet Explorer 6 for Windows Server 2003 x64 Edition (KB972260)
• Cumulative Security Update for ActiveX Killbits for Windows Server 2003 x64 Edition (KB973346)
• Security Update for Windows Server 2003 x64 Edition (KB971633)
• Security Update for Windows Server 2003 x64 Edition (KB970238)
• Security Update for Windows Server 2003 x64 Edition (KB968537)
Principled Technologies, Inc.: Real-world server consolidation: 35 HP ProLiant DL385 servers
onto 5 Dell PowerEdge M610 blade servers running Hyper-V
19
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Security Update for Windows Server 2003 x64 Edition (KB961501)
Security Update for Windows Server 2003 x64 Edition (KB959426)
Security Update for Windows Server 2003 x64 Edition (KB960803)
Security Update for Windows Server 2003 x64 Edition (KB952004)
Security Update for Windows Server 2003 x64 Edition (KB956572)
Security Update for Windows Server 2003 x64 Edition (KB923561)
Update for Windows Server 2003 x64 Edition (KB967715)
Security Update for Windows Server 2003 for x64 Edition (KB938464)
Security Update for Windows Server 2003 x64 Edition (KB960225)
Security Update for Windows Server 2003 x64 Edition (KB958687)
Security Update for Windows Server 2003 x64 Edition (KB956802)
Security Update for Windows Server 2003 x64 Edition (KB954600)
Security Update for Windows Server 2003 x64 Edition (KB952069)
Security Update for Windows Server 2003 x64 Edition (KB957097)
Security Update for Windows Server 2003 x64 Edition (KB955069)
Security Update for Windows Server 2003 x64 Edition (KB958644)
Security Update for Windows Server 2003 x64 Edition (KB956803)
Security Update for Windows Server 2003 x64 Edition (KB950974)
Security Update for Windows Server 2003 x64 Edition (KB952954)
Security Update for Outlook Express for Windows Server 2003 x64 Edition (KB951066)
Security Update for Windows Server 2003 x64 Edition (KB944338)
Security Update for Windows Server 2003 x64 Edition (KB951748)
Security Update for Windows Server 2003 x64 Edition (KB950762)
Security Update for Windows Server 2003 x64 Edition (KB945553)
Update for Windows Server 2003 x64 Edition (KB948496)
Security Update for Windows Server 2003 x64 Edition (KB943055)
Security Update for Windows Server 2003 x64 Edition (KB946026)
Security Update for Windows Server 2003 x64 Edition (KB944653)
Security Update for Windows Server 2003 x64 Edition (KB941569)
Security Update for Windows Server 2003 x64 Edition (KB943460)
Security Update for Windows Server 2003 x64 Edition (KB938127)
Security Update for Windows Server 2003 x64 Edition (KB926122)
Security Update for Windows Media Player 6.4 (KB925398)
Cumulative Security Update for Outlook Express for Windows Server 2003 x64 Edition (KB929123)
Security Update for Windows Server 2003 x64 Edition (KB924667)
Update for Windows Server 2003 x64 Edition (KB927891)
Security Update for Windows Server 2003 x64 Edition (KB932168)
Security Update for Windows Server 2003 x64 Edition (KB930178)
Security Update for Windows Server 2003 x64 Edition (KB925902)
Connecting the storage arrays to the servers
We attached the database volumes on each HP MSA30 storage to their respective HP ProLiant DL385 servers by
using a SCSI cable connected from the Smart Array 6400 Controller in the server to one SCSI connector on the
back of the HP MSA30 dual port I/O module. As we noted above, two servers shared one HP MSA30 array, with
each server occupying one port of the I/O module.
Adding additional drives to the servers via the HP Smart Array Configuration Utility, DISKPART utility,
and Disk Management Utility
1. Download and install the HP Smart Array Configuration Utility (ACU) from
http://h18000.www1.hp.com/products/servers/proliantstorage/softwaremanagement/acumatrix/index.html.
2. Open the ACU, and select the internal controller.
3. Click Create Array.
Principled Technologies, Inc.: Real-world server consolidation: 35 HP ProLiant DL385 servers
onto 5 Dell PowerEdge M610 blade servers running Hyper-V
20
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
Select the remaining two available drives, and click OK.
Click Create Logical Drive.
Leave Fault tolerance at RAID 1+0, leave the default stripe size, and click OK.
Click Save.
Select the Smart Array 6400 external controller.
Click Create Array.
Select the seven available drives, and click OK.
Click Create Logical Drive.
Choose RAID 5 for the fault tolerance selection, leave the default stripe size, and click OK.
Click Save.
Exit the ACU utility.
Click Start, Run, type cmd and press Enter.
Type diskpart
Type list disk
Type select disk # (where # is the disk number of the internal volume).
Type create partition primary size=15360 align=1024
Type assign letter=F for the internal volume.
Type list disk
Type select disk # (where # is the disk number of the external MSA30 volume).
Type create partition primary size=51200 align=1024
Type assign letter=E for the external volume.
Exit the command prompt.
Click Start, Administrative Tools, Computer Management.
Click Disk Management.
Right-click the volume, and select Format. Choose 64KB allocation unit size, NTFS file system, select
Perform Quick Format, and press OK.
29. Repeat steps 27 and 28 for the external MSA30 volume.
30. Create the folders necessary for the database. We used E:\SQLData for the SQL Server database
volume and F:\SQLLog for the SQL Server transaction log volume.
31. Repeat steps 1 through 30 on each HP ProLiant Server.
Installing SQL Server 2005 x64 on the server
1. Navigate to the SQL Server 2005 DVD, and double-click. Choose the appropriate edition if the installer
prompts you with a menu.
2. Click the Server components link under the Install section to begin the installation.
3. At the End User License Agreement, accept the licensing terms, and click Next.
4. After installing the prerequisites, click Next.
5. At the Welcome screen, click Next.
6. At the System Configuration Check, click Next. For this installation, you may ignore the possible warning
regarding IIS, as it is unnecessary for this configuration.
7. At the Registration Screen, enter the appropriate name and company.
8. At the Components to Install screen, select SQL Server Database Services and Workstation components.
9. Click Advanced.
10. Uncheck Business Intelligent Development Studio, uncheck Replication, and uncheck Books Online &
Sample Databases.
11. Click Next to start the installation.
12. At the Instance Name screen, click the Default instance radio button, and click Next.
13. At the Service Account screen, select the Use the built-in System account radio button.
14. Select Local system, and click Next.
15. At the Authentication Mode screen, select Mixed Mode.
16. Specify the “sa” logon password by entering a password in both fields, and click Next.
17. At the Collation Settings screen, click Next.
18. Accept the default Error and Usage Report Settings, and click Next.
19. At the Ready to Install screen, click Install to begin the installation.
20. At the end of the installation in the Setup Progress screen, click Next.
Principled Technologies, Inc.: Real-world server consolidation: 35 HP ProLiant DL385 servers
onto 5 Dell PowerEdge M610 blade servers running Hyper-V
21
21. Click Finish to complete the installation.
22. Click Start, All Programs, Microsoft SQL Server, SQL Server Management Studio. Connect to the SQL
Server instance.
23. Right-click the instance in Object explorer, and choose Properties.
24. Click Processors, and select the Boost SQL Server Priority checkbox. Click OK.
25. Restart the SQL Server by right-clicking the instance and choosing restart.
26. Create a SQL Server login for the ds2user (see DVD Store setup section below for the specific script to
use).
27. Click Start, Administrative Tools, and Local Security Policy.
28. Double-click Perform Volume Maintenance Tasks, and add the System account.
29. Double-click Lock Pages in Memory, and add the System account.
30. Reboot the server before installing SQL Server SP3.
Installing Service Pack 3 for Microsoft SQL Server 2005
Service Pack 3 is available for download from
http://www.microsoft.com/downloads/details.aspx?FamilyID=ae7387c3-348c-4faa-8ae5949fdfbe59c4&displaylang=en.
1. Run SQLServer2005SP3-KB955706-x64-ENU.exe.
2. At the Welcome screen, click Next.
3. At Accept the License Agreement, click Next.
4. At the Feature Selection screen, keep the defaults, and click Next.
5. At the Authentication screen, click Next.
6. At the Error and Usage Reporting Settings screen, click Next.
7. At the Running Processes screen, click Next.
8. Click Install.
9. If a compatibility error occurs during the installation, click Run program.
10. Click Next.
11. At the Installation Complete screen, click Next.
12. Click Finish.
13. Reboot the server.
DVD Store setup
Data generation overview
We built the database schema using the scripts in the DS2 distribution package, though we made a few minor
modifications. The DS2 stress tool provides options to generate 10MB, 1GB, or 100GB datasets. To get the tool
to generate the 10 GB of user data we used in this test, we had to make a few straightforward changes to the
source code and to the DVD Store application’s scripts. Note: We created our test data on a Linux system to take
advantage of the larger RAND MAX.
Editing the ds2_create_orders.c module
The module ds2_create_orders.c defines constants that define the maximum values for the customer ID and the
product ID. The constants for the 10GB database size did not exist. We added the constants for this size.
On the command line for the ds2_create_orders.c module, we specified the size. The available options were S
(small), M (medium), and L (large). We added the case T for the 10GB database. In the switch statement that sets
the values for the variables max_cust_id and max_prod_id, we added cases that assigned them the proper values
for the 10GB database size.
We recompiled the ds2_create_orders.c module on Linux, following the instructions in the header comments. We
used the following command line:
gcc –o ds2_create_orders ds2_create_orders.c –lm
Editing the ds2_create_cust.c module
We had to make the same changes to the ds2_create_cust.c module that we made to the ds2_create_orders.c
module. On the command line for the ds2_create_cust.c module, we specified the size. The available options
were S (small), M (medium), and L (large). We added the case T for the 10GB database. In the switch statement
Principled Technologies, Inc.: Real-world server consolidation: 35 HP ProLiant DL385 servers
onto 5 Dell PowerEdge M610 blade servers running Hyper-V
22
that sets the values for the variables max_cust_id and max_prod_id, we added cases that assigned them the
proper values for the 10GB database size.
We recompiled the ds2_create_cust.c module on Linux, following the instructions in the header comments. We
used the following command line:
gcc –o ds2_create_cust ds2_create_cust.c –lm
Generating the data for the 10GB database
We used shell scripts to run all four of the executables that generate the data. The distribution did not include
shell scripts for the 10GB size. We wrote shell scripts based on the ds2_create_cust_large.sh and
ds2_create_orders_large.sh scripts. The ds2_create_prod and ds2_create_inv executables did not ship with
associated shell scripts, so we created shell scripts using the instructions in the readme files. We ran the shell
scripts in the following order to generate the data for the 10GB database:
1.
2.
3.
4.
ds2_create_orders_10gb.sh
ds2_create_inv_10gb.sh
ds2_create_prod_10gb.sh
ds2_create_cust_10gb.sh
We waited until the processes finished before we moved onto the next step.
Creating the database
We modified the database creation SQL Server scripts in the DVD Store distribution package to build the
database schema, which includes the file structure, tables, indices, stored procedures, triggers, and so on. We
built a master copy of the 10GB database version for SQL Server 2005, and then used that master copy to
restore our test database to the HP ProLiant DL385 solution and Dell PowerEdge M610 solution between each
test run. We stored the backup file on the C: drive of each server or virtual machine for quick access.
We followed these steps to create the database:
1. We created the database and file structure using database creation scripts in the DS2 download. We
made size modifications specific to our 10GB database and the appropriate changes to drive letters.
2. We created database tables, stored procedures, and objects.
3. We set the database recovery model to bulk-logged to prevent excess logging.
4. We loaded the data we generated into the database. For data loading, we used the import wizard in SQL
Server Management Studio. Where necessary, we retained options from the original scripts, such as
Enable Identity Insert.
5. We created indices, full-text catalogs, primary keys, and foreign keys using the database-creation scripts.
6. We updated statistics on each table according to database-creation scripts, which sample 18 percent of
the table data.
7. On each SQL Server instance, we created a ds2user SQL Server login using the following Transact SQL
(TSQL) script:
USE [master]
GO
CREATE LOGIN [ds2user] WITH PASSWORD=N’’,
DEFAULT_DATABASE=[master],
DEFAULT_LANGUAGE=[us_english],
CHECK_EXPIRATION=OFF,
CHECK_POLICY=OFF
GO
8. We set the database recovery model back to full.
We made the following changes in the build scripts:
Principled Technologies, Inc.: Real-world server consolidation: 35 HP ProLiant DL385 servers
onto 5 Dell PowerEdge M610 blade servers running Hyper-V
23
•
•
•
•
•
Because we varied the size of the datasets, we sized the files in our scripts to reflect the database size
and the number of files per filegroup. We allowed for approximately 40 percent free space in our
database files to ensure that filegrowth activity did not occur during the testing.
We followed Microsoft’s recommendation of having 0.25 to 1 file per filegroup per core. We used two files
per filegroup on the HP ProLiant DL385 server and in the virtual machines on the Dell PowerEdge M610
blade server.
We did not use the DBCC PINTABLE command for the CATEGORIES and PRODUCTS tables, both
because Microsoft recommends against this practice and because the commands do nothing in SQL
Server 2005.
We created a SQL Server login we called ds2user and mapped a database user to this login. We made
each such user a member of the db_owner fixed database role.
Using the DVD Store scripts as a reference, we created the full-text catalog and index on the PRODUCTS
table manually in SQL Server Management Studio.
We then performed a full backup of the database. This backup allowed us to restore the databases to a pristine
state relatively quickly between tests.
Editing the workload script - ds2xdriver.cs module
To use the 10GB database we created earlier, we had to change the following constants:
•
•
•
In the routine Controller(), we changed the string sizes. We added the T option for the 10GB database
size. DS2 uses the sizes string to interpret the db_size_str option.
In the class Controller, we changed the arrays MAX_CUSTOMER and MAX_PRODUCT. To each, we
added values specifying the bounds for the customer and product IDs. The Controller() routine uses these
arrays.
We added a command line parameter for the database name:
—database_name
Editing the workload script - ds2sqlserverfns.cs module
We changed the connection string to increase the number of available connections, to not use the default
administrator (“sa”) account, and to include a parameter for the database name. We raised the available
connections limit from the default of 100 to 200 to allow room for experimentation. We created a user account we
called ds2User and used that account.
The ds2connect routine in the ds2sqlserverfns.cs module defines sConnectionString. We used the following
string; the changes we made appear in bold:
string sConnectionString = “User ID=ds2User;Initial Catalog=“+dbname+”;Max
Pool Size=200;Connection Timeout=120;Data Source=“ + Controller.target;
Recompiling the ds2sqlserverdriver.exe executable
We recompiled the ds2xdriver.cs and ds2sqlserverfns.cs module in Windows by following the instructions in the
header comments. Because the DS2 instructions were for compiling from the command line, we used the
following steps:
1. Open a command prompt.
2. Use the cd command to change to the directory containing our sources.
3. Run the batch file C:\Program Files\Microsoft Visual Studio 9.0\Common7\Tools\vsvars32.bat. This sets
up the environment variables for us.
4. Execute the following command:
csc /out:ds2sqlserverdriver.exe ds2xdriver.cs ds2sqlserverfns.cs
/d:USE_WIN32_TIMER /d:GEN_PERF_CTRS
Principled Technologies, Inc.: Real-world server consolidation: 35 HP ProLiant DL385 servers
onto 5 Dell PowerEdge M610 blade servers running Hyper-V
24
Conducting the test
To perform the test, we used a series of batch files. We stored batch files either under the C:\ServerShare folder
on each server or virtual machine or under the C:\ClientShare folder on the clients, and we used the PsExec utility
to coordinate the test procedures amongst the client machines, server machines, virtual machines, and controller.
We used simple file sharing to copy files from machine to machine, as we needed.
The testing procedure consisted of the following steps:
1. Execute batch files to clean up prior outputs on client machines, the server machine, the virtual machines,
and the controller.
2. Drop the database on each SQL Server instance.
3. Restore the database on each SQL Server instance.
4. Pause briefly to wait for background tasks to complete before the scripts shut down the VMs, and reboot
the server.
5. The scripts shut down the virtual machines, reboot the client machines and server machine, and wait for a
ping response from all test machines. In the case of the HP ProLiant DL385 solution, there were no virtual
machines.
6. After all servers and virtual machines respond, wait 10 additional minutes for any background tasks to
complete.
7. Measure idle power for 2 minutes.
8. Start the counter logs on client machines, and start the data collector set on the server under test.
9. Start the workload connections.
10. Start the workload ramp-up period.
11. Start the workload on the server, and run it for 30 minutes.
12. Stop the workload.
13. Copy all output files to the controller.
Measuring power
To record each solution’s power consumption during each test, we used up to nine Extech Instruments
(www.extech.com) 380803 Power Analyzer/Dataloggers. For the 35 HP ProLiant solutions, we connected all
power distribution units (PDU) from the three racks containing the HP ProLiant DL385 servers to the Power
Analyzers’ output load power outlets. We then plugged the power cords from the Power Analyzers’ input voltage
connection into a power outlet. The storage arrays’ power was also distributed through the PDU system in the
racks, so the nine Extech Power Analyzers measured the aggregate total of power usage of the server and
storage solution. For the five Dell PowerEdge M610 solutions, we configured the power measurement similarly,
but used only five power meters because the five Dell PowerEdge M610 solutions required less power..
We used the Power Analyzer’s Data Acquisition Software (version 2.11) to capture all recordings. We installed the
software on two separate Intel processor-based PCs, which we connected to the Power Analyzers via RS-232
cables. We captured power consumption at one-second intervals.
To gauge the idle power usage, we recorded the power usage for 2 minutes while all systems were running the
host operating system and VM operating systems, but otherwise idle, meaning the systems were not running any
test workloads. To ensure the systems were idle, we allowed the systems to sit for 10 minutes after booting
before starting the 2 minutes of idle power capture.
We then recorded the power usage (in watts) for all systems and storage during the testing at 1-second intervals.
To compute the average power usage, we averaged the power usage during the time the systems were
producing the peak performance results. Because we measured the power using nine Power Analyzers and
multiple PDUs, we added the average power of all Power together to get the total power. We call this time the
power measurement interval. See Figure 6 (idle and workload power results) for the results of these
measurements.
Principled Technologies, Inc.: Real-world server consolidation: 35 HP ProLiant DL385 servers
onto 5 Dell PowerEdge M610 blade servers running Hyper-V
25
Appendix A – Test server information
Figure 8 presents detailed information for the test servers we used in this report.
Servers
Enclosure
Blade Enclosure
General dimension information
Height (inches)
Width (inches)
Depth (inches)
U size in server rack (U)
Power supplies
Total number
Wattage of each (W)
Cooling fans
Total number
Dimensions (h x w) of each
Voltage (V)
Amps (A)
General processor setup
Number of processor packages
Number of cores per processor
package
Number of hardware threads per
core
System power management policy
CPU
Vendor
Name
Stepping
Socket type
Core frequency (GHz)
L1 cache
L2 cache
L3 cache
Platform
Vendor and model number
Motherboard model number
BIOS name and version
BIOS settings
Memory modules
Total RAM in system (GB)
Number of types of memory
modules
5 Dell PowerEdge M610
35 HP ProLiant DL385
Dell PowerEdge M1000e
N/A
17.3
17.6
29.7
10
3.4
17.5
26.0
2
6
2,360
2
575
9
3.1” x 3.5”
12
7.00
7
2.5” x 2.5”
12
1.10
2
2
4
1
2
1
OS control
Always On
Intel
Xeon X5570
D0
LGA1366
2.93
128 KB (4 x 32 KB per core)
1 MB (4 x 256 KB)
1 x 8 MB
AMD
Opteron 254
E4
Socket 940
2.80
64 KB + 64 KB
1 MB
N/A
Dell PowerEdge M610
0N582M
Dell 1.2.7 (8/31/2009)
Virtualization enabled
HP ProLiant DL385 G1
N/A
HP A05
Default
72
4
2
1
Principled Technologies, Inc.: Real-world server consolidation: 35 HP ProLiant DL385 servers
onto 5 Dell PowerEdge M610 blade servers running Hyper-V
26
Servers
5 Dell PowerEdge M610
35 HP ProLiant DL385
First type of memory modules
Vendor and model number
Type
Speed (MHz)
Speed in the system currently
running @ (MHz)
Timing/latency (tCL-tRCD-iRPtRASmin)
Size (GB)
Number of RAM modules
Chip organization
Second type of memory modules
Vendor and model number
Type
Speed (MHz)
Speed in the system currently
running @ (MHz)
Timing/Latency (tCL-tRCD-iRPtRASmin)
Size (GB)
Number of RAM modules
Chip organization
Hard disk
Vendor and model number
Number of disks in system
Size (GB)
Buffer size (MB)
RPM
Type
Operating system
Name
Build number
File system
Language
Network/storage subsystem
Vendor and model number
Type
Vendor and model number
Samsung M393B1K70BH1-CF8 (3
servers) /
Hynix HMT31GR7AFR4C-H9 (2
servers)
PC3-8500R (Samsung) /
PC3-10600R (Hynix)
1,066 (Samsung) /
1,333 (Hynix)
1,066
Hynix HYMD512G726CFP4Q-D43
PC-3200
400
400
8-8-8-24 (Samsung) /
9-9-9-36 (Hynix)
48 (all servers)
6 x 8 GB (all servers)
Double-sided (all servers)
4
4 x 1 GB
Double-sided
Samsung M393B5170EH1-CH9
PC3-10600R
1,333
N/A
N/A
N/A
1,066
N/A
8-8-8-24
N/A
24
6 x 4 GB
Double-sided
N/A
N/A
N/A
Seagate ST973452SS
2
73
16
15,000
SAS
HP BF072863BA
4
72
8
15,000
U320 SCSI
Windows Server 2008 R2
Datacenter Edition with Hyper-V
6.1.7600
NTFS
English
Windows Server 2003 Enterprise
Edition x64 R2 with SP2
5.2.3790
NTFS
English
Broadcom NetXtreme II 5709 DualPort Ethernet
Integrated
Broadcom NetXtreme II 5709 DualPort Ethernet
Broadcom NetXtreme Gigabit
Ethernet
Integrated
3-3-4-8
N/A
Principled Technologies, Inc.: Real-world server consolidation: 35 HP ProLiant DL385 servers
onto 5 Dell PowerEdge M610 blade servers running Hyper-V
27
Servers
Type
Blade pass-through modules
5 Dell PowerEdge M610
Discrete Mezzanine Card
Four Gigabit Ethernet PassThrough Modules
35 HP ProLiant DL385
N/A
N/A
USB ports
Number
Type
2 on the blade, 4 on the chassis (2
on the iKVM)
USB 2.0
3
USB 1.1
Figure 8: Detailed system configuration information for the two test configurations.
Principled Technologies, Inc.: Real-world server consolidation: 35 HP ProLiant DL385 servers
onto 5 Dell PowerEdge M610 blade servers running Hyper-V
28
Appendix B – Test storage information
Figure 9 presents detailed information for the test storage we used in this report.
Enclosure
General dimension information
Height (inches)
Width (inches)
Depth (inches)
U size in server rack
Power supplies
Total number
Wattage of each
Cooling fans
Total number
Dimensions (H x W) of each
Voltage (V)
Amps (A)
Hard disk
Vendor and model number
Number of disks in system
Size (GB)
Buffer Size (MB)
RPM
Type
Storage connectivity (Fibre Channel,
SAS, SATA, iSCSI, SCSI)
Firmware revision
Storage cache (GB)
Number of storage controllers
Number of storage ports
Maximum Network bandwidth of
storage connectivity to host
Switch type/model/firmware revision
5 Dell EqualLogic PS5000XV
18 HP StorageWorks MSA30
5.25
19.0
21.5
3
5.20
19.0
16.5
3
2
450
2
499
4
4” x 4”
12.0
1.8
2
4” X 4”
12.5
2.0
16
146
16
15,000
SAS
HP BD14685A26
HP BD1468856B
HP BD14686225
HP BD146863B3
14
146
8
10,000
SCSI U320
iSCSI
SCSI U320
v4.1.4
1
2 (1 active)
3 per controller
N/A
N/A
1
2
1 Gb x 3
N/A
Dell PowerConnect 6248
v2.2.0.3
N/A
Seagate ST3146855SS
Maxtor 8k147S0087511
Figure 9: Detailed system configuration information for the test storage.
Principled Technologies, Inc.: Real-world server consolidation: 35 HP ProLiant DL385 servers
onto 5 Dell PowerEdge M610 blade servers running Hyper-V
29
Appendix C – Payback period calculation
In calculating the estimated payback period, we assume that investment costs occur at the beginning of the year
and we spread annual costs evenly throughout the year. Dell provided the components of the Dell PowerEdge
M610 solution and list price data for that solution. The investment costs include the following:
•
•
•
Intel Xeon Processor X5570-based Dell PowerEdge M610 blade server with 72 GB of memory and two
four-core processors. The list price for this server is $12,955.00.
Dell EqualLogic PS5000XV storage with 16 450GB 15,000 RPM disks. The list price of the storage array
is $43,001. We assign three-fourths ($32,251) of the storage array acquisition costs to this solution based
on the assumption that the array is a resource that can be shared with other workloads.
Migration costs. We assume a migration cost of $667, based on our estimate of 12 person hours required
for planning and migration tasks at a burdened hourly rate of $55.61.
Figure 10 shows the payback calculation for the 5 Dell PowerEdge M610 solutions and the 35 HP ProLiant DL385
solutions.
5 Dell PowerEdge
M610 solutions
One-time initial investment (includes server,
storage hardware, and migration costs)
Monthly cost (annual costs divided by 12;
includes hardware support, maintenance and
service contracts, software licenses and support
agreements, facility costs, energy costs, and
management costs)
Payback period
$229,559
$5,431
35 HP ProLiant
DL385 solutions
$0.00
$28,649
Difference
$229,559
(HP ProLiant
DL385 is lower)
$23,218
(Dell PowerEdge
M610 is lower)
9.9 months ($229,559/$23,218
=9.9)
Figure 10: Payback calculation for the 5 Dell PowerEdge M610 solutions and the 35 HP ProLiant DL385 solutions.
The server and storage hardware costs for the 5 Dell PowerEdge M610 solutions included the list price of the Dell
PowerEdge M610 blade servers and the list price of the Dell EqualLogic PS5000XV storage solution multiplied by
the 75 percent of that solution that the test databases require. For the 35 HP ProLiant DL385 solutions, these
costs included the replacement costs for failed disks. We made the following assumptions in estimating these
costs:
•
•
•
•
The enterprise would dedicate the entire newer Dell PowerEdge M610 blade servers to the tasks that the
servers it was replacing carried out; we therefore included the entire cost of the newer Dell PowerEdge
M610 blade servers in our analysis.
Other servers in the data center would share the storage alongside the Dell PowerEdge M610 solutions.
The enterprise would apportion storage costs based on the percentage of the storage solution used.
Balancing the available storage and IOPS, we estimated that one-fourth of the storage capacity would be
available for low-demand storage or for off-hours processing.
The enterprise would dedicate all 35 HP ProLiant DL385 solution servers and 17½ of the storage
enclosures to the tasks our benchmark tests model.
The migration costs included an estimate of staff time to migrate software and data from the HP ProLiant
DL385 solutions to the newer Dell PowerEdge M610 solutions.
We made the following assumptions in estimating the support, maintenance, and service contract costs for
hardware:
•
The enterprise would typically select a 3-year support package with 7-day, 24-hour coverage, and 4-hour
response time for newer hardware, and would take over support internally after those agreements expire.
Therefore, for the Dell PowerEdge M610 solutions, we included the costs for 3 Year ProSupport for IT
Principled Technologies, Inc.: Real-world server consolidation: 35 HP ProLiant DL385 servers
onto 5 Dell PowerEdge M610 blade servers running Hyper-V
30
•
•
and Mission Critical 4HR 7x24 Onsite Pack in our cost estimates and divided those costs by three to
show the year-one cost.
For the HP ProLiant DL385 solutions, we estimated an annual in-house support cost of $400 per server
and $300 per storage array, plus costs to replace failed disks based on PT estimates of a 4 percent
annual disk replacement rate.
Support agreements, the costs for which we include in the next category, would cover replacement of
failed disks for the Dell PowerEdge M610 solutions.
We made the following assumptions in estimating the costs of software licenses and software support
agreements:
•
•
•
•
•
The HP ProLiant DL385 solutions used Windows Server 2003 R2 Enterprise Edition x64 with per-server
licenses and SQL Server 2005 Enterprise Edition with per-processor licenses, and the enterprise kept upto-date software assurance agreements for that software.
The newer Dell PowerEdge M610 solutions updated the host operating systems licenses to Windows
Server 2008 R2 Datacenter Edition licenses by upgrading five of the licenses used by the HP ProLiant
DL385 solutions to datacenter edition per-processor licenses.
The newer Dell PowerEdge M610 solutions used existing per-processor licenses for SQL Server 2005,
and support agreements for unused licenses were discontinued.
The enterprise used their Software Assurance new version rights to upgrade the licenses they needed for
the newer solutions. They saved the cost of the remaining software assurance agreements. These
agreements were due for renewal at the time of the solution acquisition.
The enterprise already owned the necessary CALS and maintained software assurance agreements for
them; thus, they fell outside the scope of the solutions.
We made the following assumptions in estimating facility costs, which include rack-based space costs and
Ethernet port costs:
•
•
•
We estimated a fixed data center cost per rack, prorated by the percentage of the rack the solutions used.
The older HP ProLiant DL385 solutions each require 2u for each of the 35 servers and 3u for each of the
18 storage arrays plus an additional 1u for a KVM, for a total of 125u, filling 1u less than three total racks;
By contrast, each of the 5 Dell M610 solution storage arrays requires 3u and the single blade server
enclosure uses 10u for a total of 25u, filling just over ½ rack, just 1/5th of the rack space of the older
solution.
We estimated an annual per-server cost for data center ports of $129 based on PT experience.
We included costs for Dell PowerConnect 6248, 48 GbE Ports, and managed switches for the Dell
PowerEdge M610 solution.
We made the following assumptions in estimating energy costs:
•
•
•
•
We calculated energy costs for active and idle power consumption values (shown in Figure 7).
The server and storage array have an average year-round power utilization that is the average of the
active and idle power from the benchmark runs, adjusted for the percentage of the storage array the
solution uses.
Average cost per kWh of power. Source: National commercial average for June 2009 from
http://www.eia.doe.gov/cneaf/electricity/epm/table5_6_a.html. The December average was $10.51.
We estimated the energy cost of cooling the servers and storage arrays to be equal to the energy cost
required for running the solutions. The total energy cost was therefore twice the cost of running the
solution.
We used the following assumptions in estimating the management costs:
•
•
Each server administrator managed 40 servers.
We included an additional cost per storage array for storage array management. These costs were in
addition to the support costs under support, maintenance, and service contract costs for hardware.
Principled Technologies, Inc.: Real-world server consolidation: 35 HP ProLiant DL385 servers
onto 5 Dell PowerEdge M610 blade servers running Hyper-V
31
About Principled Technologies
We provide industry-leading technology assessment and fact-based marketing services. We bring to every
assignment extensive experience with and expertise in all aspects of technology testing and analysis, from
researching new technologies, to developing new methodologies, to testing with existing and new tools.
When the assessment is complete, we know how to present the results to a broad range of target audiences. We
provide our clients with the materials they need, from market-focused data to use in their own collateral to custom
sales aids, such as test reports, performance assessments, and white papers. Every document reflects the results
of our trusted independent analysis.
We provide customized services that focus on our clients’ individual requirements. Whether the technology
involves hardware, software, Web sites, or services, we offer the experience, expertise, and tools to help you
assess how it will fare against its competition, its performance, whether it’s ready to go to market, and its quality
and reliability.
Our founders, Mark L. Van Name and Bill Catchings, have worked together in technology assessment for over 20
years. As journalists, they published over a thousand articles on a wide array of technology subjects. They
created and led the Ziff-Davis Benchmark Operation, which developed such industry-standard benchmarks as Ziff
Davis Media’s Winstone and WebBench. They founded and led eTesting Labs, and after the acquisition of that
company by Lionbridge Technologies were the head and CTO of VeriTest.
Principled Technologies, Inc.
1007 Slater Rd., Suite 250
Durham, NC 27703
www.principledtechnologies.com
info@principledtechnologies.com
Principled Technologies is a registered trademark of Principled Technologies, Inc.
All other product names are the trademarks of their respective owners.
Disclaimer of Warranties; Limitation of Liability:
PRINCIPLED TECHNOLOGIES, INC. HAS MADE REASONABLE EFFORTS TO ENSURE THE ACCURACY AND VALIDITY OF ITS
TESTING, HOWEVER, PRINCIPLED TECHNOLOGIES, INC. SPECIFICALLY DISCLAIMS ANY WARRANTY, EXPRESSED OR
IMPLIED, RELATING TO THE TEST RESULTS AND ANALYSIS, THEIR ACCURACY, COMPLETENESS OR QUALITY, INCLUDING
ANY IMPLIED WARRANTY OF FITNESS FOR ANY PARTICULAR PURPOSE. ALL PERSONS OR ENTITIES RELYING ON THE
RESULTS OF ANY TESTING DO SO AT THEIR OWN RISK, AND AGREE THAT PRINCIPLED TECHNOLOGIES, INC., ITS
EMPLOYEES AND ITS SUBCONTRACTORS SHALL HAVE NO LIABILITY WHATSOEVER FROM ANY CLAIM OF LOSS OR DAMAGE
ON ACCOUNT OF ANY ALLEGED ERROR OR DEFECT IN ANY TESTING PROCEDURE OR RESULT.
IN NO EVENT SHALL PRINCIPLED TECHNOLOGIES, INC. BE LIABLE FOR INDIRECT, SPECIAL, INCIDENTAL, OR
CONSEQUENTIAL DAMAGES IN CONNECTION WITH ITS TESTING, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
IN NO EVENT SHALL PRINCIPLED TECHNOLOGIES, INC.’S LIABILITY, INCLUDING FOR DIRECT DAMAGES, EXCEED THE
AMOUNTS PAID IN CONNECTION WITH PRINCIPLED TECHNOLOGIES, INC.’S TESTING. CUSTOMER’S SOLE AND EXCLUSIVE
REMEDIES ARE AS SET FORTH HEREIN.
Principled Technologies, Inc.: Real-world server consolidation: 35 HP ProLiant DL385 servers
onto 5 Dell PowerEdge M610 blade servers running Hyper-V
32