zEnterprise System - Network Architecture and Virtualization Overview

zEnterprise System - Network Architecture
and Virtualization Overview
Gus Kassimis – kassimis@us.ibm.com
Alfred B Christensen – alfredch@us.ibm.com
IBM Raleigh, NC
Session 6911
Wednesday 4-Aug-2010 – 1:30 PM to 2:30 PM
IBM Software Group – Enterprise Networking Solutions
zEnterprise System - Network Architecture and Virtualization Overview
Session number:
6911
Date and time:
Wednesday 4-Aug-2010 - 1:30 PM - 2:30 PM
Location:
Room 109 (Hynes Convention Center)
Program:
Communications Infrastructure
Project:
Communications Server
Track:
Architecture, SNA/IP Integration and z/OS Systems Programming
Classification:
Technical
Speaker:
Gus Kassimis, IBM
Alfred B Christensen, IBM
Abstract:
IBM's new zEnterprise System provides the ability to consolidate multi-tier, multi-architecture
workloads. This session will describe the networking architecture for the zEnterprise System,
including an in depth review of the new Intra-Ensemble Data Network (IEDN) that provides the
internal connectivity for this new system and how it relates to existing System z networking
configurations. This session will also describe the IBM zEnterprise Unified Resource Manager
advanced network virtualization functions that provide management of this network, including
the ability to define multiple virtual networks for workload isolation.
Page 2
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
Trademarks, notices, and disclaimers
The following terms are trademarks or registered trademarks of International Business Machines Corporation in the United States or other countries or both:
• Advanced Peer-to-Peer
Networking®
• AIX®
• alphaWorks®
• AnyNet®
• AS/400®
• BladeCenter®
• Candle®
• CICS®
• DataPower®
• DB2 Connect
• DB2®
• DRDA®
• e-business on demand®
• e-business (logo)
• e business(logo)®
• ESCON®
• FICON®
• GDDM®
• GDPS®
• Geographically Dispersed
Parallel Sysplex
• HiperSockets
• HPR Channel Connectivity
• HyperSwap
• i5/OS (logo)
• i5/OS®
• IBM eServer
• IBM (logo)®
• IBM®
• IBM zEnterprise™ System
• IMS
• InfiniBand ®
• IP PrintWay
• IPDS
• iSeries
• LANDP®
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Language Environment®
MQSeries®
MVS
NetView®
OMEGAMON®
Open Power
OpenPower
Operating System/2®
Operating System/400®
OS/2®
OS/390®
OS/400®
Parallel Sysplex®
POWER®
POWER7®
PowerVM
PR/SM
pSeries®
RACF®
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Rational Suite®
Rational®
Redbooks
Redbooks (logo)
Sysplex Timer®
System i5
System p5
System x®
System z®
System z9®
System z10
Tivoli (logo)®
Tivoli®
VTAM®
WebSphere®
xSeries®
z9®
z10 BC
z10 EC
•
•
•
•
•
•
zEnterprise
zSeries®
z/Architecture
z/OS®
z/VM®
z/VSE
* All other products may be
trademarks or registered
trademarks of their
respective companies.
The following terms are trademarks or registered trademarks of International Business Machines Corporation in the United States or other countries or both:
• Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries.
• Cell Broadband Engine is a trademark of Sony Computer Entertainment, Inc. in the United States, other countries, or both and is used under license there from.
• Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.
• Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.
• InfiniBand is a trademark and service mark of the InfiniBand Trade Association.
• Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel
Corporation or its subsidiaries in the United States and other countries.
• UNIX is a registered trademark of The Open Group in the United States and other countries.
• Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.
• ITIL is a registered trademark, and a registered community trademark of the Office of Government Commerce, and is registered in the U.S. Patent and Trademark Office.
• IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency, which is now part of the Office of Government Commerce.
Notes:
• Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any
user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload
processed. Therefore, no assurance can be given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here.
• IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply.
• All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may have
achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions.
• This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be subject to
change without notice. Consult your local IBM business contact for information on the product or services available in your area.
• All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.
• Information about non-IBM products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the
performance, compatibility, or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.
• Prices subject to change without notice. Contact your IBM representative or Business Partner for the most current pricing in your geography.
Refer to www.ibm.com/legal/us for further legal information.
Page 3
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
Agenda
‰ IBM® zEnterprise™ System Overview
‰ IBM® zEnterprise™ System Network
Virtualization and Management Overview
•
•
•
•
•
•
•
zEnterprise Node Physical Infrastructure
Communications within the Ensemble
Network and OSA Types and Attributes
External Network Access
Network Virtualization Management
Provisioning Virtual Networks
Network Access Control and Security
Disclaimer: All statements regarding IBM future direction or intent, including current product plans, are subject to
change or withdrawal without notice and represent goals and objectives only. All information is provided for
informational purposes only, on an “as is” basis, without warranty of any kind.
Page 4
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
IBM® zEnterprise™ System
Overview
Page 5
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
Information Technology Today: Limitations
Information technology today is limited by the technology and architecture
configurations available.
Web Servers
SSL/XML
Appliances
System z
Security/Directory
Servers
Application
Servers
Routers
Switches
Firewall
Servers
Caching
Appliances
DS Servers
Business Intelligence
Servers
File/Print
Servers
LAN Servers
ƒ Business processes and the applications that support them are becoming more service oriented,
modular in their construction, and integrated.
ƒ The components of these services are implemented on a variety of architectures and hosted on
heterogeneous IT infrastructures.
ƒ Approaches to managing these infrastructures along the lines of platform architecture boundaries cannot
optimize: alignment of IT with business objectives; responsiveness to change; resource utilization;
business resiliency; or overall cost of ownership.
ƒ Customers need better approach: The ability to manage the IT infrastructure and Business
Application as an integrated whole.
Page 6
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
IBM zEnterprise System – Best in Class Systems and Software Technologies
A system of systems that unifies IT for predictable service delivery
Unified
Unified management
management for
for aa smarter
smarter system:
system:
zEnterprise
zEnterprise Unified
Unified Resource
Resource Manager
Manager
ƒ Unifies management of resources,
extending IBM System z® qualities of
service end-to-end across workloads
ƒ Provides platform, hardware and workload
management
The
The world’s
world’s fastest
fastest and
and
most
most scalable
scalable system:
system:
™ 196
IBM
zEnterprise
IBM zEnterprise™
196
(z196)
(z196)
ƒ Ideal for large scale
data and transaction
serving and mission
critical applications
ƒ Most efficient platform
for Large-scale Linux®
consolidation
ƒ Leveraging a large
portfolio of z/OS® and
Linux on System z
applications
ƒ Capable of massive
scale up, over 50 Billion
Instructions per Second
(BIPS)
1
Page 7
Scale
Scale out
out to
to aa trillion
trillion
instructions
per
instructions per second:
second:
IBM
zEnterprise
IBM zEnterprise
BladeCenter
BladeCenter®® Extension
Extension
(zBX)
(zBX)
ƒ Selected IBM POWER7®
blades and IBM System x®
Blades1 for tens of
thousands of AIX® and
Linux applications
ƒ High performance
optimizers and appliances
to accelerate time to
insight and reduce cost
HMC
ƒ Dedicated high
performance private
network
All statements regarding IBM future direction and intent are subject to change or withdrawal without notice,
and represents goals and objectives only.
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
Operating System Support for zEnterprise System
ƒ Currency is key to operating system support and
exploitation of future servers
ƒ The following are the minimum operating systems
planned to run on z196:
– z/OS
• z196: z/OS V1.91 for toleration only; exploitation starts with z/OS
V1.10 with full exploitation with z/OS V1.12
• Ensemble support: z/OS V1.10
– Linux on System z distributions:
• Novell SUSE SLES 10 and SLES 11
• Red Hat RHEL
5
– z/VM
• z196: z/VM V5.4 or higher
• Ensemble support: z/VM V6.1
– z/VSE V4.1 or higher
– z/TPF V1.1 or higher
ƒ Using the general purpose blades:
– AIX 5.3, 6.1
– Linux on System x2 (SOD)
1
z/OS V1.9 support ends on Sept. 30, 2010. Lifecycle Extension for z/OS 1.9 is available Oct. 1, 2010. Note that z/OS 1.8 with the Lifecycle Extension for z/OS 1.8 and z/OS 1.7 with
the Lifecycle Extension for z/OS 1.7 are also available with toleration support only.
2
All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represents goals and objectives only.
Page 8
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
… and the Value Extends To Heterogeneous Platforms …
IBM zEnterprise BladeCenter Extension (zBX)
Machine Type: 2458 – Model 002
ƒ Integrated IBM Certified Components driven by
System z order
– Standard parts – TOR switch, BladeCenter Chassis,
Power Distribution Units, Optional Acoustic Panels
ƒ System z support
– Problem reporting, hardware and firmware updates
Optimizers
• IBM Smart Analytics
Optimizer
• WebSphere® DataPower®
appliance1
Select IBM Blades
• BladeCenter PS701 Express
• System x1
ƒ Expanding operating system support for zEnterprise
– AIX, Linux on System x1
One to four – 42u racks –
capacity for 112 blades
ƒ Simplified management
– Improved time to install and implement new
applications
– Central point of management for heterogeneous
workloads
– No change to applications
No System z software running
in zBX – Passport Advantage
software licensed to blades
… managed by the
zEnterprise Unified Resource Manager
1
Page 9
No MIPS/MSU rating
Configured for high availability
Optional rear door heat
exchanger
All statements regarding IBM future direction and intent are subject to change or withdrawal without notice,
and represents goals and objectives only.
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
zBX … Infrastructure to Support More Resources
ƒ zBX houses the multiplatform solutions key to the zEnterprise System.
– Optimizers that are dedicated to workloads.
• IBM Smart Analytics Optimizer and WebSphere DataPower appliance1
• Closed environments with hardware and software included in solution
• Individualized tools for sizing and customizing – dependant on the optimizer
– Select IBM POWER7 and System x1 blades – running any application supported by the
operating system installed on the blade – with no change.
– Mix and match Optimizer and select general purpose POWER7 and System x blades in the
same rack.
– zBX is a System z machine type for integrated fulfillment, maintenance, and support
ƒ Secure network connection between zBX and z196 for data and support.
– Fast 10 Gb Ethernet connection to the data
– Less latency – fewer ‘hops’ to get to the data
– Private, isolated network - potential to eliminate
requirement for encryption / firewalls
– Traffic on user networks not affected.
ƒ Sharing of resources – up to eight z196 servers
can attach to the zBX and have access
to solutions
ƒ Configuration, support, monitoring,
management – all by Unified Resource Manager
1
Page 10
HMC
All statements regarding IBM future direction and intent are subject to change or withdrawal without notice,
and represents goals and objectives only.
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
IBM POWER7 and System x1 Blades
General purpose processors under one management umbrella
What is it?
The zBX infrastructure can host select IBM
POWER7 and System x blades. Each blade
comes with an installed hypervisor that offers the
possibility of running an application that spans
z/OS, Linux on System z, AIX on POWER®, or
Linux on System x (SOD) 1 but have it under a
single management umbrella.
How is it different?
ƒ Complete management: Advanced
management brings operational control and
cost benefits, improved security, workload
management based on goals and policies.
ƒ Virtualized and Optimized: Virtualization
means fewer resources are required to meet
peak demands with optimized interconnection.
ƒ Integrated: Integration with System z brings
heterogeneous resources together that can be
managed as one.
ƒ Transparency: Applications certified to run on
AIX 5.3 or 6.1 will also be certified and run on
the POWER7 blade. No changes to deployed
guest images.
ƒ More applications: Brings larger application
portfolio to System z.
1
Page 11
All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represents goals and objectives only.
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
IBM Smart Analytics Optimizer
Capitalizing on breakthrough technologies to accelerate business analytics
What is it?
The IBM Smart Analytics Optimizer is a workload
optimized, appliance-like, add-on, that enables
the integration of business insights into
operational processes to drive winning
strategies. It accelerates select queries, with
unprecedented response times.
How is it different?
ƒ Performance: Unprecedented response
times to enable 'train of thought' analyses
frequently blocked by poor query
performance.
ƒ Integration: Connects to DB2® through
deep integration providing transparency to
all applications.
ƒ Self-managed workloads: Queries are
executed in the most efficient way.
ƒ Transparency: Applications connected to
DB2, are entirely unaware of IBM Smart
Analytics Optimizer.
ƒ Simplified administration: Appliance-like
hands-free operations, eliminating many
database tuning tasks.
Faster insights for enabling new opportunities
Page 12
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
WebSphere DataPower1 Appliance in the zBX
Purpose-built hardware for simplified deployment and hardened security
How is it different?
What is it?
The IBM WebSphere DataPower appliance (SOD)1
integrated in the zEnterprise System, can help
simplify, govern, and enhance the security of XML
and IT services by providing connectivity, gateway
functions, data transformation, protocol bridging,
and intelligent load distribution.
ƒ Security: VLAN support provides enforced
isolation of network traffic with secure private
networks. And integration with RACF®
security.
ƒ Improved support: Monitoring of hardware
with “call home” for current/expected
problems and support by System z Service
Support Representative.
ƒ System z packaging: Increased quality with
pre-testing of blade and zBX. Upgrade history
available to ease growth. Guided placement
of blades to optimize.
ƒ Operational controls: Monitoring rolled into
System z environment from single console.
Time synchronization with System z.
Consistent change management with Unified
Resource Manager.
1
Page 13
All statements regarding IBM future direction and intent are subject to change or withdrawal without notice,
and represents goals and objectives only.
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
Management Stack
Building an architectural construct of hardware, software, services
Service
Management
Platform
Management
Hardware
Management
Page 14
ƒ Visibility, Control and Automation for Applications, Transactions, Databases and
Data Center Resources
ƒ End-to End Workload Management and Service Level Objectives that Align IT
Management with Business Goals
ƒ Common Usage and Accounting for business accounting
ƒ Dynamic/Centralized Management of Application Workloads based on Policies
ƒ Business Resilience for multi-site recovery
ƒ End-to-end Enterprise Security
ƒ Workload based Resource Allocation and
Provisioning for zEnterprise
ƒ Physical and Virtual Resource
Management (Server, Storage, Network)
ƒ Goal Oriented Resource Management of
zEnterprise (Availability, Performance,
Energy, Security)
ƒ Ensemble Network and Storage
Management
ƒ Configuration management for
hardware / firmware
ƒ Operational controls for the
hardware / firmware
ƒ Service and Support for the
hardware / firmware
ƒ Lifecycle management for the
platform’s virtual resources
Extending with
Unified Resource Manager
ƒ Hypervisor management and creation of
virtual networks
ƒ Operational controls, service and support for
hardware / firmware
ƒ Network management of private and secure
data and support networks
ƒ Energy monitoring and management
ƒ Workload awareness and platform
performance management
ƒ Virtualization management – single view of
virtualization across the platform
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
zEnterprise extends Service Management for improved governance
Service
Management
APP
APP
Tivoli
MIDDLEWARE
APP
MULTIPLE OPERATING SYSTEMS
e.g., z/OS, z/TPF, z/VSE, z/VM,
Linux on System z
Platform
Management
APP
APP
APP
AIX
Linux on System
x1
VIRTUALIZATION – PR/SM, z/VM, PowerVM, System x Hypervisor
Unified Resource Manager
and System Director
FIRMWARE
Hardware
Management
System z
Unified Resource Manager
and System Director
Power
System x1
IBM Optimizers
Unified Resource Manager
Focused, collaborative innovation
A “complete systems” approach
1
Page 15
All statements regarding IBM future direction and intent are subject to change or withdrawal without notice,
and represents goals and objectives only.
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
zEnterprise
hardware
management
andResource
platform management
…
…
Value Made
Possible
By the Unified
Manager
Hypervisor
Simplified
installation
SimplifiedManagement
installation
Energy Management
▀ofIntegrated
deployment and
of hypervisors
hypervisors
configuration of hypervisors
▀ Hypervisors (except z/VM) shipped and
Gain
significant
time to
Gain
significant
serviced
as firmware.time to
market
with
improved
withofimproved
▀market
Management
ISO images.
speed
of
deployment
▀speed
Creationof
of deployment
virtual networks.
▀ Manage and control communication
between virtual server operating
systems and the hypervisor.
Simplified
energy
management
▀ Monitoring and
trend reporting
of CPU
energy efficiency.
▀ Ability tocost
querysavings
maximum potential power.
Energy
▀ Static power savings.
Hypervisors
Energy
Operational
Controls
Save
time, cost
and simplify
▀
Auto-discovery
and
asset managementconfiguration
support for new resources.
Operations
Decrease
problem
determination
▀ Cross platform
hardware
problem
and
resolution
time
for
crossdetection,
reporting
and
call
home.
platform
resources
▀ Physical hardware configuration,
Networks
backup and restore.
Improve and simplify cross▀ Delivery of system activity using new
platform
availability procedures
user.
Enable broader and more
granular view of resource
consumption
Performance
Virtual
Servers
HMC
Network
Factory Management
installed and configured network
▀ Management of virtual networks including access control
Improved network security with lower
latency, less complexity, no
Key
encryption/decryption
▀ Manage suite
Page 16
▀ Automate suite
Workload Awareness and
Platform
Performance
Allow
critical
workloads to
receive
resources and
Management
priority
based onmanagement
goal▀ Wizard-driven
of
oriented
policies
established
resources in accordance with
by business
requirements
specified business
service level
objectives
Smart
business
▀ HMC
provides adjustments
a single
consolidated
and consistent
based
on workload
insight
view of resources
▀ Monitor
resource
useinto
within the
Provide
deep
insight
context
of
a
business
workload
how IT resources are being
▀ Define workloads and
used
associated performance
policies
Gain
flexibility,
consistency
and
Virtual
Server Lifecycle
Management
uniformity of virtualization
▀ Single view of virtualization across platforms.
▀ Ability tothe
deploy
multiple, cross-platform
Provide
business
with fastervirtual
time
servers
within
minutes
to market
▀ Management of virtual networks including
access control
Simplified network management for
applications
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
IBM® zEnterprise™ System
Network Virtualization and Management Overview
Page 17
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
Agenda – IBM zEnterprise System Networking Overview
9 zEnterprise Node Physical Infrastructure
9 Communications within the Ensemble
9 Network and OSA Types and Attributes
9 External Network Access
9 Network Virtualization Management
9 Provisioning Virtual Networks
9 Network Access Control and Security
Notices:
1.
All statements regarding IBM future direction and intent
are subject to change or withdrawal without notice, and
represents goals and objectives only.
2.
The zEnterprise internal networks are provided with redundant
hardware – redundancy is NOT shown in this presentation
Page 18
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
zEnterprise Networking Value Points
‰Network Simplification
9Single physical network and zBX “package” (physical network integration)
9Central point of Management (zManager via the HMC/SE)
‰Secure communications
9Physical security (internal / dedicated network equipment)
9Logical security (controlled access)
9Network Virtualization and Isolation
‰High Availability
9Redundant Network
Hardware
9Logical failover
‰Unique System z QoS
9Isolated / dedicated
equipment
9Special purpose dedicated
data network & OSA-Express
(no encryption required)
Page 19
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
IBM zEnterprise System Overview
zEnterprise Node
Power Blades
xHypervisor
xHypervisor
xHypervisor
pHypervisor
Future
xHypervisor
xHypervisor
PR/SM
Smart Analytics Optimizer
z/VM
Optimizers
DataPower
x/Linux
x/Linux
AIX
x/Linux
x/Linux
AIX
x/Linux
x/Linux
AIX
x/Linux
x/Linux
Linux on System x
x/Linux
x/Linux
Linux on System x
x/Linux
x/Linux
Linux on System x
Linux on System z
Linux on System z
Linux on System z
z/OS
z/OS
z/OS
z/OS
System z Hardware Management Console
With Unified Resource Manager
X86 Blades
Z CPU, Memory and IO
System Element (SE)
Blade Center Advanced Management Module (AMM)
zBX
z196
Connecting the pieces with zManager (aka. Unified Resource Manager)!
1
Page 20
All statements regarding IBM future direction and intent are subject to change or
withdrawal without notice, and represents goals and objectives only.
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
IBM zEnterprise node with internal networks
Customer
Customer
managed
managed
management
management
network
network
zEnterprise Node
HMC
OSM
Intra
IntraNode
Node
Management
Management
Network
Network
(INMN)
(INMN)
TOR
TOR
TOR
TOR
TOR
TOR
TOR
TOR
z196
OSX
OSD
Customer
Customer
managed
managed
data
data
networks
networks
Page 21
Intra
Intra
Ensemble
Ensemble
Data
DataNetwork
Network
(IEDN)
(IEDN)
zBX
zBX
BladeCenter
chassis
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
IBM zEnterprise Node with Internal Networks
zEnterprise Ensemble
A collection of one or more
zEnterprise Nodes (including any
optionally attached zBX) that are
managed as a single logical
virtualized system by the zManager
using a Hardware Management
Console (HMC).
zEnterprise Node 1
zEnterprise
Node 2
zEnterprise
Node 3
Ensemble Member
Intra Ensemble Data Network (IEDN)
Page 22
A zEnterprise node that
has been added to an
ensemble using the HMC.
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
IBM zEnterprise Node with Internal Networks
zEnterprise Ensemble
zEnterprise Node 1
zEnterprise
Node 2
Ensemble
Member
Intra Ensemble Data Network (IEDN)
zEnterprise
Node 3
Intra Ensemble Data Network - key attributes:
1.
2.
3.
4.
5.
6.
7.
8.
Page 23
Single dedicated physical / flat layer 2 10GbE network
Comprised of IBM zEnterprise (redundant) equipment (no external / customer hardware)
Can span nodes (i.e. can be shared by all co-located nodes within the Ensemble - 10km
limit)
No layer 3 IP Routing required to communicate within the Ensemble
IP addresses (IPv4 or IPv6) are customer controlled (provisioned)
MAC addresses (prefixes) are provisioned / coordinated by zManager (HMC)
Access to the network is controlled by the zManager (HMC) via SE via OSX, hypervisors
and physical switches
Virtual servers can be isolated into multiple groups on the physical network by defining
multiple virtual networks (multiple VLANs) based on workloads and other isolation
requirements
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
IBM zEnterprise – OSA and Network Types
HMC
zEnterprise node
LP1
z/OS
LP2
z/OS
LP3
z/OS
LP5 z/VM
LP4
z/OS
VS1
VS2
VS3
Customer
external
data
network
access
OSA OSD
VS4
z/VM virtual
switch
SE
OSD
OSD
Firewall
F
I
R
E
W
A
L
L
OSX
OSX
OSM
OSM
BC1
Customer
Customer
external
external
data
data
network
network
BC2
z196
BC3
TOR
TOR
TOR
TOR
ESM
ESM
ESM
ESM
ESM
ESM
ESM
ESM
ESM
ESM
ESM
ESM
zBX
IEDN may
extend to other
zEnterprise
nodes within an
ensemble
Page 24
Intra
Ensemble
Data Network
(IEDN) OSA
OSX 10 GbE
Intra Node
Management
Network
(INMN) OSA
OSM 1 GbE
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
External Network Access – Option 1 – System z (LP) IP Router
HMC
zEnterprise node
Route via
OSD and
one or
more z/OS
images
LP1
z/OS
LP2
z/OS
LP3
z/OS
LP5 z/VM
LP4
z/OS
VS1
IP
Router
VS2
VS3
VS4
z/VM virtual
switch
SE
Firewall
OSD
OSD
F
I
R
E
W
A
L
L
OSX
OSX
OSM
OSM
BC1
Customer
Customer
external
external
data
data
network
network
BC2
z196
BC3
TOR
TOR
TOR
TOR
ESM
ESM
ESM
ESM
ESM
ESM
ESM
ESM
ESM
ESM
ESM
ESM
zBX
Page 25
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
External Network Access – Option 2 – External IP Router
HMC
zEnterprise node
And to
System z
LPARs
LP1
z/OS
LP2
z/OS
LP3
z/OS
LP5 z/VM
LP4
z/OS
VS1
VS2
VS3
VS4
z/VM virtual
switch
SE
Firewall
OSD
OSD
F
I
R
E
W
A
L
L
OSX
OSX
OSM
OSM
BC1
Customer
Customer
external
external
data
data
network
network
BC2
z196
BC3
TOR
TOR
TOR
TOR
ESM
ESM
ESM
ESM
ESM
ESM
ESM
ESM
ESM
ESM
ESM
ESM
zBX
Extend IEDN
to external
router and
route via
TOR
Page 26
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
Virtual Network Concepts – Creating Virtual Networks
Step 1. Create / Define a
Virtual Network
Network Name =
Production Net
VLAN ID = 300
Production Net
Sales Production Net
… from the
zManager (HMC)
Page 27
HMC
300
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
Virtual Network Concepts – Adding Virtual Servers
… once you have a Virtual
Network…
Step 2. …as necessary …add (associate /
authorize) Virtual Servers to the Virtual Network
“Production Net”
VLAN ID = 300
Server ID
Virtual Networks consist of
two key properties:
1.
2.
Page 28
VLAN ID (IP subnet)
List of Authorized Servers
Add Hosts to Virtual Network…
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
Deploying a Virtual Network – Example 1
Server A
IP A
single Virtual Network
subnet and VLAN ID
Server B
IP B
Server C
Server D
IP C
IP D
Single IP
Production
Net
TOR Switch
(VLAN ID = 300”)
IP E
Server E
IP F
IP G
Server G
Server F
All servers can have a
single IP interface and
all IP addresses are
from the same IP subnet
IP I
IP J
IP K
IP L
Server I
Server J
Server K
Server L
IP H
Server H
(e.g. 10.1.200.xxxx)
Multiple Interfaces are created
for redundancy!
Page 29
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
Deploying Multiple Virtual Networks – Example 2 - Isolation
Server A
IP A
Server B
IP B
Server C
Server D
IP C
IP D
1. Define Multiple Virtual
Networks
IP E
IP F
Server E
… each having unique VLAN
IDs and IP subnets
“Production Network”
VLAN ID 300 TOR Switch
“Development Network”
VLAN ID 500
Server G
Server F
2. Then add virtual
servers to each
virtual network as
needed…
IP G
IP I
IP J
IP K
IP L
Server I
Server J
Server K
Server L
IP H
Server H
…which isolates “Production Servers” from “Development Servers”
Page 30
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
Deploying Multiple Virtual Networks – Isolation
Server A
Server B
Server C
Server D
IP@ A.1
IP@ A.2
IP@ A.3
IP@ A.4
1. Define Multiple
Virtual Networks
Server E
Server F
IP@ A.5
IP@ A.6
2. Then add virtual
servers to each
virtual network
as needed…
… each having unique VLAN
IDs and IP subnets
“Marketing Network”
VLAN ID 300 TOR Switch
(subnet “A”)
“Development Network”
VLAN ID 500
(subnet “B”)
Server I
Server J
Server K
Server L
IP@ B.3
IP@ B.4
IP@ B.5
IP@ B.6
Server G
Server H
IP@ B.1
IP@ B.2
… zManager isolates “Marketing Servers” from “Development Servers”
Page 31
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
zEnterprise Virtualization and Network Access Control
VS 1
VS 2
VS 3
z/VM
VS 4
VS 5
VS 6
VS 7
VS 8
VSwitch 2
PR/SM
P
SE
VSwitch 1
Net A
OSX
BLADE 1
TOR
ESM
zBX
HMC
Page 32
Net B
BC
Net C
BLADE 2
VS VS
VS
VS
VS
VS
9
11
12
13
14
10
pHype
xHype
zManager pushes virtual network access control information to the
node and the SE propagates to control points (OSX and Hypervisors)
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
Putting It All Together…with secure access control !
Server
Server Server Server
A (G0)
B
C
(G1)
(G2)
System z
Blade Center
Chassis
D
Optimizer A
Server Server Server
E
F
G
ISAOpt
(Server ID Y)
z/OS
pHype VSwitch C
zVM VSwitch B
NVM configures specific VLANs
Blade B
Blade A
Image
VSwitch A
Image
OSX OSA
OSA Port 0
External Ports
TOR A
Internal Ports
External Ports
TOR (A) Port 0
IEDN Core Physical Network
ESM A
ESM (A)
Blade A
Port 0
Port 2
Port 1
Internal Ports
Blade B
Port 2
Management Port
Management Port
Configure (allow) all VLANs
IEDN Physical Edge
IEDN Physical Edge
zBX
(System z side)
(blade side)
HMC
BPH Ports
Note that all network components are duplicated to provide
full redundancy.. redundancy is not shown
Page 33
SE
zEnterprise zManager controls
network access at the physical and
at the virtual switches (hypervisors) !
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
Use of multiple VLANs on the IEDN – “application layer gateways” between VLANs
Virtual servers that act
as “application layer
gateways” have, under
zManager control,
access to two VLANs
VLAN3
zEnterprise node
LP1
z/OS
LP2
z/OS
LP3
z/OS
LP5 z/VM
LP4
z/OS
VS1
z/OS IP Filtering
and routing
OSD
OSD
VLAN1
TOR
TOR
Customer
Customer
external
These nodes “should”
external
data
data
be configured with
network
network
forwarding disabled.
z/OS is by default
prevented from routing
between IEDN VLANs.
Page 34
VS2
VS3
VS4
z/VM virtual
switch
OSX
OSX
ESM
ESM
SE
OSM
OSM
“Application
layer BC1
gateway”
ESM
ESM
HMC
z196
“Application
BC2 layer gateway”
BC3
ESM
ESM
ESM
ESM
ESM
ESM
TOR
TOR
ESM
ESM
zBX
VLAN2
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
Use of multiple VLANs on the IEDN – Routing between VLANs using an
external firewall
VLAN1
HMC
zEnterprise node
TOR, under
zManager control,
extends the two
VLANs to an
external firewall
LP1
z/OS
LP2
z/OS
OSX
OSX
VS1
VS2
VS3
VS4
SE
OSM
OSM
BC1
TOR may
optionally also
implement MAC
filters
Page 35
LP5 z/VM
LP4
z/OS
z/VM virtual
switch
OSD
OSD
Customer
Customer
external
external
data
data
network
network
LP3
z/OS
BC2
z196
BC3
TOR
TOR
TOR
TOR
ESM
ESM
ESM
ESM
ESM
ESM
ESM
ESM
ESM
ESM
ESM
ESM
VLAN2
zBX
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
Configuring TOR - External Network Access
Two Use Cases:
1. z10 Access
2. External IP
Router
Page 36
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
z/OS Communications Server and OSM connectivity
ƒ z/OS LPAR must participate in the ensemble
– New VTAM start option: ENSEMBLE=YES
• Required for both OSM and OSX connectivity
ƒ LPAR must be IPv6-enabled for OSM connectivity
ƒ Two IPAQENET6 interface definitions are dynamically generated and started
– If OSM CHPIDs are defined to the z/OS image, the two CHPIDs with
the lowest device numbers are assigned to these interfaces
Guest
Platform
Management
Provider
(GPMP)
ƒ TRLEs dynamically generated if connectivity allowed and CHPIDs found
ƒ Only port 0 supported
ƒ IPv6 link-local address only
ƒ Uses VLAN in access mode
– Switch handles VLAN tagging, stack unaware
z/OS TCP/IP
Two IPAQENET6 and TRLEs
dynamically built and started
ƒ Not reported to OMPROUTE
ƒ Cannot add static or dynamic routes
ƒ Supports stop, start, packet trace, NTA
OSM
OSM
ƒ Only applications permitted to EZB.OSM.sysname.tcpname can communicate over OSM interfaces
– The GPMP is the only application that needs to
Page 37
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
How bad is it enabling IPv6?
ƒ Add a NETWORK stmt. for AF_INET6
to your BPXPRMxx PARMLIB member
ƒ No changes needed to your TCP/IP Profile
– Unless you want to exploit and test
specific IPv6 features
FILESYSTYPE TYPE(INET) ENTRYPOINT(EZBPFINI)
NETWORK DOMAINNAME(AF_INET)
DOMAINNUMBER(2)
MAXSOCKETS(2000)
TYPE(INET)
NETWORK DOMAINNAME(AF_INET6)
DOMAINNUMBER(19)
MAXSOCKETS(3000)
TYPE(INET)
ƒ Testing needed in the network management area
– Netstat reports will use the LONG format
– Home-written Netstat “scraping” logic will need to be changed
– Network management products may fail if they are not prepared for IPv6
addresses
MVS TCP/IP NETSTAT CS V1R12
TCPIP Name: TCPCS
User Id Conn
State
------- -------MYINETD1 00000022 Listen
Local Socket:
9.42.130.98..23
Foreign Socket: 0.0.0.0..0
TN3270A 0000004D Establsh
Local Socket:
::ffff:9.42.105.45..23
Foreign Socket: ::ffff:9.76.144.213..4211
Application Data: EZBTNSRV TCPABC80 TSO10001 ET B
TN3270A 0000003F Listen
Local Socket:
::..23
Foreign Socket: ::..0
Application Data: EZBTNSRV LISTENER
Page 38
13:02:52
It’s a piece
of cake
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
z/OS Communications Server and OSX connectivity
ƒ Configure with INTERFACE statement
– IPAQENET and IPAQENET6
ƒ Either specify CHPID
– Dynamically created TRLE similar to HiperSockets
ƒ Or configure TRLE and point to it
– Useful in VM guest LAN environment where CHPID is unpredictable
ƒ Always uses VLAN in trunk mode
– VLANID required and must be authorized at HMC
• If not authorized, OSA activation fails
ƒ Prevents IP forwarding from OSX Ö OSX
– Sysplex distributor forwarding is allowed when using VIPAROUTE
Normal z/OS
Applications and
Subsystems
(CICS, IMS, DB2,
MQ, WAS, etc.)
z/OS TCP/IP
Define IPAQENET or IPAQENET6
Interfaces with VLAN ID that matches
HMC definitions
ƒ Supports stop, start, packet trace, NTA
ƒ To prevent external traffic from being routed to the OSX VLAN
OSX
OSX
– Define OSX as INTERFACE or IPV6_INTERFACE
– Do not enable IMPORT_DIRECT_ROUTES function
– Alternatively, do not define the OSX interfaces to OMPROUTE and tell OMPROUTE to ignore
undefined interfaces
ƒ To allow external traffic to be routed to the OSX VLAN
– Define OSX as OSPF_INTERFACE or IPV6_ISPF_INTERFACE
– Configure a non-0 value for ROUTER_PRIORITY
Page 39
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
Summary - Exploiting the intraensemble data network
Once all hardware / physical installation and System z HCD configuration tasks are
complete… then you are ready to exploit the IEDN:
Key concepts / reminders:
1. All network traffic on the IEDN must use an “authorized” VLAN ID!
2. The VLAN ID maps to a corresponding Virtual Network
3. All host images (Operating Systems) on all platforms within the Ensemble are represented as a Virtual
Server
Key zManager network related configuration tasks:
1. Virtual Network Configuration (at the HMC) consist of:
– defining a virtual network (VLAN ID)
2. Virtual Server configuration:
– Define each virtual server
– Associate each virtual server with the proper Virtual network
Hypervisors
Operations
Networks
Energy
Performance
Virtual
Servers
3. Virtual Switch configuration (if applicable – N/A to native LPs)
Finally - Operating System network configuration tasks (IP address, VLAN ID, etc.) remain within the OS –
the OS VLAN ID must match the HMC VLAN ID configuration
Page 40
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
Questions? - Thank You !
Page 41
© 2010 IBM Corporation
IBM Software Group – Enterprise Networking Solutions
For more information
URL
Content
http://www.twitter.com/IBM_Commserver
IBM Communications Server Twitter Feed
http://www.facebook.com/IBMCommserver
IBM Communications Server Facebook Fan Page
http://www.ibm.com/systems/z/
IBM System z in general
http://www.ibm.com/systems/z/hardware/networking/
IBM Mainframe System z networking
http://www.ibm.com/software/network/commserver/
IBM Software Communications Server products
http://www.ibm.com/software/network/commserver/zos/
IBM z/OS Communications Server
http://www.ibm.com/software/network/commserver/z_lin/
IBM Communications Server for Linux on System z
http://www.ibm.com/software/network/ccl/
IBM Communication Controller for Linux on System z
http://www.ibm.com/software/network/commserver/library/
IBM Communications Server library
http://www.redbooks.ibm.com
ITSO Redbooks
http://www.ibm.com/software/network/commserver/zos/support/
IBM z/OS Communications Server technical Support –
including TechNotes from service
http://www.ibm.com/support/techdocs/atsmastr.nsf/Web/TechDocs
Technical support documentation from Washington
Systems Center (techdocs, flashes, presentations,
white papers, etc.)
http://www.rfc-editor.org/rfcsearch.html
Request For Comments (RFC)
http://www.ibm.com/systems/z/os/zos/bkserv/
IBM z/OS Internet library – PDF files of all z/OS
manuals including Communications Server
For pleasant reading ….
Page 42
© 2010 IBM Corporation