Oracle Cluster Domains
Oracle Grid Infrastructure 12c Release 2
Cluster Domains
| MARCH 2017
Table of Contents
Clustering with Oracle Clusterware 12c Release 2
Oracle Grid Infrastructure 12c Release 2 Architectures
Standalone Cluster
Cluster Domain
Domain Services Cluster
Member Clusters
Domain Services Cluster
Centralized Grid Infrastructure Management Repository
Centralized ASM Storage Management Services
Accessing Centralized Storage
Rapid Home Provisioning Server
Other Services
Member Clusters
Application Member Clusters
Database Member Cluster – configured with local ASM and storage
Database Member Cluster – configured with remote ASM
Database Member Cluster – configured with directly attached shared storage
Database Member Cluster – configured with indirect access to shared storage 7
Benefits of the Cluster Domain Architecture
Cluster Domains have been introduced in Oracle Database 12c Release 2 as a cluster architecture designed to reduce
management overhead by centralizing and consolidating storage management and other common services for groups
of otherwise independent Oracle clusters. While the Standalone Cluster configuration (the only cluster architecture
available prior Oracle Grid Infrastructure 12c Rel 2) is still supported, there are significant benefits to be realized in
moving to Cluster Domains, particularly for larger cluster estates in which the management needs of the many clusters
are ever-increasing.
Oracle’s Cluster Domain architecture enables simpler, easier deployments, reduced storage management effort and
performance gains for I/O operations. Adoption of the new Cluster Domain architecture can be via new installations
through the Oracle Universal Installer, or via upgrades of established clusters.
Clustering with Oracle Clusterware 12c Release 2
Oracle Clusterware enables the clustering of otherwise independent servers so that they co-operate as a single system. As a cluster,
these servers then provide the integrated foundation which Oracle Real Application Cluster (RAC) Databases and user applications can
leverage for high availability and scalability.
The cluster of servers is coordinated via Oracle Clusterware, with cluster resources made available as required in support of the high
availability requirements of the Oracle RAC Databases and applications running on the cluster, on one or more of the clustered servers,
or nodes. Introduced in Oracle 10g Release 1, Oracle Clusterware has evolved and broadened its capabilities to meet the demand for
a more versatile and more capable infrastructure.
With Oracle Grid Infrastructure 12c Release 21, new cluster architectures are available, all based upon the Flex Cluster architecture that
was introduced with Oracle Grid Infrastructure 12c Release 1. Flex Clusters were designed to enable customers to scale their clusters
beyond established norms in support of mixed workloads, with specific support for Oracle RAC Database instances and applications in
the same cluster.
Figure 1: Flex Cluster Architecture
As Oracle Clusters have been accepted and deployed to advantage for a broad range of customer database and application
requirements, the demand has risen to both simplify the management of multi-cluster environments and to consolidate them under a
reduced management framework. This is immediately apparent to customers who have multiple clusters deployed, as the operational
demands have increased with the deployment of each new cluster.
Now, with Oracle Grid Infrastructure 12c Release 2, the customer has options in their cluster deployment, to stay with the previously
available cluster architecture (that has been well proven over the last decade) or to adopt the new multi-cluster architectures of the
Cluster Domain, especially if managing a bigger estate of Oracle Clusters.
1 Oracle Grid Infrastructure was introduced with Oracle 11g Release 1 as a composite of Oracle Clusterware and Oracle Automatic Storage Management (ASM).
Oracle Grid Infrastructure 12c Release 2 Architectures
There are now two architectures for deploying clusters using Oracle Grid Infrastructure. These are the Standalone Cluster and the new
Cluster Domain. The clustering software for these deployments is identical, differing only how they are configured during deployment.
Standalone Cluster
The Standalone Cluster consists of one or more cluster nodes configured with locally
available shared storage, a private interconnect, local instances of Automatic Storage
Management (ASM) for managing that shared storage, and the GI Management Repository
(GIMR) for storing cluster health and diagnostic information.
Figure 2: Standalone Cluster
Cluster Domain
A Cluster Domain is actually a grouping of clusters. A cluster domain consists of a single Domain Services Cluster and a number of
Member Clusters (hosting applications or databases) that utilize services offered on the Domain Services Cluster. Centralized and
consolidated services are hosted by the Domain Services Cluster, and consumed by the Member Clusters that are registered with that
Domain Services Cluster.
Figure 3: Cluster Domain
Domain Services Cluster
The Domain Services Cluster is the heart of the Cluster Domain, as it is configured to provide the services that will be utilized by the
various Member Clusters within the Cluster Domain. As per the name, it is a cluster itself, thus providing the required high availability
and scalability for the provisioned services.
The services available consist of:
» Grid Infrastructure Management Repository (GIMR),
» ASM Storage service,
» IOServer service, and
» Rapid Home Provisioning Server.
Member Clusters
The Member Clusters within a Cluster Domain are deployed to host databases and applications. They can be configured purely for
applications, in which case they lack the support for databases, purely for databases, or for a mix of databases and applications.
Depending upon the centralized services to which they subscribe, they will be configured as one of four constructs:
» Application Member Cluster – cannot support database instances, but runs with a much lighter memory footprint
» Database Member Cluster – configured with local ASM and storage
» Database Member Cluster – configured with directly attached shared storage that is managed through ASM on the Domain
Services Cluster, or
» Database Member Cluster – configured with indirect access to the shared storage managed and presented by ASM on the
Domain Services Cluster.
Database Member Clusters are designed to host databases, but may also host applications or programs that benefit from direct local
access to the local database instances (for example, Oracle GoldenGate). For a DBA managing their database and instances, the
database management on any of the Database Member Cluster constructs will appear to be identical. It is only at the cluster
deployment and administration levels that there are differences.
Domain Services Cluster
At the heart of the Oracle Cluster Domain, the Domain Services Cluster, or DSC, must be deployed first and configured with the
services required for subsequent Member Cluster deployments. Shared storage is attached to the DSC, which is managed by Oracle
ASM and configured for Member Cluster access.
Being a cluster itself, the Domain Services Cluster also has its management repository stored in the centralized Grid Infrastructure
Management Repository hosted on the DSC.
Centralized Grid Infrastructure Management Repository
The GIMR hosted on the Domain Services Cluster is configured as a Multitenant Single-Instance Database (CDB) in which each of the
Member Clusters and the DSC stores its management repository as an individual PDB. Cluster health and diagnostic information is
written to the GIMR by background processes (daemons) on each of the Member Clusters over SQL*Net. Data security and isolation
are carefully maintained by deploying separate PDB’s for each Member Cluster. Accesses to a cluster-specific data for diagnostic and
analysis purposes must be from the cluster itself, again over SQL*Net, against that cluster’s Management Database in the GIMR on the
DSC (the only exception to this is for Oracle Enterprise Manager, as it references this information across the Public Network, much as it
does for any other target).
The centralized GIMR is host to cluster health and diagnostic information for all the clusters in the Cluster Domain. As such, it is
accessed by the client applications of the Autonomous Health Framework (AHF), the Trace File Analyzer (TFA) facility and Rapid Home
Provisioning (RHP) Server across the Cluster Domain. Thus, it acts in support of the DSC’s role as the management hub.
Centralized ASM Storage Management Services
By centralizing the Oracle ASM storage management, Database Member Clusters can be deployed without configuring ASM to run
directly on those clusters. Oracle ASM storage management is consolidated onto the DSC. In addition, the ASM instances on the DSC
will now manage the consolidated storage pool for multiple Member Clusters.
Where previously, with ASM instances configured locally to manage the storage for each cluster, there would be a single, centralized
ASM deployment tasked with managing all the disks in support of the entire Cluster Domain. This increases the pool of shared disks
over which ASM can distribute shared storage for each cluster, resulting in potential I/O performance gains. It also allows ASM to
manage storage changes more efficiently. In addition, ASM’s new Flex Disk Groups feature will maintain the security and isolation of
data and data files on a database basis.
Accessing Centralized Storage
Shared storage service access from the Database Member Clusters may be via direct paths to the disk storage (i.e. the disk storage is
mounted to both the DSC and to the Database Member Cluster) or via an indirect I/O path (without disk storage mounted on the
Database Member Cluster).
Using locally attached shared storage (that is managed from the DSC) has all the benefits of local shared storage without the overhead
of running locally configured ASM instances. So, instead of registering with local ASM instances, the database instances on the
Member Cluster register with the ASM instances on the DSC. The I/O path between the database instances and the locally attached
shared storage would still be direct.
Configuring the Database Member Cluster to use an indirect I/O path to storage is simpler still, requiring no locally configured shared
storage, thus dramatically improving the ease of deploying new clusters, and changing the shared storage for those clusters (adding
disks to the storage is done at the DSC – an invisible operation to the Database Member Cluster). Instead, all database I/O operations
are channeled through the IOServer processes on the DSC. From the database instances on the Member Cluster, the database’s data
files are fully accessible and seen as individual files, exactly as they would be with locally attached shared storage. The real difference
is that the actual I/O operation is handed off to the IOServers on the DSC instead of being processed locally on the nodes of the
Member Cluster. The major benefit of this approach is that new Database Member Clusters don’t need to be configured with locally
attached shared storage, making deployment simpler and easier.
Rapid Home Provisioning Server
The Domain Services Cluster may also be configured to host a Rapid Home Provisioning (RHP) server. RHP is used to manage the
provisioning, patching and upgrading of the Oracle Database and GI software stacks and any other critical software across the Member
Clusters in the Cluster Domain. Through this service, the RHP server would be used to maintain the currency of the installations on the
Member Clusters as RHP clients, thus simplifying and standardizing the deployments across the Cluster Domain.
Other Services
Other services are expected to be added in the near future. These include services for TFA (Trace File Analyzer) for domain-wide
diagnostics and Remote ACFS wherein an ACFS file system is mounted on the Member Cluster as a client of the ASM running on the
Domain Services Cluster. The latter would provide the shared cluster file storage often required beyond just database storage, such as
for flat files, configuration files, program files, etc. in place of NFS-mounted file systems.
Member Clusters
There are four possible configurations currently supported for Member Clusters, each configured during installation.
Application Member Clusters
The distinguishing feature of an Application Member Cluster has a much smaller memory footprint and fewer required IP addresses and
has no pre-configured shared storage (in ASM or otherwise). This enables the use of smaller servers for supporting application-only
This means that the Application Member Cluster is dedicated and
optimized to support only applications or programs that may or may
not access remote databases, but without the overhead required to
support locally configured database instances. For a mission critical
application, by utilizing Oracle Clusterware functionality in defining
and managing resources, the Application Member Cluster is a
simple way to provide an initial layer of high availability and
In addition, the high availability of the applications or programs can
be enhanced with the deployment of XAG Agents, enabling the
customization of high availability capabilities for those applications
running on the Application Member Clusters.
Figure 4: Application Member Cluster
Database Member Cluster – configured with local ASM and storage
This is the a Database Member Cluster on which ASM is
configured to run locally, thus the database accesses only
locally mounted shared storage. In this scenario, only the
Management Database is offloaded to the Domain Services
Cluster and stored in the centralized GIMR. Otherwise, this is
identical to the Oracle Standalone Cluster.
Customer taking their initial steps in the world of the Cluster
Domain may favour this option, since it is a relatively simple
step to move an established Oracle Standalone Cluster into
the Cluster Domain and only offload the Management
Figure 5: Database Member Cluster with local ASM
Database Member Cluster – configured with remote ASM
There are two alternative architectures for Database Member clusters that subscribe to the ASM service on the DSC. One has the
shared storage connected directly to the cluster, while the other has no locally configured shared storage and uses the IOServer
running on the DSC for its I/O.
Database Member Cluster – configured with directly attached shared storage
This configuration takes the concept of the Oracle Standalone Cluster that one step further, by not only offloading the Management
Database, but also offloading the ASM storage management to the DSC. Storage connectivity is still seen as locally mounted, but now
the ASM instances are actually running on the DSC.
The database instances running on this Database Member
Cluster register with the remote ASM instances that provides
disk access information as required for the database files
associated only with that particular database. This preserves
the access paths that would have been available with the
Oracle Standalone Cluster, but removes the dependency
upon a locally available ASM instance. The ASM instances
running on the DSC are still highly available, but now provide
consolidated storage management for all the Member
Clusters that are registered with the DSC.
Figure 6: Database Member Cluster with Direct Storage Access
Database Member Cluster – configured with indirect access to shared storage
Under this configuration, the storage managed by ASM on the
DSC is not mounted on the Database Member Cluster.
Instead, the database instances on the Database Member
Cluster submit their I/O requests over the network to the
IOServer processes on the DSC which then pass on those
requests to the I/O subsystem on the DSC nodes. These
IOServer processes essentially act as pass thru processes for
handing off the I/O requests, without having to do any real
work. Thus, they will not add overhead to the I/O processing.
From the perspective of the database instances on the
Database Member Cluster, the DBA still sees their database
files exactly as they are used to seeing them. They can
manage and access those files exactly as they have always
Figure 7: Database Member Cluster with Indirect Access to Storage
This architecture for a Database Member Cluster that requires no locally configured shared storage would be particularly attractive as it
simplifies the deployment of RAC and RAC One Node clusters, plus inherently benefits from the consolidation of the storage itself.
Benefits of the Cluster Domain Architecture
The Cluster Domain architecture is primarily a solution for managing an ever-expanding group of Oracle clusters, thereby reducing the
management overhead in deploying new clusters, while consolidating storage management and offloading non-critical infrastructure. In
doing so, there is also the potential for I/O performance gains as storage is consolidated, and also reducing the wastage in allocating
storage on a per-cluster basis.
The Cluster Domain is a service-oriented architecture, enabling the services to be configured once and reused many times over. It
provides the security and isolation required for consolidated environments, and has the potential to provide significant benefits without
unacceptable costs.
In essence, by adopting the Cluster Domain architecture, there would be benefits to be realized in risk reduction, storage optimization,
ease of management and deployment, and better optimization of resources. In this environment, it is simple to envision DBA’s focusing
exclusively on their databases, while storage administrators managed the shared storage on the DSC, and cluster administrators
deployed new clusters in a timely fashion to readily meet the demands of the business for more processing, more databases, more
resources and more platforms.
For further details and reading on the features of Oracle Grid Infrastructure 12c Release 2, please refer to the following links:
» Oracle Webpages
» Oracle Documentation
» Oracle Clusterware Administration & Deployment Guide (covers RHP in addition to Clusterware)
» Oracle Automatic Storage Management Administrator's Guide
» Oracle Autonomous Health Framework User’s Guide
» Oracle Real Application Clusters Administration and Deployment Guide
Oracle Corporation, World Headquarters
Worldwide Inquiries
500 Oracle Parkway
Phone: +1.650.506.7000
Redwood Shores, CA 94065, USA
Fax: +1.650.506.7200
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. This document is provided for information purposes only, and the
contents hereof are subject to change without notice. This document is not warranted to be error-free, nor subject to any other
warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability or
fitness for a particular purpose. We specifically disclaim any liability with respect to this document, and no contractual obligations are
formed either directly or indirectly by this document. This document may not be reproduced or transmitted in any form or by any
means, electronic or mechanical, for any purpose, without our prior written permission.
Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.
Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and
are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are
trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group. 0116
Oracle Cluster Domains
March 2017
Author: Ian Cookson
Contributing Authors: Markus Michalewicz, Mark Scardina, Anil Nair, Jim Williams, Ricardo Gonzalez, Randy Hietter
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF