Oracle GoldenGate Performance Tuning for Oracle Database

Oracle GoldenGate Performance Tuning for Oracle Database
Oracle GoldenGate Performance
Tuning for Oracle Database
Integration
CON7773
Patricia McElroy
Distinguished Product Manager
Enterprise Replication Development
October 1st, 2014
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Restricted
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Restricted
Program Agenda
1
Oracle GoldenGate Overview
2
Diagnostic Tools
3
Performance Recommendations
4
Q&A
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Restricted
Program Agenda
1
Oracle GoldenGate Overview
2
Diagnostic Tools
3
Performance Recommendations
4
Q&A
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Restricted
How Oracle GoldenGate Works
Capture: committed transactions are captured (and can be filtered) by reading the transaction logs.
Trail: stages and queues data for routing.
Pump: distributes data for routing to target(s).
Route: data is compressed, encrypted for routing to target(s).
Delivery: applies data with transaction integrity,
transforming the data as required.
Capture
Trail
Files
Trail
Files
Pump
Delivery
LAN / WAN / Internet
Over TCP/IP
Source
Oracle & Non-Oracle
Database(s)
Target
Oracle & Non-Oracle
Database(s)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal/Restricted/Highly Restricted
6
How Oracle GoldenGate Works
Capture: committed transactions are captured (and can be filtered) by reading the transaction logs.
Trail: stages and queues data for routing.
Pump: distributes data for routing to target(s).
Route: data is compressed, encrypted for routing to target(s).
Delivery: applies data with transaction integrity,
transforming the data as required.
Capture
Delivery
Source
Oracle & Non-Oracle
Database(s)
Trail
Files
Trail
Files
Trail
Files
Pump
LAN / WAN / Internet
Over TCP/IP
Bi-directional
Pump
Trail
Files
Delivery
Capture
Target
Oracle & Non-Oracle
Database(s)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal/Restricted/Highly Restricted
7
Also Known As
• Capture = Extract
– Integrated Capture
– Integrated Extract
• Delivery = Replicat
– Integrated Delivery
– Integrated Replicat
– Integrated Apply
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Logmining Server Architecture
Oracle Database
Logmining Server
Preparer N
Reader
Preparer 2
Builder
Capture
Preparer 1
Integrated
Extract
Trail
Files
Extract
Logmining Server (Database Capture Process)
•
Reader: Reads logfile and splits into regions
•
Preparer: Scans regions of logfiles and prefilters based on extract parameters
•
Builder: Merges prepared records in SCN order
•
Capture: Formats Logical Change Records(LCRs) and passes to GoldenGate Extract
•
Requests LCRs from logmining
server
•
Performs Mapping and
Transformations
•
Writes Trail File
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Integrated Replicat
• Introduced in GoldenGate 12.1.2
• Integrated Replicat for Oracle target databases only
– Database releases: 12.1 and 11.2.0.4
• Leverages database parallel apply servers via inbound server for automatic dependency aware
parallel apply
• Minimal changes to Replicat configuration
• Single replicat, no need to use @RANGE or THREAD or other manual partitioning
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Integrated Replicat
Oracle Database
Architecture Diagram
Inbound Server
Applier N
Trail
Files
Replicat
Lightweight
Streaming API
Replicat
Receiver
Preparer
Coordinator
Applier 2
Applier 1
Inbound Server (Database Apply Process)
•
Reads the trail file
•
Receiver: Reads LCRs
•
Constructs logical change
records (LCRs)
•
Preparer: Computes the dependencies between the transactions (primary key, unique
indexes, foreign key) , grouping transactions and sorting in dependency order.
•
Transmits LCRs to Oracle
Database via the Lightweight
Streaming API
•
Coordinator: Coordinates transactions, maintains the order between applier processes.
•
Applier: Performs changes for assigned transactions, including conflict detection and
error handling.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Parallel Apply Processing
Key Features
• Dependency scheduling based on Primary Key, Unique Index, and Foreign
Key constraints at target database
– Supplemental logging at source needed for these columns
– Trail file must record these columns
• Ordering of transaction commit in apply is configurable
• COMMIT_SERIALIZATION
– FULL : commit transactions in same order as committed at source database.
– DEPENDENT_TRANSACTIONS : commit dependent transactions in correct dependency
order (default)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
BATCHSQL
• BATCHSQL supported via inbound server
• Parallelism for BATCHSQL supported
– Use BATCHTRANSOPS to tune batch size (default is 50)
• On error in BATCHSQL,
– Batched transaction rolled back
– Apply in normal mode (non-batched) via inbound server
• Larger batch sizes can cause more transaction dependency waits between
apply servers
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
BATCHSQL Improves Replicat Throughput
Changes per second
140000
120000
100000
BATCHSQL using default
settings
80000
no BATCHSQL
60000
BATCHSQL
40000
20000
0
FULL
DEPENDENT_
TRANSACTIONS
COMMIT_SERIALIZATION
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Integrated Replicat Performance
• Integrated Replicat 24% faster than 6 manually configured
parallel(and Foreign Keys disabled) Classic Replicats
– 129,000 vs. 96,000 DMLs/sec
– OLTP style workload (modified Swingbench)
• Integrated Replicat greatly simplifies configuration without manual
partitioning
– No need to understand workload (PK/UI/FK constraints)
– Foreign Key constraints enabled
– No split transaction semantics
– Autotunes apply parallelism based on workload
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Replicat Performance Comparison
BATCHSQL Enabled
Row Changes per second
140000
120000
100000
80000
60000
40000
20000
0
Workload
Classic - 3 Reps
Classic - 6 Reps
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Integrated
Program Agenda
1
Oracle GoldenGate Overview
2
Diagnostic Tools
3
Performance Recommendations
4
Q&A
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Restricted
Monitoring Latency Automatically
Latency Monitoring
• Monitor latency with Manager parameters (mgr.prm):
– LAGINFO{SECONDS | MINUTES | HOURS}
– LAGREPORT{MINUTES | HOURS}
– LAGCRITICAL{SECONDS | MINUTES | HOURS}
– Monitor ggserr.log for latency information
• Database queries of V$GOLDENGATE_CAPTURE or
V$GG_APPLY_COORDINATOR
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Monitoring GoldenGate Latency
Latency Monitoring
• ggserr.log example:
2014-01-22 18:09:00 WARNING OGG-00947 Oracle GoldenGate Manager for
Oracle, mgr.prm: Lag for EXTRACT EXT_1A is 00:01:40 (checkpoint
updated 00:00:04 ago)
• GoldenGate Management Pack
– Stand alone tool or 12c Oracle Enterprise Manager plug-in
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Process Report File
• List Active Parameters
replicat rep_1a
userid soesmall password ********
discardfile ./dirrpt/rep_1a.dsc, append, megabytes 20
REPORTCOUNT EVERY 1 MINUTES, RATE
ASSUMETARGETDEFS
MAP SOESMALL.* , TARGET SOESMALL.*
• Identifies Table Resolution on first Change Record
2014-09-19 18:57:15 INFO OGG-06506 Wildcard MAP resolved (entry DEMO.*): MAP "DEMO"."ORDERS" ,TARGET
"DEMO"."ORDERS", COMPARECOLS ( ON UPDATE ALL, ON DELETE ALL).
2014-09-19 18:57:15 INFO OGG-06511 Using following columns in default map by name: ORDER_NUMBER, PART_NUMBER,
STATUS, DELIVERY_DATE.
2014-09-19 18:57:15 INFO OGG-06510 Using the following key columns for target table DEMO.ORDERS: ORDER_NUMBER.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Process Report Statistics
• Monitor throughput per process
– REPORTCOUNT EVERY {SECONDS | MINUTES | HOURS}, RATE
– Aggressive monitoring can impact process throughput.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Extract Report File
SEND EXTRACT, LOGSTATS
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Monitoring Integrated Capture
Streams Performance Advisor (SPADV)
• Refer to Oracle Database PL/SQL Packages and Types Reference for
UTL_SPADV usage
• Example real-time statistics:
PATH 2 RUN_ID 15 RUN_TIME 2014-FEB-21 22:32:17 CCA N
|<C> OGG$CAP_EXT_1A 305260 305245 790813 LMR 0% 53.3% 33.3% "CPU + Wait
for CPU" LMP (2) 0% 0% 146.7% "CPU + Wait for CPU" LMB 73.3% 0% 6.7% ""
CAP 20% 0% 73.3% "CPU + Wait for CPU" |<Q>
"STREAMSADMIN"."OGG$Q_EXT_1A" 0.01 0.01 0 |<E> EXT_1A 304027 80076708
790813 20% 6.7% 73.3% "CPU + Wait for CPU" |<B> NO BOTTLENECK
IDENTIFIED
– LMR in flow control, and LMP is 0% idle with high CPU
– Confirm with top/CPU stats and increase Capture Parallelism
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Monitoring Integrated Replicat
Streams Performance Advisor (SPADV)
• Refer to Oracle Database PL/SQL Packages and Types Reference for
UTL_SPADV usage
• Example real-time statistics:
PATH 1 RUN_ID 16 RUN_TIME 2014-JAN-12 10:33:59 CCA Y |<R> REP_1A 1375
520424 0 0% 62.5% 37.5% |<Q> "SOESMALL"."OGGQ$REP_1A" 1375 0.01 5001
|<A> OGG$REP_1A 715 715 -1 APR 31.3% 68.8% 0% "" APC 100% 0% 0% "" APS
(7) 68.8% 0% 481.3% "free buffer waits" |<B> OGG$REP_1A APS 1219 22693
93.8% "free buffer waits“
– Replicat (REP_1a) in flow control , as is APR so look downstream
– APS high waits on free buffer waits
– Confirm database contention with AWR reports and remove it
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Performance Data Gathering
• Automatic Workload Repository (AWR)
– SQL used to apply replicated data, so important to gather and analyze
– If using DBFS database, gather AWR for it
• I/O performance data (iostat)
– Gather for disks database AND trail file locations
– Make sure there are enough resources BEFORE increasing parallelism
• CPU statistics (top)
– Need to see if any processes limited by CPU
• Oracle GoldenGate integrated healthcheck (MOS 1448324.1)
– Structured HTML file on general integrated GoldenGate health
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
DB Oracle 12c: GoldenGate AWR report
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
AWR Replication Example
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
GoldenGate Database Views
• Configuration Views
• Run Time Views
– DBA_GOLDENGATE_PRIVILEGES
– V$GOLDENGATE_CAPTURE
– DBA_GOLDENGATE_SUPPORT_MODE
– DBA_CAPTURE, DBA_CAPTURE_PARAMETERS
– V$GG_APPLY_RECEIVER
– V$GG_APPLY_READER
– DBA_GOLDENGATE_INBOUND
– V$GG_APPLY_COORDINATOR
– DBA_GG_INBOUND_PROGRESS
– DBA_APPLY, DBA_APPLY_PARAMETERS
– V$GG_APPLY_SERVER
– V$GOLDENGATE_TABLE_STATS
– DBA_APPLY_REPERROR_HANDLERS
– V$GOLDENGATE_CAPABILITIES
– DBA_APPLY_HANDLE_COLLISIONS
– DBA_APPLY_DML_CONF_HANDLERS
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
OGG HealthCheck Script
• SQL script that creates structured formatted HTML
– For specific DB release
• Summary
– Overview information
– Advice and Warnings of potential issues with configuration
• Analysis
– Correlate with other configuration to identify discrepancies
– Performance Recommendations
– Detail view information for diagnostic purposes
• Statistics
– Runtime state of Streams processing
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
OGG HealthCheck Usage
• Gather healthCheck at
– Periodically to provide baselines and snapshots over time
– Whenever an error occurs/performance worsens
– At intervals during problem times
– When the problem has been resolved
• Download HealthCheck via MOS Note 1448324.1
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Navigation Aids For Quick Access
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Integrated Extract Summary Example
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Performance Recommendation Example
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Integrated Replicat Summary
Autotuned
Server
Count
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Integrated Replicat
Apply Reader
Barrier
needed
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Integrated Replicat
Apply Servers Executing in Parallel
REPERROR Configuration
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Applying Large Transaction
Eager Apply Serializes Apply Processing
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Wait Analysis
Wait events of last 30 minutes of each OGG background process
Points towards
Buffer Cache / IO
System for further
investigation
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Tips For Analyzing Healthcheck Output
Search For
Version
Use for
Tables not supported
11.2.0.4 & 12.1
NONE = cannot be captured (from redo) or fetched by OGG
GoldenGate Sessions
All
V$SESSION info for OGG foreground and background
processes
Streams Pool Statistics
All
Separate sections for capture, logminer, and apply
GoldenGate Table
Statistics
11.2.0.4 & 12.1
Inserts, Updates, Deletes, wait dependencies, Reperror,
Handle Collisions, CDR stats per table
Logminer Database Map
All
Queries for detailed information from Logminer tables
Standby Redo Log
All
Queries V$STANDBY_LOG
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
1
Oracle GoldenGate Overview
2
Diagnostic Tools
3
Performance Recommendations
4
Q&A
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Restricted
Database Configuration
• STREAMS_POOL_SIZE
– Size appropriately for number of concurrent IE and IR processes
– Minimum 1G per IE or IR process then add 25%
– If IE or IR parameter MAX_SGA_SIZE configured, use sum of MAX_SGA_SIZE + 25%
• ENABLE_GOLDENGATE_REPLICATION
– Set to TRUE for DB 11.2.0.4 and 12.1.0.2
– Classic or Integrated Extract, Integrated or nonIntegrated Replicat
• For performance tuning, use the performance advisor available in
$ORACLE_HOME/rdbms/admin/utlspadv.sql
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Integrated Extract
Extract Parameters
• LOGALLSUPCOLS instructs extract to write supplemental logged columns to
trail file
• UPDATERECORDFORMAT COMPACT
– Single LCR with both BEFORE and AFTER images
– COMPACT reduces the amount of data sent in LCR
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Integrated Extract Parameters
TRANLOGOPTIONS INTEGRATEDPARAMS
• PARALLELISM controls number of preparers for processing logs
– Enterprise Edition allows parallelism of preparers (default is set to 2)
– Increase parallelism if preparers are CPU bound
• Processes down the chain (Builder, Capture) must have bandwidth to handle more work
• UTL_SPADV shows this information
– Standard Edition does not allow parallelism. Parallelism = 1 only.
• MAX_SGA_SIZE controls amount of Streams Pool for specific extract
– Minimum recommendation: 1000 (1G)
– Maximum recommendation: 3500 (3.5G)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Integrated Replicat
Replicat Parameters
• GoldenGate parameter BATCHSQL improves apply performance in the
majority of workloads
– If wait dependencies are high between batched transaction, REDUCE the
BATCHTRANSOPS value (default for IR is 50)
Example: BATCHSQL BATCHTRANSOPS 12
– Table Statistics in Healthcheck can identify specific tables incurring waits
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Integrated Replication Parameters
DBOPTIONS INTEGRATEDPARAMS
• COMMIT_SERIALIZATION default is DEPENDENT_TRANSACTIONS
– Set to FULL, only if apply in source commit order is required
• EAGER_SIZE threshold to begin apply of large transactions default is 9500
–
–
–
–
Eager apply serializes apply processing
For medium sized large transactions (up to 25000 LCRs), set the threshold to higher value
DBOPTIONS INTEGRATEDPARAMS(EAGER_SIZE 25001)
Make sure that streams_pool_size is appropriate to avoid Waiting For Memory issues
• MAX_SGA_SIZE controls amount of memory for IR
– Default is INFINITE (Apply allocates memory as needed and available)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Integrated Replication Parameters
DBOPTIONS INTEGRATEDPARAMS
• PARALLELISM controls minimum number of appliers
– Default is 4
– If PARALLELISM set to 1, apply parallelism is disabled.
• MAX_PARALLELISM controls maximum number of appliers
– Default is 50 for OGG 12.1.2.1 (30 for OGG 12.1.2.0)
• Autotuning of apply parallelism is default
– Computed over range of 5 intervals of PARALLELISM_INTERVAL(5) seconds
– Unneeded processes marked INACTIVE, INACTIVE processes removed after 5 minutes
– If MAX_PARALLELISM=PARALLELISM, autotuning is disabled.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Database Object Tuning
• GoldenGate uses SQL to apply changes
• AWR highlights SQL and SEGMENT issues
• ADDM recommendations as appropriate
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Further Reading
MOS Notes
• MAA white paper Oracle GoldenGate Performance Best Practices
http://www.oracle.com/technetwork/database/availability/maa-gg-performance1969630.pdf
• Note 1557031.1 Recommended patches for Integrated Capture/Replicat
• Note 1448324.1 OGG Integrated Healthcheck Script
• Note 1485620.1 Best Practices Downstream Capture
• Note 1488668.1 GoldenGate Performance Data Gathering
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Questions and Answers
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle OpenWorld 2014
49
Resources
Oracle Data
Integrator
Oracle Data Integration
Oracle
GoldenGate
Oracle Data Integration
Oracle
Enterprise
Data Quality
Oracle Enterprise
Metadata
Management
ORCL DataIntegration
blogs.oracle.com/dataint
egration
Oracle Data
Services
Integrator
OracleGoldenGate
Data Integration
http://www.oracle.com/us/products/middleware/data-integration/overview/index.html
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Oracle OpenWorld 2014
50
Safe Harbor Statement
The preceding is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Restricted
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Restricted
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement