Adapt or Perish How to Avoid a Salted Banana Five

Adapt or Perish How to Avoid a Salted Banana Five
Vol. 29, No. 1 · FEBRUARY 2015$15
Knowledge Happens
Five Database
Technology Trends
Guy Harrison prognosticates.
See page 4.
Adapt or Perish
Janis Griffin expostulates.
See page 15.
How to Avoid a
Salted Banana
Lothar Flatz elucidates.
See page 17.
Much more inside . . .
da Catrice Antoinette Meredith Daisy Nada Liz Amie Jeannine John Sherril
angelina Robbi Kathyrn Steven Gisele Tim Laura Myron Elizbeth Jack M
y Jayme Sun Darell Karan Janine Bert Larry Rodrigo Rachel Todd Nichol
dell Susan Brittaney Sterling Nedra Simon Christy Guy Clare Regena Ann
Blossom Margaret Dan Sunny Nicholle Caitlin Kareem Loyd Megan Son
me Daria Brian Torrie Renato Jan Warren Noah Andy Sharlene Andrea Mark
at Krishna Frank Arlena Cammy Chrissy Erin Manuela Karleen Tom Nick De
ieda Cherri Sabra Steven Ann Sal Alissa Amy Geri Ashlea Henry Beverly Ma
nny Nicholle Caitlin Kareem Eneida Catrice Debbie Meredith Ed Nadia Liz
annine Theron Steve Sherrill Hellen Evangelina Robbi Kathyrn Gisele Jed Ig
ron Betty Elizbeth Jeannine Rich John Eneida Catrice Antoinette Meredith
e Liz Amie Jeannine Theron Sherrill Hellen Evangelina Robbi Kathyrn Gisele
ce Antoinette Meredith Daisy Nada Liz Amie Jeannine Theron John Sherri
gelina Robbi Kathyrn Gisele Frank Alissa Antoinette Meredith Daisy Nada L
Jeannine John Sherrill Hellen
Toad enables connections
with a community of millions
The Toad World community? It’s experience. It’s knowledge.
It’s tips, tricks, and real-time help straight from the experts.
And it’s inside the latest version of Toad for Oracle. Realize
the power of connected intelligence.
Learn more at
© 2014 Dell, Inc. ALL RIGHTS RESERVED. Dell, Dell Software, the Dell Software logo and products—as identified in this document—are registered trademarks of Dell, Inc. in the U.S.A. and/or other
countries. All other trademarks and registered trademarks are property of their respective owners. PrintAd-ToadConnected-VG-25308
Professionals at Work
2015 NoCOUG Board
irst there are the IT professionals who write for the Journal. A very
Hanan Hit
special mention goes to Brian Hitchcock, who has written dozens of
Vice President
Jimmy Brock
book reviews over a 12-year period. The professional pictures on
Eric Hutchinson
the front cover are supplied by
Ganesh Sankar Balabharathi
Next, the Journal is professionally copyedited and proofread by veteran copy­
editor Karen Mead of Creative Solutions. Karen polishes phrasing and calls out
Membership Director
Vacant Position
misused words (such as “reminiscences” instead of “reminisces”). She dots every
Conference Director
Sai Devabhaktuni
i, crosses every t, checks every quote, and verifies every URL.
Then, the Journal is expertly designed by graphics duo Kenneth Lockerbie and
Vendor Coordinator
Omar Anwar
Richard Repas of San Francisco-based Giraffex.
Training Director
Vacant Position
And, finally, Jo Dziubek at Andover Printing Services deftly brings the Journal
to life on an HP Indigo digital printer.
Social Media Director
Vacant Position
This is the 113th issue of the NoCOUG Journal. Enjoy! s
—NoCOUG Journal Editor
Jimmy Brock
Journal Editor
Iggy Fernandez
Table of Contents
Special Feature.................................................. 4
Marketing Director
Ganesh Sankar Balabharathi
NoSQL Corner................................................... 10
Dell Software..................................................... 2
IOUG Liaison
Kyle Hailey
Performance Corner........................................ 15
Axxana................................................................ 9
SQL Corner.........................................................17
Embarcadero Technologies........................... 22
Product Review................................................. 20
Delphix............................................................. 26
Treasurer’s Report............................................ 23
Database Specialists....................................... 27
Tips and Tricks.................................................. 24
SolarWinds....................................................... 28
Members at Large
Jeff Mahe
Linda Yang
Board Advisor
Tim Gorman
Board Associate
Stephen Van Linge
Book Reviewer
Brian Hitchcock
Publication Notices and Submission Format
The NoCOUG Journal is published four times a year by the Northern California
Oracle Users Group (NoCOUG) approximately two weeks prior to the quarterly
educational conferences.
Please send your questions, feedback, and submissions to the NoCOUG Journal
editor at [email protected]
The submission deadline for each issue is eight weeks prior to the quarterly confer­
ence. Ar­ti­cle sub­missions should be made in Microsoft Word format via email.
Copyright © by the Northern California Oracle Users Group except where other­
wise indicated.
NoCOUG does not warrant the NoCOUG Journal to be error-free.
The NoCOUG Journal
The NoCOUG Journal is published quarterly.
Per Issue
Per Year
Quarter Page
Half Page
Full Page
Inside Cover
Personnel recruitment ads are not accepted.
[email protected]
specia l
f eature
Five Database
Technology Trends
with Guy Harrison
Guy Harrison
first started working with database technology in the mid1980s. A lot has changed in database technology over those
30 years, but I would have to say that there have been more
fundamental changes in databases over the past five years
than in the 25 years preceding. I think these changes can be
boiled down to a few key trends that are revolutionizing the
databases of today and the future. Those trends include:
➤ The end of the era in which a single type of database tech­
nology—the relational database—served as a one-sizefits-all-solution.
➤ The emergence of big data technologies, most significant­
ly Hadoop.
➤ The explosion of non-relational operational databases
(aka NoSQL) such as Cassandra and MongoDB.
➤ The increasing capacity to store data in columnar rather
than row-oriented format.
➤ The diminishment of the spinning magnetic disk as the
only viable technology for persistently storing data.
in the rapid uptake of Relational Database Management Systems
From the first release of Oracle in 1978 to the middle of the
2000s, virtually every significant database released was either
relational or claimed to be relational. The database timeline in
Figure 1 shows that virtually every database released from 1980
to 2000 was—or at least claimed to be—a relational database.
Trend #1: The End of Relational Hegemony
Database technology arguably predates digital computing:
Many of IBMs first “business machines” were tabulators—which
were able to summarize and analyze data held on punched cards.
These punched cards arguably represented a paper database.
How­ever, what we think of as computer databases emerged from
the combination of digital computing and persistent storage
media such as magnetic disks and tape.
The first digital databases—circa 1950—were based on se­
quential files that would be scanned into memory on every ac­
cess. However, during the 1960s, true database management
systems emerged that combined indexed access methods and
structured schemas. By the end of the 1960s, the hierarchical and
network database models were established and pervasive.
Edgar Codd first defined A Relational Model of Data for Large
Shared Data Banks in 1970. The relational model had several
advantages over the existing models: most significantly it sup­
ported the concept of ad-hoc queries in which the data model
was not required to anticipate all of the possible queries that
might be supported. The relational model also required an inter­
active query language—eventually standardized on SQL—that
allowed non-programmers to execute query operations. This
interactive query language broke the logjam of report writing
that plagued IT departments of the time and was a major factor
Figure 1. Database timeline
Pre Relational Era
Relational Era
Magnetic tape
“flat” (sequential) files
Magnetic Disk
Oracle V2
Printing press
Punched cards
System R
SQL Server
Network Model
Hierarchical model
Access Mechanism
Non relational Era
While some non-relational systems—Object-Oriented Data­
base Management Systems (OODBMSs) and XML databases—
were developed, none gained significant traction during this
period. For an entire generation of software professional, the
RDBMS reigned supreme.
The First Cracks Appear
The end of the relational hegemony had been foretold since the
middle of the last decade, most significantly by relational database
pioneer Mike Stonebraker, whose 2005 paper “One Size Fits All”:
An Idea Whose Time Has Come and Gone argued that modern
applications would be better served by a mix of database systems
optimized for specific workloads. At the time, the proposed “new
models” of DBMS were largely theoretical. However, Stonebraker
correctly identified the key trend: the complete dominance (though
not primary role) of the relational database was about to end.
As we can see from the timeline in Figure 1, between 2005
and 2014 literally dozens of new databases were released—al­
most none of which were fully relational.
The Third Platform
There are many factors behind the end of the relational su­
premacy, but at a high level, it is a result of the paradigm shifts in
February 2015
software application architecture. We’ve seen any number of
paradigm shifts within the last three decades, including micro­
computer to minicomputer and the introduction of the Internet.
However, arguably all of the things we’ve seen fall into three
major waves of application architecture.
IDC coined the term “the third platform” to describe the cur­
rent application landscape and contrast it to what has come be­
fore. In IDC’s widely accepted model, software applications have
undergone three fundamental changes:
Figure 2. The third platform (source: IDC)
The first platform was based on mainframe systems ac­
cessed through “dumb” terminals. All processing logic was
centralized. These systems typically supported only hun­
dreds of users, although millions of users in aggregate ac­
cessed this category of applications.
➤ The second platform emerged with the client server model
and early web applications. These applications divided ap­
plication logic into two or more tiers (one of which was
typically the database). As the Internet became a universal
WAN, these applications entered the realm of the main­
stream consumer and attracted hundreds of millions of
➤ The third platform emerged from the nexus of megatrends
that we’ve all experienced over the past 10 years: smart­
phones, social networking applications, and cloud com­
puting have combined to revolutionize not just application
architecture but digital lifestyles and our very societies.
Big Data was arguably born out of the third platform but
also strongly influenced its growth. The increasing data
generated by our mobile and social interactions creates
new avenues for user experience that in turn generate
more data. The increasing prevalence of Internet-connected
sensors—the Internet of Things (IoT)—further accelerates
the paradigm shift.
A New Database World Order
Databases supporting the third platform need to deal with
availability, throughput, and performance requirements that
were simply not in existence when the RDBMS was conceived.
The demands on databases have included some of the following
Global availability
Unstructured data and rapidly changing schemas
The NoCOUG Journal
Massive volumes—data volumes had been increasing
steadily since the genesis of the database, but new signifi­
cant pressure arose to store large amounts of unstructured
data at prices far lower than were possible with a rela­
tional schema.
Diverging storage technologies that for the first time cre­
ated an increasing divergence between the price of storing
data and the price of accessing data. Solid-state disk tech­
nologies and in-memory database options became avail­
able, which reduced the price of delivering an I/O opera­
tion, but these technologies were far more expensive when
it came to storage. It became impossible to provide the
best value dollar/GB and best value dollar/IOPS in the
same database technology and storage architecture.
The economics of various components of the technology
stack also increasingly demanded changes in core data­
base architectures. CPU and memory capacity increased
exponentially in accordance with Moore’s law, while net­
work and disk capacity increased geometrically at best. As
a result, it became increasingly more economical to move
processing to the data, rather than to move data to the pro­
Each of these will be discussed in upcoming sections, but in
overview we are currently in an era in which several different
database technologies are employed for most moderately sized
The relational system remains dominant as the underlying
storage system for traditional business systems—in-house
ERP and CRM. For instance, no non-relational system is
certified to support a SAP implementation and there is
little sign that this will change. However, as ERP and CRM
systems migrate to cloud-based SaaS alternatives, many
will be re-engineered and some may potentially adopt a
non-relational data store.
The relational database remains entrenched as the basis
for virtually all in-house data warehouses, and BI tools in
general only deliver their full functionality when working
against relational systems that support the full range of
ANSI SQL, including windowing functions, star schema
optimizations, bitmap indexes, and so on. Again, there is
some threat here from SQL-on-Hadoop, but for the most
part the real-time data warehouse remains the province of
the relational system.
Hadoop is increasingly deployed alongside the relational
data warehouse as a “data lake.” Hadoop serves as the stag­
ing area for data that has yet to find its way into a fixedschema DW and as a repository for fine-grained raw data
that may be the subject of data science projects.
Web applications—particularly those that in the past
would have been based on the LAMP (Linux, Apache,
MySQL & Perl/Python/PHP) stack—are increasing based
not on a relational system such as MySQL but on a nonrelational alternative such as MongoDB. These decisions
are largely being driven from the ground up as developers
decide on the NoSQL database in the absence of any cor­
porate mandate and in projects that adopt the DevOps
style of application development.
In addition to the widespread use cases outlined above,
there are increasingly viable niches for “NewSQL” or oth­
erwise not-quite-relational database systems. These in­
clude columnar variations on the data warehouse such as
Vertica, in-memory systems such as Hana and VoltDB,
and graph-based databases such as Neo4J.
It takes all sorts
Web Server
Teradata …)
Exalytics …)
(Oracle, SQL
Server, …)
ERP & inhouse CRM
Figure 3. Modern enterprise data architectures combine multiple
Trend #2: Big Data and Hadoop
Every few years we seem to be subjected to a new buzzword
that so rapidly dominates all conversation as to become mean­
ingless. Many believe the term “big data”—seemingly added to
any conversation that in any way relates to data—has become
such a buzzword. But in reality, big data encompasses two very
real trends in modern data management:
➤ We are required to store more, and more varied, data than
ever before.
➤ The use of data has shifted in importance: analysis of data
used to be primarily about operational efficiency; in the
modern era it has become about competitive advantage.
The Data Revolution
While there are many arguably valid definitions of big data,
my personal favorite is the Industrial Revolution metaphor
coined by Joe Hellerstein at O’Reilly. Before the Industrial
Revolution in the 1800s, all goods were hand produced. Likewise,
before the industrial revolution of data, all data was produced by
hand: entered by individuals paid to do so by the organization.
Nowadays, of course, data comes in from everywhere: custom­
ers, social networks and, increasingly, Internet-enabled devices.
From a database point of view, this involves more than just
storing qualitatively more data. We are being called upon to
manage the storage of fast amounts of unstructured, unprocessed
data. In the past data was cleansed, schematized and aggregated
prior to being loaded into a data warehouse. This still occurs, but
we are also required to store the original, unadulterated raw data.
Why? Because often it’s the analysis of this original data that can
be used to unlock competitive advantage through advanced tech­
niques of data science.
Google and Hadoop
Undeniably, Google is the pre-eminent pioneer of big data.
From the very beginning, Google was faced with the challenge of
storing and analyzing exponentially increasing volumes of data,
and managing data that was inherently flexible and evolving in
No existing database management system had either the ca­
pacity or capability of storing the volumes and varieties of data
that Google needed to master. So Google was forced to adopt its
own hardware and software stack for the purpose of building a
scalable commercial solution.
This stack involved many components but most notably in­
cluded the following building block technologies:
➤ Google File System (GFS) is a distributed file system that
al­lows the directly attached storage in very large numbers
of commodity servers to be exposed as a single logical file
➤ Map Reduce is a programming model that allows complex
problems to be broken up into simple parallelized map
functions together with reduce functions that combine the
outputs from each parallel stream. Multiple Map Reduce
pipelines can be constructed to solve a very wide variety
of problems. While somewhat simplistic (at least com­
pared to modern alternatives) Map Reduce is an algorithm
that is applicable to a wide variety of problems.
➤ Bigtable is a non-relational database system built on GFS.
It uses a normalized data model in which rows are associ­
ated with a variable, sparse, and potentially very large
number of columns, avoiding in many cases the need for
separate storage of detail records and for joins.
Google published details of the above three technologies dur­
ing the latter part of the last decade. Consequently, an opensource project arose that implemented these and other key
com­p onents of the Google stack. This project is, of course,
Hadoop, which is arguably the most significant new database
storage system to be released in the last decade.
Hadoop was nurtured at Yahoo!, which allowed it to achieve
fairly rapidly the massive scalability for which it is well known. It
provides compelling advantages over relational systems for big
data storage:
➤ It is significantly cheaper than all alternatives for storing
large amounts of data online. This is what Forrester calls
“Hadooponomics”: the ability to linearly scale data storage
and data-processing costs.
➤ It supports the storage and processing of unstructured and
semi-structured data. There is no requirement to define the
schema of a data item for storing it in Hadoop. While this
undoubtedly leads to some degree of “Hadumping”—the
thoughtless dumping of all data without analysis—it obviates
the delay and cost involved with ETL into traditional stores.
➤ Libraries of data science and advanced analysis exist for
Hadoop, providing data scientists with at least a starting
point for the analysis of data held in Hadoop.
Oozie (Workflow manager)
Map Reduce /
(RDBMS loader)
(Log Loader)
Hadoop File System (HDFS)
Figure 4. Simplified Hadoop stack
February 2015
Today, Hadoop forms the background of most big data
Trend #3: Distributed Non-Relational Databases (a.k.a.
Hadoop is an example of a non-relational database system.
However, while Hadoop arose because of pressures on analysis of
massive amounts of data, other non-relational systems were de­
veloped to meet other needs.
Traditionally, start-ups relied heavily on open-source software
stacks: Linux, Apache, and MySQL were the key components of
web start-ups such as Facebook and Twitter. However, as these
sites experienced rapid growth, it became increasingly difficult
to scale systems built on these foundations.
Means of scaling the web server are well established, but the
database presented different challenges. Some degree of relatively
easy scalability can be achieved by read offloading: using read only
replicas or distributed object cases such as Memcached. However,
at some point the master database becomes the bottleneck.
The most common solution to this dilemma is “sharding.” In
a sharded database, the largest tables are partitioned across mul­
tiple nodes. This partitioning is based on some key value, such as
a user ID. The application must be modified to know how to read
from the appropriate shard as well as typically integrating code to
handle memory-cached copies of data and the read-only replicas.
Although sharding succeeded as a solution and is still in use
in sites such as Facebook and Twitter, it is widely regarded as a
poor solution. A sharded database has lost almost all the charac­
teristics of a relational system: joins are not possibleand transac­
tional logic can no longer be expressed simply in SQL.
It has been recognized for some time that the requirements
of massive scalability and availability common to the biggest web
applications are inconsistent with the transactional model ex­
pressed in relational systems. This inconsistency is expressed in
the CAP (aka Brewer’s) theorem.
The CAP theorem states that you can only have two of these
three properties in a system:
➤ Consistency: Everyone always sees the same version of all
➤ Availability: The system can remain available when nodes
➤ Partition tolerance: The system can survive and remain
available when split in two by a network partition.
While a system such as Oracle RAC could provide high avail­
ability and strong consistency, a new model was needed for sys­
tems that would prefer to sacrifice strong consistency in order to
maintain availability across multiple data centers and geographies.
Amazon published their model for highly available data stor­
age in 2007, coining the term “Eventually Consistent” to describe
this model. Together with the Google Bigtable model discussed
earlier, these ideas inspired a variety of NoSQL databases to
emerge, including Hbase, Riak, DynamoDB, and Cassandra.
Non-relational systems were also influenced by object-orient­
ed and XML databases. Programmers have been frustrated for a
long time over the need to map program objects into relational
form. The so-called “impedance mismatch” between the two
models creates programmer overhead and hard-to-maintain
code. As a consequence of the emergence of JSON (JavaScript
Object Notation) as a widely used object document model in web
The NoCOUG Journal
• Everyone always sees
the same data
• System stays up
when network
between nodes
• System stays up
when nodes fail
Most NoSQL lives
Figure 5. CAP (Brewer’s) theorem
applications and the validation of non-relational systems, a num­
ber of JSON-based non-relational databases emerged, such as
MongoDB and Couchbase.
Today these non-relational systems—usually referred to as
NoSQL databases—are an increasingly common choice for backending web applications.
Trend #4: Columnar Databases
The natural way that most of us think about data is to imagine
the data organized into horizontal rows and vertical columns—
the same way that we might enter the data on index cards, a
paper ledger, or Excel. This organization suited early database
systems as well, since most operations were on individual rows/
records: the well-known CRUD (Create, Read, Update, Delete)
However, in data analysis it is rare to perform an analysis on
all the columns in a single row; rather, we tend to do operations
on all the rows in a single column. This shift toward columnbased processing created the opportunity for a new organiza­
tional model for data: the columnar storage model.
Row-oriented database
1001 Dick
1002 Jane
1003 Robert 17/02/80 22,000
1004 Dan
1005 Steven 11/11/81
Column-oriented database
Figure 6. Row-oriented and column-oriented databases
In a column store, columns tend to be organized together on
disk rather than by clustering rows together on disk (Figure 6).
This has several notable advantages:
➤ Aggregate operations such as SUM(), AVG(), etc., typi­
cally need to do less I/O, since all the data they need is
grouped together on disk.
➤ Compression is improved, since there tends to be more
repetition across columns than across rows.
Read Optimized Store
Bulk sequential loads
• Columnar
• Disk-based
• Highly compressed
• Bulk loadable
Asynchronous merge
Delta store
• Row oriented
• Uncompressed
Continual parallel inserts
• Single row inserts
Figure 7. Delta store within a column store
Weighed against these advantages is, primarily, a reduction in
efficiency for typical OLTP operations. An insert, for instance,
must perform as many block operations as there are columns in
the row. Continual inserts effectively require continual reorgani­
zation of the column store, which is rarely practical. For this
reason, most column stores support the idea of a “delta store”: a
staging area that is row organized. Inserts build up in this delta
store and are periodically merged with the column store. In this
interim period, queries must access both the delta store and the
column store in order to retrieve up-to-date data.
Sybase IQ was one of the first commercial column-store data­
bases. Mike Stonebraker and his colleagues defined a theoretical
model for column stores called “C-store” that heavily influenced
the design of HP Vertica and other column-oriented systems.
Oracle uses column-oriented architecture within both
Exadata Hybrid Columnar Compression (EHCC) and Oracle
Database 12c In-Memory Option.
In EHCC, each block contains data for specific columns, but
all columns for a specific row are held within a single compres­
sion unit—typically a 1 MB structure. EHCC allows Oracle to get
the higher compression rates allowed for by columnar storage
while limiting impact on single-row operations, which can still
be satisfied by a single 1 MB I/O.
We’ll consider Oracle Database 12c In-Memory Option in the
next section.
Trend #5: The Demise of the Spinning Disk
The magnetic disk device has been a ubiquitous presence
within digital computing since the 1950s. The essential architec­
Figure 8. Disk devices over the years (Photo courtesy of Paul R.
ture has changed very little over that time: one or more platters
contain magnetic charges that represent bits of information.
These magnetic charges are read and written by an actuator arm,
which moves across the disk to a specific position on the radius
of the platter and then waits for the platter to rotate to the ap­
propriate location.
The time taken to read an item of information is the sum of
the time taken to move the head into position (seek time), the
time taken to rotate the item into place (rotational latency), and
the time taken to transmit the item through the disk controller
(transfer time).
Figure 8 illustrates that while the size and density of these
devices have changed over the years, the architecture remains
virtually identical. While Moore’s law drives exponential growth
in CPU, memory, and disk density, it does not apply to the me­
chanical aspects of this performance; consequently, magnetic
disks have become an increasing drag on database performance
over the years.
The promise of solid-state disks has led some to anticipate a
day when all magnetic disks are replaced by solid-state disks.
While this might someday come to pass, in the short term the
economics of storage and the economics of I/O are at odds: mag­
netic disk technology provides a more economical medium per
unit of storage, while flash technology provides a more eco­
nomical medium for delivering high I/O rates and low latencies.
Performance HDD
Capacity HDD
Figure 9. Economics of storage for solid-state and magnetic disk
Figure 9 illustrates the two competing trends: while the cost
of I/O is reduced with solid-state technology, the cost per TB
increases. Various flavors of SSD (PCI/SATA and MLC/SLC)
offer different price and performance characteristics in mag­
netic disks (15 K vs. 7 K RPM, for instance). The SSD devices
that offer good economies of I/O offer poorer economies for
mass storage. Of course, the cost per gigabyte for SSD is drop­
ping rapidly, but not faster than the falling cost of magnetic
disks or the growth in database storage demand—especially in
the era of big data.
Since most databases include both hot and cold data—small
amounts of frequently accessed data as well as large amounts of
idle data—most databases will experience the best economic
benefit by combining both solid-state and traditional magnetic
disk technologies. This is why Exadata combines both magnetic
disks and flash disks to achieve the ability to provide the optimal
balance between storage economics and performance. If Exadata
contained only magnetic disks, it could not provide superior
February 2015
OLTP performance; if it contained only SSD, it could not offer
compelling economical storage for large databases.
Moore’s law also continues to drive an increase in memory
capacity and decrease in memory cost. Many smaller databases
can now fit entirely within the memory capacity of a single
server, and certainly within the memory capacity of a cluster. For
these databases an in-memory solution may be even more attrac­
tive than a full SSD architecture.
However, these advantages can only be realized fully by a nontraditional database architecture. Simply creating a massive buf­
fer case on top of an Oracle database, for instance, will not
provide benefits for all operations. Full table scans will still use
direct path reads to disk and checkpoints and redo operations
still need to run to a persistent storage layer.
Therefore databases that seek to be truly in memory typically
use different architectural patterns. For instance in Times 10,
data is guaranteed to always be in memory, and no user opera­
tion ever waits for disk I/O (although disks are still used to store
snapshots and transaction logs). By default, a commit writes
asynchronously to the transaction log, so that the user does not
wait on the IO. Strictly speaking this violates the ACID transac­
tion model, since a committed transaction could be lost if the
database failed before the data made it to disk.
HANA employs a mix of columnar and row-based storage,
and also requires persistent disks (often SSDs) for transaction log
and checkpoint writes.
One of the purest in-memory databases is VoltDB. VoltDB
uses a cluster of in-memory partitions that replicate data across
machines. During a commit, data is written to multiple ma­
chines, ensuring that data is not lost should there be a power
failure. This approach is called “K-safety.”
VoltDB also eliminates latch waits by allowing only a single
thread to access a partition at any moment. This is an interesting
aspect of in-memory systems, since when you eliminate I/O
waits you are likely to find that latch waits, which prevent simul­
taneous memory accesses, become the next logical bottleneck.
Oracle Database 12c In-Memory Option uses a combination
of columnar and in-memory approaches. Data is still held in the
traditional buffer cache and data files, but it can also be held in
an in-memory column format. As with other column stores,
there is a row store for deltas (the SMU) that buffers modifica­
Cloud: The Missing Trend?
It’s been true for several years that you can add the phrase “in
the cloud” to almost any technology marketing phrase to in­
crease its value. Since cloud is such a big trend, should “data­
bases in the cloud” or Database as a Service (DBaaS) be a key
database technology trend?
Possibly—in that I tend to believe that the database architec­
tures in the cloud are essentially driven by the database architec­
tures that we see on premise, especially those being pioneered in
the largest Web 2.0 companies. Furthermore, DBaaS uptake will
be slower than other cloud categories, due to some of the unique
characteristics of databases:
When the database is remote from the application it serves,
network latencies become a limiting performance factor that
cannot be overcome. A lot of work has gone into HTTP to make
it very responsive on web pages through things like staged ren­
dering and batching of request/reply packets; less work has gone
The NoCOUG Journal
into shaping database server–application server network traffic
because it’s assumed that they are both on the same local area
network or on a dedicated interconnect.
For a similar reason—assumption that the application server
and database server are on a private network—the database com­
munication and authentication protocols are not as hardened as
in HTTPS. Data items may be transmitted in plain text, and au­
thentication is often restricted to one “super” account that can do
anything. Consequently, opening the database listener port to
the Internet potentially exposes the entire database to attack or
to successful data sniffing.
Data sovereignty and other considerations apply most strong­
ly to the database, so it’s probably the last element of your onpremise servers to be moved to cloud.
None of this is to say that databases won’t move into the cloud
over time, but in my opinion they will follow applications into
the cloud rather than lead the way. s
Guy Harrison is an Oracle ACE and executive director of research
and development at Dell. He is the author of Oracle Performance
Survival Guide (Prentice Hall, 2009) and MySQL Stored Procedure
Programming (O’Reilly, with Steven Feuerstein) as well as other
books, articles, and presentations on database technology. He also
writes a monthly column for Database Trends and Applications
( He is co-author of the upcoming Oracle Exadata
Expert Handbook (Pearson, 2015). Guy can be found on the Inter­
net at and on e-mail at [email protected], and he is @guyharrison on Twitter.
Copyright © 2015, Guy Harrison
Axxana’s award winning Phoenix System
offering unprecedented Data Protection,
Cross-application consistency, Storage and
Replication agnostic.
[email protected] •
The Rise and Fall of
the SQL Empire
by Iggy Fernandez
oSQL is a “disruptive innovation” in the sense used
by Harvard professor Clayton M. Christensen. In
The Innovator’s Dilemma: When New Technologies
Cause Great Firms to Fail, Professor Christensen
defines disruptive innovations and explains why it is dangerous
to ignore them:
“Generally, disruptive innovations were technologically straightforward, consisting of off-the-shelf components put together in
a product architecture that was often simpler than prior approaches. They offered less of what customers in established
markets wanted and so could rarely be initially employed there.
They offered a different package of attributes valued only in
emerging markets remote from, and unimportant to, the mainstream.”
The personal computer was an example of a disruptive inno­
vation. It was initially targeted only at the home computing seg­
ment of the market. Established manufacturers of mainframe
computers and minicomputers did not see PC technology as a
threat to their bottom lines. Eventually, however, PC technology
came to dominate the market, and established computer manu­
facturers such as Digital Equipment Corporation, Prime, Wang,
Nixdorf, Apollo, and Silicon Graphics went out of business.
So where lies the dilemma? Professor Christensen explains:
“In every company, every day, every year, people are going into
senior management, knocking on the door saying: ‘I got a new
product for us.’ And some of those entail making better products
that you can sell for higher prices to your best customers. A
disruptive innovation generally has to cause you to go after new
markets, people who aren’t your customers. And the product
that you want to sell them is something that is just so much
more affordable and simple that your current customers can’t
buy it. And so the choice that you have to make is: Should we
make better products that we can sell for better profits to our
best customers. Or maybe we ought to make worse products
that none of our customers would buy that would ruin our
margins. What should we do? And that really is the dilemma.”
Exactly in the manner that Christensen described, the ecommerce pioneer Amazon created an in-house product called
Dynamo in 2007 to meet the performance, scalability, and avail­
ability needs of its own e-commerce platform after it concluded
that mainstream database management systems were not capable
of satisfying those needs. The most notable aspect of Dynamo
Iggy Fernandez
was the apparent break with the relational model; there was no
mention of relations, relational algebra, or SQL.
Dynamo Requirements and Assumptions
Amazon started out by using Oracle Database for its e-com­
merce platform but later switched to a proprietary database
management system called Dynamo that it built in-house.
Dynamo is the archetypal NoSQL product; it embodies all the
innovations of the NoSQL camp. The Dynamo requirements and
assumptions are documented in the paper “Dynamo: Amazon’s
Highly Available Key-value Store” (
published in 2007. Here are some excerpts from that paper:
“Customers should be able to view and add items to their shopping cart even if disks are failing, network routes are flapping,
or data centers are being destroyed by tornados. Therefore, the
service responsible for managing shopping carts requires that it
can always write to and read from its data store, and that its
data needs to be available across multiple data centers.”
“There are many services on Amazon’s platform that only need
primary-key access to a data store. For many services, such as
those that provide best seller lists, shopping carts, customer preferences, session management, sales rank, and product catalog,
the common pattern of using a relational database would lead
to inefficiencies and limit scale and availability. Dynamo provides a simple primary-key only interface to meet the requirements of these applications.”
“Experience at Amazon has shown that data stores that provide
ACID guarantees tend to have poor availability.”
“Dynamo targets applications that operate with weaker consistency (the “C” in ACID) if this results in high availability.”
“. . . since each service uses its distinct instance of Dynamo, its
initial design targets a scale of up to hundreds of storage hosts.”
To paraphrase, Amazon’s requirements were extreme perfor­
mance, extreme scalability, and extreme availability.
Functional Segmentation
Amazon’s pivotal design decision was to break its monolithic
enterprise-wide database service into simpler component ser­
vices, such as a best-seller list service, a shopping cart service, a
customer preferences service, a sales rank service, and a product
February 2015
catalog service. This avoided a single point of failure. In an inter­
view for the NoCOUG Journal, Amazon’s first database adminis­
trator, Jeremiah Wilton explains the rationale behind Amazon’s
“The best availability in the industry comes from application
software that is predicated upon a surprising assumption: The
databases upon which the software relies will inevitably fail.
The better the software’s ability to continue operating in such a
situation, the higher the overall service’s availability will be. But
isn’t Oracle unbreakable? At the database level, regardless of the
measures taken to improve availability, outages will occur from
time to time. An outage may be from a required upgrade or a
bug. Knowing this, if you engineer application software to handle
this eventuality, then a database outage will have less or no
impact on end users. In summary, there are many ways to improve a single database’s availability. But the highest availability
comes from thoughtful engineering of the entire application architecture.” (
As an example, the shopping cart service should not be affected
if the checkout service is unavailable or not performing well.
I said that this was the pivotal design decision made by
Amazon. I cannot emphasize this enough. If you resist func­
tional segmentation, you are not ready for NoSQL. If you miss
the point, you will not understand NoSQL.
Note that functional segmentation results in simple hierarchi­
cal schemas. Here is an example of a simple hierarchical schema
from Ted Codd’s 1970 paper on the relational model (http://
pdf#page=10). This simple schema stores information about
employees, their children, their job histories, and their salary
employee (man#, name, birthdate)
children (man#, childname, birthyear)
jobhistory (man#, jobdate, title)
salaryhistory (man#, jobdate, salarydate, salary)
Functional segmentation is the underpinning of NoSQL tech­
nology, but it does not present a conflict with the relational
model; it is simply a physical database design decision. Each
functional segment is usually assigned its own standalone data­
base. The collection of functional segments could be regarded as
a single distributed database. However, distributed transactions
are forbidden in the NoSQL world. Functional segmentation can
therefore result in temporary inconsistencies if, for example, the
shopping cart data is not in the same database as the product
catalog, and occasional inconsistencies result. As an Amazon
customer, I occasionally leave items in my shopping cart but
don’t complete a purchase. When I resume shopping, I some­
times get a notification that an item in my shopping chart is no
longer in stock or has been repriced. This technique is called
“eventual consistency.” Randy Shoup, one of the architects of
eBay’s ecommerce platform, explains:
“At eBay, we allow absolutely no client-side or distributed transactions of any kind – no two-phase commit. In certain well-defined
situations, we will combine multiple statements on a single database
into a single transactional operation. For the most part, however,
individual statements are auto-committed. While this intentional
relaxation of orthodox ACID properties does not guarantee imme-
The NoCOUG Journal
diate consistency everywhere, the reality is that most systems are
available the vast majority of the time. Of course, we do employ
various techniques to help the system reach eventual consistency:
careful ordering of database operations, asynchronous recovery
events, and reconciliation or settlement batches. We choose the
technique according to the consistency demands of the particular
use case.” (
Amazon’s next design decision was “sharding” or horizontal
partitioning of all the tables in a hierarchical schema. Hashpartitioning is typically used. Each table is partitioned in the
same way as the other tables in the schema, and each set of parti­
tions is placed in a separate database referred to as a “shard.” The
shards are independent of each other; that is, there is no cluster­
ing as in Oracle RAC.
Note that the hierarchical schemas that result from functional
segmentation are always shardable; that is, hierarchical schemas
are shardable by definition.
Returning to the example from Ted Codd’s 1970 paper on the
relational model:
employee (man#, name, birthdate) with primary key (man#)
children (man#, childname, birthyear) with primary key (man#, childname)
jobhistory (man#, jobdate, title) with primary key (man#, jobdate)
salaryhistory (man#, jobdate, salarydate, salary) with primary key (man#, jobdate,
Note that the jobhistory, salaryhistory, and children tables
have composite keys. In each case, the leading column of the
composite key is the man#. Therefore, all four tables can be par­
titioned using the man#.
Sharding is an essential component of NoSQL designs but it
does not present a conflict with the relational model; it too is
simply a physical database design decision. In the relational
model, the collection of standalone databases or shards can be
logically viewed as a single distributed database.
Replication and Eventual Consistency
The Dynamo developers saw that one of the keys to extreme
availability was data replication. Multiple copies of the shopping
cart are allowed to exist and, if one of the replicas becomes unre­
sponsive, the data can be served by one of the other replicas.
However, because of network latencies, the copies may occasion­
ally get out of sync, and the customer may occasionally encoun­
ter a stale version of the shopping cart. Once again, this can be
handled appropriately by the application tier; the node that falls
behind can catch up eventually, or inconsistencies can be de­
tected and resolved at an opportune time, such as at checkout.
This technique is called “eventual consistency.”
The inventor of relational theory, Dr. Codd, was acutely aware
of the potential overhead of consistency checking. In his 1970
paper, he said:
“There are, of course, several possible ways in which a system
can detect inconsistencies and respond to them. In one approach
the system checks for possible inconsistency whenever an insertion,
deletion, or key update occurs. Naturally, such checking will slow
these operations down [emphasis added]. If an inconsistency has
been generated, details are logged internally, and if it is not remedied within some reasonable time interval, either the user or some-
one responsible for the security and integrity of the data is notified.
Another approach is to conduct consistency checking as a batch
operation once a day or less frequently.” In other words, the inven­
tor of relational theory would not have found a conflict between
his relational model and the “eventual consistency” that is one
of the hallmarks of the NoSQL products of today. However, the
Dynamo developers imagined a conflict because it quite under­
standably conflated the relational model with the ACID guar­
antees of database management systems. However, ACID has
nothing to do with the relational model per se (although rela­
tional theory does come in very handy in defining consistency
constraints); pre-relational database management systems such
as IMS provided ACID guarantees and so did post-relational
object-oriented database management systems.
The tradeoff between consistency and performance is as im­
portant in the wired world of today as it was in Dr. Codd’s world.
Synchronous replication is rarely used in the relational camp, so
we cannot frown at Dynamo for not using it. Application devel­
opers in the relational camp are warned about the negative im­
pact of consistency checking, so we cannot frown on Dynamo’s
decision to permit temporary inconsistencies between func­
tional segments.
“Using primary and foreign keys can impact performance.
Avoid using them when possible.” (
“For performance reasons, the Oracle BPEL Process
Manager, Oracle Mediator, human workflow, Oracle B2B,
SOA Infrastructure, and Oracle BPM Suite schemas have
no foreign key constraints to enforce integrity.” (http://
“For database independence, applications typically do not
store the primary key-foreign key relationships in the da­
tabase itself; rather, the relationships are enforced in the
application.” (
“The ETL process commonly verifies that certain con­
straints are true. For example, it can validate all of the
foreign keys in the data coming into the fact table. This
means that you can trust it to provide clean data, instead
of implementing constraints in the data warehouse.”
The False Premise of NoSQL
The final hurdle was extreme performance, and that’s where
the Dynamo developers went astray. The Dynamo developers
believed that the relational model imposes a “join penalty” and
therefore chose to store data as “blobs.” This objection to the re­
lational model is colorfully summarized by the following state­
ment attributed to Esther Dyson, the editor of the Release 1.0
newsletter, “Using tables to store objects is like driving your car
home and then disassembling it to put it in the garage. It can be
assembled again in the morning, but one eventually asks whether
this is the most efficient way to park a car.” The statement dates
back to 1988 and was much quoted when object-oriented data­
bases were in vogue.
Since the shopping cart is an object, doesn’t disassembling it
for storage make subsequent data retrieval and updates ineffi­
cient? The belief stems from an unfounded assumption that has
found its way into every relational DBMS—that every table
should map to physical storage. In reality, the relational model is
a logical model and, therefore, it does not concern itself with
storage details at all. It would be perfectly legitimate to store the
shopping cart in a physical form that resembled a shopping cart
while still offering a relational model of the data complete with
SQL. In other words, the physical representation could be opti­
mized for the most important use case—retrieving the entire
shopping-cart object using its key—without affecting the rela­
tional model of the data. It would also be perfectly legitimate to
provide a non-relational API for the important use cases. Dr.
Codd himself gave conditional blessing to such non-relational
APIs in his 1985 Computerworld article, “Is Your DBMS Really
Relational?,” in which he says, “If a relational system has a lowlevel (single-record-at-a-time) language, that low level [should not]
be used to subvert or bypass the integrity rules and constraints
expressed in the higher level relational language (multiple-recordsat-a-time).”
The key-blob or “key-value” approach used by Dynamo
and successor products would be called “zeroth” normal form
in relational terminology. In his 1970 paper, “A Relational
Model of Data for Large Shared Data Banks,” Dr. Codd says:
“Nonatomic values can be discussed within the relational framework. Thus, some domains may have relations as elements.
These relations may, in turn, be defined on nonsimple domains, and so on. For example, one of the domains on which
the relation employee is defined might be salary history. An
element of the salary history domain is a binary relation defined on the domain date and the domain salary. The salary
history domain is the set of all such binary relations. At any
instant of time there are as many instances of the salary history
relation in the data bank as there are employees. In contrast,
there is only one instance of the employee relation.”
In common parlance, a relation with non-simple domains is
said to be in “zeroth” normal form or unnormalized. Dr. Codd
suggested that unnormalized relations should be normalized for
ease of use. Here again is the unnormalized employee relation
from Dr. Codd’s paper:
employee (
jobhistory (jobdate, title, salaryhistory (salarydate, salary)),
children (childname, birthyear)
The above unnormalized relation can be decomposed into
four normalized relations as follows.
employee' (employee#, name, birthdate)
jobhistory' (employee#, jobdate, title)
salaryhistory' (employee#, jobdate, salarydate, salary)
children' (employee#, childname, birthyear)
However, this is not to suggest that these normalized relations
must necessarily be mapped to individual buckets of physical
storage. Dr. Codd differentiated between the stored set, the
named set, and the expressible set. In the above example, we have
February 2015
one unnormalized relation and four normalized relations. If we
preferred, the unnormalized employee relation could be the only
member of the stored set. Alternatively, if we preferred, all five
relations could be part of the stored set; that is, we could legiti­
mately store redundant representations of the data. However, the
common belief blessed by current practice is that the normalized
relations should be the only members of the stored set.
Even if the stored set contains only normalized relations,
they need not map to different buckets of physical storage.
Oracle Database is unique among mainstream database
management systems in providing a convenient construct
called the “table cluster” that is suitable for hierarchical
schemas. In Dr. Codd’s example, employee# would be t he
cluster key, and rows corresponding to the same cluster
key from all four tables could be stored in the same physical
block on disk, thus avoiding the join penalty. If the cluster
was a “hash cluster,” no indexes would be required for t he
use case of retriev ing records belonging to a single cluster
key. A demonstration is available at http://iggyfernandez.
Schemaless Design
The final innovation of the NoSQL camp is “schemaless de­
sign.” In database management systems of the NoSQL kind, data
is stored in “blobs” or documents; the database management
system does not police their structure. In mainstream database
management systems on the other hand, doctrinal purity re­
quires that the schema be designed before data is inserted. Let’s
do a thought experiment.
Let’s suppose that we don’t have a schema and that the follow­
ing facts are known:
Iggy Fernandez is an employee with EMPLOYEE_ID=1
and SALARY=$1000.
Mogens Norgaard is a commissioned employee with
➤ Morten
Egan is a commissioned employee with
EMPLOYEE_ID=3, SALARY=€1000, and unknown
Could we ask the following questions and expect to re­
ceive correct answers?
Question: What is the salary of Iggy Fernandez?
Expected answer: $1000.
Question: What is the commission percentage of Iggy
Expected answer: Invalid question.
Question: What is the commission percentage of Mogens
Expected answer: 25%
Question: What is the commission percentage of Morten
Expected answer: Unknown.
If we humans can process the above data and correctly answer
the above questions, then surely we can program computers to
do so.
The NoCOUG Journal
The above data could be modeled with the following three
relations. It is certainly disruptive to suggest that this be done on
the fly by the database management system, but it is not outside
the realm of possibility.
A NoSQL company called Hadapt has already stepped for­
ward with such a feature:
“While it is true that SQL requires a schema, it is entirely untrue
that the user has to define this schema in advance before query
processing. There are many data sets out there, including
JSON, XML, and generic key-value data sets that are selfdescribing — each value is associated with some key that
describes what entity attribute this value is associated
with [emphasis added]. If these data sets are stored in Hadoop,
there is no reason why Hadoop cannot automatically generate a
virtual schema against which SQL queries can be issued. And if
this is true, users should not be forced to define a schema before
using a SQL-on-Hadoop solution — they should be able to effortlessly issue SQL against a schema that was automatically
generated for them when data was loaded into Hadoop.”
This is not really new ground. Oracle Database provides the
ability to convert XML documents into relational tables (http://
htm#ADXDB0120), though it ought to be possible to view XML
data as tables while physically storing it in XML format in order
to benefit certain use cases. It should also be possible to redun­
dantly store data in both XML and relational formats in order to
benefit other use cases.
In “Extending the Database Relational Model to Capture
More Meaning,” Dr. Codd explains how a “formatted database” is
created from a collection of facts:
“Suppose we think of a database initially as a set of formulas in
first-order predicate logic. Further, each formula has no free
variables and is in as atomic a form as possible (e.g, A & B
would be replaced by the component formulas A, B). Now suppose that most of the formulas are simple assertions of the form
Pab…z (where P is a predicate and a, b, … , z are constants),
and that the number of distinct predicates in the database is few
compared with the number of simple assertions. Such a database is usually called formatted, because the major part of it
lends itself to rather regular structuring. One obvious way is to
factor out the predicate common to a set of simple assertions
and then treat the set as an instance of an n-ary relation and the
predicate as the name of the relation.”
In other words, a collection of facts can always be organized
into a collection of relations.
NoSQL Taxonomy
NoSQL databases can be classified into the following catego­
➤ Key-value stores: The archetype is Amazon Dynamo, of
which DynamoDB is the commercial successor. Key-value
stores basically allow applications to “put” and “get” val­
ues, but each product has differentiators. For example,
DynamoDB supports “tables” (namespaces) while Oracle
NoSQL Database offers “major” and “minor” key paths.
➤ Document stores: While key-value stores treat values as
uninterpreted strings, document stores allow values to
be managed using formats such as JSON (JavaScript
Object Notation) that are conceptually similar to XML.
This allows key-value pairs to be indexed by any compo­
nent of the value just as XML data can be indexed in
mainstream database management systems.
➤ Column-family stores: Column-family stores allow data
associated with a single key to be spread over multiple
storage nodes. Each storage node only stores a subset of
the data associated with the key, hence the name “columnfamily.” A key is therefore composed of a “row key” and a
“column key.”
➤ Graph databases: Graph databases are non-relational da­
tabases that use graph concepts such as nodes and edges
to solve certain classes of problems, such as determining
the shortest route between two towns on a map. The con­
cepts of functional segmentation, sharding, replication,
eventual consistency, and schemaless design do not apply
to graph databases, so I will not discuss graph databases.
NoSQL Buyer’s Guide
NoSQL products are numerous and rapidly evolving. There is
a crying need for a continuously updated encyclopedia of
NoSQL products, but none exists. There is a crying need for an
independent benchmarking organization, but none exists. My
best advice is to do a proof of concept (POC) as well as a PSR
(Performance, Scalability, and Reliability) test before committing
to using a NoSQL product. Back in 1985, Dr. Codd had words of
advice for those who were debating between the new relational
products and the established pre-relational products of the time.
“Any buyer confronted with the decision of which DBMS to acquire should weigh three factors heavily. The first factor is the
buyer’s performance requirements, often expressed in terms of
the number of transactions that must be executed per second.
The average complexity of each transaction is also an important
consideration. Only if the performance requirements are extremely severe should buyers rule out present relational DBMS
products on this basis. Even then buyers should design performance tests of their own, rather than rely on vendor-designed
tests or vendor-declared strategies [emphasis added]. The
second factor is reduced costs for developing new databases and
new application programs . . . The third factor is protecting future investments in application programs by acquiring a DBMS
with a solid theoretical foundation . . . In every case, a relational
DBMS wins on factors two and three. In many cases, it can win
on factor one also—in spite of all the myths about per­f or­
mance.”—“An Evaluation Scheme for Database Man­agement
Systems that are claimed to be Relational”
The above advice is as solid today as it was in Dr. Codd’s day.
Oracle NoSQL Database
In May 2011, Oracle Corporation published a white paper ti­
tled “Debunking the NoSQL Hype,” the final advice being “Go for
the tried and true path. Don’t be risking your data on NoSQL databases.” (
e+NoSQL+Hype%22) However, in September of the same year,
Oracle Corporation released Oracle NoSQL Database. Oracle
suggested that the NoSQL approach was well-suited for certain
“The Oracle NoSQL Database, with its ‘No Single Point of
Failure’ architecture, is the right solution when data access is
“simple” in nature and application demands exceed the volume
or latency capability of traditional data management solutions.
For example, click-stream data from high volume web sites,
high-throughput event processing and social networking
communications all represent application domains that
produce extraordinary volumes of simple keyed data.
Monitoring online retail behavior, accessing customer profiles,
pulling up appropriate customer ads and storing and
forwarding real-time communication are examples of domains
requiring the ultimate in low-latency access. Highly distributed
applications such as real-time sensor aggregation and scalable
authentication also represent domains well-suited to Oracle
NoSQL Database.” (
Oracle NoSQL Database has two features that distinguish it
from other key-value stores: A key is the concatenation of a
“major key path” and a “minor key path.” All records with the
same “major key path” will be colocated on the same storage
node. In addition, Oracle NoSQL provides transactional support
for modifying multiple records with the same major key path.
Challeges to NoSQL
There are already proofs that performance, scalability, and
reliability can be achieved without abandoning the relational
model. For example, ScaleBase provides sharding and replication
on top of MySQL storage nodes. Another good example to study
is VoltDB, which claims to be the world’s fastest OLTP database
(though it has never published an audited TPC benchmark). A
counter-example to Amazon is eBay, which arguably has equal
scale and equally high-performance, scalability, and reliability
requirements. eBay uses performance segmentation, sharding,
replication, and eventual consistency but continues to use Oracle
(and SQL) to manage local databases. I asked Randy Shoup, one
of the architects of the eBay e-commerce platform, why eBay did
not abandon Oracle Database, and he answered in one word:
“comfort.” Here are links to some of his presentations and arti­
cles on the eBay architecture:
“eBay’s Scaling Odyssey: Growing and Evolving a Large
eCommerce Site” (
“The eBay Architecture: Striking a balance between site
stability, feature velocity, performance, and cost” (http://
w w w. a d d s i m p l i c i t y. c o m / d o w n l o a d s /
“Randy Shoup Discusses the eBay Architecture” (http://
(continued on page 26)
February 2015
P er f or mance
C orner
Adapt or Perish
by Janis Griffin
Janis Griffin
n Oracle Database 12c, the optimizer can make runtime
adjustments during the execution of the plan. It does this
by instrumenting the default plan with statistic collectors.
At runtime, statistic collectors read and buffer a small
amount of rows in each step and compare the actual statistics
with the estimates. If they are different, the step is adapted to use
the better plan in order to avoid poor performance. Currently,
the only steps that can change are join method operations, such
as changing a nested loop join to a hash join. Also, when the
optimizer is working on a parallel query, it can change the paral­
lel distribution method.
To identify whether a SQL statement is using an adaptive
plan, we need to look at a new column in the V$SQL table called
IS_RESOLVED_ADAPTIVE_PLAN. If the value of this column
is “Y,” then the plan has been adapted and is the final plan. If the
value is “N,” then the plan is adaptable, but the final plan has not
yet been chosen. This value will probably never be seen, as the
final plan is usually chosen on the first run. If the column is null,
the plan is not adaptable. Also, there is another new column in
V$SQL called IS_REOPTIMIZABLE. This column is used to tell
the optimizer to adapt the statistics. The values for this column
can be “R,” for reporting-only mode; “Y,” to signal a hard parse so
it can gather better statistics; and “N,” when no new statistics are
The parameters that control the behavior of the adaptive query
optimizer are optimizer_adaptive_features and optimizer_adap­
tive_reporting_only. By default, the optimizer_adaptive_features
option is set to TRUE and the optimizer_adaptive_reporting_only
option is set to FALSE. The “reporting only” parameter can be set
to TRUE to first test and see which queries are going to be adapted
before the optimizer actually uses them. This might be helpful
when first upgrading from an earlier version of Oracle to 12c, in
order to avoid any unexpected behavior. You can also set the opti­
mizer_feature_enable parameter to something less than 12c to
turn this feature off.
To view adaptive plans, there are two new format parameter
values for the DBMS_XPLAN.DISPLAY_CURSOR function.
The “+adaptive” format parameter will show all active and inac­
tive steps of a plan. The “+report” format parameter can be used
when the “optimizer_adaptive_reporting_only” parameter is set
to TRUE in order to see how the plan would have been adapted
if the adaptive optimizer were turned on. Figure 1 shows an ex­
ample of setting the reporting_only parameter and then using
the “report” format parameter to view what it would be.
The NoCOUG Journal
Figure 1
Figure 2 shows an example of using the “adaptive” format
parameter. Notice the inactive steps are the hash join, statistics
collector, and full table access of product. The plan adapted from
using a hash join to using nested loops.
Figure 2
Figure 3 shows an example of an adapted and reoptimized
plan for the SQL_ID beginning with “8qpak”. Notice that all of
the children have adaptive plans—identified by the “Y” in the
IS_RESOLVED_ADAPTIVE column. Also, notice that the IS_
REOPTIMIZABLE column has a value of “R” for child_number
0 because the reporting_only parameter was set to TRUE on the
first execution. In other words, the optimizer didn’t actually use
the adaptive plan. On second execution, the reporting_only pa­
rameter was set to FALSE, so the optimizer not only adapted the
plan but also sampled new statistics and found the default statis­
tics to be incorrect. Child_number 1 was marked for a reparse on
the next execution, and the value of the IS_REOPTIMIZABLE
column was set to “Y.” Finally, child_number 2 has been success­
fully reoptimized, and no more changes are currently required
(IS_REOPTIMIZABLE is set to “N”).
Figure 4 shows an example of how statistics feedback works.
For this example, the format parameter, “ALLSTATS LAST”, as
well as the hint of “gather_plan_statistics” are used to show the
estimated rows versus the actual rows returned in each step.
Figure 3
The other way for a plan to adapt is when the SQL statement
is resolved by using a parallel query. The optimizer tries to dis­
tribute the number of rows evenly across the number of pro­
cesses that it has to work with. Historically, the optimizer would
look at the number of rows and Degree of Parallelism (DOP),
and then decide how to distribute the data. This often caused
problems when there were only a few parallel processes trying
to distribute a large number of rows. Also, any data skew could
cause unequal distribution and performance issues. In 12c, a
new Hybrid Hash distribution technique tries to fix those issues.
Basically, the optimizer chooses the distribution method during
execution time by inserting statistic collectors in front of the
parallel processes on the producer side of the operation. The
optimizer will choose the Hybrid Hash distribution method if
rows are greater than the threshold, or it will choose the Broad­
cast distribution method if rows are less than the threshold. The
threshold is defined as 2 * DOP.
In Figure 4, there is an example of a Hybrid Hash distribution.
Notice that I gave it a hint to use parallel 20, and with that my
threshold is 2 times that, or 40. Notice the number of rows is
much greater than that, at 75329.
Figure 4
We briefly discussed automatic reoptimization when we were
looking at adaptive plans. However, it differs from adaptive plans
in that reoptimization changes the plan after the first execution
and not during it. It uses information gathered during the exe­
cution and compares that with the default plan. If they differ
sig­nifi­­cantly, the plan will be reoptimized on the next run. Some­
times the optimizer will store the new information gathered as a
SQL PLAN DIRECTIVE, so other similar queries can benefit
from the information. The optimizer can reoptimize a query
several times by using statistics or performance feedback to
change the plan.
Statistics feedback is used when estimated cardinalities are
wrong. This can be due to missing or inaccurate statistics, or
complex predicates. The optimizer will turn on statistics feed­
back if a table has no statistics or if there are multiple (and/or)
filter predicates on a table, or if the predicates contain complex
operations for which it can’t compute estimates.
Figure 5
In Figure 5, notice how far off the default values were. The
optimizer realized this discrepancy, so it used statistics feedback
to further optimize it.
Does the adaptive query optimizer work as advertised? Here is
a SolarWinds Database Performance Analyzer (formerly Confio
Ignite) for Oracle screenshot of one query that ran for several
days with the same work load. Basically, I had eight users running
the same order entry load over several days. This one query,
which looked up product inventory, was running every hour of
the day. You can see that it wasn’t performing very well prior to 9
a.m. In fact, the throughput or number of executions was about
10,000 per hour. During that time I had been running the adap­
tive optimizer in “report only” mode, so it was only using the
default plan. After I turned the mode off, you can see that the
plan changed by adapting, and then it was reoptimized several
times. Notice the time spent and the increase in throughput.
Figure 6
In summary, it looks like the adaptive query optimizer can
improve performance over time. s
Janis Griffin is an Oracle ACE and a database performance evangelist
at SolarWinds, the maker of Database Performance Analyzer (formerly Confio Ignite) for Oracle. Information on Database Performance
Analyzer can be found at
Copyright © 2015, Janis Griffin
February 2015
S QL C orner
How to Avoid a
Salted Banana
by Lothar Flatz
wo men sit together in a train. One of them takes out
a banana, peels it, salts it, and throws it out of the win­
dow. Then he takes another banana and repeats the
After the third banana, the second man can’t contain his curi­
osity. He asks, “Why are you throwing all of these bananas out of
the window?”
“Well,” the other man replies, “do you like salted bananas?”
Throw-Away Rows in Performance Tuning
Let’s take a moment to examine why this joke is funny. It’s
funny because it’s ridiculous: if you want to throw a banana away,
why would you go through the process of peeling and salting it
first? But now we move on to more serious stuff: Is it possible
that we do something similar in real life, for example, in database
performance tuning?
Throw-away rows are rows that were retrieved by the database
but not used in any way to compute the result of a query.
Minimizing throw-away rows is a common theme in perfor­
mance tuning ([1], paragraph 12.1.2).
The methods are intuitively plausible, but a row that is re­
trieved but not used certainly represents lost effort.
One common application of throw-away is index optimiza­
tion, as shown in Figure 1.
Lothar Flatz
index? Let’s have a look on the next figure.
Here we see the exact same execution plan as the one above,
but this time it’s focused on physical I/O.
As we can see, we have a lot more I/O requests on the table
access than on the index access. To understand this, we have to
think about how the buffer cache works. The index has 3,075 leaf
blocks, and the table has 78,954 blocks.
Each segment is accessed 175,000 times (see Figure 1). For the
index, that means 57 hits per leaf block. For the table, that would
mean roughly two hits per block.
In addition, the index is sorted based on the search criteria;
therefore, we can expect that not the whole index is retrieved but
only the part that is relevant for the search.
Thus, under normal circumstances, most of the 175,000 buf­
fer gets of the index access will be found in the buffer cache. In
comparison, a higher proportion of the buffer gets of the table
access will translate into physical reads.
Figure 2: I/O wait time compared: index access vs. table access
Adding an additional filter column in the index will therefore
reduce the overall number of physical reads.
Figure 1: Sub-optimal index access indicated by throw-away
Here we see the essential part of a SQL monitor output. We
see an index access retrieving 175k rows. When the table is ac­
cessed, only one row remains.
There must be an additional filter on the table that reduces the
rows to that level. Certainly that filter is missing in the index. All
we need to do is look up this filter in the execution plan and add
it to the index. (Note: Of course, we must also make sure that the
additional field in the index will be commonly used and not used
by just one query.)
This way we would have the throw-away in the index access
as opposed to having it in the table access.
So why would it be better to have the throw-away in the
The NoCOUG Journal
A Long-Running Query
Some years ago I received a query to tune that kept me think­
ing for years. Of course I did not work on it all the time, but oc­
casionally it popped up and I started contemplating again how it
could be solved.
At first glance, it seemed very simple. In essence it was “find
all people with the name Miller in the city of Bern.”
Figure 3: Set diagram for the Miller-in-Bern search
That seems to be a common, simple query. On second glance,
it is not that simple. Miller (actually Müller in German) is a com­
mon name in Switzerland (as well as in Germany). Bern, the
Swiss capital, has about 350,000 inhabitants.
No matter where you start searching, your first result tends to
be rather big, and the overlapping final result is comparatively
Actually, the best way would be to combine the two search
criteria. That is hard to do, however, since the two search criteria
are against different tables: one on table person and the other on
table address.
Let’s explore the different options to combine two search cri­
teria in one index using features of the Oracle database.
Our underlying assumption is that we work in an OLTP envi­
ronment, like a call center application. Actually, that is the envi­
ronment that typically generates such queries.
Most of our ideas would also work in a data warehouse, but
here the alternative solutions (see below) could be considered
more seriously.
Materialized View
One way to deal with this situation is to join table address and
table person by building a materialized view, and then building a
composite index on top of it.
In order for this to work there are a few prerequisites:
Creating a materialized view must be possible in the sense
that we have the rights and the space, etc.
In an OLTP environment we must create a fast refresh on
commit materialized view, which can be very ambitious to man­
age. We are violating normal forms, which will lead into update
anomalies that can make the refresh very resource consuming.
This solution can work out in some carefully selected sce­
narios, but it is certainly not a generic solution.
Bitmap Join Index
In principle this solution has about the same drawbacks as the
materialized view. It also holds a more serious drawback in the
lock that will be applied due to the bitmap index.
In an OLTP environment this is not an option.
Text Index
It is possible to have a multi-column, multi-table text index.
However, the refresh per commit would be challenging in an
OTLP environment.
In addition, we would have to change the queries to adopt text
index search operators.
Searching for a Solution
I came across this type of query occasionally in my profes­
sional career. After all, it is a typical query used in call centers.
Every time somebody told me, “Our call center software is tuned
to the max and will respond instantly,” I told them to search for
“Müller” in “Bern,” or for “Meier” in “Zurich.”
The result was always a jaw-droppingly long wait. After con­
templating awhile over the issue, I came up with the idea that the
query was indeed dominated by the problem of throw-away
rows, or throw-away dominated, as I’ll refer to them going for­
Let’s look at Figure 3 again. Only the orange part is the result.
The rest is throw-away.
Actually, the throw-away is far bigger than the result. So that
means the query is throw-away dominated.
The question I was asking myself was, “Why would you go
through the extra effort of reading a row from the table when
you know with high probability that you are going to throw it
away anyway?
The idea of the salted banana arrived in my head as a kind of
metaphor for the table access that might now be required. Again,
if you want to throw away your banana, why would you go
through the process of peeling and salting it first?
Our Example
To redo and test the issues with the “Müller-in-Bern” query, I
have created an example that roughly resembles the real data set.
I created a schema containing two tables.
Create table address (city_name
varchar2(30) not null,
number(7) not null,
create Table person (person_id number(7) not null,
last_name varchar2(15) not null,
As you can see the schema is simplified. It does not mean to
be realistic, but be enough to aid our discussion.
In addition these indexes exist to allow various type of join
sequences and to give the optimizer a choice.
city_idx1 on
city_idx2 on
address (city_name, person_id) compress;
address (person_id, city_name) compress;
on person(last_name, person_id) compress;
on person(person_id, last_name) compress;
I also created statistics. I used histograms for table Person but
not for table Address.
This is now the query we are discussing. The assumption is
that it runs in an OLTP system; therefore, we like to see the first
ten rows only.
select * from (
select a.city_name, p.last_name, substr(,1,1) p_data,
substr(,1,1) a_data
from ibj.address a,
ibj.person p
where p.person_id = a.person_id
and a.city_name = 'Bern'
and p.last_name = 'Müller')
where rownum < 11
Execution plans
The classical execution plan would be a nested loop join.
Figure 4: Traditional execution plan
February 2015
As we already know, the plan has its weakness in the excessive
time it takes while executing against both tables.
Of course, Oracle has identified the issue and has started tak­
ing measures against it.
As Randolf Geist notes in [2]: “Oracle introduced in recent
releases various enhancements in that area—in particular in 9i
the ‘Table Prefetching’ and in 11g the Nested Loop Join Batching
using ‘Vector/Batched I/O.’ . . . The intention of Prefetching and
Batching seems to be the same—they are both targeted toward
what is usually the most expensive part of the Nested Loop Join:
the random table lookup as part of the inner row source. By try­
ing to ‘prefetch’ or ‘batch’ physical I/O operations caused by this
random block access, Oracle attempts to minimize the I/O waits.”
In the Table prefetch, the access to the non-driving table is
done outside of the nested loop, which is shown in Figure 5. The
rowids are collected and buffered inside of the nested loop. For
adjacent blocks, the table access by rowid is done as a small vec­
tor I/O.
Nested loop batching goes one step further, as the access to
the outer tables is done as a completely separate step.
As stated in [1], paragraph
“Oracle Database 11g Release 1 (11.1) introduces a new implementation for nested loop joins to reduce overall latency for
physical I/O. When an index or a table block is not in the buffer
cache and is needed to process the join, a physical I/O is required.
In Oracle Database 11g Release 1 (11.1), Oracle Database can
batch multiple physical I/O requests and process them using a
vector I/O instead of processing them one at a time. As part of the
new implementation for nested loop joins, two NESTED LOOPS
join row sources might appear in the execution plan where only
one would have appeared in prior releases. In such cases, Oracle
Database allocates one NESTED LOOPS join row source to join
the values from the table on the outer side of the join with the
index on the inner side. A second row source is allocated to join
the result of the first join, which includes the rowids stored in
the index, with the table on the inner side of the join.”
As you will see, our proposed solution (Figure 8), which we
call an “index backbone join,” follows that same path in a more
generic manner.
But first, let’s take a close look at the runtime statistics in the
case of nested loop batching (Figure 7).
Figure 5: Table prefetching
Figure 6: Nested loop batching
As we can clearly see, the access to table Address takes most
of the time. So why would we access the table, when the row we
get will likely be thrown away? All the necessary information to
select and join the rows will already be found in the index.
The index backbone join consist of two phases: in the first
phase, only the indexes are used to find the result set rowids.
When we have the result set rowid, we do a delayed table access
for both tables.
That is the major difference in nested loop batching. Here the
access to one table is delayed.
The optimizer is not capable of doing an index backbone join,
but we can simulate it with a manual rewrite. Note that we need
a no merge hint to separate the index only scans from the table
access by rowid phase.
select * from (
select a.city_name, p.last_name, substr(,1,1) p_data,
substr(,1,1) a_data
from ibj.person p,
ibj.address a,
(select /*+ NO_MERGE */ p.rowid p_rowid, – phase 1
a.rowid a_rowid
from ibj.address a,
(continued on page 26)
Figure 7: Runtime statistics for nested loop batching
Figure 8: Runtime statistics for simulated index backbone join
The NoCOUG Journal
revie w
Virtualize the Database
and Application with
by Kyle Hailey
anaging copies of Oracle databases in non-pro­
duction environments requires massive amounts
infrastructure and personnel resources. For­tun­
ately, there is a new technology called “database
virtualization” that eliminates the majority of the resources re­
quired to create and maintain these database copies.
Oracle estimates that customers deploy, on average, 12 clones
of production databases to non-production environments. These
database clones are used to support the software development
lifecycle—developing new functionality, testing new versions of
applications by quality assurance (QA), and user acceptance test­
ing (UAT) prior to production. The clones are also used for
reporting ad hoc information queries as well as for backup
purposes. Further, Oracle predicts that this average will double
by the time Oracle 12c is fully adopted.
Today, most cloning is accomplished by creating full physical
copies of production databases. These full physical copies are
time consuming to make, requiring significant DBA time and
storage space, and they generally lead to project delays.
Development and QA processes can further increase the need
for copies. Developers generally work on separate branches of
code, which can have associated requirements for database
schema changes or specific datasets. If developers are sharing a
database copy, the job falls to the developers to make sure they
approve any changes and that these changes are compatible with
what everyone else is working on. This process of approving
changes alone can take weeks and add much more time debug­
ging when data or schema changes break other developers’ code.
Ideally, developers would operate in a sandbox with their own
copy of the production test database.
QA has to build up copies of the database with the changes
made by development before QA test suites are run. The build
time for these database copies slows down the QA process, in­
creasing the cost of QA and delaying the time it takes to find
bugs. The longer it takes to find bugs, the more work it takes to
fix those bugs, as there is more and more dependent code written
on top of them. Plus, ideally, we’d want more than one QA envi­
ronment to run QA suites in parallel, and test faster and more
As an example scenario, a customer with a 1 terabyte database
with 100 developers and 20 test suites would need close to 130
production database copies (one database copy per developer
and test suite, and a few extra for branching, merging, ad hoc
Kyle Hailey
queries, and reporting). Understandably, very few companies
have the resources (DBA time and storage) to provision these, let
alone keep them refreshed for the duration of the project.
Given the high demand for clones of production databases,
companies and DBAs often struggle to keep up and must make
sacrifices in quality or quantity. The compromises reached are
generally fewer, shared databases, partial subset databases, or a
mixture of both.
Thin-Provisioned Cloning
Thin-provisioned cloning was the first technology to address
the issue of storing large numbers of identical data blocks. Thin
provisioning introduces a new layer over a copy of a source data­
base. Each clone has a separate thin layer where the clone main­
tains its changes to the central copy, which remains unchanged.
As each clone has a separate thin layer that only it can see, each
has the appearance of being a full physical copy of the source
database. Thin provisioning can eliminate much of the space
demand of database copies, reducing the associated storage cost
of non-production database copies.
Single Point in Time
Single point in time thin-provisioned cloning is the simplest
thin-provisioning technology. Oracle first offered this technolo­
gy in a feature called Clone DB in Oracle Single point in
time thin provisioning takes a full database backup at a point in
time and allows multiple clones to open this backup. The techni­
cal innovation is allowing each clone to write any changes to a
private area; thus each clone shares the majority of data blocks
with the other clones, but the private change area makes it appear
to each clone as if it has a full-sized read/write copy of the data­
base. The downside to this technology is that it does not account
for database refreshes—any time a clone requires a newer version
of the source database, an entire new copy of the source database
has to be made.
Database Virtualization
Database virtualization takes the benefits of thin-provisioned
clones, couples it with simple management, and provides signifi­
cantly more data agility through on-demand database access.
Database virtualization takes the core technology of thinprovisioned cloning and extends it, providing the ability to:
February 2015
Automate initial source database backup, snapshots, and
redo log collection
ate repeated replay of test suites, captured workloads, and patch
Automate data retention, clearing out data older than the
designated time window
Automate provisioning a clone from any SCN or second
Provision clones from multiple sources to the same point
in time
Enable cloning of clones, branching clones, and rolling
back clones
Efficiently store all the changes from the source database
Run continually and automatically
Hardware Reduction
Database virtualization can dramatically reduce the amount
of storage required for database copies. As the majority of the
data blocks are similar, database virtualization requires storing
the changed blocks, and even those can be compressed.
Database virtualization not only saves disk space but it can
also save RAM. RAM on the virtual database hosts can be mini­
mized because virtual databases share the same data files and can
share the same blocks in the file system cache. No longer does
each copy require private memory to cache the data.
End user virtual self-service database provisioning
Be run by non-DBAs/non-sysadmins
Database virtualization technology allows a virtual database
to be made in minutes, taking up almost no space, since the vir­
tual database only creates new control files and redo log files, and
a new temporary table space. All the rest of the data is initially
shared. This allows the following advantages:
Databases on demand
Faster development
Higher-quality testing
Hardware reduction
Databases on Demand
Virtual databases can be self-provisioned in a matter of min­
utes, eliminating significant bureaucracy. Provisioning full phys­
ical copies can take weeks; virtual databases take minutes by
eliminating the data-copying time of the production database as
well as the time for requesting, discussing, processing, and allo­
cating resources. When developers need a clone, they typically
have to ask their manager, DBA, storage admin, etc. The mana­
gerial decision-making process, administrative tasks, and coor­
dination meetings often take weeks. With database virtualization
all of the overhead can be eliminated. Developers can provision
their own virtual database in minutes, with no storage overhead.
Faster Development
As the resource and operational costs of providing database
copies are eliminated with database virtualization, teams of de­
velopers can go from sharing one full physical production copy
to each having their own private copy. With a private copy of the
database, developers can change schema and metadata as fast as
they want to, instead of waiting through days or weeks of review
time to check in changes to a shared development database.
Higher-Quality Testing
More important, database virtualization speeds up QA. QA
can branch a copy of the development database in minutes and
start running test suites immediately. No longer is QA restricted
to one environment to test on, because with low resource re­
quirements of database virtualization, many parallel QA envi­
ronments can be provisioned. With database virtualization, QA
can run many tests concurrently, and the virtual databases can be
refreshed back to the original state in minutes, allowing immedi­
The NoCOUG Journal
Delphix Architecture
The Delphix Server is a software stack that implements data­
base virtualization using the Delphix file system (DxFS). The
Delphix Server automates the process of database virtualization
and management; it doesn’t require any specialized hardware and
can use any storage because Delphix maps its own file system
onto the storage. Delphix only requires an x86 box to run the
software and access to LUNs with about the same amount of the
disk space of the database to be virtualized. The source database
is backed up onto the Delphix virtual appliance via automated
RMAN APIs and the data is compressed. Delphix automates
syncing of the local copy with changes in production, freeing
data blocks outside the time retention window, and Delphix
handles the provisioning of virtual databases. A virtual database
can be provisioned from any SCN or second in time during the
retention window (typically two weeks).
Delphix File System
DxFS is a purpose-built file system optimized to efficiently
store and manage application data. DxFS merges data mapping
and file-system snapshot technology to efficiently manage and
present storage. Specifically, DxFS uses several block-aware tech­
niques that collectively eliminate over 90% of the storage re­
quired by full physical copies while also optimizing performance
of the shared storage managed by Delphix.
DxFS develops a map of unique data blocks across production
and the virtual copies created by Delphix. DxFS identifies data­
base block boundaries and compresses databases along those
database block boundaries, which is critical to maintaining per­
formance of virtual databases. Additionally, Delphix filters in­
coming data streams—eliminating temporary, empty, or scratch
blocks—driving even more data reduction.
Application Refresh with Delphix
Like a hypervisor abstracts compute resources to create vir­
tual machines, Delphix abstracts data from hardware and storage
to create virtual application instances, virtual files, and virtual
databases. By virtualizing the key components of the application
stack—the application binaries, data files, and databases—
Delphix can quickly deliver refreshed copies of the entire stack
regardless of size or complexity.
Delphix for EBS
Oracle EBS is a specific instance in which the previously de­
scribed problem arises. Oracle EBS is widespread in enterpris­
es—used extensively for ERP and CRM functions.
“dbTier” is the Oracle EBS term for everything that is related
to the database server—including Oracle Home (dbTechStack),
the EBS database itself (which is routinely tens of terabytes or
more), and the listener, among other components.
➤ EBS Database: Delphix creates a master copy of the EBS
database. This master copy is highly compressed and is
constantly updated with changes from the source database
as they occur.
➤ Oracle Home Files: Delphix creates a compressed master
copy of the Oracle Home files—the software that actually
runs the database and monitors for changes as they occur.
“appsTier” is the Oracle EBS term for everything that runs on
the application server. Delphix creates a compressed master copy
of the application binaries and configuration files. These change
relatively infrequently (when patches are applied, for example)
but are monitored for changes at regular intervals. Binaries and
files are copied on the same schedule, ensuring consistency.
Refresh Process
Delphix provisions lightweight virtual copies of all the col­
lected data (application, files, and databases) to target EBS sys­
tems. These virtual copies are extremely efficient (generally less
than 10% of the non-virtualized file size) and can be provisioned
in minutes, automatically. The target systems are the nonproduction systems that are refreshed to support ongoing de­
velopment and testing. To refresh or provision application
environments, Delphix executes the following tasks:
dbTechStack Provisioning: Delphix provisions a virtual
copy of the dbTechStack to the target database server. This
provides all the proper components for running Oracle
EBS Database Server.
Run Post-Deployment Script: Delphix runs
to configure the Oracle Home instance on the database
Start Database Listener
Run Custom Scripts: Depending on a customer’s environ­
ment, there may be additional configuration steps that are
required on the database server. After the first three steps,
custom configuration scripts can be called by Delphix.
Provision Database: When the Oracle Home is provi­
sioned and configured, a virtual copy of the database (all
the custom data) is provisioned to the database server. At
this point, the database server update is complete.
Provision appsTier binaries and files: Delphix provisions
a time-synchronized virtual copy of the application bina­
ries and configuration files to the application server.
Run Post-Deployment Script: Delphix runs
to configure the application instance on the application
Start Application
Run Custom Scripts: Depending on a customer’s environ­
ment, there may be additional configuration steps that are
required on the application server. After the previous
three steps, custom configuration scripts can be called by
Delphix. At this point, the application server is completely
When developers wish to refresh an EBS application with
Delphix, they select their instance in Delphix and click the re­
fresh button. This refresh will automatically handle all of the
above steps and fully refresh the instance in a matter of minutes.
If the developer or tester simply wants to reset the instance (re­
turn to the last clean state), the reset button can be used to clear
all new data.
By virtualizing the entire application stack, Delphix-powered
teams can deliver business results faster, with higher quality and
lower risk. Business is dependent upon IT to deliver required
applications and services, and these applications and services are
dependent upon timely, high-quality refreshes. The Delphix
Agile Data Platform enables business through fast, effective re­
fresh. s
Kyle Hailey is a technical evangelist at Delphix. Previously, he was
the designer of Embarcadero Technology’s DB Optimizer and a
principal designer for the performance pages of Oracle Enterprise
Manager. He is a member of the Oracle Oak Table, the co-author
of Oracle Insights: Tales of the Oak Table, and was a technical
editor of Oracle Wait Interface. He holds a patent in the area of
database performance diagnosis, and has been a speaker at
Hotsos, NoCOUG, RMOUG, NYOUG, Oracle OpenWorld and
DB Forum. Kyle also teaches classes around the world on Oracle
performance tuning.
Copyright © 2015, Kyle Hailey
February 2015
Many Thanks to Our Sponsors
oCOUG would like to acknowledge and thank our gen­­erous sponsors for their contributions.
Without this sponsorship, it would not be possible to present regular events while offering lowcost memberships. If your company is able to offer sponsorship at any level, please contact
NoCOUG’s president, Hanan Hit. 
Long-term event sponsorship:
Oracle Corp.
Thank you!
Gold Vendors:
➤ Axxana
➤ Database Specialists
➤ Dell Software
➤ Delphix
➤ Embarcadero Technologies
➤ SolarWinds
For information about our Gold Vendor
Program, contact the NoCOUG vendor
coordinator via email at:
[email protected]
The NoCOUG Journal
Eric Hutchinson, Treasurer
Beginning Balance
January 1, 2014
$ 56,867.13
Individual Membership
Gold Vendor Fees
Corporate Membership
Silver Vendor Fees
Conference Sponsorships
Conference Walk-in Fees
Training Day Receipts
Journal Advertising
Total Revenue
$ 51,759.12
Conference Expenses
Journal Expenses
Training Day Expenses
Board Expenses
PayPal Expenses
Software Dues
Office Expenses
Meetup Expenses
Taxes and Filing
Marketing Expenses
Total Expenses
$ 52,114.23
Ending Balance
December 31, 2014
$ 56,512.02
Many Things Oracle
by Biju Thomas
Editor’s Note: Biju publishes daily Oracle tidbits on Facebook
(fb/oraclenotes) and on Twitter (@biju_thomas). In this issue, Biju
discusses Oracle Database 12c parameters.
Program Global Area (PGA) in Oracle database is a private
memory region that contains the data and control information
for a server process. PGA is allocated per server process, used for
storing the run-time area of the cursor, sorting, bitmap merge, etc.
Automatic PGA management in Oracle Database was man­
aged by two parameters up until 11gR2, PGA_AGGREGATE_
TA R G E T a n d W O R K A R E A _ S I Z E _ P O L I C Y. W h e n
WORKAREA_SIZE_POLICY is set to AUTO (the default), auto­
matic PGA memory management is enabled; thus, there is no
need to define individual work areas (and no need to worry
PGA_AGGREGATE_TARGET specifies the target aggregate
PGA memory available to all server processes attached to the
instance. If you explicitly did not set the PGA_AGGREGATE_
TARGET parameter, Oracle sets it to 20% of the SGA or 10 MB,
whichever is greater.
If automatic memory management is enabled (MEMORY_
TARGET is set to a positive value) and PGA_AGGREGATE_
TARGET is also set to a positive value, the PGA_AGGREGATE_
TARGET value acts as the minimum value for the PGA size.
The PGA_AGGREGATE_TARGET parameter behavior or
its default values did not change in Oracle Database 12c from
11gR2, but we have a new parameter in 12c, PGA_AGGREGATE_
LIMIT, to limit the aggregate PGA memory consumed by the
Why include a limit? What was happening in 11gR2 or before
without the limit?
Well, PGA_AGGREGATE_TARGET is only a target—there
is no limit; hence, at times PGA memory usage could exceed the
target and lead to a high rate of swapping and, thus, performance
issues. To avoid this, the PGA_AGGREGATE_LIMIT parameter
sets a hard limit on the PGA size. If you do not explicitly set a
limit (provide a value), Oracle sets the default by picking the big­
gest value from this list:
2 GB,
200% of the PGA_AGGREGATE_TARGET value, or
3 MB times the value of the PROCESSES parameter.
However, it will not exceed 120% of the physical memory size
minus the total SGA size.
If you manually set the PGA_AGGREGATE_LIMIT, it must
be bigger than the default value derived by Oracle. You can set
the parameter to 0, to have no limit. If you try to set it to a
smaller value, you will get an error. In the following example, the
minimum value determined by Oracle is 909M; hence, we can­
not set it lower.
[email protected]> show parameter pga_aggregate_limit
--------------------------- ----------- -----------pga_aggregate_limit
big integer 2G
[email protected]> alter system set pga_aggregate_limit=200m ;
alter system set pga_aggregate_limit=200m
ERROR at line 1:
ORA-02097: parameter cannot be modified because specified value is invalid
ORA-00093: pga_aggregate_limit must be between 909M and 100000G
Caution: Since Oracle derives a default limit for PGA, DBAs
upgrading a database to 12c must be aware of what happens if
the process requires more memory and the total PGA exceeds
PGA_AGGREGATE_LIMIT. Once the limit is exceeded, the ses­
sions that are using the most untunable memory will have their
calls aborted. Then, if the total PGA memory usage is still over
the limit, the sessions that are using the most untunable memory
will be terminated. So, don’t be surprised if you start seeing
“PGA memory exceeded” error messages (ORA-04036) in the
alert log after upgrading to 12c.
In pre-12c database releases, VARCHAR2 and NVARCHAR2
data types were limited to 4000 bytes, while RAW data type was
limited to 2000 bytes. If there is a need to store more data in the
VARCHAR2 column, you can store it in 12c after setting the
parameter MAX_STRING_SIZE to EXTENDED. With the set­
ting MAX_STRING_SIZE to EXTENDED, you can store up to
32KB (32767 bytes) in the VARCHAR2, NVARCHAR2, and
RAW columns. Although it is an initialization parameter, there
are a few restrictions and requirements around the parameter
The default for MAX_STRING_SIZE is STANDARD in
Oracle Database 12c, where the maximum size for VARCHAR2,
NVARCHAR2, and RAW are the same as previous Oracle re­
February 2015
leases. To change the value of this parameter to extended, you
have to follow certain steps, and once you set it to EXTENDED,
there is no going back to STANDARD. To increase the maximum
size, do the following:
Shut down the database.
Start the database in UPGRADE mode.
Run script $ORACLE_HOME/rdbms/admin/utl32k.sql
connected as SYSDBA.
Shut down the database.
Start the database in NORMAL mode.
Recompile invalid objects using $ORACLE_HOME/
rdbms/admin/utlrp.sql while connected as SYSDBA.
There are several overheads associated with enabling increased
maximum size for columns, including how such columns are
stored, so make this change only if it’s absolutely necessary, after
careful testing. The feature is introduced to help migrating data
from non-Oracle databases to Oracle, where the non-Oracle
databases have a varchar maximum size of well over 4K.
Global temporary table data is stored in the temporary
tablespace (where no redo will be generated on the DML opera­
tions). When DML operations are performed on the global
temporary table, undo must be generated for read consistency.
Such undo records are written to undo tablespace, and thus redo
is generated on the undo writes.
If you perform large amounts of DML on temporary ta­
blespace data, you can reduce the amount of redo generated from
undo operations by having Oracle have two undo streams. By
setting the TEMP_UNDO_ENABLED to TRUE, a transaction
splits the undo log—the undo on the persistent tables will be
written to the undo tablespace, and undo on the temporary ta­
bles will be written to the temporary tablespace.
The parameter can be set at system level or changed at session
level. Once a session is started using either the persistent undo
area or the temporary undo area (based on the value of TEMP_
UNDO_ENABLED) for temporary tables for the first time,
changing the value of the parameter will have no effect—no
errors will be shown. If you want to enable temporary undo for
a session, make sure you enable it before doing any temporary
table DML operations. This may be a parameter that you set at
the instance level, so that you don’t need to worry about enabling
it at session level.
When an Oracle database is running on Linux/UNIX systems
you see several processes, whereas on Windows the “oracle.exe”
has several threads. By setting the THREADED_EXECUTION
parameter to TRUE, you can enable the multithreaded model on
Linux/UNIX systems. The multithreaded Oracle model enables
Oracle processes on UNIX and Linux to run as operating system
threads in separate address spaces.
In the default process model, SPID and STID columns of
V$PROCESS will have the same values, whereas in the multi­
The NoCOUG Journal
threaded model, each SPID (process) will have multiple STID
(threads) values. The EXECUTION_T YPE column in
When using threaded execution, you will not be able to log in
to the database using operating system authentication such as
“CONNECT / AS SYSDBA”. Also, for connections to work, you
must have DEDICATED_THROUGH_BROKER_<listenername>=ON set up in the listener.ora file. The database initializa­
tion parameter USE_DEDICATED_BROKER is set to TRUE
automatically when THREADED_EXECUTION=TRUE. When
this parameter is set to false (the default in process mode), the
listener spawns a dedicated server directly.
If you are working in 12c multitenant container databases,
you may be familiar with common user and local user. A com­
mon user is a database user available in the root and in every
pluggable database. Common users can connect to the root and
perform operations. The common user can perform operations
in pluggable databases if granted appropriate privileges. A local
user is a user that exist only in the pluggable database.
In, the common user name must always begin with
C##. COMMON_USER_PREFIX is a new parameter introduced
in to overcome the C## prefix restriction. In you
can define the prefix characters for the database using the
COMMON_USER_PREFIX parameter, the default is C##. Even
if you define a COMMON_USER_PREFIX value that is different
from C##, the local users still cannot begin with C##.
And, if you prefer not to have any prefix characters for com­
mon users, you can do so by setting this parameter to an empty
As the name suggests, the DB_BIG_TABLE_CACHE_
PERCENT_TARGET defines the percentage of buffer cache re­
served for automatic big table caching. The default value is 0 and
the max value is 90. The tables that are cached in the big table
cache are not defined on a per-table basis (as you define objects
in the KEEP pool). If a table is bigger than 2% of SGA, it is con­
sidered a big table.
When the big table cache is enabled, details of the big table
cache section can be viewed using the V$BT_SCAN_CACHE
view. Details such as the current ratio of the big table cache to the
buffer cache, the target ratio, and the number of objects are in
this view. The active objects in the big table cache can be queried
using the view V$BT_SCAN_OBJ_TEMPS.
This parameter and these views are introduced in as
part of the automatic big table caching feature. s
Biju Thomas is an Oracle ACE, Oracle Certified Professional, and
Certified Oracle Database SQL Expert. He is a Principal Solutions
Architect at OneNeck IT Solutions with more than 20 years of
Oracle DBA experience. He is the author of Oracle 12c and 11g
OCA, and co-author of Oracle 10g, 9i, and 8i OCP certification
guides published by Sybex/Wiley. He is a frequent presenter at
Oracle conferences and publishes articles for technical journals.
Twitter @biju_thomas.
Copyright © 2015, Biju Thomas
(continued from page 19)
(continued from page 14)
ibj.person p
where p.person_id = a.person_id
and a.city_name = 'Bern'
and p.last_name = 'Müller'
) i
where p_rowid = p.rowid
and a_rowid = a.rowid)
where rownum < 11
Although the estimates are wrong, the execution plan is ro­
bust. Before each run, the buffer cache was flushed. Therefore
the results are comparable.
I would like to thank my friend and co-inventor Björn Engsig,
who holds the U.S. Patent 2011/0119249 A1 with me. I’d also like
to thank Tim Gorman and Randolf Geist for their kind and pro­
fessional review; any flaws in this article are, of course, mine, not
theirs. As Randolf pointed out, Jonathan Lewis has documented
a similar idea in his blog [3]. It was Jonathan Lewis’s great “FAN
Hit Ratio” presentation that inspired the title of this article. s
1. Oracle Database 11g Release 1 Performance Tuning Guide
Copyright © 2015, Lothar Flatz
“Randy Shoup on eBay’s Architectural Principles” (http://
w w w. i n f o q . c o m / p r e s e n t a t i o n s / s h o u p - e b a y architectural-principles)
➤ “Scalability Best Practices: Lessons from eBay” (http://
The latest challenge to NoSQL comes from Google, which
recently created a new DBMS called F1 for its business-critical
AdWords application. Google implemented a version of Oracle
table clusters in order to avoid the join penalty. Here is a quote
from Google’s paper “F1: A Distributed SQL Database That
“In recent years, conventional wisdom in the engineering community has been that if you need a highly scalable, highthroughput data store, the only viable option is to use a NoSQL
key/value store, and to work around the lack of ACID transactional guarantees and the lack of conveniences like secondary
indexes, SQL, and so on. When we sought a replacement for
Google's MySQL data store for the AdWords product, that option was simply not feasible: the complexity of dealing with a
non-ACID data store in every part of our business logic would
be too great, and there was simply no way our business could
function without SQL queries. Instead of going NoSQL, we built
F1, a distributed relational database system that combines high
availability, the throughput and scalability of NoSQL systems,
and the functionality, usability and consistency of traditional
relational databases, including ACID transactions and SQL
queries. Google’s core AdWords business is now running completely on F1. F1 provides the SQL database functionality that
our developers are used to and our business requires. Unlike our
MySQL solution, F1 is trivial to scale up by simply adding machines.” (
The NoSQL camp put performance, scalability, and reliability
front and center but lost the opportunity to take the relational
model to the next level because—just like the relational camp—
it mistakenly believed that normalization dictates physical stor­
age choices, that non-relational APIs are forbidden by the
relational model, and that “relational” is synonymous with
ACID (Atomicity, Consistency, Isolation, and Durability).
The NoSQL camp created a number of innovations that are
disruptive in the sense used by Harvard Business School profes­
sor Clayton Christensen: functional segmentation, sharding,
replication, eventual consistency, and schemaless design. Since
these innovations are compatible with the relational model, they
should eventually be absorbed by mainstream database manage­
ment systems.
Finally, I should point out that there are very good reasons to
criticize current NoSQL products, including a lack of standards;
primitive feature sets, security, and management tools; unproven
claims; and traps for the unwary. MongoDB uses a databasewide lock for reads and writes (
manual/faq/concurrency/#what-type-of-locking-does-mongodbuse). #nuffsaid. s
Copyright © 2015, Iggy Fernandez
February 2015
Database Specialists: DBA Pro Service
Database Specialists: DBA Pro Service
• Cost-effective and flexible extension of your
IT team
• Proactive database maintenance and quick
resolution of problems by Oracle experts
• Increased database uptime
• Improved database performance
• Constant database monitoring with
Database Rx
• Onsite and offsite flexibility
• Reliable support from a stable team of DBAs
familiar with your databases
All DBA Pro services include Database Rx, our
Our Oracle Certified Professionals hav
Keeping your Oracle database systems
highly available
that Oracle syste
and skill,
alert and experience. It We
each environment is different. From large
need additional
DBA support and specialized
to small
Rx monitors
in a timely
manner, tackle s
companies that don’t require a full-time
DBA, key
offers a flexible
and other
areas: is the key. That’s why Database Specialists
in-house personnel.
After all, there’s n
service called DBA Pro. With DBA Pro, we work with you to configure a program that best suits your needs and helps you
deal with any Oracle issues that arise.Instance
You receive
basic services for development Database
systems and Rx:
prehensive plans for production and mission-critical
Messages in the alert log
All DBA Pro plans include the use of D
notification service. Depending on the
Redo log configuration
DBA Pro team receives ongoing datab
Access to experienced senior Oracle expertise when you need it
Rollback segment configuration and contention
activity and performance history, call
We work as an extension of your team to set up and manage your Oracle databases to maintain reliability, scalability,
Temporary tablespace configuration
DBAs, and monthly reports are easily
and peak performance. When you become a DBA Pro client, you are assigned a primary and secondary Database
User configuration
Specialists DBA. They’ll become intimately familiar with your systems. When you need us, just call our toll-free number
Session statistics
Database access and security
or send email for assistance from an experienced DBA during regular business hours. If you need a fuller range of
Wait events and locks
coverage with guaranteed response times, you may choose our 24 x 7 option.
Latch statistics and contention
Except for pre-authorized maintenanc
to your database or full DBA privilege
24 x 7 availability with guaranteed response time
and performance
in order
to perform
a specific task, bu
For managing mission-critical systems,
service is execution
more valuable
than being able to call on a team
of experts
to solve
to. Database
Rx uses
a database problem quickly and efficiently.
in an emergency request for help at anyustime,
knowing your
call a unique push
Index definitions
usability response time.
detailed recommendations without lo
will be answered by a Database Specialists
DBA withinand
a guaranteed
I/O and free space
DBA Pro’s mix and match service
Tablespace sizing and configuration
Database jobs
Daily review and recommendations Customer-defined
for database caremetrics and alerts
Full database administration
A Database Specialists DBA will perform a daily review of activity and alerts on your Oracle database. This aids in a proac-
tive approach to managing your database systems. After each review, you receive personalized recommendations,
By configuring acomDBA Pro service plan
ments, and action items via email. This“Database
is stored in offers
the Database
Rx Performance Portal
for future reference.
systems — including
well-rounded set of experts who can
special projects.
Monthly review and report
assist companies in a wide range of
Looking at trends and focusing on performance,
and stability
critical over time. Each
a Database
It isare
Specialists DBA will review activity and alerts on your Oracle database and prepare a comprehensive report for you.
that they are an asset to any team.”
As we work together with you as part
Proactive maintenance
in areas such as performance tuning,
Wm. Brad Gallien
When you want Database Specialists to handle ongoing proactive maintenance, we can automatically access your dataspecial projects. These can be perform
Vice President
base remotely and address issues directly — if the maintenance procedure is one you have pre-authorized us to perform.
ongoing familiarity with your system
NetForce, Inc.
You can rest assured knowing your Oracle systems are in good hands.
Onsite and offsite flexibility
You may choose to have Database Specialists consultants work onsite so they can work closely with your own DBA staff,
or you may bring us onsite only for specific
| infrastructure
S Y S T Eprojects.
I T Emay
C T Uchoose
R E |to save
D A Tmoney
A B A Son
E travel
D E S Itime
G N and
setup by having work done remotely. With DBA Pro we provide the most appropriate service program for you.
Database Spec
388 Market Street, Suite 400
Tel: 415-344-0500 | Fax: 415-344-
1 - 8 8 8 - 6 4 8 - 0 5 0 0
PERMIT NO. 11882
P.O. Box 3282
Danville, CA 94526
See what’s new in
Database Performance Analyzer 9.0
• Storage I/O analysis for better
understanding of storage performance
• Resource metric baselines to identify
normal operating thresholds
• Resource Alerts for full-alert coverage
• SQL statement analysis with expert
tuning advice
Download free trial at:
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF