Nokia Symbian SQL User manual

1
TE
RI
AL
Introduction
CO
PY
RI
GH
TE
D
MA
This book has been long in the making and written by a fairly large
and diverse group of authors. The common cause is supporting Symbian
developers in creating new phones, database-driven applications, or
porting DBMS-based application code to use Symbian SQL.
In order to understand just how important this cause is, we only
need to consider the massive growth of file storage available on today’s
mobile devices. Just a few years ago, storage space was measured in
megabytes and even desktops were shipping with 20 GB hard disks.
Modern media devices are unlikely to succeed without gigabytes of
storage. The expansion shows no signs of slowing as users become
content creators as much as content consumers. With the ability to store
thousands of songs, photos or videos on phones and the corresponding
need to catalog them, it is clear that a relational database is a necessary
component of a future-proof, scalable, mobile operating system.
Applications such as media management, as essential as they are,
only scratch the surface of what mobile databases are about. Relational
databases are an enabling technology, making possible things that we
cannot always anticipate. A great example is the Colombo Search service
discussed in Section 9.2 – a ‘desktop’ search API for Symbian that allows
searching contacts, calendar, messaging, bookmarks, notes, and more,
through a single, simple API.
1.1 Where Should You Start?
In writing this book, we envisage two major groups of readers – those
familiar with relational databases who want to re-acquire the skill on
Symbian and those familiar with Symbian C++ who want to acquire
database skills or to move from DBMS to Symbian SQL.
For both our major target groups, after this introduction, a good starting
point is Chapter 2 which offers a quick HelloWorld example using both
2
INTRODUCTION
Symbian C++ and Symbian SQL database API. Get your hands dirty
and run it on the phone. With your environment set up and a working
example, it will be considerably easier to follow the remaining text of
the book.
Prior knowledge of databases is not required – this book will get you
up to speed in the first few chapters. However, beginners in Symbian
C++ are strongly advised to read more about it before taking on this
book. There are several good books on Symbian C++ and we can warmly
recommend Symbian OS C++ for Mobile Phones Volume 3 by Richard
Harrison and Mark Shackman and Quick Recipes on Symbian OS by
Michael Aubert, the encyclopedic how-to book that no developer should
be without. Details of all the books available for Symbian developers can
be found at developer.symbian.org/wiki/index.php/Category:Book .
1.2 Symbian Terminology and Version Numbering
In June 2008, Nokia announced its intention to acquire the shares in
Symbian Ltd that it didn’t already own and create a nonprofit organization,
called the Symbian Foundation. By donating the software assets (formerly
known as Symbian OS) of Symbian Ltd, as well as its own S60 platform,
to this nonprofit entity, an open source and royalty-free mobile software
platform was created for wider industry and community collaboration
and development. The acquisition of Symbian Ltd was approved in
December 2008 and the first release of the Symbian platform occurred in
June 2009.
Most of the material in this book applies to versions of Symbian code
released before the creation of the Symbian Foundation. Table 1.1 maps
the naming and numbering of the releases of the Symbian platform,
Symbian OS and S60 covered in this book. Symbian∧ 1 is a renaming of
S60 5th Edition and Symbian OS v9.4 to denote the starting point for
the Symbian platform releases. The first version of the platform released
independently by the Symbian Foundation is Symbian∧ 2 (pronounced
‘Symbian two’).
1.3 The Relational Model and the Structured Query
Language (SQL)
For most Symbian C++ engineers looking to develop database-backed
applications, details of the relational model may be helpful but are by no
means necessary. By analogy, it is not necessary to grasp all the inner
workings of the internal combustion engine in order to be a good driver.
However, for deeper understanding of the underlying theory common
WHAT IS SYMBIAN SQL?
3
Table 1.1 Versions of the Symbian platform
Symbian
platform
S60
Symbian OS
n/a
3rd Edition
v9.1
n/a
3rd Edition, Feature Pack 1
v9.2
n/a
3rd Edition, Feature Pack 2
v9.3
Symbian∧ 1
5th Edition
v9.4
Symbian∧ 2
5th Edition, Feature Pack 1
v9.4 (and some features
back-ported from v9.5)
Symbian∧ 3
n/a1
v9.5 (and community
contributions)
Symbian∧ 4
n/a
n/a2
1 When
the Symbian Foundation is fully operational, Nokia is expected to stop marketing
S60 as an independent brand.
2 Although Symbian Ltd had a roadmap beyond v9.5, the content of future releases is
unlikely to resemble the previous plans very closely.
to all databases, Chapter 3 is an essential read. Apart from gaining a
respectable level of geekiness, concepts such as keys, views, referential
integrity, normal forms and normalization are explained there. Reading
this chapter will help you know what you are doing.
Some readers may not know about SQL. It is the language we use
to interact with databases. The Symbian platform uses an application
programming interface (API) to pass SQL queries to the database and
retrieve results. You can learn SQL in Chapter 4, which describes it in
great detail and is specifically targeted at SQLite.
Database engineers may find Chapters 3 and 4 useful for reference.
Chapter 4 in particular may be helpful for switching from other relational
database management systems to SQLite.
1.4 What Is Symbian SQL?
From its earliest days, Symbian OS was intended to support a variety
of personal information management tools. Its designers foresaw that
applications such as contact management, agendas, to-do lists, and email
would be central to the mobile user experience of the future, an idea that
is borne out today.
These applications share a common theme: they all have to store and
retrieve structured data and, perhaps more importantly, this data is often
central to the user’s everyday life. It is therefore essential that the data is
4
INTRODUCTION
protected from loss or corruption. Who would trust a phone if it lost all
its contact information when the battery ran out?
These facts led the designers of Symbian OS to include a robust set of
storage services as part of the original operating system. These included
a lightweight database called DBMS which had a slew of features and
served as a major enabler. It was used for the contacts, messaging and
multimedia applications shipping with the operating system in several
generations of mobile phones.
DBMS is famed for an extremely small footprint, especially its ability to
run in only a few kilobytes of memory. Its main advantage was that it was
built from the ground up for Symbian OS. In several scenarios – usually
with a low number of rows – DBMS produces stellar performance.
In recent years, however, shortcomings in DBMS have begun to
become apparent. Very limited support for SQL and relational operations
make porting database applications from different environments more difficult and require native applications to include complex code to perform
operations which are supported by more capable relational databases.
Several low-level operations, such as compaction and index-validity
checking, are exposed and require development attention. Although
detailed and functional, the DBMS API is somewhat different to most
modern database APIs, resulting in difficulties in porting and a skewed
learning curve. Although built with some scalability in mind, DBMS was
designed primarily for low resource usage. Thousands of records in a
database were extremely rare while memory prices were a major driving
force in device design. Today, when applications may need to store
several thousand records or more, DBMS performance does not scale
well enough to meet performance requirements.
This state of affairs triggered the search for a new database. Several candidates were considered and analyzed, and a new engine was
selected. The choice was SQLite, a database which offers many advantages: excellent SQL support, good performance, a small footprint, good
portability and unencumbered licensing – it is in the public domain.
Moreover, the SQLite project has a group of brilliant engineers and a
great community which keeps it sharp and up to date. In order to support
further development of SQLite and safeguard its future, Symbian Software
Ltd and Mozilla collaborated with the SQLite developers to found the
SQLite Consortium.1 Today, the consortium has expanded to include
Adobe and Bloomberg.
Two significant Symbian requirements could not be accommodated
by SQLite alone. First, Symbian needs the ability to securely share a
database between clients: only clients that satisfy a per-database security
policy can be allowed access. Second, in standard SQLite, if multiple
1 www.sqlite.org/consortium.html
WHAT IS SYMBIAN SQL?
5
clients share a single database then each of the clients has a private
cache of the database in its address space. This is not appropriate for
a resource-constrained mobile device; Symbian needs a model where
multiple clients share a single cache.
These needs led to the design and implementation of Symbian SQL,
which integrated SQLite within a Symbian client–server framework as
shown in Figure 1.1. Client–server IPC is the basis for the Symbian
platform security framework and a server process is the ideal mechanism
for securely sharing access to a single cache. For the design to become
a reality, Symbian worked with the SQLite developers to evolve the
database to meet the requirements. Changes were needed to enable
SQLite to share a cache between multiple connections and to improve
operational characteristics under low-memory conditions.
Application Process
Symbian SQL Process
File Server Process
Application Code
Symbian SQL
IPC Server
File Server IPC
Symbian SQL
Client (DLL)
SQ Lite Engine
(DLL)
Figure 1.1
File System
Symbian SQL architecture
A number of client APIs were considered, including the native SQLite
C API. The final choice was a native Symbian C++ API which supported
Symbian idioms, such as descriptors and asynchronous requests. Engineers familiar with databases will immediately recognize the connection
and statement classes and find that they have the expected functionality.
We focus on the API in Chapter 6 where all is explained in detail.
Symbian SQL was first delivered on devices based on Symbian OS
v9.3. Demand was sufficient for it to be offered as an installable version
backported to Symbian OS v9.2. The C++ API was ideal for early
Symbian clients but it was recognized that the standard SQLite C API
would be a valuable addition to facilitate porting SQLite-based, open
source applications. The C API was introduced in a later release. It is
covered only briefly in Chapter 7 because it is well documented online,
especially at www.sqlite.org .
6
INTRODUCTION
After the initial release of Symbian SQL, collaboration with the SQLite
developers continued to further enhance performance and functionality.
Chapter 7 explains some of these enhancements in detail.
1.5 A Well-Oiled Machine
One of the main goals of this book is to help readers to get the most
out of Symbian SQL. Symbian SQL can offer excellent performance,
however at times this requires a level of expertise and some work on
optimizing the application or database schema, or even on tuning system
parameters. In order to help in the process, Chapter 8 is entirely dedicated
to performance tuning.
Finally, we wanted to share some success stories. We have recruited
three happy engineers that have earned bragging rights in getting their
applications to fly with Symbian SQL. Chapter 9 tells their stories – if you
use a Symbian phone, you have benefited from their work too.
1.6 Tools and Troubleshooting
SQLite is great for learning and experimenting. Essentially, all that is
necessary to start experimenting is one executable (sqlite3.exe) that
is available for a variety of platforms. Using this tool, developers can
create databases, experiment with queries and analyze performance.
This is very beneficial because it eliminates the need for installing and
maintaining a separate database server or running in an OS emulator.
Once prototyping is done, we are faced with a variety of questions on
specifics, such as the compaction (auto-vacuum) policy, security options,
and page size.
A collection of common problems, including programming and execution errors, is included in Appendix A.
Further to this, Symbian SQL databases contain a metadata table. To
cater for all these requirements, Symbian provides a tool, called SDB, that
offers a systematic way of configuring all database options. Appendix B
covers the SDB tool in more detail.
1.7 Further Reading and Resources
We originally started writing this book as part of the Symbian Press
Technology series back in 2008. Since that time, a lot has changed in
the mobile software world, for us in Symbian (now Nokia) and for those
working with Symbian devices.
FURTHER READING AND RESOURCES
7
Determining the content of a technical book in a fast-moving industry
is never easy but when organizations and strategies are changing, it
becomes extremely difficult. For that reason, several sections of this book
have been revised to record the most relevant and up-to-date information.
Nevertheless, we anticipate further changes, if only to some of the URLs
we reference. For that reason, you can find a wiki page for this book
at developer.symbian.org/wiki/index.php/Inside Symbian SQL, which
also hosts any example code accompanying the book and records errata
and updates to the information presented in this version of the text.
A variety of additional resources about Symbian SQL can be found
on the Symbian developer site, developer.symbian.org , such as the API
reference for the Symbian platform and support from peer developers via
a number of discussion forums.
The official SQLite website at www.sqlite.org provides various articles
detailing the behavior of the underlying SQLite engine. These include:
• SQL syntax
• atomic commit
• locking and concurrency
• an overview of the optimizer.
The official SQLite website also offers support for the underlying SQLite
engine through mailing lists that announce future releases and enable
user discussions.