SQLAlchemy Documentation

SQLAlchemy Documentation
SQLAlchemy Documentation
Release 0.8.7
Mike Bayer
April 06, 2016
Contents
1
2
Overview
1.1 Documentation Overview . . . . . . . . . . . . . . .
1.2 Code Examples . . . . . . . . . . . . . . . . . . . . .
1.3 Installation Guide . . . . . . . . . . . . . . . . . . .
1.3.1
Supported Platforms . . . . . . . . . . . . .
1.3.2
Supported Installation Methods . . . . . . . .
1.3.3
Install via easy_install or pip . . . . . . . . .
1.3.4
Installing using setup.py . . . . . . . . . . .
1.3.5
Installing the C Extensions . . . . . . . . . .
1.3.6
Installing on Python 3 . . . . . . . . . . . . .
1.3.7
Installing a Database API . . . . . . . . . . .
1.3.8
Checking the Installed SQLAlchemy Version
1.4 0.7 to 0.8 Migration . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
3
4
4
4
4
4
5
5
5
6
6
6
6
SQLAlchemy ORM
2.1 Object Relational Tutorial . . . . . . . . . . . .
2.1.1
Version Check . . . . . . . . . . . . . .
2.1.2
Connecting . . . . . . . . . . . . . . .
2.1.3
Declare a Mapping . . . . . . . . . . .
2.1.4
Create an Instance of the Mapped Class
2.1.5
Creating a Session . . . . . . . . . . .
2.1.6
Adding New Objects . . . . . . . . . .
2.1.7
Rolling Back . . . . . . . . . . . . . .
2.1.8
Querying . . . . . . . . . . . . . . . .
2.1.9
Building a Relationship . . . . . . . . .
2.1.10 Working with Related Objects . . . . .
2.1.11 Querying with Joins . . . . . . . . . . .
2.1.12 Eager Loading . . . . . . . . . . . . .
2.1.13 Deleting . . . . . . . . . . . . . . . . .
2.1.14 Building a Many To Many Relationship
2.1.15 Further Reference . . . . . . . . . . . .
2.2 Mapper Configuration . . . . . . . . . . . . . .
2.2.1
Classical Mappings . . . . . . . . . . .
2.2.2
Customizing Column Properties . . . .
2.2.3
Deferred Column Loading . . . . . . .
2.2.4
SQL Expressions as Mapped Attributes
2.2.5
Changing Attribute Behavior . . . . . .
2.2.6
Composite Column Types . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
7
7
7
8
8
11
12
13
14
15
23
25
26
31
33
36
39
39
39
40
44
47
50
54
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
i
2.2.7
Mapping a Class against Multiple Tables . . . . . . . . . . . . . .
2.2.8
Mapping a Class against Arbitrary Selects . . . . . . . . . . . . .
2.2.9
Multiple Mappers for One Class . . . . . . . . . . . . . . . . . .
2.2.10 Constructors and Object Initialization . . . . . . . . . . . . . . .
2.2.11 Configuring a Version Counter . . . . . . . . . . . . . . . . . . .
2.2.12 Class Mapping API . . . . . . . . . . . . . . . . . . . . . . . . .
2.3 Relationship Configuration . . . . . . . . . . . . . . . . . . . . . . . . . .
2.3.1
Basic Relational Patterns . . . . . . . . . . . . . . . . . . . . . .
2.3.2
Adjacency List Relationships . . . . . . . . . . . . . . . . . . . .
2.3.3
Linking Relationships with Backref . . . . . . . . . . . . . . . .
2.3.4
Configuring how Relationship Joins . . . . . . . . . . . . . . . .
2.3.5
Rows that point to themselves / Mutually Dependent Rows . . . .
2.3.6
Mutable Primary Keys / Update Cascades . . . . . . . . . . . . .
2.3.7
Relationships API . . . . . . . . . . . . . . . . . . . . . . . . . .
2.4 Collection Configuration and Techniques . . . . . . . . . . . . . . . . . .
2.4.1
Working with Large Collections . . . . . . . . . . . . . . . . . .
2.4.2
Customizing Collection Access . . . . . . . . . . . . . . . . . . .
2.4.3
Custom Collection Implementations . . . . . . . . . . . . . . . .
2.4.4
Collection Internals . . . . . . . . . . . . . . . . . . . . . . . . .
2.5 Mapping Class Inheritance Hierarchies . . . . . . . . . . . . . . . . . . .
2.5.1
Joined Table Inheritance . . . . . . . . . . . . . . . . . . . . . .
2.5.2
Single Table Inheritance . . . . . . . . . . . . . . . . . . . . . .
2.5.3
Concrete Table Inheritance . . . . . . . . . . . . . . . . . . . . .
2.5.4
Using Relationships with Inheritance . . . . . . . . . . . . . . . .
2.5.5
Using Inheritance with Declarative . . . . . . . . . . . . . . . . .
2.6 Using the Session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.6.1
What does the Session do ? . . . . . . . . . . . . . . . . . . . . .
2.6.2
Getting a Session . . . . . . . . . . . . . . . . . . . . . . . . . .
2.6.3
Using the Session . . . . . . . . . . . . . . . . . . . . . . . . . .
2.6.4
Cascades . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.6.5
Managing Transactions . . . . . . . . . . . . . . . . . . . . . . .
2.6.6
Embedding SQL Insert/Update Expressions into a Flush . . . . .
2.6.7
Using SQL Expressions with Sessions . . . . . . . . . . . . . . .
2.6.8
Joining a Session into an External Transaction . . . . . . . . . . .
2.6.9
Contextual/Thread-local Sessions . . . . . . . . . . . . . . . . .
2.6.10 Partitioning Strategies . . . . . . . . . . . . . . . . . . . . . . . .
2.6.11 Sessions API . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.7 Querying . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.7.1
The Query Object . . . . . . . . . . . . . . . . . . . . . . . . . .
2.7.2
ORM-Specific Query Constructs . . . . . . . . . . . . . . . . . .
2.8 Relationship Loading Techniques . . . . . . . . . . . . . . . . . . . . . .
2.8.1
Using Loader Strategies: Lazy Loading, Eager Loading . . . . . .
2.8.2
Default Loading Strategies . . . . . . . . . . . . . . . . . . . . .
2.8.3
The Zen of Eager Loading . . . . . . . . . . . . . . . . . . . . .
2.8.4
What Kind of Loading to Use ? . . . . . . . . . . . . . . . . . . .
2.8.5
Routing Explicit Joins/Statements into Eagerly Loaded Collections
2.8.6
Relation Loader API . . . . . . . . . . . . . . . . . . . . . . . .
2.9 ORM Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.9.1
Attribute Events . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.9.2
Mapper Events . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.9.3
Instance Events . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.9.4
Session Events . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.9.5
Instrumentation Events . . . . . . . . . . . . . . . . . . . . . . .
2.10 ORM Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
ii
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
57
58
59
59
60
62
74
74
79
83
87
91
94
95
103
103
105
108
115
117
117
124
125
127
128
128
129
129
131
143
149
153
153
154
154
159
161
177
177
195
199
199
201
202
204
205
206
210
210
212
220
222
227
227
2.10.1 Association Proxy . . . . . . . . .
2.10.2 Declarative . . . . . . . . . . . .
2.10.3 Mutation Tracking . . . . . . . .
2.10.4 Ordering List . . . . . . . . . . .
2.10.5 Horizontal Sharding . . . . . . . .
2.10.6 Hybrid Attributes . . . . . . . . .
2.10.7 Alternate Class Instrumentation .
2.11 Examples . . . . . . . . . . . . . . . . . .
2.11.1 Adjacency List . . . . . . . . . .
2.11.2 Associations . . . . . . . . . . . .
2.11.3 Attribute Instrumentation . . . . .
2.11.4 Dogpile Caching . . . . . . . . .
2.11.5 Directed Graphs . . . . . . . . . .
2.11.6 Dynamic Relations as Dictionaries
2.11.7 Generic Associations . . . . . . .
2.11.8 Horizontal Sharding . . . . . . . .
2.11.9 Inheritance Mappings . . . . . . .
2.11.10 Large Collections . . . . . . . . .
2.11.11 Nested Sets . . . . . . . . . . . .
2.11.12 Polymorphic Associations . . . .
2.11.13 PostGIS Integration . . . . . . . .
2.11.14 Versioned Objects . . . . . . . . .
2.11.15 Vertical Attribute Mapping . . . .
2.11.16 XML Persistence . . . . . . . . .
2.12 ORM Exceptions . . . . . . . . . . . . . .
2.13 ORM Internals . . . . . . . . . . . . . . .
3
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
227
238
261
268
272
272
284
285
286
286
286
286
288
288
288
288
289
289
289
289
289
290
291
291
292
293
SQLAlchemy Core
3.1 SQL Expression Language Tutorial . . . . . .
3.1.1
Version Check . . . . . . . . . . . . .
3.1.2
Connecting . . . . . . . . . . . . . .
3.1.3
Define and Create Tables . . . . . . .
3.1.4
Insert Expressions . . . . . . . . . . .
3.1.5
Executing . . . . . . . . . . . . . . .
3.1.6
Executing Multiple Statements . . . .
3.1.7
Selecting . . . . . . . . . . . . . . .
3.1.8
Operators . . . . . . . . . . . . . . .
3.1.9
Conjunctions . . . . . . . . . . . . .
3.1.10 Using Text . . . . . . . . . . . . . . .
3.1.11 Using Aliases . . . . . . . . . . . . .
3.1.12 Using Joins . . . . . . . . . . . . . .
3.1.13 Everything Else . . . . . . . . . . . .
3.1.14 Inserts, Updates and Deletes . . . . .
3.1.15 Further Reference . . . . . . . . . . .
3.2 SQL Statements and Expressions API . . . . .
3.2.1
Column Elements and Expressions . .
3.2.2
Selectables, Tables, FROM objects . .
3.2.3
Insert, Updates, Deletes . . . . . . . .
3.2.4
SQL and Generic Functions . . . . .
3.2.5
Column and Data Types . . . . . . .
3.3 Schema Definition Language . . . . . . . . .
3.3.1
Describing Databases with MetaData
3.3.2
Reflecting Database Objects . . . . .
3.3.3
Column Insert/Update Defaults . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
327
327
327
328
328
330
330
331
332
334
336
337
338
340
341
349
352
352
352
389
434
454
458
485
485
510
515
iii
3.4
3.5
3.6
3.7
3.8
3.9
3.10
3.11
3.12
3.13
3.14
iv
3.3.4
Defining Constraints and Indexes . . . . . . . . . . . . . .
3.3.5
Customizing DDL . . . . . . . . . . . . . . . . . . . . .
Engine Configuration . . . . . . . . . . . . . . . . . . . . . . . . .
3.4.1
Supported Databases . . . . . . . . . . . . . . . . . . . .
3.4.2
Database Urls . . . . . . . . . . . . . . . . . . . . . . . .
3.4.3
Engine Creation API . . . . . . . . . . . . . . . . . . . .
3.4.4
Pooling . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.4.5
Custom DBAPI connect() arguments . . . . . . . . . . . .
3.4.6
Configuring Logging . . . . . . . . . . . . . . . . . . . .
Working with Engines and Connections . . . . . . . . . . . . . . .
3.5.1
Basic Usage . . . . . . . . . . . . . . . . . . . . . . . . .
3.5.2
Using Transactions . . . . . . . . . . . . . . . . . . . . .
3.5.3
Understanding Autocommit . . . . . . . . . . . . . . . .
3.5.4
Connectionless Execution, Implicit Execution . . . . . . .
3.5.5
Using the Threadlocal Execution Strategy . . . . . . . . .
3.5.6
Registering New Dialects . . . . . . . . . . . . . . . . . .
3.5.7
Connection / Engine API . . . . . . . . . . . . . . . . . .
Connection Pooling . . . . . . . . . . . . . . . . . . . . . . . . .
3.6.1
Connection Pool Configuration . . . . . . . . . . . . . . .
3.6.2
Switching Pool Implementations . . . . . . . . . . . . . .
3.6.3
Using a Custom Connection Function . . . . . . . . . . .
3.6.4
Constructing a Pool . . . . . . . . . . . . . . . . . . . . .
3.6.5
Pool Events . . . . . . . . . . . . . . . . . . . . . . . . .
3.6.6
Dealing with Disconnects . . . . . . . . . . . . . . . . . .
3.6.7
API Documentation - Available Pool Implementations . .
3.6.8
Pooling Plain DB-API Connections . . . . . . . . . . . .
Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.7.1
Event Registration . . . . . . . . . . . . . . . . . . . . .
3.7.2
Targets . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.7.3
Modifiers . . . . . . . . . . . . . . . . . . . . . . . . . .
3.7.4
Event Reference . . . . . . . . . . . . . . . . . . . . . . .
3.7.5
API Reference . . . . . . . . . . . . . . . . . . . . . . .
Core Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.8.1
Connection Pool Events . . . . . . . . . . . . . . . . . .
3.8.2
SQL Execution and Connection Events . . . . . . . . . .
3.8.3
Schema Events . . . . . . . . . . . . . . . . . . . . . . .
Custom SQL Constructs and Compilation Extension . . . . . . . .
3.9.1
Synopsis . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.9.2
Dialect-specific compilation rules . . . . . . . . . . . . .
3.9.3
Compiling sub-elements of a custom expression construct
3.9.4
Enabling Autocommit on a Construct . . . . . . . . . . .
3.9.5
Changing the default compilation of existing constructs . .
3.9.6
Changing Compilation of Types . . . . . . . . . . . . . .
3.9.7
Subclassing Guidelines . . . . . . . . . . . . . . . . . . .
3.9.8
Further Examples . . . . . . . . . . . . . . . . . . . . . .
Runtime Inspection API . . . . . . . . . . . . . . . . . . . . . . .
3.10.1 Available Inspection Targets . . . . . . . . . . . . . . . .
Expression Serializer Extension . . . . . . . . . . . . . . . . . . .
Deprecated Event Interfaces . . . . . . . . . . . . . . . . . . . . .
3.12.1 Execution, Connection and Cursor Events . . . . . . . . .
3.12.2 Connection Pool Events . . . . . . . . . . . . . . . . . .
Core Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . .
Core Internals . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
523
532
542
542
542
544
549
549
549
550
550
552
553
553
555
556
557
570
571
571
571
572
573
573
575
578
579
579
580
580
580
581
581
581
583
588
591
591
592
592
593
594
594
594
595
597
598
598
599
599
600
601
604
4
5
6
Dialects
4.1 Included Dialects . . . . . . . . . .
4.1.1
Drizzle . . . . . . . . . .
4.1.2
Firebird . . . . . . . . . .
4.1.3
Informix . . . . . . . . . .
4.1.4
Microsoft SQL Server . .
4.1.5
MySQL . . . . . . . . . .
4.1.6
Oracle . . . . . . . . . . .
4.1.7
PostgreSQL . . . . . . . .
4.1.8
SQLite . . . . . . . . . . .
4.1.9
Sybase . . . . . . . . . . .
4.2 External Dialects . . . . . . . . . .
4.2.1
Production Ready . . . . .
4.2.2
Experimental / Incomplete
4.2.3
Attic . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
619
619
619
623
626
626
638
657
665
683
690
692
692
692
692
Changes and Migration
5.1 Current Migration Guide . . . . . . . . . .
5.1.1
What’s New in SQLAlchemy 0.8?
5.2 Change logs . . . . . . . . . . . . . . . .
5.2.1
0.8 Changelog . . . . . . . . . . .
5.2.2
0.7 Changelog . . . . . . . . . . .
5.2.3
0.6 Changelog . . . . . . . . . . .
5.2.4
0.5 Changelog . . . . . . . . . . .
5.2.5
0.4 Changelog . . . . . . . . . . .
5.2.6
0.3 Changelog . . . . . . . . . . .
5.2.7
0.2 Changelog . . . . . . . . . . .
5.2.8
0.1 Changelog . . . . . . . . . . .
5.3 Older Migration Guides . . . . . . . . . .
5.3.1
What’s New in SQLAlchemy 0.7?
5.3.2
What’s New in SQLAlchemy 0.6?
5.3.3
What’s new in SQLAlchemy 0.5?
5.3.4
What’s new in SQLAlchemy 0.4?
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
695
695
695
717
717
750
785
828
858
890
912
919
925
925
942
956
965
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Indices and tables
977
Python Module Index
979
v
vi
SQLAlchemy Documentation, Release 0.8.7
Full table of contents. For a high level overview of all documentation, see index_toplevel.
Contents
1
SQLAlchemy Documentation, Release 0.8.7
2
Contents
CHAPTER 1
Overview
The SQLAlchemy SQL Toolkit and Object Relational Mapper is a comprehensive set of tools for working with
databases and Python. It has several distinct areas of functionality which can be used individually or combined
together. Its major components are illustrated in below, with component dependencies organized into layers:
Above, the two most significant front-facing portions of SQLAlchemy are the Object Relational Mapper and the
SQL Expression Language. SQL Expressions can be used independently of the ORM. When using the ORM, the
SQL Expression language remains part of the public facing API as it is used within object-relational configurations
and queries.
3
SQLAlchemy Documentation, Release 0.8.7
1.1 Documentation Overview
The documentation is separated into three sections: SQLAlchemy ORM, SQLAlchemy Core, and Dialects.
In SQLAlchemy ORM, the Object Relational Mapper is introduced and fully described. New users should begin with
the Object Relational Tutorial. If you want to work with higher-level SQL which is constructed automatically for you,
as well as management of Python objects, proceed to this tutorial.
In SQLAlchemy Core, the breadth of SQLAlchemy’s SQL and database integration and description services are documented, the core of which is the SQL Expression language. The SQL Expression Language is a toolkit all its
own, independent of the ORM package, which can be used to construct manipulable SQL expressions which can be
programmatically constructed, modified, and executed, returning cursor-like result sets. In contrast to the ORM’s
domain-centric mode of usage, the expression language provides a schema-centric usage paradigm. New users should
begin here with SQL Expression Language Tutorial. SQLAlchemy engine, connection, and pooling services are also
described in SQLAlchemy Core.
In Dialects, reference documentation for all provided database and DBAPI backends is provided.
1.2 Code Examples
Working code examples, mostly regarding the ORM, are included in the SQLAlchemy distribution. A description of
all the included example applications is at Examples.
There is also a wide variety of examples involving both core SQLAlchemy constructs as well as the ORM on the wiki.
See Theatrum Chemicum.
1.3 Installation Guide
1.3.1 Supported Platforms
SQLAlchemy has been tested against the following platforms:
• cPython since version 2.5, through the 2.xx series
• cPython version 3, throughout all 3.xx series
• Jython 2.5 or greater
• Pypy 1.5 or greater
Changed in version 0.8: Python 2.5 is now the minimum Python version supported.
1.3.2 Supported Installation Methods
SQLAlchemy supports installation using standard Python “distutils” or “setuptools” methodologies. An overview of
potential setups is as follows:
• Plain Python Distutils - SQLAlchemy can be installed with a clean Python install using the services provided
via Python Distutils, using the setup.py script. The C extensions as well as Python 3 builds are supported.
• Standard Setuptools - When using setuptools, SQLAlchemy can be installed via setup.py or
easy_install, and the C extensions are supported.
• pip - pip is an installer that rides on top of setuptools or distribute, replacing the usage of
easy_install. It is often preferred for its simpler mode of usage.
4
Chapter 1. Overview
SQLAlchemy Documentation, Release 0.8.7
1.3.3 Install via easy_install or pip
When easy_install or pip is available, the distribution can be downloaded from Pypi and installed in one step:
easy_install SQLAlchemy
Or with pip:
pip install SQLAlchemy
This command will download the latest version of SQLAlchemy from the Python Cheese Shop and install it to your
system.
Note: Beta releases of SQLAlchemy may not be present on Pypi, and may instead require a direct download first.
1.3.4 Installing using setup.py
Otherwise, you can install from the distribution using the setup.py script:
python setup.py install
1.3.5 Installing the C Extensions
SQLAlchemy includes C extensions which provide an extra speed boost for dealing with result sets. Currently, the
extensions are only supported on the 2.xx series of cPython, not Python 3 or Pypy.
setup.py will automatically build the extensions if an appropriate platform is detected. If the build of the C extensions fails, due to missing compiler or other issue, the setup process will output a warning message, and re-run the
build without the C extensions, upon completion reporting final status.
To run the build/install without even attempting to compile the C extensions, the DISABLE_SQLALCHEMY_CEXT
environment variable may be specified. The use case for this is either for special testing circumstances, or in the rare
case of compatibility/build issues not overcome by the usual “rebuild” mechanism:
# *** only in SQLAlchemy 0.9.4 / 0.8.6 or greater ***
export DISABLE_SQLALCHEMY_CEXT=1; python setup.py install
New in version 0.9.4,0.8.6:
Support for disabling the build of C extensions using the
DISABLE_SQLALCHEMY_CEXT environment variable has been added. This allows control of C extension
building whether or not setuptools is available, and additionally works around the fact that setuptools will possibly be
removing support for command-line switches such as --without-extensions in a future release.
For versions of SQLAlchemy prior to 0.9.4 or 0.8.6, the --without-cextensions option may be used to disable
the attempt to build C extensions, provided setupools is in use, and provided the Feature construct is supported by
the installed version of setuptools:
python setup.py --without-cextensions install
Or with pip:
pip install --global-option='--without-cextensions' SQLAlchemy
1.3. Installation Guide
5
SQLAlchemy Documentation, Release 0.8.7
1.3.6 Installing on Python 3
SQLAlchemy ships as Python 2 code. For Python 3 usage, the setup.py script will invoke the Python 2to3 tool on
the build, plugging in an extra “preprocessor” as well. The 2to3 step works with Python distutils (part of the standard
Python install) and Distribute - it will not work with a non-Distribute setuptools installation.
1.3.7 Installing a Database API
SQLAlchemy is designed to operate with a DBAPI implementation built for a particular database, and includes support
for the most popular databases. The individual database sections in Dialects enumerate the available DBAPIs for each
database, including external links.
1.3.8 Checking the Installed SQLAlchemy Version
This documentation covers SQLAlchemy version 0.8. If you’re working on a system that already has SQLAlchemy
installed, check the version from your Python prompt like this:
>>> import sqlalchemy
>>> sqlalchemy.__version__ # doctest: +SKIP
0.8.0
1.4 0.7 to 0.8 Migration
Notes on what’s changed from 0.7 to 0.8 is available here at What’s New in SQLAlchemy 0.8?.
6
Chapter 1. Overview
CHAPTER 2
SQLAlchemy ORM
Here, the Object Relational Mapper is introduced and fully described. If you want to work with higher-level SQL
which is constructed automatically for you, as well as automated persistence of Python objects, proceed first to the
tutorial.
2.1 Object Relational Tutorial
The SQLAlchemy Object Relational Mapper presents a method of associating user-defined Python classes with
database tables, and instances of those classes (objects) with rows in their corresponding tables. It includes a system that transparently synchronizes all changes in state between objects and their related rows, called a unit of work,
as well as a system for expressing database queries in terms of the user defined classes and their defined relationships
between each other.
The ORM is in contrast to the SQLAlchemy Expression Language, upon which the ORM is constructed. Whereas the
SQL Expression Language, introduced in SQL Expression Language Tutorial, presents a system of representing the
primitive constructs of the relational database directly without opinion, the ORM presents a high level and abstracted
pattern of usage, which itself is an example of applied usage of the Expression Language.
While there is overlap among the usage patterns of the ORM and the Expression Language, the similarities are more
superficial than they may at first appear. One approaches the structure and content of data from the perspective of
a user-defined domain model which is transparently persisted and refreshed from its underlying storage model. The
other approaches it from the perspective of literal schema and SQL expression representations which are explicitly
composed into messages consumed individually by the database.
A successful application may be constructed using the Object Relational Mapper exclusively. In advanced situations,
an application constructed with the ORM may make occasional usage of the Expression Language directly in certain
areas where specific database interactions are required.
The following tutorial is in doctest format, meaning each >>> line represents something you can type at a Python
command prompt, and the following text represents the expected return value.
2.1.1 Version Check
A quick check to verify that we are on at least version 0.8 of SQLAlchemy:
>>> import sqlalchemy
>>> sqlalchemy.__version__
0.8.0
7
SQLAlchemy Documentation, Release 0.8.7
2.1.2 Connecting
For this tutorial we will use an in-memory-only SQLite database. To connect we use create_engine():
>>> from sqlalchemy import create_engine
>>> engine = create_engine('sqlite:///:memory:', echo=True)
The echo flag is a shortcut to setting up SQLAlchemy logging, which is accomplished via Python’s standard
logging module. With it enabled, we’ll see all the generated SQL produced. If you are working through this
tutorial and want less output generated, set it to False. This tutorial will format the SQL behind a popup window so
it doesn’t get in our way; just click the “SQL” links to see what’s being generated.
The return value of create_engine() is an instance of Engine, and it represents the core interface to the
database, adapted through a dialect that handles the details of the database and DBAPI in use. In this case the SQLite
dialect will interpret instructions to the Python built-in sqlite3 module.
The Engine has not actually tried to connect to the database yet; that happens only the first time it is asked to perform
a task against the database. We can illustrate this by asking it to perform a simple SELECT statement:
>>> engine.execute("select 1").scalar()
select 1
()
1
As the Engine.execute() method is called, the Engine establishes a connection to the SQLite database, which
is then used to emit the SQL. The connection is then returned to an internal connection pool where it will be reused on
subsequent statement executions. While we illustrate direct usage of the Engine here, this isn’t typically necessary
when using the ORM, where the Engine, once created, is used behind the scenes by the ORM as we’ll see shortly.
See also:
Database Urls - includes examples of create_engine() connecting to several kinds of databases with links to
more information.
2.1.3 Declare a Mapping
When using the ORM, the configurational process starts by describing the database tables we’ll be dealing with, and
then by defining our own classes which will be mapped to those tables. In modern SQLAlchemy, these two tasks
are usually performed together, using a system known as Declarative, which allows us to create classes that include
directives to describe the actual database table they will be mapped to.
Classes mapped using the Declarative system are defined in terms of a base class which maintains a catalog of classes
and tables relative to that base - this is known as the declarative base class. Our application will usually have just one
instance of this base in a commonly imported module. We create the base class using the declarative_base()
function, as follows:
>>> from sqlalchemy.ext.declarative import declarative_base
>>> Base = declarative_base()
Now that we have a “base”, we can define any number of mapped classes in terms of it. We will start with just a single
table called users, which will store records for the end-users using our application. A new class called User will
be the class to which we map this table. The imports we’ll need to accomplish this include objects that represent the
components of our table, including the Column class which represents a database column, as well as the Integer
and String classes that represent basic datatypes used in columns:
>>> from sqlalchemy import Column, Integer, String
>>> class User(Base):
8
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
...
...
...
...
...
...
...
...
...
...
...
...
...
...
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
name = Column(String)
fullname = Column(String)
password = Column(String)
def __init__(self, name, fullname, password):
self.name = name
self.fullname = fullname
self.password = password
def __repr__(self):
return "<User('%s','%s', '%s')>" % (self.name, self.fullname, self.password)
The above User class establishes details about the table being mapped, including the name of the table denoted by
the __tablename__ attribute, a set of columns id, name, fullname and password, where the id column will
also be the primary key of the table. While its certainly possible that some database tables don’t have primary key
columns (as is also the case with views, which can also be mapped), the ORM in order to actually map to a particular
table needs there to be at least one column denoted as a primary key column; multiple-column, i.e. composite, primary
keys are of course entirely feasible as well.
We define a constructor via __init__() and also a __repr__() method - both are optional. The class of course
can have any number of other methods and attributes as required by the application, as it’s basically just a plain
Python class. Inheriting from Base is also only a requirement of the declarative configurational system, which itself
is optional and relatively open ended; at its core, the SQLAlchemy ORM only requires that a class be a so-called “new
style class”, that is, it inherits from object in Python 2, in order to be mapped. All classes in Python 3 are “new
style” classes.
The Non Opinionated Philosophy
In our User mapping example, it was required that we identify the name of the table in use, as well as the
names and characteristics of all columns which we care about, including which column or columns represent the
primary key, as well as some basic information about the types in use. SQLAlchemy never makes assumptions
about these decisions - the developer must always be explicit about specific conventions in use. However, that
doesn’t mean the task can’t be automated. While this tutorial will keep things explicit, developers are encouraged
to make use of helper functions as well as “Declarative Mixins” to automate their tasks in large scale applications.
The section Mixin and Custom Base Classes introduces many of these techniques.
With our User class constructed via the Declarative system, we have defined information about our table, known as
table metadata, as well as a user-defined class which is linked to this table, known as a mapped class. Declarative has
provided for us a shorthand system for what in SQLAlchemy is called a “Classical Mapping”, which specifies these
two units separately and is discussed in Classical Mappings. The table is actually represented by a datastructure known
as Table, and the mapping represented by a Mapper object generated by a function called mapper(). Declarative
performs both of these steps for us, making available the Table it has created via the __table__ attribute:
>>> User.__table__
Table('users', MetaData(None),
Column('id', Integer(), table=<users>, primary_key=True, nullable=False),
Column('name', String(), table=<users>),
Column('fullname', String(), table=<users>),
Column('password', String(), table=<users>), schema=None)
and while rarely needed, making available the Mapper object via the __mapper__ attribute:
2.1. Object Relational Tutorial
9
SQLAlchemy Documentation, Release 0.8.7
>>> User.__mapper__
<Mapper at 0x...; User>
The Declarative base class also contains a catalog of all the Table objects that have been defined called MetaData,
available via the .metadata attribute. In this example, we are defining new tables that have yet to be created in our
SQLite database, so one helpful feature the MetaData object offers is the ability to issue CREATE TABLE statements
to the database for all tables that don’t yet exist. We illustrate this by calling the MetaData.create_all()
method, passing in our Engine as a source of database connectivity. We will see that special commands are first
emitted to check for the presence of the users table, and following that the actual CREATE TABLE statement:
>>> Base.metadata.create_all(engine) # doctest:+ELLIPSIS,+NORMALIZE_WHITESPACE
PRAGMA table_info("users")
()
CREATE TABLE users (
id INTEGER NOT NULL,
name VARCHAR,
fullname VARCHAR,
password VARCHAR,
PRIMARY KEY (id)
)
()
COMMIT
10
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
Minimal Table Descriptions vs. Full Descriptions
Users familiar with the syntax of CREATE TABLE may notice that the VARCHAR columns were generated
without a length; on SQLite and Postgresql, this is a valid datatype, but on others, it’s not allowed. So if running
this tutorial on one of those databases, and you wish to use SQLAlchemy to issue CREATE TABLE, a “length”
may be provided to the String type as below:
Column(String(50))
The length field on String, as well as similar precision/scale fields available on Integer, Numeric, etc.
are not referenced by SQLAlchemy other than when creating tables.
Additionally, Firebird and Oracle require sequences to generate new primary key identifiers, and SQLAlchemy
doesn’t generate or assume these without being instructed. For that, you use the Sequence construct:
from sqlalchemy import Sequence
Column(Integer, Sequence('user_id_seq'), primary_key=True)
A full, foolproof Table generated via our declarative mapping is therefore:
class User(Base):
__tablename__ = 'users'
id = Column(Integer, Sequence('user_id_seq'), primary_key=True)
name = Column(String(50))
fullname = Column(String(50))
password = Column(String(12))
def __init__(self, name, fullname, password):
self.name = name
self.fullname = fullname
self.password = password
def __repr__(self):
return "<User('%s','%s', '%s')>" % (self.name, self.fullname, self.password)
We include this more verbose table definition separately to highlight the difference between a minimal construct
geared primarily towards in-Python usage only, versus one that will be used to emit CREATE TABLE statements
on a particular set of backends with more stringent requirements.
2.1.4 Create an Instance of the Mapped Class
With mappings complete, let’s now create and inspect a User object:
>>> ed_user = User('ed', 'Ed Jones', 'edspassword')
>>> ed_user.name
'ed'
>>> ed_user.password
'edspassword'
>>> str(ed_user.id)
'None'
The id attribute, which while not defined by our __init__() method, exists with a value of None on our User
instance due to the id column we declared in our mapping. By default, the ORM creates class attributes for all
columns present in the table being mapped. These class attributes exist as descriptors, and define instrumentation
for the mapped class. The functionality of this instrumentation includes the ability to fire on change events, track
modifications, and to automatically load new data from the database when needed.
Since we have not yet told SQLAlchemy to persist Ed Jones within the database, its id is None. When we persist
the object later, this attribute will be populated with a newly generated value.
2.1. Object Relational Tutorial
11
SQLAlchemy Documentation, Release 0.8.7
The default __init__() method
Note that in our User example we supplied an __init__() method, which receives name, fullname and
password as positional arguments. The Declarative system supplies for us a default constructor if one is not
already present, which accepts keyword arguments of the same name as that of the mapped attributes. Below we
define User without specifying a constructor:
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
name = Column(String)
fullname = Column(String)
password = Column(String)
Our User class above will make usage of the default constructor, and provide id, name, fullname, and
password as keyword arguments:
u1 = User(name='ed', fullname='Ed Jones', password='foobar')
2.1.5 Creating a Session
We’re now ready to start talking to the database. The ORM’s “handle” to the database is the Session. When we first
set up the application, at the same level as our create_engine() statement, we define a Session class which
will serve as a factory for new Session objects:
>>> from sqlalchemy.orm import sessionmaker
>>> Session = sessionmaker(bind=engine)
In the case where your application does not yet have an Engine when you define your module-level objects, just set
it up like this:
>>> Session = sessionmaker()
Later, when you create your engine with create_engine(), connect it to the Session using configure():
>>> Session.configure(bind=engine)
# once engine is available
This custom-made Session class will create new Session objects which are bound to our database. Other transactional characteristics may be defined when calling sessionmaker() as well; these are described in a later chapter.
Then, whenever you need to have a conversation with the database, you instantiate a Session:
>>> session = Session()
The above Session is associated with our SQLite-enabled Engine, but it hasn’t opened any connections yet. When
it’s first used, it retrieves a connection from a pool of connections maintained by the Engine, and holds onto it until
we commit all changes and/or close the session object.
Session Creational Patterns
The business of acquiring a Session has a good deal of variety based on the variety of types of applications
and frameworks out there. Keep in mind the Session is just a workspace for your objects, local to a particular
database connection - if you think of an application thread as a guest at a dinner party, the Session is the
guest’s plate and the objects it holds are the food (and the database...the kitchen?)! Hints on how Session is
integrated into an application are at Session Frequently Asked Questions.
12
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
2.1.6 Adding New Objects
To persist our User object, we add() it to our Session:
>>> ed_user = User('ed', 'Ed Jones', 'edspassword')
>>> session.add(ed_user)
At this point, we say that the instance is pending; no SQL has yet been issued and the object is not yet represented by
a row in the database. The Session will issue the SQL to persist Ed Jones as soon as is needed, using a process
known as a flush. If we query the database for Ed Jones, all pending information will first be flushed, and the query
is issued immediately thereafter.
For example, below we create a new Query object which loads instances of User. We “filter by” the name attribute
of ed, and indicate that we’d like only the first result in the full list of rows. A User instance is returned which is
equivalent to that which we’ve added:
>>> our_user = session.query(User).filter_by(name='ed').first() # doctest:+ELLIPSIS,+NORMALIZE_WHITES
BEGIN (implicit)
INSERT INTO users (name, fullname, password) VALUES (?, ?, ?)
('ed', 'Ed Jones', 'edspassword')
SELECT users.id AS users_id,
users.name AS users_name,
users.fullname AS users_fullname,
users.password AS users_password
FROM users
WHERE users.name = ?
LIMIT ? OFFSET ?
('ed', 1, 0)
>>> our_user
<User('ed','Ed Jones', 'edspassword')>
In fact, the Session has identified that the row returned is the same row as one already represented within its internal
map of objects, so we actually got back the identical instance as that which we just added:
>>> ed_user is our_user
True
The ORM concept at work here is known as an identity map and ensures that all operations upon a particular row
within a Session operate upon the same set of data. Once an object with a particular primary key is present in the
Session, all SQL queries on that Session will always return the same Python object for that particular primary
key; it also will raise an error if an attempt is made to place a second, already-persisted object with the same primary
key within the session.
We can add more User objects at once using add_all():
>>> session.add_all([
...
User('wendy', 'Wendy Williams', 'foobar'),
...
User('mary', 'Mary Contrary', 'xxg527'),
...
User('fred', 'Fred Flinstone', 'blah')])
Also, we’ve decided the password for Ed isn’t too secure, so lets change it:
>>> ed_user.password = 'f8s7ccs'
The Session is paying attention. It knows, for example, that Ed Jones has been modified:
>>> session.dirty
IdentitySet([<User('ed','Ed Jones', 'f8s7ccs')>])
and that three new User objects are pending:
2.1. Object Relational Tutorial
13
SQLAlchemy Documentation, Release 0.8.7
>>> session.new # doctest: +SKIP
IdentitySet([<User('wendy','Wendy Williams', 'foobar')>,
<User('mary','Mary Contrary', 'xxg527')>,
<User('fred','Fred Flinstone', 'blah')>])
We tell the Session that we’d like to issue all remaining changes to the database and commit the transaction, which
has been in progress throughout. We do this via commit():
>>> session.commit()
UPDATE users SET password=? WHERE users.id =
('f8s7ccs', 1)
INSERT INTO users (name, fullname, password)
('wendy', 'Wendy Williams', 'foobar')
INSERT INTO users (name, fullname, password)
('mary', 'Mary Contrary', 'xxg527')
INSERT INTO users (name, fullname, password)
('fred', 'Fred Flinstone', 'blah')
COMMIT
?
VALUES (?, ?, ?)
VALUES (?, ?, ?)
VALUES (?, ?, ?)
commit() flushes whatever remaining changes remain to the database, and commits the transaction. The connection
resources referenced by the session are now returned to the connection pool. Subsequent operations with this session
will occur in a new transaction, which will again re-acquire connection resources when first needed.
If we look at Ed’s id attribute, which earlier was None, it now has a value:
>>> ed_user.id # doctest: +NORMALIZE_WHITESPACE
BEGIN (implicit)
SELECT users.id AS users_id,
users.name AS users_name,
users.fullname AS users_fullname,
users.password AS users_password
FROM users
WHERE users.id = ?
(1,)
1
After the Session inserts new rows in the database, all newly generated identifiers and database-generated defaults
become available on the instance, either immediately or via load-on-first-access. In this case, the entire row was reloaded on access because a new transaction was begun after we issued commit(). SQLAlchemy by default refreshes
data from a previous transaction the first time it’s accessed within a new transaction, so that the most recent state is
available. The level of reloading is configurable as is described in Using the Session.
Session Object States
As our User object moved from being outside the Session, to inside the Session without a primary key, to
actually being inserted, it moved between three out of four available “object states” - transient, pending, and
persistent. Being aware of these states and what they mean is always a good idea - be sure to read Quickie Intro
to Object States for a quick overview.
2.1.7 Rolling Back
Since the Session works within a transaction, we can roll back changes made too. Let’s make two changes that
we’ll revert; ed_user‘s user name gets set to Edwardo:
>>> ed_user.name = 'Edwardo'
14
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
and we’ll add another erroneous user, fake_user:
>>> fake_user = User('fakeuser', 'Invalid', '12345')
>>> session.add(fake_user)
Querying the session, we can see that they’re flushed into the current transaction:
>>> session.query(User).filter(User.name.in_(['Edwardo', 'fakeuser'])).all() #doctest: +NORMALIZE_WHI
UPDATE users SET name=? WHERE users.id = ?
('Edwardo', 1)
INSERT INTO users (name, fullname, password) VALUES (?, ?, ?)
('fakeuser', 'Invalid', '12345')
SELECT users.id AS users_id,
users.name AS users_name,
users.fullname AS users_fullname,
users.password AS users_password
FROM users
WHERE users.name IN (?, ?)
('Edwardo', 'fakeuser')
[<User('Edwardo','Ed Jones', 'f8s7ccs')>, <User('fakeuser','Invalid', '12345')>]
Rolling back, we can see that ed_user‘s name is back to ed, and fake_user has been kicked out of the session:
>>> session.rollback()
ROLLBACK
>>> ed_user.name #doctest: +NORMALIZE_WHITESPACE
BEGIN (implicit)
SELECT users.id AS users_id,
users.name AS users_name,
users.fullname AS users_fullname,
users.password AS users_password
FROM users
WHERE users.id = ?
(1,)
u'ed'
>>> fake_user in session
False
issuing a SELECT illustrates the changes made to the database:
>>> session.query(User).filter(User.name.in_(['ed', 'fakeuser'])).all() #doctest: +NORMALIZE_WHITESPA
SELECT users.id AS users_id,
users.name AS users_name,
users.fullname AS users_fullname,
users.password AS users_password
FROM users
WHERE users.name IN (?, ?)
('ed', 'fakeuser')
[<User('ed','Ed Jones', 'f8s7ccs')>]
2.1.8 Querying
A Query object is created using the query() method on Session. This function takes a variable number of
arguments, which can be any combination of classes and class-instrumented descriptors. Below, we indicate a Query
which loads User instances. When evaluated in an iterative context, the list of User objects present is returned:
2.1. Object Relational Tutorial
15
SQLAlchemy Documentation, Release 0.8.7
>>> for instance in session.query(User).order_by(User.id): # doctest: +NORMALIZE_WHITESPACE
...
print instance.name, instance.fullname
SELECT users.id AS users_id,
users.name AS users_name,
users.fullname AS users_fullname,
users.password AS users_password
FROM users ORDER BY users.id
()
ed Ed Jones
wendy Wendy Williams
mary Mary Contrary
fred Fred Flinstone
The Query also accepts ORM-instrumented descriptors as arguments. Any time multiple class entities or columnbased entities are expressed as arguments to the query() function, the return result is expressed as tuples:
>>> for name, fullname in session.query(User.name, User.fullname): # doctest: +NORMALIZE_WHITESPACE
...
print name, fullname
SELECT users.name AS users_name,
users.fullname AS users_fullname
FROM users
()
ed Ed Jones
wendy Wendy Williams
mary Mary Contrary
fred Fred Flinstone
The tuples returned by Query are named tuples, supplied by the KeyedTuple class, and can be treated much like
an ordinary Python object. The names are the same as the attribute’s name for an attribute, and the class name for a
class:
>>> for row in session.query(User, User.name).all(): #doctest: +NORMALIZE_WHITESPACE
...
print row.User, row.name
SELECT users.id AS users_id,
users.name AS users_name,
users.fullname AS users_fullname,
users.password AS users_password
FROM users
()
<User('ed','Ed Jones', 'f8s7ccs')> ed
<User('wendy','Wendy Williams', 'foobar')> wendy
<User('mary','Mary Contrary', 'xxg527')> mary
<User('fred','Fred Flinstone', 'blah')> fred
You can control the names of individual column expressions using the label() construct, which is available from
any ColumnElement-derived object, as well as any class attribute which is mapped to one (such as User.name):
>>> for row in session.query(User.name.label('name_label')).all(): #doctest: +NORMALIZE_WHITESPACE
...
print(row.name_label)
SELECT users.name AS name_label
FROM users
()
ed
wendy
mary
fred
The name given to a full entity such as User, assuming that multiple entities are present in the call to query(), can
be controlled using aliased :
16
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
>>> from sqlalchemy.orm import aliased
>>> user_alias = aliased(User, name='user_alias')
>>> for row in session.query(user_alias, user_alias.name).all(): #doctest: +NORMALIZE_WHITESPACE
...
print row.user_alias
SELECT user_alias.id AS user_alias_id,
user_alias.name AS user_alias_name,
user_alias.fullname AS user_alias_fullname,
user_alias.password AS user_alias_password
FROM users AS user_alias
()
<User('ed','Ed Jones', 'f8s7ccs')>
<User('wendy','Wendy Williams', 'foobar')>
<User('mary','Mary Contrary', 'xxg527')>
<User('fred','Fred Flinstone', 'blah')>
Basic operations with Query include issuing LIMIT and OFFSET, most conveniently using Python array slices and
typically in conjunction with ORDER BY:
>>> for u in session.query(User).order_by(User.id)[1:3]: #doctest: +NORMALIZE_WHITESPACE
...
print u
SELECT users.id AS users_id,
users.name AS users_name,
users.fullname AS users_fullname,
users.password AS users_password
FROM users ORDER BY users.id
LIMIT ? OFFSET ?
(2, 1)
<User('wendy','Wendy Williams', 'foobar')>
<User('mary','Mary Contrary', 'xxg527')>
and filtering results, which is accomplished either with filter_by(), which uses keyword arguments:
>>> for name, in session.query(User.name).\
...
filter_by(fullname='Ed Jones'): # doctest: +NORMALIZE_WHITESPACE
...
print name
SELECT users.name AS users_name FROM users
WHERE users.fullname = ?
('Ed Jones',)
ed
...or filter(), which uses more flexible SQL expression language constructs. These allow you to use regular
Python operators with the class-level attributes on your mapped class:
>>> for name, in session.query(User.name).\
...
filter(User.fullname=='Ed Jones'): # doctest: +NORMALIZE_WHITESPACE
...
print name
SELECT users.name AS users_name FROM users
WHERE users.fullname = ?
('Ed Jones',)
ed
The Query object is fully generative, meaning that most method calls return a new Query object upon which
further criteria may be added. For example, to query for users named “ed” with a full name of “Ed Jones”, you can
call filter() twice, which joins criteria using AND:
>>> for user in session.query(User).\
...
filter(User.name=='ed').\
...
filter(User.fullname=='Ed Jones'): # doctest: +NORMALIZE_WHITESPACE
2.1. Object Relational Tutorial
17
SQLAlchemy Documentation, Release 0.8.7
...
print user
SELECT users.id AS users_id,
users.name AS users_name,
users.fullname AS users_fullname,
users.password AS users_password
FROM users
WHERE users.name = ? AND users.fullname = ?
('ed', 'Ed Jones')
<User('ed','Ed Jones', 'f8s7ccs')>
Common Filter Operators
Here’s a rundown of some of the most common operators used in filter():
• equals:
query.filter(User.name == 'ed')
• not equals:
query.filter(User.name != 'ed')
• LIKE:
query.filter(User.name.like('%ed%'))
• IN :
query.filter(User.name.in_(['ed', 'wendy', 'jack']))
# works with query objects too:
query.filter(User.name.in_(session.query(User.name).filter(User.name.like('%ed%'))))
• NOT IN :
query.filter(~User.name.in_(['ed', 'wendy', 'jack']))
• IS NULL:
filter(User.name == None)
# alternatively, if pep8/linters are a concern
query.filter(User.name.is_(None))
• IS NOT NULL:
filter(User.name != None)
# alternatively, if pep8/linters are a concern
query.filter(User.name.isnot(None))
• AND:
from sqlalchemy import and_
filter(and_(User.name == 'ed', User.fullname == 'Ed Jones'))
# or call filter()/filter_by() multiple times
filter(User.name == 'ed').filter(User.fullname == 'Ed Jones')
18
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
Note: Make sure you use and_() and not the Python and operator!
• OR:
from sqlalchemy import or_
filter(or_(User.name == 'ed', User.name == 'wendy'))
Note: Make sure you use or_() and not the Python or operator!
• MATCH:
query.filter(User.name.match('wendy'))
Note: match() uses a database-specific MATCH or CONTAINS function; its behavior will vary by
backend and is not available on some backends such as SQLite.
Returning Lists and Scalars
The all(), one(), and first() methods of Query immediately issue SQL and return a non-iterator value.
all() returns a list:
>>> query = session.query(User).filter(User.name.like('%ed')).order_by(User.id)
>>> query.all() #doctest: +NORMALIZE_WHITESPACE
SELECT users.id AS users_id,
users.name AS users_name,
users.fullname AS users_fullname,
users.password AS users_password
FROM users
WHERE users.name LIKE ? ORDER BY users.id
('%ed',)
[<User('ed','Ed Jones', 'f8s7ccs')>, <User('fred','Fred Flinstone', 'blah')>]
first() applies a limit of one and returns the first result as a scalar:
>>> query.first() #doctest: +NORMALIZE_WHITESPACE
SELECT users.id AS users_id,
users.name AS users_name,
users.fullname AS users_fullname,
users.password AS users_password
FROM users
WHERE users.name LIKE ? ORDER BY users.id
LIMIT ? OFFSET ?
('%ed', 1, 0)
<User('ed','Ed Jones', 'f8s7ccs')>
one(), fully fetches all rows, and if not exactly one object identity or composite row is present in the result, raises an
error:
>>> from sqlalchemy.orm.exc import MultipleResultsFound
>>> try: #doctest: +NORMALIZE_WHITESPACE
...
user = query.one()
... except MultipleResultsFound, e:
...
print e
2.1. Object Relational Tutorial
19
SQLAlchemy Documentation, Release 0.8.7
SELECT users.id AS users_id,
users.name AS users_name,
users.fullname AS users_fullname,
users.password AS users_password
FROM users
WHERE users.name LIKE ? ORDER BY users.id
('%ed',)
Multiple rows were found for one()
>>> from sqlalchemy.orm.exc import NoResultFound
>>> try: #doctest: +NORMALIZE_WHITESPACE
...
user = query.filter(User.id == 99).one()
... except NoResultFound, e:
...
print e
SELECT users.id AS users_id,
users.name AS users_name,
users.fullname AS users_fullname,
users.password AS users_password
FROM users
WHERE users.name LIKE ? AND users.id = ? ORDER BY users.id
('%ed', 99)
No row was found for one()
Using Literal SQL
Literal strings can be used flexibly with Query. Most methods accept strings in addition to SQLAlchemy clause
constructs. For example, filter() and order_by():
>>> for user in session.query(User).\
...
filter("id<224").\
...
order_by("id").all(): #doctest: +NORMALIZE_WHITESPACE
...
print user.name
SELECT users.id AS users_id,
users.name AS users_name,
users.fullname AS users_fullname,
users.password AS users_password
FROM users
WHERE id<224 ORDER BY id
()
ed
wendy
mary
fred
Bind parameters can be specified with string-based SQL, using a colon. To specify the values, use the params()
method:
>>> session.query(User).filter("id<:value and name=:name").\
...
params(value=224, name='fred').order_by(User.id).one() # doctest: +NORMALIZE_WHITESPACE
SELECT users.id AS users_id,
users.name AS users_name,
users.fullname AS users_fullname,
users.password AS users_password
FROM users
WHERE id<? and name=? ORDER BY users.id
(224, 'fred')
<User('fred','Fred Flinstone', 'blah')>
20
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
To use an entirely string-based statement, using from_statement(); just ensure that the columns clause of the
statement contains the column names normally used by the mapper (below illustrated using an asterisk):
>>> session.query(User).from_statement(
...
"SELECT * FROM users where name=:name").\
...
params(name='ed').all()
SELECT * FROM users where name=?
('ed',)
[<User('ed','Ed Jones', 'f8s7ccs')>]
You can use from_statement() to go completely “raw”, using string names to identify desired columns:
>>> session.query("id", "name", "thenumber12").\
...
from_statement("SELECT id, name, 12 as "
...
"thenumber12 FROM users where name=:name").\
...
params(name='ed').all()
SELECT id, name, 12 as thenumber12 FROM users where name=?
('ed',)
[(1, u'ed', 12)]
2.1. Object Relational Tutorial
21
SQLAlchemy Documentation, Release 0.8.7
Pros and Cons of Literal SQL
Query is constructed like the rest of SQLAlchemy, in that it tries to always allow “falling back” to a less
automated, lower level approach to things. Accepting strings for all SQL fragments is a big part of that, so that
you can bypass the need to organize SQL constructs if you know specifically what string output you’d like. But
when using literal strings, the Query no longer knows anything about that part of the SQL construct being
emitted, and has no ability to transform it to adapt to new contexts.
For example, suppose we selected User objects and ordered by the name column, using a string to indicate
name:
>>> q = session.query(User.id, User.name)
>>> q.order_by("name").all()
SELECT users.id AS users_id, users.name AS users_name
FROM users ORDER BY name
()
[(1, u'ed'), (4, u'fred'), (3, u'mary'), (2, u'wendy')]
Perfectly fine. But suppose, before we got a hold of the Query, some sophisticated transformations were
applied to it, such as below where we use from_self(), a particularly advanced method, to retrieve pairs of
user names with different numbers of characters:
>>> from sqlalchemy import func
>>> ua = aliased(User)
>>> q = q.from_self(User.id, User.name, ua.name).\
...
filter(User.name < ua.name).\
...
filter(func.length(ua.name) != func.length(User.name))
The Query now represents a select from a subquery, where User is represented twice both inside and outside
of the subquery. Telling the Query to order by “name” doesn’t really give us much guarantee which “name”
it’s going to order on. In this case it assumes “name” is against the outer “aliased” User construct:
>>> q.order_by("name").all() #doctest: +NORMALIZE_WHITESPACE
SELECT anon_1.users_id AS anon_1_users_id,
anon_1.users_name AS anon_1_users_name,
users_1.name AS users_1_name
FROM (SELECT users.id AS users_id, users.name AS users_name
FROM users) AS anon_1, users AS users_1
WHERE anon_1.users_name < users_1.name
AND length(users_1.name) != length(anon_1.users_name)
ORDER BY name
()
[(1, u'ed', u'fred'), (1, u'ed', u'mary'), (1, u'ed', u'wendy'), (3, u'mary', u'wendy'), (4, u'fred'
Only if we use the SQL element directly, in this case User.name or ua.name, do we give Query enough
information to know for sure which “name” we’d like to order on, where we can see we get different results for
each:
>>> q.order_by(ua.name).all() #doctest: +NORMALIZE_WHITESPACE
SELECT anon_1.users_id AS anon_1_users_id,
anon_1.users_name AS anon_1_users_name,
users_1.name AS users_1_name
FROM (SELECT users.id AS users_id, users.name AS users_name
FROM users) AS anon_1, users AS users_1
WHERE anon_1.users_name < users_1.name
AND length(users_1.name) != length(anon_1.users_name)
ORDER BY users_1.name
()
[(1, u'ed', u'fred'), (1, u'ed', u'mary'), (1, u'ed', u'wendy'), (3, u'mary', u'wendy'), (4, u'fred'
>>> q.order_by(User.name).all() #doctest: +NORMALIZE_WHITESPACE
SELECT anon_1.users_id AS anon_1_users_id,
anon_1.users_name AS anon_1_users_name,
users_1.name AS users_1_name
22
FROM (SELECT users.id AS users_id, users.name AS users_name Chapter 2. SQLAlchemy ORM
FROM users) AS anon_1, users AS users_1
WHERE anon_1.users_name < users_1.name
AND length(users_1.name) != length(anon_1.users_name)
SQLAlchemy Documentation, Release 0.8.7
Counting
Query includes a convenience method for counting called count():
>>> session.query(User).filter(User.name.like('%ed')).count() #doctest: +NORMALIZE_WHITESPACE
SELECT count(*) AS count_1
FROM (SELECT users.id AS users_id,
users.name AS users_name,
users.fullname AS users_fullname,
users.password AS users_password
FROM users
WHERE users.name LIKE ?) AS anon_1
('%ed',)
2
The count() method is used to determine how many rows the SQL statement would return. Looking at the generated
SQL above, SQLAlchemy always places whatever it is we are querying into a subquery, then counts the rows from
that. In some cases this can be reduced to a simpler SELECT count(*) FROM table, however modern versions
of SQLAlchemy don’t try to guess when this is appropriate, as the exact SQL can be emitted using more explicit
means.
For situations where the “thing to be counted” needs to be indicated specifically, we can specify the “count” function
directly using the expression func.count(), available from the func construct. Below we use it to return the
count of each distinct user name:
>>> from sqlalchemy import func
>>> session.query(func.count(User.name), User.name).group_by(User.name).all()
SELECT count(users.name) AS count_1, users.name AS users_name
FROM users GROUP BY users.name
()
[(1, u'ed'), (1, u'fred'), (1, u'mary'), (1, u'wendy')]
#doctest: +NORMALIZE_W
To achieve our simple SELECT count(*) FROM table, we can apply it as:
>>> session.query(func.count('*')).select_from(User).scalar()
SELECT count(?) AS count_1
FROM users
('*',)
4
The usage of select_from() can be removed if we express the count in terms of the User primary key directly:
>>> session.query(func.count(User.id)).scalar() #doctest: +NORMALIZE_WHITESPACE
SELECT count(users.id) AS count_1
FROM users
()
4
2.1.9 Building a Relationship
Let’s consider how a second table, related to User, can be mapped and queried. Users in our system can store any
number of email addresses associated with their username. This implies a basic one to many association from the
users to a new table which stores email addresses, which we will call addresses. Using declarative, we define
this table along with its mapped class, Address:
>>> from sqlalchemy import ForeignKey
>>> from sqlalchemy.orm import relationship, backref
2.1. Object Relational Tutorial
23
SQLAlchemy Documentation, Release 0.8.7
>>> class Address(Base):
...
__tablename__ = 'addresses'
...
id = Column(Integer, primary_key=True)
...
email_address = Column(String, nullable=False)
...
user_id = Column(Integer, ForeignKey('users.id'))
...
...
user = relationship("User", backref=backref('addresses', order_by=id))
...
...
def __init__(self, email_address):
...
self.email_address = email_address
...
...
def __repr__(self):
...
return "<Address('%s')>" % self.email_address
The above class introduces the ForeignKey construct, which is a directive applied to Column that indicates that
values in this column should be constrained to be values present in the named remote column. This is a core feature
of relational databases, and is the “glue” that transforms an otherwise unconnected collection of tables to have rich
overlapping relationships. The ForeignKey above expresses that values in the addresses.user_id column
should be constrained to those values in the users.id column, i.e. its primary key.
A second directive, known as relationship(), tells the ORM that the Address class itself should be linked
to the User class, using the attribute Address.user. relationship() uses the foreign key relationships
between the two tables to determine the nature of this linkage, determining that Address.user will be many-toone. A subdirective of relationship() called backref() is placed inside of relationship(), providing
details about the relationship as expressed in reverse, that of a collection of Address objects on User referenced
by User.addresses. The reverse side of a many-to-one relationship is always one-to-many. A full catalog of
available relationship() configurations is at Basic Relational Patterns.
The two complementing relationships Address.user and User.addresses are referred to as a bidirectional
relationship, and is a key feature of the SQLAlchemy ORM. The section Linking Relationships with Backref discusses
the “backref” feature in detail.
Arguments to relationship() which concern the remote class can be specified using strings, assuming the Declarative system is in use. Once all mappings are complete, these strings are evaluated as Python expressions in order to
produce the actual argument, in the above case the User class. The names which are allowed during this evaluation
include, among other things, the names of all classes which have been created in terms of the declared base. Below
we illustrate creation of the same “addresses/user” bidirectional relationship in terms of User instead of Address:
class User(Base):
# ....
addresses = relationship("Address", order_by="Address.id", backref="user")
See the docstring for relationship() for more detail on argument style.
Did you know ?
• a FOREIGN KEY constraint in most (though not all) relational databases can only link to a primary key
column, or a column that has a UNIQUE constraint.
• a FOREIGN KEY constraint that refers to a multiple column primary key, and itself has multiple columns,
is known as a “composite foreign key”. It can also reference a subset of those columns.
• FOREIGN KEY columns can automatically update themselves, in response to a change in the referenced
column or row. This is known as the CASCADE referential action, and is a built in function of the
relational database.
• FOREIGN KEY can refer to its own table. This is referred to as a “self-referential” foreign key.
• Read more about foreign keys at Foreign Key - Wikipedia.
24
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
We’ll need to create the addresses table in the database, so we will issue another CREATE from our metadata,
which will skip over tables which have already been created:
>>> Base.metadata.create_all(engine) # doctest: +NORMALIZE_WHITESPACE
PRAGMA table_info("users")
()
PRAGMA table_info("addresses")
()
CREATE TABLE addresses (
id INTEGER NOT NULL,
email_address VARCHAR NOT NULL,
user_id INTEGER,
PRIMARY KEY (id),
FOREIGN KEY(user_id) REFERENCES users (id)
)
()
COMMIT
2.1.10 Working with Related Objects
Now when we create a User, a blank addresses collection will be present. Various collection types, such as sets
and dictionaries, are possible here (see Customizing Collection Access for details), but by default, the collection is a
Python list.
>>> jack = User('jack', 'Jack Bean', 'gjffdd')
>>> jack.addresses
[]
We are free to add Address objects on our User object. In this case we just assign a full list directly:
>>> jack.addresses = [
...
Address(email_address='[email protected]'),
...
Address(email_address='[email protected]')]
When using a bidirectional relationship, elements added in one direction automatically become visible in the other
direction. This behavior occurs based on attribute on-change events and is evaluated in Python, without using any
SQL:
>>> jack.addresses[1]
<Address('[email protected]')>
>>> jack.addresses[1].user
<User('jack','Jack Bean', 'gjffdd')>
Let’s add and commit Jack Bean to the database. jack as well as the two Address members in the corresponding
addresses collection are both added to the session at once, using a process known as cascading:
>>> session.add(jack)
>>> session.commit()
INSERT INTO users (name, fullname, password) VALUES (?, ?, ?)
('jack', 'Jack Bean', 'gjffdd')
INSERT INTO addresses (email_address, user_id) VALUES (?, ?)
('[email protected]', 5)
INSERT INTO addresses (email_address, user_id) VALUES (?, ?)
('[email protected]', 5)
COMMIT
Querying for Jack, we get just Jack back. No SQL is yet issued for Jack’s addresses:
2.1. Object Relational Tutorial
25
SQLAlchemy Documentation, Release 0.8.7
>>> jack = session.query(User).\
... filter_by(name='jack').one() #doctest: +NORMALIZE_WHITESPACE
BEGIN (implicit)
SELECT users.id AS users_id,
users.name AS users_name,
users.fullname AS users_fullname,
users.password AS users_password
FROM users
WHERE users.name = ?
('jack',)
>>> jack
<User('jack','Jack Bean', 'gjffdd')>
Let’s look at the addresses collection. Watch the SQL:
>>> jack.addresses #doctest: +NORMALIZE_WHITESPACE
SELECT addresses.id AS addresses_id,
addresses.email_address AS
addresses_email_address,
addresses.user_id AS addresses_user_id
FROM addresses
WHERE ? = addresses.user_id ORDER BY addresses.id
(5,)
[<Address('[email protected]')>, <Address('[email protected]')>]
When we accessed the addresses collection, SQL was suddenly issued. This is an example of a lazy loading
relationship. The addresses collection is now loaded and behaves just like an ordinary list. We’ll cover ways to
optimize the loading of this collection in a bit.
2.1.11 Querying with Joins
Now that we have two tables, we can show some more features of Query, specifically how to create queries that deal
with both tables at the same time. The Wikipedia page on SQL JOIN offers a good introduction to join techniques,
several of which we’ll illustrate here.
To construct a simple implicit join between User and Address, we can use Query.filter() to equate their
related columns together. Below we load the User and Address entities at once using this method:
>>> for u, a in session.query(User, Address).\
...
filter(User.id==Address.user_id).\
...
filter(Address.email_address=='[email protected]').\
...
all():
# doctest: +NORMALIZE_WHITESPACE
...
print u, a
SELECT users.id AS users_id,
users.name AS users_name,
users.fullname AS users_fullname,
users.password AS users_password,
addresses.id AS addresses_id,
addresses.email_address AS addresses_email_address,
addresses.user_id AS addresses_user_id
FROM users, addresses
WHERE users.id = addresses.user_id
AND addresses.email_address = ?
('[email protected]',)
<User('jack','Jack Bean', 'gjffdd')> <Address('[email protected]')>
The actual SQL JOIN syntax, on the other hand, is most easily achieved using the Query.join() method:
26
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
>>> session.query(User).join(Address).\
...
filter(Address.email_address=='[email protected]').\
...
all() #doctest: +NORMALIZE_WHITESPACE
SELECT users.id AS users_id,
users.name AS users_name,
users.fullname AS users_fullname,
users.password AS users_password
FROM users JOIN addresses ON users.id = addresses.user_id
WHERE addresses.email_address = ?
('[email protected]',)
[<User('jack','Jack Bean', 'gjffdd')>]
Query.join() knows how to join between User and Address because there’s only one foreign key between
them. If there were no foreign keys, or several, Query.join() works better when one of the following forms are
used:
query.join(Address, User.id==Address.user_id)
query.join(User.addresses)
query.join(Address, User.addresses)
query.join('addresses')
#
#
#
#
explicit condition
specify relationship from left to right
same, with explicit target
same, using a string
As you would expect, the same idea is used for “outer” joins, using the outerjoin() function:
query.outerjoin(User.addresses)
# LEFT OUTER JOIN
The reference documentation for join() contains detailed information and examples of the calling styles accepted
by this method; join() is an important method at the center of usage for any SQL-fluent application.
Using Aliases
When querying across multiple tables, if the same table needs to be referenced more than once, SQL typically requires
that the table be aliased with another name, so that it can be distinguished against other occurrences of that table. The
Query supports this most explicitly using the aliased construct. Below we join to the Address entity twice, to
locate a user who has two distinct email addresses at the same time:
>>> from sqlalchemy.orm import aliased
>>> adalias1 = aliased(Address)
>>> adalias2 = aliased(Address)
>>> for username, email1, email2 in \
...
session.query(User.name, adalias1.email_address, adalias2.email_address).\
...
join(adalias1, User.addresses).\
...
join(adalias2, User.addresses).\
...
filter(adalias1.email_address=='[email protected]').\
...
filter(adalias2.email_address=='[email protected]'):
...
print username, email1, email2
# doctest: +NORMALIZE_WHITESPACE
SELECT users.name AS users_name,
addresses_1.email_address AS addresses_1_email_address,
addresses_2.email_address AS addresses_2_email_address
FROM users JOIN addresses AS addresses_1
ON users.id = addresses_1.user_id
JOIN addresses AS addresses_2
ON users.id = addresses_2.user_id
WHERE addresses_1.email_address = ?
AND addresses_2.email_address = ?
('[email protected]', '[email protected]')
jack [email protected] [email protected]
2.1. Object Relational Tutorial
27
SQLAlchemy Documentation, Release 0.8.7
Using Subqueries
The Query is suitable for generating statements which can be used as subqueries. Suppose we wanted to load User
objects along with a count of how many Address records each user has. The best way to generate SQL like this is to
get the count of addresses grouped by user ids, and JOIN to the parent. In this case we use a LEFT OUTER JOIN so
that we get rows back for those users who don’t have any addresses, e.g.:
SELECT users.*, adr_count.address_count FROM users LEFT OUTER JOIN
(SELECT user_id, count(*) AS address_count
FROM addresses GROUP BY user_id) AS adr_count
ON users.id=adr_count.user_id
Using the Query, we build a statement like this from the inside out. The statement accessor returns a SQL
expression representing the statement generated by a particular Query - this is an instance of a select() construct,
which are described in SQL Expression Language Tutorial:
>>> from sqlalchemy.sql import func
>>> stmt = session.query(Address.user_id, func.count('*').\
...
label('address_count')).\
...
group_by(Address.user_id).subquery()
The func keyword generates SQL functions, and the subquery() method on Query produces a SQL
expression construct representing a SELECT statement embedded within an alias (it’s actually shorthand for
query.statement.alias()).
Once we have our statement, it behaves like a Table construct, such as the one we created for users at the start of
this tutorial. The columns on the statement are accessible through an attribute called c:
>>> for u, count in session.query(User, stmt.c.address_count).\
...
outerjoin(stmt, User.id==stmt.c.user_id).order_by(User.id): # doctest: +NORMALIZE_WHITESPACE
...
print u, count
SELECT users.id AS users_id,
users.name AS users_name,
users.fullname AS users_fullname,
users.password AS users_password,
anon_1.address_count AS anon_1_address_count
FROM users LEFT OUTER JOIN
(SELECT addresses.user_id AS user_id, count(?) AS address_count
FROM addresses GROUP BY addresses.user_id) AS anon_1
ON users.id = anon_1.user_id
ORDER BY users.id
('*',)
<User('ed','Ed Jones', 'f8s7ccs')> None
<User('wendy','Wendy Williams', 'foobar')> None
<User('mary','Mary Contrary', 'xxg527')> None
<User('fred','Fred Flinstone', 'blah')> None
<User('jack','Jack Bean', 'gjffdd')> 2
Selecting Entities from Subqueries
Above, we just selected a result that included a column from a subquery. What if we wanted our subquery to map to
an entity ? For this we use aliased() to associate an “alias” of a mapped class to a subquery:
>>> stmt = session.query(Address).\
...
filter(Address.email_address != '[email protected]').\
...
subquery()
>>> adalias = aliased(Address, stmt)
>>> for user, address in session.query(User, adalias).\
28
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
...
join(adalias, User.addresses): # doctest: +NORMALIZE_WHITESPACE
...
print user, address
SELECT users.id AS users_id,
users.name AS users_name,
users.fullname AS users_fullname,
users.password AS users_password,
anon_1.id AS anon_1_id,
anon_1.email_address AS anon_1_email_address,
anon_1.user_id AS anon_1_user_id
FROM users JOIN
(SELECT addresses.id AS id,
addresses.email_address AS email_address,
addresses.user_id AS user_id
FROM addresses
WHERE addresses.email_address != ?) AS anon_1
ON users.id = anon_1.user_id
('[email protected]',)
<User('jack','Jack Bean', 'gjffdd')> <Address('[email protected]')>
Using EXISTS
The EXISTS keyword in SQL is a boolean operator which returns True if the given expression contains any rows. It
may be used in many scenarios in place of joins, and is also useful for locating rows which do not have a corresponding
row in a related table.
There is an explicit EXISTS construct, which looks like this:
>>> from sqlalchemy.sql import exists
>>> stmt = exists().where(Address.user_id==User.id)
>>> for name, in session.query(User.name).filter(stmt):
...
print name
SELECT users.name AS users_name
FROM users
WHERE EXISTS (SELECT *
FROM addresses
WHERE addresses.user_id = users.id)
()
jack
# doctest: +NORMALIZE_WHITESPACE
The Query features several operators which make usage of EXISTS automatically. Above, the statement can be
expressed along the User.addresses relationship using any():
>>> for name, in session.query(User.name).\
...
filter(User.addresses.any()):
# doctest: +NORMALIZE_WHITESPACE
...
print name
SELECT users.name AS users_name
FROM users
WHERE EXISTS (SELECT 1
FROM addresses
WHERE users.id = addresses.user_id)
()
jack
any() takes criterion as well, to limit the rows matched:
>>> for name, in session.query(User.name).\
...
filter(User.addresses.any(Address.email_address.like('%google%'))):
...
print name
2.1. Object Relational Tutorial
# doctest: +NORMALIZE_W
29
SQLAlchemy Documentation, Release 0.8.7
SELECT users.name AS users_name
FROM users
WHERE EXISTS (SELECT 1
FROM addresses
WHERE users.id = addresses.user_id AND addresses.email_address LIKE ?)
('%google%',)
jack
has() is the same operator as any() for many-to-one relationships (note the ~ operator here too, which means
“NOT”):
>>> session.query(Address).\
...
filter(~Address.user.has(User.name=='jack')).all() # doctest: +NORMALIZE_WHITESPACE
SELECT addresses.id AS addresses_id,
addresses.email_address AS addresses_email_address,
addresses.user_id AS addresses_user_id
FROM addresses
WHERE NOT (EXISTS (SELECT 1
FROM users
WHERE users.id = addresses.user_id AND users.name = ?))
('jack',)
[]
Common Relationship Operators
Here’s all the operators which build on relationships - each one is linked to its API documentation which includes full
details on usage and behavior:
• __eq__() (many-to-one “equals” comparison):
query.filter(Address.user == someuser)
• __ne__() (many-to-one “not equals” comparison):
query.filter(Address.user != someuser)
• IS NULL (many-to-one comparison, also uses __eq__()):
query.filter(Address.user == None)
• contains() (used for one-to-many collections):
query.filter(User.addresses.contains(someaddress))
• any() (used for collections):
query.filter(User.addresses.any(Address.email_address == 'bar'))
# also takes keyword arguments:
query.filter(User.addresses.any(email_address='bar'))
• has() (used for scalar references):
query.filter(Address.user.has(name='ed'))
• Query.with_parent() (used for any relationship):
session.query(Address).with_parent(someuser, 'addresses')
30
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
2.1.12 Eager Loading
Recall earlier that we illustrated a lazy loading operation, when we accessed the User.addresses collection of
a User and SQL was emitted. If you want to reduce the number of queries (dramatically, in many cases), we can
apply an eager load to the query operation. SQLAlchemy offers three types of eager loading, two of which are
automatic, and a third which involves custom criterion. All three are usually invoked via functions known as query
options which give additional instructions to the Query on how we would like various attributes to be loaded, via the
Query.options() method.
Subquery Load
In this case we’d like to indicate that User.addresses should load eagerly. A good choice for loading a set of
objects as well as their related collections is the orm.subqueryload() option, which emits a second SELECT
statement that fully loads the collections associated with the results just loaded. The name “subquery” originates from
the fact that the SELECT statement constructed directly via the Query is re-used, embedded as a subquery into a
SELECT against the related table. This is a little elaborate but very easy to use:
>>> from sqlalchemy.orm import subqueryload
>>> jack = session.query(User).\
...
options(subqueryload(User.addresses)).\
...
filter_by(name='jack').one() #doctest: +NORMALIZE_WHITESPACE
SELECT users.id AS users_id,
users.name AS users_name,
users.fullname AS users_fullname,
users.password AS users_password
FROM users
WHERE users.name = ?
('jack',)
SELECT addresses.id AS addresses_id,
addresses.email_address AS addresses_email_address,
addresses.user_id AS addresses_user_id,
anon_1.users_id AS anon_1_users_id
FROM (SELECT users.id AS users_id
FROM users WHERE users.name = ?) AS anon_1
JOIN addresses ON anon_1.users_id = addresses.user_id
ORDER BY anon_1.users_id, addresses.id
('jack',)
>>> jack
<User('jack','Jack Bean', 'gjffdd')>
>>> jack.addresses
[<Address('[email protected]')>, <Address('[email protected]')>]
Joined Load
The other automatic eager loading function is more well known and is called orm.joinedload(). This style
of loading emits a JOIN, by default a LEFT OUTER JOIN, so that the lead object as well as the related object or
collection is loaded in one step. We illustrate loading the same addresses collection in this way - note that even
though the User.addresses collection on jack is actually populated right now, the query will emit the extra join
regardless:
>>> from sqlalchemy.orm import joinedload
>>> jack = session.query(User).\
...
options(joinedload(User.addresses)).\
2.1. Object Relational Tutorial
31
SQLAlchemy Documentation, Release 0.8.7
...
filter_by(name='jack').one() #doctest: +NORMALIZE_WHITESPACE
SELECT users.id AS users_id,
users.name AS users_name,
users.fullname AS users_fullname,
users.password AS users_password,
addresses_1.id AS addresses_1_id,
addresses_1.email_address AS addresses_1_email_address,
addresses_1.user_id AS addresses_1_user_id
FROM users
LEFT OUTER JOIN addresses AS addresses_1 ON users.id = addresses_1.user_id
WHERE users.name = ? ORDER BY addresses_1.id
('jack',)
>>> jack
<User('jack','Jack Bean', 'gjffdd')>
>>> jack.addresses
[<Address('[email protected]')>, <Address('[email protected]')>]
Note that even though the OUTER JOIN resulted in two rows, we still only got one instance of User back. This is
because Query applies a “uniquing” strategy, based on object identity, to the returned entities. This is specifically so
that joined eager loading can be applied without affecting the query results.
While joinedload() has been around for a long time, subqueryload() is a newer form of eager loading.
subqueryload() tends to be more appropriate for loading related collections while joinedload() tends to be
better suited for many-to-one relationships, due to the fact that only one row is loaded for both the lead and the related
object.
joinedload() is not a replacement for join()
The join created by joinedload() is anonymously aliased such that it does not affect the query results.
An Query.order_by() or Query.filter() call cannot reference these aliased tables - so-called “user
space” joins are constructed using Query.join(). The rationale for this is that joinedload() is only
applied in order to affect how related objects or collections are loaded as an optimizing detail - it can be added or
removed with no impact on actual results. See the section The Zen of Eager Loading for a detailed description
of how this is used.
Explicit Join + Eagerload
A third style of eager loading is when we are constructing a JOIN explicitly in order to locate the primary rows, and
would like to additionally apply the extra table to a related object or collection on the primary object. This feature is
supplied via the orm.contains_eager() function, and is most typically useful for pre-loading the many-to-one
object on a query that needs to filter on that same object. Below we illustrate loading an Address row as well as the
related User object, filtering on the User named “jack” and using orm.contains_eager() to apply the “user”
columns to the Address.user attribute:
>>> from sqlalchemy.orm import contains_eager
>>> jacks_addresses = session.query(Address).\
...
join(Address.user).\
...
filter(User.name=='jack').\
...
options(contains_eager(Address.user)).\
...
all() #doctest: +NORMALIZE_WHITESPACE
SELECT users.id AS users_id,
users.name AS users_name,
users.fullname AS users_fullname,
32
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
users.password AS users_password,
addresses.id AS addresses_id,
addresses.email_address AS addresses_email_address,
addresses.user_id AS addresses_user_id
FROM addresses JOIN users ON users.id = addresses.user_id
WHERE users.name = ?
('jack',)
>>> jacks_addresses
[<Address('[email protected]')>, <Address('[email protected]')>]
>>> jacks_addresses[0].user
<User('jack','Jack Bean', 'gjffdd')>
For more information on eager loading, including how to configure various forms of loading by default, see the section
Relationship Loading Techniques.
2.1.13 Deleting
Let’s try to delete jack and see how that goes. We’ll mark as deleted in the session, then we’ll issue a count query
to see that no rows remain:
>>> session.delete(jack)
>>> session.query(User).filter_by(name='jack').count() # doctest: +NORMALIZE_WHITESPACE
UPDATE addresses SET user_id=? WHERE addresses.id = ?
(None, 1)
UPDATE addresses SET user_id=? WHERE addresses.id = ?
(None, 2)
DELETE FROM users WHERE users.id = ?
(5,)
SELECT count(*) AS count_1
FROM (SELECT users.id AS users_id,
users.name AS users_name,
users.fullname AS users_fullname,
users.password AS users_password
FROM users
WHERE users.name = ?) AS anon_1
('jack',)
0
So far, so good. How about Jack’s Address objects ?
>>> session.query(Address).filter(
...
Address.email_address.in_(['[email protected]', '[email protected]'])
... ).count() # doctest: +NORMALIZE_WHITESPACE
SELECT count(*) AS count_1
FROM (SELECT addresses.id AS addresses_id,
addresses.email_address AS addresses_email_address,
addresses.user_id AS addresses_user_id
FROM addresses
WHERE addresses.email_address IN (?, ?)) AS anon_1
('[email protected]', '[email protected]')
2
Uh oh, they’re still there ! Analyzing the flush SQL, we can see that the user_id column of each address was set to
NULL, but the rows weren’t deleted. SQLAlchemy doesn’t assume that deletes cascade, you have to tell it to do so.
2.1. Object Relational Tutorial
33
SQLAlchemy Documentation, Release 0.8.7
Configuring delete/delete-orphan Cascade
We will configure cascade options on the User.addresses relationship to change the behavior. While
SQLAlchemy allows you to add new attributes and relationships to mappings at any point in time, in this case the
existing relationship needs to be removed, so we need to tear down the mappings completely and start again - we’ll
close the Session:
>>> session.close()
and use a new declarative_base():
>>> Base = declarative_base()
Next we’ll declare the User class, adding in the addresses relationship including the cascade configuration (we’ll
leave the constructor out too):
>>> class User(Base):
...
__tablename__ = 'users'
...
...
id = Column(Integer, primary_key=True)
...
name = Column(String)
...
fullname = Column(String)
...
password = Column(String)
...
...
addresses = relationship("Address", backref='user', cascade="all, delete, delete-orphan")
...
...
def __repr__(self):
...
return "<User('%s','%s', '%s')>" % (self.name, self.fullname, self.password)
Then we recreate Address, noting that in this case we’ve created the Address.user relationship via the User
class already:
>>> class Address(Base):
...
__tablename__ = 'addresses'
...
id = Column(Integer, primary_key=True)
...
email_address = Column(String, nullable=False)
...
user_id = Column(Integer, ForeignKey('users.id'))
...
...
def __repr__(self):
...
return "<Address('%s')>" % self.email_address
Now when we load the user jack (below using get(), which loads by primary key), removing an address from the
corresponding addresses collection will result in that Address being deleted:
# load Jack by primary key
>>> jack = session.query(User).get(5)
#doctest: +NORMALIZE_WHITESPACE
BEGIN (implicit)
SELECT users.id AS users_id,
users.name AS users_name,
users.fullname AS users_fullname,
users.password AS users_password
FROM users
WHERE users.id = ?
(5,)
# remove one Address (lazy load fires off)
>>> del jack.addresses[1] #doctest: +NORMALIZE_WHITESPACE
SELECT addresses.id AS addresses_id,
34
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
addresses.email_address AS addresses_email_address,
addresses.user_id AS addresses_user_id
FROM addresses
WHERE ? = addresses.user_id
(5,)
# only one address remains
>>> session.query(Address).filter(
...
Address.email_address.in_(['[email protected]', '[email protected]'])
... ).count() # doctest: +NORMALIZE_WHITESPACE
DELETE FROM addresses WHERE addresses.id = ?
(2,)
SELECT count(*) AS count_1
FROM (SELECT addresses.id AS addresses_id,
addresses.email_address AS addresses_email_address,
addresses.user_id AS addresses_user_id
FROM addresses
WHERE addresses.email_address IN (?, ?)) AS anon_1
('[email protected]', '[email protected]')
1
Deleting Jack will delete both Jack and the remaining Address associated with the user:
>>> session.delete(jack)
>>> session.query(User).filter_by(name='jack').count() # doctest: +NORMALIZE_WHITESPACE
DELETE FROM addresses WHERE addresses.id = ?
(1,)
DELETE FROM users WHERE users.id = ?
(5,)
SELECT count(*) AS count_1
FROM (SELECT users.id AS users_id,
users.name AS users_name,
users.fullname AS users_fullname,
users.password AS users_password
FROM users
WHERE users.name = ?) AS anon_1
('jack',)
0
>>> session.query(Address).filter(
...
Address.email_address.in_(['[email protected]', '[email protected]'])
... ).count() # doctest: +NORMALIZE_WHITESPACE
SELECT count(*) AS count_1
FROM (SELECT addresses.id AS addresses_id,
addresses.email_address AS addresses_email_address,
addresses.user_id AS addresses_user_id
FROM addresses
WHERE addresses.email_address IN (?, ?)) AS anon_1
('[email protected]', '[email protected]')
0
More on Cascades
Further detail on configuration of cascades is at Cascades. The cascade functionality can also integrate smoothly
with the ON DELETE CASCADE functionality of the relational database. See Using Passive Deletes for details.
2.1. Object Relational Tutorial
35
SQLAlchemy Documentation, Release 0.8.7
2.1.14 Building a Many To Many Relationship
We’re moving into the bonus round here, but lets show off a many-to-many relationship. We’ll sneak in some other
features too, just to take a tour. We’ll make our application a blog application, where users can write BlogPost
items, which have Keyword items associated with them.
For a plain many-to-many, we need to create an un-mapped Table construct to serve as the association table. This
looks like the following:
>>>
>>>
>>>
...
...
...
from sqlalchemy import Table, Text
# association table
post_keywords = Table('post_keywords', Base.metadata,
Column('post_id', Integer, ForeignKey('posts.id')),
Column('keyword_id', Integer, ForeignKey('keywords.id'))
)
Above, we can see declaring a Table directly is a little different than declaring a mapped class. Table is a constructor function, so each individual Column argument is separated by a comma. The Column object is also given
its name explicitly, rather than it being taken from an assigned attribute name.
Next we define BlogPost and Keyword, with a relationship() linked via the post_keywords table:
>>> class BlogPost(Base):
...
__tablename__ = 'posts'
...
...
id = Column(Integer, primary_key=True)
...
user_id = Column(Integer, ForeignKey('users.id'))
...
headline = Column(String(255), nullable=False)
...
body = Column(Text)
...
...
# many to many BlogPost<->Keyword
...
keywords = relationship('Keyword', secondary=post_keywords, backref='posts')
...
...
def __init__(self, headline, body, author):
...
self.author = author
...
self.headline = headline
...
self.body = body
...
...
def __repr__(self):
...
return "BlogPost(%r, %r, %r)" % (self.headline, self.body, self.author)
>>> class Keyword(Base):
...
__tablename__ = 'keywords'
...
...
id = Column(Integer, primary_key=True)
...
keyword = Column(String(50), nullable=False, unique=True)
...
...
def __init__(self, keyword):
...
self.keyword = keyword
Above, the many-to-many relationship is BlogPost.keywords. The defining feature of a many-to-many relationship is the secondary keyword argument which references a Table object representing the association table.
This table only contains columns which reference the two sides of the relationship; if it has any other columns, such
as its own primary key, or foreign keys to other tables, SQLAlchemy requires a different usage pattern called the
“association object”, described at Association Object.
We would also like our BlogPost class to have an author field. We will add this as another bidirectional relationship, except one issue we’ll have is that a single user might have lots of blog posts. When we access User.posts,
36
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
we’d like to be able to filter results further so as not to load the entire collection. For this we use a setting accepted by
relationship() called lazy=’dynamic’, which configures an alternate loader strategy on the attribute. To
use it on the “reverse” side of a relationship(), we use the backref() function:
>>> from sqlalchemy.orm import backref
>>> # "dynamic" loading relationship to User
>>> BlogPost.author = relationship(User, backref=backref('posts', lazy='dynamic'))
Create new tables:
>>> Base.metadata.create_all(engine) # doctest: +NORMALIZE_WHITESPACE
PRAGMA table_info("users")
()
PRAGMA table_info("addresses")
()
PRAGMA table_info("posts")
()
PRAGMA table_info("keywords")
()
PRAGMA table_info("post_keywords")
()
CREATE TABLE posts (
id INTEGER NOT NULL,
user_id INTEGER,
headline VARCHAR(255) NOT NULL,
body TEXT,
PRIMARY KEY (id),
FOREIGN KEY(user_id) REFERENCES users (id)
)
()
COMMIT
CREATE TABLE keywords (
id INTEGER NOT NULL,
keyword VARCHAR(50) NOT NULL,
PRIMARY KEY (id),
UNIQUE (keyword)
)
()
COMMIT
CREATE TABLE post_keywords (
post_id INTEGER,
keyword_id INTEGER,
FOREIGN KEY(post_id) REFERENCES posts (id),
FOREIGN KEY(keyword_id) REFERENCES keywords (id)
)
()
COMMIT
Usage is not too different from what we’ve been doing. Let’s give Wendy some blog posts:
>>> wendy = session.query(User).\
...
filter_by(name='wendy').\
...
one() #doctest: +NORMALIZE_WHITESPACE
SELECT users.id AS users_id,
users.name AS users_name,
users.fullname AS users_fullname,
users.password AS users_password
FROM users
WHERE users.name = ?
2.1. Object Relational Tutorial
37
SQLAlchemy Documentation, Release 0.8.7
('wendy',)
>>> post = BlogPost("Wendy's Blog Post", "This is a test", wendy)
>>> session.add(post)
We’re storing keywords uniquely in the database, but we know that we don’t have any yet, so we can just create them:
>>> post.keywords.append(Keyword('wendy'))
>>> post.keywords.append(Keyword('firstpost'))
We can now look up all blog posts with the keyword ‘firstpost’. We’ll use the any operator to locate “blog posts where
any of its keywords has the keyword string ‘firstpost”’:
>>> session.query(BlogPost).\
...
filter(BlogPost.keywords.any(keyword='firstpost')).\
...
all() #doctest: +NORMALIZE_WHITESPACE
INSERT INTO keywords (keyword) VALUES (?)
('wendy',)
INSERT INTO keywords (keyword) VALUES (?)
('firstpost',)
INSERT INTO posts (user_id, headline, body) VALUES (?, ?, ?)
(2, "Wendy's Blog Post", 'This is a test')
INSERT INTO post_keywords (post_id, keyword_id) VALUES (?, ?)
((1, 1), (1, 2))
SELECT posts.id AS posts_id,
posts.user_id AS posts_user_id,
posts.headline AS posts_headline,
posts.body AS posts_body
FROM posts
WHERE EXISTS (SELECT 1
FROM post_keywords, keywords
WHERE posts.id = post_keywords.post_id
AND keywords.id = post_keywords.keyword_id
AND keywords.keyword = ?)
('firstpost',)
[BlogPost("Wendy's Blog Post", 'This is a test', <User('wendy','Wendy Williams', 'foobar')>)]
If we want to look up posts owned by the user wendy, we can tell the query to narrow down to that User object as a
parent:
>>> session.query(BlogPost).\
...
filter(BlogPost.author==wendy).\
...
filter(BlogPost.keywords.any(keyword='firstpost')).\
...
all() #doctest: +NORMALIZE_WHITESPACE
SELECT posts.id AS posts_id,
posts.user_id AS posts_user_id,
posts.headline AS posts_headline,
posts.body AS posts_body
FROM posts
WHERE ? = posts.user_id AND (EXISTS (SELECT 1
FROM post_keywords, keywords
WHERE posts.id = post_keywords.post_id
AND keywords.id = post_keywords.keyword_id
AND keywords.keyword = ?))
(2, 'firstpost')
[BlogPost("Wendy's Blog Post", 'This is a test', <User('wendy','Wendy Williams', 'foobar')>)]
Or we can use Wendy’s own posts relationship, which is a “dynamic” relationship, to query straight from there:
38
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
>>> wendy.posts.\
...
filter(BlogPost.keywords.any(keyword='firstpost')).\
...
all() #doctest: +NORMALIZE_WHITESPACE
SELECT posts.id AS posts_id,
posts.user_id AS posts_user_id,
posts.headline AS posts_headline,
posts.body AS posts_body
FROM posts
WHERE ? = posts.user_id AND (EXISTS (SELECT 1
FROM post_keywords, keywords
WHERE posts.id = post_keywords.post_id
AND keywords.id = post_keywords.keyword_id
AND keywords.keyword = ?))
(2, 'firstpost')
[BlogPost("Wendy's Blog Post", 'This is a test', <User('wendy','Wendy Williams', 'foobar')>)]
2.1.15 Further Reference
Query Reference: Querying
Mapper Reference: Mapper Configuration
Relationship Reference: Relationship Configuration
Session Reference: Using the Session
2.2 Mapper Configuration
This section describes a variety of configurational patterns that are usable with mappers. It assumes you’ve worked
through Object Relational Tutorial and know how to construct and use rudimentary mappers and relationships.
2.2.1 Classical Mappings
A Classical Mapping refers to the configuration of a mapped class using the mapper() function, without using the
Declarative system. As an example, start with the declarative mapping introduced in Object Relational Tutorial:
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
name = Column(String)
fullname = Column(String)
password = Column(String)
In “classical” form, the table metadata is created separately with the Table construct, then associated with the User
class via the mapper() function:
from sqlalchemy import Table, MetaData, Column, ForeignKey, Integer, String
from sqlalchemy.orm import mapper
metadata = MetaData()
user = Table('user', metadata,
Column('id', Integer, primary_key=True),
2.2. Mapper Configuration
39
SQLAlchemy Documentation, Release 0.8.7
Column('name', String(50)),
Column('fullname', String(50)),
Column('password', String(12))
)
class User(object):
def __init__(self, name, fullname, password):
self.name = name
self.fullname = fullname
self.password = password
mapper(User, user)
Information about mapped attributes, such as relationships to other classes, are provided via the properties dictionary. The example below illustrates a second Table object, mapped to a class called Address, then linked to User
via relationship():
address = Table('address', metadata,
Column('id', Integer, primary_key=True),
Column('user_id', Integer, ForeignKey('user.id')),
Column('email_address', String(50))
)
mapper(User, user, properties={
'addresses' : relationship(Address, backref='user', order_by=address.c.id)
})
mapper(Address, address)
When using classical mappings, classes must be provided directly without the benefit of the “string lookup” system
provided by Declarative. SQL expressions are typically specified in terms of the Table objects, i.e. address.c.id
above for the Address relationship, and not Address.id, as Address may not yet be linked to table metadata,
nor can we specify a string here.
Some examples in the documentation still use the classical approach, but note that the classical as well as Declarative
approaches are fully interchangeable. Both systems ultimately create the same configuration, consisting of a Table,
user-defined class, linked together with a mapper(). When we talk about “the behavior of mapper()”, this includes
when using the Declarative system as well - it’s still used, just behind the scenes.
2.2.2 Customizing Column Properties
The default behavior of mapper() is to assemble all the columns in the mapped Table into mapped object attributes,
each of which are named according to the name of the column itself (specifically, the key attribute of Column). This
behavior can be modified in several ways.
Naming Columns Distinctly from Attribute Names
A mapping by default shares the same name for a Column as that of the mapped attribute - specifically it matches the
Column.key attribute on Column, which by default is the same as the Column.name.
The name assigned to the Python attribute which maps to Column can be different from either Column.name or
Column.key just by assigning it that way, as we illustrate here in a Declarative mapping:
class User(Base):
__tablename__ = 'user'
40
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
id = Column('user_id', Integer, primary_key=True)
name = Column('user_name', String(50))
Where above User.id resolves to a column named user_id and User.name resolves to a column named
user_name.
When mapping to an existing table, the Column object can be referenced directly:
class User(Base):
__table__ = user_table
id = user_table.c.user_id
name = user_table.c.user_name
Or in a classical mapping, placed in the properties dictionary with the desired key:
mapper(User, user_table, properties={
'id': user_table.c.user_id,
'name': user_table.c.user_name,
})
In the next section we’ll examine the usage of .key more closely.
Automating Column Naming Schemes from Reflected Tables
In the previous section Naming Columns Distinctly from Attribute Names, we showed how a Column explicitly
mapped to a class can have a different attribute name than the column. But what if we aren’t listing out Column
objects explicitly, and instead are automating the production of Table objects using reflection (e.g. as described in
Reflecting Database Objects)? In this case we can make use of the DDLEvents.column_reflect() event to
intercept the production of Column objects and provide them with the Column.key of our choice:
@event.listens_for(Table, "column_reflect")
def column_reflect(inspector, table, column_info):
# set column.key = "attr_<lower_case_name>"
column_info['key'] = "attr_%s" % column_info['name'].lower()
With the above event, the reflection of Column objects will be intercepted with our event that adds a new ”.key”
element, such as in a mapping as below:
class MyClass(Base):
__table__ = Table("some_table", Base.metadata,
autoload=True, autoload_with=some_engine)
If we want to qualify our event to only react for the specific MetaData object above, we can check for it in our event:
@event.listens_for(Table, "column_reflect")
def column_reflect(inspector, table, column_info):
if table.metadata is Base.metadata:
# set column.key = "attr_<lower_case_name>"
column_info['key'] = "attr_%s" % column_info['name'].lower()
Naming All Columns with a Prefix
A quick approach to prefix column names, typically when mapping to an existing Table object, is to use
column_prefix:
2.2. Mapper Configuration
41
SQLAlchemy Documentation, Release 0.8.7
class User(Base):
__table__ = user_table
__mapper_args__ = {'column_prefix':'_'}
The above will place attribute names such as _user_id, _user_name, _password etc. on the mapped User
class.
This approach is uncommon in modern usage. For dealing with reflected tables, a more flexible approach is to use that
described in Automating Column Naming Schemes from Reflected Tables.
Using column_property for column level options
Options can be specified when mapping a Column using the column_property() function. This function explicitly creates the ColumnProperty used by the mapper() to keep track of the Column; normally, the mapper()
creates this automatically. Using column_property(), we can pass additional arguments about how we’d like the
Column to be mapped. Below, we pass an option active_history, which specifies that a change to this column’s
value should result in the former value being loaded first:
from sqlalchemy.orm import column_property
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = column_property(Column(String(50)), active_history=True)
column_property() is also used to map a single attribute to multiple columns. This use case arises when mapping
to a join() which has attributes which are equated to each other:
class User(Base):
__table__ = user.join(address)
# assign "user.id", "address.user_id" to the
# "id" attribute
id = column_property(user_table.c.id, address_table.c.user_id)
For more examples featuring this usage, see Mapping a Class against Multiple Tables.
Another place where column_property() is needed is to specify SQL expressions as mapped attributes, such as
below where we create an attribute fullname that is the string concatenation of the firstname and lastname
columns:
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
firstname = Column(String(50))
lastname = Column(String(50))
fullname = column_property(firstname + " " + lastname)
See examples of this usage at SQL Expressions as Mapped Attributes.
sqlalchemy.orm.column_property(*cols, **kw)
Provide a column-level property for use with a Mapper.
Column-based properties can normally be applied to the mapper’s properties dictionary using the Column
element directly. Use this function when the given column is not directly present within the mapper’s selectable;
examples include SQL expressions, functions, and scalar SELECT queries.
42
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
Columns that aren’t present in the mapper’s selectable won’t be persisted by the mapper and are effectively
“read-only” attributes.
Parameters
• *cols – list of Column objects to be mapped.
• active_history=False – When True, indicates that the “previous” value for a scalar
attribute should be loaded when replaced, if not already loaded. Normally, history tracking logic for simple non-primary-key scalar values only needs to be aware of the “new”
value in order to perform a flush. This flag is available for applications that make use of
attributes.get_history() or Session.is_modified() which also need to
know the “previous” value of the attribute.
New in version 0.6.6.
• comparator_factory – a class which extends ColumnProperty.Comparator
which provides custom SQL clause generation for comparison operations.
• group – a group name for this property when marked as deferred.
• deferred – when True, the column property is “deferred”, meaning that it does not load
immediately, and is instead loaded when the attribute is first accessed on an instance. See
also deferred().
• doc – optional string that will be applied as the doc on the class-bound descriptor.
• expire_on_flush=True – Disable expiry on flush. A column_property() which refers
to a SQL expression (and not a single table-bound column) is considered to be a “read only”
property; populating it has no effect on the state of data, and it can only return database
state. For this reason a column_property()’s value is expired whenever the parent object is
involved in a flush, that is, has any kind of “dirty” state within a flush. Setting this parameter
to False will have the effect of leaving any existing value present after the flush proceeds.
Note however that the Session with default expiration settings still expires all attributes
after a Session.commit() call, however.
New in version 0.7.3.
• info – Optional data dictionary which
MapperProperty.info attribute of this object.
will
be
populated
into
the
New in version 0.8.
• extension – an AttributeExtension instance, or list of extensions, which will be
prepended to the list of attribute listeners for the resulting descriptor placed on the class.
Deprecated. Please see AttributeEvents.
Mapping a Subset of Table Columns
Sometimes, a Table object was made available using the reflection process described at Reflecting Database Objects
to load the table’s structure from the database. For such a table that has lots of columns that don’t need to be referenced
in the application, the include_properties or exclude_properties arguments can specify that only a
subset of columns should be mapped. For example:
class User(Base):
__table__ = user_table
__mapper_args__ = {
'include_properties' :['user_id', 'user_name']
}
2.2. Mapper Configuration
43
SQLAlchemy Documentation, Release 0.8.7
...will map the User class to the user_table table, only including the user_id and user_name columns - the
rest are not referenced. Similarly:
class Address(Base):
__table__ = address_table
__mapper_args__ = {
'exclude_properties' : ['street', 'city', 'state', 'zip']
}
...will map the Address class to the address_table table, including all columns present except street, city,
state, and zip.
When this mapping is used, the columns that are not included will not be referenced in any SELECT statements emitted
by Query, nor will there be any mapped attribute on the mapped class which represents the column; assigning an
attribute of that name will have no effect beyond that of a normal Python attribute assignment.
In some cases, multiple columns may have the same name, such as when mapping to a join of two or more tables that
share some column name. include_properties and exclude_properties can also accommodate Column
objects to more accurately describe which columns should be included or excluded:
class UserAddress(Base):
__table__ = user_table.join(addresses_table)
__mapper_args__ = {
'exclude_properties' :[address_table.c.id],
'primary_key' : [user_table.c.id]
}
Note: insert and update defaults configured on individual Column objects, i.e. those described at metadata_defaults
including those configured by the default, update, server_default and server_onupdate arguments,
will continue to function normally even if those Column objects are not mapped. This is because in the case of
default and update, the Column object is still present on the underlying Table, thus allowing the default
functions to take place when the ORM emits an INSERT or UPDATE, and in the case of server_default and
server_onupdate, the relational database itself maintains these functions.
2.2.3 Deferred Column Loading
This feature allows particular columns of a table be loaded only upon direct access, instead of when the entity is queried
using Query. This feature is useful when one wants to avoid loading a large text or binary field into memory when
it’s not needed. Individual columns can be lazy loaded by themselves or placed into groups that lazy-load together,
using the orm.deferred() function to mark them as “deferred”. In the example below, we define a mapping that
will load each of .excerpt and .photo in separate, individual-row SELECT statements when each attribute is first
referenced on the individual object instance:
from sqlalchemy.orm import deferred
from sqlalchemy import Integer, String, Text, Binary, Column
class Book(Base):
__tablename__ = 'book'
book_id
title =
summary
excerpt
photo =
44
= Column(Integer, primary_key=True)
Column(String(200), nullable=False)
= Column(String(2000))
= deferred(Column(Text))
deferred(Column(Binary))
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
Classical mappings as always place the usage of orm.deferred() in the properties dictionary against the
table-bound Column:
mapper(Book, book_table, properties={
'photo':deferred(book_table.c.photo)
})
Deferred columns can be associated with a “group” name, so that they load together when any of them are first
accessed. The example below defines a mapping with a photos deferred group. When one .photo is accessed, all
three photos will be loaded in one SELECT statement. The .excerpt will be loaded separately when it is accessed:
class Book(Base):
__tablename__ = 'book'
book_id = Column(Integer, primary_key=True)
title = Column(String(200), nullable=False)
summary = Column(String(2000))
excerpt = deferred(Column(Text))
photo1 = deferred(Column(Binary), group='photos')
photo2 = deferred(Column(Binary), group='photos')
photo3 = deferred(Column(Binary), group='photos')
You can defer or undefer columns at the Query level using the orm.defer() and orm.undefer() query options:
from sqlalchemy.orm import defer, undefer
query = session.query(Book)
query.options(defer('summary')).all()
query.options(undefer('excerpt')).all()
And an entire “deferred group”, i.e. which uses the group keyword argument to orm.deferred(), can be undeferred using orm.undefer_group(), sending in the group name:
from sqlalchemy.orm import undefer_group
query = session.query(Book)
query.options(undefer_group('photos')).all()
Column Deferral API
sqlalchemy.orm.deferred(*columns, **kwargs)
Return a DeferredColumnProperty, which indicates this object attributes should only be loaded from its
corresponding table column when first accessed.
Used with the “properties” dictionary sent to mapper().
See also:
Deferred Column Loading
sqlalchemy.orm.defer(*key)
Return a MapperOption that will convert the column property of the given name into a deferred load.
Used with Query.options().
e.g.:
2.2. Mapper Configuration
45
SQLAlchemy Documentation, Release 0.8.7
from sqlalchemy.orm import defer
query(MyClass).options(defer("attribute_one"),
defer("attribute_two"))
A class bound descriptor is also accepted:
query(MyClass).options(
defer(MyClass.attribute_one),
defer(MyClass.attribute_two))
A “path” can be specified onto a related or collection object using a dotted name. The orm.defer() option
will be applied to that object when loaded:
query(MyClass).options(
defer("related.attribute_one"),
defer("related.attribute_two"))
To specify a path via class, send multiple arguments:
query(MyClass).options(
defer(MyClass.related, MyOtherClass.attribute_one),
defer(MyClass.related, MyOtherClass.attribute_two))
See also:
Deferred Column Loading
Parameters *key – A key representing an individual path. Multiple entries are accepted to allow a
multiple-token path for a single target, not multiple targets.
sqlalchemy.orm.undefer(*key)
Return a MapperOption that will convert the column property of the given name into a non-deferred (regular
column) load.
Used with Query.options().
e.g.:
from sqlalchemy.orm import undefer
query(MyClass).options(
undefer("attribute_one"),
undefer("attribute_two"))
A class bound descriptor is also accepted:
query(MyClass).options(
undefer(MyClass.attribute_one),
undefer(MyClass.attribute_two))
A “path” can be specified onto a related or collection object using a dotted name. The orm.undefer() option
will be applied to that object when loaded:
query(MyClass).options(
undefer("related.attribute_one"),
undefer("related.attribute_two"))
To specify a path via class, send multiple arguments:
46
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
query(MyClass).options(
undefer(MyClass.related, MyOtherClass.attribute_one),
undefer(MyClass.related, MyOtherClass.attribute_two))
See also:
orm.undefer_group() as a means to “undefer” a group of attributes at once.
Deferred Column Loading
Parameters *key – A key representing an individual path. Multiple entries are accepted to allow a
multiple-token path for a single target, not multiple targets.
sqlalchemy.orm.undefer_group(name)
Return a MapperOption that will convert the given group of deferred column properties into a non-deferred
(regular column) load.
Used with Query.options().
e.g.:
query(MyClass).options(undefer("group_one"))
See also:
Deferred Column Loading
Parameters name – String name of the deferred group. This name is established using the “group”
name to the orm.deferred() configurational function.
2.2.4 SQL Expressions as Mapped Attributes
Attributes on a mapped class can be linked to SQL expressions, which can be used in queries.
Using a Hybrid
The easiest and most flexible way to link relatively simple SQL expressions to a class is to use a so-called “hybrid
attribute”, described in the section Hybrid Attributes. The hybrid provides for an expression that works at both the
Python level as well as at the SQL expression level. For example, below we map a class User, containing attributes
firstname and lastname, and include a hybrid that will provide for us the fullname, which is the string
concatenation of the two:
from sqlalchemy.ext.hybrid import hybrid_property
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
firstname = Column(String(50))
lastname = Column(String(50))
@hybrid_property
def fullname(self):
return self.firstname + " " + self.lastname
Above, the fullname attribute is interpreted at both the instance and class level, so that it is available from an
instance:
some_user = session.query(User).first()
print some_user.fullname
2.2. Mapper Configuration
47
SQLAlchemy Documentation, Release 0.8.7
as well as usable wtihin queries:
some_user = session.query(User).filter(User.fullname == "John Smith").first()
The string concatenation example is a simple one, where the Python expression can be dual purposed at the instance
and class level. Often, the SQL expression must be distinguished from the Python expression, which can be achieved
using hybrid_property.expression(). Below we illustrate the case where a conditional needs to be present
inside the hybrid, using the if statement in Python and the sql.expression.case() construct for SQL expressions:
from sqlalchemy.ext.hybrid import hybrid_property
from sqlalchemy.sql import case
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
firstname = Column(String(50))
lastname = Column(String(50))
@hybrid_property
def fullname(self):
if self.firstname is not None:
return self.firstname + " " + self.lastname
else:
return self.lastname
@fullname.expression
def fullname(cls):
return case([
(cls.firstname != None, cls.firstname + " " + cls.lastname),
], else_ = cls.lastname)
Using column_property
The orm.column_property() function can be used to map a SQL expression in a manner similar to a regularly
mapped Column. With this technique, the attribute is loaded along with all other column-mapped attributes at load
time. This is in some cases an advantage over the usage of hybrids, as the value can be loaded up front at the same
time as the parent row of the object, particularly if the expression is one which links to other tables (typically as a
correlated subquery) to access data that wouldn’t normally be available on an already loaded object.
Disadvantages to using orm.column_property() for SQL expressions include that the expression must be compatible with the SELECT statement emitted for the class as a whole, and there are also some configurational quirks
which can occur when using orm.column_property() from declarative mixins.
Our “fullname” example can be expressed using orm.column_property() as follows:
from sqlalchemy.orm import column_property
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
firstname = Column(String(50))
lastname = Column(String(50))
fullname = column_property(firstname + " " + lastname)
Correlated subqueries may be used as well. Below we use the select() construct to create a SELECT that links
together the count of Address objects available for a particular User:
48
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
from sqlalchemy.orm import column_property
from sqlalchemy import select, func
from sqlalchemy import Column, Integer, String, ForeignKey
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Address(Base):
__tablename__ = 'address'
id = Column(Integer, primary_key=True)
user_id = Column(Integer, ForeignKey('user.id'))
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
address_count = column_property(
select([func.count(Address.id)]).\
where(Address.user_id==id).\
correlate_except(Address)
)
In the above example, we define a select() construct like the following:
select([func.count(Address.id)]).\
where(Address.user_id==id).\
correlate_except(Address)
The meaning of the above statement is, select the count of Address.id rows where the Address.user_id
column is equated to id, which in the context of the User class is the Column named id (note that id is also the
name of a Python built in function, which is not what we want to use here - if we were outside of the User class
definition, we’d use User.id).
The select.correlate_except() directive indicates that each element in the FROM clause of this select()
may be omitted from the FROM list (that is, correlated to the enclosing SELECT statement against User) except for
the one corresponding to Address. This isn’t strictly necessary, but prevents Address from being inadvertently
omitted from the FROM list in the case of a long string of joins between User and Address tables where SELECT
statements against Address are nested.
If import issues prevent the column_property() from being defined inline with the class, it can be assigned to
the class after both are configured. In Declarative this has the effect of calling Mapper.add_property() to add
an additional property after the fact:
User.address_count = column_property(
select([func.count(Address.id)]).\
where(Address.user_id==User.id)
)
For many-to-many relationships, use and_() to join the fields of the association table to both tables in a relation,
illustrated here with a classical mapping:
from sqlalchemy import and_
mapper(Author, authors, properties={
'book_count': column_property(
select([func.count(books.c.id)],
and_(
book_authors.c.author_id==authors.c.id,
book_authors.c.book_id==books.c.id
2.2. Mapper Configuration
49
SQLAlchemy Documentation, Release 0.8.7
)))
})
Using a plain descriptor
In cases where a SQL query more elaborate than what orm.column_property() or hybrid_property can
provide must be emitted, a regular Python function accessed as an attribute can be used, assuming the expression only
needs to be available on an already-loaded instance. The function is decorated with Python’s own @property decorator to mark it as a read-only attribute. Within the function, object_session() is used to locate the Session
corresponding to the current object, which is then used to emit a query:
from sqlalchemy.orm import object_session
from sqlalchemy import select, func
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
firstname = Column(String(50))
lastname = Column(String(50))
@property
def address_count(self):
return object_session(self).\
scalar(
select([func.count(Address.id)]).\
where(Address.user_id==self.id)
)
The plain descriptor approach is useful as a last resort, but is less performant in the usual case than both the hybrid
and column property approaches, in that it needs to emit a SQL query upon each access.
2.2.5 Changing Attribute Behavior
Simple Validators
A quick way to add a “validation” routine to an attribute is to use the validates() decorator. An attribute validator
can raise an exception, halting the process of mutating the attribute’s value, or can change the given value into something different. Validators, like all attribute extensions, are only called by normal userland code; they are not issued
when the ORM is populating the object:
from sqlalchemy.orm import validates
class EmailAddress(Base):
__tablename__ = 'address'
id = Column(Integer, primary_key=True)
email = Column(String)
@validates('email')
def validate_email(self, key, address):
assert '@' in address
return address
Validators also receive collection events, when items are added to a collection:
50
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
from sqlalchemy.orm import validates
class User(Base):
# ...
addresses = relationship("Address")
@validates('addresses')
def validate_address(self, key, address):
assert '@' in address.email
return address
Note that the validates() decorator is a convenience function built on top of attribute events. An application
that requires more control over configuration of attribute change behavior can make use of this system, described at
AttributeEvents.
sqlalchemy.orm.validates(*names, **kw)
Decorate a method as a ‘validator’ for one or more named properties.
Designates a method as a validator, a method which receives the name of the attribute as well as a value to
be assigned, or in the case of a collection, the value to be added to the collection. The function can then
raise validation exceptions to halt the process from continuing (where Python’s built-in ValueError and
AssertionError exceptions are reasonable choices), or can modify or replace the value before proceeding.
The function should otherwise return the given value.
Note that a validator for a collection cannot issue a load of that collection within the validation routine - this
usage raises an assertion to avoid recursion overflows. This is a reentrant condition which is not supported.
Parameters
• *names – list of attribute names to be validated.
• include_removes – if True, “remove” events will be sent as well - the validation function must accept an additional argument “is_remove” which will be a boolean.
New in version 0.7.7.
Using Descriptors and Hybrids
A more comprehensive way to produce modified behavior for an attribute is to use descriptors. These are commonly
used in Python using the property() function. The standard SQLAlchemy technique for descriptors is to create
a plain descriptor, and to have it read/write from a mapped attribute with a different name. Below we illustrate this
using Python 2.6-style properties:
class EmailAddress(Base):
__tablename__ = 'email_address'
id = Column(Integer, primary_key=True)
# name the attribute with an underscore,
# different from the column name
_email = Column("email", String)
# then create an ".email" attribute
# to get/set "._email"
@property
def email(self):
return self._email
2.2. Mapper Configuration
51
SQLAlchemy Documentation, Release 0.8.7
@email.setter
def email(self, email):
self._email = email
The approach above will work, but there’s more we can add. While our EmailAddress object will shuttle the value
through the email descriptor and into the _email mapped attribute, the class level EmailAddress.email
attribute does not have the usual expression semantics usable with Query. To provide these, we instead use the
hybrid extension as follows:
from sqlalchemy.ext.hybrid import hybrid_property
class EmailAddress(Base):
__tablename__ = 'email_address'
id = Column(Integer, primary_key=True)
_email = Column("email", String)
@hybrid_property
def email(self):
return self._email
@email.setter
def email(self, email):
self._email = email
The .email attribute, in addition to providing getter/setter behavior when we have an instance of EmailAddress,
also provides a SQL expression when used at the class level, that is, from the EmailAddress class directly:
from sqlalchemy.orm import Session
session = Session()
address = session.query(EmailAddress).\
filter(EmailAddress.email == '[email protected]').\
one()
SELECT address.email AS address_email, address.id AS address_id
FROM address
WHERE address.email = ?
('[email protected]',)
address.email = '[email protected]'
session.commit()
UPDATE address SET email=? WHERE address.id = ?
('[email protected]', 1)
COMMIT
The hybrid_property also allows us to change the behavior of the attribute, including defining separate
behaviors when the attribute is accessed at the instance level versus at the class/expression level, using the
hybrid_property.expression() modifier. Such as, if we wanted to add a host name automatically, we
might define two sets of string manipulation logic:
class EmailAddress(Base):
__tablename__ = 'email_address'
id = Column(Integer, primary_key=True)
_email = Column("email", String)
52
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
@hybrid_property
def email(self):
"""Return the value of _email up until the last twelve
characters."""
return self._email[:-12]
@email.setter
def email(self, email):
"""Set the value of _email, tacking on the twelve character
value @example.com."""
self._email = email + "@example.com"
@email.expression
def email(cls):
"""Produce a SQL expression that represents the value
of the _email column, minus the last twelve characters."""
return func.substr(cls._email, 0, func.length(cls._email) - 12)
Above, accessing the email property of an instance of EmailAddress will return the value of the _email attribute, removing or adding the hostname @example.com from the value. When we query against the email
attribute, a SQL function is rendered which produces the same effect:
address = session.query(EmailAddress).filter(EmailAddress.email == 'address').one()
SELECT address.email AS address_email, address.id AS address_id
FROM address
WHERE substr(address.email, ?, length(address.email) - ?) = ?
(0, 12, 'address')
Read more about Hybrids at Hybrid Attributes.
Synonyms
Synonyms are a mapper-level construct that applies expression behavior to a descriptor based attribute.
Changed in version 0.7: The functionality of synonym is superceded as of 0.7 by hybrid attributes.
sqlalchemy.orm.synonym(name, map_column=False, descriptor=None, comparator_factory=None,
doc=None)
Denote an attribute name as a synonym to a mapped property.
Changed in version 0.7: synonym() is superseded by the hybrid extension. See the documentation for
hybrids at Hybrid Attributes.
Used with the properties dictionary sent to mapper():
class MyClass(object):
def _get_status(self):
return self._status
def _set_status(self, value):
self._status = value
status = property(_get_status, _set_status)
mapper(MyClass, sometable, properties={
"status":synonym("_status", map_column=True)
})
2.2. Mapper Configuration
53
SQLAlchemy Documentation, Release 0.8.7
Above, the status attribute of MyClass will produce expression behavior against the table column named
status, using the Python attribute _status on the mapped class to represent the underlying value.
Parameters
• name – the name of the existing mapped property, which can be any other
MapperProperty including column-based properties and relationships.
• map_column – if True, an additional ColumnProperty is created on the mapper automatically, using the synonym’s name as the keyname of the property, and the keyname of
this synonym() as the name of the column to map.
Operator Customization
The “operators” used by the SQLAlchemy ORM and Core expression language are fully customizable. For example, the comparison expression User.name == ’ed’ makes usage of an operator built into Python itself called
operator.eq - the actual SQL construct which SQLAlchemy associates with such an operator can be modified.
New operations can be associated with column expressions as well. The operators which take place for column expressions are most directly redefined at the type level - see the section Redefining and Creating New Operators for a
description.
ORM level functions like column_property(), relationship(), and composite() also provide for operator redefinition at the ORM level, by passing a PropComparator subclass to the comparator_factory
argument of each function. Customization of operators at this level is a rare use case. See the documentation at
PropComparator for an overview.
2.2.6 Composite Column Types
Sets of columns can be associated with a single user-defined datatype. The ORM provides a single attribute which
represents the group of columns using the class you provide.
Changed in version 0.7: Composites have been simplified such that they no longer “conceal” the underlying column
based attributes. Additionally, in-place mutation is no longer automatic; see the section below on enabling mutability
to support tracking of in-place changes.
A simple example represents pairs of columns as a Point object. Point represents such a pair as .x and .y:
class Point(object):
def __init__(self, x, y):
self.x = x
self.y = y
def __composite_values__(self):
return self.x, self.y
def __repr__(self):
return "Point(x=%r, y=%r)" % (self.x, self.y)
def __eq__(self, other):
return isinstance(other, Point) and \
other.x == self.x and \
other.y == self.y
def __ne__(self, other):
return not self.__eq__(other)
54
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
The requirements for the custom datatype class are that it have a constructor which accepts positional arguments
corresponding to its column format, and also provides a method __composite_values__() which returns the
state of the object as a list or tuple, in order of its column-based attributes. It also should supply adequate __eq__()
and __ne__() methods which test the equality of two instances.
We will create a mapping to a table vertice, which represents two points as x1/y1 and x2/y2. These are created
normally as Column objects. Then, the composite() function is used to assign new attributes that will represent
sets of columns via the Point class:
from sqlalchemy import Column, Integer
from sqlalchemy.orm import composite
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Vertex(Base):
__tablename__ = 'vertice'
id
x1
y1
x2
y2
=
=
=
=
=
Column(Integer, primary_key=True)
Column(Integer)
Column(Integer)
Column(Integer)
Column(Integer)
start = composite(Point, x1, y1)
end = composite(Point, x2, y2)
A classical mapping above would define each composite() against the existing table:
mapper(Vertex, vertice_table, properties={
'start':composite(Point, vertice_table.c.x1, vertice_table.c.y1),
'end':composite(Point, vertice_table.c.x2, vertice_table.c.y2),
})
We can now persist and use Vertex instances, as well as query for them, using the .start and .end attributes
against ad-hoc Point instances:
>>> v = Vertex(start=Point(3, 4), end=Point(5, 6))
>>> session.add(v)
>>> q = session.query(Vertex).filter(Vertex.start == Point(3, 4))
>>> print q.first().start
BEGIN (implicit)
INSERT INTO vertice (x1, y1, x2, y2) VALUES (?, ?, ?, ?)
(3, 4, 5, 6)
SELECT vertice.id AS vertice_id,
vertice.x1 AS vertice_x1,
vertice.y1 AS vertice_y1,
vertice.x2 AS vertice_x2,
vertice.y2 AS vertice_y2
FROM vertice
WHERE vertice.x1 = ? AND vertice.y1 = ?
LIMIT ? OFFSET ?
(3, 4, 1, 0)
Point(x=3, y=4)
sqlalchemy.orm.composite(class_, *cols, **kwargs)
Return a composite column-based property for use with a Mapper.
See the mapping documentation section Composite Column Types for a full usage example.
2.2. Mapper Configuration
55
SQLAlchemy Documentation, Release 0.8.7
The MapperProperty returned by composite() is the CompositeProperty.
Parameters
• class_ – The “composite type” class.
• *cols – List of Column objects to be mapped.
• active_history=False – When True, indicates that the “previous” value for a scalar
attribute should be loaded when replaced, if not already loaded. See the same flag on
column_property().
Changed in version 0.7: This flag specifically becomes meaningful - previously it was a
placeholder.
• group – A group name for this property when marked as deferred.
• deferred – When True, the column property is “deferred”, meaning that it does not load
immediately, and is instead loaded when the attribute is first accessed on an instance. See
also deferred().
• comparator_factory
–
a
class
which
extends
CompositeProperty.Comparator which provides custom SQL clause generation for comparison operations.
• doc – optional string that will be applied as the doc on the class-bound descriptor.
• info – Optional data dictionary which
MapperProperty.info attribute of this object.
will
be
populated
into
the
New in version 0.8.
• extension – an AttributeExtension instance, or list of extensions, which will be
prepended to the list of attribute listeners for the resulting descriptor placed on the class.
Deprecated. Please see AttributeEvents.
Tracking In-Place Mutations on Composites
In-place changes to an existing composite value are not tracked automatically. Instead, the composite class needs to
provide events to its parent object explicitly. This task is largely automated via the usage of the MutableComposite
mixin, which uses events to associate each user-defined composite object with all parent associations. Please see the
example in Establishing Mutability on Composites.
Changed in version 0.7: In-place changes to an existing composite value are no longer tracked automatically; the
functionality is superseded by the MutableComposite class.
Redefining Comparison Operations for Composites
The “equals” comparison operation by default produces an AND of all corresponding columns equated to one another.
This can be changed using the comparator_factory argument to composite(), where we specify a custom
CompositeProperty.Comparator class to define existing or new operations. Below we illustrate the “greater
than” operator, implementing the same expression that the base “greater than” does:
from sqlalchemy.orm.properties import CompositeProperty
from sqlalchemy import sql
class PointComparator(CompositeProperty.Comparator):
def __gt__(self, other):
"""redefine the 'greater than' operation"""
56
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
return sql.and_(*[a>b for a, b in
zip(self.__clause_element__().clauses,
other.__composite_values__())])
class Vertex(Base):
___tablename__ = 'vertice'
id
x1
y1
x2
y2
=
=
=
=
=
Column(Integer, primary_key=True)
Column(Integer)
Column(Integer)
Column(Integer)
Column(Integer)
start = composite(Point, x1, y1,
comparator_factory=PointComparator)
end = composite(Point, x2, y2,
comparator_factory=PointComparator)
2.2.7 Mapping a Class against Multiple Tables
Mappers can be constructed against arbitrary relational units (called selectables) in addition to plain tables. For
example, the join() function creates a selectable unit comprised of multiple tables, complete with its own composite
primary key, which can be mapped in the same way as a Table:
from sqlalchemy import Table, Column, Integer, \
String, MetaData, join, ForeignKey
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import column_property
metadata = MetaData()
# define two Table objects
user_table = Table('user', metadata,
Column('id', Integer, primary_key=True),
Column('name', String),
)
address_table = Table('address', metadata,
Column('id', Integer, primary_key=True),
Column('user_id', Integer, ForeignKey('user.id')),
Column('email_address', String)
)
# define a join between them. This
# takes place across the user.id and address.user_id
# columns.
user_address_join = join(user_table, address_table)
Base = declarative_base()
# map to it
class AddressUser(Base):
__table__ = user_address_join
id = column_property(user_table.c.id, address_table.c.user_id)
address_id = address_table.c.id
2.2. Mapper Configuration
57
SQLAlchemy Documentation, Release 0.8.7
In the example above, the join expresses columns for both the user and the address table. The user.id and
address.user_id columns are equated by foreign key, so in the mapping they are defined as one attribute,
AddressUser.id, using column_property() to indicate a specialized column mapping. Based on this part
of the configuration, the mapping will copy new primary key values from user.id into the address.user_id
column when a flush occurs.
Additionally, the address.id column is mapped explicitly to an attribute named address_id. This is to disambiguate the mapping of the address.id column from the same-named AddressUser.id attribute, which here
has been assigned to refer to the user table combined with the address.user_id foreign key.
The natural primary key of the above mapping is the composite of (user.id, address.id), as these are the
primary key columns of the user and address table combined together. The identity of an AddressUser object
will be in terms of these two values, and is represented from an AddressUser object as (AddressUser.id,
AddressUser.address_id).
2.2.8 Mapping a Class against Arbitrary Selects
Similar to mapping against a join, a plain select() object can be used with a mapper as well. The example fragment
below illustrates mapping a class called Customer to a select() which includes a join to a subquery:
from sqlalchemy import select, func
subq = select([
func.count(orders.c.id).label('order_count'),
func.max(orders.c.price).label('highest_order'),
orders.c.customer_id
]).group_by(orders.c.customer_id).alias()
customer_select = select([customers, subq]).\
select_from(
join(customers, subq,
customers.c.id == subq.c.customer_id)
).alias()
class Customer(Base):
__table__ = customer_select
Above, the full row represented by customer_select will be all the columns of the customers table, in
addition to those columns exposed by the subq subquery, which are order_count, highest_order, and
customer_id. Mapping the Customer class to this selectable then creates a class which will contain those attributes.
When the ORM persists new instances of Customer, only the customers table will actually receive an INSERT.
This is because the primary key of the orders table is not represented in the mapping; the ORM will only emit an
INSERT into a table for which it has mapped the primary key.
Note: The practice of mapping to arbitrary SELECT statements, especially complex ones as above, is almost never
needed; it necessarily tends to produce complex queries which are often less efficient than that which would be
produced by direct query construction. The practice is to some degree based on the very early history of SQLAlchemy
where the mapper() construct was meant to represent the primary querying interface; in modern usage, the Query
object can be used to construct virtually any SELECT statement, including complex composites, and should be favored
over the “map-to-selectable” approach.
58
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
2.2.9 Multiple Mappers for One Class
In modern SQLAlchemy, a particular class is only mapped by one mapper() at a time. The rationale here is that
the mapper() modifies the class itself, not only persisting it towards a particular Table, but also instrumenting
attributes upon the class which are structured specifically according to the table metadata.
One potential use case for another mapper to exist at the same time is if we wanted to load instances of our class not
just from the immediate Table to which it is mapped, but from another selectable that is a derivation of that Table.
While there technically is a way to create such a mapper(), using the non_primary=True option, this approach
is virtually never needed. Instead, we use the functionality of the Query object to achieve this, using a method such
as Query.select_from() or Query.from_statement() to specify a derived selectable.
Another potential use is if we genuinely want instances of our class to be persisted into different tables at different
times; certain kinds of data sharding configurations may persist a particular class into tables that are identical in structure except for their name. For this kind of pattern, Python offers a better approach than the complexity of mapping
the same class multiple times, which is to instead create new mapped classes for each target table. SQLAlchemy refers
to this as the “entity name” pattern, which is described as a recipe at Entity Name.
2.2.10 Constructors and Object Initialization
Mapping imposes no restrictions or requirements on the constructor (__init__) method for the class. You are free
to require any arguments for the function that you wish, assign attributes to the instance that are unknown to the ORM,
and generally do anything else you would normally do when writing a constructor for a Python class.
The SQLAlchemy ORM does not call __init__ when recreating objects from database rows. The ORM’s process
is somewhat akin to the Python standard library’s pickle module, invoking the low level __new__ method and then
quietly restoring attributes directly on the instance rather than calling __init__.
If you need to do some setup on database-loaded instances before they’re ready to use, you can use the
@reconstructor decorator to tag a method as the ORM counterpart to __init__. SQLAlchemy will call this
method with no arguments every time it loads or reconstructs one of your instances. This is useful for recreating
transient properties that are normally assigned in your __init__:
from sqlalchemy import orm
class MyMappedClass(object):
def __init__(self, data):
self.data = data
# we need stuff on all instances, but not in the database.
self.stuff = []
@orm.reconstructor
def init_on_load(self):
self.stuff = []
When obj = MyMappedClass() is executed, Python calls the __init__ method as normal and the data argument is required. When instances are loaded during a Query operation as in query(MyMappedClass).one(),
init_on_load is called.
Any method may be tagged as the reconstructor(), even the __init__ method. SQLAlchemy will call the
reconstructor method with no arguments. Scalar (non-collection) database-mapped attributes of the instance will be
available for use within the function. Eagerly-loaded collections are generally not yet available and will usually only
contain the first element. ORM state changes made to objects at this stage will not be recorded for the next flush()
operation, so the activity within a reconstructor should be conservative.
reconstructor() is a shortcut into a larger system of “instance level” events, which can be subscribed to using
the event API - see InstanceEvents for the full API description of these events.
2.2. Mapper Configuration
59
SQLAlchemy Documentation, Release 0.8.7
sqlalchemy.orm.reconstructor(fn)
Decorate a method as the ‘reconstructor’ hook.
Designates a method as the “reconstructor”, an __init__-like method that will be called by the ORM after
the instance has been loaded from the database or otherwise reconstituted.
The reconstructor will be invoked with no arguments. Scalar (non-collection) database-mapped attributes of the
instance will be available for use within the function. Eagerly-loaded collections are generally not yet available
and will usually only contain the first element. ORM state changes made to objects at this stage will not be
recorded for the next flush() operation, so the activity within a reconstructor should be conservative.
2.2.11 Configuring a Version Counter
The Mapper supports management of a version id column, which is a single table column that increments or otherwise
updates its value each time an UPDATE to the mapped table occurs. This value is checked each time the ORM emits
an UPDATE or DELETE against the row to ensure that the value held in memory matches the database value.
Warning: Because the versioning feature relies upon comparison of the in memory record of an object, the
feature only applies to the Session.flush() process, where the ORM flushes individual in-memory rows to
the database. It does not take effect when performing a multirow UPDATE or DELETE using Query.update()
or Query.delete() methods, as these methods only emit an UPDATE or DELETE statement but otherwise do
not have direct access to the contents of those rows being affected.
The purpose of this feature is to detect when two concurrent transactions are modifying the same row at roughly the
same time, or alternatively to provide a guard against the usage of a “stale” row in a system that might be re-using data
from a previous transaction without refreshing (e.g. if one sets expire_on_commit=False with a Session, it
is possible to re-use the data from a previous transaction).
Concurrent transaction updates
When detecting concurrent updates within transactions, it is typically the case that the database’s transaction
isolation level is below the level of repeatable read; otherwise, the transaction will not be exposed to a new
row value created by a concurrent update which conflicts with the locally updated value. In this case, the
SQLAlchemy versioning feature will typically not be useful for in-transaction conflict detection, though it still
can be used for cross-transaction staleness detection.
The database that enforces repeatable reads will typically either have locked the target row against a concurrent
update, or is employing some form of multi version concurrency control such that it will emit an error when
the transaction is committed. SQLAlchemy’s version_id_col is an alternative which allows version tracking to
occur for specific tables within a transaction that otherwise might not have this isolation level set.
See also:
Repeatable Read Isolation Level - Postgresql’s implementation of repeatable read, including a description of the
error condition.
Simple Version Counting
The most straightforward way to track versions is to add an integer column to the mapped table, then establish it as
the version_id_col within the mapper options:
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
version_id = Column(Integer, nullable=False)
60
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
name = Column(String(50), nullable=False)
__mapper_args__ = {
"version_id_col": version_id
}
Above, the User mapping tracks integer versions using the column version_id. When an object of type User
is first flushed, the version_id column will be given a value of “1”. Then, an UPDATE of the table later on will
always be emitted in a manner similar to the following:
UPDATE user SET version_id=:version_id, name=:name
WHERE user.id = :user_id AND user.version_id = :user_version_id
{"name": "new name", "version_id": 2, "user_id": 1, "user_version_id": 1}
The above UPDATE statement is updating the row that not only matches user.id = 1, it also is requiring that
user.version_id = 1, where “1” is the last version identifier we’ve been known to use on this object. If a
transaction elsewhere has modifed the row independently, this version id will no longer match, and the UPDATE
statement will report that no rows matched; this is the condition that SQLAlchemy tests, that exactly one row matched
our UPDATE (or DELETE) statement. If zero rows match, that indicates our version of the data is stale, and a
StaleDataError is raised.
Custom Version Counters / Types
Other kinds of values or counters can be used for versioning. Common types include dates and GUIDs.
When using an alternate type or counter scheme, SQLAlchemy provides a hook for this scheme using the
version_id_generator argument, which accepts a version generation callable. This callable is passed the value
of the current known version, and is expected to return the subsequent version.
For example, if we wanted to track the versioning of our User class using a randomly generated GUID, we could do
this (note that some backends support a native GUID type, but we illustrate here using a simple string):
import uuid
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
version_uuid = Column(String(32))
name = Column(String(50), nullable=False)
__mapper_args__ = {
'version_id_col':version_uuid,
'version_id_generator':lambda version: uuid.uuid4().hex
}
The persistence engine will call upon uuid.uuid4() each time a User object is subject to an INSERT or an UPDATE. In this case, our version generation function can disregard the incoming value of version, as the uuid4()
function generates identifiers without any prerequisite value. If we were using a sequential versioning scheme such
as numeric or a special character system, we could make use of the given version in order to help determine the
subsequent value.
See also:
Backend-agnostic GUID Type
2.2. Mapper Configuration
61
SQLAlchemy Documentation, Release 0.8.7
2.2.12 Class Mapping API
sqlalchemy.orm.mapper(class_, local_table=None, *args, **params)
Return a new Mapper object.
This function is typically used behind the scenes via the Declarative extension. When using Declarative, many of the usual mapper() arguments are handled by the Declarative extension itself, including
class_, local_table, properties, and inherits. Other options are passed to mapper() using
the __mapper_args__ class variable:
class MyClass(Base):
__tablename__ = 'my_table'
id = Column(Integer, primary_key=True)
type = Column(String(50))
alt = Column("some_alt", Integer)
__mapper_args__ = {
'polymorphic_on' : type
}
Explicit use of mapper() is often referred to as classical mapping. The above declarative example is equivalent
in classical form to:
my_table = Table("my_table", metadata,
Column('id', Integer, primary_key=True),
Column('type', String(50)),
Column("some_alt", Integer)
)
class MyClass(object):
pass
mapper(MyClass, my_table,
polymorphic_on=my_table.c.type,
properties={
'alt':my_table.c.some_alt
})
See also:
Classical Mappings - discussion of direct usage of mapper()
Parameters
• class_ – The class to be mapped. When using Declarative, this argument is automatically
passed as the declared class itself.
• local_table – The Table or other selectable to which the class is mapped. May be
None if this mapper inherits from another mapper using single-table inheritance. When
using Declarative, this argument is automatically passed by the extension, based on what
is configured via the __table__ argument or via the Table produced as a result of the
__tablename__ and Column arguments present.
• always_refresh – If True, all query operations for this mapped class will overwrite all
data within object instances that already exist within the session, erasing any in-memory
changes with whatever information was loaded from the database. Usage of this flag is
highly discouraged; as an alternative, see the method Query.populate_existing().
• allow_partial_pks – Defaults to True. Indicates that a composite primary key with
some NULL values should be considered as possibly existing within the database. This
62
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
affects whether a mapper will assign an incoming row to an existing identity, as well as if
Session.merge() will check the database first for a particular primary key value. A
“partial primary key” can occur if one has mapped to an OUTER JOIN, for example.
• batch – Defaults to True, indicating that save operations of multiple entities can be
batched together for efficiency. Setting to False indicates that an instance will be fully
saved before saving the next instance. This is used in the extremely rare case that a
MapperEvents listener requires being called in between individual row persistence operations.
• column_prefix – A string which will be prepended to the mapped attribute name when
Column objects are automatically assigned as attributes to the mapped class. Does not
affect explicitly specified column-based properties.
See the section Naming All Columns with a Prefix for an example.
• concrete – If True, indicates this mapper should use concrete table inheritance with its
parent mapper.
See the section Concrete Table Inheritance for an example.
• eager_defaults – if True, the ORM will immediately fetch the value of servergenerated default values after an INSERT or UPDATE, rather than leaving them as expired
to be fetched on next access. This can be used for event schemes where the server-generated
values are needed immediately before the flush completes. This scheme will emit an individual SELECT statement per row inserted or updated, which note can add significant
performance overhead.
• exclude_properties – A list or set of string column names to be excluded from mapping.
See Mapping a Subset of Table Columns for an example.
• extension – A MapperExtension instance or list of MapperExtension instances which will be applied to all operations by this Mapper. Deprecated. Please see
MapperEvents.
• include_properties – An inclusive list or set of string column names to map.
See Mapping a Subset of Table Columns for an example.
• inherits – A mapped class or the corresponding Mapper of one indicating a superclass
to which this Mapper should inherit from. The mapped class here must be a subclass of
the other mapper’s class. When using Declarative, this argument is passed automatically as
a result of the natural class hierarchy of the declared classes.
See also:
Mapping Class Inheritance Hierarchies
• inherit_condition – For joined table inheritance, a SQL expression which will define
how the two tables are joined; defaults to a natural join between the two tables.
• inherit_foreign_keys – When inherit_condition is used and the columns
present are missing a ForeignKey configuration, this parameter can be used to specify
which columns are “foreign”. In most cases can be left as None.
• legacy_is_orphan – Boolean, defaults to False. When True, specifies that “legacy”
orphan consideration is to be applied to objects mapped by this mapper, which means that
a pending (that is, not persistent) object is auto-expunged from an owning Session only
when it is de-associated from all parents that specify a delete-orphan cascade towards
2.2. Mapper Configuration
63
SQLAlchemy Documentation, Release 0.8.7
this mapper. The new default behavior is that the object is auto-expunged when it is deassociated with any of its parents that specify delete-orphan cascade. This behavior is
more consistent with that of a persistent object, and allows behavior to be consistent in more
scenarios independently of whether or not an orphanable object has been flushed yet or not.
See the change note and example at The consideration of a “pending” object as an “orphan” has been made more aggressive for more detail on this change.
New in version 0.8: - the consideration of a pending object as an “orphan” has been modified to more closely match the behavior as that of persistent objects, which is that the
object is expunged from the Session as soon as it is de-associated from any of its orphanenabled parents. Previously, the pending object would be expunged only if de-associated
from all of its orphan-enabled parents. The new flag legacy_is_orphan is added to
orm.mapper() which re-establishes the legacy behavior.
• non_primary – Specify that this Mapper is in addition to the “primary” mapper, that is,
the one used for persistence. The Mapper created here may be used for ad-hoc mapping of
the class to an alternate selectable, for loading only.
The non_primary feature is rarely needed with modern usage.
• order_by – A single Column or list of Column objects for which selection operations
should use as the default ordering for entities. By default mappers have no pre-defined
ordering.
• passive_updates – Indicates UPDATE behavior of foreign key columns when a primary key column changes on a joined-table inheritance mapping. Defaults to True.
When True, it is assumed that ON UPDATE CASCADE is configured on the foreign key in
the database, and that the database will handle propagation of an UPDATE from a source
column to dependent columns on joined-table rows.
When False, it is assumed that the database does not enforce referential integrity and will
not be issuing its own CASCADE operation for an update. The unit of work process will
emit an UPDATE statement for the dependent columns during a primary key change.
See also:
Mutable Primary Keys / Update Cascades - description of a similar feature as used with
relationship()
• polymorphic_on – Specifies the column, attribute, or SQL expression used to determine
the target class for an incoming row, when inheriting classes are present.
This value is commonly a Column object that’s present in the mapped Table:
class Employee(Base):
__tablename__ = 'employee'
id = Column(Integer, primary_key=True)
discriminator = Column(String(50))
__mapper_args__ = {
"polymorphic_on":discriminator,
"polymorphic_identity":"employee"
}
It may also be specified as a SQL expression, as in this example where we use the case()
construct to provide a conditional approach:
64
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
class Employee(Base):
__tablename__ = 'employee'
id = Column(Integer, primary_key=True)
discriminator = Column(String(50))
__mapper_args__ = {
"polymorphic_on":case([
(discriminator == "EN", "engineer"),
(discriminator == "MA", "manager"),
], else_="employee"),
"polymorphic_identity":"employee"
}
It may also refer to any attribute configured with column_property(), or to the string
name of one:
class Employee(Base):
__tablename__ = 'employee'
id = Column(Integer, primary_key=True)
discriminator = Column(String(50))
employee_type = column_property(
case([
(discriminator == "EN", "engineer"),
(discriminator == "MA", "manager"),
], else_="employee")
)
__mapper_args__ = {
"polymorphic_on":employee_type,
"polymorphic_identity":"employee"
}
Changed in version 0.7.4: polymorphic_on may be specified as a SQL expression, or
refer to any attribute configured with column_property(), or to the string name of one.
When setting polymorphic_on to reference an attribute or expression that’s not present
in the locally mapped Table, yet the value of the discriminator should be persisted to
the database, the value of the discriminator is not automatically set on new instances; this
must be handled by the user, either through manual means or via event listeners. A typical
approach to establishing such a listener looks like:
from sqlalchemy import event
from sqlalchemy.orm import object_mapper
@event.listens_for(Employee, "init", propagate=True)
def set_identity(instance, *arg, **kw):
mapper = object_mapper(instance)
instance.discriminator = mapper.polymorphic_identity
Where above, we assign the value of polymorphic_identity for the mapped class to
the discriminator attribute, thus persisting the value to the discriminator column
in the database.
See also:
Mapping Class Inheritance Hierarchies
2.2. Mapper Configuration
65
SQLAlchemy Documentation, Release 0.8.7
• polymorphic_identity – Specifies the value which identifies this particular class as
returned by the column expression referred to by the polymorphic_on setting. As rows
are received, the value corresponding to the polymorphic_on column expression is compared to this value, indicating which subclass should be used for the newly reconstructed
object.
• properties – A dictionary mapping the string names of object attributes to
MapperProperty instances, which define the persistence behavior of that attribute.
Note that Column objects present in the mapped Table are automatically placed into
ColumnProperty instances upon mapping, unless overridden. When using Declarative,
this argument is passed automatically, based on all those MapperProperty instances
declared in the declared class body.
• primary_key – A list of Column objects which define the primary key to be used
against this mapper’s selectable unit. This is normally simply the primary key of the
local_table, but can be overridden here.
• version_id_col – A Column that will be used to keep a running version id of rows in
the table. This is used to detect concurrent updates or the presence of stale data in a flush.
The methodology is to detect if an UPDATE statement does not match the last known version
id, a StaleDataError exception is thrown. By default, the column must be of Integer
type, unless version_id_generator specifies an alternative version generator.
See also:
Configuring a Version Counter - discussion of version counting and rationale.
• version_id_generator – Define how new version ids should be generated. Defaults
to None, which indicates that a simple integer counting scheme be employed. To provide a
custom versioning scheme, provide a callable function of the form:
def generate_version(version):
return next_version
See also:
Custom Version Counters / Types
• with_polymorphic – A tuple in the form (<classes>, <selectable>) indicating the default style of “polymorphic” loading, that is, which tables are queried at once.
<classes> is any single or list of mappers and/or classes indicating the inherited classes that
should be loaded at once. The special value ’*’ may be used to indicate all descending
classes should be loaded immediately. The second tuple argument <selectable> indicates a
selectable that will be used to query for multiple classes.
See also:
Basic Control of Which Tables are Queried
sqlalchemy.orm.object_mapper(instance)
Given an object, return the primary Mapper associated with the object instance.
Raises sqlalchemy.orm.exc.UnmappedInstanceError if no mapping is configured.
This function is available via the inspection system as:
inspect(instance).mapper
Using the inspection system will raise sqlalchemy.exc.NoInspectionAvailable if the instance is
not part of a mapping.
66
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
sqlalchemy.orm.class_mapper(class_, configure=True)
Given a class, return the primary Mapper associated with the key.
Raises UnmappedClassError if no mapping is configured on the given class, or ArgumentError if a
non-class object is passed.
Equivalent functionality is available via the inspect() function as:
inspect(some_mapped_class)
Using the inspection system will raise sqlalchemy.exc.NoInspectionAvailable if the class is not
mapped.
sqlalchemy.orm.configure_mappers()
Initialize the inter-mapper relationships of all mappers that have been constructed thus far.
This function can be called any number of times, but in most cases is handled internally.
sqlalchemy.orm.clear_mappers()
Remove all mappers from all classes.
This function removes all instrumentation from classes and disposes of their associated mappers. Once called,
the classes are unmapped and can be later re-mapped with new mappers.
clear_mappers() is not for normal use, as there is literally no valid usage for it outside of very specific testing scenarios. Normally, mappers are permanent structural components of user-defined classes, and are never
discarded independently of their class. If a mapped class itself is garbage collected, its mapper is automatically disposed of as well. As such, clear_mappers() is only for usage in test suites that re-use the same
classes with different mappings, which is itself an extremely rare use case - the only such use case is in fact
SQLAlchemy’s own test suite, and possibly the test suites of other ORM extension libraries which intend to test
various combinations of mapper construction upon a fixed set of classes.
sqlalchemy.orm.util.identity_key(*args, **kwargs)
Generate “identity key” tuples, as are used as keys in the Session.identity_map dictionary.
This function has several call styles:
•identity_key(class, ident)
This form receives a mapped class and a primary key scalar or tuple as an argument.
E.g.:
>>> identity_key(MyClass, (1, 2))
(<class '__main__.MyClass'>, (1, 2))
param class mapped class (must be a positional argument)
param ident primary key, may be a scalar or tuple argument.
•identity_key(instance=instance)
This form will produce the identity key for a given instance. The instance need not be persistent, only that
its primary key attributes are populated (else the key will contain None for those missing values).
E.g.:
>>> instance = MyClass(1, 2)
>>> identity_key(instance=instance)
(<class '__main__.MyClass'>, (1, 2))
2.2. Mapper Configuration
67
SQLAlchemy Documentation, Release 0.8.7
In this form, the given instance is ultimately run though Mapper.identity_key_from_instance(),
which will have the effect of performing a database check for the corresponding row if the object is
expired.
param instance object instance (must be given as a keyword arg)
•identity_key(class, row=row)
This form is similar to the class/tuple form, except is passed a database result row as a RowProxy object.
E.g.:
>>> row = engine.execute("select * from table where a=1 and b=2").first()
>>> identity_key(MyClass, row=row)
(<class '__main__.MyClass'>, (1, 2))
param class mapped class (must be a positional argument)
param row RowProxy row returned by a ResultProxy (must be given as a keyword arg)
sqlalchemy.orm.util.polymorphic_union(table_map, typecolname,
cast_nulls=True)
Create a UNION statement used by a polymorphic mapper.
aliasname=’p_union’,
See Concrete Table Inheritance for an example of how this is used.
Parameters
• table_map – mapping of polymorphic identities to Table objects.
• typecolname – string name of a “discriminator” column, which will be derived from
the query, producing the polymorphic identity for each row. If None, no polymorphic
discriminator is generated.
• aliasname – name of the alias() construct generated.
• cast_nulls – if True, non-existent columns, which are represented as labeled NULLs,
will be passed into CAST. This is a legacy behavior that is problematic on some backends
such as Oracle - in which case it can be set to False.
class sqlalchemy.orm.mapper.Mapper(class_, local_table, properties=None, primary_key=None,
non_primary=False, inherits=None, inherit_condition=None,
inherit_foreign_keys=None,
extension=None,
order_by=False, always_refresh=False, version_id_col=None,
version_id_generator=None,
polymorphic_on=None,
_polymorphic_map=None,
polymorphic_identity=None,
concrete=False,
with_polymorphic=None,
allow_partial_pks=True, batch=True, column_prefix=None,
include_properties=None,
exclude_properties=None,
passive_updates=True,
eager_defaults=False,
legacy_is_orphan=False, _compiled_cache_size=100)
Bases: sqlalchemy.orm.interfaces._InspectionAttr
Define the correlation of class attributes to database table columns.
The Mapper object is instantiated using the mapper() function. For information about instantiating new
Mapper objects, see that function’s documentation.
When mapper() is used explicitly to link a user defined class with table metadata, this is referred to as classical
mapping. Modern SQLAlchemy usage tends to favor the sqlalchemy.ext.declarative extension for
class configuration, which makes usage of mapper() behind the scenes.
68
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
Given a particular class known to be mapped by the ORM, the Mapper which maintains it can be acquired
using the inspect() function:
from sqlalchemy import inspect
mapper = inspect(MyClass)
A class which was mapped by the sqlalchemy.ext.declarative extension will also have its mapper
available via the __mapper__ attribute.
__init__(class_, local_table, properties=None, primary_key=None, non_primary=False, inherits=None, inherit_condition=None, inherit_foreign_keys=None, extension=None, order_by=False, always_refresh=False, version_id_col=None, version_id_generator=None,
polymorphic_on=None,
_polymorphic_map=None,
polymorphic_identity=None,
concrete=False,
with_polymorphic=None,
allow_partial_pks=True,
batch=True,
column_prefix=None,
include_properties=None,
exclude_properties=None,
passive_updates=True,
eager_defaults=False,
legacy_is_orphan=False,
_compiled_cache_size=100)
Construct a new mapper.
Mappers are normally constructed via the mapper() function. See for details.
add_properties(dict_of_properties)
Add the given dictionary of properties to this mapper, using add_property.
add_property(key, prop)
Add an individual MapperProperty to this mapper.
If the mapper has not been configured yet, just adds the property to the initial properties dictionary sent to
the constructor. If this Mapper has already been configured, then the given MapperProperty is configured
immediately.
all_orm_descriptors
A namespace of all _InspectionAttr attributes associated with the mapped class.
These attributes are in all cases Python descriptors associated with the mapped class or its superclasses.
This namespace includes attributes that are mapped to the class as well as attributes declared by extension
modules. It includes any Python descriptor type that inherits from _InspectionAttr. This includes
QueryableAttribute, as well as extension types such as hybrid_property, hybrid_method
and AssociationProxy.
To distinguish between mapped attributes and extension attributes,
the attribute
_InspectionAttr.extension_type will refer to a constant that distinguishes between different extension types.
When dealing with a QueryableAttribute, the QueryableAttribute.property attribute
refers to the MapperProperty property, which is what you get when referring to the collection of
mapped properties via Mapper.attrs.
New in version 0.8.0.
See also:
Mapper.attrs
attrs
A namespace of all MapperProperty objects associated this mapper.
This is an object that provides each property based on its key name. For instance, the mapper for a
User class which has User.name attribute would provide mapper.attrs.name, which would be
2.2. Mapper Configuration
69
SQLAlchemy Documentation, Release 0.8.7
the ColumnProperty representing the name column. The namespace object can also be iterated, which
would yield each MapperProperty.
Mapper has several pre-filtered views of this attribute which limit the types of properties returned, inclding synonyms, column_attrs, relationships, and composites.
See also:
Mapper.all_orm_descriptors
base_mapper = None
The base-most Mapper in an inheritance chain.
In a non-inheriting scenario, this attribute will always be this Mapper. In an inheritance scenario, it
references the Mapper which is parent to all other Mapper objects in the inheritance chain.
This is a read only attribute determined during mapper construction. Behavior is undefined if directly
modified.
c = None
A synonym for columns.
cascade_iterator(type_, state, halt_on=None)
Iterate each element and its mapper in an object graph, for all relationships that meet the given cascade
rule.
Parameters
• type_ – The name of the cascade rule (i.e. save-update, delete, etc.)
• state – The lead InstanceState. child items will be processed per the relationships defined for this object’s mapper.
the return value are object instances; this provides a strong reference so that they don’t fall out of scope
immediately.
class_ = None
The Python class which this Mapper maps.
This is a read only attribute determined during mapper construction. Behavior is undefined if directly
modified.
class_manager = None
The ClassManager which maintains event listeners and class-bound descriptors for this Mapper.
This is a read only attribute determined during mapper construction. Behavior is undefined if directly
modified.
column_attrs
Return a namespace of all ColumnProperty properties maintained by this Mapper.
See also:
Mapper.attrs - namespace of all MapperProperty objects.
columns = None
A collection of Column or other scalar expression objects maintained by this Mapper.
The collection behaves the same as that of the c attribute on any Table object, except that only those
columns included in this mapping are present, and are keyed based on the attribute name defined in the
mapping, not necessarily the key attribute of the Column itself. Additionally, scalar expressions mapped
by column_property() are also present here.
This is a read only attribute determined during mapper construction. Behavior is undefined if directly
modified.
70
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
common_parent(other)
Return true if the given mapper shares a common inherited parent as this mapper.
compile()
Initialize the inter-mapper relationships of all mappers that
Deprecated since version 0.7: Mapper.compile() is replaced by configure_mappers()
have been constructed thus far.
compiled
Deprecated since version 0.7: Mapper.compiled is replaced by Mapper.configured
composites
Return a namespace of all CompositeProperty properties maintained by this Mapper.
See also:
Mapper.attrs - namespace of all MapperProperty objects.
concrete = None
Represent True if this Mapper is a concrete inheritance mapper.
This is a read only attribute determined during mapper construction. Behavior is undefined if directly
modified.
configured = None
Represent True if this Mapper has been configured.
This is a read only attribute determined during mapper construction. Behavior is undefined if directly
modified.
See also:
configure_mappers().
entity
Part of the inspection API.
Returns self.class_.
get_property(key, _configure_mappers=True)
return a MapperProperty associated with the given key.
get_property_by_column(column)
Given a Column object, return the MapperProperty which maps this column.
identity_key_from_instance(instance)
Return the identity key for the given instance, based on its primary key attributes.
If the instance’s state is expired, calling this method will result in a database check to see if the object has
been deleted. If the row no longer exists, ObjectDeletedError is raised.
This value is typically also found on the instance state under the attribute name key.
identity_key_from_primary_key(primary_key)
Return an identity-map key for use in storing/retrieving an item from an identity map.
Parameters primary_key – A list of values indicating the identifier.
identity_key_from_row(row, adapter=None)
Return an identity-map key for use in storing/retrieving an item from the identity map.
Parameters row – A RowProxy instance. The columns which are mapped by this
Mapper should be locatable in the row, preferably via the Column object directly (as
2.2. Mapper Configuration
71
SQLAlchemy Documentation, Release 0.8.7
is the case when a select() construct is executed), or via string names of the form
<tablename>_<colname>.
inherits = None
References the Mapper which this Mapper inherits from, if any.
This is a read only attribute determined during mapper construction. Behavior is undefined if directly
modified.
is_mapper = True
Part of the inspection API.
isa(other)
Return True if the this mapper inherits from the given mapper.
iterate_properties
return an iterator of all MapperProperty objects.
local_table = None
The Selectable which this Mapper manages.
Typically is an instance of Table or Alias. May also be None.
The “local” table is the selectable that the Mapper is directly responsible for managing from an attribute
access and flush perspective. For non-inheriting mappers, the local table is the same as the “mapped” table.
For joined-table inheritance mappers, local_table will be the particular sub-table of the overall “join” which
this Mapper represents. If this mapper is a single-table inheriting mapper, local_table will be None.
See also:
mapped_table.
mapped_table = None
The Selectable to which this Mapper is mapped.
Typically an instance of Table, Join, or Alias.
The “mapped” table is the selectable that the mapper selects from during queries. For non-inheriting mappers, the mapped table is the same as the “local” table. For joined-table inheritance mappers, mapped_table
references the full Join representing full rows for this particular subclass. For single-table inheritance
mappers, mapped_table references the base table.
See also:
local_table.
mapper
Part of the inspection API.
Returns self.
non_primary = None
Represent True if this Mapper is a “non-primary” mapper, e.g. a mapper that is used only to selet rows
but not for persistence management.
This is a read only attribute determined during mapper construction. Behavior is undefined if directly
modified.
polymorphic_identity = None
Represent an identifier which is matched against the polymorphic_on column during result row loading.
Used only with inheritance, this object can be of any type which is comparable to the type of column
represented by polymorphic_on.
72
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
This is a read only attribute determined during mapper construction. Behavior is undefined if directly
modified.
polymorphic_iterator()
Iterate through the collection including this mapper and all descendant mappers.
This includes not just the immediately inheriting mappers but all their inheriting mappers as well.
To iterate through an entire hierarchy, use mapper.base_mapper.polymorphic_iterator().
polymorphic_map = None
A mapping of “polymorphic identity” identifiers mapped to Mapper instances, within an inheritance
scenario.
The identifiers can be of any type which is comparable to the type of column represented by
polymorphic_on.
An inheritance chain of mappers will all reference the same polymorphic map object. The object is used
to correlate incoming result rows to target mappers.
This is a read only attribute determined during mapper construction. Behavior is undefined if directly
modified.
polymorphic_on = None
The Column or SQL expression specified as the polymorphic_on argument for this Mapper, within
an inheritance scenario.
This attribute is normally a Column instance but may also be an expression, such as one derived from
cast().
This is a read only attribute determined during mapper construction. Behavior is undefined if directly
modified.
primary_key = None
An iterable containing the collection of Column objects which comprise the ‘primary key’ of the mapped
table, from the perspective of this Mapper.
This list is against the selectable in mapped_table. In the case of inheriting mappers, some columns
may be managed by a superclass mapper. For example, in the case of a Join, the primary key is determined by all of the primary key columns across all tables referenced by the Join.
The list is also not necessarily the same as the primary key column collection associated with the underlying tables; the Mapper features a primary_key argument that can override what the Mapper
considers as primary key columns.
This is a read only attribute determined during mapper construction. Behavior is undefined if directly
modified.
primary_key_from_instance(instance)
Return the list of primary key values for the given instance.
If the instance’s state is expired, calling this method will result in a database check to see if the object has
been deleted. If the row no longer exists, ObjectDeletedError is raised.
primary_mapper()
Return the primary mapper corresponding to this mapper’s class key (class).
relationships
Return a namespace of all RelationshipProperty properties maintained by this Mapper.
See also:
Mapper.attrs - namespace of all MapperProperty objects.
2.2. Mapper Configuration
73
SQLAlchemy Documentation, Release 0.8.7
selectable
The select() construct this Mapper selects from by default.
Normally, this is equivalent to mapped_table, unless the with_polymorphic feature is in use, in
which case the full “polymorphic” selectable is returned.
self_and_descendants
The collection including this mapper and all descendant mappers.
This includes not just the immediately inheriting mappers but all their inheriting mappers as well.
single = None
Represent True if this Mapper is a single table inheritance mapper.
local_table will be None if this flag is set.
This is a read only attribute determined during mapper construction. Behavior is undefined if directly
modified.
synonyms
Return a namespace of all SynonymProperty properties maintained by this Mapper.
See also:
Mapper.attrs - namespace of all MapperProperty objects.
tables = None
An iterable containing the collection of Table objects which this Mapper is aware of.
If the mapper is mapped to a Join, or an Alias representing a Select, the individual Table objects
that comprise the full construct will be represented here.
This is a read only attribute determined during mapper construction. Behavior is undefined if directly
modified.
validators = None
An immutable dictionary of attributes which have been decorated using the validates() decorator.
The dictionary contains string attribute names as keys mapped to the actual validation method.
with_polymorphic_mappers
The list of Mapper objects included in the default “polymorphic” query.
2.3 Relationship Configuration
This section describes the relationship() function and in depth discussion of its usage. The reference material here continues into the next section, Collection Configuration and Techniques, which has additional detail on
configuration of collections via relationship().
2.3.1 Basic Relational Patterns
A quick walkthrough of the basic relational patterns.
The imports used for each of the following sections is as follows:
from sqlalchemy import Table, Column, Integer, ForeignKey
from sqlalchemy.orm import relationship, backref
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
74
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
One To Many
A one to many relationship places a foreign key on the child table referencing the parent. relationship() is then
specified on the parent, as referencing a collection of items represented by the child:
class Parent(Base):
__tablename__ = 'parent'
id = Column(Integer, primary_key=True)
children = relationship("Child")
class Child(Base):
__tablename__ = 'child'
id = Column(Integer, primary_key=True)
parent_id = Column(Integer, ForeignKey('parent.id'))
To establish a bidirectional relationship in one-to-many, where the “reverse” side is a many to one, specify the
backref option:
class Parent(Base):
__tablename__ = 'parent'
id = Column(Integer, primary_key=True)
children = relationship("Child", backref="parent")
class Child(Base):
__tablename__ = 'child'
id = Column(Integer, primary_key=True)
parent_id = Column(Integer, ForeignKey('parent.id'))
Child will get a parent attribute with many-to-one semantics.
Many To One
Many to one places a foreign key in the parent table referencing the child. relationship() is declared on the
parent, where a new scalar-holding attribute will be created:
class Parent(Base):
__tablename__ = 'parent'
id = Column(Integer, primary_key=True)
child_id = Column(Integer, ForeignKey('child.id'))
child = relationship("Child")
class Child(Base):
__tablename__ = 'child'
id = Column(Integer, primary_key=True)
Bidirectional behavior is achieved by setting backref to the value "parents", which will place a one-to-many
collection on the Child class:
class Parent(Base):
__tablename__ = 'parent'
id = Column(Integer, primary_key=True)
child_id = Column(Integer, ForeignKey('child.id'))
child = relationship("Child", backref="parents")
2.3. Relationship Configuration
75
SQLAlchemy Documentation, Release 0.8.7
One To One
One To One is essentially a bidirectional relationship with a scalar attribute on both sides. To achieve this, the
uselist flag indicates the placement of a scalar attribute instead of a collection on the “many” side of the relationship. To convert one-to-many into one-to-one:
class Parent(Base):
__tablename__ = 'parent'
id = Column(Integer, primary_key=True)
child = relationship("Child", uselist=False, backref="parent")
class Child(Base):
__tablename__ = 'child'
id = Column(Integer, primary_key=True)
parent_id = Column(Integer, ForeignKey('parent.id'))
Or to turn a one-to-many backref into one-to-one, use the backref() function to provide arguments for the reverse
side:
class Parent(Base):
__tablename__ = 'parent'
id = Column(Integer, primary_key=True)
child_id = Column(Integer, ForeignKey('child.id'))
child = relationship("Child", backref=backref("parent", uselist=False))
class Child(Base):
__tablename__ = 'child'
id = Column(Integer, primary_key=True)
Many To Many
Many to Many adds an association table between two classes. The association table is indicated by the secondary
argument to relationship(). Usually, the Table uses the MetaData object associated with the declarative
base class, so that the ForeignKey directives can locate the remote tables with which to link:
association_table = Table('association', Base.metadata,
Column('left_id', Integer, ForeignKey('left.id')),
Column('right_id', Integer, ForeignKey('right.id'))
)
class Parent(Base):
__tablename__ = 'left'
id = Column(Integer, primary_key=True)
children = relationship("Child",
secondary=association_table)
class Child(Base):
__tablename__ = 'right'
id = Column(Integer, primary_key=True)
For a bidirectional relationship, both sides of the relationship contain a collection. The backref keyword will
automatically use the same secondary argument for the reverse relationship:
association_table = Table('association', Base.metadata,
Column('left_id', Integer, ForeignKey('left.id')),
Column('right_id', Integer, ForeignKey('right.id'))
)
76
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
class Parent(Base):
__tablename__ = 'left'
id = Column(Integer, primary_key=True)
children = relationship("Child",
secondary=association_table,
backref="parents")
class Child(Base):
__tablename__ = 'right'
id = Column(Integer, primary_key=True)
The secondary argument of relationship() also accepts a callable that returns the ultimate argument, which
is evaluated only when mappers are first used. Using this, we can define the association_table at a later point,
as long as it’s available to the callable after all module initialization is complete:
class Parent(Base):
__tablename__ = 'left'
id = Column(Integer, primary_key=True)
children = relationship("Child",
secondary=lambda: association_table,
backref="parents")
With the declarative extension in use, the traditional “string name of the table” is accepted as well, matching the name
of the table as stored in Base.metadata.tables:
class Parent(Base):
__tablename__ = 'left'
id = Column(Integer, primary_key=True)
children = relationship("Child",
secondary="association",
backref="parents")
Deleting Rows from the Many to Many Table
A behavior which is unique to the secondary argument to relationship() is that the Table which is specified
here is automatically subject to INSERT and DELETE statements, as objects are added or removed from the collection.
There is no need to delete from this table manually. The act of removing a record from the collection will have the
effect of the row being deleted on flush:
# row will be deleted from the "secondary" table
# automatically
myparent.children.remove(somechild)
A question which often arises is how the row in the “secondary” table can be deleted when the child object is handed
directly to Session.delete():
session.delete(somechild)
There are several possibilities here:
• If there is a relationship() from Parent to Child, but there is not a reverse-relationship that links a
particular Child to each Parent, SQLAlchemy will not have any awareness that when deleting this particular Child object, it needs to maintain the “secondary” table that links it to the Parent. No delete of the
“secondary” table will occur.
• If there is a relationship that links a particular Child to each Parent, suppose it’s called Child.parents,
SQLAlchemy by default will load in the Child.parents collection to locate all Parent objects, and remove
2.3. Relationship Configuration
77
SQLAlchemy Documentation, Release 0.8.7
each row from the “secondary” table which establishes this link. Note that this relationship does not need to be
bidrectional; SQLAlchemy is strictly looking at every relationship() associated with the Child object
being deleted.
• A higher performing option here is to use ON DELETE CASCADE directives with the foreign keys used by
the database. Assuming the database supports this feature, the database itself can be made to automatically
delete rows in the “secondary” table as referencing rows in “child” are deleted. SQLAlchemy can be instructed
to forego actively loading in the Child.parents collection in this case using the passive_deletes
directive on relationship(); see Using Passive Deletes for more details on this.
Note again, these behaviors are only relevant to the secondary option used with relationship(). If dealing with association tables that are mapped explicitly and are not present in the secondary option of a relevant
relationship(), cascade rules can be used instead to automatically delete entities in reaction to a related entity
being deleted - see Cascades for information on this feature.
Association Object
The association object pattern is a variant on many-to-many: it’s used when your association table contains additional
columns beyond those which are foreign keys to the left and right tables. Instead of using the secondary argument,
you map a new class directly to the association table. The left side of the relationship references the association object
via one-to-many, and the association class references the right side via many-to-one. Below we illustrate an association
table mapped to the Association class which includes a column called extra_data, which is a string value that
is stored along with each association between Parent and Child:
class Association(Base):
__tablename__ = 'association'
left_id = Column(Integer, ForeignKey('left.id'), primary_key=True)
right_id = Column(Integer, ForeignKey('right.id'), primary_key=True)
extra_data = Column(String(50))
child = relationship("Child")
class Parent(Base):
__tablename__ = 'left'
id = Column(Integer, primary_key=True)
children = relationship("Association")
class Child(Base):
__tablename__ = 'right'
id = Column(Integer, primary_key=True)
The bidirectional version adds backrefs to both relationships:
class Association(Base):
__tablename__ = 'association'
left_id = Column(Integer, ForeignKey('left.id'), primary_key=True)
right_id = Column(Integer, ForeignKey('right.id'), primary_key=True)
extra_data = Column(String(50))
child = relationship("Child", backref="parent_assocs")
class Parent(Base):
__tablename__ = 'left'
id = Column(Integer, primary_key=True)
children = relationship("Association", backref="parent")
class Child(Base):
__tablename__ = 'right'
id = Column(Integer, primary_key=True)
78
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
Working with the association pattern in its direct form requires that child objects are associated with an association
instance before being appended to the parent; similarly, access from parent to child goes through the association object:
# create parent, append a child via association
p = Parent()
a = Association(extra_data="some data")
a.child = Child()
p.children.append(a)
# iterate through child objects via association, including association
# attributes
for assoc in p.children:
print assoc.extra_data
print assoc.child
To enhance the association object pattern such that direct access to the Association object is optional,
SQLAlchemy provides the Association Proxy extension. This extension allows the configuration of attributes which
will access two “hops” with a single access, one “hop” to the associated object, and a second to a target attribute.
Note: When using the association object pattern, it is advisable that the association-mapped table not be used as
the secondary argument on a relationship() elsewhere, unless that relationship() contains the option
viewonly set to True. SQLAlchemy otherwise may attempt to emit redundant INSERT and DELETE statements
on the same table, if similar state is detected on the related attribute as well as the associated object.
2.3.2 Adjacency List Relationships
The adjacency list pattern is a common relational pattern whereby a table contains a foreign key reference to itself.
This is the most common way to represent hierarchical data in flat tables. Other methods include nested sets, sometimes called “modified preorder”, as well as materialized path. Despite the appeal that modified preorder has when
evaluated for its fluency within SQL queries, the adjacency list model is probably the most appropriate pattern for
the large majority of hierarchical storage needs, for reasons of concurrency, reduced complexity, and that modified
preorder has little advantage over an application which can fully load subtrees into the application space.
In this example, we’ll work with a single mapped class called Node, representing a tree structure:
class Node(Base):
__tablename__ = 'node'
id = Column(Integer, primary_key=True)
parent_id = Column(Integer, ForeignKey('node.id'))
data = Column(String(50))
children = relationship("Node")
With this structure, a graph such as the following:
root --+---> child1
+---> child2 --+--> subchild1
|
+--> subchild2
+---> child3
Would be represented with data such as:
id
--1
2
3
parent_id
------NULL
1
1
data
---root
child1
child2
2.3. Relationship Configuration
79
SQLAlchemy Documentation, Release 0.8.7
4
5
6
3
3
1
subchild1
subchild2
child3
The relationship() configuration here works in the same way as a “normal” one-to-many relationship, with the
exception that the “direction”, i.e. whether the relationship is one-to-many or many-to-one, is assumed by default to
be one-to-many. To establish the relationship as many-to-one, an extra directive is added known as remote_side,
which is a Column or collection of Column objects that indicate those which should be considered to be “remote”:
class Node(Base):
__tablename__ = 'node'
id = Column(Integer, primary_key=True)
parent_id = Column(Integer, ForeignKey('node.id'))
data = Column(String(50))
parent = relationship("Node", remote_side=[id])
Where above, the id column is applied as the remote_side of the parent relationship(), thus establishing
parent_id as the “local” side, and the relationship then behaves as a many-to-one.
As always, both directions can be combined into a bidirectional relationship using the backref() function:
class Node(Base):
__tablename__ = 'node'
id = Column(Integer, primary_key=True)
parent_id = Column(Integer, ForeignKey('node.id'))
data = Column(String(50))
children = relationship("Node",
backref=backref('parent', remote_side=[id])
)
There are several examples included with SQLAlchemy illustrating self-referential strategies; these include Adjacency
List and XML Persistence.
Composite Adjacency Lists
A sub-category of the adjacency list relationship is the rare case where a particular column is present on both the
“local” and “remote” side of the join condition. An example is the Folder class below; using a composite primary
key, the account_id column refers to itself, to indicate sub folders which are within the same account as that of the
parent; while folder_id refers to a specific folder within that account:
class Folder(Base):
__tablename__ = 'folder'
__table_args__ = (
ForeignKeyConstraint(
['account_id', 'parent_id'],
['folder.account_id', 'folder.folder_id']),
)
account_id = Column(Integer, primary_key=True)
folder_id = Column(Integer, primary_key=True)
parent_id = Column(Integer)
name = Column(String)
parent_folder = relationship("Folder",
backref="child_folders",
remote_side=[account_id, folder_id]
)
80
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
Above, we pass account_id into the remote_side list. relationship() recognizes that the account_id
column here is on both sides, and aligns the “remote” column along with the folder_id column, which it recognizes
as uniquely present on the “remote” side.
New in version 0.8: Support for self-referential composite keys in relationship() where a column points to
itself.
Self-Referential Query Strategies
Querying of self-referential structures works like any other query:
# get all nodes named 'child2'
session.query(Node).filter(Node.data=='child2')
However extra care is needed when attempting to join along the foreign key from one level of the tree to the next.
In SQL, a join from a table to itself requires that at least one side of the expression be “aliased” so that it can be
unambiguously referred to.
Recall from Using Aliases in the ORM tutorial that the orm.aliased() construct is normally used to provide an
“alias” of an ORM entity. Joining from Node to itself using this technique looks like:
from sqlalchemy.orm import aliased
nodealias = aliased(Node)
session.query(Node).filter(Node.data=='subchild1').\
join(nodealias, Node.parent).\
filter(nodealias.data=="child2").\
all()
SELECT node.id AS node_id,
node.parent_id AS node_parent_id,
node.data AS node_data
FROM node JOIN node AS node_1
ON node.parent_id = node_1.id
WHERE node.data = ?
AND node_1.data = ?
['subchild1', 'child2']
Query.join() also includes a feature known as Query.join.aliased that can shorten the verbosity selfreferential joins, at the expense of query flexibility. This feature performs a similar “aliasing” step to that above,
without the need for an explicit entity. Calls to Query.filter() and similar subsequent to the aliased join will
adapt the Node entity to be that of the alias:
session.query(Node).filter(Node.data=='subchild1').\
join(Node.parent, aliased=True).\
filter(Node.data=='child2').\
all()
SELECT node.id AS node_id,
node.parent_id AS node_parent_id,
node.data AS node_data
FROM node
JOIN node AS node_1 ON node_1.id = node.parent_id
WHERE node.data = ? AND node_1.data = ?
['subchild1', 'child2']
To add criterion to multiple points along a longer join, add Query.join.from_joinpoint to the additional
join() calls:
2.3. Relationship Configuration
81
SQLAlchemy Documentation, Release 0.8.7
# get all nodes named 'subchild1' with a
# parent named 'child2' and a grandparent 'root'
session.query(Node).\
filter(Node.data=='subchild1').\
join(Node.parent, aliased=True).\
filter(Node.data=='child2').\
join(Node.parent, aliased=True, from_joinpoint=True).\
filter(Node.data=='root').\
all()
SELECT node.id AS node_id,
node.parent_id AS node_parent_id,
node.data AS node_data
FROM node
JOIN node AS node_1 ON node_1.id = node.parent_id
JOIN node AS node_2 ON node_2.id = node_1.parent_id
WHERE node.data = ?
AND node_1.data = ?
AND node_2.data = ?
['subchild1', 'child2', 'root']
Query.reset_joinpoint() will also remove the “aliasing” from filtering calls:
session.query(Node).\
join(Node.children, aliased=True).\
filter(Node.data == 'foo').\
reset_joinpoint().\
filter(Node.data == 'bar')
For an example of using Query.join.aliased to arbitrarily join along a chain of self-referential nodes, see XML
Persistence.
Configuring Self-Referential Eager Loading
Eager loading of relationships occurs using joins or outerjoins from parent to child table during a normal query operation, such that the parent and its immediate child collection or reference can be populated from a single SQL statement, or a second statement for all immediate child collections. SQLAlchemy’s joined and subquery eager loading use
aliased tables in all cases when joining to related items, so are compatible with self-referential joining. However, to use
eager loading with a self-referential relationship, SQLAlchemy needs to be told how many levels deep it should join
and/or query; otherwise the eager load will not take place at all. This depth setting is configured via join_depth:
class Node(Base):
__tablename__ = 'node'
id = Column(Integer, primary_key=True)
parent_id = Column(Integer, ForeignKey('node.id'))
data = Column(String(50))
children = relationship("Node",
lazy="joined",
join_depth=2)
session.query(Node).all()
SELECT node_1.id AS node_1_id,
node_1.parent_id AS node_1_parent_id,
node_1.data AS node_1_data,
node_2.id AS node_2_id,
node_2.parent_id AS node_2_parent_id,
node_2.data AS node_2_data,
node.id AS node_id,
82
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
node.parent_id AS node_parent_id,
node.data AS node_data
FROM node
LEFT OUTER JOIN node AS node_2
ON node.id = node_2.parent_id
LEFT OUTER JOIN node AS node_1
ON node_2.id = node_1.parent_id
[]
2.3.3 Linking Relationships with Backref
The backref keyword argument was first introduced in Object Relational Tutorial, and has been mentioned throughout many of the examples here. What does it actually do ? Let’s start with the canonical User and Address scenario:
from sqlalchemy import Integer, ForeignKey, String, Column
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship
Base = declarative_base()
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String)
addresses = relationship("Address", backref="user")
class Address(Base):
__tablename__ = 'address'
id = Column(Integer, primary_key=True)
email = Column(String)
user_id = Column(Integer, ForeignKey('user.id'))
The above configuration establishes a collection of Address objects on User called User.addresses. It also
establishes a .user attribute on Address which will refer to the parent User object.
In fact, the backref keyword is only a common shortcut for placing a second relationship() onto the
Address mapping, including the establishment of an event listener on both sides which will mirror attribute operations in both directions. The above configuration is equivalent to:
from sqlalchemy import Integer, ForeignKey, String, Column
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship
Base = declarative_base()
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String)
addresses = relationship("Address", back_populates="user")
class Address(Base):
__tablename__ = 'address'
id = Column(Integer, primary_key=True)
email = Column(String)
2.3. Relationship Configuration
83
SQLAlchemy Documentation, Release 0.8.7
user_id = Column(Integer, ForeignKey('user.id'))
user = relationship("User", back_populates="addresses")
Above, we add a .user relationship to Address explicitly. On both relationships, the back_populates directive
tells each relationship about the other one, indicating that they should establish “bidirectional” behavior between each
other. The primary effect of this configuration is that the relationship adds event handlers to both attributes which
have the behavior of “when an append or set event occurs here, set ourselves onto the incoming attribute using this
particular attribute name”. The behavior is illustrated as follows. Start with a User and an Address instance. The
.addresses collection is empty, and the .user attribute is None:
>>> u1 = User()
>>> a1 = Address()
>>> u1.addresses
[]
>>> print a1.user
None
However, once the Address is appended to the u1.addresses collection, both the collection and the scalar
attribute have been populated:
>>> u1.addresses.append(a1)
>>> u1.addresses
[<__main__.Address object at 0x12a6ed0>]
>>> a1.user
<__main__.User object at 0x12a6590>
This behavior of course works in reverse for removal operations as well, as well as for equivalent operations on both
sides. Such as when .user is set again to None, the Address object is removed from the reverse collection:
>>> a1.user = None
>>> u1.addresses
[]
The manipulation of the .addresses collection and the .user attribute occurs entirely in Python without any
interaction with the SQL database. Without this behavior, the proper state would be apparent on both sides once
the data has been flushed to the database, and later reloaded after a commit or expiration operation occurs. The
backref/back_populates behavior has the advantage that common bidirectional operations can reflect the correct state without requiring a database round trip.
Remember, when the backref keyword is used on a single relationship, it’s exactly the same as if the above two
relationships were created individually using back_populates on each.
Backref Arguments
We’ve established that the backref keyword is merely a shortcut for building two individual relationship()
constructs that refer to each other. Part of the behavior of this shortcut is that certain configurational arguments
applied to the relationship() will also be applied to the other direction - namely those arguments that describe
the relationship at a schema level, and are unlikely to be different in the reverse direction. The usual case here is a
many-to-many relationship() that has a secondary argument, or a one-to-many or many-to-one which has a
primaryjoin argument (the primaryjoin argument is discussed in Specifying Alternate Join Conditions). Such
as if we limited the list of Address objects to those which start with “tony”:
from sqlalchemy import Integer, ForeignKey, String, Column
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship
84
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
Base = declarative_base()
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String)
addresses = relationship("Address",
primaryjoin="and_(User.id==Address.user_id, "
"Address.email.startswith('tony'))",
backref="user")
class Address(Base):
__tablename__ = 'address'
id = Column(Integer, primary_key=True)
email = Column(String)
user_id = Column(Integer, ForeignKey('user.id'))
We can observe, by inspecting the resulting property, that both sides of the relationship have this join condition applied:
>>> print
"user".id
>>>
>>> print
"user".id
>>>
User.addresses.property.primaryjoin
= address.user_id AND address.email LIKE :email_1 || '%%'
Address.user.property.primaryjoin
= address.user_id AND address.email LIKE :email_1 || '%%'
This reuse of arguments should pretty much do the “right thing” - it uses only arguments that are applicable, and in the
case of a many-to- many relationship, will reverse the usage of primaryjoin and secondaryjoin to correspond
to the other direction (see the example in Self-Referential Many-to-Many Relationship for this).
It’s very often the case however that we’d like to specify arguments that are specific to just the side where we happened
to place the “backref”. This includes relationship() arguments like lazy, remote_side, cascade and
cascade_backrefs. For this case we use the backref() function in place of a string:
# <other imports>
from sqlalchemy.orm import backref
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String)
addresses = relationship("Address",
backref=backref("user", lazy="joined"))
Where above, we placed a lazy="joined" directive only on the Address.user side, indicating that when a
query against Address is made, a join to the User entity should be made automatically which will populate the
.user attribute of each returned Address. The backref() function formatted the arguments we gave it into
a form that is interpreted by the receiving relationship() as additional arguments to be applied to the new
relationship it creates.
One Way Backrefs
An unusual case is that of the “one way backref”. This is where the “back-populating” behavior of the backref is only
desirable in one direction. An example of this is a collection which contains a filtering primaryjoin condition.
We’d like to append items to this collection as needed, and have them populate the “parent” object on the incoming
2.3. Relationship Configuration
85
SQLAlchemy Documentation, Release 0.8.7
object. However, we’d also like to have items that are not part of the collection, but still have the same “parent”
association - these items should never be in the collection.
Taking our previous example, where we established a primaryjoin that limited the collection only to Address
objects whose email address started with the word tony, the usual backref behavior is that all items populate in both
directions. We wouldn’t want this behavior for a case like the following:
>>> u1 = User()
>>> a1 = Address(email='mary')
>>> a1.user = u1
>>> u1.addresses
[<__main__.Address object at 0x1411910>]
Above, the Address object that doesn’t match the criterion of “starts with ‘tony”’ is present in the addresses
collection of u1. After these objects are flushed, the transaction committed and their attributes expired for a re-load,
the addresses collection will hit the database on next access and no longer have this Address object present, due
to the filtering condition. But we can do away with this unwanted side of the “backref” behavior on the Python side
by using two separate relationship() constructs, placing back_populates only on one side:
from sqlalchemy import Integer, ForeignKey, String, Column
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship
Base = declarative_base()
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String)
addresses = relationship("Address",
primaryjoin="and_(User.id==Address.user_id, "
"Address.email.startswith('tony'))",
back_populates="user")
class Address(Base):
__tablename__ = 'address'
id = Column(Integer, primary_key=True)
email = Column(String)
user_id = Column(Integer, ForeignKey('user.id'))
user = relationship("User")
With the above scenario, appending an Address object to the .addresses collection of a User will always
establish the .user attribute on that Address:
>>> u1 = User()
>>> a1 = Address(email='tony')
>>> u1.addresses.append(a1)
>>> a1.user
<__main__.User object at 0x1411850>
However, applying a User to the .user attribute of an Address, will not append the Address object to the
collection:
>>> a2 = Address(email='mary')
>>> a2.user = u1
>>> a2 in u1.addresses
False
Of course, we’ve disabled some of the usefulness of backref here, in that when we do append an Address that
corresponds to the criteria of email.startswith(’tony’), it won’t show up in the User.addresses col86
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
lection until the session is flushed, and the attributes reloaded after a commit or expire operation. While we could
consider an attribute event that checks this criterion in Python, this starts to cross the line of duplicating too much SQL
behavior in Python. The backref behavior itself is only a slight transgression of this philosophy - SQLAlchemy tries
to keep these to a minimum overall.
2.3.4 Configuring how Relationship Joins
relationship() will normally create a join between two tables by examining the foreign key relationship between
the two tables to determine which columns should be compared. There are a variety of situations where this behavior
needs to be customized.
Handling Multiple Join Paths
One of the most common situations to deal with is when there are more than one foreign key path between two tables.
Consider a Customer class that contains two foreign keys to an Address class:
from sqlalchemy import Integer, ForeignKey, String, Column
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship
Base = declarative_base()
class Customer(Base):
__tablename__ = 'customer'
id = Column(Integer, primary_key=True)
name = Column(String)
billing_address_id = Column(Integer, ForeignKey("address.id"))
shipping_address_id = Column(Integer, ForeignKey("address.id"))
billing_address = relationship("Address")
shipping_address = relationship("Address")
class Address(Base):
__tablename__ = 'address'
id = Column(Integer, primary_key=True)
street = Column(String)
city = Column(String)
state = Column(String)
zip = Column(String)
The above mapping, when we attempt to use it, will produce the error:
sqlalchemy.exc.AmbiguousForeignKeysError: Could not determine join
condition between parent/child tables on relationship
Customer.billing_address - there are multiple foreign key
paths linking the tables. Specify the 'foreign_keys' argument,
providing a list of those columns which should be
counted as containing a foreign key reference to the parent table.
The above message is pretty long. There are many potential messages that relationship() can return, which
have been carefully tailored to detect a variety of common configurational issues; most will suggest the additional
configuration that’s needed to resolve the ambiguity or other missing information.
In this case, the message wants us to qualify each relationship() by instructing for each one which foreign key
column should be considered, and the appropriate form is as follows:
2.3. Relationship Configuration
87
SQLAlchemy Documentation, Release 0.8.7
class Customer(Base):
__tablename__ = 'customer'
id = Column(Integer, primary_key=True)
name = Column(String)
billing_address_id = Column(Integer, ForeignKey("address.id"))
shipping_address_id = Column(Integer, ForeignKey("address.id"))
billing_address = relationship("Address", foreign_keys=[billing_address_id])
shipping_address = relationship("Address", foreign_keys=[shipping_address_id])
Above, we specify the foreign_keys argument, which is a Column or list of Column objects which indicate those columns to be considered “foreign”, or in other words, the columns that contain a value referring to
a parent table. Loading the Customer.billing_address relationship from a Customer object will use
the value present in billing_address_id in order to identify the row in Address to be loaded; similarly,
shipping_address_id is used for the shipping_address relationship. The linkage of the two columns also
plays a role during persistence; the newly generated primary key of a just-inserted Address object will be copied
into the appropriate foreign key column of an associated Customer object during a flush.
When specifying foreign_keys with Declarative, we can also use string names to specify, however it is important
that if using a list, the list is part of the string:
billing_address = relationship("Address", foreign_keys="[Customer.billing_address_id]")
In this specific example, the list is not necessary in any case as there’s only one Column we need:
billing_address = relationship("Address", foreign_keys="Customer.billing_address_id")
Changed in version 0.8: relationship() can resolve ambiguity between foreign key targets on the basis of the
foreign_keys argument alone; the primaryjoin argument is no longer needed in this situation.
Specifying Alternate Join Conditions
The default behavior of relationship() when constructing a join is that it equates the value of primary key
columns on one side to that of foreign-key-referring columns on the other. We can change this criterion to be anything we’d like using the primaryjoin argument, as well as the secondaryjoin argument in the case when a
“secondary” table is used.
In the example below, using the User class as well as an Address class which stores a street address, we create a
relationship boston_addresses which will only load those Address objects which specify a city of “Boston”:
from sqlalchemy import Integer, ForeignKey, String, Column
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship
Base = declarative_base()
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String)
boston_addresses = relationship("Address",
primaryjoin="and_(User.id==Address.user_id, "
"Address.city=='Boston')")
class Address(Base):
__tablename__ = 'address'
id = Column(Integer, primary_key=True)
88
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
user_id = Column(Integer, ForeignKey('user.id'))
street = Column(String)
city = Column(String)
state = Column(String)
zip = Column(String)
Within this string SQL expression, we made use of the and_() conjunction construct to establish two distinct predicates for the join condition - joining both the User.id and Address.user_id columns to each other, as well
as limiting rows in Address to just city=’Boston’. When using Declarative, rudimentary SQL functions like
and_() are automatically available in the evaluated namespace of a string relationship() argument.
The custom criteria we use in a primaryjoin is generally only significant when SQLAlchemy is rendering SQL in
order to load or represent this relationship. That is, it’s used in the SQL statement that’s emitted in order to perform
a per-attribute lazy load, or when a join is constructed at query time, such as via Query.join(), or via the eager
“joined” or “subquery” styles of loading. When in-memory objects are being manipulated, we can place any Address
object we’d like into the boston_addresses collection, regardless of what the value of the .city attribute is.
The objects will remain present in the collection until the attribute is expired and re-loaded from the database where the
criterion is applied. When a flush occurs, the objects inside of boston_addresses will be flushed unconditionally,
assigning value of the primary key user.id column onto the foreign-key-holding address.user_id column for
each row. The city criteria has no effect here, as the flush process only cares about synchronizing primary key values
into referencing foreign key values.
Creating Custom Foreign Conditions
Another element of the primary join condition is how those columns considered “foreign” are determined. Usually,
some subset of Column objects will specify ForeignKey, or otherwise be part of a ForeignKeyConstraint
that’s relevant to the join condition. relationship() looks to this foreign key status as it decides how it
should load and persist data for this relationship. However, the primaryjoin argument can be used to create a
join condition that doesn’t involve any “schema” level foreign keys. We can combine primaryjoin along with
foreign_keys and remote_side explicitly in order to establish such a join.
Below, a class HostEntry joins to itself, equating the string content column to the ip_address column, which
is a Postgresql type called INET. We need to use cast() in order to cast one side of the join to the type of the other:
from sqlalchemy import cast, String, Column, Integer
from sqlalchemy.orm import relationship
from sqlalchemy.dialects.postgresql import INET
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class HostEntry(Base):
__tablename__ = 'host_entry'
id = Column(Integer, primary_key=True)
ip_address = Column(INET)
content = Column(String(50))
# relationship() using explicit foreign_keys, remote_side
parent_host = relationship("HostEntry",
primaryjoin=ip_address == cast(content, INET),
foreign_keys=content,
remote_side=ip_address
)
2.3. Relationship Configuration
89
SQLAlchemy Documentation, Release 0.8.7
The above relationship will produce a join like:
SELECT host_entry.id, host_entry.ip_address, host_entry.content
FROM host_entry JOIN host_entry AS host_entry_1
ON host_entry_1.ip_address = CAST(host_entry.content AS INET)
An alternative syntax to the above is to use the foreign() and remote() annotations, inline within the
primaryjoin expression. This syntax represents the annotations that relationship() normally applies by
itself to the join condition given the foreign_keys and remote_side arguments; the functions are provided in
the API in the rare case that relationship() can’t determine the exact location of these features on its own:
from sqlalchemy.orm import foreign, remote
class HostEntry(Base):
__tablename__ = 'host_entry'
id = Column(Integer, primary_key=True)
ip_address = Column(INET)
content = Column(String(50))
# relationship() using explicit foreign() and remote() annotations
# in lieu of separate arguments
parent_host = relationship("HostEntry",
primaryjoin=remote(ip_address) == \
cast(foreign(content), INET),
)
Self-Referential Many-to-Many Relationship
Many to many relationships can be customized by one or both of primaryjoin and secondaryjoin - the latter
is significant for a relationship that specifies a many-to-many reference using the secondary argument. A common
situation which involves the usage of primaryjoin and secondaryjoin is when establishing a many-to-many
relationship from a class to itself, as shown below:
from sqlalchemy import Integer, ForeignKey, String, Column, Table
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship
Base = declarative_base()
node_to_node = Table("node_to_node", Base.metadata,
Column("left_node_id", Integer, ForeignKey("node.id"), primary_key=True),
Column("right_node_id", Integer, ForeignKey("node.id"), primary_key=True)
)
class Node(Base):
__tablename__ = 'node'
id = Column(Integer, primary_key=True)
label = Column(String)
right_nodes = relationship("Node",
secondary=node_to_node,
primaryjoin=id==node_to_node.c.left_node_id,
secondaryjoin=id==node_to_node.c.right_node_id,
backref="left_nodes"
)
Where above, SQLAlchemy can’t know automatically which columns should connect to which for the right_nodes
and left_nodes relationships. The primaryjoin and secondaryjoin arguments establish how we’d like to
90
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
join to the association table. In the Declarative form above, as we are declaring these conditions within the Python
block that corresponds to the Node class, the id variable is available directly as the Column object we wish to join
with.
A classical mapping situation here is similar, where node_to_node can be joined to node.c.id:
from sqlalchemy import Integer, ForeignKey, String, Column, Table, MetaData
from sqlalchemy.orm import relationship, mapper
metadata = MetaData()
node_to_node = Table("node_to_node", metadata,
Column("left_node_id", Integer, ForeignKey("node.id"), primary_key=True),
Column("right_node_id", Integer, ForeignKey("node.id"), primary_key=True)
)
node = Table("node", metadata,
Column('id', Integer, primary_key=True),
Column('label', String)
)
class Node(object):
pass
mapper(Node, node, properties={
'right_nodes':relationship(Node,
secondary=node_to_node,
primaryjoin=node.c.id==node_to_node.c.left_node_id,
secondaryjoin=node.c.id==node_to_node.c.right_node_id,
backref="left_nodes"
)})
Note that in both examples, the backref keyword specifies a left_nodes backref - when relationship()
creates the second relationship in the reverse direction, it’s smart enough to reverse the primaryjoin and
secondaryjoin arguments.
Building Query-Enabled Properties
Very ambitious custom join conditions may fail to be directly persistable, and in some cases may not even load
correctly. To remove the persistence part of the equation, use the flag viewonly on the relationship(), which
establishes it as a read-only attribute (data written to the collection will be ignored on flush()). However, in extreme
cases, consider using a regular Python property in conjunction with Query as follows:
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
def _get_addresses(self):
return object_session(self).query(Address).with_parent(self).filter(...).all()
addresses = property(_get_addresses)
2.3.5 Rows that point to themselves / Mutually Dependent Rows
This is a very specific case where relationship() must perform an INSERT and a second UPDATE in order to properly
populate a row (and vice versa an UPDATE and DELETE in order to delete without violating foreign key constraints).
The two use cases are:
2.3. Relationship Configuration
91
SQLAlchemy Documentation, Release 0.8.7
• A table contains a foreign key to itself, and a single row will have a foreign key value pointing to its own primary
key.
• Two tables each contain a foreign key referencing the other table, with a row in each table referencing the other.
For example:
user
--------------------------------user_id
name
related_user_id
1
'ed'
1
Or:
widget
------------------------------------------widget_id
name
favorite_entry_id
1
'somewidget'
5
entry
--------------------------------entry_id
name
widget_id
5
'someentry'
1
In the first case, a row points to itself. Technically, a database that uses sequences such as PostgreSQL or Oracle
can INSERT the row at once using a previously generated value, but databases which rely upon autoincrement-style
primary key identifiers cannot. The relationship() always assumes a “parent/child” model of row population
during flush, so unless you are populating the primary key/foreign key columns directly, relationship() needs
to use two statements.
In the second case, the “widget” row must be inserted before any referring “entry” rows, but then the “favorite_entry_id” column of that “widget” row cannot be set until the “entry” rows have been generated. In this case,
it’s typically impossible to insert the “widget” and “entry” rows using just two INSERT statements; an UPDATE must
be performed in order to keep foreign key constraints fulfilled. The exception is if the foreign keys are configured
as “deferred until commit” (a feature some databases support) and if the identifiers were populated manually (again
essentially bypassing relationship()).
To enable the usage of a supplementary UPDATE statement, we use the post_update option of
relationship(). This specifies that the linkage between the two rows should be created using an UPDATE
statement after both rows have been INSERTED; it also causes the rows to be de-associated with each other via UPDATE before a DELETE is emitted. The flag should be placed on just one of the relationships, preferably the manyto-one side. Below we illustrate a complete example, including two ForeignKey constructs, one which specifies
use_alter to help with emitting CREATE TABLE statements:
from sqlalchemy import Integer, ForeignKey, Column
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship
Base = declarative_base()
class Entry(Base):
__tablename__ = 'entry'
entry_id = Column(Integer, primary_key=True)
widget_id = Column(Integer, ForeignKey('widget.widget_id'))
name = Column(String(50))
class Widget(Base):
__tablename__ = 'widget'
widget_id = Column(Integer, primary_key=True)
favorite_entry_id = Column(Integer,
ForeignKey('entry.entry_id',
use_alter=True,
name="fk_favorite_entry"))
name = Column(String(50))
92
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
entries = relationship(Entry, primaryjoin=
widget_id==Entry.widget_id)
favorite_entry = relationship(Entry,
primaryjoin=
favorite_entry_id==Entry.entry_id,
post_update=True)
When a structure against the above configuration is flushed, the “widget” row will be INSERTed minus the “favorite_entry_id” value, then all the “entry” rows will be INSERTed referencing the parent “widget” row, and then an
UPDATE statement will populate the “favorite_entry_id” column of the “widget” table (it’s one row at a time for the
time being):
>>> w1 = Widget(name='somewidget')
>>> e1 = Entry(name='someentry')
>>> w1.favorite_entry = e1
>>> w1.entries = [e1]
>>> session.add_all([w1, e1])
>>> session.commit()
BEGIN (implicit)
INSERT INTO widget (favorite_entry_id, name) VALUES (?, ?)
(None, 'somewidget')
INSERT INTO entry (widget_id, name) VALUES (?, ?)
(1, 'someentry')
UPDATE widget SET favorite_entry_id=? WHERE widget.widget_id = ?
(1, 1)
COMMIT
An additional configuration we can specify is to supply a more comprehensive foreign key constraint on Widget,
such that it’s guaranteed that favorite_entry_id refers to an Entry that also refers to this Widget. We can
use a composite foreign key, as illustrated below:
from sqlalchemy import Integer, ForeignKey, String, \
Column, UniqueConstraint, ForeignKeyConstraint
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship
Base = declarative_base()
class Entry(Base):
__tablename__ = 'entry'
entry_id = Column(Integer, primary_key=True)
widget_id = Column(Integer, ForeignKey('widget.widget_id'))
name = Column(String(50))
__table_args__ = (
UniqueConstraint("entry_id", "widget_id"),
)
class Widget(Base):
__tablename__ = 'widget'
widget_id = Column(Integer, autoincrement='ignore_fk', primary_key=True)
favorite_entry_id = Column(Integer)
name = Column(String(50))
__table_args__ = (
ForeignKeyConstraint(
["widget_id", "favorite_entry_id"],
2.3. Relationship Configuration
93
SQLAlchemy Documentation, Release 0.8.7
["entry.widget_id", "entry.entry_id"],
name="fk_favorite_entry", use_alter=True
),
)
entries = relationship(Entry, primaryjoin=
widget_id==Entry.widget_id,
foreign_keys=Entry.widget_id)
favorite_entry = relationship(Entry,
primaryjoin=
favorite_entry_id==Entry.entry_id,
foreign_keys=favorite_entry_id,
post_update=True)
The above mapping features a composite ForeignKeyConstraint bridging the widget_id and
favorite_entry_id columns. To ensure that Widget.widget_id remains an “autoincrementing” column we
specify autoincrement to the value "ignore_fk" on Column, and additionally on each relationship()
we must limit those columns considered as part of the foreign key for the purposes of joining and cross-population.
2.3.6 Mutable Primary Keys / Update Cascades
When the primary key of an entity changes, related items which reference the primary key must also be updated as
well. For databases which enforce referential integrity, it’s required to use the database’s ON UPDATE CASCADE
functionality in order to propagate primary key changes to referenced foreign keys - the values cannot be out of sync
for any moment.
For databases that don’t support this, such as SQLite and MySQL without their referential integrity options turned
on, the passive_updates flag can be set to False, most preferably on a one-to-many or many-to-many
relationship(), which instructs SQLAlchemy to issue UPDATE statements individually for objects referenced
in the collection, loading them into memory if not already locally present. The passive_updates flag can also
be False in conjunction with ON UPDATE CASCADE functionality, although in that case the unit of work will be
issuing extra SELECT and UPDATE statements unnecessarily.
A typical mutable primary key setup might look like:
class User(Base):
__tablename__ = 'user'
username = Column(String(50), primary_key=True)
fullname = Column(String(100))
# passive_updates=False *only* needed if the database
# does not implement ON UPDATE CASCADE
addresses = relationship("Address", passive_updates=False)
class Address(Base):
__tablename__ = 'address'
email = Column(String(50), primary_key=True)
username = Column(String(50),
ForeignKey('user.username', onupdate="cascade")
)
passive_updates is set to True by default, indicating that ON UPDATE CASCADE is expected to be in place
in the usual case for foreign keys that expect to have a mutating parent key.
A passive_updates setting of False may be configured on any direction of relationship, i.e. one-to-many, many-
94
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
to-one, and many-to-many, although it is much more effective when placed just on the one-to-many or many-to-many
side. Configuring the passive_updates to False only on the many-to-one side will have only a partial effect, as
the unit of work searches only through the current identity map for objects that may be referencing the one with a
mutating primary key, not throughout the database.
2.3.7 Relationships API
sqlalchemy.orm.relationship(argument, secondary=None, **kwargs)
Provide a relationship between two mapped classes.
This corresponds to a parent-child or associative table relationship. The constructed class is an instance of
RelationshipProperty.
A typical relationship(), used in a classical mapping:
mapper(Parent, properties={
'children': relationship(Child)
})
Some arguments accepted by relationship() optionally accept a callable function, which when called
produces the desired value. The callable is invoked by the parent Mapper at “mapper initialization” time, which
happens only when mappers are first used, and is assumed to be after all mappings have been constructed. This
can be used to resolve order-of-declaration and other dependency issues, such as if Child is declared below
Parent in the same file:
mapper(Parent, properties={
"children":relationship(lambda: Child,
order_by=lambda: Child.id)
})
When using the Declarative extension, the Declarative initializer allows string arguments to be passed to
relationship(). These string arguments are converted into callables that evaluate the string as Python
code, using the Declarative class-registry as a namespace. This allows the lookup of related classes to be automatic via their string name, and removes the need to import related classes at all into the local module space:
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Parent(Base):
__tablename__ = 'parent'
id = Column(Integer, primary_key=True)
children = relationship("Child", order_by="Child.id")
See also:
Relationship Configuration - Full introductory and reference documentation for relationship().
Building a Relationship - ORM tutorial introduction.
Parameters
• argument – a mapped class, or actual Mapper instance, representing the target of the
relationship.
argument may also be passed as a callable function which is evaluated at mapper initialization time, and may be passed as a Python-evaluable string when using Declarative.
See also:
2.3. Relationship Configuration
95
SQLAlchemy Documentation, Release 0.8.7
Configuring Relationships - further detail on relationship configuration when using Declarative.
• secondary – for a many-to-many relationship, specifies the intermediary table, and is
typically an instance of Table. In less common circumstances, the argument may also be
specified as an Alias construct, or even a Join construct.
secondary may also be passed as a callable function which is evaluated at mapper initialization time. When using Declarative, it may also be a string argument noting the name
of a Table that is present in the MetaData collection associated with the parent-mapped
Table.
The secondary keyword argument is typically applied in the case where the intermediary
Table is not otherwise exprssed in any direct class mapping. If the “secondary” table is also
explicitly mapped elsewhere (e.g. as in Association Object), one should consider applying
the viewonly flag so that this relationship() is not used for persistence operations
which may conflict with those of the association object pattern.
See also:
Many To Many - Reference example of “many to many”.
Building a Many To Many Relationship - ORM tutorial introduction to many-to-many relationships.
Self-Referential Many-to-Many Relationship - Specifics on using many-to-many in a selfreferential case.
Configuring Many-to-Many Relationships - Additional options when using Declarative.
Association Object - an alternative to secondary when composing association table relationships, allowing additional attributes to be specified on the association table.
• active_history=False – When True, indicates that the “previous” value for a
many-to-one reference should be loaded when replaced, if not already loaded. Normally,
history tracking logic for simple many-to-ones only needs to be aware of the “new” value
in order to perform a flush. This flag is available for applications that make use of
attributes.get_history() which also need to know the “previous” value of the
attribute.
• backref – indicates the string name of a property to be placed on the related mapper’s
class that will handle this relationship in the other direction. The other property will be created automatically when the mappers are configured. Can also be passed as a backref()
object to control the configuration of the new relationship.
See also:
Linking Relationships with Backref - Introductory documentation and examples.
back_populates - alternative form of backref specification.
backref() - allows control over relationship() configuration when using
backref.
• back_populates – Takes a string name and has the same meaning as backref, except the complementing property is not created automatically, and instead must be configured explicitly on the other mapper. The complementing property should also indicate
back_populates to this relationship to ensure proper functioning.
See also:
Linking Relationships with Backref - Introductory documentation and examples.
96
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
backref - alternative form of backref specification.
• cascade – a comma-separated list of cascade rules which determines how Session operations should be “cascaded” from parent to child. This defaults to False, which means the
default cascade should be used - this default cascade is "save-update, merge".
The available cascades are save-update, merge, expunge, delete,
delete-orphan, and refresh-expire. An additional option, all indicates shorthand for "save-update, merge, refresh-expire, expunge, delete",
and is often used as in "all, delete-orphan" to indicate that related objects should
follow along with the parent object in all cases, and be deleted when de-associated.
See also:
Cascades - Full detail on each of the available cascade options.
Configuring delete/delete-orphan Cascade - Tutorial example describing a delete cascade.
• cascade_backrefs=True – a boolean value indicating if the save-update cascade
should operate along an assignment event intercepted by a backref. When set to False, the
attribute managed by this relationship will not cascade an incoming transient object into the
session of a persistent parent, if the event is received via backref.
See also:
Controlling Cascade on Backrefs - Full discussion and examples on how the
cascade_backrefs option is used.
• collection_class – a class or callable that returns a new list-holding object. will be
used in place of a plain list for storing elements.
See also:
Customizing Collection Access - Introductory documentation and examples.
• comparator_factory
–
a
class
which
extends
RelationshipProperty.Comparator which provides custom SQL clause
generation for comparison operations.
See also:
PropComparator - some detail on redefining comparators at this level.
Operator Customization - Brief intro to this feature.
• distinct_target_key=None – Indicate if a “subquery” eager load should apply the
DISTINCT keyword to the innermost SELECT statement. When left as None, the DISTINCT keyword will be applied in those cases when the target columns do not comprise the
full primary key of the target table. When set to True, the DISTINCT keyword is applied
to the innermost SELECT unconditionally.
It may be desirable to set this flag to False when the DISTINCT is reducing performance of
the innermost subquery beyond that of what duplicate innermost rows may be causing.
New in version 0.8.3: - distinct_target_key allows the subquery eager loader to
apply a DISTINCT modifier to the innermost SELECT.
See also:
Relationship Loading Techniques - includes an introduction to subquery eager loading.
• doc – docstring which will be applied to the resulting descriptor.
• extension – an AttributeExtension instance, or list of extensions, which will be
prepended to the list of attribute listeners for the resulting descriptor placed on the class.
2.3. Relationship Configuration
97
SQLAlchemy Documentation, Release 0.8.7
Deprecated since version 0.7: Please see AttributeEvents.
• foreign_keys – a list of columns which are to be used as “foreign key” columns,
or columns which refer to the value in a remote column, within the context of this
relationship() object’s primaryjoin condition. That is, if the primaryjoin
condition of this relationship() is a.id == b.a_id, and the values in
b.a_id are required to be present in a.id, then the “foreign key” column of this
relationship() is b.a_id.
In normal cases, the foreign_keys parameter is not required. relationship() will
automatically determine which columns in the primaryjoin conditition are to be considered “foreign key” columns based on those Column objects that specify ForeignKey,
or are otherwise listed as referencing columns in a ForeignKeyConstraint construct.
foreign_keys is only needed when:
1. There is more than one way to construct a join from the local table to the remote table, as
there are multiple foreign key references present. Setting foreign_keys will limit the
relationship() to consider just those columns specified here as “foreign”.
Changed in version 0.8: A multiple-foreign key join ambiguity can be resolved by setting
the foreign_keys parameter alone, without the need to explicitly set primaryjoin
as well.
2. The Table being mapped does not actually have ForeignKey or
ForeignKeyConstraint constructs present, often because the table was reflected
from a database that does not support foreign key reflection (MySQL MyISAM).
3. The primaryjoin argument is used to construct a non-standard join condition, which
makes use of columns or expressions that do not normally refer to their “parent” column,
such as a join condition expressed by a complex comparison using a SQL function.
The relationship() construct will raise informative error messages that suggest the
use of the foreign_keys parameter when presented with an ambiguous condition. In
typical cases, if relationship() doesn’t raise any exceptions, the foreign_keys
parameter is usually not needed.
foreign_keys may also be passed as a callable function which is evaluated at mapper
initialization time, and may be passed as a Python-evaluable string when using Declarative.
See also:
Handling Multiple Join Paths
Creating Custom Foreign Conditions
foreign() - allows direct annotation of the “foreign” columns within a primaryjoin
condition.
New in version 0.8: The foreign() annotation can also be applied directly to the
primaryjoin expression, which is an alternate, more specific system of describing which
columns in a particular primaryjoin should be considered “foreign”.
• info – Optional data dictionary which
MapperProperty.info attribute of this object.
will
be
populated
into
the
New in version 0.8.
• innerjoin=False – when True, joined eager loads will use an inner join to join against
related tables instead of an outer join. The purpose of this option is generally one of performance, as inner joins generally perform better than outer joins.
98
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
This flag can be set to True when the relationship references an object via many-to-one
using local foreign keys that are not nullable, or when the reference is one-to-one or a
collection that is guaranteed to have one or at least one entry.
See also:
What Kind of Loading to Use ? - Discussion of some details of various loader options.
• join_depth – when non-None, an integer value indicating how many levels deep “eager”
loaders should join on a self-referring or cyclical relationship. The number counts how
many times the same Mapper shall be present in the loading condition along a particular
join branch. When left at its default of None, eager loaders will stop chaining when they
encounter a the same target mapper which is already higher up in the chain. This option
applies both to joined- and subquery- eager loaders.
See also:
Configuring Self-Referential Eager Loading - Introductory documentation and examples.
• lazy=’select’ – specifies how the related items should be loaded. Default value is
select. Values include:
– select - items should be loaded lazily when the property is first accessed, using a
separate SELECT statement, or identity map fetch for simple many-to-one references.
– immediate - items should be loaded as the parents are loaded, using a separate SELECT
statement, or identity map fetch for simple many-to-one references.
– joined - items should be loaded “eagerly” in the same query as that of the parent, using
a JOIN or LEFT OUTER JOIN. Whether the join is “outer” or not is determined by the
innerjoin parameter.
– subquery - items should be loaded “eagerly” as the parents are loaded, using one additional SQL statement, which issues a JOIN to a subquery of the original statement, for
each collection requested.
– noload - no loading should occur at any time. This is to support “write-only” attributes,
or attributes which are populated in some manner specific to the application.
– dynamic - the attribute will return a pre-configured Query object for all read operations, onto which further filtering operations can be applied before iterating the results.
See the section Dynamic Relationship Loaders for more details.
– True - a synonym for ‘select’
– False - a synonym for ‘joined’
– None - a synonym for ‘noload’
See also:
Relationship Loading Techniques - Full documentation on relationship loader configuration.
Dynamic Relationship Loaders - detail on the dynamic option.
• load_on_pending=False – Indicates loading behavior for transient or pending parent
objects.
When set to True, causes the lazy-loader to issue a query for a parent object that is not
persistent, meaning it has never been flushed. This may take effect for a pending object
when autoflush is disabled, or for a transient object that has been “attached” to a Session
but is not part of its pending collection.
2.3. Relationship Configuration
99
SQLAlchemy Documentation, Release 0.8.7
The load_on_pending flag does not improve behavior when the ORM is used normally
- object references should be constructed at the object level, not at the foreign key level, so
that they are present in an ordinary way before a flush proceeds. This flag is not not intended
for general use.
See also:
Session.enable_relationship_loading() - this method establishes “load on
pending” behavior for the whole object, and also allows loading on objects that remain
transient or detached.
• order_by – indicates the ordering that should be applied when loading these items.
order_by is expected to refer to one of the Column objects to which the target class
is mapped, or the attribute itself bound to the target class which refers to the column.
order_by may also be passed as a callable function which is evaluated at mapper initialization time, and may be passed as a Python-evaluable string when using Declarative.
• passive_deletes=False – Indicates loading behavior during delete operations.
A value of True indicates that unloaded child items should not be loaded during a delete
operation on the parent. Normally, when a parent item is deleted, all child items are loaded
so that they can either be marked as deleted, or have their foreign key to the parent set
to NULL. Marking this flag as True usually implies an ON DELETE <CASCADE|SET
NULL> rule is in place which will handle updating/deleting child rows on the database
side.
Additionally, setting the flag to the string value ‘all’ will disable the “nulling out” of the child
foreign keys, when there is no delete or delete-orphan cascade enabled. This is typically
used when a triggering or error raise scenario is in place on the database side. Note that the
foreign key attributes on in-session child objects will not be changed after a flush occurs so
this is a very special use-case setting.
See also:
Using Passive Deletes - Introductory documentation and examples.
• passive_updates=True – Indicates loading and INSERT/UPDATE/DELETE behavior when the source of a foreign key value changes (i.e. an “on update” cascade), which are
typically the primary key columns of the source row.
When True, it is assumed that ON UPDATE CASCADE is configured on the foreign key in
the database, and that the database will handle propagation of an UPDATE from a source
column to dependent rows. Note that with databases which enforce referential integrity (i.e.
PostgreSQL, MySQL with InnoDB tables), ON UPDATE CASCADE is required for this
operation. The relationship() will update the value of the attribute on related items which
are locally present in the session during a flush.
When False, it is assumed that the database does not enforce referential integrity and will
not be issuing its own CASCADE operation for an update. The relationship() will issue the
appropriate UPDATE statements to the database in response to the change of a referenced
key, and items locally present in the session during a flush will also be refreshed.
This flag should probably be set to False if primary key changes are expected and the
database in use doesn’t support CASCADE (i.e. SQLite, MySQL MyISAM tables).
See also:
Mutable Primary Keys / Update Cascades - Introductory documentation and examples.
mapper.passive_updates - a similar flag which takes effect for joined-table inheritance mappings.
100
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
• post_update – this indicates that the relationship should be handled by a second UPDATE statement after an INSERT or before a DELETE. Currently, it also will issue an
UPDATE after the instance was UPDATEd as well, although this technically should be
improved. This flag is used to handle saving bi-directional dependencies between two individual rows (i.e. each row references the other), where it would otherwise be impossible
to INSERT or DELETE both rows fully since one row exists before the other. Use this
flag when a particular mapping arrangement will incur two rows that are dependent on each
other, such as a table that has a one-to-many relationship to a set of child rows, and also has
a column that references a single child row within that list (i.e. both tables contain a foreign
key to each other). If a flush operation returns an error that a “cyclical dependency” was
detected, this is a cue that you might want to use post_update to “break” the cycle.
See also:
Rows that point to themselves / Mutually Dependent Rows - Introductory documentation and
examples.
• primaryjoin – a SQL expression that will be used as the primary join of this child
object against the parent object, or in a many-to-many relationship the join of the primary
object to the association table. By default, this value is computed based on the foreign key
relationships of the parent and child tables (or association table).
primaryjoin may also be passed as a callable function which is evaluated at mapper
initialization time, and may be passed as a Python-evaluable string when using Declarative.
See also:
Specifying Alternate Join Conditions
• remote_side – used for self-referential relationships, indicates the column or list of
columns that form the “remote side” of the relationship.
relationship.remote_side may also be passed as a callable function which is evaluated at mapper initialization time, and may be passed as a Python-evaluable string when
using Declarative.
Changed in version 0.8: The remote() annotation can also be applied directly to the
primaryjoin expression, which is an alternate, more specific system of describing which
columns in a particular primaryjoin should be considered “remote”.
See also:
Adjacency List Relationships - in-depth explaination of how remote_side is used to
configure self-referential relationships.
remote() - an annotation function that accomplishes the same purpose as
remote_side, typically when a custom primaryjoin condition is used.
• query_class – a Query subclass that will be used as the base of the “appender query” returned by a “dynamic” relationship, that is, a relationship that specifies
lazy="dynamic" or was otherwise constructed using the orm.dynamic_loader()
function.
See also:
Dynamic Relationship Loaders - Introduction to “dynamic” relationship loaders.
• secondaryjoin – a SQL expression that will be used as the join of an association table
to the child object. By default, this value is computed based on the foreign key relationships
of the association and child tables.
2.3. Relationship Configuration
101
SQLAlchemy Documentation, Release 0.8.7
secondaryjoin may also be passed as a callable function which is evaluated at mapper
initialization time, and may be passed as a Python-evaluable string when using Declarative.
See also:
Specifying Alternate Join Conditions
• single_parent – when True, installs a validator which will prevent objects from being
associated with more than one parent at a time. This is used for many-to-one or many-tomany relationships that should be treated either as one-to-one or one-to-many. Its usage
is optional, except for relationship() constructs which are many-to-one or many-tomany and also specify the delete-orphan cascade option. The relationship()
construct itself will raise an error instructing when this option is required.
See also:
Cascades - includes detail on when the single_parent flag may be appropriate.
• uselist – a boolean that indicates if this property should be loaded as a list or a scalar.
In most cases, this value is determined automatically by relationship() at mapper
configuration time, based on the type and direction of the relationship - one to many forms a
list, many to one forms a scalar, many to many is a list. If a scalar is desired where normally
a list would be present, such as a bi-directional one-to-one relationship, set uselist to
False.
The uselist flag is also available on an existing relationship() construct as a readonly attribute, which can be used to determine if this relationship() deals with collections or scalar attributes:
>>> User.addresses.property.uselist
True
See also:
One To One - Introduction to the “one to one” relationship pattern, which is typically when
the uselist flag is needed.
• viewonly=False – when set to True, the relationship is used only for loading objects,
and not for any persistence operation. A relationship() which specifies viewonly
can work with a wider range of SQL operations within the primaryjoin condition, including operations that feature the use of a variety of comparison operators as well as SQL
functions such as cast(). The viewonly flag is also of general use when defining any
kind of relationship() that doesn’t represent the full set of related objects, to prevent
modifications of the collection from resulting in persistence operations.
See also:
relationship_custom_operator - Introduces the most common use case for viewonly, that
of a non-equality comparison in a primaryjoin condition.
sqlalchemy.orm.backref(name, **kwargs)
Create a back reference with explicit keyword arguments, which are the same arguments one can send to
relationship().
Used with the backref keyword argument to relationship() in place of a string argument, e.g.:
'items':relationship(SomeItem, backref=backref('parent', lazy='subquery'))
sqlalchemy.orm.relation(*arg, **kw)
A synonym for relationship().
102
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
sqlalchemy.orm.dynamic_loader(argument, **kw)
Construct a dynamically-loading mapper property.
This is essentially the same as using the lazy=’dynamic’ argument with relationship():
dynamic_loader(SomeClass)
# is the same as
relationship(SomeClass, lazy="dynamic")
See the section Dynamic Relationship Loaders for more details on dynamic loading.
sqlalchemy.orm.foreign(expr)
Annotate a portion of a primaryjoin expression with a ‘foreign’ annotation.
See the section Creating Custom Foreign Conditions for a description of use.
New in version 0.8.
See also:
Creating Custom Foreign Conditions
remote()
sqlalchemy.orm.remote(expr)
Annotate a portion of a primaryjoin expression with a ‘remote’ annotation.
See the section Creating Custom Foreign Conditions for a description of use.
New in version 0.8.
See also:
Creating Custom Foreign Conditions
foreign()
2.4 Collection Configuration and Techniques
The relationship() function defines a linkage between two classes. When the linkage defines a one-to-many
or many-to-many relationship, it’s represented as a Python collection when objects are loaded and manipulated. This
section presents additional information about collection configuration and techniques.
2.4.1 Working with Large Collections
The default behavior of relationship() is to fully load the collection of items in, as according to the loading
strategy of the relationship. Additionally, the Session by default only knows how to delete objects which are
actually present within the session. When a parent instance is marked for deletion and flushed, the Session loads its
full list of child items in so that they may either be deleted as well, or have their foreign key value set to null; this is
to avoid constraint violations. For large collections of child items, there are several strategies to bypass full loading of
child items both at load time as well as deletion time.
2.4. Collection Configuration and Techniques
103
SQLAlchemy Documentation, Release 0.8.7
Dynamic Relationship Loaders
A key feature to enable management of a large collection is the so-called “dynamic” relationship. This is an optional
form of relationship() which returns a Query object in place of a collection when accessed. filter()
criterion may be applied as well as limits and offsets, either explicitly or via array slices:
class User(Base):
__tablename__ = 'user'
posts = relationship(Post, lazy="dynamic")
jack = session.query(User).get(id)
# filter Jack's blog posts
posts = jack.posts.filter(Post.headline=='this is a post')
# apply array slices
posts = jack.posts[5:20]
The dynamic relationship supports limited write operations, via the append() and remove() methods:
oldpost = jack.posts.filter(Post.headline=='old post').one()
jack.posts.remove(oldpost)
jack.posts.append(Post('new post'))
Since the read side of the dynamic relationship always queries the database, changes to the underlying collection will
not be visible until the data has been flushed. However, as long as “autoflush” is enabled on the Session in use, this
will occur automatically each time the collection is about to emit a query.
To place a dynamic relationship on a backref, use the backref() function in conjunction with lazy=’dynamic’:
class Post(Base):
__table__ = posts_table
user = relationship(User,
backref=backref('posts', lazy='dynamic')
)
Note that eager/lazy loading options cannot be used in conjunction dynamic relationships at this time.
Note:
The dynamic_loader() function is essentially the same as relationship() with the
lazy=’dynamic’ argument specified.
Warning: The “dynamic” loader applies to collections only. It is not valid to use “dynamic” loaders with manyto-one, one-to-one, or uselist=False relationships. Newer versions of SQLAlchemy emit warnings or exceptions in
these cases.
Setting Noload
A “noload” relationship never loads from the database, even when accessed. It is configured using lazy=’noload’:
class MyClass(Base):
__tablename__ = 'some_table'
children = relationship(MyOtherClass, lazy='noload')
104
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
Above, the children collection is fully writeable, and changes to it will be persisted to the database as well as
locally available for reading at the time they are added. However when instances of MyClass are freshly loaded from
the database, the children collection stays empty.
Using Passive Deletes
Use passive_deletes to disable child object loading on a DELETE operation, in conjunction with “ON DELETE
(CASCADE|SET NULL)” on your database to automatically cascade deletes to child objects:
class MyClass(Base):
__tablename__ = 'mytable'
id = Column(Integer, primary_key=True)
children = relationship("MyOtherClass",
cascade="all, delete-orphan",
passive_deletes=True)
class MyOtherClass(Base):
__tablename__ = 'myothertable'
id = Column(Integer, primary_key=True)
parent_id = Column(Integer,
ForeignKey('mytable.id', ondelete='CASCADE')
)
Note: To use “ON DELETE CASCADE”, the underlying database engine must support foreign keys.
• When using MySQL, an appropriate storage engine must be selected. See CREATE TABLE arguments including
Storage Engines for details.
• When using SQLite, foreign key support must be enabled explicitly. See Foreign Key Support for details.
When passive_deletes is applied, the children relationship will not be loaded into memory when an instance
of MyClass is marked for deletion. The cascade="all, delete-orphan" will take effect for instances of
MyOtherClass which are currently present in the session; however for instances of MyOtherClass which are
not loaded, SQLAlchemy assumes that “ON DELETE CASCADE” rules will ensure that those rows are deleted by
the database.
2.4.2 Customizing Collection Access
Mapping a one-to-many or many-to-many relationship results in a collection of values accessible through an attribute
on the parent instance. By default, this collection is a list:
class Parent(Base):
__tablename__ = 'parent'
parent_id = Column(Integer, primary_key=True)
children = relationship(Child)
parent = Parent()
parent.children.append(Child())
print parent.children[0]
Collections are not limited to lists. Sets, mutable sequences and almost any other Python object that can act as a container can be used in place of the default list, by specifying the collection_class option on relationship():
2.4. Collection Configuration and Techniques
105
SQLAlchemy Documentation, Release 0.8.7
class Parent(Base):
__tablename__ = 'parent'
parent_id = Column(Integer, primary_key=True)
# use a set
children = relationship(Child, collection_class=set)
parent = Parent()
child = Child()
parent.children.add(child)
assert child in parent.children
Dictionary Collections
A little extra detail is needed when using a dictionary as a collection. This because objects are always loaded
from the database as lists, and a key-generation strategy must be available to populate the dictionary correctly. The
attribute_mapped_collection() function is by far the most common way to achieve a simple dictionary
collection. It produces a dictionary class that will apply a particular attribute of the mapped class as a key. Below we
map an Item class containing a dictionary of Note items keyed to the Note.keyword attribute:
from
from
from
from
sqlalchemy import Column, Integer, String, ForeignKey
sqlalchemy.orm import relationship
sqlalchemy.orm.collections import attribute_mapped_collection
sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Item(Base):
__tablename__ = 'item'
id = Column(Integer, primary_key=True)
notes = relationship("Note",
collection_class=attribute_mapped_collection('keyword'),
cascade="all, delete-orphan")
class Note(Base):
__tablename__ = 'note'
id = Column(Integer, primary_key=True)
item_id = Column(Integer, ForeignKey('item.id'), nullable=False)
keyword = Column(String)
text = Column(String)
def __init__(self, keyword, text):
self.keyword = keyword
self.text = text
Item.notes is then a dictionary:
>>> item = Item()
>>> item.notes['a'] = Note('a', 'atext')
>>> item.notes.items()
{'a': <__main__.Note object at 0x2eaaf0>}
attribute_mapped_collection() will ensure that the .keyword attribute of each Note complies with the
key in the dictionary. Such as, when assigning to Item.notes, the dictionary key we supply must match that of the
actual Note object:
106
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
item = Item()
item.notes = {
'a': Note('a', 'atext'),
'b': Note('b', 'btext')
}
The attribute which attribute_mapped_collection() uses as a key does not need to be mapped at all! Using
a regular Python @property allows virtually any detail or combination of details about the object to be used as the
key, as below when we establish it as a tuple of Note.keyword and the first ten letters of the Note.text field:
class Item(Base):
__tablename__ = 'item'
id = Column(Integer, primary_key=True)
notes = relationship("Note",
collection_class=attribute_mapped_collection('note_key'),
backref="item",
cascade="all, delete-orphan")
class Note(Base):
__tablename__ = 'note'
id = Column(Integer, primary_key=True)
item_id = Column(Integer, ForeignKey('item.id'), nullable=False)
keyword = Column(String)
text = Column(String)
@property
def note_key(self):
return (self.keyword, self.text[0:10])
def __init__(self, keyword, text):
self.keyword = keyword
self.text = text
Above we added a Note.item backref. Assigning to this reverse relationship, the Note is added to the
Item.notes dictionary and the key is generated for us automatically:
>>> item = Item()
>>> n1 = Note("a", "atext")
>>> n1.item = item
>>> item.notes
{('a', 'atext'): <__main__.Note object at 0x2eaaf0>}
Other built-in dictionary types include column_mapped_collection(),
attribute_mapped_collection() except given the Column object directly:
which
is
almost
like
from sqlalchemy.orm.collections import column_mapped_collection
class Item(Base):
__tablename__ = 'item'
id = Column(Integer, primary_key=True)
notes = relationship("Note",
collection_class=column_mapped_collection(Note.__table__.c.keyword),
cascade="all, delete-orphan")
as well as mapped_collection() which is passed any callable function. Note that it’s usually easier to use
attribute_mapped_collection() along with a @property as mentioned earlier:
from sqlalchemy.orm.collections import mapped_collection
2.4. Collection Configuration and Techniques
107
SQLAlchemy Documentation, Release 0.8.7
class Item(Base):
__tablename__ = 'item'
id = Column(Integer, primary_key=True)
notes = relationship("Note",
collection_class=mapped_collection(lambda note: note.text[0:10]),
cascade="all, delete-orphan")
Dictionary mappings are often combined with the “Association Proxy” extension to produce streamlined dictionary
views. See Proxying to Dictionary Based Collections and Composite Association Proxies for examples.
sqlalchemy.orm.collections.attribute_mapped_collection(attr_name)
A dictionary-based collection type with attribute-based keying.
Returns a MappedCollection factory with a keying based on the ‘attr_name’ attribute of entities in the
collection, where attr_name is the string name of the attribute.
The key value must be immutable for the lifetime of the object. You can not, for example, map on foreign key
values if those key values will change during the session, i.e. from None to a database-assigned integer after a
session flush.
sqlalchemy.orm.collections.column_mapped_collection(mapping_spec)
A dictionary-based collection type with column-based keying.
Returns a MappedCollection factory with a keying function generated from mapping_spec, which may be
a Column or a sequence of Columns.
The key value must be immutable for the lifetime of the object. You can not, for example, map on foreign key
values if those key values will change during the session, i.e. from None to a database-assigned integer after a
session flush.
sqlalchemy.orm.collections.mapped_collection(keyfunc)
A dictionary-based collection type with arbitrary keying.
Returns a MappedCollection factory with a keying function generated from keyfunc, a callable that takes
an entity and returns a key value.
The key value must be immutable for the lifetime of the object. You can not, for example, map on foreign key
values if those key values will change during the session, i.e. from None to a database-assigned integer after a
session flush.
2.4.3 Custom Collection Implementations
You can use your own types for collections as well. In simple cases, inherting from list or set, adding custom
behavior, is all that’s needed. In other cases, special decorators are needed to tell SQLAlchemy more detail about how
the collection operates.
108
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
Do I need a custom collection implementation?
In most cases not at all! The most common use cases for a “custom” collection is one that validates or marshals
incoming values into a new form, such as a string that becomes a class instance, or one which goes a step beyond
and represents the data internally in some fashion, presenting a “view” of that data on the outside of a different
form.
For the first use case, the orm.validates() decorator is by far the simplest way to intercept incoming values
in all cases for the purposes of validation and simple marshaling. See Simple Validators for an example of this.
For the second use case, the Association Proxy extension is a well-tested, widely used system that provides a
read/write “view” of a collection in terms of some attribute present on the target object. As the target attribute
can be a @property that returns virtually anything, a wide array of “alternative” views of a collection can be
constructed with just a few functions. This approach leaves the underlying mapped collection unaffected and
avoids the need to carefully tailor collection behavior on a method-by-method basis.
Customized collections are useful when the collection needs to have special behaviors upon access or mutation
operations that can’t otherwise be modeled externally to the collection. They can of course be combined with
the above two approaches.
Collections in SQLAlchemy are transparently instrumented. Instrumentation means that normal operations on the collection are tracked and result in changes being written to the database at flush time. Additionally, collection operations
can fire events which indicate some secondary operation must take place. Examples of a secondary operation include
saving the child item in the parent’s Session (i.e. the save-update cascade), as well as synchronizing the state
of a bi-directional relationship (i.e. a backref()).
The collections package understands the basic interface of lists, sets and dicts and will automatically apply instrumentation to those built-in types and their subclasses. Object-derived types that implement a basic collection interface are
detected and instrumented via duck-typing:
class ListLike(object):
def __init__(self):
self.data = []
def append(self, item):
self.data.append(item)
def remove(self, item):
self.data.remove(item)
def extend(self, items):
self.data.extend(items)
def __iter__(self):
return iter(self.data)
def foo(self):
return 'foo'
append, remove, and extend are known list-like methods, and will be instrumented automatically. __iter__ is
not a mutator method and won’t be instrumented, and foo won’t be either.
Duck-typing (i.e. guesswork) isn’t rock-solid, of course, so you can be explicit about the interface you are implementing by providing an __emulates__ class attribute:
class SetLike(object):
__emulates__ = set
def __init__(self):
self.data = set()
def append(self, item):
self.data.add(item)
def remove(self, item):
self.data.remove(item)
def __iter__(self):
return iter(self.data)
2.4. Collection Configuration and Techniques
109
SQLAlchemy Documentation, Release 0.8.7
This class looks list-like because of append, but __emulates__ forces it to set-like. remove is known to be part
of the set interface and will be instrumented.
But this class won’t work quite yet: a little glue is needed to adapt it for use by SQLAlchemy. The ORM needs to
know which methods to use to append, remove and iterate over members of the collection. When using a type like
list or set, the appropriate methods are well-known and used automatically when present. This set-like class does
not provide the expected add method, so we must supply an explicit mapping for the ORM via a decorator.
Annotating Custom Collections via Decorators
Decorators can be used to tag the individual methods the ORM needs to manage collections. Use them when your
class doesn’t quite meet the regular interface for its container type, or when you otherwise would like to use a different
method to get the job done.
from sqlalchemy.orm.collections import collection
class SetLike(object):
__emulates__ = set
def __init__(self):
self.data = set()
@collection.appender
def append(self, item):
self.data.add(item)
def remove(self, item):
self.data.remove(item)
def __iter__(self):
return iter(self.data)
And that’s all that’s needed to complete the example. SQLAlchemy will add instances via the append method.
remove and __iter__ are the default methods for sets and will be used for removing and iteration. Default
methods can be changed as well:
from sqlalchemy.orm.collections import collection
class MyList(list):
@collection.remover
def zark(self, item):
# do something special...
@collection.iterator
def hey_use_this_instead_for_iteration(self):
# ...
There is no requirement to be list-, or set-like at all. Collection classes can be any shape, so long as they have the
append, remove and iterate interface marked for SQLAlchemy’s use. Append and remove methods will be called with
a mapped entity as the single argument, and iterator methods are called with no arguments and must return an iterator.
class sqlalchemy.orm.collections.collection
Decorators for entity collection classes.
The decorators fall into two groups: annotations and interception recipes.
110
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
The annotating decorators (appender, remover, iterator, linker, converter, internally_instrumented) indicate the
method’s purpose and take no arguments. They are not written with parens:
@collection.appender
def append(self, append): ...
The recipe decorators all require parens, even those that take no arguments:
@collection.adds('entity')
def insert(self, position, entity): ...
@collection.removes_return()
def popitem(self): ...
static adds(arg)
Mark the method as adding an entity to the collection.
Adds “add to collection” handling to the method. The decorator argument indicates which method argument holds the SQLAlchemy-relevant value. Arguments can be specified positionally (i.e. integer) or by
name:
@collection.adds(1)
def push(self, item): ...
@collection.adds('entity')
def do_stuff(self, thing, entity=None): ...
static appender(fn)
Tag the method as the collection appender.
The appender method is called with one positional argument: the value to append. The method will be
automatically decorated with ‘adds(1)’ if not already decorated:
@collection.appender
def add(self, append): ...
# or, equivalently
@collection.appender
@collection.adds(1)
def add(self, append): ...
# for mapping type, an 'append' may kick out a previous value
# that occupies that slot. consider d['a'] = 'foo'- any previous
# value in d['a'] is discarded.
@collection.appender
@collection.replaces(1)
def add(self, entity):
key = some_key_func(entity)
previous = None
if key in self:
previous = self[key]
self[key] = entity
return previous
If the value to append is not allowed in the collection, you may raise an exception. Something to remember
is that the appender will be called for each object mapped by a database query. If the database contains
rows that violate your collection semantics, you will need to get creative to fix the problem, as access via
the collection will not work.
If the appender method is internally instrumented, you must also receive the keyword argument
2.4. Collection Configuration and Techniques
111
SQLAlchemy Documentation, Release 0.8.7
‘_sa_initiator’ and ensure its promulgation to collection events.
static converter(fn)
Tag the method as the collection converter.
This optional method will be called when a collection is being replaced entirely, as in:
myobj.acollection = [newvalue1, newvalue2]
The converter method will receive the object being assigned and should return an iterable of values suitable
for use by the appender method. A converter must not assign values or mutate the collection, its sole
job is to adapt the value the user provides into an iterable of values for the ORM’s use.
The default converter implementation will use duck-typing to do the conversion. A dict-like collection will
be convert into an iterable of dictionary values, and other types will simply be iterated:
@collection.converter
def convert(self, other): ...
If the duck-typing of the object does not match the type of this collection, a TypeError is raised.
Supply an implementation of this method if you want to expand the range of possible types that can be
assigned in bulk or perform validation on the values about to be assigned.
static internally_instrumented(fn)
Tag the method as instrumented.
This tag will prevent any decoration from being applied to the method. Use this if you are orchestrating
your own calls to collection_adapter() in one of the basic SQLAlchemy interface methods, or to
prevent an automatic ABC method decoration from wrapping your implementation:
# normally an 'extend' method on a list-like class would be
# automatically intercepted and re-implemented in terms of
# SQLAlchemy events and append(). your implementation will
# never be called, unless:
@collection.internally_instrumented
def extend(self, items): ...
static iterator(fn)
Tag the method as the collection remover.
The iterator method is called with no arguments. It is expected to return an iterator over all collection
members:
@collection.iterator
def __iter__(self): ...
static link(fn)
deprecated; synonym for collection.linker().
static linker(fn)
Tag the method as a “linked to attribute” event handler.
This optional event handler will be called when the collection class is linked to or unlinked from the
InstrumentedAttribute. It is invoked immediately after the ‘_sa_adapter’ property is set on the instance. A
single argument is passed: the collection adapter that has been linked, or None if unlinking.
static remover(fn)
Tag the method as the collection remover.
The remover method is called with one positional argument: the value to remove. The method will be
automatically decorated with removes_return() if not already decorated:
112
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
@collection.remover
def zap(self, entity): ...
# or, equivalently
@collection.remover
@collection.removes_return()
def zap(self, ): ...
If the value to remove is not present in the collection, you may raise an exception or return None to ignore
the error.
If the remove method is internally instrumented, you must also receive the keyword argument
‘_sa_initiator’ and ensure its promulgation to collection events.
static removes(arg)
Mark the method as removing an entity in the collection.
Adds “remove from collection” handling to the method. The decorator argument indicates which method
argument holds the SQLAlchemy-relevant value to be removed. Arguments can be specified positionally
(i.e. integer) or by name:
@collection.removes(1)
def zap(self, item): ...
For methods where the value to remove is not known at call-time, use collection.removes_return.
static removes_return()
Mark the method as removing an entity in the collection.
Adds “remove from collection” handling to the method. The return value of the method, if any, is considered the value to remove. The method arguments are not inspected:
@collection.removes_return()
def pop(self): ...
For methods where the value to remove is known at call-time, use collection.remove.
static replaces(arg)
Mark the method as replacing an entity in the collection.
Adds “add to collection” and “remove from collection” handling to the method. The decorator argument
indicates which method argument holds the SQLAlchemy-relevant value to be added, and return value, if
any will be considered the value to remove.
Arguments can be specified positionally (i.e. integer) or by name:
@collection.replaces(2)
def __setitem__(self, index, item): ...
Custom Dictionary-Based Collections
The MappedCollection class can be used as a base class for your custom types or as a mix-in to quickly add dict
collection support to other classes. It uses a keying function to delegate to __setitem__ and __delitem__:
from sqlalchemy.util import OrderedDict
from sqlalchemy.orm.collections import MappedCollection
class NodeMap(OrderedDict, MappedCollection):
"""Holds 'Node' objects, keyed by the 'name' attribute with insert order maintained."""
2.4. Collection Configuration and Techniques
113
SQLAlchemy Documentation, Release 0.8.7
def __init__(self, *args, **kw):
MappedCollection.__init__(self, keyfunc=lambda node: node.name)
OrderedDict.__init__(self, *args, **kw)
When subclassing MappedCollection, user-defined versions of __setitem__() or __delitem__() should
be decorated with collection.internally_instrumented(), if they call down to those same methods on
MappedCollection. This because the methods on MappedCollection are already instrumented - calling
them from within an already instrumented call can cause events to be fired off repeatedly, or inappropriately, leading
to internal state corruption in rare cases:
from sqlalchemy.orm.collections import MappedCollection,\
collection
class MyMappedCollection(MappedCollection):
"""Use @internally_instrumented when your methods
call down to already-instrumented methods.
"""
@collection.internally_instrumented
def __setitem__(self, key, value, _sa_initiator=None):
# do something with key, value
super(MyMappedCollection, self).__setitem__(key, value, _sa_initiator)
@collection.internally_instrumented
def __delitem__(self, key, _sa_initiator=None):
# do something with key
super(MyMappedCollection, self).__delitem__(key, _sa_initiator)
The ORM understands the dict interface just like lists and sets, and will automatically instrument all dict-like methods if you choose to subclass dict or provide dict-like collection behavior in a duck-typed class. You must decorate appender and remover methods, however- there are no compatible methods in the basic dictionary interface for
SQLAlchemy to use by default. Iteration will go through itervalues() unless otherwise decorated.
Note: Due to a bug in MappedCollection prior to version 0.7.6, this workaround usually needs to be called before a
custom subclass of MappedCollection which uses collection.internally_instrumented() can be
used:
from sqlalchemy.orm.collections import _instrument_class, MappedCollection
_instrument_class(MappedCollection)
This will ensure that the MappedCollection has been properly initialized with custom __setitem__() and
__delitem__() methods before used in a custom subclass.
class sqlalchemy.orm.collections.MappedCollection(keyfunc)
Bases: __builtin__.dict
A basic dictionary-based collection class.
Extends dict with the minimal bag semantics that collection classes require. set and remove are implemented
in terms of a keying function: any callable that takes an object and returns an object for use as a dictionary key.
__init__(keyfunc)
Create a new collection with keying provided by keyfunc.
keyfunc may be any callable any callable that takes an object and returns an object for use as a dictionary
key.
114
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
The keyfunc will be called every time the ORM needs to add a member by value-only (such as when
loading instances from the database) or remove a member. The usual cautions about dictionary keying
apply- keyfunc(object) should return the same output for the life of the collection. Keying based on
mutable properties can result in unreachable instances “lost” in the collection.
clear() → None. Remove all items from D.
pop(k[, d ]) → v, remove specified key and return the corresponding value.
If key is not found, d is returned if given, otherwise KeyError is raised
popitem() → (k, v), remove and return some (key, value) pair as a
2-tuple; but raise KeyError if D is empty.
remove(value, _sa_initiator=None)
Remove an item by value, consulting the keyfunc for the key.
set(value, _sa_initiator=None)
Add an item by value, consulting the keyfunc for the key.
setdefault(k[, d ]) → D.get(k,d), also set D[k]=d if k not in D
update([E ], **F) → None. Update D from dict/iterable E and F.
If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method,
does: for (k, v) in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]
Instrumentation and Custom Types
Many custom types and existing library classes can be used as a entity collection type as-is without further ado.
However, it is important to note that the instrumentation process will modify the type, adding decorators around
methods automatically.
The decorations are lightweight and no-op outside of relationships, but they do add unneeded overhead when triggered
elsewhere. When using a library class as a collection, it can be good practice to use the “trivial subclass” trick to restrict
the decorations to just your usage in relationships. For example:
class MyAwesomeList(some.great.library.AwesomeList):
pass
# ... relationship(..., collection_class=MyAwesomeList)
The ORM uses this approach for built-ins, quietly substituting a trivial subclass when a list, set or dict is used
directly.
2.4.4 Collection Internals
Various internal methods.
sqlalchemy.orm.collections.bulk_replace(values, existing_adapter, new_adapter)
Load a new collection, firing events based on prior like membership.
Appends instances in values onto the new_adapter. Events will be fired for any instance not present in
the existing_adapter. Any instances in existing_adapter not present in values will have remove
events fired upon them.
Parameters
• values – An iterable of collection member instances
• existing_adapter – A CollectionAdapter of instances to be replaced
2.4. Collection Configuration and Techniques
115
SQLAlchemy Documentation, Release 0.8.7
• new_adapter – An empty CollectionAdapter to load with values
class sqlalchemy.orm.collections.collection
Decorators for entity collection classes.
The decorators fall into two groups: annotations and interception recipes.
The annotating decorators (appender, remover, iterator, linker, converter, internally_instrumented) indicate the
method’s purpose and take no arguments. They are not written with parens:
@collection.appender
def append(self, append): ...
The recipe decorators all require parens, even those that take no arguments:
@collection.adds('entity')
def insert(self, position, entity): ...
@collection.removes_return()
def popitem(self): ...
sqlalchemy.orm.collections.collection_adapter(collection)
Fetch the CollectionAdapter for a collection.
class sqlalchemy.orm.collections.CollectionAdapter(attr, owner_state, data)
Bridges between the ORM and arbitrary Python collections.
Proxies base-level collection operations (append, remove, iterate) to the underlying Python collection, and emits
add/remove events for entities entering or leaving the collection.
The ORM uses CollectionAdapter exclusively for interaction with entity collections.
The usage of getattr()/setattr() is currently to allow injection of custom methods, such as to unwrap Zope security
proxies.
class sqlalchemy.orm.collections.InstrumentedDict
Bases: __builtin__.dict
An instrumented version of the built-in dict.
class sqlalchemy.orm.collections.InstrumentedList
Bases: __builtin__.list
An instrumented version of the built-in list.
class sqlalchemy.orm.collections.InstrumentedSet
Bases: __builtin__.set
An instrumented version of the built-in set.
sqlalchemy.orm.collections.prepare_instrumentation(factory)
Prepare a callable for future use as a collection class factory.
Given a collection class factory (either a type or no-arg callable), return another factory that will produce compatible instances when called.
This function is responsible for converting collection_class=list into the run-time behavior of collection_class=InstrumentedList.
116
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
2.5 Mapping Class Inheritance Hierarchies
SQLAlchemy supports three forms of inheritance: single table inheritance, where several types of classes are represented by a single table, concrete table inheritance, where each type of class is represented by independent tables,
and joined table inheritance, where the class hierarchy is broken up among dependent tables, each class represented
by its own table that only includes those attributes local to that class.
The most common forms of inheritance are single and joined table, while concrete inheritance presents more configurational challenges.
When mappers are configured in an inheritance relationship, SQLAlchemy has the ability to load elements polymorphically, meaning that a single query can return objects of multiple types.
2.5.1 Joined Table Inheritance
In joined table inheritance, each class along a particular classes’ list of parents is represented by a unique table. The
total set of attributes for a particular instance is represented as a join along all tables in its inheritance path. Here, we
first define the Employee class. This table will contain a primary key column (or columns), and a column for each
attribute that’s represented by Employee. In this case it’s just name:
class Employee(Base):
__tablename__ = 'employee'
id = Column(Integer, primary_key=True)
name = Column(String(50))
type = Column(String(50))
__mapper_args__ = {
'polymorphic_identity':'employee',
'polymorphic_on':type
}
The mapped table also has a column called type. The purpose of this column is to act as the discriminator, and
stores a value which indicates the type of object represented within the row. The column may be of any datatype,
though string and integer are the most common.
The discriminator column is only needed if polymorphic loading is desired, as is usually the case. It is not strictly
necessary that it be present directly on the base mapped table, and can instead be defined on a derived select statement
that’s used when the class is queried; however, this is a much more sophisticated configuration scenario.
The mapping receives additional arguments via the __mapper_args__ dictionary. Here the type column is explicitly stated as the discriminator column, and the polymorphic identity of employee is also given; this is the value
that will be stored in the polymorphic discriminator column for instances of this class.
We next define Engineer and Manager subclasses of Employee. Each contains columns that represent the
attributes unique to the subclass they represent. Each table also must contain a primary key column (or columns), and
in most cases a foreign key reference to the parent table:
class Engineer(Employee):
__tablename__ = 'engineer'
id = Column(Integer, ForeignKey('employee.id'), primary_key=True)
engineer_name = Column(String(30))
__mapper_args__ = {
'polymorphic_identity':'engineer',
}
class Manager(Employee):
2.5. Mapping Class Inheritance Hierarchies
117
SQLAlchemy Documentation, Release 0.8.7
__tablename__ = 'manager'
id = Column(Integer, ForeignKey('employee.id'), primary_key=True)
manager_name = Column(String(30))
__mapper_args__ = {
'polymorphic_identity':'manager',
}
It is standard practice that the same column is used for both the role of primary key as well as foreign key to the parent
table, and that the column is also named the same as that of the parent table. However, both of these practices are
optional. Separate columns may be used for primary key and parent-relationship, the column may be named differently
than that of the parent, and even a custom join condition can be specified between parent and child tables instead of
using a foreign key.
Joined inheritance primary keys
One natural effect of the joined table inheritance configuration is that the identity of any mapped object can be
determined entirely from the base table. This has obvious advantages, so SQLAlchemy always considers the
primary key columns of a joined inheritance class to be those of the base table only. In other words, the id
columns of both the engineer and manager tables are not used to locate Engineer or Manager objects only the value in employee.id is considered. engineer.id and manager.id are still of course critical
to the proper operation of the pattern overall as they are used to locate the joined row, once the parent row has
been determined within a statement.
With the joined inheritance mapping complete, querying against Employee will return a combination of Employee,
Engineer and Manager objects. Newly saved Engineer, Manager, and Employee objects will automatically
populate the employee.type column with engineer, manager, or employee, as appropriate.
Basic Control of Which Tables are Queried
The orm.with_polymorphic() function and the with_polymorphic() method of Query affects the specific tables which the Query selects from. Normally, a query such as this:
session.query(Employee).all()
...selects only from the employee table. When loading fresh from the database, our joined-table setup will query
from the parent table only, using SQL such as this:
SELECT employee.id AS employee_id,
employee.name AS employee_name, employee.type AS employee_type
FROM employee
[]
As attributes are requested from those Employee objects which are represented in either the engineer or
manager child tables, a second load is issued for the columns in that related row, if the data was not already loaded.
So above, after accessing the objects you’d see further SQL issued along the lines of:
SELECT manager.id AS manager_id,
manager.manager_data AS manager_manager_data
FROM manager
WHERE ? = manager.id
[5]
SELECT engineer.id AS engineer_id,
engineer.engineer_info AS engineer_engineer_info
FROM engineer
118
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
WHERE ? = engineer.id
[2]
This behavior works well when issuing searches for small numbers of items, such as when using Query.get(),
since the full range of joined tables are not pulled in to the SQL statement unnecessarily. But when querying a larger
span of rows which are known to be of many types, you may want to actively join to some or all of the joined tables.
The with_polymorphic feature provides this.
Telling our query to polymorphically load Engineer and Manager objects, we can use the
orm.with_polymorphic() function to create a new aliased class which represents a select of the base
table combined with outer joins to each of the inheriting tables:
from sqlalchemy.orm import with_polymorphic
eng_plus_manager = with_polymorphic(Employee, [Engineer, Manager])
query = session.query(eng_plus_manager)
The above produces a query which joins the employee table to both the engineer and manager tables like the
following:
query.all()
SELECT employee.id AS employee_id,
engineer.id AS engineer_id,
manager.id AS manager_id,
employee.name AS employee_name,
employee.type AS employee_type,
engineer.engineer_info AS engineer_engineer_info,
manager.manager_data AS manager_manager_data
FROM employee
LEFT OUTER JOIN engineer
ON employee.id = engineer.id
LEFT OUTER JOIN manager
ON employee.id = manager.id
[]
The entity returned by orm.with_polymorphic() is an AliasedClass object, which can be used in a
Query like any other alias, including named attributes for those attributes on the Employee class. In our example, eng_plus_manager becomes the entity that we use to refer to the three-way outer join above. It also includes
namespaces for each class named in the list of classes, so that attributes specific to those subclasses can be called upon
as well. The following example illustrates calling upon attributes specific to Engineer as well as Manager in terms
of eng_plus_manager:
eng_plus_manager = with_polymorphic(Employee, [Engineer, Manager])
query = session.query(eng_plus_manager).filter(
or_(
eng_plus_manager.Engineer.engineer_info=='x',
eng_plus_manager.Manager.manager_data=='y'
)
)
orm.with_polymorphic() accepts a single class or mapper, a list of classes/mappers, or the string ’*’ to
indicate all subclasses:
# join to the engineer table
entity = with_polymorphic(Employee, Engineer)
# join to the engineer and manager tables
2.5. Mapping Class Inheritance Hierarchies
119
SQLAlchemy Documentation, Release 0.8.7
entity = with_polymorphic(Employee, [Engineer, Manager])
# join to all subclass tables
entity = query.with_polymorphic(Employee, '*')
# use with Query
session.query(entity).all()
It also accepts a third argument selectable which replaces the automatic join creation and instead selects directly
from the selectable given. This feature is normally used with “concrete” inheritance, described later, but can be used
with any kind of inheritance setup in the case that specialized SQL should be used to load polymorphically:
# custom selectable
employee = Employee.__table__
manager = Manager.__table__
engineer = Engineer.__table__
entity = with_polymorphic(
Employee,
[Engineer, Manager],
employee.outerjoin(manager).outerjoin(engineer)
)
# use with Query
session.query(entity).all()
Note that if you only need to load a single subtype, such as just the Engineer objects,
orm.with_polymorphic() is not needed since you would query against the Engineer class directly.
Query.with_polymorphic() has the same purpose as orm.with_polymorphic(), except is not as flexible
in its usage patterns in that it only applies to the first full mapping, which then impacts all occurrences of that class or
the target subclasses within the Query. For simple cases it might be considered to be more succinct:
session.query(Employee).with_polymorphic([Engineer, Manager]).\
filter(or_(Engineer.engineer_info=='w', Manager.manager_data=='q'))
New in version 0.8: orm.with_polymorphic(), an improved version of Query.with_polymorphic()
method.
The mapper also accepts with_polymorphic as a configurational argument so that the joined-style load will be
issued automatically. This argument may be the string ’*’, a list of classes, or a tuple consisting of either, followed
by a selectable:
class Employee(Base):
__tablename__ = 'employee'
id = Column(Integer, primary_key=True)
type = Column(String(20))
__mapper_args__ = {
'polymorphic_on':type,
'polymorphic_identity':'employee',
'with_polymorphic':'*'
}
class Engineer(Employee):
__tablename__ = 'engineer'
id = Column(Integer, ForeignKey('employee.id'), primary_key=True)
__mapper_args__ = {'polymorphic_identity':'engineer'}
class Manager(Employee):
120
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
__tablename__ = 'manager'
id = Column(Integer, ForeignKey('employee.id'), primary_key=True)
__mapper_args__ = {'polymorphic_identity':'manager'}
The above mapping will produce a query similar to that of with_polymorphic(’*’) for every query of
Employee objects.
Using orm.with_polymorphic() or Query.with_polymorphic() will override the mapper-level
with_polymorphic setting.
sqlalchemy.orm.with_polymorphic(base, classes, selectable=False, polymorphic_on=None,
aliased=False, innerjoin=False, _use_mapper_path=False)
Produce an AliasedClass construct which specifies columns for descendant mappers of the given base.
New in version 0.8: orm.with_polymorphic() is in addition to the existing Query method
Query.with_polymorphic(), which has the same purpose but is not as flexible in its usage.
Using this method will ensure that each descendant mapper’s tables are included in the FROM clause, and will
allow filter() criterion to be used against those tables. The resulting instances will also have those columns
already loaded so that no “post fetch” of those columns will be required.
See the examples at Basic Control of Which Tables are Queried.
Parameters
• base – Base class to be aliased.
• classes – a single class or mapper, or list of class/mappers, which inherit from the base
class. Alternatively, it may also be the string ’*’, in which case all descending mapped
classes will be added to the FROM clause.
• aliased – when True, the selectable will be wrapped in an alias, that is (SELECT
* FROM <fromclauses>) AS anon_1. This can be important when using the
with_polymorphic() to create the target of a JOIN on a backend that does not support parenthesized joins, such as SQLite and older versions of MySQL.
• selectable – a table or select() statement that will be used in place of the generated
FROM clause. This argument is required if any of the desired classes use concrete table
inheritance, since SQLAlchemy currently cannot generate UNIONs among tables automatically. If used, the selectable argument must represent the full set of tables and columns
mapped by every mapped class. Otherwise, the unaccounted mapped columns will result in
their table being appended directly to the FROM clause which will usually lead to incorrect
results.
• polymorphic_on – a column to be used as the “discriminator” column for the given
selectable. If not given, the polymorphic_on attribute of the base classes’ mapper will be
used, if any. This is useful for mappings that don’t have polymorphic loading behavior by
default.
• innerjoin – if True, an INNER JOIN will be used. This should only be specified if
querying for one specific subtype only
Advanced Control of Which Tables are Queried
The with_polymorphic functions work fine for simplistic scenarios. However, direct control of table rendering is
called for, such as the case when one wants to render to only the subclass table and not the parent table.
This use case can be achieved by using the mapped Table objects directly. For example, to query the name of
employees with particular criterion:
2.5. Mapping Class Inheritance Hierarchies
121
SQLAlchemy Documentation, Release 0.8.7
engineer = Engineer.__table__
manager = Manager.__table__
session.query(Employee.name).\
outerjoin((engineer, engineer.c.employee_id==Employee.employee_id)).\
outerjoin((manager, manager.c.employee_id==Employee.employee_id)).\
filter(or_(Engineer.engineer_info=='w', Manager.manager_data=='q'))
The base table, in this case the “employees” table, isn’t always necessary. A SQL query is always more efficient
with fewer joins. Here, if we wanted to just load information specific to manager or engineer, we can instruct
Query to use only those tables. The FROM clause is determined by what’s specified in the Session.query(),
Query.filter(), or Query.select_from() methods:
session.query(Manager.manager_data).select_from(manager)
session.query(engineer.c.id).\
filter(engineer.c.engineer_info==manager.c.manager_data)
Creating Joins to Specific Subtypes
The of_type() method is a helper which allows the construction of joins along relationship() paths while
narrowing the criterion to specific subclasses. Suppose the employees table represents a collection of employees
which are associated with a Company object. We’ll add a company_id column to the employees table and a new
table companies:
class Company(Base):
__tablename__ = 'company'
id = Column(Integer, primary_key=True)
name = Column(String(50))
employees = relationship("Employee",
backref='company',
cascade='all, delete-orphan')
class Employee(Base):
__tablename__ = 'employee'
id = Column(Integer, primary_key=True)
type = Column(String(20))
company_id = Column(Integer, ForeignKey('company.id'))
__mapper_args__ = {
'polymorphic_on':type,
'polymorphic_identity':'employee',
'with_polymorphic':'*'
}
class Engineer(Employee):
__tablename__ = 'engineer'
id = Column(Integer, ForeignKey('employee.id'), primary_key=True)
engineer_info = Column(String(50))
__mapper_args__ = {'polymorphic_identity':'engineer'}
class Manager(Employee):
__tablename__ = 'manager'
id = Column(Integer, ForeignKey('employee.id'), primary_key=True)
manager_data = Column(String(50))
__mapper_args__ = {'polymorphic_identity':'manager'}
When querying from Company onto the Employee relationship, the join() method as well as the any() and
122
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
has() operators will create a join from company to employee, without including engineer or manager in the
mix. If we wish to have criterion which is specifically against the Engineer class, we can tell those methods to join
or subquery against the joined table representing the subclass using the of_type() operator:
session.query(Company).\
join(Company.employees.of_type(Engineer)).\
filter(Engineer.engineer_info=='someinfo')
A longhand version of this would involve spelling out the full target selectable within a 2-tuple:
employee = Employee.__table__
engineer = Engineer.__table__
session.query(Company).\
join((employee.join(engineer), Company.employees)).\
filter(Engineer.engineer_info=='someinfo')
of_type() accepts a single class argument. More flexibility can be achieved either by joining to an explicit join as
above, or by using the orm.with_polymorphic() function to create a polymorphic selectable:
manager_and_engineer = with_polymorphic(
Employee, [Manager, Engineer],
aliased=True)
session.query(Company).\
join(manager_and_engineer, Company.employees).\
filter(
or_(manager_and_engineer.Engineer.engineer_info=='someinfo',
manager_and_engineer.Manager.manager_data=='somedata')
)
Above, we use the aliased=True argument with orm.with_polymorhpic() so that the right hand side of
the join between Company and manager_and_engineer is converted into an aliased subquery. Some backends,
such as SQLite and older versions of MySQL can’t handle a FROM clause of the following form:
FROM x JOIN (y JOIN z ON <onclause>) ON <onclause>
Using aliased=True instead renders it more like:
FROM x JOIN (SELECT * FROM y JOIN z ON <onclause>) AS anon_1 ON <onclause>
The above join can also be expressed more succinctly by combining of_type() with the polymorphic construct:
manager_and_engineer = with_polymorphic(
Employee, [Manager, Engineer],
aliased=True)
session.query(Company).\
join(Company.employees.of_type(manager_and_engineer)).\
filter(
or_(manager_and_engineer.Engineer.engineer_info=='someinfo',
manager_and_engineer.Manager.manager_data=='somedata')
)
The any() and has() operators also can be used with of_type() when the embedded criterion is in terms of a
subclass:
session.query(Company).\
filter(
Company.employees.of_type(Engineer).
2.5. Mapping Class Inheritance Hierarchies
123
SQLAlchemy Documentation, Release 0.8.7
any(Engineer.engineer_info=='someinfo')
).all()
Note that the any() and has() are both shorthand for a correlated EXISTS query. To build one by hand looks like:
session.query(Company).filter(
exists([1],
and_(Engineer.engineer_info=='someinfo',
employees.c.company_id==companies.c.company_id),
from_obj=employees.join(engineers)
)
).all()
The EXISTS subquery above selects from the join of employees to engineers, and also specifies criterion which
correlates the EXISTS subselect back to the parent companies table.
New in version 0.8: of_type() accepts orm.aliased() and orm.with_polymorphic() constructs in conjunction with Query.join(), any() and has().
Eager Loading of Specific or Polymorphic Subtypes
The joinedload(), subqueryload(), contains_eager() and other loading-related options also support
paths which make use of of_type(). Below we load Company rows while eagerly loading related Engineer
objects, querying the employee and engineer tables simultaneously:
session.query(Company).\
options(subqueryload_all(Company.employees.of_type(Engineer),
Engineer.machines))
As is the case with Query.join(), of_type() also can be used with eager loading and
orm.with_polymorphic() at the same time, so that all sub-attributes of all referenced subtypes can be
loaded:
manager_and_engineer = with_polymorphic(
Employee, [Manager, Engineer],
aliased=True)
session.query(Company).\
options(
joinedload(Company.employees.of_type(manager_and_engineer))
)
)
New in version 0.8: joinedload(), subqueryload(), contains_eager() and related loader
options support paths that are qualified with of_type(), supporting single target types as well as
orm.with_polymorphic() targets.
2.5.2 Single Table Inheritance
Single table inheritance is where the attributes of the base class as well as all subclasses are represented within a single
table. A column is present in the table for every attribute mapped to the base class and all subclasses; the columns
which correspond to a single subclass are nullable. This configuration looks much like joined-table inheritance except
there’s only one table. In this case, a type column is required, as there would be no other way to discriminate between
classes. The table is specified in the base mapper only; for the inheriting classes, leave their table parameter blank:
124
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
class Employee(Base):
__tablename__ = 'employee'
id = Column(Integer, primary_key=True)
name = Column(String(50))
manager_data = Column(String(50))
engineer_info = Column(String(50))
type = Column(String(20))
__mapper_args__ = {
'polymorphic_on':type,
'polymorphic_identity':'employee'
}
class Manager(Employee):
__mapper_args__ = {
'polymorphic_identity':'manager'
}
class Engineer(Employee):
__mapper_args__ = {
'polymorphic_identity':'engineer'
}
Note that the mappers for the derived classes Manager and Engineer omit the __tablename__, indicating they do
not have a mapped table of their own.
2.5.3 Concrete Table Inheritance
Note: this section is currently using classical mappings. The Declarative system fully supports concrete inheritance
however. See the links below for more information on using declarative with concrete table inheritance.
This form of inheritance maps each class to a distinct table, as below:
employees_table = Table('employees', metadata,
Column('employee_id', Integer, primary_key=True),
Column('name', String(50)),
)
managers_table = Table('managers', metadata,
Column('employee_id', Integer, primary_key=True),
Column('name', String(50)),
Column('manager_data', String(50)),
)
engineers_table = Table('engineers', metadata,
Column('employee_id', Integer, primary_key=True),
Column('name', String(50)),
Column('engineer_info', String(50)),
)
Notice in this case there is no type column. If polymorphic loading is not required, there’s no advantage to using
inherits here; you just define a separate mapper for each class.
mapper(Employee, employees_table)
mapper(Manager, managers_table)
mapper(Engineer, engineers_table)
2.5. Mapping Class Inheritance Hierarchies
125
SQLAlchemy Documentation, Release 0.8.7
To load polymorphically, the with_polymorphic argument is required, along with a selectable indicating how
rows should be loaded. In this case we must construct a UNION of all three tables. SQLAlchemy includes a helper
function to create these called polymorphic_union(), which will map all the different columns into a structure
of selects with the same numbers and names of columns, and also generate a virtual type column for each subselect:
pjoin = polymorphic_union({
'employee': employees_table,
'manager': managers_table,
'engineer': engineers_table
}, 'type', 'pjoin')
employee_mapper = mapper(Employee, employees_table,
with_polymorphic=('*', pjoin),
polymorphic_on=pjoin.c.type,
polymorphic_identity='employee')
manager_mapper = mapper(Manager, managers_table,
inherits=employee_mapper,
concrete=True,
polymorphic_identity='manager')
engineer_mapper = mapper(Engineer, engineers_table,
inherits=employee_mapper,
concrete=True,
polymorphic_identity='engineer')
Upon select, the polymorphic union produces a query like this:
session.query(Employee).all()
SELECT pjoin.type AS pjoin_type,
pjoin.manager_data AS pjoin_manager_data,
pjoin.employee_id AS pjoin_employee_id,
pjoin.name AS pjoin_name, pjoin.engineer_info AS pjoin_engineer_info
FROM (
SELECT employees.employee_id AS employee_id,
CAST(NULL AS VARCHAR(50)) AS manager_data, employees.name AS name,
CAST(NULL AS VARCHAR(50)) AS engineer_info, 'employee' AS type
FROM employees
UNION ALL
SELECT managers.employee_id AS employee_id,
managers.manager_data AS manager_data, managers.name AS name,
CAST(NULL AS VARCHAR(50)) AS engineer_info, 'manager' AS type
FROM managers
UNION ALL
SELECT engineers.employee_id AS employee_id,
CAST(NULL AS VARCHAR(50)) AS manager_data, engineers.name AS name,
engineers.engineer_info AS engineer_info, 'engineer' AS type
FROM engineers
) AS pjoin
[]
Concrete Inheritance with Declarative
New in version 0.7.3: The Declarative module includes helpers for concrete inheritance. See Using the Concrete
Helpers for more information.
126
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
2.5.4 Using Relationships with Inheritance
Both joined-table and single table inheritance scenarios produce mappings which are usable in relationship()
functions; that is, it’s possible to map a parent object to a child object which is polymorphic. Similarly, inheriting
mappers can have relationship() objects of their own at any level, which are inherited to each child class.
The only requirement for relationships is that there is a table relationship between parent and child. An example is
the following modification to the joined table inheritance example, which sets a bi-directional relationship between
Employee and Company:
employees_table = Table('employees', metadata,
Column('employee_id', Integer, primary_key=True),
Column('name', String(50)),
Column('company_id', Integer, ForeignKey('companies.company_id'))
)
companies = Table('companies', metadata,
Column('company_id', Integer, primary_key=True),
Column('name', String(50)))
class Company(object):
pass
mapper(Company, companies, properties={
'employees': relationship(Employee, backref='company')
})
Relationships with Concrete Inheritance
In a concrete inheritance scenario, mapping relationships is more challenging since the distinct classes do not share
a table. In this case, you can establish a relationship from parent to child if a join condition can be constructed from
parent to child, if each child table contains a foreign key to the parent:
companies = Table('companies', metadata,
Column('id', Integer, primary_key=True),
Column('name', String(50)))
employees_table = Table('employees', metadata,
Column('employee_id', Integer, primary_key=True),
Column('name', String(50)),
Column('company_id', Integer, ForeignKey('companies.id'))
)
managers_table = Table('managers', metadata,
Column('employee_id', Integer, primary_key=True),
Column('name', String(50)),
Column('manager_data', String(50)),
Column('company_id', Integer, ForeignKey('companies.id'))
)
engineers_table = Table('engineers', metadata,
Column('employee_id', Integer, primary_key=True),
Column('name', String(50)),
Column('engineer_info', String(50)),
Column('company_id', Integer, ForeignKey('companies.id'))
)
mapper(Employee, employees_table,
2.5. Mapping Class Inheritance Hierarchies
127
SQLAlchemy Documentation, Release 0.8.7
with_polymorphic=('*', pjoin),
polymorphic_on=pjoin.c.type,
polymorphic_identity='employee')
mapper(Manager, managers_table,
inherits=employee_mapper,
concrete=True,
polymorphic_identity='manager')
mapper(Engineer, engineers_table,
inherits=employee_mapper,
concrete=True,
polymorphic_identity='engineer')
mapper(Company, companies, properties={
'employees': relationship(Employee)
})
The big limitation with concrete table inheritance is that relationship() objects placed on each concrete mapper
do not propagate to child mappers. If you want to have the same relationship() objects set up on all concrete mappers, they must be configured manually on each. To configure back references in such a configuration the
back_populates keyword may be used instead of backref, such as below where both A(object) and B(A)
bidirectionally reference C:
ajoin = polymorphic_union({
'a':a_table,
'b':b_table
}, 'type', 'ajoin')
mapper(A, a_table, with_polymorphic=('*', ajoin),
polymorphic_on=ajoin.c.type, polymorphic_identity='a',
properties={
'some_c':relationship(C, back_populates='many_a')
})
mapper(B, b_table,inherits=A, concrete=True,
polymorphic_identity='b',
properties={
'some_c':relationship(C, back_populates='many_a')
})
mapper(C, c_table, properties={
'many_a':relationship(A, collection_class=set,
back_populates='some_c'),
})
2.5.5 Using Inheritance with Declarative
Declarative makes inheritance configuration more intuitive. See the docs at Inheritance Configuration.
2.6 Using the Session
The orm.mapper() function and declarative extensions are the primary configurational interface for the ORM.
Once mappings are configured, the primary usage interface for persistence operations is the Session.
128
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
2.6.1 What does the Session do ?
In the most general sense, the Session establishes all conversations with the database and represents a “holding
zone” for all the objects which you’ve loaded or associated with it during its lifespan. It provides the entrypoint to
acquire a Query object, which sends queries to the database using the Session object’s current database connection,
populating result rows into objects that are then stored in the Session, inside a structure called the Identity Map - a
data structure that maintains unique copies of each object, where “unique” means “only one object with a particular
primary key”.
The Session begins in an essentially stateless form. Once queries are issued or other objects are persisted with it, it
requests a connection resource from an Engine that is associated either with the Session itself or with the mapped
Table objects being operated upon. This connection represents an ongoing transaction, which remains in effect until
the Session is instructed to commit or roll back its pending state.
All changes to objects maintained by a Session are tracked - before the database is queried again or before the
current transaction is committed, it flushes all pending changes to the database. This is known as the Unit of Work
pattern.
When using a Session, it’s important to note that the objects which are associated with it are proxy objects to the
transaction being held by the Session - there are a variety of events that will cause objects to re-access the database
in order to keep synchronized. It is possible to “detach” objects from a Session, and to continue using them, though
this practice has its caveats. It’s intended that usually, you’d re-associate detached objects with another Session
when you want to work with them again, so that they can resume their normal task of representing database state.
2.6.2 Getting a Session
Session is a regular Python class which can be directly instantiated. However, to standardize how sessions are
configured and acquired, the sessionmaker class is normally used to create a top level Session configuration
which can then be used throughout an application without the need to repeat the configurational arguments.
The usage of sessionmaker is illustrated below:
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
# an Engine, which the Session will use for connection
# resources
some_engine = create_engine('postgresql://scott:[email protected]/')
# create a configured "Session" class
Session = sessionmaker(bind=some_engine)
# create a Session
session = Session()
# work with sess
myobject = MyObject('foo', 'bar')
session.add(myobject)
session.commit()
Above, the sessionmaker call creates a factory for us, which we assign to the name Session. This factory, when
called, will create a new Session object using the configurational arguments we’ve given the factory. In this case,
as is typical, we’ve configured the factory to specify a particular Engine for connection resources.
A typical setup will associate the sessionmaker with an Engine, so that each Session generated will use this
Engine to acquire connection resources. This association can be set up as in the example above, using the bind
argument.
2.6. Using the Session
129
SQLAlchemy Documentation, Release 0.8.7
When you write your application, place the sessionmaker factory at the global level. This factory can then be used
by the rest of the applcation as the source of new Session instances, keeping the configuration for how Session
objects are constructed in one place.
The sessionmaker factory can also be used in conjunction with other helpers, which are passed a user-defined
sessionmaker that is then maintained by the helper. Some of these helpers are discussed in the section When do I
construct a Session, when do I commit it, and when do I close it?.
Adding Additional Configuration to an Existing sessionmaker()
A common scenario is where the sessionmaker is invoked at module import time, however the generation of one
or more Engine instances to be associated with the sessionmaker has not yet proceeded. For this use case,
the sessionmaker construct offers the sessionmaker.configure() method, which will place additional
configuration directives into an existing sessionmaker that will take place when the construct is invoked:
from sqlalchemy.orm import sessionmaker
from sqlalchemy import create_engine
# configure Session class with desired options
Session = sessionmaker()
# later, we create the engine
engine = create_engine('postgresql://...')
# associate it with our custom Session class
Session.configure(bind=engine)
# work with the session
session = Session()
Creating Ad-Hoc Session Objects with Alternate Arguments
For the use case where an application needs to create a new Session with special arguments that deviate from
what is normally used throughout the application, such as a Session that binds to an alternate source of connectivity, or a Session that should have other arguments such as expire_on_commit established differently
from what most of the application wants, specific arguments can be passed to the sessionmaker factory’s
sessionmaker.__call__() method. These arguments will override whatever configurations have already been
placed, such as below, where a new Session is constructed against a specific Connection:
# at the module level, the global sessionmaker,
# bound to a specific Engine
Session = sessionmaker(bind=engine)
# later, some unit of code wants to create a
# Session that is bound to a specific Connection
conn = engine.connect()
session = Session(bind=conn)
The typical rationale for the association of a Session with a specific Connection is that of a test fixture that
maintains an external transaction - see Joining a Session into an External Transaction for an example of this.
130
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
2.6.3 Using the Session
Quickie Intro to Object States
It’s helpful to know the states which an instance can have within a session:
• Transient - an instance that’s not in a session, and is not saved to the database; i.e. it has no database identity.
The only relationship such an object has to the ORM is that its class has a mapper() associated with it.
• Pending - when you add() a transient instance, it becomes pending. It still wasn’t actually flushed to the
database yet, but it will be when the next flush occurs.
• Persistent - An instance which is present in the session and has a record in the database. You get persistent
instances by either flushing so that the pending instances become persistent, or by querying the database for
existing instances (or moving persistent instances from other sessions into your local session).
• Detached - an instance which has a record in the database, but is not in any session. There’s nothing wrong with
this, and you can use objects normally when they’re detached, except they will not be able to issue any SQL in
order to load collections or attributes which are not yet loaded, or were marked as “expired”.
Knowing these states is important, since the Session tries to be strict about ambiguous operations (such as trying to
save the same object to two different sessions at the same time).
Getting the Current State of an Object
The actual state of any mapped object can be viewed at any time using the inspect() system:
>>> from sqlalchemy import inspect
>>> insp = inspect(my_object)
>>> insp.persistent
True
See also:
InstanceState.transient
InstanceState.pending
InstanceState.persistent
InstanceState.detached
Session Frequently Asked Questions
When do I make a sessionmaker?
Just one time, somewhere in your application’s global scope. It should be looked upon as part of your application’s
configuration. If your application has three .py files in a package, you could, for example, place the sessionmaker
line in your __init__.py file; from that point on your other modules say “from mypackage import Session”. That
way, everyone else just uses Session(), and the configuration of that session is controlled by that central point.
If your application starts up, does imports, but does not know what database it’s going to be connecting to, you can
bind the Session at the “class” level to the engine later on, using sessionmaker.configure().
In the examples in this section, we will frequently show the sessionmaker being created right above the line
where we actually invoke Session. But that’s just for example’s sake! In reality, the sessionmaker would be
somewhere at the module level. The calls to instantiate Session would then be placed at the point in the application
where database conversations begin.
2.6. Using the Session
131
SQLAlchemy Documentation, Release 0.8.7
When do I construct a Session, when do I commit it, and when do I close it?
tl;dr;
As a general rule, keep the lifecycle of the session separate and external from functions and objects that access
and/or manipulate database data.
A Session is typically constructed at the beginning of a logical operation where database access is potentially
anticipated.
The Session, whenever it is used to talk to the database, begins a database transaction as soon as it starts communicating. Assuming the autocommit flag is left at its recommended default of False, this transaction remains in
progress until the Session is rolled back, committed, or closed. The Session will begin a new transaction if it is
used again, subsequent to the previous transaction ending; from this it follows that the Session is capable of having
a lifespan across many transactions, though only one at a time. We refer to these two concepts as transaction scope
and session scope.
The implication here is that the SQLAlchemy ORM is encouraging the developer to establish these two scopes in
their application, including not only when the scopes begin and end, but also the expanse of those scopes, for example
should a single Session instance be local to the execution flow within a function or method, should it be a global
object used by the entire application, or somewhere in between these two.
The burden placed on the developer to determine this scope is one area where the SQLAlchemy ORM necessarily has
a strong opinion about how the database should be used. The unit of work pattern is specifically one of accumulating
changes over time and flushing them periodically, keeping in-memory state in sync with what’s known to be present
in a local transaction. This pattern is only effective when meaningful transaction scopes are in place.
It’s usually not very hard to determine the best points at which to begin and end the scope of a Session, though the
wide variety of application architectures possible can introduce challenging situations.
A common choice is to tear down the Session at the same time the transaction ends, meaning the transaction and
session scopes are the same. This is a great choice to start out with as it removes the need to consider session scope as
separate from transaction scope.
While there’s no one-size-fits-all recommendation for how transaction scope should be determined, there are common
patterns. Especially if one is writing a web application, the choice is pretty much established.
A web application is the easiest case because such an appication is already constructed around a single, consistent
scope - this is the request, which represents an incoming request from a browser, the processing of that request to
formulate a response, and finally the delivery of that response back to the client. Integrating web applications with
the Session is then the straightforward task of linking the scope of the Session to that of the request. The
Session can be established as the request begins, or using a lazy initialization pattern which establishes one as
soon as it is needed. The request then proceeds, with some system in place where application logic can access the
current Session in a manner associated with how the actual request object is accessed. As the request ends, the
Session is torn down as well, usually through the usage of event hooks provided by the web framework. The
transaction used by the Session may also be committed at this point, or alternatively the application may opt for an
explicit commit pattern, only committing for those requests where one is warranted, but still always tearing down the
Session unconditionally at the end.
Some web frameworks include infrastructure to assist in the task of aligning the lifespan of a Session with that
of a web request. This includes products such as Flask-SQLAlchemy, for usage in conjunction with the Flask web
framework, and Zope-SQLAlchemy, typically used with the Pyramid framework. SQLAlchemy recommends that
these products be used as available.
In those situations where the integration libraries are not provided or are insufficient, SQLAlchemy includes its own
“helper” class known as scoped_session. A tutorial on the usage of this object is at Contextual/Thread-local
132
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
Sessions. It provides both a quick way to associate a Session with the current thread, as well as patterns to associate
Session objects with other kinds of scopes.
As mentioned before, for non-web applications there is no one clear pattern, as applications themselves don’t have
just one pattern of architecture. The best strategy is to attempt to demarcate “operations”, points at which a particular
thread begins to perform a series of operations for some period of time, which can be committed at the end. Some
examples:
• A background daemon which spawns off child forks would want to create a Session local to each child
process, work with that Session through the life of the “job” that the fork is handling, then tear it down when
the job is completed.
• For a command-line script, the application would create a single, global Session that is established when the
program begins to do its work, and commits it right as the program is completing its task.
• For a GUI interface-driven application, the scope of the Session may best be within the scope of a usergenerated event, such as a button push. Or, the scope may correspond to explicit user interaction, such as the
user “opening” a series of records, then “saving” them.
As a general rule, the application should manage the lifecycle of the session externally to functions that deal with
specific data. This is a fundamental separation of concerns which keeps data-specific operations agnostic of the
context in which they access and manipulate that data.
E.g. don’t do this:
### this is the **wrong way to do it** ###
class ThingOne(object):
def go(self):
session = Session()
try:
session.query(FooBar).update({"x": 5})
session.commit()
except:
session.rollback()
raise
class ThingTwo(object):
def go(self):
session = Session()
try:
session.query(Widget).update({"q": 18})
session.commit()
except:
session.rollback()
raise
def run_my_program():
ThingOne().go()
ThingTwo().go()
Keep the lifecycle of the session (and usually the transaction) separate and external:
### this is a **better** (but not the only) way to do it ###
class ThingOne(object):
def go(self, session):
session.query(FooBar).update({"x": 5})
class ThingTwo(object):
2.6. Using the Session
133
SQLAlchemy Documentation, Release 0.8.7
def go(self, session):
session.query(Widget).update({"q": 18})
def run_my_program():
session = Session()
try:
ThingOne().go(session)
ThingTwo().go(session)
session.commit()
except:
session.rollback()
raise
finally:
session.close()
The advanced developer will try to keep the details of session, transaction and exception management as far as possible
from the details of the program doing its work. For example, we can further separate concerns using a context manager:
### another way (but again *not the only way*) to do it ###
from contextlib import contextmanager
@contextmanager
def session_scope():
"""Provide a transactional scope around a series of operations."""
session = Session()
try:
yield session
session.commit()
except:
session.rollback()
raise
finally:
session.close()
def run_my_program():
with session_scope() as session:
ThingOne().go(session)
ThingTwo().go(session)
Is the Session a cache?
Yeee...no. It’s somewhat used as a cache, in that it implements the identity map pattern, and stores objects
keyed to their primary key. However, it doesn’t do any kind of query caching. This means, if you say
session.query(Foo).filter_by(name=’bar’), even if Foo(name=’bar’) is right there, in the identity map, the session has no idea about that. It has to issue SQL to the database, get the rows back, and then when it
sees the primary key in the row, then it can look in the local identity map and see that the object is already there. It’s
only when you say query.get({some primary key}) that the Session doesn’t have to issue a query.
Additionally, the Session stores object instances using a weak reference by default. This also defeats the purpose of
using the Session as a cache.
The Session is not designed to be a global object from which everyone consults as a “registry” of objects. That’s
more the job of a second level cache. SQLAlchemy provides a pattern for implementing second level caching using
dogpile.cache, via the Dogpile Caching example.
134
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
How can I get the Session for a certain object?
Use the object_session() classmethod available on Session:
session = Session.object_session(someobject)
The newer Runtime Inspection API system can also be used:
from sqlalchemy import inspect
session = inspect(someobject).session
Is the session thread-safe?
The Session is very much intended to be used in a non-concurrent fashion, which usually means in only one thread
at a time.
The Session should be used in such a way that one instance exists for a single series of operations within a single transaction. One expedient way to get this effect is by associating a Session with the current thread (see
Contextual/Thread-local Sessions for background). Another is to use a pattern where the Session is passed between
functions and is otherwise not shared with other threads.
The bigger point is that you should not want to use the session with multiple concurrent threads. That would be like
having everyone at a restaurant all eat from the same plate. The session is a local “workspace” that you use for a
specific set of tasks; you don’t want to, or need to, share that session with other threads who are doing some other task.
Making sure the Session is only used in a single concurrent thread at a time is called a “share nothing” approach
to concurrency. But actually, not sharing the Session implies a more significant pattern; it means not just the
Session object itself, but also all objects that are associated with that Session, must be kept within the scope
of a single concurrent thread. The set of mapped objects associated with a Session are essentially proxies for
data within database rows accessed over a database connection, and so just like the Session itself, the whole set
of objects is really just a large-scale proxy for a database connection (or connections). Ultimately, it’s mostly the
DBAPI connection itself that we’re keeping away from concurrent access; but since the Session and all the objects
associated with it are all proxies for that DBAPI connection, the entire graph is essentially not safe for concurrent
access.
If there are in fact multiple threads participating in the same task, then you may consider sharing the session and its
objects between those threads; however, in this extremely unusual scenario the application would need to ensure that
a proper locking scheme is implemented so that there isn’t concurrent access to the Session or its state. A more
common approach to this situation is to maintain a single Session per concurrent thread, but to instead copy objects
from one Session to another, often using the Session.merge() method to copy the state of an object into a new
object local to a different Session.
Querying
The query() function takes one or more entities and returns a new Query object which will issue mapper queries
within the context of this Session. An entity is defined as a mapped class, a Mapper object, an orm-enabled descriptor,
or an AliasedClass object:
# query from a class
session.query(User).filter_by(name='ed').all()
# query with multiple classes, returns tuples
session.query(User, Address).join('addresses').filter_by(name='ed').all()
# query using orm-enabled descriptors
session.query(User.name, User.fullname).all()
2.6. Using the Session
135
SQLAlchemy Documentation, Release 0.8.7
# query from a mapper
user_mapper = class_mapper(User)
session.query(user_mapper)
When Query returns results, each object instantiated is stored within the identity map. When a row matches an
object which is already present, the same object is returned. In the latter case, whether or not the row is populated
onto an existing object depends upon whether the attributes of the instance have been expired or not. A defaultconfigured Session automatically expires all instances along transaction boundaries, so that with a normally isolated
transaction, there shouldn’t be any issue of instances representing data which is stale with regards to the current
transaction.
The Query object is introduced in great detail in Object Relational Tutorial, and further documented in Querying.
Adding New or Existing Items
add() is used to place instances in the session. For transient (i.e. brand new) instances, this will have the effect of
an INSERT taking place for those instances upon the next flush. For instances which are persistent (i.e. were loaded
by this session), they are already present and do not need to be added. Instances which are detached (i.e. have been
removed from a session) may be re-associated with a session using this method:
user1 = User(name='user1')
user2 = User(name='user2')
session.add(user1)
session.add(user2)
session.commit()
# write changes to the database
To add a list of items to the session at once, use add_all():
session.add_all([item1, item2, item3])
The add() operation cascades along the save-update cascade. For more details see the section Cascades.
Merging
merge() transfers state from an outside object into a new or already existing instance within a session. It also
reconciles the incoming data against the state of the database, producing a history stream which will be applied
towards the next flush, or alternatively can be made to produce a simple “transfer” of state without producing change
history or accessing the database. Usage is as follows:
merged_object = session.merge(existing_object)
When given an instance, it follows these steps:
• It examines the primary key of the instance. If it’s present, it attempts to locate that instance in the local identity
map. If the load=True flag is left at its default, it also checks the database for this primary key if not located
locally.
• If the given instance has no primary key, or if no instance can be found with the primary key given, a new
instance is created.
• The state of the given instance is then copied onto the located/newly created instance. For attributes which are
present on the source instance, the value is transferred to the target instance. For mapped attributes which aren’t
present on the source, the attribute is expired on the target instance, discarding its existing value.
If the load=True flag is left at its default, this copy process emits events and will load the target object’s
unloaded collections for each attribute present on the source object, so that the incoming state can be reconciled
136
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
against what’s present in the database. If load is passed as False, the incoming data is “stamped” directly
without producing any history.
• The operation is cascaded to related objects and collections, as indicated by the merge cascade (see Cascades).
• The new instance is returned.
With merge(), the given “source” instance is not modified nor is it associated with the target Session, and remains
available to be merged with any number of other Session objects. merge() is useful for taking the state of any
kind of object structure without regard for its origins or current session associations and copying its state into a new
session. Here’s some examples:
• An application which reads an object structure from a file and wishes to save it to the database might parse the
file, build up the structure, and then use merge() to save it to the database, ensuring that the data within the
file is used to formulate the primary key of each element of the structure. Later, when the file has changed, the
same process can be re-run, producing a slightly different object structure, which can then be merged in again,
and the Session will automatically update the database to reflect those changes, loading each object from the
database by primary key and then updating its state with the new state given.
• An application is storing objects in an in-memory cache, shared by many Session objects simultaneously.
merge() is used each time an object is retrieved from the cache to create a local copy of it in each Session
which requests it. The cached object remains detached; only its state is moved into copies of itself that are local
to individual Session objects.
In the caching use case, it’s common to use the load=False flag to remove the overhead of reconciling the
object’s state with the database. There’s also a “bulk” version of merge() called merge_result() that was
designed to work with cache-extended Query objects - see the section Dogpile Caching.
• An application wants to transfer the state of a series of objects into a Session maintained by a worker thread
or other concurrent system. merge() makes a copy of each object to be placed into this new Session. At
the end of the operation, the parent thread/process maintains the objects it started with, and the thread/worker
can proceed with local copies of those objects.
In the “transfer between threads/processes” use case, the application may want to use the load=False flag as
well to avoid overhead and redundant SQL queries as the data is transferred.
Merge Tips
merge() is an extremely useful method for many purposes. However, it deals with the intricate border between
objects that are transient/detached and those that are persistent, as well as the automated transference of state. The
wide variety of scenarios that can present themselves here often require a more careful approach to the state of objects.
Common problems with merge usually involve some unexpected state regarding the object being passed to merge().
Lets use the canonical example of the User and Address objects:
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String(50), nullable=False)
addresses = relationship("Address", backref="user")
class Address(Base):
__tablename__ = 'address'
id = Column(Integer, primary_key=True)
email_address = Column(String(50), nullable=False)
user_id = Column(Integer, ForeignKey('user.id'), nullable=False)
2.6. Using the Session
137
SQLAlchemy Documentation, Release 0.8.7
Assume a User object with one Address, already persistent:
>>> u1 = User(name='ed', addresses=[Address(email_address='[email protected]')])
>>> session.add(u1)
>>> session.commit()
We now create a1, an object outside the session, which we’d like to merge on top of the existing Address:
>>> existing_a1 = u1.addresses[0]
>>> a1 = Address(id=existing_a1.id)
A surprise would occur if we said this:
>>> a1.user = u1
>>> a1 = session.merge(a1)
>>> session.commit()
sqlalchemy.orm.exc.FlushError: New instance <Address at 0x1298f50>
with identity key (<class '__main__.Address'>, (1,)) conflicts with
persistent instance <Address at 0x12a25d0>
Why is that ? We weren’t careful with our cascades. The assignment of a1.user to a persistent object cascaded to
the backref of User.addresses and made our a1 object pending, as though we had added it. Now we have two
Address objects in the session:
>>> a1 = Address()
>>> a1.user = u1
>>> a1 in session
True
>>> existing_a1 in session
True
>>> a1 is existing_a1
False
Above, our a1 is already pending in the session. The subsequent merge() operation essentially does nothing.
Cascade can be configured via the cascade option on relationship(), although in this case it would mean
removing the save-update cascade from the User.addresses relationship - and usually, that behavior is extremely convenient. The solution here would usually be to not assign a1.user to an object already persistent in the
target session.
The cascade_backrefs=False option of relationship() will also prevent the Address from being added
to the session via the a1.user = u1 assignment.
Further detail on cascade operation is at Cascades.
Another example of unexpected state:
>>> a1 = Address(id=existing_a1.id, user_id=u1.id)
>>> assert a1.user is None
>>> True
>>> a1 = session.merge(a1)
>>> session.commit()
sqlalchemy.exc.IntegrityError: (IntegrityError) address.user_id
may not be NULL
Here, we accessed a1.user, which returned its default value of None, which as a result of this access, has been placed
in the __dict__ of our object a1. Normally, this operation creates no change event, so the user_id attribute takes
precedence during a flush. But when we merge the Address object into the session, the operation is equivalent to:
>>> existing_a1.id = existing_a1.id
>>> existing_a1.user_id = u1.id
>>> existing_a1.user = None
138
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
Where above, both user_id and user are assigned to, and change events are emitted for both. The user association
takes precedence, and None is applied to user_id, causing a failure.
Most merge() issues can be examined by first checking - is the object prematurely in the session ?
>>> a1 = Address(id=existing_a1, user_id=user.id)
>>> assert a1 not in session
>>> a1 = session.merge(a1)
Or is there state on the object that we don’t want ? Examining __dict__ is a quick way to check:
>>> a1 = Address(id=existing_a1, user_id=user.id)
>>> a1.user
>>> a1.__dict__
{'_sa_instance_state': <sqlalchemy.orm.state.InstanceState object at 0x1298d10>,
'user_id': 1,
'id': 1,
'user': None}
>>> # we don't want user=None merged, remove it
>>> del a1.user
>>> a1 = session.merge(a1)
>>> # success
>>> session.commit()
Deleting
The delete() method places an instance into the Session’s list of objects to be marked as deleted:
# mark two objects to be deleted
session.delete(obj1)
session.delete(obj2)
# commit (or flush)
session.commit()
Deleting from Collections
A common confusion that arises regarding delete() is when objects which are members of a collection are being
deleted. While the collection member is marked for deletion from the database, this does not impact the collection
itself in memory until the collection is expired. Below, we illustrate that even after an Address object is marked for
deletion, it’s still present in the collection associated with the parent User, even after a flush:
>>> address = user.addresses[1]
>>> session.delete(address)
>>> session.flush()
>>> address in user.addresses
True
When the above session is committed, all attributes are expired. The next access of user.addresses will re-load
the collection, revealing the desired state:
>>> session.commit()
>>> address in user.addresses
False
The usual practice of deleting items within collections is to forego the usage of delete() directly, and instead use
cascade behavior to automatically invoke the deletion as a result of removing the object from the parent collection.
The delete-orphan cascade accomplishes this, as illustrated in the example below:
2.6. Using the Session
139
SQLAlchemy Documentation, Release 0.8.7
mapper(User, users_table, properties={
'addresses':relationship(Address, cascade="all, delete, delete-orphan")
})
del user.addresses[1]
session.flush()
Where above, upon removing the Address object from the User.addresses collection, the delete-orphan
cascade has the effect of marking the Address object for deletion in the same way as passing it to delete().
See also Cascades for detail on cascades.
Deleting based on Filter Criterion
The caveat with Session.delete() is that you need to have an object handy already in order to delete. The Query
includes a delete() method which deletes based on filtering criteria:
session.query(User).filter(User.id==7).delete()
The Query.delete() method includes functionality to “expire” objects already in the session which match the
criteria. However it does have some caveats, including that “delete” and “delete-orphan” cascades won’t be fully
expressed for collections which are already loaded. See the API docs for delete() for more details.
Flushing
When the Session is used with its default configuration, the flush step is nearly always done transparently. Specifically, the flush occurs before any individual Query is issued, as well as within the commit() call before the
transaction is committed. It also occurs before a SAVEPOINT is issued when begin_nested() is used.
Regardless of the autoflush setting, a flush can always be forced by issuing flush():
session.flush()
The “flush-on-Query” aspect of the behavior can be disabled by constructing sessionmaker with the flag
autoflush=False:
Session = sessionmaker(autoflush=False)
Additionally, autoflush can be temporarily disabled by setting the autoflush flag at any time:
mysession = Session()
mysession.autoflush = False
Some autoflush-disable recipes are available at DisableAutoFlush.
The flush process always occurs within a transaction, even if the Session has been configured with
autocommit=True, a setting that disables the session’s persistent transactional state. If no transaction is present,
flush() creates its own transaction and commits it. Any failures during flush will always result in a rollback of
whatever transaction is present. If the Session is not in autocommit=True mode, an explicit call to rollback()
is required after a flush fails, even though the underlying transaction will have been rolled back already - this is so that
the overall nesting pattern of so-called “subtransactions” is consistently maintained.
Committing
commit() is used to commit the current transaction. It always issues flush() beforehand to flush any remaining
state to the database; this is independent of the “autoflush” setting. If no transaction is present, it raises an error. Note
140
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
that the default behavior of the Session is that a “transaction” is always present; this behavior can be disabled by
setting autocommit=True. In autocommit mode, a transaction can be initiated by calling the begin() method.
Note: The term “transaction” here refers to a transactional construct within the Session itself which may be maintaining zero or more actual database (DBAPI) transactions. An individual DBAPI connection begins participation in
the “transaction” as it is first used to execute a SQL statement, then remains present until the session-level “transaction”
is completed. See Managing Transactions for further detail.
Another behavior of commit() is that by default it expires the state of all instances present after the commit is
complete. This is so that when the instances are next accessed, either through attribute access or by them being present
in a Query result set, they receive the most recent state. To disable this behavior, configure sessionmaker with
expire_on_commit=False.
Normally, instances loaded into the Session are never changed by subsequent queries; the assumption is that the
current transaction is isolated so the state most recently loaded is correct as long as the transaction continues. Setting
autocommit=True works against this model to some degree since the Session behaves in exactly the same way
with regard to attribute state, except no transaction is present.
Rolling Back
rollback() rolls back the current transaction. With a default configured session, the post-rollback state of the
session is as follows:
• All transactions are rolled back and all connections returned to the connection pool, unless the Session was
bound directly to a Connection, in which case the connection is still maintained (but still rolled back).
• Objects which were initially in the pending state when they were added to the Session within the lifespan
of the transaction are expunged, corresponding to their INSERT statement being rolled back. The state of their
attributes remains unchanged.
• Objects which were marked as deleted within the lifespan of the transaction are promoted back to the persistent
state, corresponding to their DELETE statement being rolled back. Note that if those objects were first pending
within the transaction, that operation takes precedence instead.
• All objects not expunged are fully expired.
With that state understood, the Session may safely continue usage after a rollback occurs.
When a flush() fails, typically for reasons like primary key, foreign key, or “not nullable” constraint violations,
a rollback() is issued automatically (it’s currently not possible for a flush to continue after a partial failure).
However, the flush process always uses its own transactional demarcator called a subtransaction, which is described
more fully in the docstrings for Session. What it means here is that even though the database transaction has been
rolled back, the end user must still issue rollback() to fully reset the state of the Session.
Expunging
Expunge removes an object from the Session, sending persistent instances to the detached state, and pending instances
to the transient state:
session.expunge(obj1)
To remove all items, call expunge_all() (this method was formerly known as clear()).
2.6. Using the Session
141
SQLAlchemy Documentation, Release 0.8.7
Closing
The close() method issues a expunge_all(), and releases any transactional/connection resources. When connections are returned to the connection pool, transactional state is rolled back as well.
Refreshing / Expiring
The Session normally works in the context of an ongoing transaction (with the default setting of autoflush=False). Most
databases offer “isolated” transactions - this refers to a series of behaviors that allow the work within a transaction to
remain consistent as time passes, regardless of the activities outside of that transaction. A key feature of a high degree
of transaction isolation is that emitting the same SELECT statement twice will return the same results as when it was
called the first time, even if the data has been modified in another transaction.
For this reason, the Session gains very efficient behavior by loading the attributes of each instance only once.
Subsequent reads of the same row in the same transaction are assumed to have the same value. The user application
also gains directly from this assumption, that the transaction is regarded as a temporary shield against concurrent
changes - a good application will ensure that isolation levels are set appropriately such that this assumption can be
made, given the kind of data being worked with.
To clear out the currently loaded state on an instance, the instance or its individual attributes can be marked as “expired”, which results in a reload to occur upon next access of any of the instance’s attrbutes. The instance can also be
immediately reloaded from the database. The expire() and refresh() methods achieve this:
# immediately re-load attributes on obj1, obj2
session.refresh(obj1)
session.refresh(obj2)
# expire objects obj1, obj2, attributes will be reloaded
# on the next access:
session.expire(obj1)
session.expire(obj2)
When an expired object reloads, all non-deferred column-based attributes are loaded in one query. Current behavior
for expired relationship-based attributes is that they load individually upon access - this behavior may be enhanced in
a future release. When a refresh is invoked on an object, the ultimate operation is equivalent to a Query.get(), so
any relationships configured with eager loading should also load within the scope of the refresh operation.
refresh() and expire() also support being passed a list of individual attribute names in which to be refreshed.
These names can refer to any attribute, column-based or relationship based:
# immediately re-load the attributes 'hello', 'world' on obj1, obj2
session.refresh(obj1, ['hello', 'world'])
session.refresh(obj2, ['hello', 'world'])
# expire the attributes 'hello', 'world' objects obj1, obj2, attributes will be reloaded
# on the next access:
session.expire(obj1, ['hello', 'world'])
session.expire(obj2, ['hello', 'world'])
The full contents of the session may be expired at once using expire_all():
session.expire_all()
Note that expire_all() is called automatically whenever commit() or rollback() are called. If using
the session in its default mode of autocommit=False and with a well-isolated transactional environment (which is
provided by most backends with the notable exception of MySQL MyISAM), there is virtually no reason to ever call
expire_all() directly - plenty of state will remain on the current transaction until it is rolled back or committed
or otherwise removed.
142
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
refresh() and expire() similarly are usually only necessary when an UPDATE or DELETE has been issued
manually within the transaction using Session.execute().
Session Attributes
The Session itself acts somewhat like a set-like collection. All items present may be accessed using the iterator
interface:
for obj in session:
print obj
And presence may be tested for using regular “contains” semantics:
if obj in session:
print "Object is present"
The session is also keeping track of all newly created (i.e. pending) objects, all objects which have had changes since
they were last loaded or saved (i.e. “dirty”), and everything that’s been marked as deleted:
# pending objects recently added to the Session
session.new
# persistent objects which currently have changes detected
# (this collection is now created on the fly each time the property is called)
session.dirty
# persistent objects that have been marked as deleted via session.delete(obj)
session.deleted
# dictionary of all persistent objects, keyed on their
# identity key
session.identity_map
(Documentation: Session.new, Session.dirty, Session.deleted, Session.identity_map).
Note that objects within the session are by default weakly referenced. This means that when they are dereferenced in
the outside application, they fall out of scope from within the Session as well and are subject to garbage collection
by the Python interpreter. The exceptions to this include objects which are pending, objects which are marked as
deleted, or persistent objects which have pending changes on them. After a full flush, these collections are all empty,
and all objects are again weakly referenced. To disable the weak referencing behavior and force all objects within the
session to remain until explicitly expunged, configure sessionmaker with the weak_identity_map=False
setting.
2.6.4 Cascades
Mappers support the concept of configurable cascade behavior on relationship() constructs. This refers to how
operations performed on a “parent” object relative to a particular Session should be propagated to items referred to
by that relationship (e.g. “child” objects), and is affected by the relationship.cascade option.
The default behavior of cascade is limited to cascades of the so-called save-update and merge settings. The typical
“alternative” setting for cascade is to add the delete and delete-orphan options; these settings are appropriate for related
objects which only exist as long as they are attached to their parent, and are otherwise deleted.
Cascade behavior is configured using the by changing the cascade option on relationship():
2.6. Using the Session
143
SQLAlchemy Documentation, Release 0.8.7
class Order(Base):
__tablename__ = 'order'
items = relationship("Item", cascade="all, delete-orphan")
customer = relationship("User", cascade="save-update")
To set cascades on a backref, the same flag can be used with the backref() function, which ultimately feeds its
arguments back into relationship():
class Item(Base):
__tablename__ = 'item'
order = relationship("Order",
backref=backref("items", cascade="all, delete-orphan")
)
The Origins of Cascade
SQLAlchemy’s notion of cascading behavior on relationships, as well as the options to configure them, are
primarily derived from the similar feature in the Hibernate ORM; Hibernate refers to “cascade” in a few places
such as in Example: Parent/Child. If cascades are confusing, we’ll refer to their conclusion, stating “The sections
we have just covered can be a bit confusing. However, in practice, it all works out nicely.”
The default value of cascade is save-update, merge. The typical alternative setting for this parameter is either
all or more commonly all, delete-orphan. The all symbol is a synonym for save-update, merge,
refresh-expire, expunge, delete, and using it in conjunction with delete-orphan indicates that the
child object should follow along with its parent in all cases, and be deleted once it is no longer associated with that
parent.
The list of available values which can be specified for the cascade parameter are described in the following subsections.
save-update
save-update cascade indicates that when an object is placed into a Session via Session.add(), all the
objects associated with it via this relationship() should also be added to that same Session. Suppose we have
an object user1 with two related objects address1, address2:
>>> user1 = User()
>>> address1, address2 = Address(), Address()
>>> user1.addresses = [address1, address2]
If we add user1 to a Session, it will also add address1, address2 implicitly:
>>> sess = Session()
>>> sess.add(user1)
>>> address1 in sess
True
save-update cascade also affects attribute operations for objects that are already present in a Session. If we add
a third object, address3 to the user1.addresses collection, it becomes part of the state of that Session:
>>>
>>>
>>>
>>>
144
address3 = Address()
user1.append(address3)
address3 in sess
True
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
save-update has the possibly surprising behavior which is that persistent objects which were removed from a
collection or in some cases a scalar attribute may also be pulled into the Session of a parent object; this is so that
the flush process may handle that related object appropriately. This case can usually only arise if an object is removed
from one Session and added to another:
>>> user1 = sess1.query(User).filter_by(id=1).first()
>>> address1 = user1.addresses[0]
>>> sess1.close()
# user1, address1 no longer associated with sess1
>>> user1.addresses.remove(address1) # address1 no longer associated with user1
>>> sess2 = Session()
>>> sess2.add(user1)
# ... but it still gets added to the new session,
>>> address1 in sess2 # because it's still "pending" for flush
True
The save-update cascade is on by default, and is typically taken for granted; it simplifies code by allowing a
single call to Session.add() to register an entire structure of objects within that Session at once. While it can
be disabled, there is usually not a need to do so.
One case where save-update cascade does sometimes get in the way is in that it takes place in both directions for
bi-directional relationships, e.g. backrefs, meaning that the association of a child object with a particular parent can
have the effect of the parent object being implicitly associated with that child object’s Session; this pattern, as well
as how to modify its behavior using the cascade_backrefs flag, is discussed in the section Controlling Cascade
on Backrefs.
delete
The delete cascade indicates that when a “parent” object is marked for deletion, its related “child” objects should
also be marked for deletion. If for example we we have a relationship User.addresses with delete cascade
configured:
class User(Base):
# ...
addresses = relationship("Address", cascade="save-update, merge, delete")
If using the above mapping, we have a User object and two related Address objects:
>>> user1 = sess.query(User).filter_by(id=1).first()
>>> address1, address2 = user1.addresses
If we mark user1 for deletion, after the flush operation proceeds, address1 and address2 will also be deleted:
>>> sess.delete(user1)
>>> sess.commit()
DELETE FROM address WHERE address.id = ?
((1,), (2,))
DELETE FROM user WHERE user.id = ?
(1,)
COMMIT
Alternatively, if our User.addresses relationship does not have delete cascade, SQLAlchemy’s default behavior is to instead de-associate address1 and address2 from user1 by setting their foreign key reference to NULL.
Using a mapping as follows:
class User(Base):
# ...
addresses = relationship("Address")
2.6. Using the Session
145
SQLAlchemy Documentation, Release 0.8.7
Upon deletion of a parent User object, the rows in address are not deleted, but are instead de-associated:
>>> sess.delete(user1)
>>> sess.commit()
UPDATE address SET user_id=? WHERE address.id = ?
(None, 1)
UPDATE address SET user_id=? WHERE address.id = ?
(None, 2)
DELETE FROM user WHERE user.id = ?
(1,)
COMMIT
delete cascade is more often than not used in conjunction with delete-orphan cascade, which will emit a DELETE
for the related row if the “child” object is deassociated from the parent. The combination of delete and
delete-orphan cascade covers both situations where SQLAlchemy has to decide between setting a foreign key
column to NULL versus deleting the row entirely.
146
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
ORM-level “delete” cascade vs. FOREIGN KEY level “ON DELETE” cascade
The behavior of SQLAlchemy’s “delete” cascade has a lot of overlap with the ON DELETE CASCADE feature
of a database foreign key, as well as with that of the ON DELETE SET NULL foreign key setting when “delete”
cascade is not specified. Database level “ON DELETE” cascades are specific to the “FOREIGN KEY” construct
of the relational database; SQLAlchemy allows configuration of these schema-level constructs at the DDL level
using options on ForeignKeyConstraint which are described at ON UPDATE and ON DELETE.
It is important to note the differences between the ORM and the relational database’s notion of “cascade” as well
as how they integrate:
• A database level ON DELETE cascade is configured effectively on the many-to-one side of the relationship; that is, we configure it relative to the FOREIGN KEY constraint that is the “many” side of a
relationship. At the ORM level, this direction is reversed. SQLAlchemy handles the deletion of “child”
objects relative to a “parent” from the “parent” side, which means that delete and delete-orphan
cascade are configured on the one-to-many side.
• Database level foreign keys with no ON DELETE setting are often used to prevent a parent row from
being removed, as it would necessarily leave an unhandled related row present. If this behavior is desired
in a one-to-many relationship, SQLAlchemy’s default behavior of setting a foreign key to NULL can be
caught in one of two ways:
– The easiest and most common is just to set the foreign-key-holding column to NOT NULL at the
database schema level. An attempt by SQLAlchemy to set the column to NULL will fail with a
simple NOT NULL constraint exception.
– The other, more special case way is to set the passive_deletes flag to the string "all". This
has the effect of entirely disabling SQLAlchemy’s behavior of setting the foreign key column to
NULL, and a DELETE will be emitted for the parent row without any affect on the child row, even
if the child row is present in memory. This may be desirable in the case when database-level foreign
key triggers, either special ON DELETE settings or otherwise, need to be activated in all cases when
a parent row is deleted.
• Database level ON DELETE cascade is vastly more efficient than that of SQLAlchemy. The database
can chain a series of cascade operations across many relationships at once; e.g. if row A is deleted, all the
related rows in table B can be deleted, and all the C rows related to each of those B rows, and on and on,
all within the scope of a single DELETE statement. SQLAlchemy on the other hand, in order to support
the cascading delete operation fully, has to individually load each related collection in order to target all
rows that then may have further related collections. That is, SQLAlchemy isn’t sophisticated enough to
emit a DELETE for all those related rows at once within this context.
• SQLAlchemy doesn’t need to be this sophisticated, as we instead provide smooth integration with the
database’s own ON DELETE functionality, by using the passive_deletes option in conjunction with
properly configured foreign key constraints. Under this behavior, SQLAlchemy only emits DELETE for
those rows that are already locally present in the Session; for any collections that are unloaded, it leaves
them to the database to handle, rather than emitting a SELECT for them. The section Using Passive
Deletes provides an example of this use.
• While database-level ON DELETE functionality works only on the “many” side of a relationship,
SQLAlchemy’s “delete” cascade has limited ability to operate in the reverse direction as well, meaning it can be configured on the “many” side to delete an object on the “one” side when the reference on
the “many” side is deleted. However this can easily result in constraint violations if there are other objects
referring to this “one” side from the “many”, so it typically is only useful when a relationship is in fact a
“one to one”. The single_parent flag should be used to establish an in-Python assertion for this case.
When using a relationship() that also includes a many-to-many table using the secondary option,
SQLAlchemy’s delete cascade handles the rows in this many-to-many table automatically. Just like, as described
in Deleting Rows from the Many to Many Table, the addition or removal of an object from a many-to-many collection
results in the INSERT or DELETE of a row in the many-to-many table, the delete cascade, when activated as the result of a parent object delete operation, will DELETE not just the row in the “child” table but also in the many-to-many
table.
2.6. Using the Session
147
SQLAlchemy Documentation, Release 0.8.7
delete-orphan
delete-orphan cascade adds behavior to the delete cascade, such that a child object will be marked for deletion
when it is de-associated from the parent, not just when the parent is marked for deletion. This is a common feature
when dealing with a related object that is “owned” by its parent, with a NOT NULL foreign key, so that removal of
the item from the parent collection results in its deletion.
delete-orphan cascade implies that each child object can only have one parent at a time, so is configured in
the vast majority of cases on a one-to-many relationship. Setting it on a many-to-one or many-to-many relationship is more awkward; for this use case, SQLAlchemy requires that the relationship() be configured with the
single_parent argument, establishes Python-side validation that ensures the object is associated with only one
parent at a time.
merge
merge cascade indicates that the Session.merge() operation should be propagated from a parent that’s the
subject of the Session.merge() call down to referred objects. This cascade is also on by default.
refresh-expire
refresh-expire is an uncommon option, indicating that the Session.expire() operation should be propagated from a parent down to referred objects. When using Session.refresh(), the referred objects are expired
only, but not actually refreshed.
expunge
expunge cascade indicates that when the parent object is removed from the Session using
Session.expunge(), the operation should be propagated down to referred objects.
Controlling Cascade on Backrefs
The save-update cascade by default takes place on attribute change events emitted from backrefs. This is probably a
confusing statement more easily described through demonstration; it means that, given a mapping such as this:
mapper(Order, order_table, properties={
'items' : relationship(Item, backref='order')
})
If an Order is already in the session, and is assigned to the order attribute of an Item, the backref appends the
Order to the items collection of that Order, resulting in the save-update cascade taking place:
>>> o1 = Order()
>>> session.add(o1)
>>> o1 in session
True
>>> i1 = Item()
>>> i1.order = o1
>>> i1 in o1.items
True
>>> i1 in session
True
This behavior can be disabled using the cascade_backrefs flag:
148
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
mapper(Order, order_table, properties={
'items' : relationship(Item, backref='order',
cascade_backrefs=False)
})
So above, the assignment of i1.order = o1 will append i1 to the items collection of o1, but will not add
i1 to the session. You can, of course, add() i1 to the session at a later point. This option may be helpful for
situations where an object needs to be kept out of a session until it’s construction is completed, but still needs to be
given associations to objects which are already persistent in the target session.
2.6.5 Managing Transactions
A newly constructed Session may be said to be in the “begin” state. In this state, the Session has not established
any connection or transactional state with any of the Engine objects that may be associated with it.
The Session then receives requests to operate upon a database connection. Typically, this means it is
called upon to execute SQL statements using a particular Engine, which may be via Session.query(),
Session.execute(), or within a flush operation of pending data, which occurs when such state exists and
Session.commit() or Session.flush() is called.
As these requests are received, each new Engine encountered is associated with an ongoing transactional state
maintained by the Session. When the first Engine is operated upon, the Session can be said to have left
the “begin” state and entered “transactional” state. For each Engine encountered, a Connection is associated
with it, which is acquired via the Engine.contextual_connect() method. If a Connection was directly
associated with the Session (see Joining a Session into an External Transaction for an example of this), it is added
to the transactional state directly.
For each Connection, the Session also maintains a Transaction object, which is acquired by calling Connection.begin() on each Connection, or if the Session object has
been established using the flag twophase=True, a TwoPhaseTransaction object acquired via
Connection.begin_twophase(). These transactions are all committed or rolled back corresponding to
the invocation of the Session.commit() and Session.rollback() methods. A commit operation will also
call the TwoPhaseTransaction.prepare() method on all transactions if applicable.
When the transactional state is completed after a rollback or commit, the Session releases all Transaction
and Connection resources, and goes back to the “begin” state, which will again invoke new Connection and
Transaction objects as new requests to emit SQL statements are received.
The example below illustrates this lifecycle:
engine = create_engine("...")
Session = sessionmaker(bind=engine)
# new session.
no connections are in use.
session = Session()
try:
# first query. a Connection is acquired
# from the Engine, and a Transaction
# started.
item1 = session.query(Item).get(1)
# second query. the same Connection/Transaction
# are used.
item2 = session.query(Item).get(2)
# pending changes are created.
item1.foo = 'bar'
2.6. Using the Session
149
SQLAlchemy Documentation, Release 0.8.7
item2.bar = 'foo'
# commit. The pending changes above
# are flushed via flush(), the Transaction
# is committed, the Connection object closed
# and discarded, the underlying DBAPI connection
# returned to the connection pool.
session.commit()
except:
# on rollback, the same closure of state
# as that of commit proceeds.
session.rollback()
raise
Using SAVEPOINT
SAVEPOINT transactions, if supported by the underlying engine, may be delineated using the begin_nested()
method:
Session = sessionmaker()
session = Session()
session.add(u1)
session.add(u2)
session.begin_nested() # establish a savepoint
session.add(u3)
session.rollback() # rolls back u3, keeps u1 and u2
session.commit() # commits u1 and u2
begin_nested() may be called any number of times, which will issue a new SAVEPOINT with a unique identifier
for each call. For each begin_nested() call, a corresponding rollback() or commit() must be issued.
When begin_nested() is called, a flush() is unconditionally issued (regardless of the autoflush setting).
This is so that when a rollback() occurs, the full state of the session is expired, thus causing all subsequent
attribute/instance access to reference the full state of the Session right before begin_nested() was called.
begin_nested(), in the same manner as the less often used begin() method, returns a transactional object
which also works as a context manager. It can be succinctly used around individual record inserts in order to catch
things like unique constraint exceptions:
for record in records:
try:
with session.begin_nested():
session.merge(record)
except:
print "Skipped record %s" % record
session.commit()
Autocommit Mode
The example of Session transaction lifecycle illustrated at the start of Managing Transactions applies to a Session
configured in the default mode of autocommit=False. Constructing a Session with autocommit=True
produces a Session placed into “autocommit” mode, where each SQL statement invoked by a Session.query()
or Session.execute() occurs using a new connection from the connection pool, discarding it after results have
150
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
been iterated. The Session.flush() operation still occurs within the scope of a single transaction, though this
transaction is closed out after the Session.flush() operation completes.
Warning: “autocommit” mode should not be considered for general use. If used, it should always be combined
with the usage of Session.begin() and Session.commit(), to ensure a transaction demarcation.
Executing queries outside of a demarcated transaction is a legacy mode of usage, and can in some cases lead to
concurrent connection checkouts.
In the absence of a demarcated transaction, the Session cannot make appropriate decisions as to when
autoflush should occur nor when auto-expiration should occur, so these features should be disabled with
autoflush=False, expire_on_commit=False.
Modern usage of “autocommit” is for framework integrations that need to control specifically when the “begin” state occurs. A session which is configured with autocommit=True may be placed into the “begin” state using the Session.begin() method. After the cycle completes upon Session.commit() or
Session.rollback(), connection and transaction resources are released and the Session goes back into “autocommit” mode, until Session.begin() is called again:
Session = sessionmaker(bind=engine, autocommit=True)
session = Session()
session.begin()
try:
item1 = session.query(Item).get(1)
item2 = session.query(Item).get(2)
item1.foo = 'bar'
item2.bar = 'foo'
session.commit()
except:
session.rollback()
raise
The Session.begin() method also returns a transactional token which is compatible with the Python 2.6 with
statement:
Session = sessionmaker(bind=engine, autocommit=True)
session = Session()
with session.begin():
item1 = session.query(Item).get(1)
item2 = session.query(Item).get(2)
item1.foo = 'bar'
item2.bar = 'foo'
Using Subtransactions with Autocommit
A subtransaction indicates usage of the Session.begin() method in conjunction with the
subtransactions=True flag. This produces a non-transactional, delimiting construct that allows nesting
of calls to begin() and commit(). Its purpose is to allow the construction of code that can function within a
transaction both independently of any external code that starts a transaction, as well as within a block that has already
demarcated a transaction.
subtransactions=True is generally only useful in conjunction with autocommit, and is equivalent to the pattern
described at Nesting of Transaction Blocks, where any number of functions can call Connection.begin() and
Transaction.commit() as though they are the initiator of the transaction, but in fact may be participating in an
already ongoing transaction:
# method_a starts a transaction and calls method_b
def method_a(session):
2.6. Using the Session
151
SQLAlchemy Documentation, Release 0.8.7
session.begin(subtransactions=True)
try:
method_b(session)
session.commit() # transaction is committed here
except:
session.rollback() # rolls back the transaction
raise
# method_b also starts a transaction, but when
# called from method_a participates in the ongoing
# transaction.
def method_b(session):
session.begin(subtransactions=True)
try:
session.add(SomeObject('bat', 'lala'))
session.commit() # transaction is not committed yet
except:
session.rollback() # rolls back the transaction, in this case
# the one that was initiated in method_a().
raise
# create a Session and call method_a
session = Session(autocommit=True)
method_a(session)
session.close()
Subtransactions are used by the Session.flush() process to ensure that the flush operation takes place within a
transaction, regardless of autocommit. When autocommit is disabled, it is still useful in that it forces the Session
into a “pending rollback” state, as a failed flush cannot be resumed in mid-operation, where the end user still maintains
the “scope” of the transaction overall.
Enabling Two-Phase Commit
For backends which support two-phase operaration (currently MySQL and PostgreSQL), the session can be instructed
to use two-phase commit semantics. This will coordinate the committing of transactions across databases so that the
transaction is either committed or rolled back in all databases. You can also prepare() the session for interacting
with transactions not managed by SQLAlchemy. To use two phase transactions set the flag twophase=True on the
session:
engine1 = create_engine('postgresql://db1')
engine2 = create_engine('postgresql://db2')
Session = sessionmaker(twophase=True)
# bind User operations to engine 1, Account operations to engine 2
Session.configure(binds={User:engine1, Account:engine2})
session = Session()
# .... work with accounts and users
# commit. session will issue a flush to all DBs, and a prepare step to all DBs,
# before committing both transactions
session.commit()
152
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
2.6.6 Embedding SQL Insert/Update Expressions into a Flush
This feature allows the value of a database column to be set to a SQL expression instead of a literal value. It’s especially
useful for atomic updates, calling stored procedures, etc. All you do is assign an expression to an attribute:
class SomeClass(object):
pass
mapper(SomeClass, some_table)
someobject = session.query(SomeClass).get(5)
# set 'value' attribute to a SQL expression adding one
someobject.value = some_table.c.value + 1
# issues "UPDATE some_table SET value=value+1"
session.commit()
This technique works both for INSERT and UPDATE statements. After the flush/commit operation, the value
attribute on someobject above is expired, so that when next accessed the newly generated value will be loaded
from the database.
2.6.7 Using SQL Expressions with Sessions
SQL expressions and strings can be executed via the Session within its transactional context. This is most easily
accomplished using the execute() method, which returns a ResultProxy in the same manner as an Engine or
Connection:
Session = sessionmaker(bind=engine)
session = Session()
# execute a string statement
result = session.execute("select * from table where id=:id", {'id':7})
# execute a SQL expression construct
result = session.execute(select([mytable]).where(mytable.c.id==7))
The current Connection held by the Session is accessible using the connection() method:
connection = session.connection()
The examples above deal with a Session that’s bound to a single Engine or Connection. To execute statements
using a Session which is bound either to multiple engines, or none at all (i.e. relies upon bound metadata), both
execute() and connection() accept a mapper keyword argument, which is passed a mapped class or Mapper
instance, which is used to locate the proper context for the desired engine:
Session = sessionmaker()
session = Session()
# need to specify mapper or class when executing
result = session.execute("select * from table where id=:id", {'id':7}, mapper=MyMappedClass)
result = session.execute(select([mytable], mytable.c.id==7), mapper=MyMappedClass)
connection = session.connection(MyMappedClass)
2.6. Using the Session
153
SQLAlchemy Documentation, Release 0.8.7
2.6.8 Joining a Session into an External Transaction
If a Connection is being used which is already in a transactional state (i.e. has a Transaction established), a
Session can be made to participate within that transaction by just binding the Session to that Connection. The
usual rationale for this is a test suite that allows ORM code to work freely with a Session, including the ability to
call Session.commit(), where afterwards the entire database interaction is rolled back:
from sqlalchemy.orm import sessionmaker
from sqlalchemy import create_engine
from unittest import TestCase
# global application scope.
Session = sessionmaker()
create Session class, engine
engine = create_engine('postgresql://...')
class SomeTest(TestCase):
def setUp(self):
# connect to the database
self.connection = engine.connect()
# begin a non-ORM transaction
self.trans = self.connection.begin()
# bind an individual Session to the connection
self.session = Session(bind=self.connection)
def test_something(self):
# use the session in tests.
self.session.add(Foo())
self.session.commit()
def tearDown(self):
# rollback - everything that happened with the
# Session above (including calls to commit())
# is rolled back.
self.trans.rollback()
self.session.close()
# return connection to the Engine
self.connection.close()
Above, we issue Session.commit() as well as Transaction.rollback(). This is an example of where we
take advantage of the Connection object’s ability to maintain subtransactions, or nested begin/commit-or-rollback
pairs where only the outermost begin/commit pair actually commits the transaction, or if the outermost block rolls
back, everything is rolled back.
2.6.9 Contextual/Thread-local Sessions
Recall from the section When do I construct a Session, when do I commit it, and when do I close it?, the concept
of “session scopes” was introduced, with an emphasis on web applications and the practice of linking the scope of a
Session with that of a web request. Most modern web frameworks include integration tools so that the scope of the
Session can be managed automatically, and these tools should be used as they are available.
SQLAlchemy includes its own helper object, which helps with the establishment of user-defined Session scopes. It
is also used by third-party integration systems to help construct their integration schemes.
154
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
The object is the scoped_session object, and it represents a registry of Session objects. If you’re not familiar
with the registry pattern, a good introduction can be found in Patterns of Enterprise Architecture.
Note: The scoped_session object is a very popular and useful object used by many SQLAlchemy applications.
However, it is important to note that it presents only one approach to the issue of Session management. If you’re
new to SQLAlchemy, and especially if the term “thread-local variable” seems strange to you, we recommend that if
possible you familiarize first with an off-the-shelf integration system such as Flask-SQLAlchemy or zope.sqlalchemy.
A scoped_session is constructed by calling it, passing it a factory which can create new Session objects. A
factory is just something that produces a new object when called, and in the case of Session, the most common
factory is the sessionmaker, introduced earlier in this section. Below we illustrate this usage:
>>> from sqlalchemy.orm import scoped_session
>>> from sqlalchemy.orm import sessionmaker
>>> session_factory = sessionmaker(bind=some_engine)
>>> Session = scoped_session(session_factory)
The scoped_session object we’ve created will now call upon the sessionmaker when we “call” the registry:
>>> some_session = Session()
Above, some_session is an instance of Session, which we can now use to talk to the database. This same
Session is also present within the scoped_session registry we’ve created. If we call upon the registry a second
time, we get back the same Session:
>>> some_other_session = Session()
>>> some_session is some_other_session
True
This pattern allows disparate sections of the application to call upon a global scoped_session, so that all those
areas may share the same session without the need to pass it explicitly. The Session we’ve established in our registry
will remain, until we explicitly tell our registry to dispose of it, by calling scoped_session.remove():
>>> Session.remove()
The scoped_session.remove() method first calls Session.close() on the current Session, which
has the effect of releasing any connection/transactional resources owned by the Session first, then discarding the
Session itself. “Releasing” here means that connections are returned to their connection pool and any transactional
state is rolled back, ultimately using the rollback() method of the underlying DBAPI connection.
At this point, the scoped_session object is “empty”, and will create a new Session when called again. As
illustrated below, this is not the same Session we had before:
>>> new_session = Session()
>>> new_session is some_session
False
The above series of steps illustrates the idea of the “registry” pattern in a nutshell. With that basic idea in hand, we
can discuss some of the details of how this pattern proceeds.
Implicit Method Access
The job of the scoped_session is simple; hold onto a Session for all who ask for it. As a means of producing
more transparent access to this Session, the scoped_session also includes proxy behavior, meaning that the
registry itself can be treated just like a Session directly; when methods are called on this object, they are proxied
to the underlying Session being maintained by the registry:
2.6. Using the Session
155
SQLAlchemy Documentation, Release 0.8.7
Session = scoped_session(some_factory)
# equivalent to:
#
# session = Session()
# print session.query(MyClass).all()
#
print Session.query(MyClass).all()
The above code accomplishes the same task as that of acquiring the current Session by calling upon the registry,
then using that Session.
Thread-Local Scope
Users who are familiar with multithreaded programming will note that representing anything as a global variable is
usually a bad idea, as it implies that the global object will be accessed by many threads concurrently. The Session
object is entirely designed to be used in a non-concurrent fashion, which in terms of multithreading means “only
in one thread at a time”. So our above example of scoped_session usage, where the same Session object is
maintained across multiple calls, suggests that some process needs to be in place such that mutltiple calls across many
threads don’t actually get a handle to the same session. We call this notion thread local storage, which means, a
special object is used that will maintain a distinct object per each application thread. Python provides this via the
threading.local() construct. The scoped_session object by default uses this object as storage, so that a single
Session is maintained for all who call upon the scoped_session registry, but only within the scope of a single
thread. Callers who call upon the registry in a different thread get a Session instance that is local to that other thread.
Using this technique, the scoped_session provides a quick and relatively simple (if one is familiar with threadlocal storage) way of providing a single, global object in an application that is safe to be called upon from multiple
threads.
The scoped_session.remove() method, as always, removes the current Session associated with the thread,
if any. However, one advantage of the threading.local() object is that if the application thread itself ends, the
“storage” for that thread is also garbage collected. So it is in fact “safe” to use thread local scope with an application
that spawns and tears down threads, without the need to call scoped_session.remove(). However, the scope
of transactions themselves, i.e. ending them via Session.commit() or Session.rollback(), will usually
still be something that must be explicitly arranged for at the appropriate time, unless the application actually ties the
lifespan of a thread to the lifespan of a transaction.
Using Thread-Local Scope with Web Applications
As discussed in the section When do I construct a Session, when do I commit it, and when do I close it?, a web
application is architected around the concept of a web request, and integrating such an application with the Session
usually implies that the Session will be associated with that request. As it turns out, most Python web frameworks,
with notable exceptions such as the asynchronous frameworks Twisted and Tornado, use threads in a simple way, such
that a particular web request is received, processed, and completed within the scope of a single worker thread. When
the request ends, the worker thread is released to a pool of workers where it is available to handle another request.
This simple correspondence of web request and thread means that to associate a Session with a thread implies it is
also associated with the web request running within that thread, and vice versa, provided that the Session is created
only after the web request begins and torn down just before the web request ends. So it is a common practice to use
scoped_session as a quick way to integrate the Session with a web application. The sequence diagram below
illustrates this flow:
Web Server
-------------startup
->
156
Web Framework
-------------Web framework
SQLAlchemy ORM Code
-----------------------------# Session registry is established
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
initializes
incoming
web request
->
web request
starts
Session = scoped_session(sessionmaker())
->
# The registry is *optionally*
# called upon explicitly to create
# a Session local to the thread and/or request
Session()
# the Session registry can otherwise
# be used at any time, creating the
# request-local Session() if not present,
# or returning the existing one
Session.query(MyClass) # ...
Session.add(some_object) # ...
# if data was modified, commit the
# transaction
Session.commit()
outgoing web
response
web request ends
-> # the registry is instructed to
# remove the Session
Session.remove()
sends output
<-
<-
Using the above flow, the process of integrating the Session with the web application has exactly two requirements:
1. Create a single scoped_session registry when the web application first starts, ensuring that this object is
accessible by the rest of the application.
2. Ensure that scoped_session.remove() is called when the web request ends, usually by integrating with
the web framework’s event system to establish an “on request end” event.
As noted earlier, the above pattern is just one potential way to integrate a Session with a web framework, one which
in particular makes the significant assumption that the web framework associates web requests with application
threads. It is however strongly recommended that the integration tools provided with the web framework itself
be used, if available, instead of scoped_session.
In particular, while using a thread local can be convenient, it is preferable that the Session be associated directly
with the request, rather than with the current thread. The next section on custom scopes details a more advanced
configuration which can combine the usage of scoped_session with direct request based scope, or any kind of
scope.
Using Custom Created Scopes
The scoped_session object’s default behavior of “thread local” scope is only one of many options on how to
“scope” a Session. A custom scope can be defined based on any existing system of getting at “the current thing we
are working with”.
Suppose a web framework defines a library function get_current_request(). An application built using this
framework can call this function at any time, and the result will be some kind of Request object that represents
the current request being processed. If the Request object is hashable, then this function can be easily integrated
with scoped_session to associate the Session with the request. Below we illustrate this in conjunction with
a hypothetical event marker provided by the web framework on_request_end, which allows code to be invoked
whenever a request ends:
2.6. Using the Session
157
SQLAlchemy Documentation, Release 0.8.7
from my_web_framework import get_current_request, on_request_end
from sqlalchemy.orm import scoped_session, sessionmaker
Session = scoped_session(sessionmaker(bind=some_engine), scopefunc=get_current_request)
@on_request_end
def remove_session(req):
Session.remove()
Above, we instantiate scoped_session in the usual way, except that we pass our request-returning function as
the “scopefunc”. This instructs scoped_session to use this function to generate a dictionary key whenever the
registry is called upon to return the current Session. In this case it is particularly important that we ensure a reliable
“remove” system is implemented, as this dictionary is not otherwise self-managed.
Contextual Session API
class sqlalchemy.orm.scoping.scoped_session(session_factory, scopefunc=None)
Provides scoped management of Session objects.
See Contextual/Thread-local Sessions for a tutorial.
__call__(**kw)
Return the current Session, creating it using the session factory if not present.
Parameters **kw – Keyword arguments will be passed to the session factory callable, if an
existing Session is not present. If the Session is present and keyword arguments have
been passed, InvalidRequestError is raised.
__init__(session_factory, scopefunc=None)
Construct a new scoped_session.
Parameters
• session_factory – a factory to create new Session instances. This is usually, but
not necessarily, an instance of sessionmaker.
• scopefunc – optional function which defines the current scope. If not passed,
the scoped_session object assumes “thread-local” scope, and will use a Python
threading.local() in order to maintain the current Session. If passed, the function should return a hashable token; this token will be used as the key in a dictionary in
order to store and retrieve the current Session.
configure(**kwargs)
reconfigure the sessionmaker used by this scoped_session.
See sessionmaker.configure().
query_property(query_cls=None)
return a class property which produces a Query object against the class and the current Session when
called.
e.g.:
Session = scoped_session(sessionmaker())
class MyClass(object):
query = Session.query_property()
# after mappers are defined
result = MyClass.query.filter(MyClass.name=='foo').all()
158
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
Produces instances of the session’s configured query class by default. To override and use a custom implementation, provide a query_cls callable. The callable will be invoked with the class’s mapper as a
positional argument and a session keyword argument.
There is no limit to the number of query properties placed on a class.
remove()
Dispose of the current Session, if present.
This will first call Session.close() method on the current Session, which releases any existing
transactional/connection resources still being held; transactions specifically are rolled back. The Session
is then discarded. Upon next usage within the same scope, the scoped_session will produce a new
Session object.
class sqlalchemy.util.ScopedRegistry(createfunc, scopefunc)
A Registry that can store one or multiple instances of a single class on the basis of a “scope” function.
The object implements __call__ as the “getter”, so by calling myregistry() the contained object is
returned for the current scope.
Parameters
• createfunc – a callable that returns a new object to be placed in the registry
• scopefunc – a callable that will return a key to store/retrieve an object.
__init__(createfunc, scopefunc)
Construct a new ScopedRegistry.
Parameters
• createfunc – A creation function that will generate a new value for the current scope,
if none is present.
• scopefunc – A function that returns a hashable token representing the current scope
(such as, current thread identifier).
clear()
Clear the current scope, if any.
has()
Return True if an object is present in the current scope.
set(obj)
Set the value for the current scope.
class sqlalchemy.util.ThreadLocalRegistry(createfunc)
Bases: sqlalchemy.util._collections.ScopedRegistry
A ScopedRegistry that uses a threading.local() variable for storage.
2.6.10 Partitioning Strategies
Simple Vertical Partitioning
Vertical partitioning places different kinds of objects, or different tables, across multiple databases:
engine1 = create_engine('postgresql://db1')
engine2 = create_engine('postgresql://db2')
Session = sessionmaker(twophase=True)
2.6. Using the Session
159
SQLAlchemy Documentation, Release 0.8.7
# bind User operations to engine 1, Account operations to engine 2
Session.configure(binds={User:engine1, Account:engine2})
session = Session()
Above, operations against either class will make usage of the Engine linked to that class. Upon a flush operation,
similar rules take place to ensure each class is written to the right database.
The transactions among the multiple databases can optionally be coordinated via two phase commit, if the underlying
backend supports it. See Enabling Two-Phase Commit for an example.
Custom Vertical Partitioning
More comprehensive rule-based class-level partitioning can be built by overriding the Session.get_bind()
method. Below we illustrate a custom Session which delivers the following rules:
1. Flush operations are delivered to the engine named master.
2. Operations on objects that subclass MyOtherClass all occur on the other engine.
3. Read operations for all other classes occur on a random choice of the slave1 or slave2 database.
engines = {
'master':create_engine("sqlite:///master.db"),
'other':create_engine("sqlite:///other.db"),
'slave1':create_engine("sqlite:///slave1.db"),
'slave2':create_engine("sqlite:///slave2.db"),
}
from sqlalchemy.orm import Session, sessionmaker
import random
class RoutingSession(Session):
def get_bind(self, mapper=None, clause=None):
if mapper and issubclass(mapper.class_, MyOtherClass):
return engines['other']
elif self._flushing:
return engines['master']
else:
return engines[
random.choice(['slave1','slave2'])
]
The above Session class is plugged in using the class_ argument to sessionmaker:
Session = sessionmaker(class_=RoutingSession)
This approach can be combined with multiple MetaData objects, using an approach such as that of using the declarative __abstract__ keyword, described at __abstract__.
Horizontal Partitioning
Horizontal partitioning partitions the rows of a single table (or a set of tables) across multiple databases.
See the “sharding” example: Horizontal Sharding.
160
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
2.6.11 Sessions API
Session and sessionmaker()
class sqlalchemy.orm.session.sessionmaker(bind=None,
class_=<class
‘sqlalchemy.orm.session.Session’>,
autoflush=True,
autocommit=False,
expire_on_commit=True, **kw)
Bases: sqlalchemy.orm.session._SessionClassMethods
A configurable Session factory.
The sessionmaker factory generates new Session objects when called, creating them given the configurational arguments established here.
e.g.:
# global scope
Session = sessionmaker(autoflush=False)
# later, in a local scope, create and use a session:
sess = Session()
Any keyword arguments sent to the constructor itself will override the “configured” keywords:
Session = sessionmaker()
# bind an individual session to a connection
sess = Session(bind=connection)
The class also includes a method configure(), which can be used to specify additional keyword arguments to
the factory, which will take effect for subsequent Session objects generated. This is usually used to associate
one or more Engine objects with an existing sessionmaker factory before it is first used:
# application starts
Session = sessionmaker()
# ... later
engine = create_engine('sqlite:///foo.db')
Session.configure(bind=engine)
sess = Session()
__call__(**local_kw)
Produce a new Session object using the configuration established in this sessionmaker.
In Python, the __call__ method is invoked on an object when it is “called” in the same way as a
function:
Session = sessionmaker()
session = Session() # invokes sessionmaker.__call__()
__init__(bind=None, class_=<class ‘sqlalchemy.orm.session.Session’>, autoflush=True, autocommit=False, expire_on_commit=True, **kw)
Construct a new sessionmaker.
All arguments here except for class_ correspond to arguments accepted by Session directly. See the
Session.__init__() docstring for more details on parameters.
Parameters
2.6. Using the Session
161
SQLAlchemy Documentation, Release 0.8.7
• bind – a Engine or other Connectable with which newly created Session objects
will be associated.
• class_ – class to use in order to create new Session objects. Defaults to Session.
• autoflush – The autoflush setting to use with newly created Session objects.
• autocommit – The autocommit setting to use with newly created Session objects.
• expire_on_commit=True – the expire_on_commit setting to use with newly created
Session objects.
• **kw – all other keyword arguments are passed to the constructor of newly created
Session objects.
close_all()
inherited from the close_all() method of _SessionClassMethods
Close all sessions in memory.
configure(**new_kw)
(Re)configure the arguments for this sessionmaker.
e.g.:
Session = sessionmaker()
Session.configure(bind=create_engine('sqlite://'))
identity_key(*args, **kwargs)
inherited from the identity_key() method of _SessionClassMethods
Return an identity key.
This is an alias of util.identity_key().
object_session(instance)
inherited from the object_session() method of _SessionClassMethods
Return the Session to which an object belongs.
This is an alias of object_session().
class sqlalchemy.orm.session.Session(bind=None, autoflush=True, expire_on_commit=True,
_enable_transaction_accounting=True,
autocommit=False, twophase=False, weak_identity_map=True,
binds=None,
extension=None,
query_cls=<class
‘sqlalchemy.orm.query.Query’>)
Bases: sqlalchemy.orm.session._SessionClassMethods
Manages persistence operations for ORM-mapped objects.
The Session’s usage paradigm is described at Using the Session.
__init__(bind=None,
autoflush=True,
expire_on_commit=True,
_enable_transaction_accounting=True,
autocommit=False,
twophase=False,
weak_identity_map=True,
binds=None,
extension=None,
query_cls=<class
‘sqlalchemy.orm.query.Query’>)
Construct a new Session.
See also the sessionmaker function which is used to generate a Session-producing callable with a
given set of arguments.
Parameters
162
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
• autocommit –
Warning: The autocommit flag is not for general use, and if it is used, queries should
only be invoked within the span of a Session.begin() / Session.commit()
pair. Executing queries outside of a demarcated transaction is a legacy mode of usage,
and can in some cases lead to concurrent connection checkouts.
Defaults to False. When True, the Session does not keep a persistent transaction running, and will acquire connections from the engine on an as-needed basis, returning them
immediately after their use. Flushes will begin and commit (or possibly rollback) their own
transaction if no transaction is present. When using this mode, the Session.begin()
method is used to explicitly start transactions.
See also:
Autocommit Mode
• autoflush – When True, all query operations will issue a flush() call to this
Session before proceeding. This is a convenience feature so that flush() need not
be called repeatedly in order for database queries to retrieve results. It’s typical that
autoflush is used in conjunction with autocommit=False. In this scenario, explicit calls to flush() are rarely needed; you usually only need to call commit()
(which flushes) to finalize changes.
• bind – An optional Engine or Connection to which this Session should be bound.
When specified, all SQL operations performed by this session will execute via this connectable.
• binds –
An optional dictionary which contains more granular “bind” information than the
bind parameter provides. This dictionary can map individual :class‘.Table‘ instances
as well as Mapper instances to individual Engine or Connection objects. Operations which proceed relative to a particular Mapper will consult this dictionary
for the direct Mapper instance as well as the mapper’s mapped_table attribute
in order to locate a connectable to use. The full resolution is described in the
Session.get_bind(). Usage looks like:
Session = sessionmaker(binds={
SomeMappedClass: create_engine('postgresql://engine1'),
somemapper: create_engine('postgresql://engine2'),
some_table: create_engine('postgresql://engine3'),
})
Also see the Session.bind_mapper() and Session.bind_table() methods.
• class_
–
Specify
an
alternate
class
other
than
sqlalchemy.orm.session.Session which should be used by the returned
class. This is the only argument that is local to the sessionmaker function, and is not
sent directly to the constructor for Session.
• _enable_transaction_accounting – Defaults to True. A legacy-only flag
which when False disables all 0.5-style object accounting on transaction boundaries,
including auto-expiry of instances on rollback and commit, maintenance of the “new” and
“deleted” lists upon rollback, and autoflush of pending changes upon begin(), all of
which are interdependent.
• expire_on_commit – Defaults to True. When True, all instances will be fully expired after each commit(), so that all attribute/object access subsequent to a completed
transaction will load from the most recent database state.
2.6. Using the Session
163
SQLAlchemy Documentation, Release 0.8.7
• extension – An optional SessionExtension instance, or a list of such instances,
which will receive pre- and post- commit and flush events, as well as a post-rollback event.
Deprecated. Please see SessionEvents.
• query_cls – Class which should be used to create new Query objects, as returned by
the query() method. Defaults to Query.
• twophase – When True, all transactions will be started as a “two phase” transaction,
i.e. using the “two phase” semantics of the database in use along with an XID. During a
commit(), after flush() has been issued for all attached databases, the prepare()
method on each database’s TwoPhaseTransaction will be called. This allows each
database to roll back the entire transaction, before each transaction is committed.
• weak_identity_map – Defaults to True - when set to False, objects placed in the
Session will be strongly referenced until explicitly removed or the Session is closed.
Deprecated - this option is obsolete.
add(instance, _warn=True)
Place an object in the Session.
Its state will be persisted to the database on the next flush operation.
Repeated calls to add() will be ignored. The opposite of add() is expunge().
add_all(instances)
Add the given collection of instances to this Session.
begin(subtransactions=False, nested=False)
Begin a transaction on this Session.
If this Session is already within a transaction, either a plain transaction or nested transaction, an error is
raised, unless subtransactions=True or nested=True is specified.
The subtransactions=True flag indicates that this begin() can create a subtransaction if a transaction is already in progress. For documentation on subtransactions, please see Using Subtransactions
with Autocommit.
The nested flag begins a SAVEPOINT transaction and is equivalent to calling begin_nested().
For documentation on SAVEPOINT transactions, please see Using SAVEPOINT.
begin_nested()
Begin a nested transaction on this Session.
The target database(s) must support SQL SAVEPOINTs or a SQLAlchemy-supported vendor implementation of the idea.
For documentation on SAVEPOINT transactions, please see Using SAVEPOINT.
bind_mapper(mapper, bind)
Bind operations for a mapper to a Connectable.
mapper A mapper instance or mapped class
bind Any Connectable: a Engine or Connection.
All subsequent operations involving this mapper will use the given bind.
bind_table(table, bind)
Bind operations on a Table to a Connectable.
table A Table instance
bind Any Connectable: a Engine or Connection.
All subsequent operations involving this Table will use the given bind.
164
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
close()
Close this Session.
This clears all items and ends any transaction in progress.
If this session were created with autocommit=False, a new transaction is immediately begun. Note
that this new transaction does not use any connection resources until they are first needed.
close_all()
inherited from the close_all() method of _SessionClassMethods
Close all sessions in memory.
commit()
Flush pending changes and commit the current transaction.
If no transaction is in progress, this method raises an InvalidRequestError.
By default, the Session also expires all database loaded state on all ORM-managed attributes after
transaction commit. This so that subsequent operations load the most recent data from the database. This
behavior can be disabled using the expire_on_commit=False option to sessionmaker or the
Session constructor.
If a subtransaction is in effect (which occurs when begin() is called multiple times), the subtransaction
will be closed, and the next call to commit() will operate on the enclosing transaction.
When using the Session in its default mode of autocommit=False, a new transaction will be begun
immediately after the commit, but note that the newly begun transaction does not use any connection
resources until the first SQL is actually emitted.
See also:
Committing
connection(mapper=None, clause=None, bind=None, close_with_result=False, **kw)
Return a Connection object corresponding to this Session object’s transactional state.
If this Session is configured with autocommit=False, either the Connection corresponding
to the current transaction is returned, or if no transaction is in progress, a new one is begun and the
Connection returned (note that no transactional state is established with the DBAPI until the first SQL
statement is emitted).
Alternatively, if this Session is configured with autocommit=True, an ad-hoc Connection is
returned using Engine.contextual_connect() on the underlying Engine.
Ambiguity in multi-bind or unbound Session objects can be resolved through any of the optional keyword arguments. This ultimately makes usage of the get_bind() method for resolution.
Parameters
• bind – Optional Engine to be used as the bind. If this engine is already involved in
an ongoing transaction, that connection will be used. This argument takes precedence
over mapper, clause.
• mapper – Optional mapper() mapped class, used to identify the appropriate bind.
This argument takes precedence over clause.
• clause – A ClauseElement (i.e. select(), text(), etc.) which will be
used to locate a bind, if a bind cannot otherwise be identified.
• close_with_result – Passed to Engine.connect(), indicating the
Connection should be considered “single use”, automatically closing when the
first result set is closed. This flag only has an effect if this Session is configured
with autocommit=True and does not already have a transaction in progress.
2.6. Using the Session
165
SQLAlchemy Documentation, Release 0.8.7
• **kw – Additional keyword arguments are sent to get_bind(), allowing additional arguments to be passed to custom implementations of get_bind().
delete(instance)
Mark an instance as deleted.
The database delete operation occurs upon flush().
deleted
The set of all instances marked as ‘deleted’ within this Session
dirty
The set of all persistent instances considered dirty.
E.g.:
some_mapped_object in session.dirty
Instances are considered dirty when they were modified but not deleted.
Note that this ‘dirty’ calculation is ‘optimistic’; most attribute-setting or collection modification operations will mark an instance as ‘dirty’ and place it in this set, even if there is no net change to the attribute’s
value. At flush time, the value of each attribute is compared to its previously saved value, and if there’s
no net change, no SQL operation will occur (this is a more expensive operation so it’s only done at flush
time).
To check if an instance has actionable net changes to its attributes, use the Session.is_modified()
method.
enable_relationship_loading(obj)
Associate an object with this Session for related object loading.
Warning: enable_relationship_loading() exists to serve special use cases and is not
recommended for general use.
Accesses of attributes mapped with relationship() will attempt to load a value from the database
using this Session as the source of connectivity. The values will be loaded based on foreign key
values present on this object - it follows that this functionality generally only works for many-to-onerelationships.
The object will be attached to this session, but will not participate in any persistence operations; its state
for almost all purposes will remain either “transient” or “detached”, except for the case of relationship
loading.
Also note that backrefs will often not work as expected. Altering a relationship-bound attribute on the
target object may not fire off a backref event, if the effective value is what was already loaded from a
foreign-key-holding value.
The
Session.enable_relationship_loading()
method
is
similar
to
the
load_on_pending
flag
on
relationship().
Unlike
that
flag,
Session.enable_relationship_loading() allows an object to remain transient while
still being able to load related items.
To
make
a
transient
object
associated
with
a
Session
via
Session.enable_relationship_loading() pending, add it to the Session using
Session.add() normally.
Session.enable_relationship_loading() does not improve behavior when the ORM is used
normally - object references should be constructed at the object level, not at the foreign key level, so that
they are present in an ordinary way before flush() proceeds. This method is not intended for general use.
166
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
New in version 0.8.
See also:
load_on_pending at relationship() - this flag allows per-relationship loading of many-to-ones
on items that are pending.
execute(clause, params=None, mapper=None, bind=None, **kw)
Execute a SQL expression construct or string statement within the current transaction.
Returns a ResultProxy representing results of the statement execution, in the same manner as that of
an Engine or Connection.
E.g.:
result = session.execute(
user_table.select().where(user_table.c.id == 5)
)
execute() accepts any executable clause construct, such as select(), insert(), update(),
delete(), and text(). Plain SQL strings can be passed as well, which in the case of
Session.execute() only will be interpreted the same as if it were passed via a text() construct.
That is, the following usage:
result = session.execute(
"SELECT * FROM user WHERE id=:param",
{"param":5}
)
is equivalent to:
from sqlalchemy import text
result = session.execute(
text("SELECT * FROM user WHERE id=:param"),
{"param":5}
)
The second positional argument to Session.execute() is an optional parameter set. Similar to that
of Connection.execute(), whether this is passed as a single dictionary, or a list of dictionaries,
determines whether the DBAPI cursor’s execute() or executemany() is used to execute the statement. An INSERT construct may be invoked for a single row:
result = session.execute(users.insert(), {"id": 7, "name": "somename"})
or for multiple rows:
result = session.execute(users.insert(), [
{"id": 7, "name": "somename7"},
{"id": 8, "name": "somename8"},
{"id": 9, "name": "somename9"}
])
The statement is executed within the current transactional context of this Session.
The
Connection which is used to execute the statement can also be acquired directly by calling the
Session.connection() method. Both methods use a rule-based resolution scheme in order to determine the Connection, which in the average case is derived directly from the “bind” of the Session
itself, and in other cases can be based on the mapper() and Table objects passed to the method; see
the documentation for Session.get_bind() for a full description of this scheme.
The Session.execute() method does not invoke autoflush.
2.6. Using the Session
167
SQLAlchemy Documentation, Release 0.8.7
The ResultProxy returned by the Session.execute() method is returned with the
“close_with_result” flag set to true; the significance of this flag is that if this Session is autocommitting and does not have a transaction-dedicated Connection available, a temporary Connection is
established for the statement execution, which is closed (meaning, returned to the connection pool) when
the ResultProxy has consumed all available data. This applies only when the Session is configured
with autocommit=True and no transaction has been started.
Parameters
• clause – An executable statement (i.e. an Executable expression such as
expression.select()) or string SQL statement to be executed.
• params – Optional dictionary, or list of dictionaries, containing bound parameter
values. If a single dictionary, single-row execution occurs; if a list of dictionaries,
an “executemany” will be invoked. The keys in each dictionary must correspond to
parameter names present in the statement.
• mapper – Optional mapper() or mapped class, used to identify the appropriate
bind. This argument takes precedence over clause when locating a bind. See
Session.get_bind() for more details.
• bind – Optional Engine to be used as the bind. If this engine is already involved in
an ongoing transaction, that connection will be used. This argument takes precedence
over mapper and clause when locating a bind.
• **kw – Additional keyword arguments are sent to Session.get_bind() to allow extensibility of “bind” schemes.
See also:
SQL Expression Language Tutorial - Tutorial on using Core SQL constructs.
Working with Engines and Connections - Further information on direct statement execution.
Connection.execute() - core level statement execution method, which is Session.execute()
ultimately uses in order to execute the statement.
expire(instance, attribute_names=None)
Expire the attributes on an instance.
Marks the attributes of an instance as out of date. When an expired attribute is next accessed, a query will
be issued to the Session object’s current transactional context in order to load all expired attributes for
the given instance. Note that a highly isolated transaction will return the same values as were previously
read in that same transaction, regardless of changes in database state outside of that transaction.
To expire all objects in the Session simultaneously, use Session.expire_all().
The Session object’s default behavior is to expire all state whenever the Session.rollback() or
Session.commit() methods are called, so that new state can be loaded for the new transaction. For
this reason, calling Session.expire() only makes sense for the specific case that a non-ORM SQL
statement was emitted in the current transaction.
Parameters
• instance – The instance to be refreshed.
• attribute_names – optional list of string attribute names indicating a subset of
attributes to be expired.
expire_all()
Expires all persistent instances within this Session.
168
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
When any attributes on a persistent instance is next accessed, a query will be issued using the Session
object’s current transactional context in order to load all expired attributes for the given instance. Note that
a highly isolated transaction will return the same values as were previously read in that same transaction,
regardless of changes in database state outside of that transaction.
To expire individual objects and individual attributes on those objects, use Session.expire().
The Session object’s default behavior is to expire all state whenever the Session.rollback()
or Session.commit() methods are called, so that new state can be loaded for the new transaction.
For this reason, calling Session.expire_all() should not be needed when autocommit is False,
assuming the transaction is isolated.
expunge(instance)
Remove the instance from this Session.
This will free all internal references to the instance. Cascading will be applied according to the expunge
cascade rule.
expunge_all()
Remove all object instances from this Session.
This is equivalent to calling expunge(obj) on all objects in this Session.
flush(objects=None)
Flush all the object changes to the database.
Writes out all pending object creations, deletions and modifications to the database as INSERTs,
DELETEs, UPDATEs, etc. Operations are automatically ordered by the Session’s unit of work dependency solver.
Database operations will be issued in the current transactional context and do not affect the state of the
transaction, unless an error occurs, in which case the entire transaction is rolled back. You may flush() as
often as you like within a transaction to move changes from Python to the database’s transaction buffer.
For autocommit Sessions with no active manual transaction, flush() will create a transaction on the fly
that surrounds the entire set of operations into the flush.
Parameters objects – Optional; restricts the flush operation to operate only on elements
that are in the given collection.
This feature is for an extremely narrow set of use cases where particular objects may need
to be operated upon before the full flush() occurs. It is not intended for general use.
get_bind(mapper=None, clause=None)
Return a “bind” to which this Session is bound.
The “bind” is usually an instance of Engine, except in the case where the Session has been explicitly
bound directly to a Connection.
For a multiply-bound or unbound Session, the mapper or clause arguments are used to determine
the appropriate bind to return.
Note that the “mapper” argument is usually present when Session.get_bind() is called via an ORM
operation such as a Session.query(), each individual INSERT/UPDATE/DELETE operation within
a Session.flush(), call, etc.
The order of resolution is:
1.if mapper given and session.binds is present, locate a bind based on mapper.
2.if clause given and session.binds is present, locate a bind based on Table objects found in the given
clause present in session.binds.
3.if session.bind is present, return that.
2.6. Using the Session
169
SQLAlchemy Documentation, Release 0.8.7
4.if clause given, attempt to return a bind linked to the MetaData ultimately associated with the
clause.
5.if mapper given, attempt to return a bind linked to the MetaData ultimately associated with the
Table or other selectable to which the mapper is mapped.
6.No bind can be found, UnboundExecutionError is raised.
Parameters
• mapper – Optional mapper() mapped class or instance of Mapper. The bind can
be derived from a Mapper first by consulting the “binds” map associated with this
Session, and secondly by consulting the MetaData associated with the Table to
which the Mapper is mapped for a bind.
• clause – A ClauseElement (i.e. select(), text(), etc.). If the mapper
argument is not present or could not produce a bind, the given expression construct
will be searched for a bound element, typically a Table associated with bound
MetaData.
identity_key(*args, **kwargs)
inherited from the identity_key() method of _SessionClassMethods
Return an identity key.
This is an alias of util.identity_key().
identity_map = None
A mapping of object identities to objects themselves.
Iterating through Session.identity_map.values() provides access to the full set of persistent
objects (i.e., those that have row identity) currently in the session.
See also:
identity_key() - helper function to produce the keys used in this dictionary.
is_active
True if this Session is in “transaction mode” and is not in “partial rollback” state.
The Session in its default mode of autocommit=False is essentially always in “transaction
mode”, in that a SessionTransaction is associated with it as soon as it is instantiated. This
SessionTransaction is immediately replaced with a new one as soon as it is ended, due to a rollback, commit, or close operation.
“Transaction mode” does not indicate whether or not actual database connection resources are in use; the
SessionTransaction object coordinates among zero or more actual database transactions, and starts
out with none, accumulating individual DBAPI connections as different data sources are used within its
scope. The best way to track when a particular Session has actually begun to use DBAPI resources
is to implement a listener using the SessionEvents.after_begin() method, which will deliver
both the Session as well as the target Connection to a user-defined event listener.
The “partial rollback” state refers to when an “inner” transaction, typically used during a flush, encounters an error and emits a rollback of the DBAPI connection. At this point, the Session is in “partial
rollback” and awaits for the user to call Session.rollback(), in order to close out the transaction
stack. It is in this “partial rollback” period that the is_active flag returns False. After the call to
Session.rollback(), the SessionTransaction is replaced with a new one and is_active
returns True again.
When a Session is used in autocommit=True mode, the SessionTransaction is only instantiated within the scope of a flush call, or when Session.begin() is called. So is_active will
170
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
always be False outside of a flush or Session.begin() block in this mode, and will be True
within the Session.begin() block as long as it doesn’t enter “partial rollback” state.
From all the above, it follows that the only purpose to this flag is for application frameworks that wish
to detect is a “rollback” is necessary within a generic error handling routine, for Session objects that
would otherwise be in “partial rollback” mode. In a typical integration case, this is also not necessary as
it is standard practice to emit Session.rollback() unconditionally within the outermost exception
catch.
To track the transactional state of a Session fully, use event listeners, primarily
the
SessionEvents.after_begin(),
SessionEvents.after_commit(),
SessionEvents.after_rollback() and related events.
is_modified(instance, include_collections=True, passive=True)
Return True if the given instance has locally modified attributes.
This method retrieves the history for each instrumented attribute on the instance and performs a comparison of the current value to its previously committed value, if any.
It is in effect a more expensive and accurate version of checking for the given instance in the
Session.dirty collection; a full test for each attribute’s net “dirty” status is performed.
E.g.:
return session.is_modified(someobject)
Changed in version 0.8: When using SQLAlchemy 0.7 and earlier, the passive flag should always
be explicitly set to True, else SQL loads/autoflushes may proceed which can affect the modified state
itself: session.is_modified(someobject, passive=True). In 0.8 and above, the behavior
is corrected and this flag is ignored.
A few caveats to this method apply:
•Instances present in the Session.dirty collection may report False when tested with this
method. This is because the object may have received change events via attribute mutation, thus
placing it in Session.dirty, but ultimately the state is the same as that loaded from the database,
resulting in no net change here.
•Scalar attributes may not have recorded the previously set value when a new value was applied, if the
attribute was not loaded, or was expired, at the time the new value was received - in these cases, the
attribute is assumed to have a change, even if there is ultimately no net change against its database
value. SQLAlchemy in most cases does not need the “old” value when a set event occurs, so it skips
the expense of a SQL call if the old value isn’t present, based on the assumption that an UPDATE of
the scalar value is usually needed, and in those few cases where it isn’t, is less expensive on average
than issuing a defensive SELECT.
The “old” value is fetched unconditionally upon set only if the attribute container has the
active_history flag set to True. This flag is set typically for primary key attributes and
scalar object references that are not a simple many-to-one. To set this flag for any arbitrary mapped
column, use the active_history argument with column_property().
Parameters
• instance – mapped instance to be tested for pending changes.
• include_collections – Indicates if multivalued collections should be included
in the operation. Setting this to False is a way to detect only local-column based
properties (i.e. scalar columns or many-to-one foreign keys) that would result in an
UPDATE for this instance upon flush.
2.6. Using the Session
171
SQLAlchemy Documentation, Release 0.8.7
• passive – Changed in version 0.8: Ignored for backwards compatibility. When
using SQLAlchemy 0.7 and earlier, this flag should always be set to True.
merge(instance, load=True)
Copy the state of a given instance into a corresponding instance within this Session.
Session.merge() examines the primary key attributes of the source instance, and attempts to reconcile it with an instance of the same primary key in the session. If not found locally, it attempts to load
the object from the database based on primary key, and if none can be located, creates a new instance.
The state of each attribute on the source instance is then copied to the target instance. The resulting target
instance is then returned by the method; the original source instance is left unmodified, and un-associated
with the Session if not already.
This operation cascades to associated instances if the association is mapped with cascade="merge".
See Merging for a detailed discussion of merging.
Parameters
• instance – Instance to be merged.
• load – Boolean, when False, merge() switches into a “high performance” mode
which causes it to forego emitting history events as well as all database access. This
flag is used for cases such as transferring graphs of objects into a Session from a
second level cache, or to transfer just-loaded objects into the Session owned by a
worker thread or process without re-querying the database.
The load=False use case adds the caveat that the given object has to be in a “clean”
state, that is, has no pending changes to be flushed - even if the incoming object is detached from any Session. This is so that when the merge operation populates local
attributes and cascades to related objects and collections, the values can be “stamped”
onto the target object as is, without generating any history or attribute events, and
without the need to reconcile the incoming data with any existing related objects or
collections that might not be loaded. The resulting objects from load=False are
always produced as “clean”, so it is only appropriate that the given objects should be
“clean” as well, else this suggests a mis-use of the method.
new
The set of all instances marked as ‘new’ within this Session.
no_autoflush
Return a context manager that disables autoflush.
e.g.:
with session.no_autoflush:
some_object = SomeClass()
session.add(some_object)
# won't autoflush
some_object.related_thing = session.query(SomeRelated).first()
Operations that proceed within the with: block will not be subject to flushes occurring upon query
access. This is useful when initializing a series of objects which involve existing database queries, where
the uncompleted object should not yet be flushed.
New in version 0.7.6.
object_session(instance)
inherited from the object_session() method of _SessionClassMethods
172
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
Return the Session to which an object belongs.
This is an alias of object_session().
prepare()
Prepare the current transaction in progress for two phase commit.
If no transaction is in progress, this method raises an InvalidRequestError.
Only root transactions of two phase sessions can be prepared. If the current transaction is not such, an
InvalidRequestError is raised.
prune()
Remove unreferenced instances cached in the identity map.
Deprecated since version 0.7: The non-weak-referencing identity map feature is no longer needed.
Note that this method is only meaningful if “weak_identity_map” is set to False. The default weak identity
map is self-pruning.
Removes any object in this Session’s identity map that is not referenced in user code, modified, new or
scheduled for deletion. Returns the number of objects pruned.
query(*entities, **kwargs)
Return a new Query object corresponding to this Session.
refresh(instance, attribute_names=None, lockmode=None)
Expire and refresh the attributes on the given instance.
A query will be issued to the database and all attributes will be refreshed with their current database value.
Lazy-loaded relational attributes will remain lazily loaded, so that the instance-wide refresh operation will
be followed immediately by the lazy load of that attribute.
Eagerly-loaded relational attributes will eagerly load within the single refresh operation.
Note that a highly isolated transaction will return the same values as were previously read in that same
transaction, regardless of changes in database state outside of that transaction - usage of refresh()
usually only makes sense if non-ORM SQL statement were emitted in the ongoing transaction, or if
autocommit mode is turned on.
Parameters
• attribute_names – optional. An iterable collection of string attribute names
indicating a subset of attributes to be refreshed.
• lockmode – Passed to the Query as used by with_lockmode().
rollback()
Rollback the current transaction in progress.
If no transaction is in progress, this method is a pass-through.
This method rolls back the current transaction or nested transaction regardless of subtransactions being
in effect. All subtransactions up to the first real transaction are closed. Subtransactions occur when
begin() is called multiple times.
See also:
Rolling Back
scalar(clause, params=None, mapper=None, bind=None, **kw)
Like execute() but return a scalar result.
transaction = None
The current active or inactive SessionTransaction.
2.6. Using the Session
173
SQLAlchemy Documentation, Release 0.8.7
class sqlalchemy.orm.session.SessionTransaction(session, parent=None, nested=False)
A Session-level transaction.
SessionTransaction is a mostly behind-the-scenes object not normally referenced directly by application
code. It coordinates among multiple Connection objects, maintaining a database transaction for each one
individually, committing or rolling them back all at once. It also provides optional two-phase commit behavior
which can augment this coordination operation.
The Session.transaction attribute of Session refers to the current SessionTransaction object
in use, if any.
A SessionTransaction is associated with a Session in its default mode of autocommit=False
immediately, associated with no database connections. As the Session is called upon to emit SQL on behalf
of various Engine or Connection objects, a corresponding Connection and associated Transaction
is added to a collection within the SessionTransaction object, becoming one of the connection/transaction
pairs maintained by the SessionTransaction.
The lifespan of the SessionTransaction ends when the Session.commit(),
Session.rollback() or Session.close() methods are called.
At this point, the
SessionTransaction removes its association with its parent Session. A Session that is in
autocommit=False mode will create a new SessionTransaction to replace it immediately, whereas
a Session that’s in autocommit=True mode will remain without a SessionTransaction until the
Session.begin() method is called.
Another detail of SessionTransaction behavior is that it is capable of “nesting”. This means that the
Session.begin() method can be called while an existing SessionTransaction is already present,
producing a new SessionTransaction that temporarily replaces the parent SessionTransaction.
When a SessionTransaction is produced as nested, it assigns itself to the Session.transaction
attribute. When it is ended via Session.commit() or Session.rollback(), it restores its parent
SessionTransaction back onto the Session.transaction attribute. The behavior is effectively a
stack, where Session.transaction refers to the current head of the stack.
The purpose of this stack is to allow nesting of Session.rollback() or Session.commit()
calls in context with various flavors of Session.begin(). This nesting behavior applies to when
Session.begin_nested() is used to emit a SAVEPOINT transaction, and is also used to produce a socalled “subtransaction” which allows a block of code to use a begin/rollback/commit sequence regardless of
whether or not its enclosing code block has begun a transaction. The flush() method, whether called explicitly or via autoflush, is the primary consumer of the “subtransaction” feature, in that it wishes to guarantee that
it works within in a transaction block regardless of whether or not the Session is in transactional mode when
the method is called.
See also:
Session.rollback()
Session.commit()
Session.begin()
Session.begin_nested()
Session.is_active
SessionEvents.after_commit()
SessionEvents.after_rollback()
SessionEvents.after_soft_rollback()
174
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
Session Utilites
sqlalchemy.orm.session.make_transient(instance)
Make the given instance ‘transient’.
This will remove its association with any session and additionally will remove its “identity key”, such that it’s
as though the object were newly constructed, except retaining its values. It also resets the “deleted” flag on the
state if this object had been explicitly deleted by its session.
Attributes which were “expired” or deferred at the instance level are reverted to undefined, and will not trigger
any loads.
sqlalchemy.orm.session.object_session(instance)
Return the Session to which instance belongs.
If the instance is not a mapped instance, an error is raised.
Attribute and State Management Utilities
These functions are provided by the SQLAlchemy attribute instrumentation API to provide a detailed interface for
dealing with instances, attribute values, and history. Some of them are useful when constructing event listener functions, such as those described in ORM Events.
sqlalchemy.orm.util.object_state(instance)
Given an object, return the InstanceState associated with the object.
Raises sqlalchemy.orm.exc.UnmappedInstanceError if no mapping is configured.
Equivalent functionality is available via the inspect() function as:
inspect(instance)
Using the inspection system will raise sqlalchemy.exc.NoInspectionAvailable if the instance is
not part of a mapping.
sqlalchemy.orm.attributes.del_attribute(instance, key)
Delete the value of an attribute, firing history events.
This function may be used regardless of instrumentation applied directly to the class, i.e. no descriptors are
required. Custom attribute management schemes will need to make usage of this method to establish attribute
state as understood by SQLAlchemy.
sqlalchemy.orm.attributes.get_attribute(instance, key)
Get the value of an attribute, firing any callables required.
This function may be used regardless of instrumentation applied directly to the class, i.e. no descriptors are
required. Custom attribute management schemes will need to make usage of this method to make usage of
attribute state as understood by SQLAlchemy.
sqlalchemy.orm.attributes.get_history(obj, key, passive=<symbol ‘PASSIVE_OFF>)
Return a History record for the given object and attribute key.
Parameters
• obj – an object whose class is instrumented by the attributes package.
• key – string attribute name.
• passive – indicates loading behavior for the attribute if the value is not already present.
This is a bitflag attribute, which defaults to the symbol PASSIVE_OFF indicating all
necessary SQL should be emitted.
2.6. Using the Session
175
SQLAlchemy Documentation, Release 0.8.7
sqlalchemy.orm.attributes.init_collection(obj, key)
Initialize a collection attribute and return the collection adapter.
This function is used to provide direct access to collection internals for a previously unloaded attribute. e.g.:
collection_adapter = init_collection(someobject, 'elements')
for elem in values:
collection_adapter.append_without_event(elem)
For an easier way to do the above, see set_committed_value().
obj is an instrumented object instance. An InstanceState is accepted directly for backwards compatibility but
this usage is deprecated.
sqlalchemy.orm.attributes.flag_modified(instance, key)
Mark an attribute on an instance as ‘modified’.
This sets the ‘modified’ flag on the instance and establishes an unconditional change event for the given attribute.
sqlalchemy.orm.attributes.instance_state()
Return the InstanceState for a given mapped object.
This function is the internal version of object_state(). The object_state() and/or the inspect()
function is preferred here as they each emit an informative exception if the given object is not mapped.
sqlalchemy.orm.instrumentation.is_instrumented(instance, key)
Return True if the given attribute on the given instance is instrumented by the attributes package.
This function may be used regardless of instrumentation applied directly to the class, i.e. no descriptors are
required.
sqlalchemy.orm.attributes.set_attribute(instance, key, value)
Set the value of an attribute, firing history events.
This function may be used regardless of instrumentation applied directly to the class, i.e. no descriptors are
required. Custom attribute management schemes will need to make usage of this method to establish attribute
state as understood by SQLAlchemy.
sqlalchemy.orm.attributes.set_committed_value(instance, key, value)
Set the value of an attribute with no history events.
Cancels any previous history present. The value should be a scalar value for scalar-holding attributes, or an
iterable for any collection-holding attribute.
This is the same underlying method used when a lazy loader fires off and loads additional data from the database.
In particular, this method can be used by application code which has loaded additional attributes or collections
through separate queries, which can then be attached to an instance as though it were part of its original loaded
state.
class sqlalchemy.orm.attributes.History
Bases: sqlalchemy.orm.attributes.History
A 3-tuple of added, unchanged and deleted values, representing the changes which have occurred on an instrumented attribute.
The easiest way to get a History object for a particular attribute on an object is to use the inspect()
function:
from sqlalchemy import inspect
hist = inspect(myobject).attrs.myattribute.history
Each tuple member is an iterable sequence:
176
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
•added - the collection of items added to the attribute (the first tuple element).
•unchanged - the collection of items that have not changed on the attribute (the second tuple element).
•deleted - the collection of items that have been removed from the attribute (the third tuple element).
empty()
Return True if this History has no changes and no existing, unchanged state.
has_changes()
Return True if this History has changes.
non_added()
Return a collection of unchanged + deleted.
non_deleted()
Return a collection of added + unchanged.
sum()
Return a collection of added + unchanged + deleted.
2.7 Querying
This section provides API documentation for the Query object and related constructs.
For an in-depth introduction to querying with the SQLAlchemy ORM, please see the Object Relational Tutorial.
2.7.1 The Query Object
Query is produced in terms of a given Session, using the query() function:
q = session.query(SomeMappedClass)
Following is the full interface for the Query object.
class sqlalchemy.orm.query.Query(entities, session=None)
ORM-level SQL construction object.
Query is the source of all SELECT statements generated by the ORM, both those formulated by end-user query
operations as well as by high level internal operations such as related collection loading. It features a generative
interface whereby successive calls return a new Query object, a copy of the former with additional criteria and
options associated with it.
Query objects are normally initially generated using the query() method of Session. For a full walkthrough of Query usage, see the Object Relational Tutorial.
add_column(column)
Add a column expression to the list of result columns to be returned.
Pending deprecation: add_column() will be superseded by add_columns().
add_columns(*column)
Add one or more column expressions to the list of result columns to be returned.
add_entity(entity, alias=None)
add a mapped entity to the list of result columns to be returned.
2.7. Querying
177
SQLAlchemy Documentation, Release 0.8.7
all()
Return the results represented by this Query as a list.
This results in an execution of the underlying query.
as_scalar()
Return the full SELECT statement represented by this Query, converted to a scalar subquery.
Analogous to sqlalchemy.sql.expression.SelectBase.as_scalar().
New in version 0.6.5.
autoflush(setting)
Return a Query with a specific ‘autoflush’ setting.
Note that a Session with autoflush=False will not autoflush, even if this flag is set to True at the Query
level. Therefore this flag is usually used only to disable autoflush for a specific Query.
column_descriptions
Return metadata about the columns which would be returned by this Query.
Format is a list of dictionaries:
user_alias = aliased(User, name='user2')
q = sess.query(User, User.id, user_alias)
# this expression:
q.column_descriptions
# would return:
[
{
'name':'User',
'type':User,
'aliased':False,
'expr':User,
},
{
'name':'id',
'type':Integer(),
'aliased':False,
'expr':User.id,
},
{
'name':'user2',
'type':User,
'aliased':True,
'expr':user_alias
}
]
correlate(*args)
Return a Query construct which will correlate the given FROM clauses to that of an enclosing Query
or select().
The method here accepts mapped classes, aliased() constructs, and mapper() constructs as arguments, which are resolved into expression constructs, in addition to appropriate expression constructs.
The correlation arguments are ultimately passed to Select.correlate() after coercion to expression
constructs.
178
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
The correlation arguments take effect in such cases as when Query.from_self() is used, or when a
subquery as returned by Query.subquery() is embedded in another select() construct.
count()
Return a count of rows this Query would return.
This generates the SQL for this Query as follows:
SELECT count(1) AS count_1 FROM (
SELECT <rest of query follows...>
) AS anon_1
Changed in version 0.7: The above scheme is newly refined as of 0.7b3.
For fine grained control over specific columns to count, to skip the usage of a subquery or otherwise
control of the FROM clause, or to use other aggregate functions, use func expressions in conjunction
with query(), i.e.:
from sqlalchemy import func
# count User records, without
# using a subquery.
session.query(func.count(User.id))
# return count of user "id" grouped
# by "name"
session.query(func.count(User.id)).\
group_by(User.name)
from sqlalchemy import distinct
# count distinct "name" values
session.query(func.count(distinct(User.name)))
cte(name=None, recursive=False)
Return the full SELECT statement represented by this Query represented as a common table expression
(CTE).
New in version 0.7.6.
Parameters and usage are the same as those of the SelectBase.cte() method; see that method for
further details.
Here is the Postgresql WITH RECURSIVE example. Note that, in this example, the included_parts
cte and the incl_alias alias of it are Core selectables, which means the columns are accessed via
the .c. attribute. The parts_alias object is an orm.aliased() instance of the Part entity, so
column-mapped attributes are available directly:
from sqlalchemy.orm import aliased
class Part(Base):
__tablename__ = 'part'
part = Column(String, primary_key=True)
sub_part = Column(String, primary_key=True)
quantity = Column(Integer)
included_parts = session.query(
Part.sub_part,
Part.part,
Part.quantity).\
filter(Part.part=="our part").\
2.7. Querying
179
SQLAlchemy Documentation, Release 0.8.7
cte(name="included_parts", recursive=True)
incl_alias = aliased(included_parts, name="pr")
parts_alias = aliased(Part, name="p")
included_parts = included_parts.union_all(
session.query(
parts_alias.sub_part,
parts_alias.part,
parts_alias.quantity).\
filter(parts_alias.part==incl_alias.c.sub_part)
)
q = session.query(
included_parts.c.sub_part,
func.sum(included_parts.c.quantity).
label('total_quantity')
).\
group_by(included_parts.c.sub_part)
See also:
SelectBase.cte()
delete(synchronize_session=’evaluate’)
Perform a bulk delete query.
Deletes rows matched by this query from the database.
Parameters synchronize_session – chooses the strategy for the removal of matched
objects from the session. Valid values are:
False - don’t synchronize the session. This option is the most efficient and is reliable
once the session is expired, which typically occurs after a commit(), or explicitly using
expire_all(). Before the expiration, objects may still remain in the session which were in
fact deleted which can lead to confusing results if they are accessed via get() or already
loaded collections.
’fetch’ - performs a select query before the delete to find objects that are matched by
the delete query and need to be removed from the session. Matched objects are removed
from the session.
’evaluate’ - Evaluate the query’s criteria in Python straight on the objects in the session. If evaluation of the criteria isn’t implemented, an error is raised. In that case you
probably want to use the ‘fetch’ strategy as a fallback.
The expression evaluator currently doesn’t account for differing string collations between
the database and Python.
Returns the count of rows matched as returned by the database’s “row count” feature.
This method has several key caveats:
•The method does not offer in-Python cascading of relationships - it is assumed that ON DELETE
CASCADE/SET NULL/etc. is configured for any foreign key references which require it, otherwise
the database may emit an integrity violation if foreign key references are being enforced.
After the DELETE, dependent objects in the Session which were impacted by an ON DELETE
may not contain the current state, or may have been deleted. This issue is resolved once the
Session is expired, which normally occurs upon Session.commit() or can be forced by
using Session.expire_all(). Accessing an expired object whose row has been deleted will
180
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
invoke a SELECT to locate the row; when the row is not found, an ObjectDeletedError is
raised.
•The MapperEvents.before_delete() and MapperEvents.after_delete() events
are not invoked from this method. Instead, the SessionEvents.after_bulk_delete()
method is provided to act upon a mass DELETE of entity rows.
See also:
Query.update()
Inserts, Updates and Deletes - Core SQL tutorial
distinct(*criterion)
Apply a DISTINCT to the query and return the newly resulting Query.
Parameters *expr – optional column expressions. When present, the Postgresql dialect will
render a DISTINCT ON (<expressions>>) construct.
enable_assertions(value)
Control whether assertions are generated.
When set to False, the returned Query will not assert its state before certain operations, including that
LIMIT/OFFSET has not been applied when filter() is called, no criterion exists when get() is called, and
no “from_statement()” exists when filter()/order_by()/group_by() etc. is called. This more permissive
mode is used by custom Query subclasses to specify criterion or other modifiers outside of the usual
usage patterns.
Care should be taken to ensure that the usage pattern is even possible. A statement applied by
from_statement() will override any criterion set by filter() or order_by(), for example.
enable_eagerloads(value)
Control whether or not eager joins and subqueries are rendered.
When set to False, the returned Query will not render eager joins regardless of joinedload(),
subqueryload() options or mapper-level lazy=’joined’/lazy=’subquery’ configurations.
This is used primarily when nesting the Query’s statement into a subquery or other selectable.
except_(*q)
Produce an EXCEPT of this Query against one or more queries.
Works the same way as union(). See that method for usage examples.
except_all(*q)
Produce an EXCEPT ALL of this Query against one or more queries.
Works the same way as union(). See that method for usage examples.
execution_options(**kwargs)
Set non-SQL options which take effect during execution.
The options are the same as those accepted by Connection.execution_options().
Note that the stream_results execution option is enabled automatically if the yield_per()
method is used.
exists()
A convenience method that turns a query into an EXISTS subquery of the form EXISTS (SELECT 1
FROM ... WHERE ...).
e.g.:
2.7. Querying
181
SQLAlchemy Documentation, Release 0.8.7
q = session.query(User).filter(User.name == 'fred')
session.query(q.exists())
Producing SQL similar to:
SELECT EXISTS (
SELECT 1 FROM users WHERE users.name = :name_1
) AS anon_1
New in version 0.8.1.
filter(*criterion)
apply the given filtering criterion to a copy of this Query, using SQL expressions.
e.g.:
session.query(MyClass).filter(MyClass.name == 'some name')
Multiple criteria may be specified as comma separated; the effect is that they will be joined together using
the and_() function:
session.query(MyClass).\
filter(MyClass.name == 'some name', MyClass.id > 5)
The criterion is any SQL expression object applicable to the WHERE clause of a select. String expressions
are coerced into SQL expression constructs via the text() construct.
See also:
Query.filter_by() - filter on keyword expressions.
filter_by(**kwargs)
apply the given filtering criterion to a copy of this Query, using keyword expressions.
e.g.:
session.query(MyClass).filter_by(name = 'some name')
Multiple criteria may be specified as comma separated; the effect is that they will be joined together using
the and_() function:
session.query(MyClass).\
filter_by(name = 'some name', id = 5)
The keyword expressions are extracted from the primary entity of the query, or the last entity that was the
target of a call to Query.join().
See also:
Query.filter() - filter on SQL expressions.
first()
Return the first result of this Query or None if the result doesn’t contain any row.
first() applies a limit of one within the generated SQL, so that only one primary entity row is generated
on the server side (note this may consist of multiple result rows if join-loaded collections are present).
Calling first() results in an execution of the underlying query.
from_self(*entities)
return a Query that selects from this Query’s SELECT statement.
*entities - optional list of entities which will replace those being selected.
182
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
from_statement(statement)
Execute the given SELECT statement and return results.
This method bypasses all internal statement compilation, and the statement is executed without modification.
The statement argument is either a string, a select() construct, or a text() construct, and should
return the set of columns appropriate to the entity class represented by this Query.
get(ident)
Return an instance based on the given primary key identifier, or None if not found.
E.g.:
my_user = session.query(User).get(5)
some_object = session.query(VersionedFoo).get((5, 10))
get() is special in that it provides direct access to the identity map of the owning Session. If the
given primary key identifier is present in the local identity map, the object is returned directly from this
collection and no SQL is emitted, unless the object has been marked fully expired. If not present, a
SELECT is performed in order to locate the object.
get() also will perform a check if the object is present in the identity map and marked as expired
- a SELECT is emitted to refresh the object as well as to ensure that the row is still present. If not,
ObjectDeletedError is raised.
get() is only used to return a single mapped instance, not multiple instances or individual column
constructs, and strictly on a single primary key value. The originating Query must be constructed in
this way, i.e. against a single mapped entity, with no additional filtering criterion. Loading options via
options() may be applied however, and will be used if the object is not yet locally present.
A lazy-loading, many-to-one attribute configured by relationship(), using a simple foreign-keyto-primary-key criterion, will also use an operation equivalent to get() in order to retrieve the target
value from the local identity map before querying the database. See Relationship Loading Techniques for
further details on relationship loading.
Parameters ident – A scalar or tuple value representing the primary key. For a composite
primary key, the order of identifiers corresponds in most cases to that of the mapped
Table object’s primary key columns. For a mapper() that was given the primary
key argument during construction, the order of identifiers corresponds to the elements
present in this collection.
Returns The object instance, or None.
group_by(*criterion)
apply one or more GROUP BY criterion to the query and return the newly resulting Query
having(criterion)
apply a HAVING criterion to the query and return the newly resulting Query.
having() is used in conjunction with group_by().
HAVING criterion makes it possible to use filters on aggregate functions like COUNT, SUM, AVG, MAX,
and MIN, eg.:
q = session.query(User.id).\
join(User.addresses).\
group_by(User.id).\
having(func.count(Address.id) > 2)
2.7. Querying
183
SQLAlchemy Documentation, Release 0.8.7
instances(cursor, _Query__context=None)
Given a ResultProxy cursor as returned by connection.execute(), return an ORM result as an iterator.
e.g.:
result = engine.execute("select * from users")
for u in session.query(User).instances(result):
print u
intersect(*q)
Produce an INTERSECT of this Query against one or more queries.
Works the same way as union(). See that method for usage examples.
intersect_all(*q)
Produce an INTERSECT ALL of this Query against one or more queries.
Works the same way as union(). See that method for usage examples.
join(*props, **kwargs)
Create a SQL JOIN against this Query object’s criterion and apply generatively, returning the newly
resulting Query.
Simple Relationship Joins
Consider a mapping between two classes User and Address, with a relationship User.addresses
representing a collection of Address objects associated with each User. The most common usage
of join() is to create a JOIN along this relationship, using the User.addresses attribute as an
indicator for how this should occur:
q = session.query(User).join(User.addresses)
Where above, the call to join() along User.addresses will result in SQL equivalent to:
SELECT user.* FROM user JOIN address ON user.id = address.user_id
In the above example we refer to User.addresses as passed to join() as the on clause, that is,
it indicates how the “ON” portion of the JOIN should be constructed. For a single-entity query such as
the one above (i.e. we start by selecting only from User and nothing else), the relationship can also be
specified by its string name:
q = session.query(User).join("addresses")
join() can also accommodate multiple “on clause” arguments to produce a chain of joins, such as below
where a join across four related entities is constructed:
q = session.query(User).join("orders", "items", "keywords")
The above would be shorthand for three separate calls to join(), each using an explicit attribute to
indicate the source entity:
q = session.query(User).\
join(User.orders).\
join(Order.items).\
join(Item.keywords)
Joins to a Target Entity or Selectable
A second form of join() allows any mapped entity or core selectable construct as a target. In this usage,
join() will attempt to create a JOIN along the natural foreign key relationship between two entities:
184
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
q = session.query(User).join(Address)
The above calling form of join() will raise an error if either there are no foreign keys between the two
entities, or if there are multiple foreign key linkages between them. In the above calling form, join()
is called upon to create the “on clause” automatically for us. The target can be any mapped entity or
selectable, such as a Table:
q = session.query(User).join(addresses_table)
Joins to a Target with an ON Clause
The third calling form allows both the target entity as well as the ON clause to be passed explicitly.
Suppose for example we wanted to join to Address twice, using an alias the second time. We use
aliased() to create a distinct alias of Address, and join to it using the target, onclause form,
so that the alias can be specified explicitly as the target along with the relationship to instruct how the ON
clause should proceed:
a_alias = aliased(Address)
q = session.query(User).\
join(User.addresses).\
join(a_alias, User.addresses).\
filter(Address.email_address=='[email protected]').\
filter(a_alias.email_address=='[email protected]')
Where above, the generated SQL would be similar to:
SELECT user.* FROM user
JOIN address ON user.id = address.user_id
JOIN address AS address_1 ON user.id=address_1.user_id
WHERE address.email_address = :email_address_1
AND address_1.email_address = :email_address_2
The two-argument calling form of join() also allows us to construct arbitrary joins with SQL-oriented
“on clause” expressions, not relying upon configured relationships at all. Any SQL expression can be
passed as the ON clause when using the two-argument form, which should refer to the target entity in
some way as well as an applicable source entity:
q = session.query(User).join(Address, User.id==Address.user_id)
Changed in version 0.7: In SQLAlchemy 0.6 and earlier, the two argument form of join() requires the
usage of a tuple: query(User).join((Address, User.id==Address.user_id)). This
calling form is accepted in 0.7 and further, though is not necessary unless multiple join conditions are
passed to a single join() call, which itself is also not generally necessary as it is now equivalent to
multiple calls (this wasn’t always the case).
Advanced Join Targeting and Adaption
There is a lot of flexibility in what the “target” can be when using join(). As noted previously, it also
accepts Table constructs and other selectables such as alias() and select() constructs, with either
the one or two-argument forms:
addresses_q = select([Address.user_id]).\
where(Address.email_address.endswith("@bar.com")).\
alias()
q = session.query(User).\
join(addresses_q, addresses_q.c.user_id==User.id)
2.7. Querying
185
SQLAlchemy Documentation, Release 0.8.7
join() also features the ability to adapt a relationship() -driven ON clause to the target selectable. Below we construct a JOIN from User to a subquery against Address, allowing the relationship denoted by User.addresses to adapt itself to the altered target:
address_subq = session.query(Address).\
filter(Address.email_address == '[email protected]').\
subquery()
q = session.query(User).join(address_subq, User.addresses)
Producing SQL similar to:
SELECT user.* FROM user
JOIN (
SELECT address.id AS id,
address.user_id AS user_id,
address.email_address AS email_address
FROM address
WHERE address.email_address = :email_address_1
) AS anon_1 ON user.id = anon_1.user_id
The above form allows one to fall back onto an explicit ON clause at any time:
q = session.query(User).\
join(address_subq, User.id==address_subq.c.user_id)
Controlling what to Join From
While join() exclusively deals with the “right” side of the JOIN, we can also control the “left” side, in
those cases where it’s needed, using select_from(). Below we construct a query against Address
but can still make usage of User.addresses as our ON clause by instructing the Query to select first
from the User entity:
q = session.query(Address).select_from(User).\
join(User.addresses).\
filter(User.name == 'ed')
Which will produce SQL similar to:
SELECT address.* FROM user
JOIN address ON user.id=address.user_id
WHERE user.name = :name_1
Constructing Aliases Anonymously
join() can construct anonymous aliases using the aliased=True flag. This feature is useful when a
query is being joined algorithmically, such as when querying self-referentially to an arbitrary depth:
q = session.query(Node).\
join("children", "children", aliased=True)
When aliased=True is used, the actual “alias” construct is not explicitly available. To work with it,
methods such as Query.filter() will adapt the incoming entity to the last join point:
q = session.query(Node).\
join("children", "children", aliased=True).\
filter(Node.name == 'grandchild 1')
When using automatic aliasing, the from_joinpoint=True argument can allow a multi-node join to
be broken into multiple calls to join(), so that each path along the way can be further filtered:
186
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
q = session.query(Node).\
join("children", aliased=True).\
filter(Node.name='child 1').\
join("children", aliased=True, from_joinpoint=True).\
filter(Node.name == 'grandchild 1')
The filtering aliases above can then be reset back to the original Node entity using
reset_joinpoint():
q = session.query(Node).\
join("children", "children", aliased=True).\
filter(Node.name == 'grandchild 1').\
reset_joinpoint().\
filter(Node.name == 'parent 1)
For an example of aliased=True, see the distribution example XML Persistence which illustrates an
XPath-like query system using algorithmic joins.
Parameters
• *props – A collection of one or more join conditions, each consisting of a
relationship-bound attribute or string relationship name representing an “on clause”,
or a single target entity, or a tuple in the form of (target, onclause). A special
two-argument calling form of the form target, onclause is also accepted.
• aliased=False – If True, indicate that the JOIN target should be anonymously
aliased. Subsequent calls to filter and similar will adapt the incoming criterion to
the target alias, until reset_joinpoint() is called.
• from_joinpoint=False – When using aliased=True, a setting of True here
will cause the join to be from the most recent joined target, rather than starting back
from the original FROM clauses of the query.
See also:
Querying with Joins in the ORM tutorial.
Mapping Class Inheritance Hierarchies for details on how join() is used for inheritance relationships.
orm.join() - a standalone ORM-level join function, used internally by Query.join(), which in
previous SQLAlchemy versions was the primary ORM-level joining interface.
label(name)
Return the full SELECT statement represented by this Query, converted to a scalar subquery with a label
of the given name.
Analogous to sqlalchemy.sql.expression.SelectBase.label().
New in version 0.6.5.
limit(limit)
Apply a LIMIT to the query and return the newly resulting
Query.
merge_result(iterator, load=True)
Merge a result into this Query object’s Session.
Given an iterator returned by a Query of the same structure as this one, return an identical iterator
of results, with all mapped instances merged into the session using Session.merge(). This is an
optimized method which will merge all mapped instances, preserving the structure of the result rows and
unmapped columns with less method overhead than that of calling Session.merge() explicitly for
each value.
2.7. Querying
187
SQLAlchemy Documentation, Release 0.8.7
The structure of the results is determined based on the column list of this Query - if these do not correspond, unchecked errors will occur.
The ‘load’ argument is the same as that of Session.merge().
For an example of how merge_result() is used, see the source code for the example Dogpile
Caching, where merge_result() is used to efficiently restore state from a cache back into a target
Session.
offset(offset)
Apply an OFFSET to the query and return the newly resulting Query.
one()
Return exactly one result or raise an exception.
Raises sqlalchemy.orm.exc.NoResultFound if the query selects no rows.
Raises
sqlalchemy.orm.exc.MultipleResultsFound if multiple object identities are returned, or if
multiple rows are returned for a query that does not return object identities.
Note that an entity query, that is, one which selects one or more mapped classes as opposed to individual
column attributes, may ultimately represent many rows but only one row of unique entity or entities - this
is a successful result for one().
Calling one() results in an execution of the underlying query.
Changed in version 0.6: one() fully fetches all results instead of applying any kind of limit, so that the
“unique”-ing of entities does not conceal multiple object identities.
options(*args)
Return a new Query object, applying the given list of mapper options.
Most supplied options regard changing how column- and relationship-mapped attributes are loaded. See
the sections Deferred Column Loading and Relationship Loading Techniques for reference documentation.
order_by(*criterion)
apply one or more ORDER BY criterion to the query and return the newly resulting Query
All existing ORDER BY settings can be suppressed by passing None - this will suppress any ORDER
BY configured on mappers as well.
Alternatively, an existing ORDER BY setting on the Query object can be entirely cancelled by passing
False as the value - use this before calling methods where an ORDER BY is invalid.
outerjoin(*props, **kwargs)
Create a left outer join against this Query object’s criterion and apply generatively, returning the newly
resulting Query.
Usage is the same as the join() method.
params(*args, **kwargs)
add values for bind parameters which may have been specified in filter().
parameters may be specified using **kwargs, or optionally a single dictionary as the first positional argument. The reason for both is that **kwargs is convenient, however some parameter dictionaries contain
unicode keys in which case **kwargs cannot be used.
populate_existing()
Return a Query that will expire and refresh all instances as they are loaded, or reused from the current
Session.
188
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
populate_existing() does not improve behavior when the ORM is used normally - the Session
object’s usual behavior of maintaining a transaction and expiring all attributes after rollback or commit
handles object state automatically. This method is not intended for general use.
prefix_with(*prefixes)
Apply the prefixes to the query and return the newly resulting Query.
Parameters *prefixes – optional prefixes, typically strings, not using any commas. In
particular is useful for MySQL keywords.
e.g.:
query = sess.query(User.name).\
prefix_with('HIGH_PRIORITY').\
prefix_with('SQL_SMALL_RESULT', 'ALL')
Would render:
SELECT HIGH_PRIORITY SQL_SMALL_RESULT ALL users.name AS users_name
FROM users
New in version 0.7.7.
reset_joinpoint()
Return a new Query, where the “join point” has been reset back to the base FROM entities of the query.
This method is usually used in conjunction with the aliased=True feature of the join() method.
See the example in join() for how this is used.
scalar()
Return the first element of the first result or None if no rows present. If multiple rows are returned, raises
MultipleResultsFound.
>>> session.query(Item).scalar()
<Item>
>>> session.query(Item.id).scalar()
1
>>> session.query(Item.id).filter(Item.id < 0).scalar()
None
>>> session.query(Item.id, Item.name).scalar()
1
>>> session.query(func.count(Parent.id)).scalar()
20
This results in an execution of the underlying query.
select_entity_from(from_obj)
Set the FROM clause of this Query to a core selectable, applying it as a replacement FROM clause for
corresponding mapped entities.
This method is currently equivalent to the Query.select_from() method, but in 0.9 these two methods will diverge in functionality.
In addition to changing the FROM list, the method will also apply the given selectable to replace the
FROM which the selected entities would normally select from.
The given from_obj must be an instance of a FromClause, e.g. a select() or Alias construct.
An
example
would
be
a
Query
that
selects
User
entities,
but
uses
Query.select_entity_from() to have the entities selected from a select() construct
instead of the base user table:
2.7. Querying
189
SQLAlchemy Documentation, Release 0.8.7
select_stmt = select([User]).where(User.id == 7)
q = session.query(User).\
select_entity_from(select_stmt).\
filter(User.name == 'ed')
The query generated will select User entities directly from the given select() construct, and will be:
SELECT anon_1.id AS anon_1_id, anon_1.name AS anon_1_name
FROM (SELECT "user".id AS id, "user".name AS name
FROM "user"
WHERE "user".id = :id_1) AS anon_1
WHERE anon_1.name = :name_1
Notice above that even the WHERE criterion was “adapted” such that the anon_1 subquery effectively
replaces all references to the user table, except for the one that it refers to internally.
Compare this to Query.select_from(), which as of version 0.9, does not affect existing entities.
The statement below:
q = session.query(User).\
select_from(select_stmt).\
filter(User.name == 'ed')
Produces SQL where both the user table as well as the select_stmt construct are present as separate
elements in the FROM clause. No “adaptation” of the user table is applied:
SELECT "user".id AS user_id, "user".name AS user_name
FROM "user", (SELECT "user".id AS id, "user".name AS name
FROM "user"
WHERE "user".id = :id_1) AS anon_1
WHERE "user".name = :name_1
Query.select_entity_from() maintains an older behavior of Query.select_from(). In
modern usage, similar results can also be achieved using aliased():
select_stmt = select([User]).where(User.id == 7)
user_from_select = aliased(User, select_stmt.alias())
q = session.query(user_from_select)
Parameters from_obj – a FromClause object that will replace the FROM clause of this
Query.
See also:
Query.select_from()
New in version 0.8.2: Query.select_entity_from() was added to specify the specific behavior
of entity replacement, however the Query.select_from() maintains this behavior as well until 0.9.
select_from(*from_obj)
Set the FROM clause of this Query explicitly.
Query.select_from() is often used in conjunction with Query.join() in order to control which
entity is selected from on the “left” side of the join.
The entity or selectable object here effectively replaces the “left edge” of any calls to join(), when no
joinpoint is otherwise established - usually, the default “join point” is the leftmost entity in the Query
object’s list of entities to be selected.
190
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
A typical example:
q = session.query(Address).select_from(User).\
join(User.addresses).\
filter(User.name == 'ed')
Which produces SQL equivalent to:
SELECT address.* FROM user
JOIN address ON user.id=address.user_id
WHERE user.name = :name_1
Parameters *from_obj – collection of one or more entities to apply to the FROM clause.
Entities can be mapped classes, AliasedClass objects, Mapper objects as well as
core FromClause elements like subqueries.
Note:
Query.select_from() features a deprecated behavior whereby when passed a
FromClause element, such as a select construct, it will apply that select construct to replace the FROM
clause that an existing entity is joined from. This behavior is being removed in SQLAlchemy 0.9, to be
replaced with the Query.select_entity_from() method. Applications which rely on this behavior to re-base query entities to an arbitrary selectable should transition to this method before upgrading to
0.9.
See also:
join()
Query.select_entity_from()
selectable
Return the Select object emitted by this Query.
Used for inspect() compatibility, this is equivalent to:
query.enable_eagerloads(False).with_labels().statement
slice(start, stop)
apply LIMIT/OFFSET to the Query based on a ” “range and return the newly resulting Query.
statement
The full SELECT statement represented by this Query.
The statement by default will not have disambiguating labels applied to the construct unless
with_labels(True) is called first.
subquery(name=None, with_labels=False, reduce_columns=False)
return the full SELECT statement represented by this Query, embedded within an Alias.
Eager JOIN generation within the query is disabled.
Parameters
• name – string name to be assigned as the alias; this is passed through to
FromClause.alias(). If None, a name will be deterministically generated at
compile time.
• with_labels – if True, with_labels() will be called on the Query first to
apply table-qualified labels to all columns.
2.7. Querying
191
SQLAlchemy Documentation, Release 0.8.7
• reduce_columns – if True, Select.reduce_columns() will be called on
the resulting select() construct, to remove same-named columns where one also
refers to the other via foreign key or WHERE clause equivalence.
Changed in version 0.8: the with_labels and reduce_columns keyword arguments were added.
union(*q)
Produce a UNION of this Query against one or more queries.
e.g.:
q1 = sess.query(SomeClass).filter(SomeClass.foo=='bar')
q2 = sess.query(SomeClass).filter(SomeClass.bar=='foo')
q3 = q1.union(q2)
The method accepts multiple Query objects so as to control the level of nesting. A series of union()
calls such as:
x.union(y).union(z).all()
will nest on each union(), and produces:
SELECT * FROM (SELECT * FROM (SELECT * FROM X UNION
SELECT * FROM y) UNION SELECT * FROM Z)
Whereas:
x.union(y, z).all()
produces:
SELECT * FROM (SELECT * FROM X UNION SELECT * FROM y UNION
SELECT * FROM Z)
Note that many database backends do not allow ORDER BY to be rendered on a query called within
UNION, EXCEPT, etc. To disable all ORDER BY clauses including those configured on mappers, issue query.order_by(None) - the resulting Query object will not render ORDER BY within its
SELECT statement.
union_all(*q)
Produce a UNION ALL of this Query against one or more queries.
Works the same way as union(). See that method for usage examples.
update(values, synchronize_session=’evaluate’)
Perform a bulk update query.
Updates rows matched by this query in the database.
Parameters
• values – a dictionary with attributes names as keys and literal values or sql expressions as values.
• synchronize_session – chooses the strategy to update the attributes on objects
in the session. Valid values are:
False - don’t synchronize the session. This option is the most efficient and is reliable
once the session is expired, which typically occurs after a commit(), or explicitly using
expire_all(). Before the expiration, updated objects may still remain in the session
with stale values on their attributes, which can lead to confusing results.
192
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
’fetch’ - performs a select query before the update to find objects that are matched
by the update query. The updated attributes are expired on matched objects.
’evaluate’ - Evaluate the Query’s criteria in Python straight on the objects in the
session. If evaluation of the criteria isn’t implemented, an exception is raised.
The expression evaluator currently doesn’t account for differing string collations between the database and Python.
Returns the count of rows matched as returned by the database’s “row count” feature.
This method has several key caveats:
•The method does not offer in-Python cascading of relationships - it is assumed that ON UPDATE
CASCADE is configured for any foreign key references which require it, otherwise the database
may emit an integrity violation if foreign key references are being enforced.
After the UPDATE, dependent objects in the Session which were impacted by an ON UPDATE CASCADE may not contain the current state; this issue is resolved once the Session
is expired, which normally occurs upon Session.commit() or can be forced by using
Session.expire_all().
•As of 0.8, this method will support multiple table updates, as detailed in Multiple Table Updates, and
this behavior does extend to support updates of joined-inheritance and other multiple table mappings.
However, the join condition of an inheritance mapper is currently not automatically rendered.
Care must be taken in any multiple-table update to explicitly include the joining condition between
those tables, even in mappings where this is normally automatic. E.g. if a class Engineer subclasses Employee, an UPDATE of the Engineer local table using criteria against the Employee
local table might look like:
session.query(Engineer).\
filter(Engineer.id == Employee.id).\
filter(Employee.name == 'dilbert').\
update({"engineer_type": "programmer"})
•The MapperEvents.before_update() and MapperEvents.after_update() events
are not invoked from this method. Instead, the SessionEvents.after_bulk_update()
method is provided to act upon a mass UPDATE of entity rows.
See also:
Query.delete()
Inserts, Updates and Deletes - Core SQL tutorial
value(column)
Return a scalar result corresponding to the given column expression.
values(*columns)
Return an iterator yielding result tuples corresponding to the given list of columns
whereclause
A readonly attribute which returns the current WHERE criterion for this Query.
This returned value is a SQL expression construct, or None if no criterion has been established.
with_entities(*entities)
Return a new Query replacing the SELECT list with the given entities.
e.g.:
2.7. Querying
193
SQLAlchemy Documentation, Release 0.8.7
# Users, filtered on some arbitrary criterion
# and then ordered by related email address
q = session.query(User).\
join(User.address).\
filter(User.name.like('%ed%')).\
order_by(Address.email)
# given *only* User.id==5, Address.email, and 'q', what
# would the *next* User in the result be ?
subq = q.with_entities(Address.email).\
order_by(None).\
filter(User.id==5).\
subquery()
q = q.join((subq, subq.c.email < Address.email)).\
limit(1)
New in version 0.6.5.
with_hint(selectable, text, dialect_name=’*’)
Add an indexing hint for the given entity or selectable to this Query.
Functionality is passed straight through to with_hint(), with the addition that selectable can be
a Table, Alias, or ORM entity / mapped class /etc.
with_labels()
Apply column labels to the return value of Query.statement.
Indicates that this Query’s statement accessor should return a SELECT statement that applies labels to all
columns in the form <tablename>_<columnname>; this is commonly used to disambiguate columns from
multiple tables which have the same name.
When the Query actually issues SQL to load rows, it always uses column labeling.
with_lockmode(mode)
Return a new Query object with the specified locking mode.
Parameters mode – a string representing the desired locking mode. A corresponding value is
passed to the for_update parameter of select() when the query is executed. Valid
values are:
’update’ - passes for_update=True, which translates to FOR UPDATE (standard
SQL, supported by most dialects)
’update_nowait’ - passes for_update=’nowait’, which translates to FOR
UPDATE NOWAIT (supported by Oracle, PostgreSQL 8.1 upwards)
’read’ - passes for_update=’read’, which translates to LOCK IN SHARE
MODE (for MySQL), and FOR SHARE (for PostgreSQL)
’read_nowait’ - passes for_update=’read_nowait’, which translates to FOR
SHARE NOWAIT (supported by PostgreSQL).
New in version 0.7.7: FOR SHARE and FOR SHARE NOWAIT (PostgreSQL).
with_parent(instance, property=None)
Add filtering criterion that relates the given instance to a child object or collection, using its attribute state
as well as an established relationship() configuration.
The method uses the with_parent() function to generate the clause, the result of which is passed to
Query.filter().
194
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
Parameters are the same as with_parent(), with the exception that the given property can be None,
in which case a search is performed against this Query object’s target mapper.
with_polymorphic(cls_or_mappers, selectable=None, polymorphic_on=None)
Load columns for inheriting classes.
Query.with_polymorphic() applies transformations to the “main” mapped class represented by
this Query. The “main” mapped class here means the Query object’s first argument is a full class, i.e.
session.query(SomeClass). These transformations allow additional tables to be present in the
FROM clause so that columns for a joined-inheritance subclass are available in the query, both for the
purposes of load-time efficiency as well as the ability to use these columns at query time.
See the documentation section Basic Control of Which Tables are Queried for details on how this method
is used.
Changed in version 0.8: A new and more flexible function orm.with_polymorphic() supersedes
Query.with_polymorphic(), as it can apply the equivalent functionality to any set of columns or
classes in the Query, not just the “zero mapper”. See that function for a description of arguments.
with_session(session)
Return a Query that will use the given Session.
with_transformation(fn)
Return a new Query object transformed by the given function.
E.g.:
def filter_something(criterion):
def transform(q):
return q.filter(criterion)
return transform
q = q.with_transformation(filter_something(x==5))
This allows ad-hoc recipes to be created for Query objects. See the example at Building Transformers.
New in version 0.7.4.
yield_per(count)
Yield only count rows at a time.
WARNING: use this method with caution; if the same instance is present in more than one batch of rows,
end-user changes to attributes will be overwritten.
In particular, it’s usually impossible to use this setting with eagerly loaded collections (i.e. any
lazy=’joined’ or ‘subquery’) since those collections will be cleared for a new load when encountered
in a subsequent result batch. In the case of ‘subquery’ loading, the full result for all rows is fetched which
generally defeats the purpose of yield_per().
Also note that while yield_per() will set the stream_results execution option to True, currently
this is only understood by psycopg2 dialect which will stream results using server side cursors instead
of pre-buffer all rows for this query. Other DBAPIs pre-buffer all rows before making them available.
2.7.2 ORM-Specific Query Constructs
sqlalchemy.orm.aliased(element, alias=None, name=None, adapt_on_names=False)
Produce an alias of the given element, usually an AliasedClass instance.
E.g.:
2.7. Querying
195
SQLAlchemy Documentation, Release 0.8.7
my_alias = aliased(MyClass)
session.query(MyClass, my_alias).filter(MyClass.id > my_alias.id)
The aliased() function is used to create an ad-hoc mapping of a mapped class to a new selectable.
By default, a selectable is generated from the normally mapped selectable (typically a Table) using the
FromClause.alias() method. However, aliased() can also be used to link the class to a new
select() statement. Also, the with_polymorphic() function is a variant of aliased() that is
intended to specify a so-called “polymorphic selectable”, that corresponds to the union of several joinedinheritance subclasses at once.
For convenience, the aliased() function also accepts plain FromClause constructs, such as a Table or
select() construct. In those cases, the FromClause.alias() method is called on the object and the new
Alias object returned. The returned Alias is not ORM-mapped in this case.
Parameters
• element – element to be aliased. Is normally a mapped class, but for convenience can
also be a FromClause element.
• alias – Optional selectable unit to map the element to. This should normally be
a Alias object corresponding to the Table to which the class is mapped, or to a
select() construct that is compatible with the mapping. By default, a simple anonymous alias of the mapped table is generated.
• name – optional string name to use for the alias, if not specified by the alias parameter.
The name, among other things, forms the attribute name that will be accessible via tuples
returned by a Query object.
• adapt_on_names – if True, more liberal “matching” will be used when mapping the
mapped columns of the ORM entity to those of the given selectable - a name-based match
will be performed if the given selectable doesn’t otherwise have a column that corresponds to one on the entity. The use case for this is when associating an entity with some
derived selectable such as one that uses aggregate functions:
class UnitPrice(Base):
__tablename__ = 'unit_price'
...
unit_id = Column(Integer)
price = Column(Numeric)
aggregated_unit_price = Session.query(
func.sum(UnitPrice.price).label('price')
).group_by(UnitPrice.unit_id).subquery()
aggregated_unit_price = aliased(UnitPrice,
alias=aggregated_unit_price, adapt_on_names=True)
Above, functions on aggregated_unit_price which refer to .price will return
the fund.sum(UnitPrice.price).label(’price’) column, as it is matched
on the name “price”. Ordinarily, the “price” function wouldn’t have any “column correspondence” to the actual UnitPrice.price column as it is not a proxy of the original.
New in version 0.7.3.
class sqlalchemy.orm.util.AliasedClass(cls, alias=None, name=None, adapt_on_names=False,
with_polymorphic_mappers=(),
with_polymorphic_discriminator=None,
base_alias=None, use_mapper_path=False)
196
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
Represents an “aliased” form of a mapped class for usage with Query.
The ORM equivalent of a sqlalchemy.sql.expression.alias() construct, this object mimics the
mapped class using a __getattr__ scheme and maintains a reference to a real Alias object.
Usage is via the orm.aliased() function, or alternatively via the orm.with_polymorphic() function.
Usage example:
# find all pairs of users with the same name
user_alias = aliased(User)
session.query(User, user_alias).\
join((user_alias, User.id > user_alias.id)).\
filter(User.name==user_alias.name)
The resulting object is an instance of AliasedClass. This object implements an attribute scheme which
produces the same attribute and method interface as the original mapped class, allowing AliasedClass to
be compatible with any attribute technique which works on the original class, including hybrid attributes (see
Hybrid Attributes).
The AliasedClass can be inspected for its underlying Mapper, aliased selectable, and other information
using inspect():
from sqlalchemy import inspect
my_alias = aliased(MyClass)
insp = inspect(my_alias)
The resulting inspection object is an instance of AliasedInsp.
See aliased() and with_polymorphic() for construction argument descriptions.
class sqlalchemy.orm.util.AliasedInsp(entity,
mapper,
selectable,
name,
with_polymorphic_mappers,
polymorphic_on,
_base_alias, _use_mapper_path)
Bases: sqlalchemy.orm.interfaces._InspectionAttr
Provide an inspection interface for an AliasedClass object.
The AliasedInsp object is returned given an AliasedClass using the inspect() function:
from sqlalchemy import inspect
from sqlalchemy.orm import aliased
my_alias = aliased(MyMappedClass)
insp = inspect(my_alias)
Attributes on AliasedInsp include:
•entity - the AliasedClass represented.
•mapper - the Mapper mapping the underlying class.
•selectable - the Alias construct which ultimately represents an aliased Table or Select construct.
•name - the name of the alias. Also is used as the attribute name when returned in a result tuple from
Query.
•with_polymorphic_mappers - collection of Mapper objects indicating all those mappers expressed in the select construct for the AliasedClass.
•polymorphic_on - an alternate column or SQL expression which will be used as the “discriminator”
for a polymorphic load.
2.7. Querying
197
SQLAlchemy Documentation, Release 0.8.7
See also:
Runtime Inspection API
class sqlalchemy.util.KeyedTuple
Bases: __builtin__.tuple
tuple subclass that adds labeled names.
E.g.:
>>> k = KeyedTuple([1, 2, 3], labels=["one", "two", "three"])
>>> k.one
1
>>> k.two
2
Result rows returned by Query that contain multiple ORM entities and/or column expressions make use of this
class to return rows.
The KeyedTuple exhibits similar behavior to the collections.namedtuple() construct provided in
the Python standard library, however is architected very differently. Unlike collections.namedtuple(),
KeyedTuple is does not rely on creation of custom subtypes in order to represent a new series of
keys, instead each KeyedTuple instance receives its list of keys in place. The subtype approach of
collections.namedtuple() introduces significant complexity and performance overhead, which is not
necessary for the Query object’s use case.
Changed in version 0.8: Compatibility methods with collections.namedtuple() have been added including KeyedTuple._fields and KeyedTuple._asdict().
See also:
Querying
_asdict()
Return the contents of this KeyedTuple as a dictionary.
This method provides compatibility with collections.namedtuple(), with the exception that the
dictionary returned is not ordered.
New in version 0.8.
_fields
Return a tuple of string key names for this KeyedTuple.
This method provides compatibility with collections.namedtuple().
New in version 0.8.
See also:
KeyedTuple.keys()
keys()
Return a list of string key names for this KeyedTuple.
See also:
KeyedTuple._fields
sqlalchemy.orm.join(left, right, onclause=None, isouter=False, join_to_left=None)
Produce an inner join between left and right clauses.
orm.join() is an extension to the core join interface provided by sql.expression.join(), where the
left and right selectables may be not only core selectable objects such as Table, but also mapped classes or
198
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
AliasedClass instances. The “on” clause can be a SQL expression, or an attribute or string name referencing
a configured relationship().
orm.join() is not commonly needed in modern usage, as its functionality is encapsulated within that of the
Query.join() method, which features a significant amount of automation beyond orm.join() by itself.
Explicit usage of orm.join() with Query involves usage of the Query.select_from() method, as in:
from sqlalchemy.orm import join
session.query(User).\
select_from(join(User, Address, User.addresses)).\
filter(Address.email_address=='[email protected]')
In modern SQLAlchemy the above join can be written more succinctly as:
session.query(User).\
join(User.addresses).\
filter(Address.email_address=='[email protected]')
See Query.join() for information on modern usage of ORM level joins.
Changed in version 0.8.1: - the join_to_left parameter is no longer used, and is deprecated.
sqlalchemy.orm.outerjoin(left, right, onclause=None, join_to_left=None)
Produce a left outer join between left and right clauses.
This is the “outer join” version of the orm.join() function, featuring the same behavior except that an
OUTER JOIN is generated. See that function’s documentation for other usage details.
sqlalchemy.orm.with_parent(instance, prop)
Create filtering criterion that relates this query’s primary entity to the given related instance, using established
relationship() configuration.
The SQL rendered is the same as that rendered when a lazy loader would fire off from the given parent on that
attribute, meaning that the appropriate state is taken from the parent object in Python without the need to render
joins to the parent table in the rendered statement.
Changed in version 0.6.4: This method accepts parent instances in all persistence states, including transient,
persistent, and detached. Only the requisite primary key/foreign key attributes need to be populated. Previous
versions didn’t work with transient instances.
Parameters
• instance – An instance which has some relationship().
• property – String property name, or class-bound attribute, which indicates what relationship from the instance should be used to reconcile the parent/child relationship.
2.8 Relationship Loading Techniques
A big part of SQLAlchemy is providing a wide range of control over how related objects get loaded when querying.
This behavior can be configured at mapper construction time using the lazy parameter to the relationship()
function, as well as by using options with the Query object.
2.8.1 Using Loader Strategies: Lazy Loading, Eager Loading
By default, all inter-object relationships are lazy loading. The scalar or collection attribute associated with a
relationship() contains a trigger which fires the first time the attribute is accessed. This trigger, in all but
one case, issues a SQL call at the point of access in order to load the related object or objects:
2.8. Relationship Loading Techniques
199
SQLAlchemy Documentation, Release 0.8.7
>>> jack.addresses
SELECT addresses.id AS addresses_id, addresses.email_address AS addresses_email_address,
addresses.user_id AS addresses_user_id
FROM addresses
WHERE ? = addresses.user_id
[5]
[<Address(u'[email protected]')>, <Address(u'[email protected]')>]
The one case where SQL is not emitted is for a simple many-to-one relationship, when the related object can be
identified by its primary key alone and that object is already present in the current Session.
This default behavior of “load upon attribute access” is known as “lazy” or “select” loading - the name “select” because
a “SELECT” statement is typically emitted when the attribute is first accessed.
In the Object Relational Tutorial, we introduced the concept of Eager Loading. We used an option in conjunction
with the Query object in order to indicate that a relationship should be loaded at the same time as the parent, within
a single SQL query. This option, known as joinedload(), connects a JOIN (by default a LEFT OUTER join) to
the statement and populates the scalar/collection from the same result set as that of the parent:
>>> jack = session.query(User).\
... options(joinedload('addresses')).\
... filter_by(name='jack').all() #doctest: +NORMALIZE_WHITESPACE
SELECT addresses_1.id AS addresses_1_id, addresses_1.email_address AS addresses_1_email_address,
addresses_1.user_id AS addresses_1_user_id, users.id AS users_id, users.name AS users_name,
users.fullname AS users_fullname, users.password AS users_password
FROM users LEFT OUTER JOIN addresses AS addresses_1 ON users.id = addresses_1.user_id
WHERE users.name = ?
['jack']
In addition to “joined eager loading”, a second option for eager loading exists, called “subquery eager loading”. This
kind of eager loading emits an additional SQL statement for each collection requested, aggregated across all parent
objects:
>>> jack = session.query(User).\
... options(subqueryload('addresses')).\
... filter_by(name='jack').all()
SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname,
users.password AS users_password
FROM users
WHERE users.name = ?
('jack',)
SELECT addresses.id AS addresses_id, addresses.email_address AS addresses_email_address,
addresses.user_id AS addresses_user_id, anon_1.users_id AS anon_1_users_id
FROM (SELECT users.id AS users_id
FROM users
WHERE users.name = ?) AS anon_1 JOIN addresses ON anon_1.users_id = addresses.user_id
ORDER BY anon_1.users_id, addresses.id
('jack',)
The default loader strategy for any relationship() is configured by the lazy keyword argument, which defaults to select - this indicates a “select” statement . Below we set it as joined so that the children relationship
is eager loading, using a join:
# load the 'children' collection using LEFT OUTER JOIN
mapper(Parent, parent_table, properties={
200
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
'children': relationship(Child, lazy='joined')
})
We can also set it to eagerly load using a second query for all collections, using subquery:
# load the 'children' attribute using a join to a subquery
mapper(Parent, parent_table, properties={
'children': relationship(Child, lazy='subquery')
})
When querying, all three choices of loader strategy are available on a per-query basis, using the joinedload(),
subqueryload() and lazyload() query options:
# set children to load lazily
session.query(Parent).options(lazyload('children')).all()
# set children to load eagerly with a join
session.query(Parent).options(joinedload('children')).all()
# set children to load eagerly with a second statement
session.query(Parent).options(subqueryload('children')).all()
To reference a relationship that is deeper than one level, separate the names by periods:
session.query(Parent).options(joinedload('foo.bar.bat')).all()
When using dot-separated names with joinedload() or subqueryload(), the option applies only to the actual
attribute named, and not its ancestors. For example, suppose a mapping from A to B to C, where the relationships,
named atob and btoc, are both lazy-loading. A statement like the following:
session.query(A).options(joinedload('atob.btoc')).all()
will load only A objects to start. When the atob attribute on each A is accessed, the returned B objects will eagerly
load their C objects.
Therefore, to modify the eager load to load both atob as well as btoc, place joinedloads for both:
session.query(A).options(joinedload('atob'), joinedload('atob.btoc')).all()
or more succinctly just use joinedload_all() or subqueryload_all():
session.query(A).options(joinedload_all('atob.btoc')).all()
There are two other loader strategies available, dynamic loading and no loading; these are described in Working with
Large Collections.
2.8.2 Default Loading Strategies
New in version 0.7.5: Default loader strategies as a new feature.
2.8. Relationship Loading Techniques
201
SQLAlchemy Documentation, Release 0.8.7
Each of joinedload(), subqueryload(), lazyload(), and noload() can be used to set the default style
of relationship() loading for a particular query, affecting all relationship() -mapped attributes not otherwise specified in the Query. This feature is available by passing the string ’*’ as the argument to any of these
options:
session.query(MyClass).options(lazyload('*'))
Above, the lazyload(’*’) option will supersede the lazy setting of all relationship() constructs in
use for that query, except for those which use the ’dynamic’ style of loading. If some relationships specify
lazy=’joined’ or lazy=’subquery’, for example, using lazyload(’*’) will unilaterally cause all those
relationships to use ’select’ loading, e.g. emit a SELECT statement when each attribute is accessed.
The option does not supersede loader options stated in the query, such as eagerload(), subqueryload(), etc.
The query below will still use joined loading for the widget relationship:
session.query(MyClass).options(
lazyload('*'),
joinedload(MyClass.widget)
)
If multiple ’*’ options are passed, the last one overrides those previously passed.
2.8.3 The Zen of Eager Loading
The philosophy behind loader strategies is that any set of loading schemes can be applied to a particular query, and
the results don’t change - only the number of SQL statements required to fully load related objects and collections
changes. A particular query might start out using all lazy loads. After using it in context, it might be revealed that
particular attributes or collections are always accessed, and that it would be more efficient to change the loader strategy
for these. The strategy can be changed with no other modifications to the query, the results will remain identical, but
fewer SQL statements would be emitted. In theory (and pretty much in practice), nothing you can do to the Query
would make it load a different set of primary or related objects based on a change in loader strategy.
How joinedload() in particular achieves this result of not impacting entity rows returned in any way is that
it creates an anonymous alias of the joins it adds to your query, so that they can’t be referenced by other parts of
the query. For example, the query below uses joinedload() to create a LEFT OUTER JOIN from users to
addresses, however the ORDER BY added against Address.email_address is not valid - the Address
entity is not named in the query:
>>> jack = session.query(User).\
... options(joinedload(User.addresses)).\
... filter(User.name=='jack').\
... order_by(Address.email_address).all()
SELECT addresses_1.id AS addresses_1_id, addresses_1.email_address AS addresses_1_email_address,
addresses_1.user_id AS addresses_1_user_id, users.id AS users_id, users.name AS users_name,
users.fullname AS users_fullname, users.password AS users_password
FROM users LEFT OUTER JOIN addresses AS addresses_1 ON users.id = addresses_1.user_id
WHERE users.name = ? ORDER BY addresses.email_address
<-- this part is wrong !
['jack']
Above, ORDER BY addresses.email_address is not valid since addresses is not in the FROM list. The
correct way to load the User records and order by email address is to use Query.join():
202
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
>>>
...
...
...
jack = session.query(User).\
join(User.addresses).\
filter(User.name=='jack').\
order_by(Address.email_address).all()
SELECT users.id AS users_id, users.name AS users_name,
users.fullname AS users_fullname, users.password AS users_password
FROM users JOIN addresses ON users.id = addresses.user_id
WHERE users.name = ? ORDER BY addresses.email_address
['jack']
The statement above is of course not the same as the previous one, in that the columns from addresses are not
included in the result at all. We can add joinedload() back in, so that there are two joins - one is that which we
are ordering on, the other is used anonymously to load the contents of the User.addresses collection:
>>> jack = session.query(User).\
... join(User.addresses).\
... options(joinedload(User.addresses)).\
... filter(User.name=='jack').\
... order_by(Address.email_address).all()
SELECT addresses_1.id AS addresses_1_id, addresses_1.email_address AS addresses_1_email_address,
addresses_1.user_id AS addresses_1_user_id, users.id AS users_id, users.name AS users_name,
users.fullname AS users_fullname, users.password AS users_password
FROM users JOIN addresses ON users.id = addresses.user_id
LEFT OUTER JOIN addresses AS addresses_1 ON users.id = addresses_1.user_id
WHERE users.name = ? ORDER BY addresses.email_address
['jack']
What we see above is that our usage of Query.join() is to supply JOIN clauses we’d like to use in subsequent query criterion, whereas our usage of joinedload() only concerns itself with the loading of the
User.addresses collection, for each User in the result. In this case, the two joins most probably appear redundant - which they are. If we wanted to use just one JOIN for collection loading as well as ordering, we use
the contains_eager() option, described in Routing Explicit Joins/Statements into Eagerly Loaded Collections
below. But to see why joinedload() does what it does, consider if we were filtering on a particular Address:
>>> jack = session.query(User).\
... join(User.addresses).\
... options(joinedload(User.addresses)).\
... filter(User.name=='jack').\
... filter(Address.email_address=='[email protected]').\
... all()
SELECT addresses_1.id AS addresses_1_id, addresses_1.email_address AS addresses_1_email_address,
addresses_1.user_id AS addresses_1_user_id, users.id AS users_id, users.name AS users_name,
users.fullname AS users_fullname, users.password AS users_password
FROM users JOIN addresses ON users.id = addresses.user_id
LEFT OUTER JOIN addresses AS addresses_1 ON users.id = addresses_1.user_id
WHERE users.name = ? AND addresses.email_address = ?
['jack', '[email protected]']
Above, we can see that the two JOINs have very different roles. One will match exactly one row, that of the join
of User and Address where Address.email_address==’[email protected]’. The other LEFT
OUTER JOIN will match all Address rows related to User, and is only used to populate the User.addresses
collection, for those User objects that are returned.
By changing the usage of joinedload() to another style of loading, we can change how the collection is loaded
completely independently of SQL used to retrieve the actual User rows we want. Below we change joinedload()
2.8. Relationship Loading Techniques
203
SQLAlchemy Documentation, Release 0.8.7
into subqueryload():
>>> jack = session.query(User).\
... join(User.addresses).\
... options(subqueryload(User.addresses)).\
... filter(User.name=='jack').\
... filter(Address.email_address=='[email protected]').\
... all()
SELECT users.id AS users_id, users.name AS users_name,
users.fullname AS users_fullname, users.password AS users_password
FROM users JOIN addresses ON users.id = addresses.user_id
WHERE users.name = ? AND addresses.email_address = ?
['jack', '[email protected]']
# ... subqueryload() emits a SELECT in order
# to load all address records ...
When using joined eager loading, if the query contains a modifier that impacts the rows returned externally to the
joins, such as when using DISTINCT, LIMIT, OFFSET or equivalent, the completed statement is first wrapped inside
a subquery, and the joins used specifically for joined eager loading are applied to the subquery. SQLAlchemy’s joined
eager loading goes the extra mile, and then ten miles further, to absolutely ensure that it does not affect the end result
of the query, only the way collections and related objects are loaded, no matter what the format of the query is.
2.8.4 What Kind of Loading to Use ?
Which type of loading to use typically comes down to optimizing the tradeoff between number of SQL executions,
complexity of SQL emitted, and amount of data fetched. Lets take two examples, a relationship() which
references a collection, and a relationship() that references a scalar many-to-one reference.
• One to Many Collection
• When using the default lazy loading, if you load 100 objects, and then access a collection on each of them, a total
of 101 SQL statements will be emitted, although each statement will typically be a simple SELECT without any
joins.
• When using joined loading, the load of 100 objects and their collections will emit only one SQL statement.
However, the total number of rows fetched will be equal to the sum of the size of all the collections, plus one
extra row for each parent object that has an empty collection. Each row will also contain the full set of columns
represented by the parents, repeated for each collection item - SQLAlchemy does not re-fetch these columns
other than those of the primary key, however most DBAPIs (with some exceptions) will transmit the full data
of each parent over the wire to the client connection in any case. Therefore joined eager loading only makes
sense when the size of the collections are relatively small. The LEFT OUTER JOIN can also be performance
intensive compared to an INNER join.
• When using subquery loading, the load of 100 objects will emit two SQL statements. The second statement
will fetch a total number of rows equal to the sum of the size of all collections. An INNER JOIN is used, and
a minimum of parent columns are requested, only the primary keys. So a subquery load makes sense when the
collections are larger.
• When multiple levels of depth are used with joined or subquery loading, loading collections-within- collections
will multiply the total number of rows fetched in a cartesian fashion. Both forms of eager loading always join
from the original parent class.
• Many to One Reference
• When using the default lazy loading, a load of 100 objects will like in the case of the collection emit as many
as 101 SQL statements. However - there is a significant exception to this, in that if the many-to-one reference
204
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
is a simple foreign key reference to the target’s primary key, each reference will be checked first in the current
identity map using Query.get(). So here, if the collection of objects references a relatively small set of
target objects, or the full set of possible target objects have already been loaded into the session and are strongly
referenced, using the default of lazy=’select’ is by far the most efficient way to go.
• When using joined loading, the load of 100 objects will emit only one SQL statement. The join will be a LEFT
OUTER JOIN, and the total number of rows will be equal to 100 in all cases. If you know that each parent
definitely has a child (i.e. the foreign key reference is NOT NULL), the joined load can be configured with
innerjoin set to True, which is usually specified within the relationship(). For a load of objects
where there are many possible target references which may have not been loaded already, joined loading with
an INNER JOIN is extremely efficient.
• Subquery loading will issue a second load for all the child objects, so for a load of 100 objects there would be
two SQL statements emitted. There’s probably not much advantage here over joined loading, however, except
perhaps that subquery loading can use an INNER JOIN in all cases whereas joined loading requires that the
foreign key is NOT NULL.
2.8.5 Routing Explicit Joins/Statements into Eagerly Loaded Collections
The behavior of joinedload() is such that joins are created automatically, using anonymous aliases as targets, the
results of which are routed into collections and scalar references on loaded objects. It is often the case that a query
already includes the necessary joins which represent a particular collection or scalar reference, and the joins added by
the joinedload feature are redundant - yet you’d still like the collections/references to be populated.
For this SQLAlchemy supplies the contains_eager() option. This option is used in the same manner as the
joinedload() option except it is assumed that the Query will specify the appropriate joins explicitly. Below it’s
used with a from_statement load:
# mapping is the users->addresses mapping
mapper(User, users_table, properties={
'addresses': relationship(Address, addresses_table)
})
# define a query on USERS with an outer join to ADDRESSES
statement = users_table.outerjoin(addresses_table).select().apply_labels()
# construct a Query object which expects the "addresses" results
query = session.query(User).options(contains_eager('addresses'))
# get results normally
r = query.from_statement(statement)
It works just as well with an inline Query.join() or Query.outerjoin():
session.query(User).outerjoin(User.addresses).options(contains_eager(User.addresses)).all()
If the “eager” portion of the statement is “aliased”, the alias keyword argument to contains_eager() may be
used to indicate it. This is a string alias name or reference to an actual Alias (or other selectable) object:
# use an alias of the Address entity
adalias = aliased(Address)
# construct a Query object which expects the "addresses" results
query = session.query(User).\
outerjoin(adalias, User.addresses).\
2.8. Relationship Loading Techniques
205
SQLAlchemy Documentation, Release 0.8.7
options(contains_eager(User.addresses, alias=adalias))
# get results normally
r = query.all()
SELECT users.user_id AS users_user_id, users.user_name AS users_user_name, adalias.address_id AS adal
adalias.user_id AS adalias_user_id, adalias.email_address AS adalias_email_address, (...other columns
FROM users LEFT OUTER JOIN email_addresses AS email_addresses_1 ON users.user_id = email_addresses_1.
The alias argument is used only as a source of columns to match up to the result set. You can use it to match up
the result to arbitrary label names in a string SQL statement, by passing a select() which links those labels to the
mapped Table:
# label the columns of the addresses table
eager_columns = select([
addresses.c.address_id.label('a1'),
addresses.c.email_address.label('a2'),
addresses.c.user_id.label('a3')])
# select from a raw SQL statement which uses those label names for the
# addresses table. contains_eager() matches them up.
query = session.query(User).\
from_statement("select users.*, addresses.address_id as a1, "
"addresses.email_address as a2, addresses.user_id as a3 "
"from users left outer join addresses on users.user_id=addresses.user_id").\
options(contains_eager(User.addresses, alias=eager_columns))
The path given as the argument to contains_eager() needs to be a full path from the starting entity. For example
if we were loading Users->orders->Order->items->Item, the string version would look like:
query(User).options(contains_eager('orders', 'items'))
Or using the class-bound descriptor:
query(User).options(contains_eager(User.orders, Order.items))
2.8.6 Relation Loader API
sqlalchemy.orm.contains_alias(alias)
Return a MapperOption that will indicate to the query that the main table has been aliased.
This is used in the very rare case that contains_eager() is being used in conjunction with a user-defined
SELECT statement that aliases the parent table. E.g.:
# define an aliased UNION called 'ulist'
statement = users.select(users.c.user_id==7).\
union(users.select(users.c.user_id>7)).\
alias('ulist')
# add on an eager load of "addresses"
statement = statement.outerjoin(addresses).\
select().apply_labels()
# create query, indicating "ulist" will be an
# alias for the main table, "addresses"
206
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
# property should be eager loaded
query = session.query(User).options(
contains_alias('ulist'),
contains_eager('addresses'))
# then get results via the statement
results = query.from_statement(statement).all()
Parameters alias – is the string name of an alias, or a Alias object representing the alias.
sqlalchemy.orm.contains_eager(*keys, **kwargs)
Return a MapperOption that will indicate to the query that the given attribute should be eagerly loaded from
columns currently in the query.
Used with options().
The option is used in conjunction with an explicit join that loads the desired rows, i.e.:
sess.query(Order).\
join(Order.user).\
options(contains_eager(Order.user))
The above query would join from the Order entity to its related User entity, and the returned Order objects
would have the Order.user attribute pre-populated.
contains_eager() also accepts an alias argument, which is the string name of an alias, an alias()
construct, or an aliased() construct. Use this when the eagerly-loaded rows are to come from an aliased
table:
user_alias = aliased(User)
sess.query(Order).\
join((user_alias, Order.user)).\
options(contains_eager(Order.user, alias=user_alias))
See also eagerload() for the “automatic” version of this functionality.
For additional examples of contains_eager() see Routing Explicit Joins/Statements into Eagerly Loaded
Collections.
sqlalchemy.orm.eagerload(*args, **kwargs)
A synonym for joinedload().
sqlalchemy.orm.eagerload_all(*args, **kwargs)
A synonym for joinedload_all()
sqlalchemy.orm.immediateload(*keys)
Return a MapperOption that will convert the property of the given name or series of mapped attributes into
an immediate load.
The “immediate” load means the attribute will be fetched with a separate SELECT statement per parent in the
same way as lazy loading - except the loader is guaranteed to be called at load time before the parent object is
returned in the result.
The normal behavior of lazy loading applies - if the relationship is a simple many-to-one, and the child object is
already present in the Session, no SELECT statement will be emitted.
Used with options().
See also: lazyload(), eagerload(), subqueryload()
New in version 0.6.5.
2.8. Relationship Loading Techniques
207
SQLAlchemy Documentation, Release 0.8.7
sqlalchemy.orm.joinedload(*keys, **kw)
Return a MapperOption that will convert the property of the given name or series of mapped attributes into
an joined eager load.
Changed in version 0.6beta3: This function is known as eagerload() in all versions of SQLAlchemy prior
to version 0.6beta3, including the 0.5 and 0.4 series. eagerload() will remain available for the foreseeable
future in order to enable cross-compatibility.
Used with options().
examples:
# joined-load the "orders" collection on "User"
query(User).options(joinedload(User.orders))
# joined-load the "keywords" collection on each "Item",
# but not the "items" collection on "Order" - those
# remain lazily loaded.
query(Order).options(joinedload(Order.items, Item.keywords))
# to joined-load across both, use joinedload_all()
query(Order).options(joinedload_all(Order.items, Item.keywords))
# set the default strategy to be 'joined'
query(Order).options(joinedload('*'))
joinedload() also accepts a keyword argument innerjoin=True which indicates using an inner join instead
of an outer:
query(Order).options(joinedload(Order.user, innerjoin=True))
Note: The join created by joinedload() is anonymously aliased such that it does not affect the query
results. An Query.order_by() or Query.filter() call cannot reference these aliased tables - socalled “user space” joins are constructed using Query.join(). The rationale for this is that joinedload()
is only applied in order to affect how related objects or collections are loaded as an optimizing detail - it can
be added or removed with no impact on actual results. See the section The Zen of Eager Loading for a detailed
description of how this is used, including how to use a single explicit JOIN for filtering/ordering and eager
loading simultaneously.
See also: subqueryload(), lazyload()
sqlalchemy.orm.joinedload_all(*keys, **kw)
Return a MapperOption that will convert all properties along the given dot-separated path or series of mapped
attributes into an joined eager load.
Changed in version 0.6beta3: This function is known as eagerload_all() in all versions of SQLAlchemy
prior to version 0.6beta3, including the 0.5 and 0.4 series. eagerload_all() will remain available for the
foreseeable future in order to enable cross-compatibility.
Used with options().
For example:
query.options(joinedload_all('orders.items.keywords'))...
will set all of orders, orders.items, and orders.items.keywords to load in one joined eager load.
Individual descriptors are accepted as arguments as well:
208
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
query.options(joinedload_all(User.orders, Order.items, Item.keywords))
The keyword arguments accept a flag innerjoin=True|False which will override the value of the innerjoin flag
specified on the relationship().
See also: subqueryload_all(), lazyload()
sqlalchemy.orm.lazyload(*keys)
Return a MapperOption that will convert the property of the given name or series of mapped attributes into a
lazy load.
Used with options().
See also: eagerload(), subqueryload(), immediateload()
sqlalchemy.orm.noload(*keys)
Return a MapperOption that will convert the property of the given name or series of mapped attributes into a
non-load.
Used with options().
See also: lazyload(), eagerload(), subqueryload(), immediateload()
sqlalchemy.orm.subqueryload(*keys)
Return a MapperOption that will convert the property of the given name or series of mapped attributes into
an subquery eager load.
Used with options().
examples:
# subquery-load the "orders" collection on "User"
query(User).options(subqueryload(User.orders))
# subquery-load the "keywords" collection on each "Item",
# but not the "items" collection on "Order" - those
# remain lazily loaded.
query(Order).options(subqueryload(Order.items, Item.keywords))
# to subquery-load across both, use subqueryload_all()
query(Order).options(subqueryload_all(Order.items, Item.keywords))
# set the default strategy to be 'subquery'
query(Order).options(subqueryload('*'))
See also: joinedload(), lazyload()
sqlalchemy.orm.subqueryload_all(*keys)
Return a MapperOption that will convert all properties along the given dot-separated path or series of mapped
attributes into a subquery eager load.
Used with options().
For example:
query.options(subqueryload_all('orders.items.keywords'))...
will set all of orders, orders.items, and orders.items.keywords to load in one subquery eager
load.
Individual descriptors are accepted as arguments as well:
2.8. Relationship Loading Techniques
209
SQLAlchemy Documentation, Release 0.8.7
query.options(subqueryload_all(User.orders, Order.items,
Item.keywords))
See also: joinedload_all(), lazyload(), immediateload()
2.9 ORM Events
The ORM includes a wide variety of hooks available for subscription.
New in version 0.7: The event supersedes the previous system of “extension” classes.
For an introduction to the event API, see Events. Non-ORM events such as those regarding connections and low-level
statement execution are described in Core Events.
2.9.1 Attribute Events
class sqlalchemy.orm.events.AttributeEvents
Bases: sqlalchemy.event.Events
Define events for object attributes.
These are typically defined on the class-bound descriptor for the target class.
e.g.:
from sqlalchemy import event
def my_append_listener(target, value, initiator):
print "received append event for target: %s" % target
event.listen(MyClass.collection, 'append', my_append_listener)
Listeners have the option to return a possibly modified version of the value, when the retval=True flag is
passed to listen():
def validate_phone(target, value, oldvalue, initiator):
"Strip non-numeric characters from a phone number"
return re.sub(r'(?![0-9])', '', value)
# setup listener on UserContact.phone attribute, instructing
# it to use the return value
listen(UserContact.phone, 'set', validate_phone, retval=True)
A validation function like the above can also raise an exception such as ValueError to halt the operation.
Several modifiers are available to the listen() function.
Parameters
• active_history=False – When True, indicates that the “set” event would like
to receive the “old” value being replaced unconditionally, even if this requires firing off database loads. Note that active_history can also be set directly via
column_property() and relationship().
• propagate=False – When True, the listener function will be established not just for
the class attribute given, but for attributes of the same name on all current subclasses of
210
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
that class, as well as all future subclasses of that class, using an additional listener that
listens for instrumentation events.
• raw=False – When True, the “target” argument to the event will be the
InstanceState management object, rather than the mapped instance itself.
• retval=False – when True, the user-defined event listening must return the “value”
argument from the function. This gives the listening function the opportunity to change
the value that is ultimately used for a “set” or “append” event.
append(target, value, initiator)
Receive a collection append event.
Parameters
• target – the object instance receiving the event. If the listener is registered with
raw=True, this will be the InstanceState object.
• value – the value being appended. If this listener is registered with retval=True,
the listener function must return this value, or a new value which replaces it.
• initiator – the attribute implementation object which initiated this event.
Returns if the event was registered with retval=True, the given value, or a new effective
value, should be returned.
remove(target, value, initiator)
Receive a collection remove event.
Parameters
• target – the object instance receiving the event. If the listener is registered with
raw=True, this will be the InstanceState object.
• value – the value being removed.
• initiator – the attribute implementation object which initiated this event.
Returns No return value is defined for this event.
set(target, value, oldvalue, initiator)
Receive a scalar set event.
Parameters
• target – the object instance receiving the event. If the listener is registered with
raw=True, this will be the InstanceState object.
• value – the value being set. If this listener is registered with retval=True, the
listener function must return this value, or a new value which replaces it.
• oldvalue – the previous value being replaced.
This may also be
the symbol NEVER_SET or NO_VALUE. If the listener is registered with
active_history=True, the previous value of the attribute will be loaded from
the database if the existing value is currently unloaded or expired.
• initiator – the attribute implementation object which initiated this event.
Returns if the event was registered with retval=True, the given value, or a new effective
value, should be returned.
2.9. ORM Events
211
SQLAlchemy Documentation, Release 0.8.7
2.9.2 Mapper Events
class sqlalchemy.orm.events.MapperEvents
Bases: sqlalchemy.event.Events
Define events specific to mappings.
e.g.:
from sqlalchemy import event
def my_before_insert_listener(mapper, connection, target):
# execute a stored procedure upon INSERT,
# apply the value to the row to be inserted
target.calculated_value = connection.scalar(
"select my_special_function(%d)"
% target.special_number)
# associate the listener function with SomeClass,
# to execute during the "before_insert" hook
event.listen(
SomeClass, 'before_insert', my_before_insert_listener)
Available targets include:
•mapped classes
•unmapped superclasses of mapped or to-be-mapped classes (using the propagate=True flag)
•Mapper objects
•the Mapper class itself and the mapper() function indicate listening for all mappers.
Changed in version 0.8.0: mapper events can be associated with unmapped superclasses of mapped classes.
Mapper events provide hooks into critical sections of the mapper, including those related to object instrumentation, object loading, and object persistence.
In particular, the persistence methods
before_insert(), and before_update() are popular places to augment the state being persisted however, these methods operate with several significant restrictions. The user is encouraged to evaluate the
SessionEvents.before_flush() and SessionEvents.after_flush() methods as more flexible and user-friendly hooks in which to apply additional database state during a flush.
When using MapperEvents, several modifiers are available to the event.listen() function.
Parameters
• propagate=False – When True, the event listener should be applied to all inheriting
mappers and/or the mappers of inheriting classes, as well as any mapper which is the
target of this listener.
• raw=False – When True, the “target” argument passed to applicable event listener
functions will be the instance’s InstanceState management object, rather than the
mapped instance itself.
• retval=False – when True, the user-defined event function must have a return value,
the purpose of which is either to control subsequent event propagation, or to otherwise
alter the operation in progress by the mapper. Possible return values are:
– sqlalchemy.orm.interfaces.EXT_CONTINUE - continue event processing normally.
– sqlalchemy.orm.interfaces.EXT_STOP - cancel all subsequent event
handlers in the chain.
212
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
– other values - the return value specified by specific listeners, such as
translate_row() or create_instance().
after_configured()
Called after a series of mappers have been configured.
This corresponds to the orm.configure_mappers() call, which note is usually called automatically
as mappings are first used.
Theoretically this event is called once per application, but is actually called any time new mappers have
been affected by a orm.configure_mappers() call. If new mappings are constructed after existing
ones have already been used, this event can be called again.
after_delete(mapper, connection, target)
Receive an object instance after a DELETE statement has been emitted corresponding to that instance.
This event is used to emit additional SQL statements on the given connection as well as to perform
application specific bookkeeping related to a deletion event.
The event is often called for a batch of objects of the same class after their DELETE statements have been
emitted at once in a previous step.
Warning: Mapper-level flush events are designed to operate on attributes local to the immediate
object being handled and via SQL operations with the given Connection only. Handlers here
should not make alterations to the state of the Session overall, and in general should not affect any
relationship() -mapped attributes, as session cascade rules will not function properly, nor is it
always known if the related class has already been handled. Operations that are not supported in
mapper events include:
•Session.add()
•Session.delete()
•Mapped collection append, add, remove, delete, discard, etc.
•Mapped relationship attribute set/del events, i.e.
someobject.related =
someotherobject
Operations which manipulate the state of the object relative to other objects are better handled:
•In the __init__() method of the mapped object itself, or another method designed to establish some particular state.
•In a @validates handler, see Simple Validators
•Within the SessionEvents.before_flush() event.
Parameters
• mapper – the Mapper which is the target of this event.
• connection – the Connection being used to emit DELETE statements for this
instance. This provides a handle into the current transaction on the target database
specific to this instance.
• target – the mapped instance being deleted. If the event is configured with
raw=True, this will instead be the InstanceState state-management object associated with the instance.
Returns No return value is supported by this event.
after_insert(mapper, connection, target)
Receive an object instance after an INSERT statement is emitted corresponding to that instance.
This event is used to modify in-Python-only state on the instance after an INSERT occurs, as well as to
emit additional SQL statements on the given connection.
The event is often called for a batch of objects of the same class after their INSERT statements have been
emitted at once in a previous step. In the extremely rare case that this is not desirable, the mapper() can
2.9. ORM Events
213
SQLAlchemy Documentation, Release 0.8.7
be configured with batch=False, which will cause batches of instances to be broken up into individual
(and more poorly performing) event->persist->event steps.
Warning: Mapper-level flush events are designed to operate on attributes local to the immediate
object being handled and via SQL operations with the given Connection only. Handlers here
should not make alterations to the state of the Session overall, and in general should not affect any
relationship() -mapped attributes, as session cascade rules will not function properly, nor is it
always known if the related class has already been handled. Operations that are not supported in
mapper events include:
•Session.add()
•Session.delete()
•Mapped collection append, add, remove, delete, discard, etc.
•Mapped relationship attribute set/del events, i.e.
someobject.related =
someotherobject
Operations which manipulate the state of the object relative to other objects are better handled:
•In the __init__() method of the mapped object itself, or another method designed to establish some particular state.
•In a @validates handler, see Simple Validators
•Within the SessionEvents.before_flush() event.
Parameters
• mapper – the Mapper which is the target of this event.
• connection – the Connection being used to emit INSERT statements for this
instance. This provides a handle into the current transaction on the target database
specific to this instance.
• target – the mapped instance being persisted. If the event is configured with
raw=True, this will instead be the InstanceState state-management object associated with the instance.
Returns No return value is supported by this event.
after_update(mapper, connection, target)
Receive an object instance after an UPDATE statement is emitted corresponding to that instance.
This event is used to modify in-Python-only state on the instance after an UPDATE occurs, as well as to
emit additional SQL statements on the given connection.
This method is called for all instances that are marked as “dirty”, even those which have no net changes to
their column-based attributes, and for which no UPDATE statement has proceeded. An object is marked
as dirty when any of its column-based attributes have a “set attribute” operation called or when any of its
collections are modified. If, at update time, no column-based attributes have any net changes, no UPDATE
statement will be issued. This means that an instance being sent to after_update() is not a guarantee
that an UPDATE statement has been issued.
To detect if the column-based attributes on the object have net changes, and therefore resulted
in an UPDATE statement, use object_session(instance).is_modified(instance,
include_collections=False).
The event is often called for a batch of objects of the same class after their UPDATE statements have been
emitted at once in a previous step. In the extremely rare case that this is not desirable, the mapper() can
be configured with batch=False, which will cause batches of instances to be broken up into individual
(and more poorly performing) event->persist->event steps.
214
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
Warning: Mapper-level flush events are designed to operate on attributes local to the immediate
object being handled and via SQL operations with the given Connection only. Handlers here
should not make alterations to the state of the Session overall, and in general should not affect any
relationship() -mapped attributes, as session cascade rules will not function properly, nor is it
always known if the related class has already been handled. Operations that are not supported in
mapper events include:
•Session.add()
•Session.delete()
•Mapped collection append, add, remove, delete, discard, etc.
•Mapped relationship attribute set/del events, i.e.
someobject.related =
someotherobject
Operations which manipulate the state of the object relative to other objects are better handled:
•In the __init__() method of the mapped object itself, or another method designed to establish some particular state.
•In a @validates handler, see Simple Validators
•Within the SessionEvents.before_flush() event.
Parameters
• mapper – the Mapper which is the target of this event.
• connection – the Connection being used to emit UPDATE statements for this
instance. This provides a handle into the current transaction on the target database
specific to this instance.
• target – the mapped instance being persisted. If the event is configured with
raw=True, this will instead be the InstanceState state-management object associated with the instance.
Returns No return value is supported by this event.
append_result(mapper, context, row, target, result, **flags)
Receive an object instance before that instance is appended to a result list.
This is a rarely used hook which can be used to alter the construction of a result list returned by Query.
Parameters
• mapper – the Mapper which is the target of this event.
• context – the QueryContext, which includes a handle to the current Query in
progress as well as additional state information.
• row – the result row being handled. This may be an actual RowProxy or may be a
dictionary containing Column objects as keys.
• target – the mapped instance being populated. If the event is configured with
raw=True, this will instead be the InstanceState state-management object associated with the instance.
• result – a list-like object where results are being appended.
• **flags – Additional state information about the current handling of the row.
Returns If this method is registered with retval=True, a return value of EXT_STOP will
prevent the instance from being appended to the given result list, whereas a return value
of EXT_CONTINUE will result in the default behavior of appending the value to the result
list.
before_delete(mapper, connection, target)
Receive an object instance before a DELETE statement is emitted corresponding to that instance.
2.9. ORM Events
215
SQLAlchemy Documentation, Release 0.8.7
This event is used to emit additional SQL statements on the given connection as well as to perform
application specific bookkeeping related to a deletion event.
The event is often called for a batch of objects of the same class before their DELETE statements are
emitted at once in a later step.
Warning: Mapper-level flush events are designed to operate on attributes local to the immediate
object being handled and via SQL operations with the given Connection only. Handlers here
should not make alterations to the state of the Session overall, and in general should not affect any
relationship() -mapped attributes, as session cascade rules will not function properly, nor is it
always known if the related class has already been handled. Operations that are not supported in
mapper events include:
•Session.add()
•Session.delete()
•Mapped collection append, add, remove, delete, discard, etc.
•Mapped relationship attribute set/del events, i.e.
someobject.related =
someotherobject
Operations which manipulate the state of the object relative to other objects are better handled:
•In the __init__() method of the mapped object itself, or another method designed to establish some particular state.
•In a @validates handler, see Simple Validators
•Within the SessionEvents.before_flush() event.
Parameters
• mapper – the Mapper which is the target of this event.
• connection – the Connection being used to emit DELETE statements for this
instance. This provides a handle into the current transaction on the target database
specific to this instance.
• target – the mapped instance being deleted. If the event is configured with
raw=True, this will instead be the InstanceState state-management object associated with the instance.
Returns No return value is supported by this event.
before_insert(mapper, connection, target)
Receive an object instance before an INSERT statement is emitted corresponding to that instance.
This event is used to modify local, non-object related attributes on the instance before an INSERT occurs,
as well as to emit additional SQL statements on the given connection.
The event is often called for a batch of objects of the same class before their INSERT statements are
emitted at once in a later step. In the extremely rare case that this is not desirable, the mapper() can be
configured with batch=False, which will cause batches of instances to be broken up into individual
(and more poorly performing) event->persist->event steps.
216
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
Warning: Mapper-level flush events are designed to operate on attributes local to the immediate
object being handled and via SQL operations with the given Connection only. Handlers here
should not make alterations to the state of the Session overall, and in general should not affect any
relationship() -mapped attributes, as session cascade rules will not function properly, nor is it
always known if the related class has already been handled. Operations that are not supported in
mapper events include:
•Session.add()
•Session.delete()
•Mapped collection append, add, remove, delete, discard, etc.
•Mapped relationship attribute set/del events, i.e.
someobject.related =
someotherobject
Operations which manipulate the state of the object relative to other objects are better handled:
•In the __init__() method of the mapped object itself, or another method designed to establish some particular state.
•In a @validates handler, see Simple Validators
•Within the SessionEvents.before_flush() event.
Parameters
• mapper – the Mapper which is the target of this event.
• connection – the Connection being used to emit INSERT statements for this
instance. This provides a handle into the current transaction on the target database
specific to this instance.
• target – the mapped instance being persisted. If the event is configured with
raw=True, this will instead be the InstanceState state-management object associated with the instance.
Returns No return value is supported by this event.
before_update(mapper, connection, target)
Receive an object instance before an UPDATE statement is emitted corresponding to that instance.
This event is used to modify local, non-object related attributes on the instance before an UPDATE occurs,
as well as to emit additional SQL statements on the given connection.
This method is called for all instances that are marked as “dirty”, even those which have no net changes
to their column-based attributes. An object is marked as dirty when any of its column-based attributes
have a “set attribute” operation called or when any of its collections are modified. If, at update time, no
column-based attributes have any net changes, no UPDATE statement will be issued. This means that an
instance being sent to before_update() is not a guarantee that an UPDATE statement will be issued,
although you can affect the outcome here by modifying attributes so that a net change in value does exist.
To detect if the column-based attributes on the object have net changes, and will therefore generate an UPDATE statement, use object_session(instance).is_modified(instance,
include_collections=False).
The event is often called for a batch of objects of the same class before their UPDATE statements are
emitted at once in a later step. In the extremely rare case that this is not desirable, the mapper() can be
configured with batch=False, which will cause batches of instances to be broken up into individual
(and more poorly performing) event->persist->event steps.
2.9. ORM Events
217
SQLAlchemy Documentation, Release 0.8.7
Warning: Mapper-level flush events are designed to operate on attributes local to the immediate
object being handled and via SQL operations with the given Connection only. Handlers here
should not make alterations to the state of the Session overall, and in general should not affect any
relationship() -mapped attributes, as session cascade rules will not function properly, nor is it
always known if the related class has already been handled. Operations that are not supported in
mapper events include:
•Session.add()
•Session.delete()
•Mapped collection append, add, remove, delete, discard, etc.
•Mapped relationship attribute set/del events, i.e.
someobject.related =
someotherobject
Operations which manipulate the state of the object relative to other objects are better handled:
•In the __init__() method of the mapped object itself, or another method designed to establish some particular state.
•In a @validates handler, see Simple Validators
•Within the SessionEvents.before_flush() event.
Parameters
• mapper – the Mapper which is the target of this event.
• connection – the Connection being used to emit UPDATE statements for this
instance. This provides a handle into the current transaction on the target database
specific to this instance.
• target – the mapped instance being persisted. If the event is configured with
raw=True, this will instead be the InstanceState state-management object associated with the instance.
Returns No return value is supported by this event.
create_instance(mapper, context, row, class_)
Receive a row when a new object instance is about to be created from that row.
The method can choose to create the instance itself, or it can return EXT_CONTINUE to indicate normal
object creation should take place. This listener is typically registered with retval=True.
Parameters
• mapper – the Mapper which is the target of this event.
• context – the QueryContext, which includes a handle to the current Query in
progress as well as additional state information.
• row – the result row being handled. This may be an actual RowProxy or may be a
dictionary containing Column objects as keys.
• class_ – the mapped class.
Returns When configured with retval=True, the return value should be a newly created
instance of the mapped class, or EXT_CONTINUE indicating that default object construction should take place.
instrument_class(mapper, class_)
Receive a class when the mapper is first constructed, before instrumentation is applied to the mapped
class.
This event is the earliest phase of mapper construction. Most attributes of the mapper are not yet initialized.
218
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
This listener can either be applied to the Mapper class overall, or to any un-mapped class which serves
as a base for classes that will be mapped (using the propagate=True flag):
Base = declarative_base()
@event.listens_for(Base, "instrument_class", propagate=True)
def on_new_class(mapper, cls_):
" ... "
Parameters
• mapper – the Mapper which is the target of this event.
• class_ – the mapped class.
mapper_configured(mapper, class_)
Called when the mapper for the class is fully configured.
This event is the latest phase of mapper construction, and is invoked when the mapped classes are first
used, so that relationships between mappers can be resolved. When the event is called, the mapper should
be in its final state.
While the configuration event normally occurs automatically, it can be forced to occur ahead of time, in the
case where the event is needed before any actual mapper usage, by using the configure_mappers()
function.
Parameters
• mapper – the Mapper which is the target of this event.
• class_ – the mapped class.
populate_instance(mapper, context, row, target, **flags)
Receive an instance before that instance has its attributes populated.
This usually corresponds to a newly loaded instance but may also correspond to an already-loaded instance
which has unloaded attributes to be populated. The method may be called many times for a single instance,
as multiple result rows are used to populate eagerly loaded collections.
Most usages of this hook are obsolete. For a generic “object has been newly created from a row” hook,
use InstanceEvents.load().
Parameters
• mapper – the Mapper which is the target of this event.
• context – the QueryContext, which includes a handle to the current Query in
progress as well as additional state information.
• row – the result row being handled. This may be an actual RowProxy or may be a
dictionary containing Column objects as keys.
• target – the mapped instance. If the event is configured with raw=True, this
will instead be the InstanceState state-management object associated with the
instance.
Returns When configured with retval=True, a return value of EXT_STOP will bypass
instance population by the mapper. A value of EXT_CONTINUE indicates that default
instance population should take place.
translate_row(mapper, context, row)
Perform pre-processing on the given result row and return a new row instance.
2.9. ORM Events
219
SQLAlchemy Documentation, Release 0.8.7
This listener is typically registered with retval=True. It is called when the mapper first receives a row,
before the object identity or the instance itself has been derived from that row. The given row may or may
not be a RowProxy object - it will always be a dictionary-like object which contains mapped columns
as keys. The returned object should also be a dictionary-like object which recognizes mapped columns as
keys.
Parameters
• mapper – the Mapper which is the target of this event.
• context – the QueryContext, which includes a handle to the current Query in
progress as well as additional state information.
• row – the result row being handled. This may be an actual RowProxy or may be a
dictionary containing Column objects as keys.
Returns When configured with retval=True, the function should return a dictionary-like
row object, or EXT_CONTINUE, indicating the original row should be used.
2.9.3 Instance Events
class sqlalchemy.orm.events.InstanceEvents
Bases: sqlalchemy.event.Events
Define events specific to object lifecycle.
e.g.:
from sqlalchemy import event
def my_load_listener(target, context):
print "on load!"
event.listen(SomeClass, 'load', my_load_listener)
Available targets include:
•mapped classes
•unmapped superclasses of mapped or to-be-mapped classes (using the propagate=True flag)
•Mapper objects
•the Mapper class itself and the mapper() function indicate listening for all mappers.
Changed in version 0.8.0: instance events can be associated with unmapped superclasses of mapped classes.
Instance events are closely related to mapper events, but are more specific to the instance and its instrumentation,
rather than its system of persistence.
When using InstanceEvents, several modifiers are available to the event.listen() function.
Parameters
• propagate=False – When True, the event listener should be applied to all inheriting
classes as well as the class which is the target of this listener.
• raw=False – When True, the “target” argument passed to applicable event listener
functions will be the instance’s InstanceState management object, rather than the
mapped instance itself.
220
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
expire(target, attrs)
Receive an object instance after its attributes or some subset have been expired.
‘keys’ is a list of attribute names. If None, the entire state was expired.
Parameters
• target – the mapped instance. If the event is configured with raw=True, this
will instead be the InstanceState state-management object associated with the
instance.
• attrs – iterable collection of attribute names which were expired, or None if all
attributes were expired.
first_init(manager, cls)
Called when the first instance of a particular mapping is called.
init(target, args, kwargs)
Receive an instance when its constructor is called.
This method is only called during a userland construction of an object. It is not called when an object is
loaded from the database.
init_failure(target, args, kwargs)
Receive an instance when its constructor has been called, and raised an exception.
This method is only called during a userland construction of an object. It is not called when an object is
loaded from the database.
load(target, context)
Receive an object instance after it has been created via __new__, and after initial attribute population
has occurred.
This typically occurs when the instance is created based on incoming result rows, and is only called once
for that instance’s lifetime.
Note that during a result-row load, this method is called upon the first row received for this instance. Note
that some attributes and collections may or may not be loaded or even initialized, depending on what’s
present in the result rows.
Parameters
• target – the mapped instance. If the event is configured with raw=True, this
will instead be the InstanceState state-management object associated with the
instance.
• context – the QueryContext corresponding to the current Query in progress.
This argument may be None if the load does not correspond to a Query, such as
during Session.merge().
pickle(target, state_dict)
Receive an object instance when its associated state is being pickled.
Parameters
• target – the mapped instance. If the event is configured with raw=True, this
will instead be the InstanceState state-management object associated with the
instance.
• state_dict – the dictionary returned by InstanceState.__getstate__,
containing the state to be pickled.
refresh(target, context, attrs)
Receive an object instance after one or more attributes have been refreshed from a query.
2.9. ORM Events
221
SQLAlchemy Documentation, Release 0.8.7
Parameters
• target – the mapped instance. If the event is configured with raw=True, this
will instead be the InstanceState state-management object associated with the
instance.
• context – the QueryContext corresponding to the current Query in progress.
• attrs – iterable collection of attribute names which were populated, or None if all
column-mapped, non-deferred attributes were populated.
resurrect(target)
Receive an object instance as it is ‘resurrected’ from garbage collection, which occurs when a “dirty” state
falls out of scope.
Parameters target – the mapped instance. If the event is configured with raw=True,
this will instead be the InstanceState state-management object associated with the
instance.
unpickle(target, state_dict)
Receive an object instance after its associated state has been unpickled.
Parameters
• target – the mapped instance. If the event is configured with raw=True, this
will instead be the InstanceState state-management object associated with the
instance.
• state_dict – the dictionary sent to InstanceState.__setstate__, containing the state dictionary which was pickled.
2.9.4 Session Events
class sqlalchemy.orm.events.SessionEvents
Bases: sqlalchemy.event.Events
Define events specific to Session lifecycle.
e.g.:
from sqlalchemy import event
from sqlalchemy.orm import sessionmaker
def my_before_commit(session):
print "before commit!"
Session = sessionmaker()
event.listen(Session, "before_commit", my_before_commit)
The listen() function will accept Session objects as well as the return result of sessionmaker() and
scoped_session().
Additionally, it accepts the Session class which will apply listeners to all Session instances globally.
after_attach(session, instance)
Execute after an instance is attached to a session.
This is called after an add, delete or merge.
222
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
Note: As of 0.8, this event fires off after the item has been fully associated with the session, which
is different than previous releases. For event handlers that require the object not yet be part of session
state (such as handlers which may autoflush while the target object is not yet complete) consider the new
before_attach() event.
See also:
before_attach()
after_begin(session, transaction, connection)
Execute after a transaction is begun on a connection
Parameters
• session – The target Session.
• transaction – The SessionTransaction.
• connection – The Connection object which will be used for SQL statements.
See also:
before_commit()
after_commit()
after_transaction_create()
after_transaction_end()
after_bulk_delete(session, query, query_context, result)
Execute after a bulk delete operation to the session.
This is called as a result of the Query.delete() method.
Parameters
• query – the Query object that this update operation was called upon.
• query_context – The QueryContext object, corresponding to the invocation
of an ORM query.
• result – the ResultProxy returned as a result of the bulk DELETE operation.
after_bulk_update(session, query, query_context, result)
Execute after a bulk update operation to the session.
This is called as a result of the Query.update() method.
Parameters
• query – the Query object that this update operation was called upon.
• query_context – The QueryContext object, corresponding to the invocation
of an ORM query.
• result – the ResultProxy returned as a result of the bulk UPDATE operation.
after_commit(session)
Execute after a commit has occurred.
Note: The after_commit() hook is not per-flush, that is, the Session can emit SQL to the
database many times within the scope of a transaction. For interception of these events, use the
2.9. ORM Events
223
SQLAlchemy Documentation, Release 0.8.7
before_flush(), after_flush(), or after_flush_postexec() events.
Note:
The Session is not in an active tranasction when the after_commit() event is invoked, and therefore can not emit SQL. To emit SQL corresponding to every transaction, use the
before_commit() event.
Parameters session – The target Session.
See also:
before_commit()
after_begin()
after_transaction_create()
after_transaction_end()
after_flush(session, flush_context)
Execute after flush has completed, but before commit has been called.
Note that the session’s state is still in pre-flush, i.e. ‘new’, ‘dirty’, and ‘deleted’ lists still show pre-flush
state as well as the history settings on instance attributes.
Parameters
• session – The target Session.
• flush_context – Internal UOWTransaction object which handles the details
of the flush.
See also:
before_flush()
after_flush_postexec()
after_flush_postexec(session, flush_context)
Execute after flush has completed, and after the post-exec state occurs.
This will be when the ‘new’, ‘dirty’, and ‘deleted’ lists are in their final state. An actual commit() may or
may not have occurred, depending on whether or not the flush started its own transaction or participated
in a larger transaction.
Parameters
• session – The target Session.
• flush_context – Internal UOWTransaction object which handles the details
of the flush.
See also:
before_flush()
after_flush()
after_rollback(session)
Execute after a real DBAPI rollback has occurred.
Note that this event only fires when the actual rollback against the database occurs - it does not fire
each time the Session.rollback() method is called, if the underlying DBAPI transaction has
224
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
already been rolled back. In many cases, the Session will not be in an “active” state during this
event, as the current transaction is not valid. To acquire a Session which is active after the outermost
rollback has proceeded, use the SessionEvents.after_soft_rollback() event, checking the
Session.is_active flag.
Parameters session – The target Session.
after_soft_rollback(session, previous_transaction)
Execute after any rollback has occurred, including “soft” rollbacks that don’t actually emit at the DBAPI
level.
This corresponds to both nested and outer rollbacks, i.e. the innermost rollback that calls the DBAPI’s
rollback() method, as well as the enclosing rollback calls that only pop themselves from the transaction
stack.
The given Session can be used to invoke SQL and Session.query() operations after an outermost
rollback by first checking the Session.is_active flag:
@event.listens_for(Session, "after_soft_rollback")
def do_something(session, previous_transaction):
if session.is_active:
session.execute("select * from some_table")
Parameters
• session – The target Session.
• previous_transaction – The SessionTransaction
transactional marker object which was just closed. The current SessionTransaction for the given
Session is available via the Session.transaction attribute.
New in version 0.7.3.
after_transaction_create(session, transaction)
Execute when a new SessionTransaction is created.
This event differs from after_begin() in that it occurs for each SessionTransaction overall, as opposed to when transactions are begun on individual database connections. It is also
invoked for nested transactions and subtransactions, and is always matched by a corresponding
after_transaction_end() event (assuming normal operation of the Session).
Parameters
• session – the target Session.
• transaction – the target SessionTransaction.
New in version 0.8.
See also:
after_transaction_end()
after_transaction_end(session, transaction)
Execute when the span of a SessionTransaction ends.
This event differs from after_commit() in that it corresponds to all SessionTransaction objects in use, including those for nested transactions and subtransactions, and is always matched by a
corresponding after_transaction_create() event.
Parameters
• session – the target Session.
2.9. ORM Events
225
SQLAlchemy Documentation, Release 0.8.7
• transaction – the target SessionTransaction.
New in version 0.8.
See also:
after_transaction_create()
before_attach(session, instance)
Execute before an instance is attached to a session.
This is called before an add, delete or merge causes the object to be part of the session.
New in version 0.8.: Note that after_attach() now fires off after the item is part of the session.
before_attach() is provided for those cases where the item should not yet be part of the session
state.
See also:
after_attach()
before_commit(session)
Execute before commit is called.
Note: The before_commit() hook is not per-flush, that is, the Session can emit SQL to
the database many times within the scope of a transaction. For interception of these events, use the
before_flush(), after_flush(), or after_flush_postexec() events.
Parameters session – The target Session.
See also:
after_commit()
after_begin()
after_transaction_create()
after_transaction_end()
before_flush(session, flush_context, instances)
Execute before flush process has started.
Parameters
• session – The target Session.
• flush_context – Internal UOWTransaction object which handles the details
of the flush.
• instances – Usually None, this is the collection of objects which can be passed
to the Session.flush() method (note this usage is deprecated).
See also:
after_flush()
after_flush_postexec()
226
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
2.9.5 Instrumentation Events
class sqlalchemy.orm.events.InstrumentationEvents
Bases: sqlalchemy.event.Events
Events related to class instrumentation events.
The listeners here support being established against any new style class, that is any object that is a subclass
of ‘type’. Events will then be fired off for events against that class. If the “propagate=True” flag is passed to
event.listen(), the event will fire off for subclasses of that class as well.
The Python type builtin is also accepted as a target, which when used has the effect of events being emitted
for all classes.
Note the “propagate” flag here is defaulted to True, unlike the other class level events where it defaults to
False. This means that new subclasses will also be the subject of these events, when a listener is established
on a superclass.
Changed in version 0.8: - events here will emit based on comparing the incoming class to the type of class passed
to event.listen(). Previously, the event would fire for any class unconditionally regardless of what class
was sent for listening, despite documentation which stated the contrary.
attribute_instrument(cls, key, inst)
Called when an attribute is instrumented.
class_instrument(cls)
Called after the given class is instrumented.
To get at the ClassManager, use manager_of_class().
class_uninstrument(cls)
Called before the given class is uninstrumented.
To get at the ClassManager, use manager_of_class().
2.10 ORM Extensions
SQLAlchemy has a variety of ORM extensions available, which add additional functionality to the core behavior.
The extensions build almost entirely on public core and ORM APIs and users should be encouraged to read their source
code to further their understanding of their behavior. In particular the “Horizontal Sharding”, “Hybrid Attributes”, and
“Mutation Tracking” extensions are very succinct.
2.10.1 Association Proxy
associationproxy is used to create a read/write view of a target attribute across a relationship. It essentially conceals the usage of a “middle” attribute between two endpoints, and can be used to cherry-pick fields from a collection
of related objects or to reduce the verbosity of using the association object pattern. Applied creatively, the association
proxy allows the construction of sophisticated collections and dictionary views of virtually any geometry, persisted to
the database using standard, transparently configured relational patterns.
Simplifying Scalar Collections
Consider a many-to-many mapping between two classes, User and Keyword. Each User can have any number of
Keyword objects, and vice-versa (the many-to-many pattern is described at Many To Many):
2.10. ORM Extensions
227
SQLAlchemy Documentation, Release 0.8.7
from sqlalchemy import Column, Integer, String, ForeignKey, Table
from sqlalchemy.orm import relationship
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String(64))
kw = relationship("Keyword", secondary=lambda: userkeywords_table)
def __init__(self, name):
self.name = name
class Keyword(Base):
__tablename__ = 'keyword'
id = Column(Integer, primary_key=True)
keyword = Column('keyword', String(64))
def __init__(self, keyword):
self.keyword = keyword
userkeywords_table = Table('userkeywords', Base.metadata,
Column('user_id', Integer, ForeignKey("user.id"),
primary_key=True),
Column('keyword_id', Integer, ForeignKey("keyword.id"),
primary_key=True)
)
Reading and manipulating the collection of “keyword” strings associated with User requires traversal from each
collection element to the .keyword attribute, which can be awkward:
>>> user = User('jek')
>>> user.kw.append(Keyword('cheese inspector'))
>>> print(user.kw)
[<__main__.Keyword object at 0x12bf830>]
>>> print(user.kw[0].keyword)
cheese inspector
>>> print([keyword.keyword for keyword in user.kw])
['cheese inspector']
The association_proxy is applied to the User class to produce a “view” of the kw relationship, which only
exposes the string value of .keyword associated with each Keyword object:
from sqlalchemy.ext.associationproxy import association_proxy
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String(64))
kw = relationship("Keyword", secondary=lambda: userkeywords_table)
def __init__(self, name):
self.name = name
228
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
# proxy the 'keyword' attribute from the 'kw' relationship
keywords = association_proxy('kw', 'keyword')
We can now reference the .keywords collection as a listing of strings, which is both readable and writable. New
Keyword objects are created for us transparently:
>>> user = User('jek')
>>> user.keywords.append('cheese inspector')
>>> user.keywords
['cheese inspector']
>>> user.keywords.append('snack ninja')
>>> user.kw
[<__main__.Keyword object at 0x12cdd30>, <__main__.Keyword object at 0x12cde30>]
The AssociationProxy object produced by the association_proxy() function is an instance of a Python
descriptor. It is always declared with the user-defined class being mapped, regardless of whether Declarative or
classical mappings via the mapper() function are used.
The proxy functions by operating upon the underlying mapped attribute or collection in response to operations, and
changes made via the proxy are immediately apparent in the mapped attribute, as well as vice versa. The underlying
attribute remains fully accessible.
When first accessed, the association proxy performs introspection operations on the target collection so that its behavior corresponds correctly. Details such as if the locally proxied attribute is a collection (as is typical) or a scalar
reference, as well as if the collection acts like a set, list, or dictionary is taken into account, so that the proxy should
act just like the underlying collection or attribute does.
Creation of New Values
When a list append() event (or set add(), dictionary __setitem__(), or scalar assignment event) is intercepted by the
association proxy, it instantiates a new instance of the “intermediary” object using its constructor, passing as a single
argument the given value. In our example above, an operation like:
user.keywords.append('cheese inspector')
Is translated by the association proxy into the operation:
user.kw.append(Keyword('cheese inspector'))
The example works here because we have designed the constructor for Keyword to accept a single positional argument, keyword. For those cases where a single-argument constructor isn’t feasible, the association proxy’s creational
behavior can be customized using the creator argument, which references a callable (i.e. Python function) that will
produce a new object instance given the singular argument. Below we illustrate this using a lambda as is typical:
class User(Base):
# ...
# use Keyword(keyword=kw) on append() events
keywords = association_proxy('kw', 'keyword',
creator=lambda kw: Keyword(keyword=kw))
The creator function accepts a single argument in the case of a list- or set- based collection, or a scalar attribute. In
the case of a dictionary-based collection, it accepts two arguments, “key” and “value”. An example of this is below in
Proxying to Dictionary Based Collections.
2.10. ORM Extensions
229
SQLAlchemy Documentation, Release 0.8.7
Simplifying Association Objects
The “association object” pattern is an extended form of a many-to-many relationship, and is described at Association
Object. Association proxies are useful for keeping “association objects” out the way during regular use.
Suppose our userkeywords table above had additional columns which we’d like to map explicitly, but in most
cases we don’t require direct access to these attributes. Below, we illustrate a new mapping which introduces the
UserKeyword class, which is mapped to the userkeywords table illustrated earlier. This class adds an additional
column special_key, a value which we occasionally want to access, but not in the usual case. We create an
association proxy on the User class called keywords, which will bridge the gap from the user_keywords
collection of User to the .keyword attribute present on each UserKeyword:
from sqlalchemy import Column, Integer, String, ForeignKey
from sqlalchemy.orm import relationship, backref
from sqlalchemy.ext.associationproxy import association_proxy
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String(64))
# association proxy of "user_keywords" collection
# to "keyword" attribute
keywords = association_proxy('user_keywords', 'keyword')
def __init__(self, name):
self.name = name
class UserKeyword(Base):
__tablename__ = 'user_keyword'
user_id = Column(Integer, ForeignKey('user.id'), primary_key=True)
keyword_id = Column(Integer, ForeignKey('keyword.id'), primary_key=True)
special_key = Column(String(50))
# bidirectional attribute/collection of "user"/"user_keywords"
user = relationship(User,
backref=backref("user_keywords",
cascade="all, delete-orphan")
)
# reference to the "Keyword" object
keyword = relationship("Keyword")
def __init__(self, keyword=None, user=None, special_key=None):
self.user = user
self.keyword = keyword
self.special_key = special_key
class Keyword(Base):
__tablename__ = 'keyword'
id = Column(Integer, primary_key=True)
keyword = Column('keyword', String(64))
def __init__(self, keyword):
230
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
self.keyword = keyword
def __repr__(self):
return 'Keyword(%s)' % repr(self.keyword)
With the above configuration, we can operate upon the .keywords collection of each User object, and the usage of
UserKeyword is concealed:
>>> user = User('log')
>>> for kw in (Keyword('new_from_blammo'), Keyword('its_big')):
...
user.keywords.append(kw)
...
>>> print(user.keywords)
[Keyword('new_from_blammo'), Keyword('its_big')]
Where above, each .keywords.append() operation is equivalent to:
>>> user.user_keywords.append(UserKeyword(Keyword('its_heavy')))
The UserKeyword association object has two attributes here which are populated; the .keyword attribute is populated directly as a result of passing the Keyword object as the first argument. The .user argument is then assigned
as the UserKeyword object is appended to the User.user_keywords collection, where the bidirectional relationship configured between User.user_keywords and UserKeyword.user results in a population of the
UserKeyword.user attribute. The special_key argument above is left at its default value of None.
For those cases where we do want special_key to have a value, we create the UserKeyword object explicitly.
Below we assign all three attributes, where the assignment of .user has the effect of the UserKeyword being
appended to the User.user_keywords collection:
>>> UserKeyword(Keyword('its_wood'), user, special_key='my special key')
The association proxy returns to us a collection of Keyword objects represented by all these operations:
>>> user.keywords
[Keyword('new_from_blammo'), Keyword('its_big'), Keyword('its_heavy'), Keyword('its_wood')]
Proxying to Dictionary Based Collections
The association proxy can proxy to dictionary based collections as well. SQLAlchemy mappings usually use the
attribute_mapped_collection() collection type to create dictionary collections, as well as the extended
techniques described in Custom Dictionary-Based Collections.
The association proxy adjusts its behavior when it detects the usage of a dictionary-based collection. When new values
are added to the dictionary, the association proxy instantiates the intermediary object by passing two arguments to the
creation function instead of one, the key and the value. As always, this creation function defaults to the constructor of
the intermediary class, and can be customized using the creator argument.
Below, we modify our UserKeyword example such that the User.user_keywords collection will now be
mapped using a dictionary, where the UserKeyword.special_key argument will be used as the key for the
dictionary. We then apply a creator argument to the User.keywords proxy so that these values are assigned
appropriately when new elements are added to the dictionary:
2.10. ORM Extensions
231
SQLAlchemy Documentation, Release 0.8.7
from
from
from
from
from
sqlalchemy import Column, Integer, String, ForeignKey
sqlalchemy.orm import relationship, backref
sqlalchemy.ext.associationproxy import association_proxy
sqlalchemy.ext.declarative import declarative_base
sqlalchemy.orm.collections import attribute_mapped_collection
Base = declarative_base()
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String(64))
# proxy to 'user_keywords', instantiating UserKeyword
# assigning the new key to 'special_key', values to
# 'keyword'.
keywords = association_proxy('user_keywords', 'keyword',
creator=lambda k, v:
UserKeyword(special_key=k, keyword=v)
)
def __init__(self, name):
self.name = name
class UserKeyword(Base):
__tablename__ = 'user_keyword'
user_id = Column(Integer, ForeignKey('user.id'), primary_key=True)
keyword_id = Column(Integer, ForeignKey('keyword.id'), primary_key=True)
special_key = Column(String)
# bidirectional user/user_keywords relationships, mapping
# user_keywords with a dictionary against "special_key" as key.
user = relationship(User, backref=backref(
"user_keywords",
collection_class=attribute_mapped_collection("special_key"),
cascade="all, delete-orphan"
)
)
keyword = relationship("Keyword")
class Keyword(Base):
__tablename__ = 'keyword'
id = Column(Integer, primary_key=True)
keyword = Column('keyword', String(64))
def __init__(self, keyword):
self.keyword = keyword
def __repr__(self):
return 'Keyword(%s)' % repr(self.keyword)
We illustrate the .keywords collection as a dictionary, mapping the UserKeyword.string_key value to
Keyword objects:
>>> user = User('log')
>>> user.keywords['sk1'] = Keyword('kw1')
232
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
>>> user.keywords['sk2'] = Keyword('kw2')
>>> print(user.keywords)
{'sk1': Keyword('kw1'), 'sk2': Keyword('kw2')}
Composite Association Proxies
Given our previous examples of proxying from relationship to scalar attribute, proxying across an association object,
and proxying dictionaries, we can combine all three techniques together to give User a keywords dictionary that
deals strictly with the string value of special_key mapped to the string keyword. Both the UserKeyword and
Keyword classes are entirely concealed. This is achieved by building an association proxy on User that refers to an
association proxy present on UserKeyword:
from sqlalchemy import Column, Integer, String, ForeignKey
from sqlalchemy.orm import relationship, backref
from sqlalchemy.ext.associationproxy import association_proxy
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm.collections import attribute_mapped_collection
Base = declarative_base()
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String(64))
# the same 'user_keywords'->'keyword' proxy as in
# the basic dictionary example
keywords = association_proxy(
'user_keywords',
'keyword',
creator=lambda k, v:
UserKeyword(special_key=k, keyword=v)
)
def __init__(self, name):
self.name = name
class UserKeyword(Base):
__tablename__ = 'user_keyword'
user_id = Column(Integer, ForeignKey('user.id'), primary_key=True)
keyword_id = Column(Integer, ForeignKey('keyword.id'),
primary_key=True)
special_key = Column(String)
user = relationship(User, backref=backref(
"user_keywords",
collection_class=attribute_mapped_collection("special_key"),
cascade="all, delete-orphan"
)
)
# the relationship to Keyword is now called
# 'kw'
kw = relationship("Keyword")
2.10. ORM Extensions
233
SQLAlchemy Documentation, Release 0.8.7
# 'keyword' is changed to be a proxy to the
# 'keyword' attribute of 'Keyword'
keyword = association_proxy('kw', 'keyword')
class Keyword(Base):
__tablename__ = 'keyword'
id = Column(Integer, primary_key=True)
keyword = Column('keyword', String(64))
def __init__(self, keyword):
self.keyword = keyword
User.keywords is now a dictionary of string to string, where UserKeyword and Keyword objects are created
and removed for us transparently using the association proxy. In the example below, we illustrate usage of the assignment operator, also appropriately handled by the association proxy, to apply a dictionary value to the collection at
once:
>>> user = User('log')
>>> user.keywords = {
...
'sk1':'kw1',
...
'sk2':'kw2'
... }
>>> print(user.keywords)
{'sk1': 'kw1', 'sk2': 'kw2'}
>>> user.keywords['sk3'] = 'kw3'
>>> del user.keywords['sk2']
>>> print(user.keywords)
{'sk1': 'kw1', 'sk3': 'kw3'}
>>> # illustrate un-proxied usage
... print(user.user_keywords['sk3'].kw)
<__main__.Keyword object at 0x12ceb90>
One caveat with our example above is that because Keyword objects are created for each dictionary set operation, the
example fails to maintain uniqueness for the Keyword objects on their string name, which is a typical requirement
for a tagging scenario such as this one. For this use case the recipe UniqueObject, or a comparable creational strategy,
is recommended, which will apply a “lookup first, then create” strategy to the constructor of the Keyword class, so
that an already existing Keyword is returned if the given name is already present.
Querying with Association Proxies
The AssociationProxy features simple SQL construction capabilities which relate down to
the underlying relationship() in use as well as the target attribute.
For example, the
RelationshipProperty.Comparator.any() and RelationshipProperty.Comparator.has()
operations are available, and will produce a “nested” EXISTS clause, such as in our basic association object example:
>>> print(session.query(User).filter(User.keywords.any(keyword='jek')))
SELECT user.id AS user_id, user.name AS user_name
FROM user
WHERE EXISTS (SELECT 1
FROM user_keyword
WHERE user.id = user_keyword.user_id AND (EXISTS (SELECT 1
FROM keyword
WHERE keyword.id = user_keyword.keyword_id AND keyword.keyword = :keyword_1)))
234
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
For a proxy to a scalar attribute, __eq__() is supported:
>>> print(session.query(UserKeyword).filter(UserKeyword.keyword == 'jek'))
SELECT user_keyword.*
FROM user_keyword
WHERE EXISTS (SELECT 1
FROM keyword
WHERE keyword.id = user_keyword.keyword_id AND keyword.keyword = :keyword_1)
and .contains() is available for a proxy to a scalar collection:
>>> print(session.query(User).filter(User.keywords.contains('jek')))
SELECT user.*
FROM user
WHERE EXISTS (SELECT 1
FROM userkeywords, keyword
WHERE user.id = userkeywords.user_id
AND keyword.id = userkeywords.keyword_id
AND keyword.keyword = :keyword_1)
AssociationProxy can be used with Query.join() somewhat manually using the attr attribute in a starargs context:
q = session.query(User).join(*User.keywords.attr)
New in version 0.7.3: attr attribute in a star-args context.
attr is composed of AssociationProxy.local_attr and AssociationProxy.remote_attr, which
are just synonyms for the actual proxied attributes, and can also be used for querying:
uka = aliased(UserKeyword)
ka = aliased(Keyword)
q = session.query(User).\
join(uka, User.keywords.local_attr).\
join(ka, User.keywords.remote_attr)
New in version 0.7.3: AssociationProxy.local_attr and AssociationProxy.remote_attr, synonyms for the actual proxied attributes, and usable for querying.
API Documentation
sqlalchemy.ext.associationproxy.association_proxy(target_collection, attr, **kw)
Return a Python property implementing a view of a target attribute which references an attribute on members of
the target.
The returned value is an instance of AssociationProxy.
Implements a Python property representing a relationship as a collection of simpler values, or a scalar value.
The proxied property will mimic the collection type of the target (list, dict or set), or, in the case of a one to one
relationship, a simple scalar value.
Parameters
• target_collection – Name of the attribute we’ll proxy to. This attribute is typically mapped by relationship() to link to a target collection, but can also be a
many-to-one or non-scalar relationship.
2.10. ORM Extensions
235
SQLAlchemy Documentation, Release 0.8.7
• attr – Attribute on the associated instance or instances we’ll proxy for.
For example, given a target collection of [obj1, obj2], a list created by this proxy property
would look like [getattr(obj1, attr), getattr(obj2, attr)]
If the relationship is one-to-one or otherwise uselist=False, then simply: getattr(obj, attr)
• creator – optional.
When new items are added to this proxied collection, new instances of the class collected
by the target collection will be created. For list and set collections, the target class constructor will be called with the ‘value’ for the new instance. For dict types, two arguments
are passed: key and value.
If you want to construct instances differently, supply a creator function that takes arguments as above and returns instances.
For scalar relationships, creator() will be called if the target is None. If the target is
present, set operations are proxied to setattr() on the associated object.
If you have an associated object with multiple attributes, you may set up multiple association proxies mapping to different attributes. See the unit tests for examples, and
for examples of how creator() functions can be used to construct the scalar relationship
on-demand in this situation.
• **kw – Passes along any other keyword arguments to AssociationProxy.
class sqlalchemy.ext.associationproxy.AssociationProxy(target_collection,
creator=None,
set_factory=None,
proxy_factory=None,
proxy_bulk_set=None)
Bases: sqlalchemy.orm.interfaces._InspectionAttr
attr,
get-
A descriptor that presents a read/write view of an object attribute.
__init__(target_collection, attr, creator=None,
proxy_bulk_set=None)
Construct a new AssociationProxy.
getset_factory=None,
proxy_factory=None,
The association_proxy() function is provided as the usual entrypoint here, though
AssociationProxy can be instantiated and/or subclassed directly.
Parameters
• target_collection – Name of the collection we’ll proxy to, usually created
with relationship().
• attr – Attribute on the collected instances we’ll proxy for. For example, given a
target collection of [obj1, obj2], a list created by this proxy property would look like
[getattr(obj1, attr), getattr(obj2, attr)]
• creator – Optional. When new items are added to this proxied collection, new
instances of the class collected by the target collection will be created. For list and
set collections, the target class constructor will be called with the ‘value’ for the new
instance. For dict types, two arguments are passed: key and value.
If you want to construct instances differently, supply a ‘creator’ function that takes
arguments as above and returns instances.
• getset_factory – Optional. Proxied attribute access is automatically handled by
routines that get and set values based on the attr argument for this proxy.
236
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
If you would like to customize this behavior, you may supply a getset_factory callable
that produces a tuple of getter and setter functions. The factory is called with two
arguments, the abstract type of the underlying collection and this proxy instance.
• proxy_factory – Optional. The type of collection to emulate is determined by
sniffing the target collection. If your collection type can’t be determined by duck
typing or you’d like to use a different collection implementation, you may supply a
factory function to produce those collections. Only applicable to non-scalar relationships.
• proxy_bulk_set – Optional, use with proxy_factory. See the _set() method for
details.
any(criterion=None, **kwargs)
Produce a proxied ‘any’ expression using EXISTS.
This expression will be a composed product using the RelationshipProperty.Comparator.any()
and/or RelationshipProperty.Comparator.has() operators of the underlying proxied attributes.
attr
Return a tuple of (local_attr, remote_attr).
This attribute is convenient when specifying a join using Query.join() across two relationships:
sess.query(Parent).join(*Parent.proxied.attr)
New in version 0.7.3.
See also:
AssociationProxy.local_attr
AssociationProxy.remote_attr
contains(obj)
Produce a proxied ‘contains’ expression using EXISTS.
This expression will be a composed product using the RelationshipProperty.Comparator.any()
, RelationshipProperty.Comparator.has(), and/or RelationshipProperty.Comparator.contain
operators of the underlying proxied attributes.
extension_type = <symbol ‘ASSOCIATION_PROXY>
has(criterion=None, **kwargs)
Produce a proxied ‘has’ expression using EXISTS.
This expression will be a composed product using the RelationshipProperty.Comparator.any()
and/or RelationshipProperty.Comparator.has() operators of the underlying proxied attributes.
is_attribute = False
local_attr
The ‘local’ MapperProperty referenced by this AssociationProxy.
New in version 0.7.3.
See also:
AssociationProxy.attr
AssociationProxy.remote_attr
2.10. ORM Extensions
237
SQLAlchemy Documentation, Release 0.8.7
remote_attr
The ‘remote’ MapperProperty referenced by this AssociationProxy.
New in version 0.7.3.
See also:
AssociationProxy.attr
AssociationProxy.local_attr
scalar
Return True if this AssociationProxy proxies a scalar relationship on the local side.
target_class
The intermediary class handled by this AssociationProxy.
Intercepted append/set/assignment events will result in the generation of new instances of this class.
sqlalchemy.ext.associationproxy.ASSOCIATION_PROXY = <symbol ‘ASSOCIATION_PROXY>
2.10.2 Declarative
Synopsis
SQLAlchemy object-relational configuration involves the combination of Table, mapper(), and class objects to
define a mapped class. declarative allows all three to be expressed at once within the class declaration. As
much as possible, regular SQLAlchemy schema and ORM constructs are used directly, so that configuration between
“classical” ORM usage and declarative remain highly similar.
As a simple example:
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class SomeClass(Base):
__tablename__ = 'some_table'
id = Column(Integer, primary_key=True)
name = Column(String(50))
Above, the declarative_base() callable returns a new base class from which all mapped classes should inherit.
When the class definition is completed, a new Table and mapper() will have been generated.
The resulting table and mapper are accessible via __table__ and __mapper__ attributes on the SomeClass
class:
# access the mapped Table
SomeClass.__table__
# access the Mapper
SomeClass.__mapper__
Defining Attributes
In the previous example, the Column objects are automatically named with the name of the attribute to which they
are assigned.
238
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
To name columns explicitly with a name distinct from their mapped attribute, just give the column a name. Below, column “some_table_id” is mapped to the “id” attribute of SomeClass, but in SQL will be represented as “some_table_id”:
class SomeClass(Base):
__tablename__ = 'some_table'
id = Column("some_table_id", Integer, primary_key=True)
Attributes may be added to the class after its construction, and they will be added to the underlying Table and
mapper() definitions as appropriate:
SomeClass.data = Column('data', Unicode)
SomeClass.related = relationship(RelatedInfo)
Classes which are constructed using declarative can interact freely with classes that are mapped explicitly with
mapper().
It is recommended, though not required, that all tables share the same underlying MetaData object, so that stringconfigured ForeignKey references can be resolved without issue.
Accessing the MetaData
The declarative_base() base class contains a MetaData object where newly defined Table objects are collected. This object is intended to be accessed directly for MetaData-specific operations. Such as, to issue CREATE
statements for all tables:
engine = create_engine('sqlite://')
Base.metadata.create_all(engine)
declarative_base() can also receive a pre-existing MetaData object, which allows a declarative setup to be
associated with an already existing traditional collection of Table objects:
mymetadata = MetaData()
Base = declarative_base(metadata=mymetadata)
Configuring Relationships
Relationships to other classes are done in the usual way, with the added feature that the class specified to
relationship() may be a string name. The “class registry” associated with Base is used at mapper compilation time to resolve the name into the actual class object, which is expected to have been defined once the mapper
configuration is used:
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
name = Column(String(50))
addresses = relationship("Address", backref="user")
class Address(Base):
__tablename__ = 'addresses'
id = Column(Integer, primary_key=True)
2.10. ORM Extensions
239
SQLAlchemy Documentation, Release 0.8.7
email = Column(String(50))
user_id = Column(Integer, ForeignKey('users.id'))
Column constructs, since they are just that, are immediately usable, as below where we define a primary join condition
on the Address class using them:
class Address(Base):
__tablename__ = 'addresses'
id = Column(Integer, primary_key=True)
email = Column(String(50))
user_id = Column(Integer, ForeignKey('users.id'))
user = relationship(User, primaryjoin=user_id == User.id)
In addition to the main argument for relationship(), other arguments which depend upon the columns present
on an as-yet undefined class may also be specified as strings. These strings are evaluated as Python expressions. The
full namespace available within this evaluation includes all classes mapped for this declarative base, as well as the
contents of the sqlalchemy package, including expression functions like desc() and func:
class User(Base):
# ....
addresses = relationship("Address",
order_by="desc(Address.email)",
primaryjoin="Address.user_id==User.id")
For the case where more than one module contains a class of the same name, string class names can also be specified
as module-qualified paths within any of these string expressions:
class User(Base):
# ....
addresses = relationship("myapp.model.address.Address",
order_by="desc(myapp.model.address.Address.email)",
primaryjoin="myapp.model.address.Address.user_id=="
"myapp.model.user.User.id")
The qualified path can be any partial path that removes ambiguity between the names. For example, to disambiguate between myapp.model.address.Address and myapp.model.lookup.Address, we can specify
address.Address or lookup.Address:
class User(Base):
# ....
addresses = relationship("address.Address",
order_by="desc(address.Address.email)",
primaryjoin="address.Address.user_id=="
"User.id")
New in version 0.8: module-qualified paths can be used when specifying string arguments with Declarative, in order
to specify specific modules.
Two alternatives also exist to using string-based attributes. A lambda can also be used, which will be evaluated after
all mappers have been configured:
class User(Base):
# ...
addresses = relationship(lambda: Address,
240
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
order_by=lambda: desc(Address.email),
primaryjoin=lambda: Address.user_id==User.id)
Or, the relationship can be added to the class explicitly after the classes are available:
User.addresses = relationship(Address,
primaryjoin=Address.user_id==User.id)
Configuring Many-to-Many Relationships
Many-to-many relationships are also declared in the same way with declarative as with traditional mappings. The
secondary argument to relationship() is as usual passed a Table object, which is typically declared in the
traditional way. The Table usually shares the MetaData object used by the declarative base:
keywords = Table(
'keywords', Base.metadata,
Column('author_id', Integer, ForeignKey('authors.id')),
Column('keyword_id', Integer, ForeignKey('keywords.id'))
)
class Author(Base):
__tablename__ = 'authors'
id = Column(Integer, primary_key=True)
keywords = relationship("Keyword", secondary=keywords)
Like other relationship() arguments, a string is accepted as well, passing the string name of the table as defined
in the Base.metadata.tables collection:
class Author(Base):
__tablename__ = 'authors'
id = Column(Integer, primary_key=True)
keywords = relationship("Keyword", secondary="keywords")
As with traditional mapping, its generally not a good idea to use a Table as the “secondary” argument which is also
mapped to a class, unless the relationship() is declared with viewonly=True. Otherwise, the unit-of-work
system may attempt duplicate INSERT and DELETE statements against the underlying table.
Defining SQL Expressions
See SQL Expressions as Mapped Attributes for examples on declaratively mapping attributes to SQL expressions.
Table Configuration
Table arguments other than the name, metadata, and mapped Column arguments are specified using the
__table_args__ class attribute. This attribute accommodates both positional as well as keyword arguments that
are normally sent to the Table constructor. The attribute can be specified in one of two forms. One is as a dictionary:
class MyClass(Base):
__tablename__ = 'sometable'
__table_args__ = {'mysql_engine':'InnoDB'}
2.10. ORM Extensions
241
SQLAlchemy Documentation, Release 0.8.7
The other, a tuple, where each argument is positional (usually constraints):
class MyClass(Base):
__tablename__ = 'sometable'
__table_args__ = (
ForeignKeyConstraint(['id'], ['remote_table.id']),
UniqueConstraint('foo'),
)
Keyword arguments can be specified with the above form by specifying the last argument as a dictionary:
class MyClass(Base):
__tablename__ = 'sometable'
__table_args__ = (
ForeignKeyConstraint(['id'], ['remote_table.id']),
UniqueConstraint('foo'),
{'autoload':True}
)
Using a Hybrid Approach with __table__
As an alternative to __tablename__, a direct Table construct may be used. The Column objects, which in this
case require their names, will be added to the mapping just like a regular mapping to a table:
class MyClass(Base):
__table__ = Table('my_table', Base.metadata,
Column('id', Integer, primary_key=True),
Column('name', String(50))
)
__table__ provides a more focused point of control for establishing table metadata, while still getting most of the
benefits of using declarative. An application that uses reflection might want to load table metadata elsewhere and pass
it to declarative classes:
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
Base.metadata.reflect(some_engine)
class User(Base):
__table__ = metadata.tables['user']
class Address(Base):
__table__ = metadata.tables['address']
Some configuration schemes may find it more appropriate to use __table__, such as those which already take
advantage of the data-driven nature of Table to customize and/or automate schema definition.
Note that when the __table__ approach is used, the object is immediately usable as a plain Table within the class
declaration body itself, as a Python class is only another syntactical block. Below this is illustrated by using the id
column in the primaryjoin condition of a relationship():
242
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
class MyClass(Base):
__table__ = Table('my_table', Base.metadata,
Column('id', Integer, primary_key=True),
Column('name', String(50))
)
widgets = relationship(Widget,
primaryjoin=Widget.myclass_id==__table__.c.id)
Similarly, mapped attributes which refer to __table__ can be placed inline, as below where we assign the name
column to the attribute _name, generating a synonym for name:
from sqlalchemy.ext.declarative import synonym_for
class MyClass(Base):
__table__ = Table('my_table', Base.metadata,
Column('id', Integer, primary_key=True),
Column('name', String(50))
)
_name = __table__.c.name
@synonym_for("_name")
def name(self):
return "Name: %s" % _name
Using Reflection with Declarative
It’s easy to set up a Table that uses autoload=True in conjunction with a mapped class:
class MyClass(Base):
__table__ = Table('mytable', Base.metadata,
autoload=True, autoload_with=some_engine)
However, one improvement that can be made here is to not require the Engine to be available when classes are being
first declared. To achieve this, use the DeferredReflection mixin, which sets up mappings only after a special
prepare(engine) step is called:
from sqlalchemy.ext.declarative import declarative_base, DeferredReflection
Base = declarative_base(cls=DeferredReflection)
class Foo(Base):
__tablename__ = 'foo'
bars = relationship("Bar")
class Bar(Base):
__tablename__ = 'bar'
# illustrate overriding of "bar.foo_id" to have
# a foreign key constraint otherwise not
# reflected, such as when using MySQL
foo_id = Column(Integer, ForeignKey('foo.id'))
Base.prepare(e)
2.10. ORM Extensions
243
SQLAlchemy Documentation, Release 0.8.7
New in version 0.8: Added DeferredReflection.
Mapper Configuration
Declarative makes use of the mapper() function internally when it creates the mapping to the declared table. The
options for mapper() are passed directly through via the __mapper_args__ class attribute. As always, arguments
which reference locally mapped columns can reference them directly from within the class declaration:
from datetime import datetime
class Widget(Base):
__tablename__ = 'widgets'
id = Column(Integer, primary_key=True)
timestamp = Column(DateTime, nullable=False)
__mapper_args__ = {
'version_id_col': timestamp,
'version_id_generator': lambda v:datetime.now()
}
Inheritance Configuration
Declarative supports all three forms of inheritance as intuitively as possible. The inherits mapper keyword argument is not needed as declarative will determine this from the class itself. The various “polymorphic” keyword
arguments are specified using __mapper_args__.
Joined Table Inheritance
Joined table inheritance is defined as a subclass that defines its own table:
class Person(Base):
__tablename__ = 'people'
id = Column(Integer, primary_key=True)
discriminator = Column('type', String(50))
__mapper_args__ = {'polymorphic_on': discriminator}
class Engineer(Person):
__tablename__ = 'engineers'
__mapper_args__ = {'polymorphic_identity': 'engineer'}
id = Column(Integer, ForeignKey('people.id'), primary_key=True)
primary_language = Column(String(50))
Note that above, the Engineer.id attribute, since it shares the same attribute name as the Person.id attribute,
will in fact represent the people.id and engineers.id columns together, with the “Engineer.id” column
taking precedence if queried directly. To provide the Engineer class with an attribute that represents only the
engineers.id column, give it a different attribute name:
244
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
class Engineer(Person):
__tablename__ = 'engineers'
__mapper_args__ = {'polymorphic_identity': 'engineer'}
engineer_id = Column('id', Integer, ForeignKey('people.id'),
primary_key=True)
primary_language = Column(String(50))
Changed in version 0.7: joined table inheritance favors the subclass column over that of the superclass, such as
querying above for Engineer.id. Prior to 0.7 this was the reverse.
Single Table Inheritance
Single table inheritance is defined as a subclass that does not have its own table; you just leave out the __table__
and __tablename__ attributes:
class Person(Base):
__tablename__ = 'people'
id = Column(Integer, primary_key=True)
discriminator = Column('type', String(50))
__mapper_args__ = {'polymorphic_on': discriminator}
class Engineer(Person):
__mapper_args__ = {'polymorphic_identity': 'engineer'}
primary_language = Column(String(50))
When the above mappers are configured, the Person class is mapped to the people table before the
primary_language column is defined, and this column will not be included in its own mapping. When
Engineer then defines the primary_language column, the column is added to the people table so that
it is included in the mapping for Engineer and is also part of the table’s full set of columns. Columns
which are not mapped to Person are also excluded from any other single or joined inheriting classes using the
exclude_properties mapper argument. Below, Manager will have all the attributes of Person and Manager
but not the primary_language attribute of Engineer:
class Manager(Person):
__mapper_args__ = {'polymorphic_identity': 'manager'}
golf_swing = Column(String(50))
The attribute exclusion logic is provided by the exclude_properties mapper argument, and declarative’s default
behavior can be disabled by passing an explicit exclude_properties collection (empty or otherwise) to the
__mapper_args__.
Resolving Column Conflicts Note above that the primary_language and golf_swing columns are “moved
up” to be applied to Person.__table__, as a result of their declaration on a subclass that has no table of its own.
A tricky case comes up when two subclasses want to specify the same column, as below:
class Person(Base):
__tablename__ = 'people'
id = Column(Integer, primary_key=True)
discriminator = Column('type', String(50))
__mapper_args__ = {'polymorphic_on': discriminator}
class Engineer(Person):
__mapper_args__ = {'polymorphic_identity': 'engineer'}
2.10. ORM Extensions
245
SQLAlchemy Documentation, Release 0.8.7
start_date = Column(DateTime)
class Manager(Person):
__mapper_args__ = {'polymorphic_identity': 'manager'}
start_date = Column(DateTime)
Above, the start_date column declared on both Engineer and Manager will result in an error:
sqlalchemy.exc.ArgumentError: Column 'start_date' on class
<class '__main__.Manager'> conflicts with existing
column 'people.start_date'
In a situation like this, Declarative can’t be sure of the intent, especially if the start_date columns had, for
example, different types. A situation like this can be resolved by using declared_attr to define the Column
conditionally, taking care to return the existing column via the parent __table__ if it already exists:
from sqlalchemy.ext.declarative import declared_attr
class Person(Base):
__tablename__ = 'people'
id = Column(Integer, primary_key=True)
discriminator = Column('type', String(50))
__mapper_args__ = {'polymorphic_on': discriminator}
class Engineer(Person):
__mapper_args__ = {'polymorphic_identity': 'engineer'}
@declared_attr
def start_date(cls):
"Start date column, if not present already."
return Person.__table__.c.get('start_date', Column(DateTime))
class Manager(Person):
__mapper_args__ = {'polymorphic_identity': 'manager'}
@declared_attr
def start_date(cls):
"Start date column, if not present already."
return Person.__table__.c.get('start_date', Column(DateTime))
Above, when Manager is mapped, the start_date column is already present on the Person class. Declarative
lets us return that Column as a result in this case, where it knows to skip re-assigning the same column. If the
mapping is mis-configured such that the start_date column is accidentally re-assigned to a different table (such
as, if we changed Manager to be joined inheritance without fixing start_date), an error is raised which indicates
an existing Column is trying to be re-assigned to a different owning Table.
New in version 0.8: declared_attr can be used on a non-mixin class, and the returned Column or other mapped
attribute will be applied to the mapping as any other attribute. Previously, the resulting attribute would be ignored, and
also result in a warning being emitted when a subclass was created.
New in version 0.8: declared_attr, when used either with a mixin or non-mixin declarative class, can return an
existing Column already assigned to the parent Table, to indicate that the re-assignment of the Column should be
skipped, however should still be mapped on the target class, in order to resolve duplicate column conflicts.
The same concept can be used with mixin classes (see Mixin and Custom Base Classes):
246
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
class Person(Base):
__tablename__ = 'people'
id = Column(Integer, primary_key=True)
discriminator = Column('type', String(50))
__mapper_args__ = {'polymorphic_on': discriminator}
class HasStartDate(object):
@declared_attr
def start_date(cls):
return cls.__table__.c.get('start_date', Column(DateTime))
class Engineer(HasStartDate, Person):
__mapper_args__ = {'polymorphic_identity': 'engineer'}
class Manager(HasStartDate, Person):
__mapper_args__ = {'polymorphic_identity': 'manager'}
The above mixin checks the local __table__ attribute for the column. Because we’re using single table inheritance,
we’re sure that in this case, cls.__table__ refers to People.__table__. If we were mixing joined- and
single-table inheritance, we might want our mixin to check more carefully if cls.__table__ is really the Table
we’re looking for.
Concrete Table Inheritance
Concrete is defined as a subclass which has its own table and sets the concrete keyword argument to True:
class Person(Base):
__tablename__ = 'people'
id = Column(Integer, primary_key=True)
name = Column(String(50))
class Engineer(Person):
__tablename__ = 'engineers'
__mapper_args__ = {'concrete':True}
id = Column(Integer, primary_key=True)
primary_language = Column(String(50))
name = Column(String(50))
Usage of an abstract base class is a little less straightforward as it requires usage of polymorphic_union(), which
needs to be created with the Table objects before the class is built:
engineers = Table('engineers', Base.metadata,
Column('id', Integer, primary_key=True),
Column('name', String(50)),
Column('primary_language', String(50))
)
managers = Table('managers', Base.metadata,
Column('id', Integer, primary_key=True),
Column('name', String(50)),
Column('golf_swing', String(50))
)
punion = polymorphic_union({
'engineer':engineers,
'manager':managers
2.10. ORM Extensions
247
SQLAlchemy Documentation, Release 0.8.7
}, 'type', 'punion')
class Person(Base):
__table__ = punion
__mapper_args__ = {'polymorphic_on':punion.c.type}
class Engineer(Person):
__table__ = engineers
__mapper_args__ = {'polymorphic_identity':'engineer', 'concrete':True}
class Manager(Person):
__table__ = managers
__mapper_args__ = {'polymorphic_identity':'manager', 'concrete':True}
Using the Concrete Helpers Helper classes provides a simpler pattern for concrete inheritance. With these objects,
the __declare_last__ helper is used to configure the “polymorphic” loader for the mapper after all subclasses
have been declared.
New in version 0.7.3.
An abstract base can be declared using the AbstractConcreteBase class:
from sqlalchemy.ext.declarative import AbstractConcreteBase
class Employee(AbstractConcreteBase, Base):
pass
To have a concrete employee table, use ConcreteBase instead:
from sqlalchemy.ext.declarative import ConcreteBase
class Employee(ConcreteBase, Base):
__tablename__ = 'employee'
employee_id = Column(Integer, primary_key=True)
name = Column(String(50))
__mapper_args__ = {
'polymorphic_identity':'employee',
'concrete':True}
Either Employee base can be used in the normal fashion:
class Manager(Employee):
__tablename__ = 'manager'
employee_id = Column(Integer, primary_key=True)
name = Column(String(50))
manager_data = Column(String(40))
__mapper_args__ = {
'polymorphic_identity':'manager',
'concrete':True}
class Engineer(Employee):
__tablename__ = 'engineer'
employee_id = Column(Integer, primary_key=True)
name = Column(String(50))
engineer_info = Column(String(40))
248
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
__mapper_args__ = {'polymorphic_identity':'engineer',
'concrete':True}
Mixin and Custom Base Classes
A common need when using declarative is to share some functionality, such as a set of common columns, some
common table options, or other mapped properties, across many classes. The standard Python idioms for this is to
have the classes inherit from a base which includes these common features.
When using declarative, this idiom is allowed via the usage of a custom declarative base class, as well as a
“mixin” class which is inherited from in addition to the primary base. Declarative includes several helper features to
make this work in terms of how mappings are declared. An example of some commonly mixed-in idioms is below:
from sqlalchemy.ext.declarative import declared_attr
class MyMixin(object):
@declared_attr
def __tablename__(cls):
return cls.__name__.lower()
__table_args__ = {'mysql_engine': 'InnoDB'}
__mapper_args__= {'always_refresh': True}
id =
Column(Integer, primary_key=True)
class MyModel(MyMixin, Base):
name = Column(String(1000))
Where above, the class MyModel will contain an “id” column as the primary key, a __tablename__ attribute that
derives from the name of the class itself, as well as __table_args__ and __mapper_args__ defined by the
MyMixin mixin class.
There’s no fixed convention over whether MyMixin precedes Base or not. Normal Python method resolution rules
apply, and the above example would work just as well with:
class MyModel(Base, MyMixin):
name = Column(String(1000))
This works because Base here doesn’t define any of the variables that MyMixin defines, i.e. __tablename__,
__table_args__, id, etc. If the Base did define an attribute of the same name, the class placed first in the inherits
list would determine which attribute is used on the newly defined class.
Augmenting the Base
In addition to using a pure mixin, most of the techniques in this section can also be applied to the base class itself, for
patterns that should apply to all classes derived from a particular base. This is achieved using the cls argument of the
declarative_base() function:
from sqlalchemy.ext.declarative import declared_attr
class Base(object):
@declared_attr
2.10. ORM Extensions
249
SQLAlchemy Documentation, Release 0.8.7
def __tablename__(cls):
return cls.__name__.lower()
__table_args__ = {'mysql_engine': 'InnoDB'}
id =
Column(Integer, primary_key=True)
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base(cls=Base)
class MyModel(Base):
name = Column(String(1000))
Where above, MyModel and all other classes that derive from Base will have a table name derived from the class
name, an id primary key column, as well as the “InnoDB” engine for MySQL.
Mixing in Columns
The most basic way to specify a column on a mixin is by simple declaration:
class TimestampMixin(object):
created_at = Column(DateTime, default=func.now())
class MyModel(TimestampMixin, Base):
__tablename__ = 'test'
id = Column(Integer, primary_key=True)
name = Column(String(1000))
Where above, all declarative classes that include TimestampMixin will also have a column created_at that
applies a timestamp to all row insertions.
Those familiar with the SQLAlchemy expression language know that the object identity of clause elements defines
their role in a schema. Two Table objects a and b may both have a column called id, but the way these are
differentiated is that a.c.id and b.c.id are two distinct Python objects, referencing their parent tables a and b
respectively.
In the case of the mixin column, it seems that only one Column object is explicitly created, yet the ultimate
created_at column above must exist as a distinct Python object for each separate destination class. To accomplish this, the declarative extension creates a copy of each Column object encountered on a class that is detected as a
mixin.
This copy mechanism is limited to simple columns that have no foreign keys, as a ForeignKey itself contains
references to columns which can’t be properly recreated at this level. For columns that have foreign keys, as well as
for the variety of mapper-level constructs that require destination-explicit context, the declared_attr decorator is
provided so that patterns common to many classes can be defined as callables:
from sqlalchemy.ext.declarative import declared_attr
class ReferenceAddressMixin(object):
@declared_attr
def address_id(cls):
return Column(Integer, ForeignKey('address.id'))
class User(ReferenceAddressMixin, Base):
250
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
Where above, the address_id class-level callable is executed at the point at which the User class is constructed,
and the declarative extension can use the resulting Column object as returned by the method without the need to copy
it.
Changed in version >: 0.6.5 Rename 0.6.5 sqlalchemy.util.classproperty into declared_attr.
Columns generated by declared_attr can also be referenced by __mapper_args__ to a limited degree, currently by polymorphic_on and version_id_col, by specifying the classdecorator itself into the dictionary the declarative extension will resolve them at class construction time:
class MyMixin:
@declared_attr
def type_(cls):
return Column(String(50))
__mapper_args__= {'polymorphic_on':type_}
class MyModel(MyMixin, Base):
__tablename__='test'
id = Column(Integer, primary_key=True)
Mixing in Relationships
Relationships created by relationship() are provided with declarative mixin classes exclusively using the
declared_attr approach, eliminating any ambiguity which could arise when copying a relationship and its possibly column-bound contents. Below is an example which combines a foreign key column and a relationship so that
two classes Foo and Bar can both be configured to reference a common target class via many-to-one:
class RefTargetMixin(object):
@declared_attr
def target_id(cls):
return Column('target_id', ForeignKey('target.id'))
@declared_attr
def target(cls):
return relationship("Target")
class Foo(RefTargetMixin, Base):
__tablename__ = 'foo'
id = Column(Integer, primary_key=True)
class Bar(RefTargetMixin, Base):
__tablename__ = 'bar'
id = Column(Integer, primary_key=True)
class Target(Base):
__tablename__ = 'target'
id = Column(Integer, primary_key=True)
relationship() definitions which require explicit primaryjoin, order_by etc. expressions should use the string
forms for these arguments, so that they are evaluated as late as possible. To reference the mixin class in these expressions, use the given cls to get its name:
2.10. ORM Extensions
251
SQLAlchemy Documentation, Release 0.8.7
class RefTargetMixin(object):
@declared_attr
def target_id(cls):
return Column('target_id', ForeignKey('target.id'))
@declared_attr
def target(cls):
return relationship("Target",
primaryjoin="Target.id==%s.target_id" % cls.__name__
)
Mixing in deferred(), column_property(), and other MapperProperty classes
Like relationship(), all MapperProperty subclasses such as deferred(), column_property(),
etc. ultimately involve references to columns, and therefore, when used with declarative mixins, have the
declared_attr requirement so that no reliance on copying is needed:
class SomethingMixin(object):
@declared_attr
def dprop(cls):
return deferred(Column(Integer))
class Something(SomethingMixin, Base):
__tablename__ = "something"
Mixing in Association Proxy and Other Attributes
Mixins can specify user-defined attributes as well as other extension units such as association_proxy(). The
usage of declared_attr is required in those cases where the attribute must be tailored specifically to the target
subclass. An example is when constructing multiple association_proxy() attributes which each target a different type of child object. Below is an association_proxy() / mixin example which provides a scalar list of
string values to an implementing class:
from
from
from
from
sqlalchemy import Column, Integer, ForeignKey, String
sqlalchemy.orm import relationship
sqlalchemy.ext.associationproxy import association_proxy
sqlalchemy.ext.declarative import declarative_base, declared_attr
Base = declarative_base()
class HasStringCollection(object):
@declared_attr
def _strings(cls):
class StringAttribute(Base):
__tablename__ = cls.string_table_name
id = Column(Integer, primary_key=True)
value = Column(String(50), nullable=False)
parent_id = Column(Integer,
ForeignKey('%s.id' % cls.__tablename__),
nullable=False)
def __init__(self, value):
252
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
self.value = value
return relationship(StringAttribute)
@declared_attr
def strings(cls):
return association_proxy('_strings', 'value')
class TypeA(HasStringCollection, Base):
__tablename__ = 'type_a'
string_table_name = 'type_a_strings'
id = Column(Integer(), primary_key=True)
class TypeB(HasStringCollection, Base):
__tablename__ = 'type_b'
string_table_name = 'type_b_strings'
id = Column(Integer(), primary_key=True)
Above, the HasStringCollection mixin produces a relationship() which refers to a newly generated
class called StringAttribute. The StringAttribute class is generated with it’s own Table definition
which is local to the parent class making usage of the HasStringCollection mixin. It also produces an
association_proxy() object which proxies references to the strings attribute onto the value attribute of
each StringAttribute instance.
TypeA or TypeB can be instantiated given the constructor argument strings, a list of strings:
ta = TypeA(strings=['foo', 'bar'])
tb = TypeA(strings=['bat', 'bar'])
This list will generate a collection of StringAttribute objects, which are persisted into a table that’s local to
either the type_a_strings or type_b_strings table:
>>> print ta._strings
[<__main__.StringAttribute object at 0x10151cd90>,
<__main__.StringAttribute object at 0x10151ce10>]
When constructing the association_proxy(), the declared_attr decorator must be used so that a distinct
association_proxy() object is created for each of the TypeA and TypeB classes.
New in version 0.8: declared_attr is usable with non-mapped attributes, including user-defined attributes as well
as association_proxy().
Controlling table inheritance with mixins
The __tablename__ attribute in conjunction with the hierarchy of classes involved in a declarative mixin scenario
controls what type of table inheritance, if any, is configured by the declarative extension.
If the __tablename__ is computed by a mixin, you may need to control which classes get the computed attribute
in order to get the type of table inheritance you require.
For example, if you had a mixin that computes __tablename__ but where you wanted to use that mixin in a single
table inheritance hierarchy, you can explicitly specify __tablename__ as None to indicate that the class should
not have a table mapped:
2.10. ORM Extensions
253
SQLAlchemy Documentation, Release 0.8.7
from sqlalchemy.ext.declarative import declared_attr
class Tablename:
@declared_attr
def __tablename__(cls):
return cls.__name__.lower()
class Person(Tablename, Base):
id = Column(Integer, primary_key=True)
discriminator = Column('type', String(50))
__mapper_args__ = {'polymorphic_on': discriminator}
class Engineer(Person):
__tablename__ = None
__mapper_args__ = {'polymorphic_identity': 'engineer'}
primary_language = Column(String(50))
Alternatively, you can make the mixin intelligent enough to only return a __tablename__ in the event that no table
is already mapped in the inheritance hierarchy. To help with this, a has_inherited_table() helper function is
provided that returns True if a parent class already has a mapped table.
As an example, here’s a mixin that will only allow single table inheritance:
from sqlalchemy.ext.declarative import declared_attr
from sqlalchemy.ext.declarative import has_inherited_table
class Tablename(object):
@declared_attr
def __tablename__(cls):
if has_inherited_table(cls):
return None
return cls.__name__.lower()
class Person(Tablename, Base):
id = Column(Integer, primary_key=True)
discriminator = Column('type', String(50))
__mapper_args__ = {'polymorphic_on': discriminator}
class Engineer(Person):
primary_language = Column(String(50))
__mapper_args__ = {'polymorphic_identity': 'engineer'}
Combining Table/Mapper Arguments from Multiple Mixins
In the case of __table_args__ or __mapper_args__ specified with declarative mixins, you may want to combine some parameters from several mixins with those you wish to define on the class iteself. The declared_attr
decorator can be used here to create user-defined collation routines that pull from multiple collections:
from sqlalchemy.ext.declarative import declared_attr
class MySQLSettings(object):
__table_args__ = {'mysql_engine':'InnoDB'}
class MyOtherMixin(object):
__table_args__ = {'info':'foo'}
254
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
class MyModel(MySQLSettings, MyOtherMixin, Base):
__tablename__='my_model'
@declared_attr
def __table_args__(cls):
args = dict()
args.update(MySQLSettings.__table_args__)
args.update(MyOtherMixin.__table_args__)
return args
id =
Column(Integer, primary_key=True)
Creating Indexes with Mixins
To define a named, potentially multicolumn Index that applies to all tables derived from a mixin, use the “inline”
form of Index and establish it as part of __table_args__:
class MyMixin(object):
a = Column(Integer)
b = Column(Integer)
@declared_attr
def __table_args__(cls):
return (Index('test_idx_%s' % cls.__tablename__, 'a', 'b'),)
class MyModel(MyMixin, Base):
__tablename__ = 'atable'
c = Column(Integer,primary_key=True)
Special Directives
__declare_last__()
The __declare_last__() hook allows definition of a class level function that is automatically called by the
MapperEvents.after_configured() event, which occurs after mappings are assumed to be completed and
the ‘configure’ step has finished:
class MyClass(Base):
@classmethod
def __declare_last__(cls):
""
# do something with mappings
New in version 0.7.3.
__abstract__
__abstract__ causes declarative to skip the production of a table or mapper for the class entirely. A class can
be added within a hierarchy in the same way as mixin (see Mixin and Custom Base Classes), allowing subclasses to
extend just from the special class:
2.10. ORM Extensions
255
SQLAlchemy Documentation, Release 0.8.7
class SomeAbstractBase(Base):
__abstract__ = True
def some_helpful_method(self):
""
@declared_attr
def __mapper_args__(cls):
return {"helpful mapper arguments":True}
class MyMappedClass(SomeAbstractBase):
""
One possible use of __abstract__ is to use a distinct MetaData for different bases:
Base = declarative_base()
class DefaultBase(Base):
__abstract__ = True
metadata = MetaData()
class OtherBase(Base):
__abstract__ = True
metadata = MetaData()
Above, classes which inherit from DefaultBase will use one MetaData as the registry of tables, and those which
inherit from OtherBase will use a different one. The tables themselves can then be created perhaps within distinct
databases:
DefaultBase.metadata.create_all(some_engine)
OtherBase.metadata_create_all(some_other_engine)
New in version 0.7.3.
Class Constructor
As a convenience feature, the declarative_base() sets a default constructor on classes which takes keyword
arguments, and assigns them to the named attributes:
e = Engineer(primary_language='python')
Sessions
Note that declarative does nothing special with sessions, and is only intended as an easier way to configure
mappers and Table objects. A typical application setup using scoped_session might look like:
engine = create_engine('postgresql://scott:[email protected]/test')
Session = scoped_session(sessionmaker(autocommit=False,
autoflush=False,
bind=engine))
Base = declarative_base()
Mapped instances then make usage of Session in the usual way.
256
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
API Reference
sqlalchemy.ext.declarative.declarative_base(bind=None, metadata=None, mapper=None,
cls=<type
‘object’>,
name=’Base’,
constructor=<function
__init__>,
class_registry=None,
metaclass=<class
‘sqlalchemy.ext.declarative.api.DeclarativeMeta’>)
Construct a base class for declarative class definitions.
The new base class will be given a metaclass that produces appropriate Table objects and makes the appropriate
mapper() calls based on the information provided declaratively in the class and any subclasses of the class.
Parameters
• bind – An optional Connectable, will be assigned the bind attribute on the
MetaData instance.
• metadata – An optional MetaData instance. All Table objects implicitly declared
by subclasses of the base will share this MetaData. A MetaData instance will be created
if none is provided. The MetaData instance will be available via the metadata attribute
of the generated declarative base class.
• mapper – An optional callable, defaults to mapper(). Will be used to map subclasses
to their Tables.
• cls – Defaults to object. A type to use as the base for the generated declarative base
class. May be a class or tuple of classes.
• name – Defaults to Base. The display name for the generated class. Customizing this
is not required, but can improve clarity in tracebacks and debugging.
• constructor – Defaults to _declarative_constructor(), an __init__ implementation that assigns **kwargs for declared fields and relationships to an instance.
If None is supplied, no __init__ will be provided and construction will fall back to
cls.__init__ by way of the normal Python semantics.
• class_registry – optional dictionary that will serve as the registry of class
names-> mapped classes when string names are used to identify classes inside of
relationship() and others. Allows two or more declarative base classes to share
the same registry of class names for simplified inter-base relationships.
• metaclass – Defaults to DeclarativeMeta. A metaclass or __metaclass__ compatible callable to use as the meta type of the generated declarative base class.
See also:
as_declarative()
sqlalchemy.ext.declarative.as_declarative(**kw)
Class decorator for declarative_base().
Provides a syntactical shortcut to the cls argument sent to declarative_base(), allowing the base class
to be converted in-place to a “declarative” base:
from sqlalchemy.ext.declarative import as_declarative
@as_declarative()
class Base(object):
@declared_attr
def __tablename__(cls):
return cls.__name__.lower()
id = Column(Integer, primary_key=True)
2.10. ORM Extensions
257
SQLAlchemy Documentation, Release 0.8.7
class MyMappedClass(Base):
# ...
All keyword arguments passed to as_declarative() are passed along to declarative_base().
New in version 0.8.3.
See also:
declarative_base()
class sqlalchemy.ext.declarative.declared_attr(fget, *arg, **kw)
Bases: sqlalchemy.orm.interfaces._MappedAttribute, __builtin__.property
Mark a class-level method as representing the definition of a mapped property or special declarative member
name.
@declared_attr turns the attribute into a scalar-like property that can be invoked from the uninstantiated class.
Declarative treats attributes specifically marked with @declared_attr as returning a construct that is specific to
mapping or declarative table configuration. The name of the attribute is that of what the non-dynamic version
of the attribute would be.
@declared_attr is more often than not applicable to mixins, to define relationships that are to be applied to
different implementors of the class:
class ProvidesUser(object):
"A mixin that adds a 'user' relationship to classes."
@declared_attr
def user(self):
return relationship("User")
It also can be applied to mapped classes, such as to provide a “polymorphic” scheme for inheritance:
class Employee(Base):
id = Column(Integer, primary_key=True)
type = Column(String(50), nullable=False)
@declared_attr
def __tablename__(cls):
return cls.__name__.lower()
@declared_attr
def __mapper_args__(cls):
if cls.__name__ == 'Employee':
return {
"polymorphic_on":cls.type,
"polymorphic_identity":"Employee"
}
else:
return {"polymorphic_identity":cls.__name__}
Changed in version 0.8: declared_attr can be used with non-ORM or extension attributes, such as userdefined attributes or association_proxy() objects, which will be assigned to the class at class construction time.
sqlalchemy.ext.declarative.api._declarative_constructor(self, **kwargs)
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in kwargs.
258
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any
mapped columns or relationships.
sqlalchemy.ext.declarative.has_inherited_table(cls)
Given a class, return True if any of the classes it inherits from has a mapped table, otherwise return False.
sqlalchemy.ext.declarative.synonym_for(name, map_column=False)
Decorator, make a Python @property a query synonym for a column.
A decorator version of synonym(). The function being decorated is the ‘descriptor’, otherwise passes its
arguments through to synonym():
@synonym_for('col')
@property
def prop(self):
return 'special sauce'
The regular synonym() is also usable directly in a declarative setting and may be convenient for read/write
properties:
prop = synonym('col', descriptor=property(_read_prop, _write_prop))
sqlalchemy.ext.declarative.comparable_using(comparator_factory)
Decorator, allow a Python @property to be used in query criteria.
This is a decorator front end to comparable_property() that passes through the comparator_factory and
the function being decorated:
@comparable_using(MyComparatorType)
@property
def prop(self):
return 'special sauce'
The regular comparable_property() is also usable directly in a declarative setting and may be convenient
for read/write properties:
prop = comparable_property(MyComparatorType)
sqlalchemy.ext.declarative.instrument_declarative(cls, registry, metadata)
Given a class, configure the class declaratively, using the given registry, which can be any dictionary, and
MetaData object.
class sqlalchemy.ext.declarative.AbstractConcreteBase
Bases: sqlalchemy.ext.declarative.api.ConcreteBase
A helper class for ‘concrete’ declarative mappings.
AbstractConcreteBase will use the polymorphic_union() function automatically, against all tables
mapped as a subclass to this class. The function is called via the __declare_last__() function, which is
essentially a hook for the MapperEvents.after_configured() event.
AbstractConcreteBase does not produce a mapped table for the class itself.
ConcreteBase, which does.
Compare to
Example:
from sqlalchemy.ext.declarative import AbstractConcreteBase
class Employee(AbstractConcreteBase, Base):
pass
class Manager(Employee):
2.10. ORM Extensions
259
SQLAlchemy Documentation, Release 0.8.7
__tablename__ = 'manager'
employee_id = Column(Integer, primary_key=True)
name = Column(String(50))
manager_data = Column(String(40))
__mapper_args__ = {
'polymorphic_identity':'manager',
'concrete':True}
class sqlalchemy.ext.declarative.ConcreteBase
A helper class for ‘concrete’ declarative mappings.
ConcreteBase will use the polymorphic_union() function automatically, against all tables mapped as
a subclass to this class. The function is called via the __declare_last__() function, which is essentially
a hook for the MapperEvents.after_configured() event.
ConcreteBase produces a mapped table for the class itself. Compare to AbstractConcreteBase,
which does not.
Example:
from sqlalchemy.ext.declarative import ConcreteBase
class Employee(ConcreteBase, Base):
__tablename__ = 'employee'
employee_id = Column(Integer, primary_key=True)
name = Column(String(50))
__mapper_args__ = {
'polymorphic_identity':'employee',
'concrete':True}
class Manager(Employee):
__tablename__ = 'manager'
employee_id = Column(Integer, primary_key=True)
name = Column(String(50))
manager_data = Column(String(40))
__mapper_args__ = {
'polymorphic_identity':'manager',
'concrete':True}
class sqlalchemy.ext.declarative.DeferredReflection
A helper class for construction of mappings based on a deferred reflection step.
Normally, declarative can be used with reflection by setting a Table object using autoload=True as the
__table__ attribute on a declarative class. The caveat is that the Table must be fully reflected, or at the very
least have a primary key column, at the point at which a normal declarative mapping is constructed, meaning
the Engine must be available at class declaration time.
The DeferredReflection mixin moves the construction of mappers to be at a later point, after a specific
method is called which first reflects all Table objects created so far. Classes can define it as such:
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.ext.declarative import DeferredReflection
Base = declarative_base()
class MyClass(DeferredReflection, Base):
__tablename__ = 'mytable'
Above, MyClass is not yet mapped. After a series of classes have been defined in the above fashion, all tables
can be reflected and mappings created using DeferredReflection.prepare():
260
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
engine = create_engine("someengine://...")
DeferredReflection.prepare(engine)
The DeferredReflection mixin can be applied to individual classes, used as the base for the declarative
base itself, or used in a custom abstract class. Using an abstract base allows that only a subset of classes to be
prepared for a particular prepare step, which is necessary for applications that use more than one engine. For
example, if an application has two engines, you might use two bases, and prepare each separately, e.g.:
class ReflectedOne(DeferredReflection, Base):
__abstract__ = True
class ReflectedTwo(DeferredReflection, Base):
__abstract__ = True
class MyClass(ReflectedOne):
__tablename__ = 'mytable'
class MyOtherClass(ReflectedOne):
__tablename__ = 'myothertable'
class YetAnotherClass(ReflectedTwo):
__tablename__ = 'yetanothertable'
# ... etc.
Above, the class hierarchies for ReflectedOne and ReflectedTwo can be configured separately:
ReflectedOne.prepare(engine_one)
ReflectedTwo.prepare(engine_two)
New in version 0.8.
2.10.3 Mutation Tracking
Provide support for tracking of in-place changes to scalar values, which are propagated into ORM change events on
owning parent objects.
New in version 0.7: sqlalchemy.ext.mutable replaces SQLAlchemy’s legacy approach to in-place mutations
of scalar values; see Mutation event extension, supersedes “mutable=True”.
Establishing Mutability on Scalar Column Values
A typical example of a “mutable” structure is a Python dictionary. Following the example introduced in Column and
Data Types, we begin with a custom type that marshals Python dictionaries into JSON strings before being persisted:
from sqlalchemy.types import TypeDecorator, VARCHAR
import json
class JSONEncodedDict(TypeDecorator):
"Represents an immutable structure as a json-encoded string."
impl = VARCHAR
def process_bind_param(self, value, dialect):
if value is not None:
value = json.dumps(value)
2.10. ORM Extensions
261
SQLAlchemy Documentation, Release 0.8.7
return value
def process_result_value(self, value, dialect):
if value is not None:
value = json.loads(value)
return value
The usage of json is only for the purposes of example. The sqlalchemy.ext.mutable extension can be used
with any type whose target Python type may be mutable, including PickleType, postgresql.ARRAY, etc.
When using the sqlalchemy.ext.mutable extension, the value itself tracks all parents which reference it. Below, we illustrate the a simple version of the MutableDict dictionary object, which applies the Mutable mixin to
a plain Python dictionary:
from sqlalchemy.ext.mutable import Mutable
class MutableDict(Mutable, dict):
@classmethod
def coerce(cls, key, value):
"Convert plain dictionaries to MutableDict."
if not isinstance(value, MutableDict):
if isinstance(value, dict):
return MutableDict(value)
# this call will raise ValueError
return Mutable.coerce(key, value)
else:
return value
def __setitem__(self, key, value):
"Detect dictionary set events and emit change events."
dict.__setitem__(self, key, value)
self.changed()
def __delitem__(self, key):
"Detect dictionary del events and emit change events."
dict.__delitem__(self, key)
self.changed()
The above dictionary class takes the approach of subclassing the Python built-in dict to produce a dict subclass
which routes all mutation events through __setitem__. There are variants on this approach, such as subclassing
UserDict.UserDict or collections.MutableMapping; the part that’s important to this example is that
the Mutable.changed() method is called whenever an in-place change to the datastructure takes place.
We also redefine the Mutable.coerce() method which will be used to convert any values that are not instances
of MutableDict, such as the plain dictionaries returned by the json module, into the appropriate type. Defining this method is optional; we could just as well created our JSONEncodedDict such that it always returns an
instance of MutableDict, and additionally ensured that all calling code uses MutableDict explicitly. When
Mutable.coerce() is not overridden, any values applied to a parent object which are not instances of the mutable
type will raise a ValueError.
Our new MutableDict type offers a class method as_mutable() which we can use within column metadata to
associate with types. This method grabs the given type object or class and associates a listener that will detect all
future mappings of this type, applying event listening instrumentation to the mapped attribute. Such as, with classical
262
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
table metadata:
from sqlalchemy import Table, Column, Integer
my_data = Table('my_data', metadata,
Column('id', Integer, primary_key=True),
Column('data', MutableDict.as_mutable(JSONEncodedDict))
)
Above, as_mutable() returns an instance of JSONEncodedDict (if the type object was not an instance already),
which will intercept any attributes which are mapped against this type. Below we establish a simple mapping against
the my_data table:
from sqlalchemy import mapper
class MyDataClass(object):
pass
# associates mutation listeners with MyDataClass.data
mapper(MyDataClass, my_data)
The MyDataClass.data member will now be notified of in place changes to its value.
There’s no difference in usage when using declarative:
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class MyDataClass(Base):
__tablename__ = 'my_data'
id = Column(Integer, primary_key=True)
data = Column(MutableDict.as_mutable(JSONEncodedDict))
Any in-place changes to the MyDataClass.data member will flag the attribute as “dirty” on the parent object:
>>> from sqlalchemy.orm import Session
>>>
>>>
>>>
>>>
sess = Session()
m1 = MyDataClass(data={'value1':'foo'})
sess.add(m1)
sess.commit()
>>> m1.data['value1'] = 'bar'
>>> assert m1 in sess.dirty
True
The MutableDict can be associated with all future instances of JSONEncodedDict in one step, using associate_with(). This is similar to as_mutable() except it will intercept all occurrences of
MutableDict in all mappings unconditionally, without the need to declare it individually:
MutableDict.associate_with(JSONEncodedDict)
class MyDataClass(Base):
__tablename__ = 'my_data'
2.10. ORM Extensions
263
SQLAlchemy Documentation, Release 0.8.7
id = Column(Integer, primary_key=True)
data = Column(JSONEncodedDict)
Supporting Pickling
The key to the sqlalchemy.ext.mutable extension relies upon the placement of a
weakref.WeakKeyDictionary upon the value object, which stores a mapping of parent mapped objects
keyed to the attribute name under which they are associated with this value. WeakKeyDictionary objects are
not picklable, due to the fact that they contain weakrefs and function callbacks. In our case, this is a good thing,
since if this dictionary were picklable, it could lead to an excessively large pickle size for our value objects that are
pickled by themselves outside of the context of the parent. The developer responsibility here is only to provide a
__getstate__ method that excludes the _parents() collection from the pickle stream:
class MyMutableType(Mutable):
def __getstate__(self):
d = self.__dict__.copy()
d.pop('_parents', None)
return d
With our dictionary example, we need to return the contents of the dict itself (and also restore them on __setstate__):
class MutableDict(Mutable, dict):
# ....
def __getstate__(self):
return dict(self)
def __setstate__(self, state):
self.update(state)
In the case that our mutable value object is pickled as it is attached to one or more parent objects that are also part
of the pickle, the Mutable mixin will re-establish the Mutable._parents collection on each value object as the
owning parents themselves are unpickled.
Establishing Mutability on Composites
Composites are a special ORM feature which allow a single scalar attribute to be assigned an object value which
represents information “composed” from one or more columns from the underlying mapped table. The usual example
is that of a geometric “point”, and is introduced in Composite Column Types.
Changed in version 0.7: The internals of orm.composite() have been greatly simplified and in-place mutation detection is no longer enabled by default; instead, the user-defined value must detect changes on its own and
propagate them to all owning parents. The sqlalchemy.ext.mutable extension provides the helper class
MutableComposite, which is a slight variant on the Mutable class.
As is the case with Mutable, the user-defined composite class subclasses MutableComposite as a mixin, and
detects and delivers change events to its parents via the MutableComposite.changed() method. In the case
of a composite class, the detection is usually via the usage of Python descriptors (i.e. @property), or alternatively
via the special Python method __setattr__(). Below we expand upon the Point class introduced in Composite
Column Types to subclass MutableComposite and to also route attribute set events via __setattr__ to the
MutableComposite.changed() method:
264
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
from sqlalchemy.ext.mutable import MutableComposite
class Point(MutableComposite):
def __init__(self, x, y):
self.x = x
self.y = y
def __setattr__(self, key, value):
"Intercept set events"
# set the attribute
object.__setattr__(self, key, value)
# alert all parents to the change
self.changed()
def __composite_values__(self):
return self.x, self.y
def __eq__(self, other):
return isinstance(other, Point) and \
other.x == self.x and \
other.y == self.y
def __ne__(self, other):
return not self.__eq__(other)
The MutableComposite class uses a Python metaclass to automatically establish listeners for any usage of
orm.composite() that specifies our Point type. Below, when Point is mapped to the Vertex class, listeners are established which will route change events from Point objects to each of the Vertex.start and
Vertex.end attributes:
from sqlalchemy.orm import composite, mapper
from sqlalchemy import Table, Column
vertices = Table('vertices', metadata,
Column('id', Integer, primary_key=True),
Column('x1', Integer),
Column('y1', Integer),
Column('x2', Integer),
Column('y2', Integer),
)
class Vertex(object):
pass
mapper(Vertex, vertices, properties={
'start': composite(Point, vertices.c.x1, vertices.c.y1),
'end': composite(Point, vertices.c.x2, vertices.c.y2)
})
Any in-place changes to the Vertex.start or Vertex.end members will flag the attribute as “dirty” on the
parent object:
>>> from sqlalchemy.orm import Session
2.10. ORM Extensions
265
SQLAlchemy Documentation, Release 0.8.7
>>>
>>>
>>>
>>>
sess = Session()
v1 = Vertex(start=Point(3, 4), end=Point(12, 15))
sess.add(v1)
sess.commit()
>>> v1.end.x = 8
>>> assert v1 in sess.dirty
True
Coercing Mutable Composites
The MutableBase.coerce() method is also supported on composite types.
In the case of
MutableComposite, the MutableBase.coerce() method is only called for attribute set operations, not load
operations. Overriding the MutableBase.coerce() method is essentially equivalent to using a validates()
validation routine for all attributes which make use of the custom composite type:
class Point(MutableComposite):
# other Point methods
# ...
def coerce(cls, key, value):
if isinstance(value, tuple):
value = Point(*value)
elif not isinstance(value, Point):
raise ValueError("tuple or Point expected")
return value
New in version 0.7.10,0.8.0b2: Support for the MutableBase.coerce() method in conjunction with objects of
type MutableComposite.
Supporting Pickling
As is the case with Mutable, the MutableComposite helper class uses a weakref.WeakKeyDictionary
available via the MutableBase._parents() attribute which isn’t picklable. If we need to pickle instances of
Point or its owning class Vertex, we at least need to define a __getstate__ that doesn’t include the _parents
dictionary. Below we define both a __getstate__ and a __setstate__ that package up the minimal form of
our Point class:
class Point(MutableComposite):
# ...
def __getstate__(self):
return self.x, self.y
def __setstate__(self, state):
self.x, self.y = state
As with Mutable, the MutableComposite augments the pickling process of the parent’s object-relational state
so that the MutableBase._parents() collection is restored to all Point objects.
266
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
API Reference
class sqlalchemy.ext.mutable.MutableBase
Common base class to Mutable and MutableComposite.
_parents
Dictionary of parent object->attribute name on the parent.
This attribute is a so-called “memoized” property.
It initializes itself with a new
weakref.WeakKeyDictionary the first time it is accessed, returning the same object upon
subsequent access.
classmethod coerce(key, value)
Given a value, coerce it into the target type.
Can be overridden by custom subclasses to coerce incoming data into a particular type.
By default, raises ValueError.
This method is called in different scenarios depending on if the parent class is of type Mutable or of
type MutableComposite. In the case of the former, it is called for both attribute-set operations as
well as during ORM loading operations. For the latter, it is only called during attribute-set operations; the
mechanics of the composite() construct handle coercion during load operations.
Parameters
• key – string name of the ORM-mapped attribute being set.
• value – the incoming value.
Returns the method should return the coerced value, or raise ValueError if the coercion
cannot be completed.
class sqlalchemy.ext.mutable.Mutable
Bases: sqlalchemy.ext.mutable.MutableBase
Mixin that defines transparent propagation of change events to a parent object.
See the example in Establishing Mutability on Scalar Column Values for usage information.
classmethod as_mutable(sqltype)
Associate a SQL type with this mutable Python type.
This establishes listeners that will detect ORM mappings against the given type, adding mutation event
trackers to those mappings.
The type is returned, unconditionally as an instance, so that as_mutable() can be used inline:
Table('mytable', metadata,
Column('id', Integer, primary_key=True),
Column('data', MyMutableType.as_mutable(PickleType))
)
Note that the returned type is always an instance, even if a class is given, and that only columns which are
declared specifically with that type instance receive additional instrumentation.
To associate a particular mutable type with all occurrences of a particular type, use the
Mutable.associate_with() classmethod of the particular Mutable() subclass to establish a
global association.
Warning: The listeners established by this method are global to all mappers, and are not garbage
collected. Only use as_mutable() for types that are permanent to an application, not with ad-hoc
types else this will cause unbounded growth in memory usage.
2.10. ORM Extensions
267
SQLAlchemy Documentation, Release 0.8.7
classmethod associate_with(sqltype)
Associate this wrapper with all future mapped columns of the given type.
This is a convenience method that calls associate_with_attribute automatically.
Warning: The listeners established by this method are global to all mappers, and are not garbage
collected. Only use associate_with() for types that are permanent to an application, not with
ad-hoc types else this will cause unbounded growth in memory usage.
classmethod associate_with_attribute(attribute)
Establish this type as a mutation listener for the given mapped descriptor.
changed()
Subclasses should call this method whenever change events occur.
class sqlalchemy.ext.mutable.MutableComposite
Bases: sqlalchemy.ext.mutable.MutableBase
Mixin that defines transparent propagation of change events on a SQLAlchemy “composite” object to its owning
parent or parents.
See the example in Establishing Mutability on Composites for usage information.
Members
class sqlalchemy.ext.mutable.MutableDict
Bases: sqlalchemy.ext.mutable.Mutable, __builtin__.dict
A dictionary type that implements Mutable.
New in version 0.8.
Members
2.10.4 Ordering List
A custom list that manages index/position information for contained elements.
author Jason Kirtland
orderinglist is a helper for mutable ordered relationships. It will intercept list operations performed on a
relationship()-managed collection and automatically synchronize changes in list position onto a target scalar
attribute.
Example: A slide table, where each row refers to zero or more entries in a related bullet table. The bullets
within a slide are displayed in order based on the value of the position column in the bullet table. As entries
are reordered in memory, the value of the position attribute should be updated to reflect the new sort order:
Base = declarative_base()
class Slide(Base):
__tablename__ = 'slide'
id = Column(Integer, primary_key=True)
name = Column(String)
bullets = relationship("Bullet", order_by="Bullet.position")
class Bullet(Base):
__tablename__ = 'bullet'
268
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
id = Column(Integer, primary_key=True)
slide_id = Column(Integer, ForeignKey('slide.id'))
position = Column(Integer)
text = Column(String)
The standard relationship mapping will produce a list-like attribute on each Slide containing all related Bullet
objects, but coping with changes in ordering is not handled automatically. When appending a Bullet into
Slide.bullets, the Bullet.position attribute will remain unset until manually assigned. When the Bullet
is inserted into the middle of the list, the following Bullet objects will also need to be renumbered.
The OrderingList object automates this task, managing the position attribute on all Bullet objects in the
collection. It is constructed using the ordering_list() factory:
from sqlalchemy.ext.orderinglist import ordering_list
Base = declarative_base()
class Slide(Base):
__tablename__ = 'slide'
id = Column(Integer, primary_key=True)
name = Column(String)
bullets = relationship("Bullet", order_by="Bullet.position",
collection_class=ordering_list('position'))
class Bullet(Base):
__tablename__ = 'bullet'
id = Column(Integer, primary_key=True)
slide_id = Column(Integer, ForeignKey('slide.id'))
position = Column(Integer)
text = Column(String)
With the above mapping the Bullet.position attribute is managed:
s = Slide()
s.bullets.append(Bullet())
s.bullets.append(Bullet())
s.bullets[1].position
>>> 1
s.bullets.insert(1, Bullet())
s.bullets[2].position
>>> 2
The OrderingList construct only works with changes to a collection, and not the initial load from the database,
and requires that the list be sorted when loaded. Therefore, be sure to specify order_by on the relationship()
against the target ordering attribute, so that the ordering is correct when first loaded.
2.10. ORM Extensions
269
SQLAlchemy Documentation, Release 0.8.7
Warning: OrderingList only provides limited functionality when a primary key column or unique column is
the target of the sort. Operations that are unsupported or are problematic include:
• two entries must trade values. This is not supported directly in the case of a primary key or unique constraint
because it means at least one row would need to be temporarily removed first, or changed to a third, neutral
value while the switch occurs.
• an entry must be deleted in order to make room for a new entry. SQLAlchemy’s unit of work performs all INSERTs before DELETEs within a single flush. In the case of a primary key, it will trade an INSERT/DELETE
of the same primary key for an UPDATE statement in order to lessen the impact of this limitation, however
this does not take place for a UNIQUE column. A future feature will allow the “DELETE before INSERT”
behavior to be possible, allevating this limitation, though this feature will require explicit configuration at
the mapper level for sets of columns that are to be handled in this way.
ordering_list() takes the name of the related object’s ordering attribute as an argument. By default, the
zero-based integer index of the object’s position in the ordering_list() is synchronized with the ordering attribute: index 0 will get position 0, index 1 position 1, etc. To start numbering at 1 or some other integer, provide
count_from=1.
API Reference
sqlalchemy.ext.orderinglist.ordering_list(attr, count_from=None, **kw)
Prepares an OrderingList factory for use in mapper definitions.
Returns an object suitable for use as an argument to a Mapper relationship’s collection_class option.
e.g.:
from sqlalchemy.ext.orderinglist import ordering_list
class Slide(Base):
__tablename__ = 'slide'
id = Column(Integer, primary_key=True)
name = Column(String)
bullets = relationship("Bullet", order_by="Bullet.position",
collection_class=ordering_list('position'))
Parameters
• attr – Name of the mapped attribute to use for storage and retrieval of ordering information
• count_from – Set up an integer-based ordering, starting at count_from. For example, ordering_list(’pos’, count_from=1) would create a 1-based list in
SQL, storing the value in the ‘pos’ column. Ignored if ordering_func is supplied.
Additional arguments are passed to the OrderingList constructor.
sqlalchemy.ext.orderinglist.count_from_0(index, collection)
Numbering function: consecutive integers starting at 0.
sqlalchemy.ext.orderinglist.count_from_1(index, collection)
Numbering function: consecutive integers starting at 1.
sqlalchemy.ext.orderinglist.count_from_n_factory(start)
Numbering function: consecutive integers starting at arbitrary start.
270
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
class sqlalchemy.ext.orderinglist.OrderingList(ordering_attr=None, ordering_func=None,
reorder_on_append=False)
Bases: __builtin__.list
A custom list that manages position information for its children.
The OrderingList object is normally set up using the ordering_list() factory function, used in conjunction with the relationship() function.
__init__(ordering_attr=None, ordering_func=None, reorder_on_append=False)
A custom list that manages position information for its children.
OrderingList is a collection_class list implementation that syncs position in a Python list
with a position attribute on the mapped objects.
This implementation relies on the list starting in the proper order, so be sure to put an order_by on
your relationship.
Parameters
• ordering_attr – Name of the attribute that stores the object’s order in the relationship.
• ordering_func – Optional. A function that maps the position in the Python list to
a value to store in the ordering_attr. Values returned are usually (but need not
be!) integers.
An ordering_func is called with two positional parameters: the index of the
element in the list, and the list itself.
If omitted, Python list indexes are used for the attribute values. Two basic prebuilt numbering functions are provided in this module: count_from_0 and
count_from_1. For more exotic examples like stepped numbering, alphabetical
and Fibonacci numbering, see the unit tests.
• reorder_on_append – Default False. When appending an object with an
existing (non-None) ordering value, that value will be left untouched unless
reorder_on_append is true. This is an optimization to avoid a variety of dangerous unexpected database writes.
SQLAlchemy will add instances to the list via append() when your object loads. If for
some reason the result set from the database skips a step in the ordering (say, row ‘1’
is missing but you get ‘2’, ‘3’, and ‘4’), reorder_on_append=True would immediately
renumber the items to ‘1’, ‘2’, ‘3’. If you have multiple sessions making changes, any
of whom happen to load this collection even in passing, all of the sessions would try
to “clean up” the numbering in their commits, possibly causing all but one to fail with
a concurrent modification error.
Recommend leaving this with the default of False, and just call reorder() if you’re
doing append() operations with previously ordered instances or when doing some
housekeeping after manual sql operations.
append(entity)
L.append(object) – append object to end
insert(index, entity)
L.insert(index, object) – insert object before index
pop([index ]) → item – remove and return item at index (default last).
Raises IndexError if list is empty or index is out of range.
remove(entity)
L.remove(value) – remove first occurrence of value. Raises ValueError if the value is not present.
2.10. ORM Extensions
271
SQLAlchemy Documentation, Release 0.8.7
reorder()
Synchronize ordering for the entire collection.
Sweeps through the list and ensures that each object has accurate ordering information set.
2.10.5 Horizontal Sharding
Horizontal sharding support.
Defines a rudimental ‘horizontal sharding’ system which allows a Session to distribute queries and persistence operations across multiple databases.
For a usage example, see the Horizontal Sharding example included in the source distribution.
API Documentation
class sqlalchemy.ext.horizontal_shard.ShardedSession(shard_chooser,
id_chooser,
query_chooser,
shards=None,
query_cls=<class
‘sqlalchemy.ext.horizontal_shard.ShardedQuery’>,
**kwargs)
Bases: sqlalchemy.orm.session.Session
__init__(shard_chooser,
id_chooser,
query_chooser,
shards=None,
‘sqlalchemy.ext.horizontal_shard.ShardedQuery’>, **kwargs)
Construct a ShardedSession.
query_cls=<class
Parameters
• shard_chooser – A callable which, passed a Mapper, a mapped instance, and
possibly a SQL clause, returns a shard ID. This id may be based off of the attributes
present within the object, or on some round-robin scheme. If the scheme is based
on a selection, it should set whatever state on the instance to mark it in the future as
participating in that shard.
• id_chooser – A callable, passed a query and a tuple of identity values, which
should return a list of shard ids where the ID might reside. The databases will be
queried in the order of this listing.
• query_chooser – For a given Query, returns the list of shard_ids where the query
should be issued. Results from all shards returned will be combined together into a
single listing.
• shards – A dictionary of string shard names to Engine objects.
class sqlalchemy.ext.horizontal_shard.ShardedQuery(*args, **kwargs)
Bases: sqlalchemy.orm.query.Query
set_shard(shard_id)
return a new query, limited to a single shard ID.
all subsequent operations with the returned query will be against the single shard regardless of other state.
2.10.6 Hybrid Attributes
Define attributes on ORM-mapped classes that have “hybrid” behavior.
“hybrid” means the attribute has distinct behaviors defined at the class level and at the instance level.
272
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
The hybrid extension provides a special form of method decorator, is around 50 lines of code and has almost no
dependencies on the rest of SQLAlchemy. It can, in theory, work with any descriptor-based expression system.
Consider a mapping Interval, representing integer start and end values. We can define higher level functions
on mapped classes that produce SQL expressions at the class level, and Python expression evaluation at the instance
level. Below, each function decorated with hybrid_method or hybrid_property may receive self as an
instance of the class, or as the class itself:
from
from
from
from
sqlalchemy import Column, Integer
sqlalchemy.ext.declarative import declarative_base
sqlalchemy.orm import Session, aliased
sqlalchemy.ext.hybrid import hybrid_property, hybrid_method
Base = declarative_base()
class Interval(Base):
__tablename__ = 'interval'
id = Column(Integer, primary_key=True)
start = Column(Integer, nullable=False)
end = Column(Integer, nullable=False)
def __init__(self, start, end):
self.start = start
self.end = end
@hybrid_property
def length(self):
return self.end - self.start
@hybrid_method
def contains(self, point):
return (self.start <= point) & (point <= self.end)
@hybrid_method
def intersects(self, other):
return self.contains(other.start) | self.contains(other.end)
Above, the length property returns the difference between the end and start attributes. With an instance of
Interval, this subtraction occurs in Python, using normal Python descriptor mechanics:
>>> i1 = Interval(5, 10)
>>> i1.length
5
When dealing with the Interval class itself, the hybrid_property descriptor evaluates the function body given
the Interval class as the argument, which when evaluated with SQLAlchemy expression mechanics returns a new
SQL expression:
>>> print Interval.length
interval."end" - interval.start
>>> print Session().query(Interval).filter(Interval.length > 10)
SELECT interval.id AS interval_id, interval.start AS interval_start,
interval."end" AS interval_end
FROM interval
WHERE interval."end" - interval.start > :param_1
2.10. ORM Extensions
273
SQLAlchemy Documentation, Release 0.8.7
ORM methods such as filter_by() generally use getattr() to locate attributes, so can also be used with hybrid
attributes:
>>> print Session().query(Interval).filter_by(length=5)
SELECT interval.id AS interval_id, interval.start AS interval_start,
interval."end" AS interval_end
FROM interval
WHERE interval."end" - interval.start = :param_1
The Interval class example also illustrates two methods, contains() and intersects(), decorated with
hybrid_method. This decorator applies the same idea to methods that hybrid_property applies to attributes.
The methods return boolean values, and take advantage of the Python | and & bitwise operators to produce equivalent
instance-level and SQL expression-level boolean behavior:
>>> i1.contains(6)
True
>>> i1.contains(15)
False
>>> i1.intersects(Interval(7, 18))
True
>>> i1.intersects(Interval(25, 29))
False
>>> print Session().query(Interval).filter(Interval.contains(15))
SELECT interval.id AS interval_id, interval.start AS interval_start,
interval."end" AS interval_end
FROM interval
WHERE interval.start <= :start_1 AND interval."end" > :end_1
>>> ia = aliased(Interval)
>>> print Session().query(Interval, ia).filter(Interval.intersects(ia))
SELECT interval.id AS interval_id, interval.start AS interval_start,
interval."end" AS interval_end, interval_1.id AS interval_1_id,
interval_1.start AS interval_1_start, interval_1."end" AS interval_1_end
FROM interval, interval AS interval_1
WHERE interval.start <= interval_1.start
AND interval."end" > interval_1.start
OR interval.start <= interval_1."end"
AND interval."end" > interval_1."end"
Defining Expression Behavior Distinct from Attribute Behavior
Our usage of the & and | bitwise operators above was fortunate, considering our functions operated on two boolean
values to return a new one. In many cases, the construction of an in-Python function and a SQLAlchemy SQL
expression have enough differences that two separate Python expressions should be defined. The hybrid decorators
define the hybrid_property.expression() modifier for this purpose. As an example we’ll define the radius
of the interval, which requires the usage of the absolute value function:
from sqlalchemy import func
class Interval(object):
# ...
@hybrid_property
def radius(self):
274
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
return abs(self.length) / 2
@radius.expression
def radius(cls):
return func.abs(cls.length) / 2
Above the Python function abs() is used for instance-level operations, the SQL function ABS() is used via the
func object for class-level expressions:
>>> i1.radius
2
>>> print Session().query(Interval).filter(Interval.radius > 5)
SELECT interval.id AS interval_id, interval.start AS interval_start,
interval."end" AS interval_end
FROM interval
WHERE abs(interval."end" - interval.start) / :abs_1 > :param_1
Defining Setters
Hybrid properties can also define setter methods. If we wanted length above, when set, to modify the endpoint
value:
class Interval(object):
# ...
@hybrid_property
def length(self):
return self.end - self.start
@length.setter
def length(self, value):
self.end = self.start + value
The length(self, value) method is now called upon set:
>>>
>>>
5
>>>
>>>
17
i1 = Interval(5, 10)
i1.length
i1.length = 12
i1.end
Working with Relationships
There’s no essential difference when creating hybrids that work with related objects as opposed to column-based data.
The need for distinct expressions tends to be greater. Two variants of we’ll illustrate are the “join-dependent” hybrid,
and the “correlated subquery” hybrid.
2.10. ORM Extensions
275
SQLAlchemy Documentation, Release 0.8.7
Join-Dependent Relationship Hybrid
Consider the following declarative mapping which relates a User to a SavingsAccount:
from
from
from
from
sqlalchemy import Column, Integer, ForeignKey, Numeric, String
sqlalchemy.orm import relationship
sqlalchemy.ext.declarative import declarative_base
sqlalchemy.ext.hybrid import hybrid_property
Base = declarative_base()
class SavingsAccount(Base):
__tablename__ = 'account'
id = Column(Integer, primary_key=True)
user_id = Column(Integer, ForeignKey('user.id'), nullable=False)
balance = Column(Numeric(15, 5))
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String(100), nullable=False)
accounts = relationship("SavingsAccount", backref="owner")
@hybrid_property
def balance(self):
if self.accounts:
return self.accounts[0].balance
else:
return None
@balance.setter
def balance(self, value):
if not self.accounts:
account = Account(owner=self)
else:
account = self.accounts[0]
account.balance = value
@balance.expression
def balance(cls):
return SavingsAccount.balance
The above hybrid property balance works with the first SavingsAccount entry in the list of accounts for this
user. The in-Python getter/setter methods can treat accounts as a Python list available on self.
However, at the expression level, it’s expected that the User class will be used in an appropriate context such that an
appropriate join to SavingsAccount will be present:
>>> print Session().query(User, User.balance).\
...
join(User.accounts).filter(User.balance > 5000)
SELECT "user".id AS user_id, "user".name AS user_name,
account.balance AS account_balance
FROM "user" JOIN account ON "user".id = account.user_id
WHERE account.balance > :balance_1
276
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
Note however, that while the instance level accessors need to worry about whether self.accounts is even present,
this issue expresses itself differently at the SQL expression level, where we basically would use an outer join:
>>> from sqlalchemy import or_
>>> print (Session().query(User, User.balance).outerjoin(User.accounts).
...
filter(or_(User.balance < 5000, User.balance == None)))
SELECT "user".id AS user_id, "user".name AS user_name,
account.balance AS account_balance
FROM "user" LEFT OUTER JOIN account ON "user".id = account.user_id
WHERE account.balance < :balance_1 OR account.balance IS NULL
Correlated Subquery Relationship Hybrid
We can, of course, forego being dependent on the enclosing query’s usage of joins in favor of the correlated subquery,
which can portably be packed into a single column expression. A correlated subquery is more portable, but often
performs more poorly at the SQL level. Using the same technique illustrated at Using column_property, we can adjust
our SavingsAccount example to aggregate the balances for all accounts, and use a correlated subquery for the
column expression:
from
from
from
from
from
sqlalchemy import Column, Integer, ForeignKey, Numeric, String
sqlalchemy.orm import relationship
sqlalchemy.ext.declarative import declarative_base
sqlalchemy.ext.hybrid import hybrid_property
sqlalchemy import select, func
Base = declarative_base()
class SavingsAccount(Base):
__tablename__ = 'account'
id = Column(Integer, primary_key=True)
user_id = Column(Integer, ForeignKey('user.id'), nullable=False)
balance = Column(Numeric(15, 5))
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String(100), nullable=False)
accounts = relationship("SavingsAccount", backref="owner")
@hybrid_property
def balance(self):
return sum(acc.balance for acc in self.accounts)
@balance.expression
def balance(cls):
return select([func.sum(SavingsAccount.balance)]).\
where(SavingsAccount.user_id==cls.id).\
label('total_balance')
The above recipe will give us the balance column which renders a correlated SELECT:
>>> print s.query(User).filter(User.balance > 400)
SELECT "user".id AS user_id, "user".name AS user_name
FROM "user"
2.10. ORM Extensions
277
SQLAlchemy Documentation, Release 0.8.7
WHERE (SELECT sum(account.balance) AS sum_1
FROM account
WHERE account.user_id = "user".id) > :param_1
Building Custom Comparators
The hybrid property also includes a helper that allows construction of custom comparators. A comparator object
allows one to customize the behavior of each SQLAlchemy expression operator individually. They are useful when
creating custom types that have some highly idiosyncratic behavior on the SQL side.
The example class below allows case-insensitive comparisons on the attribute named word_insensitive:
from
from
from
from
sqlalchemy.ext.hybrid import Comparator, hybrid_property
sqlalchemy import func, Column, Integer, String
sqlalchemy.orm import Session
sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class CaseInsensitiveComparator(Comparator):
def __eq__(self, other):
return func.lower(self.__clause_element__()) == func.lower(other)
class SearchWord(Base):
__tablename__ = 'searchword'
id = Column(Integer, primary_key=True)
word = Column(String(255), nullable=False)
@hybrid_property
def word_insensitive(self):
return self.word.lower()
@word_insensitive.comparator
def word_insensitive(cls):
return CaseInsensitiveComparator(cls.word)
Above, SQL expressions against word_insensitive will apply the LOWER() SQL function to both sides:
>>> print Session().query(SearchWord).filter_by(word_insensitive="Trucks")
SELECT searchword.id AS searchword_id, searchword.word AS searchword_word
FROM searchword
WHERE lower(searchword.word) = lower(:lower_1)
The CaseInsensitiveComparator above implements part of the ColumnOperators interface. A “coercion” operation like lowercasing can be applied to all comparison operations (i.e. eq, lt, gt, etc.) using
Operators.operate():
class CaseInsensitiveComparator(Comparator):
def operate(self, op, other):
return op(func.lower(self.__clause_element__()), func.lower(other))
278
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
Hybrid Value Objects
Note in our previous example, if we were to compare the word_insensitive attribute of a SearchWord
instance to a plain Python string, the plain Python string would not be coerced to lower case - the
CaseInsensitiveComparator we built, being returned by @word_insensitive.comparator, only applies to the SQL side.
A more comprehensive form of the custom comparator is to construct a Hybrid Value Object. This technique applies
the target value or expression to a value object which is then returned by the accessor in all cases. The value object
allows control of all operations upon the value as well as how compared values are treated, both on the SQL expression
side as well as the Python value side. Replacing the previous CaseInsensitiveComparator class with a new
CaseInsensitiveWord class:
class CaseInsensitiveWord(Comparator):
"Hybrid value representing a lower case representation of a word."
def __init__(self, word):
if isinstance(word, basestring):
self.word = word.lower()
elif isinstance(word, CaseInsensitiveWord):
self.word = word.word
else:
self.word = func.lower(word)
def operate(self, op, other):
if not isinstance(other, CaseInsensitiveWord):
other = CaseInsensitiveWord(other)
return op(self.word, other.word)
def __clause_element__(self):
return self.word
def __str__(self):
return self.word
key = 'word'
"Label to apply to Query tuple results"
Above, the CaseInsensitiveWord object represents self.word, which may be a SQL function, or may be a
Python native. By overriding operate() and __clause_element__() to work in terms of self.word, all
comparison operations will work against the “converted” form of word, whether it be SQL side or Python side. Our
SearchWord class can now deliver the CaseInsensitiveWord object unconditionally from a single hybrid call:
class SearchWord(Base):
__tablename__ = 'searchword'
id = Column(Integer, primary_key=True)
word = Column(String(255), nullable=False)
@hybrid_property
def word_insensitive(self):
return CaseInsensitiveWord(self.word)
The word_insensitive attribute now has case-insensitive comparison behavior universally, including SQL expression vs. Python expression (note the Python value is converted to lower case on the Python side here):
2.10. ORM Extensions
279
SQLAlchemy Documentation, Release 0.8.7
>>> print Session().query(SearchWord).filter_by(word_insensitive="Trucks")
SELECT searchword.id AS searchword_id, searchword.word AS searchword_word
FROM searchword
WHERE lower(searchword.word) = :lower_1
SQL expression versus SQL expression:
>>> sw1 = aliased(SearchWord)
>>> sw2 = aliased(SearchWord)
>>> print Session().query(
...
sw1.word_insensitive,
...
sw2.word_insensitive).\
...
filter(
...
sw1.word_insensitive > sw2.word_insensitive
...
)
SELECT lower(searchword_1.word) AS lower_1,
lower(searchword_2.word) AS lower_2
FROM searchword AS searchword_1, searchword AS searchword_2
WHERE lower(searchword_1.word) > lower(searchword_2.word)
Python only expression:
>>> ws1 = SearchWord(word="SomeWord")
>>> ws1.word_insensitive == "sOmEwOrD"
True
>>> ws1.word_insensitive == "XOmEwOrX"
False
>>> print ws1.word_insensitive
someword
The Hybrid Value pattern is very useful for any kind of value that may have multiple representations, such as timestamps, time deltas, units of measurement, currencies and encrypted passwords.
See also:
Hybrids and Value Agnostic Types - on the techspot.zzzeek.org blog
Value Agnostic Types, Part II - on the techspot.zzzeek.org blog
Building Transformers
A transformer is an object which can receive a Query object and return a new one. The Query object includes a
method with_transformation() that returns a new Query transformed by the given function.
We can combine this with the Comparator class to produce one type of recipe which can both set up the FROM
clause of a query as well as assign filtering criterion.
Consider a mapped class Node, which assembles using adjacency list into a hierarchical tree pattern:
from
from
from
Base
sqlalchemy import Column, Integer, ForeignKey
sqlalchemy.orm import relationship
sqlalchemy.ext.declarative import declarative_base
= declarative_base()
class Node(Base):
__tablename__ = 'node'
id =Column(Integer, primary_key=True)
280
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
parent_id = Column(Integer, ForeignKey('node.id'))
parent = relationship("Node", remote_side=id)
Suppose we wanted to add an accessor grandparent. This would return the parent of Node.parent. When
we have an instance of Node, this is simple:
from sqlalchemy.ext.hybrid import hybrid_property
class Node(Base):
# ...
@hybrid_property
def grandparent(self):
return self.parent.parent
For the expression, things are not so clear. We’d need to construct a Query where we join() twice along
Node.parent to get to the grandparent. We can instead return a transforming callable that we’ll combine
with the Comparator class to receive any Query object, and return a new one that’s joined to the Node.parent
attribute and filtered based on the given criterion:
from sqlalchemy.ext.hybrid import Comparator
class GrandparentTransformer(Comparator):
def operate(self, op, other):
def transform(q):
cls = self.__clause_element__()
parent_alias = aliased(cls)
return q.join(parent_alias, cls.parent).\
filter(op(parent_alias.parent, other))
return transform
Base = declarative_base()
class Node(Base):
__tablename__ = 'node'
id =Column(Integer, primary_key=True)
parent_id = Column(Integer, ForeignKey('node.id'))
parent = relationship("Node", remote_side=id)
@hybrid_property
def grandparent(self):
return self.parent.parent
@grandparent.comparator
def grandparent(cls):
return GrandparentTransformer(cls)
The GrandparentTransformer overrides the core Operators.operate() method at the base of the
Comparator hierarchy to return a query-transforming callable, which then runs the given comparison operation
in a particular context. Such as, in the example above, the operate method is called, given the Operators.eq
callable as well as the right side of the comparison Node(id=5). A function transform is then returned which
will transform a Query first to join to Node.parent, then to compare parent_alias using Operators.eq
against the left and right sides, passing into Query.filter:
2.10. ORM Extensions
281
SQLAlchemy Documentation, Release 0.8.7
>>> from sqlalchemy.orm import Session
>>> session = Session()
>>> session.query(Node).\
...
with_transformation(Node.grandparent==Node(id=5)).\
...
all()
SELECT node.id AS node_id, node.parent_id AS node_parent_id
FROM node JOIN node AS node_1 ON node_1.id = node.parent_id
WHERE :param_1 = node_1.parent_id
We can modify the pattern to be more verbose but flexible by separating the “join” step from the “filter” step. The tricky
part here is ensuring that successive instances of GrandparentTransformer use the same AliasedClass
object against Node. Below we use a simple memoizing approach that associates a GrandparentTransformer
with each class:
class Node(Base):
# ...
@grandparent.comparator
def grandparent(cls):
# memoize a GrandparentTransformer
# per class
if '_gp' not in cls.__dict__:
cls._gp = GrandparentTransformer(cls)
return cls._gp
class GrandparentTransformer(Comparator):
def __init__(self, cls):
self.parent_alias = aliased(cls)
@property
def join(self):
def go(q):
return q.join(self.parent_alias, Node.parent)
return go
def operate(self, op, other):
return op(self.parent_alias.parent, other)
>>> session.query(Node).\
...
with_transformation(Node.grandparent.join).\
...
filter(Node.grandparent==Node(id=5))
SELECT node.id AS node_id, node.parent_id AS node_parent_id
FROM node JOIN node AS node_1 ON node_1.id = node.parent_id
WHERE :param_1 = node_1.parent_id
The “transformer” pattern is an experimental pattern that starts to make usage of some functional programming
paradigms. While it’s only recommended for advanced and/or patient developers, there’s probably a whole lot of
amazing things it can be used for.
282
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
API Reference
class sqlalchemy.ext.hybrid.hybrid_method(func, expr=None)
Bases: sqlalchemy.orm.interfaces._InspectionAttr
A decorator which allows definition of a Python object method with both instance-level and class-level behavior.
__init__(func, expr=None)
Create a new hybrid_method.
Usage is typically via decorator:
from sqlalchemy.ext.hybrid import hybrid_method
class SomeClass(object):
@hybrid_method
def value(self, x, y):
return self._value + x + y
@value.expression
def value(self, x, y):
return func.some_function(self._value, x, y)
expression(expr)
Provide a modifying decorator that defines a SQL-expression producing method.
class sqlalchemy.ext.hybrid.hybrid_property(fget, fset=None, fdel=None, expr=None)
Bases: sqlalchemy.orm.interfaces._InspectionAttr
A decorator which allows definition of a Python descriptor with both instance-level and class-level behavior.
__init__(fget, fset=None, fdel=None, expr=None)
Create a new hybrid_property.
Usage is typically via decorator:
from sqlalchemy.ext.hybrid import hybrid_property
class SomeClass(object):
@hybrid_property
def value(self):
return self._value
@value.setter
def value(self, value):
self._value = value
comparator(comparator)
Provide a modifying decorator that defines a custom comparator producing method.
The return value of the decorated method should be an instance of Comparator.
deleter(fdel)
Provide a modifying decorator that defines a value-deletion method.
expression(expr)
Provide a modifying decorator that defines a SQL-expression producing method.
setter(fset)
Provide a modifying decorator that defines a value-setter method.
2.10. ORM Extensions
283
SQLAlchemy Documentation, Release 0.8.7
class sqlalchemy.ext.hybrid.Comparator(expression)
Bases: sqlalchemy.orm.interfaces.PropComparator
A helper class that allows easy construction of custom PropComparator classes for usage with hybrids.
sqlalchemy.ext.hybrid.HYBRID_METHOD = <symbol ‘HYBRID_METHOD>
sqlalchemy.ext.hybrid.HYBRID_PROPERTY = <symbol ‘HYBRID_PROPERTY>
2.10.7 Alternate Class Instrumentation
Extensible class instrumentation.
The sqlalchemy.ext.instrumentation package provides for alternate systems of class instrumentation
within the ORM. Class instrumentation refers to how the ORM places attributes on the class which maintain data
and track changes to that data, as well as event hooks installed on the class.
Note: The extension package is provided for the benefit of integration with other object management packages, which
already perform their own instrumentation. It is not intended for general use.
For examples of how the instrumentation extension is used, see the example Attribute Instrumentation.
Changed in version 0.8: The sqlalchemy.orm.instrumentation was split out so that all functionality
having to do with non-standard instrumentation was moved out to sqlalchemy.ext.instrumentation.
When imported, the module installs itself within sqlalchemy.orm.instrumentation so that it
takes effect, including recognition of __sa_instrumentation_manager__ on mapped classes, as well
instrumentation_finders being used to determine class instrumentation resolution.
API Reference
sqlalchemy.ext.instrumentation.INSTRUMENTATION_MANAGER = ‘__sa_instrumentation_manager__’
Attribute, elects custom instrumentation when present on a mapped class.
Allows a class to specify a slightly or wildly different technique for tracking changes made to mapped attributes
and collections.
Only one instrumentation implementation is allowed in a given object inheritance hierarchy.
The value of this attribute must be a callable and will be passed a class object. The callable must return one of:
•An instance of an InstrumentationManager or subclass
•An object implementing all or some of InstrumentationManager (TODO)
•A dictionary of callables, implementing all or some of the above (TODO)
•An instance of a ClassManager or subclass
This
attribute
is
consulted
by
SQLAlchemy
instrumentation
resolution,
once
the
sqlalchemy.ext.instrumentation module has been imported. If custom finders are installed
in the global instrumentation_finders list, they may or may not choose to honor this attribute.
class sqlalchemy.ext.instrumentation.InstrumentationManager(class_)
User-defined class instrumentation extension.
InstrumentationManager can be subclassed in order to change how class instrumentation proceeds. This
class exists for the purposes of integration with other object management frameworks which would like to entirely modify the instrumentation methodology of the ORM, and is not intended for regular usage. For interception of class instrumentation events, see InstrumentationEvents.
284
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
The API for this class should be considered as semi-stable, and may change slightly with new releases.
Changed
in
version
0.8:
InstrumentationManager
was
moved
sqlalchemy.orm.instrumentation to sqlalchemy.ext.instrumentation.
from
dict_getter(class_)
dispose(class_, manager)
get_instance_dict(class_, instance)
initialize_instance_dict(class_, instance)
install_descriptor(class_, key, inst)
install_member(class_, key, implementation)
install_state(class_, instance, state)
instrument_attribute(class_, key, inst)
instrument_collection_class(class_, key, collection_class)
manage(class_, manager)
manager_getter(class_)
post_configure_attribute(class_, key, inst)
remove_state(class_, instance)
state_getter(class_)
uninstall_descriptor(class_, key)
uninstall_member(class_, key)
sqlalchemy.ext.instrumentation.instrumentation_finders = [<function find_native_user_instrumentation_ho
An extensible sequence of callables which return instrumentation implementations
When a class is registered, each callable will be passed a class object. If None is returned, the next finder in the
sequence is consulted. Otherwise the return must be an instrumentation factory that follows the same guidelines
as sqlalchemy.ext.instrumentation.INSTRUMENTATION_MANAGER.
By default, the only finder is find_native_user_instrumentation_hook, which searches for INSTRUMENTATION_MANAGER. If all finders return None, standard ClassManager instrumentation is used.
class sqlalchemy.ext.instrumentation.ExtendedInstrumentationRegistry
Bases: sqlalchemy.orm.instrumentation.InstrumentationFactory
Extends InstrumentationFactory with additional bookkeeping, to accommodate multiple types of class
managers.
Members
2.11 Examples
The SQLAlchemy distribution includes a variety of code examples illustrating a select set of patterns, some typical
and some not so typical. All are runnable and can be found in the /examples directory of the distribution. Each
example contains a README in its __init__.py file, each of which are listed below.
Additional SQLAlchemy examples,
some user
http://www.sqlalchemy.org/trac/wiki/UsageRecipes.
2.11. Examples
contributed,
are
available
on
the
wiki
at
285
SQLAlchemy Documentation, Release 0.8.7
2.11.1 Adjacency List
Location: /examples/adjacency_list/ An example of a dictionary-of-dictionaries structure mapped using an adjacency
list model.
E.g.:
node = TreeNode('rootnode')
node.append('node1')
node.append('node3')
session.add(node)
session.commit()
dump_tree(node)
2.11.2 Associations
Location: /examples/association/ Examples illustrating the usage of the “association object” pattern, where an intermediary class mediates the relationship between two classes that are associated in a many-to-many pattern.
This directory includes the following examples:
• basic_association.py - illustrate a many-to-many relationship between an “Order” and a collection of “Item”
objects, associating a purchase price with each via an association object called “OrderItem”
• proxied_association.py
same
example
as
basic_association,
adding
in
usage
sqlalchemy.ext.associationproxy to make explicit references to “OrderItem” optional.
of
• dict_of_sets_with_default.py - an advanced association proxy example which illustrates nesting of association
proxies to produce multi-level Python collections, in this case a dictionary with string keys and sets of integers
as values, which conceal the underlying mapped classes.
2.11.3 Attribute Instrumentation
Location: /examples/custom_attributes/ Two examples illustrating modifications to SQLAlchemy’s attribute management system.
listen_for_events.py illustrates the usage of AttributeExtension to intercept attribute events.
It additionally illustrates a way to automatically attach these listeners to all class attributes using a
InstrumentationManager.
custom_management.py illustrates much deeper usage of InstrumentationManager as well as collection
adaptation, to completely change the underlying method used to store state on an object. This example was developed
to illustrate techniques which would be used by other third party object instrumentation systems to interact with
SQLAlchemy’s event system and is only intended for very intricate framework integrations.
2.11.4 Dogpile Caching
Location: /examples/dogpile_caching/ Illustrates how to embed dogpile.cache functionality within the Query object,
allowing full cache control as well as the ability to pull “lazy loaded” attributes from long term cache as well.
Changed in version 0.8: The example was modernized to use dogpile.cache, replacing Beaker as the caching library
in use.
In this demo, the following techniques are illustrated:
286
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
• Using custom subclasses of Query
• Basic technique of circumventing Query to pull from a custom cache source instead of the database.
• Rudimental caching with dogpile.cache, using “regions” which allow global control over a fixed set of configurations.
• Using custom MapperOption objects to configure options on a Query, including the ability to invoke the
options deep within an object graph when lazy loads occur.
E.g.:
# query for Person objects, specifying cache
q = Session.query(Person).options(FromCache("default"))
# specify that each Person's "addresses" collection comes from
# cache too
q = q.options(RelationshipCache(Person.addresses, "default"))
# query
print q.all()
To run, both SQLAlchemy and dogpile.cache must be installed or on the current PYTHONPATH. The demo will
create a local directory for datafiles, insert initial data, and run. Running the demo a second time will utilize the cache
files already present, and exactly one SQL statement against two tables will be emitted - the displayed result however
will utilize dozens of lazyloads that all pull from cache.
The demo scripts themselves, in order of complexity, are run as Python modules so that relative imports work:
python -m examples.dogpile_caching.helloworld
python -m examples.dogpile_caching.relationship_caching
python -m examples.dogpile_caching.advanced
python -m examples.dogpile_caching.local_session_caching
Listing of files:
environment.py - Establish the Session, a dictionary of “regions”, a sample cache region against a .dbm
file, data / cache file paths, and configurations, bootstrap fixture data if necessary.
caching_query.py - Represent functions and classes which allow the usage of Dogpile caching with
SQLAlchemy. Introduces a query option called FromCache.
model.py - The datamodel, which represents Person that has multiple Address objects, each with PostalCode, City, Country
fixture_data.py - creates demo PostalCode, Address, Person objects in the database.
helloworld.py - the basic idea.
relationship_caching.py - Illustrates how to add cache options on relationship endpoints, so that lazyloads
load from cache.
advanced.py - Further examples of how to use FromCache. Combines techniques from the first two
scripts.
local_session_caching.py - Grok everything so far ? This example creates a new dogpile.cache backend
that will persist data in a dictionary which is local to the current session. remove() the session and the
cache is gone.
2.11. Examples
287
SQLAlchemy Documentation, Release 0.8.7
2.11.5 Directed Graphs
Location: /examples/graphs/ An example of persistence for a directed graph structure. The graph is stored as a
collection of edges, each referencing both a “lower” and an “upper” node in a table of nodes. Basic persistence and
querying for lower- and upper- neighbors are illustrated:
n2 = Node(2)
n5 = Node(5)
n2.add_neighbor(n5)
print n2.higher_neighbors()
2.11.6 Dynamic Relations as Dictionaries
Location: /examples/dynamic_dict/ Illustrates how to place a dictionary-like facade on top of a “dynamic” relation,
so that dictionary operations (assuming simple string keys) can operate upon a large collection without loading the full
collection at once.
2.11.7 Generic Associations
Location: /examples/generic_associations Illustrates various methods of associating multiple types of parents with a
particular child object.
The examples all use the declarative extension along with declarative mixins. Each one presents the identical use case
at the end - two classes, Customer and Supplier, both subclassing the HasAddresses mixin, which ensures
that the parent class is provided with an addresses collection which contains Address objects.
The configurations include:
• table_per_related.py - illustrates a distinct table per related collection.
• table_per_association.py - illustrates a shared collection table, using a table per association.
• discriminator_on_association.py - shared collection table and shared association table, including
a discriminator column.
• generic_fk.py - imitates the approach taken by popular frameworks such as Django and Ruby on Rails to
create a so-called “generic foreign key”.
The discriminator_on_association.py and generic_fk.py scripts are modernized versions of recipes
presented in the 2007 blog post Polymorphic Associations with SQLAlchemy. .
2.11.8 Horizontal Sharding
Location: /examples/sharding A basic example of using the SQLAlchemy Sharding API. Sharding refers to horizontally scaling data across multiple databases.
The basic components of a “sharded” mapping are:
• multiple databases, each assigned a ‘shard id’
• a function which can return a single shard id, given an instance to be saved; this is called “shard_chooser”
• a function which can return a list of shard ids which apply to a particular instance identifier; this is called
“id_chooser”. If it returns all shard ids, all shards will be searched.
• a function which can return a list of shard ids to try, given a particular Query (“query_chooser”). If it returns all
shard ids, all shards will be queried and the results joined together.
288
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
In this example, four sqlite databases will store information about weather data on a database-per-continent basis. We
provide example shard_chooser, id_chooser and query_chooser functions. The query_chooser illustrates inspection of
the SQL expression element in order to attempt to determine a single shard being requested.
The construction of generic sharding routines is an ambitious approach to the issue of organizing instances among
multiple databases. For a more plain-spoken alternative, the “distinct entity” approach is a simple method of assigning
objects to different tables (and potentially database nodes) in an explicit way - described on the wiki at EntityName.
2.11.9 Inheritance Mappings
Location: /examples/inheritance/ Working examples of single-table, joined-table, and concrete-table inheritance as
described in datamapping_inheritance.
2.11.10 Large Collections
Location: /examples/large_collection/ Large collection example.
Illustrates the options to use with relationship() when the list of related objects is very large, including:
• “dynamic” relationships which query slices of data as accessed
• how to use ON DELETE CASCADE in conjunction with passive_deletes=True to greatly improve the
performance of related collection deletion.
2.11.11 Nested Sets
Location: /examples/nested_sets/ Illustrates a rudimentary way to implement the “nested sets” pattern for hierarchical
data using the SQLAlchemy ORM.
2.11.12 Polymorphic Associations
See Generic Associations for a modern version of polymorphic associations.
2.11.13 PostGIS Integration
Location: /examples/postgis A naive example illustrating techniques to help embed PostGIS functionality.
This example was originally developed in the hopes that it would be extrapolated into a comprehensive PostGIS
integration layer. We are pleased to announce that this has come to fruition as GeoAlchemy.
The example illustrates:
• a DDL extension which allows CREATE/DROP to work in conjunction with AddGeometryColumn/DropGeometryColumn
• a Geometry type, as well as a few subtypes, which convert result row values to a GIS-aware object, and also
integrates with the DDL extension.
• a GIS-aware object which stores a raw geometry value and provides a factory for functions such as AsText().
• an ORM comparator which can override standard column methods on mapped objects to produce GIS operators.
• an attribute event listener that intercepts strings and converts to GeomFromText().
• a standalone operator example.
2.11. Examples
289
SQLAlchemy Documentation, Release 0.8.7
The implementation is limited to only public, well known and simple to use extension points.
E.g.:
print session.query(Road).filter(Road.road_geom.intersects(r1.road_geom)).all()
2.11.14 Versioned Objects
Location: /examples/versioning Illustrates an extension which creates version tables for entities and stores records for
each change. The same idea as Elixir’s versioned extension, but more efficient (uses attribute API to get history) and
handles class inheritance. The given extensions generate an anonymous “history” class which represents historical
versions of the target object.
Usage is illustrated via a unit test module test_versioning.py, which can be run via nose:
cd examples/versioning
nosetests -v
A fragment of example usage, using declarative:
from history_meta import Versioned, versioned_session
Base = declarative_base()
class SomeClass(Versioned, Base):
__tablename__ = 'sometable'
id = Column(Integer, primary_key=True)
name = Column(String(50))
def __eq__(self, other):
assert type(other) is SomeClass and other.id == self.id
Session = sessionmaker(bind=engine)
versioned_session(Session)
sess = Session()
sc = SomeClass(name='sc1')
sess.add(sc)
sess.commit()
sc.name = 'sc1modified'
sess.commit()
assert sc.version == 2
SomeClassHistory = SomeClass.__history_mapper__.class_
assert sess.query(SomeClassHistory).\
filter(SomeClassHistory.version == 1).\
all() \
== [SomeClassHistory(version=1, name='sc1')]
The Versioned mixin is designed to work with declarative. To use the extension with classical mappers, the
_history_mapper function can be applied:
290
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
from history_meta import _history_mapper
m = mapper(SomeClass, sometable)
_history_mapper(m)
SomeHistoryClass = SomeClass.__history_mapper__.class_
2.11.15 Vertical Attribute Mapping
Location: /examples/vertical Illustrates “vertical table” mappings.
A “vertical table” refers to a technique where individual attributes of an object are stored as distinct rows in a table.
The “vertical table” technique is used to persist objects which can have a varied set of attributes, at the expense of
simple query control and brevity. It is commonly found in content/document management systems in order to represent
user-created structures flexibly.
Two variants on the approach are given. In the second, each row references a “datatype” which contains information
about the type of information stored in the attribute, such as integer, string, or date.
Example:
shrew = Animal(u'shrew')
shrew[u'cuteness'] = 5
shrew[u'weasel-like'] = False
shrew[u'poisonous'] = True
session.add(shrew)
session.flush()
q = (session.query(Animal).
filter(Animal.facts.any(
and_(AnimalFact.key == u'weasel-like',
AnimalFact.value == True))))
print 'weasel-like animals', q.all()
2.11.16 XML Persistence
Location: /examples/elementtree/ Illustrates three strategies for persisting and querying XML documents as represented by ElementTree in a relational database. The techniques do not apply any mappings to the ElementTree objects
directly, so are compatible with the native cElementTree as well as lxml, and can be adapted to suit any kind of DOM
representation system. Querying along xpath-like strings is illustrated as well.
In order of complexity:
• pickle.py - Quick and dirty, serialize the whole DOM into a BLOB column. While the example is very
brief, it has very limited functionality.
• adjacency_list.py - Each DOM node is stored in an individual table row, with attributes represented in
a separate table. The nodes are associated in a hierarchy using an adjacency list structure. A query function
is introduced which can search for nodes along any path with a given structure of attributes, basically a (very
narrow) subset of xpath.
• optimized_al.py - Uses the same strategy as adjacency_list.py, but associates each DOM row
with its owning document row, so that a full document of DOM nodes can be loaded using O(1) queries - the
construction of the “hierarchy” is performed after the load in a non-recursive fashion and is much more efficient.
2.11. Examples
291
SQLAlchemy Documentation, Release 0.8.7
E.g.:
# parse an XML file and persist in the database
doc = ElementTree.parse("test.xml")
session.add(Document(file, doc))
session.commit()
# locate documents with a certain path/attribute structure
for document in find_document('/somefile/header/field2[@attr=foo]'):
# dump the XML
print document
2.12 ORM Exceptions
SQLAlchemy ORM exceptions.
sqlalchemy.orm.exc.ConcurrentModificationError
alias of StaleDataError
exception sqlalchemy.orm.exc.DetachedInstanceError
An attempt to access unloaded attributes on a mapped instance that is detached.
exception sqlalchemy.orm.exc.FlushError
A invalid condition was detected during flush().
exception sqlalchemy.orm.exc.MultipleResultsFound
A single database result was required but more than one were found.
sqlalchemy.orm.exc.NO_STATE = (<type ‘exceptions.AttributeError’>, <type ‘exceptions.KeyError’>)
Exception types that may be raised by instrumentation implementations.
exception sqlalchemy.orm.exc.NoResultFound
A database result was required but none was found.
exception sqlalchemy.orm.exc.ObjectDeletedError(state, msg=None)
A refresh operation failed to retrieve the database row corresponding to an object’s known primary key identity.
A refresh operation proceeds when an expired attribute is accessed on an object, or when Query.get() is
used to retrieve an object which is, upon retrieval, detected as expired. A SELECT is emitted for the target row
based on primary key; if no row is returned, this exception is raised.
The true meaning of this exception is simply that no row exists for the primary key identifier associated with
a persistent object. The row may have been deleted, or in some cases the primary key updated to a new value,
outside of the ORM’s management of the target object.
exception sqlalchemy.orm.exc.ObjectDereferencedError
An operation cannot complete due to an object being garbage collected.
exception sqlalchemy.orm.exc.StaleDataError
An operation encountered database state that is unaccounted for.
Conditions which cause this to happen include:
•A flush may have attempted to update or delete rows and an unexpected number of rows were matched
during the UPDATE or DELETE statement. Note that when version_id_col is used, rows in UPDATE or
DELETE statements are also matched against the current known version identifier.
•A mapped object with version_id_col was refreshed, and the version number coming back from the
database does not match that of the object itself.
292
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
•A object is detached from its parent object, however the object was previously attached to a different
parent identity which was garbage collected, and a decision cannot be made if the new parent was really
the most recent “parent”.
New in version 0.7.4.
exception sqlalchemy.orm.exc.UnmappedClassError(cls, msg=None)
An mapping operation was requested for an unknown class.
exception sqlalchemy.orm.exc.UnmappedColumnError
Mapping operation was requested on an unknown column.
exception sqlalchemy.orm.exc.UnmappedError
Base for exceptions that involve expected mappings not present.
exception sqlalchemy.orm.exc.UnmappedInstanceError(obj, msg=None)
An mapping operation was requested for an unknown instance.
2.13 ORM Internals
Key ORM constructs, not otherwise covered in other sections, are listed here.
class sqlalchemy.orm.state.AttributeState(state, key)
Provide an inspection interface corresponding to a particular attribute on a particular mapped object.
The AttributeState object is accessed via the InstanceState.attrs collection of a particular
InstanceState:
from sqlalchemy import inspect
insp = inspect(some_mapped_object)
attr_state = insp.attrs.some_attribute
history
Return the current pre-flush change history for this attribute, via the History interface.
loaded_value
The current value of this attribute as loaded from the database.
If the value has not been loaded, or is otherwise not present in the object’s dictionary, returns NO_VALUE.
value
Return the value of this attribute.
This operation is equivalent to accessing the object’s attribute directly or via getattr(), and will fire
off any pending loader callables if needed.
class sqlalchemy.orm.instrumentation.ClassManager(class_)
Bases: __builtin__.dict
tracks state information at the class level.
__eq__
inherited from the __eq__ attribute of dict
x.__eq__(y) <==> x==y
__le__
inherited from the __le__ attribute of dict
x.__le__(y) <==> x<=y
2.13. ORM Internals
293
SQLAlchemy Documentation, Release 0.8.7
__lt__
inherited from the __lt__ attribute of dict
x.__lt__(y) <==> x<y
__ne__
inherited from the __ne__ attribute of dict
x.__ne__(y) <==> x!=y
clear() → None. Remove all items from D.
inherited from the clear() method of dict
copy() → a shallow copy of D
inherited from the copy() method of dict
dispose()
Dissasociate this manager from its class.
fromkeys(S[, v ]) → New dict with keys from S and values equal to v.
inherited from the fromkeys() method of dict
v defaults to None.
get(k[, d ]) → D[k] if k in D, else d. d defaults to None.
inherited from the get() method of dict
has_key(k) → True if D has a key k, else False
inherited from the has_key() method of dict
has_parent(state, key, optimistic=False)
TODO
items() → list of D’s (key, value) pairs, as 2-tuples
inherited from the items() method of dict
iteritems() → an iterator over the (key, value) items of D
inherited from the iteritems() method of dict
iterkeys() → an iterator over the keys of D
inherited from the iterkeys() method of dict
itervalues() → an iterator over the values of D
inherited from the itervalues() method of dict
keys() → list of D’s keys
inherited from the keys() method of dict
manage()
Mark this instance as the manager for its class.
original_init
x.__init__(...) initializes x; see help(type(x)) for signature
pop(k[, d ]) → v, remove specified key and return the corresponding value.
inherited from the pop() method of dict
If key is not found, d is returned if given, otherwise KeyError is raised
popitem() → (k, v), remove and return some (key, value) pair as a
inherited from the popitem() method of dict
2-tuple; but raise KeyError if D is empty.
294
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
setdefault(k[, d ]) → D.get(k,d), also set D[k]=d if k not in D
inherited from the setdefault() method of dict
state_getter()
Return a (instance) -> InstanceState callable.
“state getter” callables should raise either KeyError or AttributeError if no InstanceState could be found
for the instance.
unregister()
remove all instrumentation established by this ClassManager.
update([E ], **F) → None. Update D from dict/iterable E and F.
inherited from the update() method of dict
If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method,
does: for (k, v) in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]
values() → list of D’s values
inherited from the values() method of dict
viewitems() → a set-like object providing a view on D’s items
inherited from the viewitems() method of dict
viewkeys() → a set-like object providing a view on D’s keys
inherited from the viewkeys() method of dict
viewvalues() → an object providing a view on D’s values
inherited from the viewvalues() method of dict
class sqlalchemy.orm.properties.ColumnProperty(*columns, **kwargs)
Bases: sqlalchemy.orm.interfaces.StrategizedProperty
Describes an object attribute that corresponds to a table column.
Public constructor is the orm.column_property() function.
ColumnComparator
alias of Comparator
class Comparator(prop, parentmapper, adapter=None)
Bases: sqlalchemy.orm.interfaces.PropComparator
Produce boolean, comparison, and other operators for ColumnProperty attributes.
See the documentation for PropComparator for a brief overview.
See also:
PropComparator
ColumnOperators
Redefining and Creating New Operators
TypeEngine.comparator_factory
__eq__(other)
inherited from the __eq__() method of ColumnOperators
Implement the == operator.
In a column context, produces the clause a = b. If the target is None, produces a IS NULL.
2.13. ORM Internals
295
SQLAlchemy Documentation, Release 0.8.7
__le__(other)
inherited from the __le__() method of ColumnOperators
Implement the <= operator.
In a column context, produces the clause a <= b.
__lt__(other)
inherited from the __lt__() method of ColumnOperators
Implement the < operator.
In a column context, produces the clause a < b.
__ne__(other)
inherited from the __ne__() method of ColumnOperators
Implement the != operator.
In a column context, produces the clause a != b. If the target is None, produces a IS NOT
NULL.
adapted(adapter)
inherited from the adapted() method of PropComparator
Return a copy of this PropComparator which will use the given adaption function on the local side
of generated expressions.
any(criterion=None, **kwargs)
inherited from the any() method of PropComparator
Return true if this collection contains any member that meets the given criterion.
The usual implementation of any() is RelationshipProperty.Comparator.any().
Parameters
• criterion – an optional ClauseElement formulated against the member class’
table or attributes.
• **kwargs – key/value pairs corresponding to member class attribute names which
will be compared via equality to the corresponding values.
asc()
inherited from the asc() method of ColumnOperators
Produce a asc() clause against the parent object.
between(cleft, cright)
inherited from the between() method of ColumnOperators
Produce a between() clause against the parent object, given the lower and upper range.
collate(collation)
inherited from the collate() method of ColumnOperators
Produce a collate() clause against the parent object, given the collation string.
concat(other)
inherited from the concat() method of ColumnOperators
Implement the ‘concat’ operator.
In a column context, produces the clause a || b, or uses the concat() operator on MySQL.
296
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
contains(other, **kwargs)
inherited from the contains() method of ColumnOperators
Implement the ‘contains’ operator.
In a column context, produces the clause LIKE ’%<other>%’
desc()
inherited from the desc() method of ColumnOperators
Produce a desc() clause against the parent object.
distinct()
inherited from the distinct() method of ColumnOperators
Produce a distinct() clause against the parent object.
endswith(other, **kwargs)
inherited from the endswith() method of ColumnOperators
Implement the ‘endswith’ operator.
In a column context, produces the clause LIKE ’%<other>’
has(criterion=None, **kwargs)
inherited from the has() method of PropComparator
Return true if this element references a member which meets the given criterion.
The usual implementation of has() is RelationshipProperty.Comparator.has().
Parameters
• criterion – an optional ClauseElement formulated against the member class’
table or attributes.
• **kwargs – key/value pairs corresponding to member class attribute names which
will be compared via equality to the corresponding values.
ilike(other, escape=None)
inherited from the ilike() method of ColumnOperators
Implement the ilike operator.
In a column context, produces the clause a ILIKE other.
E.g.:
select([sometable]).where(sometable.c.column.ilike("%foobar%"))
Parameters
• other – expression to be compared
• escape – optional escape character, renders the ESCAPE keyword, e.g.:
somecolumn.ilike("foo/%bar", escape="/")
See also:
ColumnOperators.like()
2.13. ORM Internals
297
SQLAlchemy Documentation, Release 0.8.7
in_(other)
inherited from the in_() method of ColumnOperators
Implement the in operator.
In a column context, produces the clause a IN other. “other” may be a tuple/list of column
expressions, or a select() construct.
is_(other)
inherited from the is_() method of ColumnOperators
Implement the IS operator.
Normally, IS is generated automatically when comparing to a value of None, which resolves to
NULL. However, explicit usage of IS may be desirable if comparing to boolean values on certain
platforms.
New in version 0.7.9.
See also:
ColumnOperators.isnot()
isnot(other)
inherited from the isnot() method of ColumnOperators
Implement the IS NOT operator.
Normally, IS NOT is generated automatically when comparing to a value of None, which resolves
to NULL. However, explicit usage of IS NOT may be desirable if comparing to boolean values on
certain platforms.
New in version 0.7.9.
See also:
ColumnOperators.is_()
like(other, escape=None)
inherited from the like() method of ColumnOperators
Implement the like operator.
In a column context, produces the clause a LIKE other.
E.g.:
select([sometable]).where(sometable.c.column.like("%foobar%"))
Parameters
• other – expression to be compared
• escape – optional escape character, renders the ESCAPE keyword, e.g.:
somecolumn.like("foo/%bar", escape="/")
See also:
ColumnOperators.ilike()
match(other, **kwargs)
inherited from the match() method of ColumnOperators
Implements a database-specific ‘match’ operator.
298
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
match() attempts to resolve to a MATCH-like function or operator provided by the backend.
Examples include:
•Postgresql - renders x @@ to_tsquery(y)
•MySQL - renders MATCH (x) AGAINST (y IN BOOLEAN MODE)
•Oracle - renders CONTAINS(x, y)
•other backends may provide special implementations.
•Backends without any special implementation will emit the operator as “MATCH”. This is
compatible with SQlite, for example.
notilike(other, escape=None)
inherited from the notilike() method of ColumnOperators
implement the NOT ILIKE operator.
This is equivalent to using negation with ColumnOperators.ilike(), i.e. ~x.ilike(y).
New in version 0.8.
See also:
ColumnOperators.ilike()
notin_(other)
inherited from the notin_() method of ColumnOperators
implement the NOT IN operator.
This is equivalent to using negation with ColumnOperators.in_(), i.e. ~x.in_(y).
New in version 0.8.
See also:
ColumnOperators.in_()
notlike(other, escape=None)
inherited from the notlike() method of ColumnOperators
implement the NOT LIKE operator.
This is equivalent to using negation with ColumnOperators.like(), i.e. ~x.like(y).
New in version 0.8.
See also:
ColumnOperators.like()
nullsfirst()
inherited from the nullsfirst() method of ColumnOperators
Produce a nullsfirst() clause against the parent object.
nullslast()
inherited from the nullslast() method of ColumnOperators
Produce a nullslast() clause against the parent object.
of_type(class_)
inherited from the of_type() method of PropComparator
Redefine this object in terms of a polymorphic subclass.
Returns a new PropComparator from which further criterion can be evaluated.
2.13. ORM Internals
299
SQLAlchemy Documentation, Release 0.8.7
e.g.:
query.join(Company.employees.of_type(Engineer)).\
filter(Engineer.name=='foo')
Parameters class_ – a class or mapper indicating that criterion will be against this
specific subclass.
op(opstring, precedence=0)
inherited from the op() method of Operators
produce a generic operator function.
e.g.:
somecolumn.op("*")(5)
produces:
somecolumn * 5
This function can also be used to make bitwise operators explicit. For example:
somecolumn.op('&')(0xff)
is a bitwise AND of the value in somecolumn.
Parameters
• operator – a string which will be output as the infix operator between this element and the expression passed to the generated function.
• precedence – precedence to apply to the operator, when parenthesizing expressions. A lower number will cause the expression to be parenthesized when applied
against another operator with higher precedence. The default value of 0 is lower
than all operators except for the comma (,) and AS operators. A value of 100 will
be higher or equal to all operators, and -100 will be lower than or equal to all operators.
New in version 0.8: - added the ‘precedence’ argument.
See also:
Redefining and Creating New Operators
startswith(other, **kwargs)
inherited from the startswith() method of ColumnOperators
Implement the startwith operator.
In a column context, produces the clause LIKE ’<other>%’
ColumnProperty.__init__(*columns, **kwargs)
Construct a ColumnProperty.
Note the public constructor is the orm.column_property() function.
Parameters
• *columns – The list of columns describes a single object property. If there are multiple tables joined together for the mapper, this list represents the equivalent column
as it appears across each table.
• group –
300
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
• deferred –
• comparator_factory –
• descriptor –
• expire_on_flush –
• extension –
• info – Optional data dictionary which will be populated into the info attribute of
this object.
ColumnProperty.cascade_iterator(type_, state, visited_instances=None, halt_on=None)
inherited from the cascade_iterator() method of MapperProperty
Iterate through instances related to the given instance for a particular ‘cascade’, starting with this MapperProperty.
Return an iterator3-tuples (instance, mapper, state).
Note that the ‘cascade’ collection on this MapperProperty is checked first for the given type before cascade_iterator is called.
See PropertyLoader for the related instance implementation.
ColumnProperty.class_attribute
inherited from the class_attribute attribute of MapperProperty
Return the class-bound descriptor corresponding to this MapperProperty.
This is basically a getattr() call:
return getattr(self.parent.class_, self.key)
I.e. if this MapperProperty were named addresses, and the class to which it is mapped is User,
this sequence is possible:
>>> from sqlalchemy import inspect
>>> mapper = inspect(User)
>>> addresses_property = mapper.attrs.addresses
>>> addresses_property.class_attribute is User.addresses
True
>>> User.addresses.property is addresses_property
True
ColumnProperty.compare(operator, value, **kw)
inherited from the compare() method of MapperProperty
Return a compare operation for the columns represented by this MapperProperty to the given value,
which may be a column value or an instance. ‘operator’ is an operator from the operators module, or from
sql.Comparator.
By default uses the PropComparator attached to this MapperProperty under the attribute name “comparator”.
ColumnProperty.expression
Return the primary column or expression for this ColumnProperty.
ColumnProperty.info
inherited from the info attribute of MapperProperty
Info dictionary associated with the object, allowing user-defined data to be associated with this
MapperProperty.
2.13. ORM Internals
301
SQLAlchemy Documentation, Release 0.8.7
The dictionary is generated when first accessed. Alternatively, it can be specified as a constructor argument to the column_property(), relationship(), or composite() functions.
New in version 0.8: Added support for .info to all MapperProperty subclasses.
See also:
QueryableAttribute.info
SchemaItem.info
ColumnProperty.init()
inherited from the init() method of MapperProperty
Called after all mappers are created to assemble relationships between mappers and perform other postmapper-creation initialization steps.
ColumnProperty.is_primary()
inherited from the is_primary() method of MapperProperty
Return True if this MapperProperty‘s mapper is the primary mapper for its class.
This flag is used to indicate that the MapperProperty can define attribute instrumentation for the class
at the class level (as opposed to the individual instance level).
class sqlalchemy.orm.descriptor_props.CompositeProperty(class_, *attrs, **kwargs)
Bases: sqlalchemy.orm.descriptor_props.DescriptorProperty
Defines a “composite” mapped attribute, representing a collection of columns as one attribute.
CompositeProperty is constructed using the composite() function.
See also:
Composite Column Types
class Comparator(prop, parentmapper, adapter=None)
Bases: sqlalchemy.orm.interfaces.PropComparator
Produce boolean, comparison, and other operators for CompositeProperty attributes.
See the example in Redefining Comparison Operations for Composites for an overview of usage , as well
as the documentation for PropComparator.
See also:
PropComparator
ColumnOperators
Redefining and Creating New Operators
TypeEngine.comparator_factory
CompositeProperty.do_init()
Initialization which occurs after the CompositeProperty has been associated with its parent mapper.
CompositeProperty.get_history(state, dict_, passive=<symbol ‘PASSIVE_OFF>)
Provided for userland code that uses attributes.get_history().
class sqlalchemy.orm.interfaces._InspectionAttr
A base class applied to all ORM objects that can be returned by the inspect() function.
The attributes defined here allow the usage of simple boolean checks to test basic facts about the object returned.
302
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
While the boolean checks here are basically the same as using the Python isinstance() function, the flags here
can be used without the need to import all of these classes, and also such that the SQLAlchemy class system
can change while leaving the flags here intact for forwards-compatibility.
extension_type = <symbol ‘NOT_EXTENSION>
The extension type, if any. Defaults to interfaces.NOT_EXTENSION
New in version 0.8.0.
See also:
HYBRID_METHOD
HYBRID_PROPERTY
ASSOCIATION_PROXY
is_aliased_class = False
True if this object is an instance of AliasedClass.
is_attribute = False
True if this object is a Python descriptor.
This can refer to one of many types. Usually a QueryableAttribute which handles attributes events
on behalf of a MapperProperty. But can also be an extension type such as AssociationProxy
or hybrid_property. The _InspectionAttr.extension_type will refer to a constant identifying the specific subtype.
See also:
Mapper.all_orm_descriptors
is_clause_element = False
True if this object is an instance of ClauseElement.
is_instance = False
True if this object is an instance of InstanceState.
is_mapper = False
True if this object is an instance of Mapper.
is_property = False
True if this object is an instance of MapperProperty.
is_selectable = False
Return True if this object is an instance of Selectable.
class sqlalchemy.orm.state.InstanceState(obj, manager)
Bases: sqlalchemy.orm.interfaces._InspectionAttr
tracks state information at the instance level.
The InstanceState is a key object used by the SQLAlchemy ORM in order to track the state of an object;
it is created the moment an object is instantiated, typically as a result of instrumentation which SQLAlchemy
applies to the __init__() method of the class.
InstanceState is also a semi-public object, available for runtime inspection as to the state of a mapped
instance, including information such as its current status within a particular Session and details about data on
individual attributes. The public API in order to acquire a InstanceState object is to use the inspect()
system:
>>> from sqlalchemy import inspect
>>> insp = inspect(some_mapped_object)
2.13. ORM Internals
303
SQLAlchemy Documentation, Release 0.8.7
See also:
Runtime Inspection API
__call__(state, passive)
__call__ allows the InstanceState to act as a deferred callable for loading expired attributes, which is also
serializable (picklable).
attrs
Return a namespace representing each attribute on the mapped object, including its current value and
history.
The returned object is an instance of AttributeState. This object allows inspection of the current
data within an attribute as well as attribute history since the last flush.
detached
Return true if the object is detached.
See also:
Quickie Intro to Object States
dict
Return the instance dict used by the object.
Under normal circumstances, this is always synonymous with the __dict__ attribute of the mapped
object, unless an alternative instrumentation system has been configured.
In the case that the actual object has been garbage collected, this accessor returns a blank dictionary.
expired_attributes
Return the set of keys which are ‘expired’ to be loaded by the manager’s deferred scalar loader, assuming
no pending changes.
see also the unmodified collection which is intersected against this set when a refresh operation occurs.
has_identity
Return True if this object has an identity key.
This should always have the same value as the expression state.persistent or
state.detached.
identity
Return the mapped identity of the mapped object. This is the primary key identity as persisted by the
ORM which can always be passed directly to Query.get().
Returns None if the object has no primary key identity.
Note: An object which is transient or pending does not have a mapped identity until it is flushed, even if
its attributes include primary key values.
identity_key
Return the identity key for the mapped object.
This is the key used to locate the object within the Session.identity_map mapping. It contains the
identity as returned by identity within it.
mapper
Return the Mapper used for this mapepd object.
object
Return the mapped object represented by this InstanceState.
304
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
pending
Return true if the object is pending.
See also:
Quickie Intro to Object States
persistent
Return true if the object is persistent.
See also:
Quickie Intro to Object States
session
Return the owning Session for this instance, or None if none available.
transient
Return true if the object is transient.
See also:
Quickie Intro to Object States
unloaded
Return the set of keys which do not have a loaded value.
This includes expired attributes and any other attribute that was never populated or modified.
unmodified
Return the set of keys which have no uncommitted changes
unmodified_intersection(keys)
Return self.unmodified.intersection(keys).
class sqlalchemy.orm.attributes.InstrumentedAttribute(class_,
key,
impl=None,
comparator=None,
parententity=None, of_type=None)
Bases: sqlalchemy.orm.attributes.QueryableAttribute
Class bound instrumented attribute which adds basic descriptor methods.
See QueryableAttribute for a description of most features.
__delete__(instance)
__get__(instance, owner)
__set__(instance, value)
class sqlalchemy.orm.interfaces.MapperProperty
Bases: sqlalchemy.orm.interfaces._MappedAttribute, sqlalchemy.orm.interfaces._InspectionAt
Manage the relationship of a Mapper to a single class attribute, as well as that attribute as it appears on individual instances of the class, including attribute instrumentation, attribute access, loading behavior, and dependency
calculations.
The most common occurrences of MapperProperty are the mapped Column, which is represented in a mapping as an instance of ColumnProperty, and a reference to another class produced by relationship(),
represented in the mapping as an instance of RelationshipProperty.
cascade = frozenset([])
The set of ‘cascade’ attribute names.
This collection is checked before the ‘cascade_iterator’ method is called.
2.13. ORM Internals
305
SQLAlchemy Documentation, Release 0.8.7
cascade_iterator(type_, state, visited_instances=None, halt_on=None)
Iterate through instances related to the given instance for a particular ‘cascade’, starting with this MapperProperty.
Return an iterator3-tuples (instance, mapper, state).
Note that the ‘cascade’ collection on this MapperProperty is checked first for the given type before cascade_iterator is called.
See PropertyLoader for the related instance implementation.
class_attribute
Return the class-bound descriptor corresponding to this MapperProperty.
This is basically a getattr() call:
return getattr(self.parent.class_, self.key)
I.e. if this MapperProperty were named addresses, and the class to which it is mapped is User,
this sequence is possible:
>>> from sqlalchemy import inspect
>>> mapper = inspect(User)
>>> addresses_property = mapper.attrs.addresses
>>> addresses_property.class_attribute is User.addresses
True
>>> User.addresses.property is addresses_property
True
compare(operator, value, **kw)
Return a compare operation for the columns represented by this MapperProperty to the given value,
which may be a column value or an instance. ‘operator’ is an operator from the operators module, or from
sql.Comparator.
By default uses the PropComparator attached to this MapperProperty under the attribute name “comparator”.
create_row_processor(context, path, mapper, row, adapter)
Return a 3-tuple consisting of three row processing functions.
do_init()
Perform subclass-specific initialization post-mapper-creation steps.
This is a template method called by the MapperProperty object’s init() method.
info
Info dictionary associated with the object, allowing user-defined data to be associated with this
MapperProperty.
The dictionary is generated when first accessed. Alternatively, it can be specified as a constructor argument to the column_property(), relationship(), or composite() functions.
New in version 0.8: Added support for .info to all MapperProperty subclasses.
See also:
QueryableAttribute.info
SchemaItem.info
init()
Called after all mappers are created to assemble relationships between mappers and perform other postmapper-creation initialization steps.
306
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
is_primary()
Return True if this MapperProperty‘s mapper is the primary mapper for its class.
This flag is used to indicate that the MapperProperty can define attribute instrumentation for the class
at the class level (as opposed to the individual instance level).
merge(session, source_state, source_dict, dest_state, dest_dict, load, _recursive)
Merge the attribute represented by this MapperProperty from source to destination object
post_instrument_class(mapper)
Perform instrumentation adjustments that need to occur after init() has completed.
setup(context, entity, path, adapter, **kwargs)
Called by Query for the purposes of constructing a SQL statement.
Each MapperProperty associated with the target mapper processes the statement referenced by the query
context, adding columns and/or criterion as appropriate.
sqlalchemy.orm.interfaces.NOT_EXTENSION = <symbol ‘NOT_EXTENSION>
class sqlalchemy.orm.interfaces.PropComparator(prop, parentmapper, adapter=None)
Bases: sqlalchemy.sql.operators.ColumnOperators
Defines boolean, comparison, and other operators for MapperProperty objects.
SQLAlchemy allows for operators to be redefined at both the Core and ORM level. PropComparator
is the base class of operator redefinition for ORM-level operations, including those of ColumnProperty,
RelationshipProperty, and CompositeProperty.
Note: With the advent of Hybrid properties introduced in SQLAlchemy 0.7, as well as Core-level operator
redefinition in SQLAlchemy 0.8, the use case for user-defined PropComparator instances is extremely rare.
See Hybrid Attributes as well as Redefining and Creating New Operators.
User-defined subclasses of PropComparator may be created.
The built-in Python comparison and math operator methods, such as operators.ColumnOperators.__eq__(),
operators.ColumnOperators.__lt__(), and operators.ColumnOperators.__add__(),
can be overridden to provide new operator behavior. The custom PropComparator is passed to the
MapperProperty instance via the comparator_factory argument. In each case, the appropriate
subclass of PropComparator should be used:
# definition of custom PropComparator subclasses
from sqlalchemy.orm.properties import \
ColumnProperty,\
CompositeProperty,\
RelationshipProperty
class MyColumnComparator(ColumnProperty.Comparator):
def __eq__(self, other):
return self.__clause_element__() == other
class MyRelationshipComparator(RelationshipProperty.Comparator):
def any(self, expression):
"define the 'any' operation"
# ...
class MyCompositeComparator(CompositeProperty.Comparator):
def __gt__(self, other):
"redefine the 'greater than' operation"
2.13. ORM Internals
307
SQLAlchemy Documentation, Release 0.8.7
return sql.and_(*[a>b for a, b in
zip(self.__clause_element__().clauses,
other.__composite_values__())])
# application of custom PropComparator subclasses
from sqlalchemy.orm import column_property, relationship, composite
from sqlalchemy import Column, String
class SomeMappedClass(Base):
some_column = column_property(Column("some_column", String),
comparator_factory=MyColumnComparator)
some_relationship = relationship(SomeOtherClass,
comparator_factory=MyRelationshipComparator)
some_composite = composite(
Column("a", String), Column("b", String),
comparator_factory=MyCompositeComparator
)
Note that for column-level operator redefinition, it’s usually simpler to define the operators at the Core level,
using the TypeEngine.comparator_factory attribute. See Redefining and Creating New Operators for
more detail.
See also:
ColumnProperty.Comparator
RelationshipProperty.Comparator
CompositeProperty.Comparator
ColumnOperators
Redefining and Creating New Operators
TypeEngine.comparator_factory
__eq__(other)
inherited from the __eq__() method of ColumnOperators
Implement the == operator.
In a column context, produces the clause a = b. If the target is None, produces a IS NULL.
__le__(other)
inherited from the __le__() method of ColumnOperators
Implement the <= operator.
In a column context, produces the clause a <= b.
__lt__(other)
inherited from the __lt__() method of ColumnOperators
Implement the < operator.
In a column context, produces the clause a < b.
__ne__(other)
inherited from the __ne__() method of ColumnOperators
308
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
Implement the != operator.
In a column context, produces the clause a != b. If the target is None, produces a IS NOT NULL.
adapted(adapter)
Return a copy of this PropComparator which will use the given adaption function on the local side of
generated expressions.
any(criterion=None, **kwargs)
Return true if this collection contains any member that meets the given criterion.
The usual implementation of any() is RelationshipProperty.Comparator.any().
Parameters
• criterion – an optional ClauseElement formulated against the member class’ table or attributes.
• **kwargs – key/value pairs corresponding to member class attribute names which
will be compared via equality to the corresponding values.
asc()
inherited from the asc() method of ColumnOperators
Produce a asc() clause against the parent object.
between(cleft, cright)
inherited from the between() method of ColumnOperators
Produce a between() clause against the parent object, given the lower and upper range.
collate(collation)
inherited from the collate() method of ColumnOperators
Produce a collate() clause against the parent object, given the collation string.
concat(other)
inherited from the concat() method of ColumnOperators
Implement the ‘concat’ operator.
In a column context, produces the clause a || b, or uses the concat() operator on MySQL.
contains(other, **kwargs)
inherited from the contains() method of ColumnOperators
Implement the ‘contains’ operator.
In a column context, produces the clause LIKE ’%<other>%’
desc()
inherited from the desc() method of ColumnOperators
Produce a desc() clause against the parent object.
distinct()
inherited from the distinct() method of ColumnOperators
Produce a distinct() clause against the parent object.
endswith(other, **kwargs)
inherited from the endswith() method of ColumnOperators
Implement the ‘endswith’ operator.
In a column context, produces the clause LIKE ’%<other>’
2.13. ORM Internals
309
SQLAlchemy Documentation, Release 0.8.7
has(criterion=None, **kwargs)
Return true if this element references a member which meets the given criterion.
The usual implementation of has() is RelationshipProperty.Comparator.has().
Parameters
• criterion – an optional ClauseElement formulated against the member class’ table or attributes.
• **kwargs – key/value pairs corresponding to member class attribute names which
will be compared via equality to the corresponding values.
ilike(other, escape=None)
inherited from the ilike() method of ColumnOperators
Implement the ilike operator.
In a column context, produces the clause a ILIKE other.
E.g.:
select([sometable]).where(sometable.c.column.ilike("%foobar%"))
Parameters
• other – expression to be compared
• escape – optional escape character, renders the ESCAPE keyword, e.g.:
somecolumn.ilike("foo/%bar", escape="/")
See also:
ColumnOperators.like()
in_(other)
inherited from the in_() method of ColumnOperators
Implement the in operator.
In a column context, produces the clause a IN other. “other” may be a tuple/list of column expressions, or a select() construct.
is_(other)
inherited from the is_() method of ColumnOperators
Implement the IS operator.
Normally, IS is generated automatically when comparing to a value of None, which resolves to NULL.
However, explicit usage of IS may be desirable if comparing to boolean values on certain platforms.
New in version 0.7.9.
See also:
ColumnOperators.isnot()
isnot(other)
inherited from the isnot() method of ColumnOperators
Implement the IS NOT operator.
310
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
Normally, IS NOT is generated automatically when comparing to a value of None, which resolves to
NULL. However, explicit usage of IS NOT may be desirable if comparing to boolean values on certain
platforms.
New in version 0.7.9.
See also:
ColumnOperators.is_()
like(other, escape=None)
inherited from the like() method of ColumnOperators
Implement the like operator.
In a column context, produces the clause a LIKE other.
E.g.:
select([sometable]).where(sometable.c.column.like("%foobar%"))
Parameters
• other – expression to be compared
• escape – optional escape character, renders the ESCAPE keyword, e.g.:
somecolumn.like("foo/%bar", escape="/")
See also:
ColumnOperators.ilike()
match(other, **kwargs)
inherited from the match() method of ColumnOperators
Implements a database-specific ‘match’ operator.
match() attempts to resolve to a MATCH-like function or operator provided by the backend. Examples
include:
•Postgresql - renders x @@ to_tsquery(y)
•MySQL - renders MATCH (x) AGAINST (y IN BOOLEAN MODE)
•Oracle - renders CONTAINS(x, y)
•other backends may provide special implementations.
•Backends without any special implementation will emit the operator as “MATCH”. This is compatible with SQlite, for example.
notilike(other, escape=None)
inherited from the notilike() method of ColumnOperators
implement the NOT ILIKE operator.
This is equivalent to using negation with ColumnOperators.ilike(), i.e. ~x.ilike(y).
New in version 0.8.
See also:
ColumnOperators.ilike()
2.13. ORM Internals
311
SQLAlchemy Documentation, Release 0.8.7
notin_(other)
inherited from the notin_() method of ColumnOperators
implement the NOT IN operator.
This is equivalent to using negation with ColumnOperators.in_(), i.e. ~x.in_(y).
New in version 0.8.
See also:
ColumnOperators.in_()
notlike(other, escape=None)
inherited from the notlike() method of ColumnOperators
implement the NOT LIKE operator.
This is equivalent to using negation with ColumnOperators.like(), i.e. ~x.like(y).
New in version 0.8.
See also:
ColumnOperators.like()
nullsfirst()
inherited from the nullsfirst() method of ColumnOperators
Produce a nullsfirst() clause against the parent object.
nullslast()
inherited from the nullslast() method of ColumnOperators
Produce a nullslast() clause against the parent object.
of_type(class_)
Redefine this object in terms of a polymorphic subclass.
Returns a new PropComparator from which further criterion can be evaluated.
e.g.:
query.join(Company.employees.of_type(Engineer)).\
filter(Engineer.name=='foo')
Parameters class_ – a class or mapper indicating that criterion will be against this specific
subclass.
op(opstring, precedence=0)
inherited from the op() method of Operators
produce a generic operator function.
e.g.:
somecolumn.op("*")(5)
produces:
somecolumn * 5
This function can also be used to make bitwise operators explicit. For example:
somecolumn.op('&')(0xff)
312
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
is a bitwise AND of the value in somecolumn.
Parameters
• operator – a string which will be output as the infix operator between this element
and the expression passed to the generated function.
• precedence – precedence to apply to the operator, when parenthesizing expressions. A lower number will cause the expression to be parenthesized when applied
against another operator with higher precedence. The default value of 0 is lower than
all operators except for the comma (,) and AS operators. A value of 100 will be
higher or equal to all operators, and -100 will be lower than or equal to all operators.
New in version 0.8: - added the ‘precedence’ argument.
See also:
Redefining and Creating New Operators
operate(op, *other, **kwargs)
inherited from the operate() method of Operators
Operate on an argument.
This is the lowest level of operation, raises NotImplementedError by default.
Overriding this on a subclass can allow common behavior to be applied to all operations. For example,
overriding ColumnOperators to apply func.lower() to the left and right side:
class MyComparator(ColumnOperators):
def operate(self, op, other):
return op(func.lower(self), func.lower(other))
Parameters
• op – Operator callable.
• *other – the ‘other’ side of the operation. Will be a single scalar for most operations.
• **kwargs – modifiers. These may be passed by special operators such as
ColumnOperators.contains().
reverse_operate(op, other, **kwargs)
inherited from the reverse_operate() method of Operators
Reverse operate on an argument.
Usage is the same as operate().
startswith(other, **kwargs)
inherited from the startswith() method of ColumnOperators
Implement the startwith operator.
In a column context, produces the clause LIKE ’<other>%’
2.13. ORM Internals
313
SQLAlchemy Documentation, Release 0.8.7
class sqlalchemy.orm.properties.RelationshipProperty(argument,
secondary=None,
primaryjoin=None,
secondaryjoin=None,
foreign_keys=None, uselist=None,
order_by=False,
backref=None, back_populates=None,
post_update=False,
cascade=False,
extension=None,
viewonly=False,
lazy=True,
collection_class=None,
passive_deletes=False,
passive_updates=True,
remote_side=None,
enable_typechecks=True,
join_depth=None,
comparator_factory=None,
single_parent=False,
innerjoin=False,
distinct_target_key=False,
doc=None, active_history=False,
cascade_backrefs=True,
load_on_pending=False,
strategy_class=None,
_local_remote_pairs=None,
query_class=None, info=None)
Bases: sqlalchemy.orm.interfaces.StrategizedProperty
Describes an object property that holds a single item or list of items that correspond to a related database table.
Public constructor is the orm.relationship() function.
See also:
Relationship Configuration
class Comparator(prop, parentmapper, of_type=None, adapter=None)
Bases: sqlalchemy.orm.interfaces.PropComparator
Produce boolean, comparison, and other operators for RelationshipProperty attributes.
See the documentation for PropComparator for a brief overview of ORM level operator definition.
See also:
PropComparator
ColumnProperty.Comparator
ColumnOperators
Redefining and Creating New Operators
TypeEngine.comparator_factory
__eq__(other)
Implement the == operator.
In a many-to-one context, such as:
MyClass.some_prop == <some object>
this will typically produce a clause such as:
314
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
mytable.related_id == <some id>
Where <some id> is the primary key of the given object.
The == operator provides partial functionality for non- many-to-one comparisons:
•Comparisons against collections are not supported. Use contains().
•Compared to a scalar one-to-many, will produce a clause that compares the target columns in
the parent to the given target.
•Compared to a scalar many-to-many, an alias of the association table will be rendered as well,
forming a natural join that is part of the main body of the query. This will not work for queries
that go beyond simple AND conjunctions of comparisons, such as those which use OR. Use explicit joins, outerjoins, or has() for more comprehensive non-many-to-one scalar membership
tests.
•Comparisons against None given in a one-to-many or many-to-many context produce a NOT
EXISTS clause.
__init__(prop, parentmapper, of_type=None, adapter=None)
Construction of RelationshipProperty.Comparator is internal to the ORM’s attribute
mechanics.
__le__(other)
inherited from the __le__() method of ColumnOperators
Implement the <= operator.
In a column context, produces the clause a <= b.
__lt__(other)
inherited from the __lt__() method of ColumnOperators
Implement the < operator.
In a column context, produces the clause a < b.
__ne__(other)
Implement the != operator.
In a many-to-one context, such as:
MyClass.some_prop != <some object>
This will typically produce a clause such as:
mytable.related_id != <some id>
Where <some id> is the primary key of the given object.
The != operator provides partial functionality for non- many-to-one comparisons:
•Comparisons against collections are not supported. Use contains() in conjunction with
not_().
•Compared to a scalar one-to-many, will produce a clause that compares the target columns in
the parent to the given target.
•Compared to a scalar many-to-many, an alias of the association table will be rendered as well,
forming a natural join that is part of the main body of the query. This will not work for queries
that go beyond simple AND conjunctions of comparisons, such as those which use OR. Use
explicit joins, outerjoins, or has() in conjunction with not_() for more comprehensive nonmany-to-one scalar membership tests.
2.13. ORM Internals
315
SQLAlchemy Documentation, Release 0.8.7
•Comparisons against None given in a one-to-many or many-to-many context produce an EXISTS clause.
adapted(adapter)
Return a copy of this PropComparator which will use the given adaption function on the local side
of generated expressions.
any(criterion=None, **kwargs)
Produce an expression that tests a collection against particular criterion, using EXISTS.
An expression like:
session.query(MyClass).filter(
MyClass.somereference.any(SomeRelated.x==2)
)
Will produce a query like:
SELECT * FROM my_table WHERE
EXISTS (SELECT 1 FROM related WHERE related.my_id=my_table.id
AND related.x=2)
Because any() uses a correlated subquery, its performance is not nearly as good when compared
against large target tables as that of using a join.
any() is particularly useful for testing for empty collections:
session.query(MyClass).filter(
~MyClass.somereference.any()
)
will produce:
SELECT * FROM my_table WHERE
NOT EXISTS (SELECT 1 FROM related WHERE
related.my_id=my_table.id)
any() is only valid for collections, i.e. a relationship() that has uselist=True. For
scalar references, use has().
asc()
inherited from the asc() method of ColumnOperators
Produce a asc() clause against the parent object.
between(cleft, cright)
inherited from the between() method of ColumnOperators
Produce a between() clause against the parent object, given the lower and upper range.
collate(collation)
inherited from the collate() method of ColumnOperators
Produce a collate() clause against the parent object, given the collation string.
concat(other)
inherited from the concat() method of ColumnOperators
Implement the ‘concat’ operator.
In a column context, produces the clause a || b, or uses the concat() operator on MySQL.
contains(other, **kwargs)
Return a simple expression that tests a collection for containment of a particular item.
316
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
contains() is only valid for a collection, i.e. a relationship() that implements one-tomany or many-to-many with uselist=True.
When used in a simple one-to-many context, an expression like:
MyClass.contains(other)
Produces a clause like:
mytable.id == <some id>
Where <some id> is the value of the foreign key attribute on other which refers to the primary
key of its parent object. From this it follows that contains() is very useful when used with
simple one-to-many operations.
For many-to-many operations, the behavior of contains() has more caveats. The association
table will be rendered in the statement, producing an “implicit” join, that is, includes multiple tables
in the FROM clause which are equated in the WHERE clause:
query(MyClass).filter(MyClass.contains(other))
Produces a query like:
SELECT * FROM my_table, my_association_table AS
my_association_table_1 WHERE
my_table.id = my_association_table_1.parent_id
AND my_association_table_1.child_id = <some id>
Where <some id> would be the primary key of other. From the above, it is clear that
contains() will not work with many-to-many collections when used in queries that move beyond
simple AND conjunctions, such as multiple contains() expressions joined by OR. In such cases
subqueries or explicit “outer joins” will need to be used instead. See any() for a less-performant
alternative using EXISTS, or refer to Query.outerjoin() as well as Querying with Joins for
more details on constructing outer joins.
desc()
inherited from the desc() method of ColumnOperators
Produce a desc() clause against the parent object.
distinct()
inherited from the distinct() method of ColumnOperators
Produce a distinct() clause against the parent object.
endswith(other, **kwargs)
inherited from the endswith() method of ColumnOperators
Implement the ‘endswith’ operator.
In a column context, produces the clause LIKE ’%<other>’
has(criterion=None, **kwargs)
Produce an expression that tests a scalar reference against particular criterion, using EXISTS.
An expression like:
session.query(MyClass).filter(
MyClass.somereference.has(SomeRelated.x==2)
)
Will produce a query like:
2.13. ORM Internals
317
SQLAlchemy Documentation, Release 0.8.7
SELECT * FROM my_table WHERE
EXISTS (SELECT 1 FROM related WHERE
related.id==my_table.related_id AND related.x=2)
Because has() uses a correlated subquery, its performance is not nearly as good when compared
against large target tables as that of using a join.
has() is only valid for scalar references, i.e. a relationship() that has uselist=False.
For collection references, use any().
ilike(other, escape=None)
inherited from the ilike() method of ColumnOperators
Implement the ilike operator.
In a column context, produces the clause a ILIKE other.
E.g.:
select([sometable]).where(sometable.c.column.ilike("%foobar%"))
Parameters
• other – expression to be compared
• escape – optional escape character, renders the ESCAPE keyword, e.g.:
somecolumn.ilike("foo/%bar", escape="/")
See also:
ColumnOperators.like()
in_(other)
Produce an IN clause - this is not implemented for relationship()-based attributes at this time.
is_(other)
inherited from the is_() method of ColumnOperators
Implement the IS operator.
Normally, IS is generated automatically when comparing to a value of None, which resolves to
NULL. However, explicit usage of IS may be desirable if comparing to boolean values on certain
platforms.
New in version 0.7.9.
See also:
ColumnOperators.isnot()
isnot(other)
inherited from the isnot() method of ColumnOperators
Implement the IS NOT operator.
Normally, IS NOT is generated automatically when comparing to a value of None, which resolves
to NULL. However, explicit usage of IS NOT may be desirable if comparing to boolean values on
certain platforms.
New in version 0.7.9.
See also:
318
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
ColumnOperators.is_()
like(other, escape=None)
inherited from the like() method of ColumnOperators
Implement the like operator.
In a column context, produces the clause a LIKE other.
E.g.:
select([sometable]).where(sometable.c.column.like("%foobar%"))
Parameters
• other – expression to be compared
• escape – optional escape character, renders the ESCAPE keyword, e.g.:
somecolumn.like("foo/%bar", escape="/")
See also:
ColumnOperators.ilike()
mapper
The target Mapper referred to by this :class:‘.RelationshipProperty.Comparator.
This is the “target” or “remote” side of the relationship().
match(other, **kwargs)
inherited from the match() method of ColumnOperators
Implements a database-specific ‘match’ operator.
match() attempts to resolve to a MATCH-like function or operator provided by the backend.
Examples include:
•Postgresql - renders x @@ to_tsquery(y)
•MySQL - renders MATCH (x) AGAINST (y IN BOOLEAN MODE)
•Oracle - renders CONTAINS(x, y)
•other backends may provide special implementations.
•Backends without any special implementation will emit the operator as “MATCH”. This is
compatible with SQlite, for example.
notilike(other, escape=None)
inherited from the notilike() method of ColumnOperators
implement the NOT ILIKE operator.
This is equivalent to using negation with ColumnOperators.ilike(), i.e. ~x.ilike(y).
New in version 0.8.
See also:
ColumnOperators.ilike()
notin_(other)
inherited from the notin_() method of ColumnOperators
implement the NOT IN operator.
2.13. ORM Internals
319
SQLAlchemy Documentation, Release 0.8.7
This is equivalent to using negation with ColumnOperators.in_(), i.e. ~x.in_(y).
New in version 0.8.
See also:
ColumnOperators.in_()
notlike(other, escape=None)
inherited from the notlike() method of ColumnOperators
implement the NOT LIKE operator.
This is equivalent to using negation with ColumnOperators.like(), i.e. ~x.like(y).
New in version 0.8.
See also:
ColumnOperators.like()
nullsfirst()
inherited from the nullsfirst() method of ColumnOperators
Produce a nullsfirst() clause against the parent object.
nullslast()
inherited from the nullslast() method of ColumnOperators
Produce a nullslast() clause against the parent object.
of_type(cls)
Produce a construct that represents a particular ‘subtype’ of attribute for the parent class.
Currently this is usable in conjunction with Query.join() and Query.outerjoin().
op(opstring, precedence=0)
inherited from the op() method of Operators
produce a generic operator function.
e.g.:
somecolumn.op("*")(5)
produces:
somecolumn * 5
This function can also be used to make bitwise operators explicit. For example:
somecolumn.op('&')(0xff)
is a bitwise AND of the value in somecolumn.
Parameters
• operator – a string which will be output as the infix operator between this element and the expression passed to the generated function.
• precedence – precedence to apply to the operator, when parenthesizing expressions. A lower number will cause the expression to be parenthesized when applied
against another operator with higher precedence. The default value of 0 is lower
than all operators except for the comma (,) and AS operators. A value of 100 will
be higher or equal to all operators, and -100 will be lower than or equal to all operators.
320
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
New in version 0.8: - added the ‘precedence’ argument.
See also:
Redefining and Creating New Operators
operate(op, *other, **kwargs)
inherited from the operate() method of Operators
Operate on an argument.
This is the lowest level of operation, raises NotImplementedError by default.
Overriding this on a subclass can allow common behavior to be applied to all operations. For example, overriding ColumnOperators to apply func.lower() to the left and right side:
class MyComparator(ColumnOperators):
def operate(self, op, other):
return op(func.lower(self), func.lower(other))
Parameters
• op – Operator callable.
• *other – the ‘other’ side of the operation. Will be a single scalar for most operations.
• **kwargs – modifiers. These may be passed by special operators such as
ColumnOperators.contains().
reverse_operate(op, other, **kwargs)
inherited from the reverse_operate() method of Operators
Reverse operate on an argument.
Usage is the same as operate().
startswith(other, **kwargs)
inherited from the startswith() method of ColumnOperators
Implement the startwith operator.
In a column context, produces the clause LIKE ’<other>%’
RelationshipProperty.cascade
Return the current cascade setting for this RelationshipProperty.
RelationshipProperty.class_attribute
inherited from the class_attribute attribute of MapperProperty
Return the class-bound descriptor corresponding to this MapperProperty.
This is basically a getattr() call:
return getattr(self.parent.class_, self.key)
I.e. if this MapperProperty were named addresses, and the class to which it is mapped is User,
this sequence is possible:
>>> from sqlalchemy import inspect
>>> mapper = inspect(User)
>>> addresses_property = mapper.attrs.addresses
>>> addresses_property.class_attribute is User.addresses
True
2.13. ORM Internals
321
SQLAlchemy Documentation, Release 0.8.7
>>> User.addresses.property is addresses_property
True
RelationshipProperty.info
inherited from the info attribute of MapperProperty
Info dictionary associated with the object, allowing user-defined data to be associated with this
MapperProperty.
The dictionary is generated when first accessed. Alternatively, it can be specified as a constructor argument to the column_property(), relationship(), or composite() functions.
New in version 0.8: Added support for .info to all MapperProperty subclasses.
See also:
QueryableAttribute.info
SchemaItem.info
RelationshipProperty.init()
inherited from the init() method of MapperProperty
Called after all mappers are created to assemble relationships between mappers and perform other postmapper-creation initialization steps.
RelationshipProperty.is_primary()
inherited from the is_primary() method of MapperProperty
Return True if this MapperProperty‘s mapper is the primary mapper for its class.
This flag is used to indicate that the MapperProperty can define attribute instrumentation for the class
at the class level (as opposed to the individual instance level).
RelationshipProperty.mapper
Return the targeted Mapper for this RelationshipProperty.
This is a lazy-initializing static attribute.
RelationshipProperty.table
Return the selectable linked to this
Deprecated since version 0.7: Use .target
RelationshipProperty object’s target Mapper.
class sqlalchemy.orm.descriptor_props.SynonymProperty(name,
map_column=None,
descriptor=None,
comparator_factory=None, doc=None)
Bases: sqlalchemy.orm.descriptor_props.DescriptorProperty
cascade_iterator(type_, state, visited_instances=None, halt_on=None)
inherited from the cascade_iterator() method of MapperProperty
Iterate through instances related to the given instance for a particular ‘cascade’, starting with this MapperProperty.
Return an iterator3-tuples (instance, mapper, state).
Note that the ‘cascade’ collection on this MapperProperty is checked first for the given type before cascade_iterator is called.
See PropertyLoader for the related instance implementation.
322
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
class_attribute
inherited from the class_attribute attribute of MapperProperty
Return the class-bound descriptor corresponding to this MapperProperty.
This is basically a getattr() call:
return getattr(self.parent.class_, self.key)
I.e. if this MapperProperty were named addresses, and the class to which it is mapped is User,
this sequence is possible:
>>> from sqlalchemy import inspect
>>> mapper = inspect(User)
>>> addresses_property = mapper.attrs.addresses
>>> addresses_property.class_attribute is User.addresses
True
>>> User.addresses.property is addresses_property
True
compare(operator, value, **kw)
inherited from the compare() method of MapperProperty
Return a compare operation for the columns represented by this MapperProperty to the given value,
which may be a column value or an instance. ‘operator’ is an operator from the operators module, or from
sql.Comparator.
By default uses the PropComparator attached to this MapperProperty under the attribute name “comparator”.
create_row_processor(context, path, mapper, row, adapter)
inherited from the create_row_processor() method of MapperProperty
Return a 3-tuple consisting of three row processing functions.
do_init()
inherited from the do_init() method of MapperProperty
Perform subclass-specific initialization post-mapper-creation steps.
This is a template method called by the MapperProperty object’s init() method.
info
inherited from the info attribute of MapperProperty
Info dictionary associated with the object, allowing user-defined data to be associated with this
MapperProperty.
The dictionary is generated when first accessed. Alternatively, it can be specified as a constructor argument to the column_property(), relationship(), or composite() functions.
New in version 0.8: Added support for .info to all MapperProperty subclasses.
See also:
QueryableAttribute.info
SchemaItem.info
init()
inherited from the init() method of MapperProperty
Called after all mappers are created to assemble relationships between mappers and perform other postmapper-creation initialization steps.
2.13. ORM Internals
323
SQLAlchemy Documentation, Release 0.8.7
is_primary()
inherited from the is_primary() method of MapperProperty
Return True if this MapperProperty‘s mapper is the primary mapper for its class.
This flag is used to indicate that the MapperProperty can define attribute instrumentation for the class
at the class level (as opposed to the individual instance level).
merge(session, source_state, source_dict, dest_state, dest_dict, load, _recursive)
inherited from the merge() method of MapperProperty
Merge the attribute represented by this MapperProperty from source to destination object
post_instrument_class(mapper)
inherited from the post_instrument_class() method of MapperProperty
Perform instrumentation adjustments that need to occur after init() has completed.
setup(context, entity, path, adapter, **kwargs)
inherited from the setup() method of MapperProperty
Called by Query for the purposes of constructing a SQL statement.
Each MapperProperty associated with the target mapper processes the statement referenced by the query
context, adding columns and/or criterion as appropriate.
class sqlalchemy.orm.query.QueryContext(query)
class sqlalchemy.orm.attributes.QueryableAttribute(class_, key, impl=None, comparator=None,
parententity=None,
of_type=None)
Bases: sqlalchemy.orm.interfaces._MappedAttribute, sqlalchemy.orm.interfaces._InspectionAt
sqlalchemy.orm.interfaces.PropComparator
Base class for descriptor objects that intercept attribute events on behalf of a MapperProperty object. The
actual MapperProperty is accessible via the QueryableAttribute.property attribute.
See also:
InstrumentedAttribute
MapperProperty
Mapper.all_orm_descriptors
Mapper.attrs
Inherited-members
info
Return the ‘info’ dictionary for the underlying SQL element.
The behavior here is as follows:
•If the attribute is a column-mapped property, i.e. ColumnProperty, which is mapped directly
to a schema-level Column object, this attribute will return the SchemaItem.info dictionary
associated with the core-level Column object.
•If the attribute is a ColumnProperty but is mapped to any other kind of SQL expression other
than a Column, the attribute will refer to the MapperProperty.info dictionary associated
directly with the ColumnProperty, assuming the SQL expression itself does not have its own
.info attribute (which should be the case, unless a user-defined SQL construct has defined one).
324
Chapter 2. SQLAlchemy ORM
SQLAlchemy Documentation, Release 0.8.7
•If the attribute refers to any other kind of MapperProperty,
including
RelationshipProperty, the attribute will refer to the MapperProperty.info dictionary associated with that MapperProperty.
•To access the MapperProperty.info dictionary of the MapperProperty unconditionally, including for a ColumnProperty that’s associated directly with a schema.Column,
the attribute can be referred to using QueryableAttribute.property attribute, as
MyClass.someattribute.property.info.
New in version 0.8.0.
See also:
SchemaItem.info
MapperProperty.info
parent
Return an inspection instance representing the parent.
This will be either an instance of Mapper or AliasedInsp, depending upon the nature of the parent
entity which this attribute is associated with.
property
Return the MapperProperty associated with this QueryableAttribute.
Return values here will commonly be instances of ColumnProperty or RelationshipProperty.
2.13. ORM Internals
325
SQLAlchemy Documentation, Release 0.8.7
326
Chapter 2. SQLAlchemy ORM
CHAPTER 3
SQLAlchemy Core
The breadth of SQLAlchemy’s SQL rendering engine, DBAPI integration, transaction integration, and schema description services are documented here. In contrast to the ORM’s domain-centric mode of usage, the SQL Expression
Language provides a schema-centric usage paradigm.
3.1 SQL Expression Language Tutorial
The SQLAlchemy Expression Language presents a system of representing relational database structures and expressions using Python constructs. These constructs are modeled to resemble those of the underlying database as closely
as possible, while providing a modicum of abstraction of the various implementation differences between database
backends. While the constructs attempt to represent equivalent concepts between backends with consistent structures,
they do not conceal useful concepts that are unique to particular subsets of backends. The Expression Language therefore presents a method of writing backend-neutral SQL expressions, but does not attempt to enforce that expressions
are backend-neutral.
The Expression Language is in contrast to the Object Relational Mapper, which is a distinct API that builds on top
of the Expression Language. Whereas the ORM, introduced in Object Relational Tutorial, presents a high level and
abstracted pattern of usage, which itself is an example of applied usage of the Expression Language, the Expression
Language presents a system of representing the primitive constructs of the relational database directly without opinion.
While there is overlap among the usage patterns of the ORM and the Expression Language, the similarities are more
superficial than they may at first appear. One approaches the structure and content of data from the perspective of
a user-defined domain model which is transparently persisted and refreshed from its underlying storage model. The
other approaches it from the perspective of literal schema and SQL expression representations which are explicitly
composed into messages consumed individually by the database.
A successful application may be constructed using the Expression Language exclusively, though the application will
need to define its own system of translating application concepts into individual database messages and from individual database result sets. Alternatively, an application constructed with the ORM may, in advanced scenarios, make
occasional usage of the Expression Language directly in certain areas where specific database interactions are required.
The following tutorial is in doctest format, meaning each >>> line represents something you can type at a Python
command prompt, and the following text represents the expected return value. The tutorial has no prerequisites.
3.1.1 Version Check
A quick check to verify that we are on at least version 0.8 of SQLAlchemy:
327
SQLAlchemy Documentation, Release 0.8.7
>>> import sqlalchemy
>>> sqlalchemy.__version__ # doctest:+SKIP
0.8.0
3.1.2 Connecting
For this tutorial we will use an in-memory-only SQLite database. This is an easy way to test things without needing
to have an actual database defined anywhere. To connect we use create_engine():
>>> from sqlalchemy import create_engine
>>> engine = create_engine('sqlite:///:memory:', echo=True)
The echo flag is a shortcut to setting up SQLAlchemy logging, which is accomplished via Python’s standard
logging module. With it enabled, we’ll see all the generated SQL produced. If you are working through this
tutorial and want less output generated, set it to False. This tutorial will format the SQL behind a popup window so
it doesn’t get in our way; just click the “SQL” links to see what’s being generated.
The return value of create_engine() is an instance of Engine, and it represents the core interface to the
database, adapted through a dialect that handles the details of the database and DBAPI in use. In this case the SQLite
dialect will interpret instructions to the Python built-in sqlite3 module.
Lazy Connecting
The Engine, when first returned by create_engine(), has not actually tried to connect to the database
yet; that happens only the first time it is asked to perform a task against the database.
The first time a method like Engine.execute() or Engine.connect() is called, the Engine establishes a
real DBAPI connection to the database, which is then used to emit the SQL.
See also:
Database Urls - includes examples of create_engine() connecting to several kinds of databases with links to
more information.
3.1.3 Define and Create Tables
The SQL Expression Language constructs its expressions in most cases against table columns. In SQLAlchemy, a
column is most often represented by an object called Column, and in all cases a Column is associated with a Table.
A collection of Table objects and their associated child objects is referred to as database metadata. In this tutorial
we will explicitly lay out several Table objects, but note that SA can also “import” whole sets of Table objects
automatically from an existing database (this process is called table reflection).
We define our tables all within a catalog called MetaData, using the Table construct, which resembles regular SQL
CREATE TABLE statements. We’ll make two tables, one of which represents “users” in an application, and another
which represents zero or more “email addresses” for each row in the “users” table:
>>>
>>>
>>>
...
...
...
...
328
from sqlalchemy import Table, Column, Integer, String, MetaData, ForeignKey
metadata = MetaData()
users = Table('users', metadata,
Column('id', Integer, primary_key=True),
Column('name', String),
Column('fullname', String),
)
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
>>> addresses = Table('addresses', metadata,
...
Column('id', Integer, primary_key=True),
...
Column('user_id', None, ForeignKey('users.id')),
...
Column('email_address', String, nullable=False)
... )
All about how to define Table objects, as well as how to create them from an existing database automatically, is
described in Describing Databases with MetaData.
Next, to tell the MetaData we’d actually like to create our selection of tables for real inside the SQLite database, we
use create_all(), passing it the engine instance which points to our database. This will check for the presence
of each table first before creating, so it’s safe to call multiple times:
>>> metadata.create_all(engine) #doctest: +NORMALIZE_WHITESPACE
PRAGMA table_info("users")
()
PRAGMA table_info("addresses")
()
CREATE TABLE users (
id INTEGER NOT NULL,
name VARCHAR,
fullname VARCHAR,
PRIMARY KEY (id)
)
()
COMMIT
CREATE TABLE addresses (
id INTEGER NOT NULL,
user_id INTEGER,
email_address VARCHAR NOT NULL,
PRIMARY KEY (id),
FOREIGN KEY(user_id) REFERENCES users (id)
)
()
COMMIT
Note: Users familiar with the syntax of CREATE TABLE may notice that the VARCHAR columns were generated
without a length; on SQLite and Postgresql, this is a valid datatype, but on others, it’s not allowed. So if running this
tutorial on one of those databases, and you wish to use SQLAlchemy to issue CREATE TABLE, a “length” may be
provided to the String type as below:
Column('name', String(50))
The length field on String, as well as similar precision/scale fields available on Integer, Numeric, etc. are not
referenced by SQLAlchemy other than when creating tables.
Additionally, Firebird and Oracle require sequences to generate new primary key identifiers, and SQLAlchemy doesn’t
generate or assume these without being instructed. For that, you use the Sequence construct:
from sqlalchemy import Sequence
Column('id', Integer, Sequence('user_id_seq'), primary_key=True)
A full, foolproof Table is therefore:
3.1. SQL Expression Language Tutorial
329
SQLAlchemy Documentation, Release 0.8.7
users = Table('users', metadata,
Column('id', Integer, Sequence('user_id_seq'), primary_key=True),
Column('name', String(50)),
Column('fullname', String(50)),
Column('password', String(12))
)
We include this more verbose Table construct separately to highlight the difference between a minimal construct
geared primarily towards in-Python usage only, versus one that will be used to emit CREATE TABLE statements on a
particular set of backends with more stringent requirements.
3.1.4 Insert Expressions
The first SQL expression we’ll create is the Insert construct, which represents an INSERT statement. This is
typically created relative to its target table:
>>> ins = users.insert()
To see a sample of the SQL this construct produces, use the str() function:
>>> str(ins)
'INSERT INTO users (id, name, fullname) VALUES (:id, :name, :fullname)'
Notice above that the INSERT statement names every column in the users table. This can be limited by using the
values() method, which establishes the VALUES clause of the INSERT explicitly:
>>> ins = users.insert().values(name='jack', fullname='Jack Jones')
>>> str(ins)
'INSERT INTO users (name, fullname) VALUES (:name, :fullname)'
Above, while the values method limited the VALUES clause to just two columns, the actual data we placed in
values didn’t get rendered into the string; instead we got named bind parameters. As it turns out, our data is stored
within our Insert construct, but it typically only comes out when the statement is actually executed; since the data
consists of literal values, SQLAlchemy automatically generates bind parameters for them. We can peek at this data
for now by looking at the compiled form of the statement:
>>> ins.compile().params
{'fullname': 'Jack Jones', 'name': 'jack'}
3.1.5 Executing
The interesting part of an Insert is executing it. In this tutorial, we will generally focus on the most explicit method
of executing a SQL construct, and later touch upon some “shortcut” ways to do it. The engine object we created
is a repository for database connections capable of issuing SQL to the database. To acquire a connection, we use the
connect() method:
>>> conn = engine.connect()
>>> conn
<sqlalchemy.engine.base.Connection object at 0x...>
330
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
The Connection object represents an actively checked out DBAPI connection resource. Lets feed it our Insert
object and see what happens:
>>> result = conn.execute(ins)
INSERT INTO users (name, fullname) VALUES (?, ?)
('jack', 'Jack Jones')
COMMIT
So the INSERT statement was now issued to the database. Although we got positional “qmark” bind parameters
instead of “named” bind parameters in the output. How come ? Because when executed, the Connection used the
SQLite dialect to help generate the statement; when we use the str() function, the statement isn’t aware of this
dialect, and falls back onto a default which uses named parameters. We can view this manually as follows:
>>> ins.bind = engine
>>> str(ins)
'INSERT INTO users (name, fullname) VALUES (?, ?)'
What about the result variable we got when we called execute() ? As the SQLAlchemy Connection object
references a DBAPI connection, the result, known as a ResultProxy object, is analogous to the DBAPI cursor
object. In the case of an INSERT, we can get important information from it, such as the primary key values which
were generated from our statement:
>>> result.inserted_primary_key
[1]
The value of 1 was automatically generated by SQLite, but only because we did not specify the id column in our
Insert statement; otherwise, our explicit value would have been used. In either case, SQLAlchemy always knows
how to get at a newly generated primary key value, even though the method of generating them is different across
different databases; each database’s Dialect knows the specific steps needed to determine the correct value (or
values; note that inserted_primary_key returns a list so that it supports composite primary keys).
3.1.6 Executing Multiple Statements
Our insert example above was intentionally a little drawn out to show some various behaviors of expression language
constructs. In the usual case, an Insert statement is usually compiled against the parameters sent to the execute()
method on Connection, so that there’s no need to use the values keyword with Insert. Lets create a generic
Insert statement again and use it in the “normal” way:
>>> ins = users.insert()
>>> conn.execute(ins, id=2, name='wendy', fullname='Wendy Williams') # doctest: +ELLIPSIS
INSERT INTO users (id, name, fullname) VALUES (?, ?, ?)
(2, 'wendy', 'Wendy Williams')
COMMIT
<sqlalchemy.engine.result.ResultProxy object at 0x...>
Above, because we specified all three columns in the execute() method, the compiled Insert included all three
columns. The Insert statement is compiled at execution time based on the parameters we specified; if we specified
fewer parameters, the Insert would have fewer entries in its VALUES clause.
To issue many inserts using DBAPI’s executemany() method, we can send in a list of dictionaries each containing
a distinct set of parameters to be inserted, as we do here to add some email addresses:
3.1. SQL Expression Language Tutorial
331
SQLAlchemy Documentation, Release 0.8.7
>>> conn.execute(addresses.insert(), [ # doctest: +ELLIPSIS
...
{'user_id': 1, 'email_address' : '[email protected]'},
...
{'user_id': 1, 'email_address' : '[email protected]'},
...
{'user_id': 2, 'email_address' : '[email protected]'},
...
{'user_id': 2, 'email_address' : '[email protected]'},
... ])
INSERT INTO addresses (user_id, email_address) VALUES (?, ?)
((1, '[email protected]'), (1, '[email protected]'), (2, '[email protected]'), (2, '[email protected]'))
COMMIT
<sqlalchemy.engine.result.ResultProxy object at 0x...>
Above, we again relied upon SQLite’s automatic generation of primary key identifiers for each addresses row.
When executing multiple sets of parameters, each dictionary must have the same set of keys; i.e. you cant have fewer
keys in some dictionaries than others. This is because the Insert statement is compiled against the first dictionary
in the list, and it’s assumed that all subsequent argument dictionaries are compatible with that statement.
3.1.7 Selecting
We began with inserts just so that our test database had some data in it. The more interesting part of the data is selecting
it ! We’ll cover UPDATE and DELETE statements later. The primary construct used to generate SELECT statements
is the select() function:
>>> from sqlalchemy.sql import select
>>> s = select([users])
>>> result = conn.execute(s) # doctest: +NORMALIZE_WHITESPACE
SELECT users.id, users.name, users.fullname
FROM users
()
Above, we issued a basic select() call, placing the users table within the COLUMNS clause of the select, and
then executing. SQLAlchemy expanded the users table into the set of each of its columns, and also generated a
FROM clause for us. The result returned is again a ResultProxy object, which acts much like a DBAPI cursor,
including methods such as fetchone() and fetchall(). The easiest way to get rows from it is to just iterate:
>>> for row in result:
...
print row
(1, u'jack', u'Jack Jones')
(2, u'wendy', u'Wendy Williams')
Above, we see that printing each row produces a simple tuple-like result. We have more options at accessing the data
in each row. One very common way is through dictionary access, using the string names of columns:
>>> result = conn.execute(s) # doctest: +NORMALIZE_WHITESPACE
SELECT users.id, users.name, users.fullname
FROM users
()
>>> row = result.fetchone()
>>> print "name:", row['name'], "; fullname:", row['fullname']
name: jack ; fullname: Jack Jones
Integer indexes work as well:
332
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
>>> row = result.fetchone()
>>> print "name:", row[1], "; fullname:", row[2]
name: wendy ; fullname: Wendy Williams
But another way, whose usefulness will become apparent later on, is to use the Column objects directly as keys:
>>> for row in conn.execute(s): # doctest: +NORMALIZE_WHITESPACE
...
print "name:", row[users.c.name], "; fullname:", row[users.c.fullname]
SELECT users.id, users.name, users.fullname
FROM users
()
name: jack ; fullname: Jack Jones
name: wendy ; fullname: Wendy Williams
Result sets which have pending rows remaining should be explicitly closed before discarding. While the cursor and
connection resources referenced by the ResultProxy will be respectively closed and returned to the connection
pool when the object is garbage collected, it’s better to make it explicit as some database APIs are very picky about
such things:
>>> result.close()
If we’d like to more carefully control the columns which are placed in the COLUMNS clause of the select, we reference
individual Column objects from our Table. These are available as named attributes off the c attribute of the Table
object:
>>> s = select([users.c.name, users.c.fullname])
>>> result = conn.execute(s) # doctest: +NORMALIZE_WHITESPACE
SELECT users.name, users.fullname
FROM users
()
>>> for row in result: #doctest: +NORMALIZE_WHITESPACE
...
print row
(u'jack', u'Jack Jones')
(u'wendy', u'Wendy Williams')
Lets observe something interesting about the FROM clause. Whereas the generated statement contains two distinct
sections, a “SELECT columns” part and a “FROM table” part, our select() construct only has a list containing
columns. How does this work ? Let’s try putting two tables into our select() statement:
>>> for row in conn.execute(select([users, addresses])):
...
print row # doctest: +NORMALIZE_WHITESPACE
SELECT users.id, users.name, users.fullname, addresses.id, addresses.user_id, addresses.email_address
FROM users, addresses
()
(1, u'jack', u'Jack Jones', 1, 1, u'[email protected]')
(1, u'jack', u'Jack Jones', 2, 1, u'[email protected]')
(1, u'jack', u'Jack Jones', 3, 2, u'[email protected]')
(1, u'jack', u'Jack Jones', 4, 2, u'[email protected]')
(2, u'wendy', u'Wendy Williams', 1, 1, u'[email protected]')
(2, u'wendy', u'Wendy Williams', 2, 1, u'[email protected]')
(2, u'wendy', u'Wendy Williams', 3, 2, u'[email protected]')
(2, u'wendy', u'Wendy Williams', 4, 2, u'[email protected]')
3.1. SQL Expression Language Tutorial
333
SQLAlchemy Documentation, Release 0.8.7
It placed both tables into the FROM clause. But also, it made a real mess. Those who are familiar with SQL joins know
that this is a Cartesian product; each row from the users table is produced against each row from the addresses
table. So to put some sanity into this statement, we need a WHERE clause. We do that using Select.where():
>>> s = select([users, addresses]).where(users.c.id == addresses.c.user_id)
>>> for row in conn.execute(s):
...
print row # doctest: +NORMALIZE_WHITESPACE
SELECT users.id, users.name, users.fullname, addresses.id,
addresses.user_id, addresses.email_address
FROM users, addresses
WHERE users.id = addresses.user_id
()
(1, u'jack', u'Jack Jones', 1, 1, u'[email protected]')
(1, u'jack', u'Jack Jones', 2, 1, u'[email protected]')
(2, u'wendy', u'Wendy Williams', 3, 2, u'[email protected]')
(2, u'wendy', u'Wendy Williams', 4, 2, u'[email protected]')
So that looks a lot better, we added an expression to our select() which had the effect of adding WHERE
users.id = addresses.user_id to our statement, and our results were managed down so that the join of
users and addresses rows made sense. But let’s look at that expression? It’s using just a Python equality operator between two different Column objects. It should be clear that something is up. Saying 1 == 1 produces True,
and 1 == 2 produces False, not a WHERE clause. So lets see exactly what that expression is doing:
>>> users.c.id == addresses.c.user_id #doctest: +ELLIPSIS
<sqlalchemy.sql.expression.BinaryExpression object at 0x...>
Wow, surprise ! This is neither a True nor a False. Well what is it ?
>>> str(users.c.id == addresses.c.user_id)
'users.id = addresses.user_id'
As you can see, the == operator is producing an object that is very much like the Insert and select() objects
we’ve made so far, thanks to Python’s __eq__() builtin; you call str() on it and it produces SQL. By now, one
can see that everything we are working with is ultimately the same type of object. SQLAlchemy terms the base class
of all of these expressions as ColumnElement.
3.1.8 Operators
Since we’ve stumbled upon SQLAlchemy’s operator paradigm, let’s go through some of its capabilities. We’ve seen
how to equate two columns to each other:
>>> print users.c.id == addresses.c.user_id
users.id = addresses.user_id
If we use a literal value (a literal meaning, not a SQLAlchemy clause object), we get a bind parameter:
>>> print users.c.id == 7
users.id = :id_1
The 7 literal is embedded the resulting ColumnElement; we can use the same trick we did with the Insert object
to see it:
334
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
>>> (users.c.id == 7).compile().params
{u'id_1': 7}
Most Python operators, as it turns out, produce a SQL expression here, like equals, not equals, etc.:
>>> print users.c.id != 7
users.id != :id_1
>>> # None converts to IS NULL
>>> print users.c.name == None
users.name IS NULL
>>> # reverse works too
>>> print 'fred' > users.c.name
users.name < :name_1
If we add two integer columns together, we get an addition expression:
>>> print users.c.id + addresses.c.id
users.id + addresses.id
Interestingly, the type of the Column is important! If we use + with two string based columns (recall we put types
like Integer and String on our Column objects at the beginning), we get something different:
>>> print users.c.name + users.c.fullname
users.name || users.fullname
Where || is the string concatenation operator used on most databases. But not all of them. MySQL users, fear not:
>>> print (users.c.name + users.c.fullname).\
...
compile(bind=create_engine('mysql://'))
concat(users.name, users.fullname)
The above illustrates the SQL that’s generated for an Engine that’s connected to a MySQL database; the || operator
now compiles as MySQL’s concat() function.
If you have come across an operator which really isn’t available, you can always use the ColumnOperators.op()
method; this generates whatever operator you need:
>>> print users.c.name.op('tiddlywinks')('foo')
users.name tiddlywinks :name_1
This function can also be used to make bitwise operators explicit. For example:
somecolumn.op('&')(0xff)
is a bitwise AND of the value in somecolumn.
3.1. SQL Expression Language Tutorial
335
SQLAlchemy Documentation, Release 0.8.7
Operator Customization
While ColumnOperators.op() is handy to get at a custom operator in a hurry, the Core supports fundamental
customization and extension of the operator system at the type level. The behavior of existing operators can be
modified on a per-type basis, and new operations can be defined which become available for all column expressions
that are part of that particular type. See the section Redefining and Creating New Operators for a description.
3.1.9 Conjunctions
We’d like to show off some of our operators inside of select() constructs. But we need to lump them together a
little more, so let’s first introduce some conjunctions. Conjunctions are those little words like AND and OR that put
things together. We’ll also hit upon NOT. and_(), or_(), and not_() can work from the corresponding functions
SQLAlchemy provides (notice we also throw in a like()):
>>> from sqlalchemy.sql import and_, or_, not_
>>> print and_(
...
users.c.name.like('j%'),
...
users.c.id == addresses.c.user_id, #doctest: +NORMALIZE_WHITESPACE
...
or_(
...
addresses.c.email_address == '[email protected]',
...
addresses.c.email_address == '[email protected]'
...
),
...
not_(users.c.id > 5)
...
)
users.name LIKE :name_1 AND users.id = addresses.user_id AND
(addresses.email_address = :email_address_1
OR addresses.email_address = :email_address_2)
AND users.id <= :id_1
And you can also use the re-jiggered bitwise AND, OR and NOT operators, although because of Python operator
precedence you have to watch your parenthesis:
>>> print users.c.name.like('j%') & (users.c.id == addresses.c.user_id) &
...
(
...
(addresses.c.email_address == '[email protected]') | \
...
(addresses.c.email_address == '[email protected]')
...
) \
...
& ~(users.c.id>5) # doctest: +NORMALIZE_WHITESPACE
users.name LIKE :name_1 AND users.id = addresses.user_id AND
(addresses.email_address = :email_address_1
OR addresses.email_address = :email_address_2)
AND users.id <= :id_1
\
So with all of this vocabulary, let’s select all users who have an email address at AOL or MSN, whose name starts with
a letter between “m” and “z”, and we’ll also generate a column containing their full name combined with their email
address. We will add two new constructs to this statement, between() and label(). between() produces
a BETWEEN clause, and label() is used in a column expression to produce labels using the AS keyword; it’s
recommended when selecting from expressions that otherwise would not have a name:
>>> s = select([(users.c.fullname +
...
", " + addresses.c.email_address).
...
label('title')]).\
...
where(
...
and_(
336
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
...
users.c.id == addresses.c.user_id,
...
users.c.name.between('m', 'z'),
...
or_(
...
addresses.c.email_address.like('%@aol.com'),
...
addresses.c.email_address.like('%@msn.com')
...
)
...
)
...
)
>>> conn.execute(s).fetchall() #doctest: +NORMALIZE_WHITESPACE
SELECT users.fullname || ? || addresses.email_address AS title
FROM users, addresses
WHERE users.id = addresses.user_id AND users.name BETWEEN ? AND ? AND
(addresses.email_address LIKE ? OR addresses.email_address LIKE ?)
(', ', 'm', 'z', '%@aol.com', '%@msn.com')
[(u'Wendy Williams, [email protected]',)]
Once again, SQLAlchemy figured out the FROM clause for our statement. In fact it will determine the FROM clause
based on all of its other bits; the columns clause, the where clause, and also some other elements which we haven’t
covered yet, which include ORDER BY, GROUP BY, and HAVING.
A shortcut to using and_() is to chain together multiple where() clauses. The above can also be written as:
>>> s = select([(users.c.fullname +
...
", " + addresses.c.email_address).
...
label('title')]).\
...
where(users.c.id == addresses.c.user_id).\
...
where(users.c.name.between('m', 'z')).\
...
where(
...
or_(
...
addresses.c.email_address.like('%@aol.com'),
...
addresses.c.email_address.like('%@msn.com')
...
)
...
)
>>> conn.execute(s).fetchall() #doctest: +NORMALIZE_WHITESPACE
SELECT users.fullname || ? || addresses.email_address AS title
FROM users, addresses
WHERE users.id = addresses.user_id AND users.name BETWEEN ? AND ? AND
(addresses.email_address LIKE ? OR addresses.email_address LIKE ?)
(', ', 'm', 'z', '%@aol.com', '%@msn.com')
[(u'Wendy Williams, [email protected]',)]
The way that we can build up a select() construct through successive method calls is called method chaining.
3.1.10 Using Text
Our last example really became a handful to type. Going from what one understands to be a textual SQL expression into a Python construct which groups components together in a programmatic style can be hard. That’s why
SQLAlchemy lets you just use strings too. The text() construct represents any textual statement, in a backendagnostic way. To use bind parameters with text(), always use the named colon format. Such as below, we create a
text() and execute it, feeding in the bind parameters to the execute() method:
>>> from sqlalchemy.sql import text
>>> s = text(
...
"SELECT users.fullname || ', ' || addresses.email_address AS title "
...
"FROM users, addresses "
3.1. SQL Expression Language Tutorial
337
SQLAlchemy Documentation, Release 0.8.7
...
"WHERE users.id = addresses.user_id "
...
"AND users.name BETWEEN :x AND :y "
...
"AND (addresses.email_address LIKE :e1 "
...
"OR addresses.email_address LIKE :e2)")
>>> conn.execute(s, x='m', y='z', e1='%@aol.com', e2='%@msn.com').fetchall() # doctest:+NORMALIZE_WHI
SELECT users.fullname || ', ' || addresses.email_address AS title
FROM users, addresses
WHERE users.id = addresses.user_id AND users.name BETWEEN ? AND ? AND
(addresses.email_address LIKE ? OR addresses.email_address LIKE ?)
('m', 'z', '%@aol.com', '%@msn.com')
[(u'Wendy Williams, [email protected]',)]
To gain a “hybrid” approach, the select() construct accepts strings for most of its arguments. Below we combine
the usage of strings with our constructed select() object, by using the select() object to structure the statement,
and strings to provide all the content within the structure. For this example, SQLAlchemy is not given any Column or
Table objects in any of its expressions, so it cannot generate a FROM clause. So we also use the select_from()
method, which accepts a FromClause or string expression to be placed within the FROM clause:
>>> s = select([
...
"users.fullname || ', ' || addresses.email_address AS title"
...
]).\
...
where(
...
and_(
...
"users.id = addresses.user_id",
...
"users.name BETWEEN 'm' AND 'z'",
...
"(addresses.email_address LIKE :x OR addresses.email_address LIKE :y)"
...
)
...
).select_from('users, addresses')
>>> conn.execute(s, x='%@aol.com', y='%@msn.com').fetchall() #doctest: +NORMALIZE_WHITESPACE
SELECT users.fullname || ', ' || addresses.email_address AS title
FROM users, addresses
WHERE users.id = addresses.user_id AND users.name BETWEEN 'm' AND 'z'
AND (addresses.email_address LIKE ? OR addresses.email_address LIKE ?)
('%@aol.com', '%@msn.com')
[(u'Wendy Williams, [email protected]',)]
Going from constructed SQL to text, we lose some capabilities. We lose the capability for SQLAlchemy to compile our
expression to a specific target database; above, our expression won’t work with MySQL since it has no || construct.
It also becomes more tedious for SQLAlchemy to be made aware of the datatypes in use; for example, if our bind
parameters required UTF-8 encoding before going in, or conversion from a Python datetime into a string (as is
required with SQLite), we would have to add extra information to our text() construct. Similar issues arise on the
result set side, where SQLAlchemy also performs type-specific data conversion in some cases; still more information
can be added to text() to work around this. But what we really lose from our statement is the ability to manipulate
it, transform it, and analyze it. These features are critical when using the ORM, which makes heavy usage of relational
transformations. To show off what we mean, we’ll first introduce the ALIAS construct and the JOIN construct, just so
we have some juicier bits to play with.
3.1.11 Using Aliases
The alias in SQL corresponds to a “renamed” version of a table or SELECT statement, which occurs anytime you
say “SELECT .. FROM sometable AS someothername”. The AS creates a new name for the table. Aliases are a key
construct as they allow any table or subquery to be referenced by a unique name. In the case of a table, this allows the
same table to be named in the FROM clause multiple times. In the case of a SELECT statement, it provides a parent
name for the columns represented by the statement, allowing them to be referenced relative to this name.
338
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
In SQLAlchemy, any Table, select() construct, or other selectable can be turned into an alias using the
FromClause.alias() method, which produces a Alias construct. As an example, suppose we know that our
user jack has two particular email addresses. How can we locate jack based on the combination of those two addresses? To accomplish this, we’d use a join to the addresses table, once for each address. We create two Alias
constructs against addresses, and then use them both within a select() construct:
>>> a1 = addresses.alias()
>>> a2 = addresses.alias()
>>> s = select([users]).\
...
where(and_(
...
users.c.id == a1.c.user_id,
...
users.c.id == a2.c.user_id,
...
a1.c.email_address == '[email protected]',
...
a2.c.email_address == '[email protected]'
...
))
>>> conn.execute(s).fetchall() # doctest: +NORMALIZE_WHITESPACE
SELECT users.id, users.name, users.fullname
FROM users, addresses AS addresses_1, addresses AS addresses_2
WHERE users.id = addresses_1.user_id
AND users.id = addresses_2.user_id
AND addresses_1.email_address = ?
AND addresses_2.email_address = ?
('[email protected]', '[email protected]')
[(1, u'jack', u'Jack Jones')]
Note that the Alias construct generated the names addresses_1 and addresses_2 in the final SQL result. The
generation of these names is determined by the position of the construct within the statement. If we created a query
using only the second a2 alias, the name would come out as addresses_1. The generation of the names is also
deterministic, meaning the same SQLAlchemy statement construct will produce the identical SQL string each time it
is rendered for a particular dialect.
Since on the outside, we refer to the alias using the Alias construct itself, we don’t need to be concerned about
the generated name. However, for the purposes of debugging, it can be specified by passing a string name to the
FromClause.alias() method:
>>> a1 = addresses.alias('a1')
Aliases can of course be used for anything which you can SELECT from, including SELECT statements themselves.
We can self-join the users table back to the select() we’ve created by making an alias of the entire statement.
The correlate(None) directive is to avoid SQLAlchemy’s attempt to “correlate” the inner users table with the
outer one:
>>> a1 = s.correlate(None).alias()
>>> s = select([users.c.name]).where(users.c.id == a1.c.id)
>>> conn.execute(s).fetchall() # doctest: +NORMALIZE_WHITESPACE
SELECT users.name
FROM users,
(SELECT users.id AS id, users.name AS name, users.fullname AS fullname
FROM users, addresses AS addresses_1, addresses AS addresses_2
WHERE users.id = addresses_1.user_id AND users.id = addresses_2.user_id
AND addresses_1.email_address = ?
AND addresses_2.email_address = ?) AS anon_1
WHERE users.id = anon_1.id
('[email protected]', '[email protected]')
[(u'jack',)]
3.1. SQL Expression Language Tutorial
339
SQLAlchemy Documentation, Release 0.8.7
3.1.12 Using Joins
We’re halfway along to being able to construct any SELECT expression. The next cornerstone of the SELECT is the
JOIN expression. We’ve already been doing joins in our examples, by just placing two tables in either the columns
clause or the where clause of the select() construct. But if we want to make a real “JOIN” or “OUTERJOIN”
construct, we use the join() and outerjoin() methods, most commonly accessed from the left table in the join:
>>> print users.join(addresses)
users JOIN addresses ON users.id = addresses.user_id
The alert reader will see more surprises; SQLAlchemy figured out how to JOIN the two tables ! The ON condition
of the join, as it’s called, was automatically generated based on the ForeignKey object which we placed on the
addresses table way at the beginning of this tutorial. Already the join() construct is looking like a much better
way to join tables.
Of course you can join on whatever expression you want, such as if we want to join on all users who use the same
name in their email address as their username:
>>> print users.join(addresses,
...
addresses.c.email_address.like(users.c.name + '%')
...
)
users JOIN addresses ON addresses.email_address LIKE (users.name || :name_1)
When we create a select() construct, SQLAlchemy looks around at the tables we’ve mentioned and then places
them in the FROM clause of the statement. When we use JOINs however, we know what FROM clause we want, so
here we make use of the select_from() method:
>>> s = select([users.c.fullname]).select_from(
...
users.join(addresses,
...
addresses.c.email_address.like(users.c.name + '%'))
...
)
>>> conn.execute(s).fetchall() # doctest: +NORMALIZE_WHITESPACE
SELECT users.fullname
FROM users JOIN addresses ON addresses.email_address LIKE (users.name || ?)
('%',)
[(u'Jack Jones',), (u'Jack Jones',), (u'Wendy Williams',)]
The outerjoin() method creates LEFT OUTER JOIN constructs, and is used in the same way as join():
>>> s = select([users.c.fullname]).select_from(users.outerjoin(addresses))
>>> print s # doctest: +NORMALIZE_WHITESPACE
SELECT users.fullname
FROM users
LEFT OUTER JOIN addresses ON users.id = addresses.user_id
That’s the output outerjoin() produces, unless, of course, you’re stuck in a gig using Oracle prior to version 9,
and you’ve set up your engine (which would be using OracleDialect) to use Oracle-specific SQL:
>>> from sqlalchemy.dialects.oracle import dialect as OracleDialect
>>> print s.compile(dialect=OracleDialect(use_ansi=False)) # doctest: +NORMALIZE_WHITESPACE
SELECT users.fullname
FROM users, addresses
WHERE users.id = addresses.user_id(+)
340
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
If you don’t know what that SQL means, don’t worry ! The secret tribe of Oracle DBAs don’t want their black magic
being found out ;).
See also:
expression.join()
expression.outerjoin()
Join
3.1.13 Everything Else
The concepts of creating SQL expressions have been introduced. What’s left are more variants of the same themes.
So now we’ll catalog the rest of the important things we’ll need to know.
Bind Parameter Objects
Throughout all these examples, SQLAlchemy is busy creating bind parameters wherever literal expressions occur.
You can also specify your own bind parameters with your own names, and use the same statement repeatedly. The
bindparam() construct is used to produce a bound parameter with a given name. While SQLAlchemy always refers
to bound parameters by name on the API side, the database dialect converts to the appropriate named or positional
style at execution time, as here where it converts to positional for SQLite:
>>> from sqlalchemy.sql import bindparam
>>> s = users.select(users.c.name == bindparam('username'))
>>> conn.execute(s, username='wendy').fetchall() # doctest: +NORMALIZE_WHITESPACE
SELECT users.id, users.name, users.fullname
FROM users
WHERE users.name = ?
('wendy',)
[(2, u'wendy', u'Wendy Williams')]
Another important aspect of bindparam() is that it may be assigned a type. The type of the bind parameter will
determine its behavior within expressions and also how the data bound to it is processed before being sent off to the
database:
>>> s = users.select(users.c.name.like(bindparam('username', type_=String) + text("'%'")))
>>> conn.execute(s, username='wendy').fetchall() # doctest: +NORMALIZE_WHITESPACE
SELECT users.id, users.name, users.fullname
FROM users
WHERE users.name LIKE (? || '%')
('wendy',)
[(2, u'wendy', u'Wendy Williams')]
bindparam() constructs of the same name can also be used multiple times, where only a single named value is
needed in the execute parameters:
>>> s = select([users, addresses]).\
...
where(
...
or_(
...
users.c.name.like(
...
bindparam('name', type_=String) + text("'%'")),
...
addresses.c.email_address.like(
...
bindparam('name', type_=String) + text("'@%'"))
3.1. SQL Expression Language Tutorial
341
SQLAlchemy Documentation, Release 0.8.7
...
)
...
).\
...
select_from(users.outerjoin(addresses)).\
...
order_by(addresses.c.id)
>>> conn.execute(s, name='jack').fetchall() # doctest: +NORMALIZE_WHITESPACE
SELECT users.id, users.name, users.fullname, addresses.id,
addresses.user_id, addresses.email_address
FROM users LEFT OUTER JOIN addresses ON users.id = addresses.user_id
WHERE users.name LIKE (? || '%') OR addresses.email_address LIKE (? || '@%')
ORDER BY addresses.id
('jack', 'jack')
[(1, u'jack', u'Jack Jones', 1, 1, u'[email protected]'), (1, u'jack', u'Jack Jones', 2, 1, u'[email protected]
See also:
bindparam()
Functions
SQL functions are created using the func keyword, which generates functions using attribute access:
>>> from sqlalchemy.sql import func
>>> print func.now()
now()
>>> print func.concat('x', 'y')
concat(:param_1, :param_2)
By “generates”, we mean that any SQL function is created based on the word you choose:
>>> print func.xyz_my_goofy_function()
xyz_my_goofy_function()
Certain function names are known by SQLAlchemy, allowing special behavioral rules to be applied. Some for example
are “ANSI” functions, which mean they don’t get the parenthesis added after them, such as CURRENT_TIMESTAMP:
>>> print func.current_timestamp()
CURRENT_TIMESTAMP
Functions are most typically used in the columns clause of a select statement, and can also be labeled as well as given
a type. Labeling a function is recommended so that the result can be targeted in a result row based on a string name,
and assigning it a type is required when you need result-set processing to occur, such as for Unicode conversion and
date conversions. Below, we use the result function scalar() to just read the first column of the first row and then
close the result; the label, even though present, is not important in this case:
>>> conn.execute(
...
select([
...
func.max(addresses.c.email_address, type_=String).
...
label('maxemail')
...
])
...
).scalar() # doctest: +NORMALIZE_WHITESPACE
SELECT max(addresses.email_address) AS maxemail
FROM addresses
()
u'[email protected]'
342
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
Databases such as PostgreSQL and Oracle which support functions that return whole result sets can be assembled
into selectable units, which can be used in statements. Such as, a database function calculate() which takes the
parameters x and y, and returns three columns which we’d like to name q, z and r, we can construct using “lexical”
column objects as well as bind parameters:
>>> from sqlalchemy.sql import column
>>> calculate = select([column('q'), column('z'), column('r')]).\
...
select_from(
...
func.calculate(
...
bindparam('x'),
...
bindparam('y')
...
)
...
)
>>> calc = calculate.alias()
>>> print select([users]).where(users.c.id > calc.c.z) # doctest: +NORMALIZE_WHITESPACE
SELECT users.id, users.name, users.fullname
FROM users, (SELECT q, z, r
FROM calculate(:x, :y)) AS anon_1
WHERE users.id > anon_1.z
If we wanted to use our calculate statement twice with different bind parameters, the unique_params()
function will create copies for us, and mark the bind parameters as “unique” so that conflicting names are isolated.
Note we also make two separate aliases of our selectable:
>>> calc1 = calculate.alias('c1').unique_params(x=17, y=45)
>>> calc2 = calculate.alias('c2').unique_params(x=5, y=12)
>>> s = select([users]).\
...
where(users.c.id.between(calc1.c.z, calc2.c.z))
>>> print s # doctest: +NORMALIZE_WHITESPACE
SELECT users.id, users.name, users.fullname
FROM users,
(SELECT q, z, r FROM calculate(:x_1, :y_1)) AS c1,
(SELECT q, z, r FROM calculate(:x_2, :y_2)) AS c2
WHERE users.id BETWEEN c1.z AND c2.z
>>> s.compile().params
{u'x_2': 5, u'y_2': 12, u'y_1': 45, u'x_1': 17}
See also:
func
Window Functions
Any FunctionElement, including functions generated by func, can be turned into a “window function”, that is
an OVER clause, using the FunctionElement.over() method:
>>> s = select([
...
users.c.id,
...
func.row_number().over(order_by=users.c.name)
...
])
>>> print s # doctest: +NORMALIZE_WHITESPACE
SELECT users.id, row_number() OVER (ORDER BY users.name) AS anon_1
FROM users
3.1. SQL Expression Language Tutorial
343
SQLAlchemy Documentation, Release 0.8.7
See also:
over()
FunctionElement.over()
Unions and Other Set Operations
Unions come in two flavors, UNION and UNION ALL, which are available via module level functions union() and
union_all():
>>> from sqlalchemy.sql import union
>>> u = union(
...
addresses.select().
...
where(addresses.c.email_address == '[email protected]'),
...
addresses.select().
...
where(addresses.c.email_address.like('%@yahoo.com')),
... ).order_by(addresses.c.email_address)
>>> conn.execute(u).fetchall() # doctest: +NORMALIZE_WHITESPACE
SELECT addresses.id, addresses.user_id, addresses.email_address
FROM addresses
WHERE addresses.email_address = ?
UNION
SELECT addresses.id, addresses.user_id, addresses.email_address
FROM addresses
WHERE addresses.email_address LIKE ? ORDER BY addresses.email_address
('[email protected]', '%@yahoo.com')
[(1, 1, u'[email protected]')]
Also available, though not supported on all databases, are intersect(), intersect_all(), except_(), and
except_all():
>>> from sqlalchemy.sql import except_
>>> u = except_(
...
addresses.select().
...
where(addresses.c.email_address.like('%@%.com')),
...
addresses.select().
...
where(addresses.c.email_address.like('%@msn.com'))
... )
>>> conn.execute(u).fetchall() # doctest: +NORMALIZE_WHITESPACE
SELECT addresses.id, addresses.user_id, addresses.email_address
FROM addresses
WHERE addresses.email_address LIKE ?
EXCEPT
SELECT addresses.id, addresses.user_id, addresses.email_address
FROM addresses
WHERE addresses.email_address LIKE ?
('%@%.com', '%@msn.com')
[(1, 1, u'[email protected]'), (4, 2, u'[email protected]')]
A common issue with so-called “compound” selectables arises due to the fact that they nest with parenthesis. SQLite
in particular doesn’t like a statement that starts with parenthesis. So when nesting a “compound” inside a “compound”,
it’s often necessary to apply .alias().select() to the first element of the outermost compound, if that element
is also a compound. For example, to nest a “union” and a “select” inside of “except_”, SQLite will want the “union”
to be stated as a subquery:
344
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
>>> u = except_(
...
union(
...
addresses.select().
...
where(addresses.c.email_address.like('%@yahoo.com')),
...
addresses.select().
...
where(addresses.c.email_address.like('%@msn.com'))
...
).alias().select(),
# apply subquery here
...
addresses.select(addresses.c.email_address.like('%@msn.com'))
... )
>>> conn.execute(u).fetchall()
# doctest: +NORMALIZE_WHITESPACE
SELECT anon_1.id, anon_1.user_id, anon_1.email_address
FROM (SELECT addresses.id AS id, addresses.user_id AS user_id,
addresses.email_address AS email_address
FROM addresses
WHERE addresses.email_address LIKE ?
UNION
SELECT addresses.id AS id,
addresses.user_id AS user_id,
addresses.email_address AS email_address
FROM addresses
WHERE addresses.email_address LIKE ?) AS anon_1
EXCEPT
SELECT addresses.id, addresses.user_id, addresses.email_address
FROM addresses
WHERE addresses.email_address LIKE ?
('%@yahoo.com', '%@msn.com', '%@msn.com')
[(1, 1, u'[email protected]')]
See also:
union()
union_all()
intersect()
intersect_all()
except_()
except_all()
Scalar Selects
A scalar select is a SELECT that returns exactly one row and one column. It can then be used as a column expression.
A scalar select is often a correlated subquery, which relies upon the enclosing SELECT statement in order to acquire
at least one of its FROM clauses.
The select() construct can be modified to act as a column expression by calling either the as_scalar() or
label() method:
>>> stmt = select([func.count(addresses.c.id)]).\
...
where(users.c.id == addresses.c.user_id).\
...
as_scalar()
The above construct is now a ScalarSelect object, and is no longer part of the FromClause hierarchy; it instead
is within the ColumnElement family of expression constructs. We can place this construct the same as any other
column within another select():
3.1. SQL Expression Language Tutorial
345
SQLAlchemy Documentation, Release 0.8.7
>>> conn.execute(select([users.c.name, stmt])).fetchall()
SELECT users.name, (SELECT count(addresses.id) AS count_1
FROM addresses
WHERE users.id = addresses.user_id) AS anon_1
FROM users
()
[(u'jack', 2), (u'wendy', 2)]
# doctest: +NORMALIZE_WHITESPACE
To apply a non-anonymous column name to our scalar select, we create it using SelectBase.label() instead:
>>> stmt = select([func.count(addresses.c.id)]).\
...
where(users.c.id == addresses.c.user_id).\
...
label("address_count")
>>> conn.execute(select([users.c.name, stmt])).fetchall() # doctest: +NORMALIZE_WHITESPACE
SELECT users.name, (SELECT count(addresses.id) AS count_1
FROM addresses
WHERE users.id = addresses.user_id) AS address_count
FROM users
()
[(u'jack', 2), (u'wendy', 2)]
See also:
Select.as_scalar()
Select.label()
Correlated Subqueries
Notice in the examples on Scalar Selects, the FROM clause of each embedded select did not contain the users table
in its FROM clause. This is because SQLAlchemy automatically correlates embedded FROM objects to that of an
enclosing query, if present, and if the inner SELECT statement would still have at least one FROM clause of its own.
For example:
>>> stmt = select([addresses.c.user_id]).\
...
where(addresses.c.user_id == users.c.id).\
...
where(addresses.c.email_address == '[email protected]')
>>> enclosing_stmt = select([users.c.name]).where(users.c.id == stmt)
>>> conn.execute(enclosing_stmt).fetchall() # doctest: +NORMALIZE_WHITESPACE
SELECT users.name
FROM users
WHERE users.id = (SELECT addresses.user_id
FROM addresses
WHERE addresses.user_id = users.id
AND addresses.email_address = ?)
('[email protected]',)
[(u'jack',)]
Auto-correlation will usually do what’s expected, however it can also be controlled. For example, if we wanted
a statement to correlate only to the addresses table but not the users table, even if both were present in the
enclosing SELECT, we use the correlate() method to specify those FROM clauses that may be correlated:
>>> stmt = select([users.c.id]).\
...
where(users.c.id == addresses.c.user_id).\
...
where(users.c.name == 'jack').\
346
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
...
correlate(addresses)
>>> enclosing_stmt = select(
...
[users.c.name, addresses.c.email_address]).\
...
select_from(users.join(addresses)).\
...
where(users.c.id == stmt)
>>> conn.execute(enclosing_stmt).fetchall() # doctest: +NORMALIZE_WHITESPACE
SELECT users.name, addresses.email_address
FROM users JOIN addresses ON users.id = addresses.user_id
WHERE users.id = (SELECT users.id
FROM users
WHERE users.id = addresses.user_id AND users.name = ?)
('jack',)
[(u'jack', u'[email protected]'), (u'jack', u'[email protected]')]
To entirely disable a statement from correlating, we can pass None as the argument:
>>> stmt = select([users.c.id]).\
...
where(users.c.name == 'wendy').\
...
correlate(None)
>>> enclosing_stmt = select([users.c.name]).\
...
where(users.c.id == stmt)
>>> conn.execute(enclosing_stmt).fetchall() # doctest: +NORMALIZE_WHITESPACE
SELECT users.name
FROM users
WHERE users.id = (SELECT users.id
FROM users
WHERE users.name = ?)
('wendy',)
[(u'wendy',)]
We can also control correlation via exclusion, using the Select.correlate_except() method. Such as, we
can write our SELECT for the users table by telling it to correlate all FROM clauses except for users:
>>> stmt = select([users.c.id]).\
...
where(users.c.id == addresses.c.user_id).\
...
where(users.c.name == 'jack').\
...
correlate_except(users)
>>> enclosing_stmt = select(
...
[users.c.name, addresses.c.email_address]).\
...
select_from(users.join(addresses)).\
...
where(users.c.id == stmt)
>>> conn.execute(enclosing_stmt).fetchall() # doctest: +NORMALIZE_WHITESPACE
SELECT users.name, addresses.email_address
FROM users JOIN addresses ON users.id = addresses.user_id
WHERE users.id = (SELECT users.id
FROM users
WHERE users.id = addresses.user_id AND users.name = ?)
('jack',)
[(u'jack', u'[email protected]'), (u'jack', u'[email protected]')]
Ordering, Grouping, Limiting, Offset...ing...
Ordering is done by passing column expressions to the order_by() method:
3.1. SQL Expression Language Tutorial
347
SQLAlchemy Documentation, Release 0.8.7
>>> stmt = select([users.c.name]).order_by(users.c.name)
>>> conn.execute(stmt).fetchall() # doctest: +NORMALIZE_WHITESPACE
SELECT users.name
FROM users ORDER BY users.name
()
[(u'jack',), (u'wendy',)]
Ascending or descending can be controlled using the asc() and desc() modifiers:
>>> stmt = select([users.c.name]).order_by(users.c.name.desc())
>>> conn.execute(stmt).fetchall() # doctest: +NORMALIZE_WHITESPACE
SELECT users.name
FROM users ORDER BY users.name DESC
()
[(u'wendy',), (u'jack',)]
Grouping refers to the GROUP BY clause, and is usually used in conjunction with aggregate functions to establish
groups of rows to be aggregated. This is provided via the group_by() method:
>>> stmt = select([users.c.name, func.count(addresses.c.id)]).\
...
select_from(users.join(addresses)).\
...
group_by(users.c.name)
>>> conn.execute(stmt).fetchall() # doctest: +NORMALIZE_WHITESPACE
SELECT users.name, count(addresses.id) AS count_1
FROM users JOIN addresses
ON users.id = addresses.user_id
GROUP BY users.name
()
[(u'jack', 2), (u'wendy', 2)]
HAVING can be used to filter results on an aggregate value, after GROUP BY has been applied. It’s available here via
the having() method:
>>> stmt = select([users.c.name, func.count(addresses.c.id)]).\
...
select_from(users.join(addresses)).\
...
group_by(users.c.name).\
...
having(func.length(users.c.name) > 4)
>>> conn.execute(stmt).fetchall() # doctest: +NORMALIZE_WHITESPACE
SELECT users.name, count(addresses.id) AS count_1
FROM users JOIN addresses
ON users.id = addresses.user_id
GROUP BY users.name
HAVING length(users.name) > ?
(4,)
[(u'wendy', 2)]
A common system of dealing with duplicates in composed SELECT statements is the DISTINCT modifier. A simple
DISTINCT clause can be added using the Select.distinct() method:
>>> stmt = select([users.c.name]).\
...
where(addresses.c.email_address.
...
contains(users.c.name)).\
...
distinct()
>>> conn.execute(stmt).fetchall() # doctest: +NORMALIZE_WHITESPACE
SELECT DISTINCT users.name
348
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
FROM users, addresses
WHERE addresses.email_address LIKE '%%' || users.name || '%%'
()
[(u'jack',), (u'wendy',)]
Most database backends support a system of limiting how many rows are returned, and the majority also feature a
means of starting to return rows after a given “offset”. While common backends like Postgresql, MySQL and SQLite
support LIMIT and OFFSET keywords, other backends need to refer to more esoteric features such as “window
functions” and row ids to achieve the same effect. The limit() and offset() methods provide an easy abstraction
into the current backend’s methodology:
>>> stmt = select([users.c.name, addresses.c.email_address]).\
...
select_from(users.join(addresses)).\
...
limit(1).offset(1)
>>> conn.execute(stmt).fetchall() # doctest: +NORMALIZE_WHITESPACE
SELECT users.name, addresses.email_address
FROM users JOIN addresses ON users.id = addresses.user_id
LIMIT ? OFFSET ?
(1, 1)
[(u'jack', u'[email protected]')]
3.1.14 Inserts, Updates and Deletes
We’ve seen insert() demonstrated earlier in this tutorial. Where insert() produces INSERT, the update()
method produces UPDATE. Both of these constructs feature a method called values() which specifies the VALUES
or SET clause of the statement.
The values() method accommodates any column expression as a value:
>>> stmt = users.update().\
...
values(fullname="Fullname: " + users.c.name)
>>> conn.execute(stmt) #doctest: +ELLIPSIS
UPDATE users SET fullname=(? || users.name)
('Fullname: ',)
COMMIT
<sqlalchemy.engine.result.ResultProxy object at 0x...>
When using insert() or update() in an “execute many” context, we may also want to specify named bound
parameters which we can refer to in the argument list. The two constructs will automatically generate bound placeholders for any column names passed in the dictionaries sent to execute() at execution time. However, if we wish
to use explicitly targeted named parameters with composed expressions, we need to use the bindparam() construct.
When using bindparam() with insert() or update(), the names of the table’s columns themselves are reserved for the “automatic” generation of bind names. We can combine the usage of implicitly available bind names
and explicitly named parameters as in the example below:
>>> stmt = users.insert().\
...
values(name=bindparam('_name') + " .. name")
>>> conn.execute(stmt, [
# doctest: +ELLIPSIS
...
{'id':4, '_name':'name1'},
...
{'id':5, '_name':'name2'},
...
{'id':6, '_name':'name3'},
...
])
INSERT INTO users (id, name) VALUES (?, (? || ?))
((4, 'name1', ' .. name'), (5, 'name2', ' .. name'), (6, 'name3', ' .. name'))
3.1. SQL Expression Language Tutorial
349
SQLAlchemy Documentation, Release 0.8.7
COMMIT
<sqlalchemy.engine.result.ResultProxy object at 0x...>
An UPDATE statement is emitted using the update() construct. This works much like an INSERT, except there is
an additional WHERE clause that can be specified:
>>> stmt = users.update().\
...
where(users.c.name == 'jack').\
...
values(name='ed')
>>> conn.execute(stmt) #doctest: +ELLIPSIS
UPDATE users SET name=? WHERE users.name = ?
('ed', 'jack')
COMMIT
<sqlalchemy.engine.result.ResultProxy object at 0x...>
When using update() in an “execute many” context, we may wish to also use explicitly named bound parameters
in the WHERE clause. Again, bindparam() is the construct used to achieve this:
>>> stmt = users.update().\
...
where(users.c.name == bindparam('oldname')).\
...
values(name=bindparam('newname'))
>>> conn.execute(stmt, [
...
{'oldname':'jack', 'newname':'ed'},
...
{'oldname':'wendy', 'newname':'mary'},
...
{'oldname':'jim', 'newname':'jake'},
...
]) #doctest: +ELLIPSIS
UPDATE users SET name=? WHERE users.name = ?
(('ed', 'jack'), ('mary', 'wendy'), ('jake', 'jim'))
COMMIT
<sqlalchemy.engine.result.ResultProxy object at 0x...>
Correlated Updates
A correlated update lets you update a table using selection from another table, or the same table:
>>> stmt = select([addresses.c.email_address]).\
...
where(addresses.c.user_id == users.c.id).\
...
limit(1)
>>> conn.execute(users.update().values(fullname=stmt)) #doctest: +ELLIPSIS,+NORMALIZE_WHITESPACE
UPDATE users SET fullname=(SELECT addresses.email_address
FROM addresses
WHERE addresses.user_id = users.id
LIMIT ? OFFSET ?)
(1, 0)
COMMIT
<sqlalchemy.engine.result.ResultProxy object at 0x...>
Multiple Table Updates
New in version 0.7.4.
The Postgresql, Microsoft SQL Server, and MySQL backends all support UPDATE statements that refer to multiple
tables. For PG and MSSQL, this is the “UPDATE FROM” syntax, which updates one table at a time, but can reference
350
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
additional tables in an additional “FROM” clause that can then be referenced in the WHERE clause directly. On
MySQL, multiple tables can be embedded into a single UPDATE statement separated by a comma. The SQLAlchemy
update() construct supports both of these modes implicitly, by specifying multiple tables in the WHERE clause:
stmt = users.update().\
values(name='ed wood').\
where(users.c.id == addresses.c.id).\
where(addresses.c.email_address.startswith('ed%'))
conn.execute(stmt)
The resulting SQL from the above statement would render as:
UPDATE users SET name=:name FROM addresses
WHERE users.id = addresses.id AND
addresses.email_address LIKE :email_address_1 || '%%'
When using MySQL, columns from each table can be assigned to in the SET clause directly, using the dictionary form
passed to Update.values():
stmt = users.update().\
values({
users.c.name:'ed wood',
addresses.c.email_address:'[email protected]'
}).\
where(users.c.id == addresses.c.id).\
where(addresses.c.email_address.startswith('ed%'))
The tables are referenced explicitly in the SET clause:
UPDATE users, addresses SET addresses.email_address=%s,
users.name=%s WHERE users.id = addresses.id
AND addresses.email_address LIKE concat(%s, '%%')
SQLAlchemy doesn’t do anything special when these constructs are used on a non-supporting database. The UPDATE
FROM syntax generates by default when multiple tables are present, and the statement will be rejected by the database
if this syntax is not supported.
Deletes
Finally, a delete. This is accomplished easily enough using the delete() construct:
>>> conn.execute(addresses.delete()) #doctest: +ELLIPSIS
DELETE FROM addresses
()
COMMIT
<sqlalchemy.engine.result.ResultProxy object at 0x...>
>>> conn.execute(users.delete().where(users.c.name > 'm')) #doctest: +ELLIPSIS
DELETE FROM users WHERE users.name > ?
('m',)
COMMIT
<sqlalchemy.engine.result.ResultProxy object at 0x...>
3.1. SQL Expression Language Tutorial
351
SQLAlchemy Documentation, Release 0.8.7
Matched Row Counts
Both of update() and delete() are associated with matched row counts. This is a number indicating the number
of rows that were matched by the WHERE clause. Note that by “matched”, this includes rows where no UPDATE
actually took place. The value is available as rowcount:
>>> result = conn.execute(users.delete()) #doctest: +ELLIPSIS
DELETE FROM users
()
COMMIT
>>> result.rowcount
1
3.1.15 Further Reference
Expression Language Reference: SQL Statements and Expressions API
Database Metadata Reference: Describing Databases with MetaData
Engine Reference: Engine Configuration
Connection Reference: Working with Engines and Connections
Types Reference: Column and Data Types
3.2 SQL Statements and Expressions API
This section presents the API reference for the SQL Expression Language. For a full introduction to its usage, see
SQL Expression Language Tutorial.
3.2.1 Column Elements and Expressions
The most fundamental part of the SQL expression API are the “column elements”, which allow for basic SQL expression support. The core of all SQL expression constructs is the ClauseElement, which is the base for several
sub-branches. The ColumnElement class is the fundamental unit used to construct any kind of typed SQL expression.
sqlalchemy.sql.expression.and_(*clauses)
Produce a conjunction of expressions joined by AND.
E.g.:
from sqlalchemy import and_
stmt = select([users_table]).where(
and_(
users_table.c.name == 'wendy',
users_table.c.enrolled == True
)
)
The and_() conjunction is also available using the Python & operator (though note that compound expressions
need to be parenthesized in order to function with Python operator precedence behavior):
352
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
stmt = select([users_table]).where(
(users_table.c.name == 'wendy') &
(users_table.c.enrolled == True)
)
The and_() operation is also implicit in some cases; the Select.where() method for example can be
invoked multiple times against a statement, which will have the effect of each clause being combined using
and_():
stmt = select([users_table]).\
where(users_table.c.name == 'wendy').\
where(users_table.c.enrolled == True)
See also:
or_()
sqlalchemy.sql.expression.asc(column)
Produce an ascending ORDER BY clause element.
e.g.:
from sqlalchemy import asc
stmt = select([users_table]).order_by(asc(users_table.c.name))
will produce SQL as:
SELECT id, name FROM user ORDER BY name ASC
The asc() function is a standalone version of the ColumnElement.asc() method available on all SQL
expressions, e.g.:
stmt = select([users_table]).order_by(users_table.c.name.asc())
Parameters column – A ColumnElement (e.g. scalar SQL expression) with which to apply
the asc() operation.
See also:
desc()
nullsfirst()
nullslast()
Select.order_by()
sqlalchemy.sql.expression.between(expr, lower_bound, upper_bound)
Produce a BETWEEN predicate clause.
E.g.:
from sqlalchemy import between
stmt = select([users_table]).where(between(users_table.c.id, 5, 7))
Would produce SQL resembling:
SELECT id, name FROM user WHERE id BETWEEN :id_1 AND :id_2
The between() function is a standalone version of the ColumnElement.between() method available
on all SQL expressions, as in:
3.2. SQL Statements and Expressions API
353
SQLAlchemy Documentation, Release 0.8.7
stmt = select([users_table]).where(users_table.c.id.between(5, 7))
All arguments passed to between(), including the left side column expression, are coerced from Python scalar
values if a the value is not a ColumnElement subclass. For example, three fixed values can be compared as
in:
print(between(5, 3, 7))
Which would produce:
:param_1 BETWEEN :param_2 AND :param_3
Parameters
• expr – a column expression, typically a ColumnElement instance or alternatively a
Python scalar expression to be coerced into a column expression, serving as the left side
of the BETWEEN expression.
• lower_bound – a column or Python scalar expression serving as the lower bound of
the right side of the BETWEEN expression.
• upper_bound – a column or Python scalar expression serving as the upper bound of
the right side of the BETWEEN expression.
See also:
ColumnElement.between()
sqlalchemy.sql.expression.bindparam(key,
value=<symbol ‘NO_ARG>,
unique=False,
required=<symbol
quote=None, callable_=None)
Produce a “bound expression”.
type_=None,
‘NO_ARG>,
The return value is an instance of BindParameter; this is a ColumnElement subclass which represents
a so-called “placeholder” value in a SQL expression, the value of which is supplied at the point at which the
statement in executed against a database connection.
In SQLAlchemy, the bindparam() construct has the ability to carry along the actual value that will be ultimately used at expression time. In this way, it serves not just as a “placeholder” for eventual population, but also
as a means of representing so-called “unsafe” values which should not be rendered directly in a SQL statement,
but rather should be passed along to the DBAPI as values which need to be correctly escaped and potentially
handled for type-safety.
When using bindparam() explicitly, the use case is typically one of traditional deferment of parameters; the
bindparam() construct accepts a name which can then be referred to at execution time:
from sqlalchemy import bindparam
stmt = select([users_table]).\
where(users_table.c.name == bindparam('username'))
The above statement, when rendered, will produce SQL similar to:
SELECT id, name FROM user WHERE name = :username
In order to populate the value of :username above, the value would typically be applied at execution time to
a method like Connection.execute():
result = connection.execute(stmt, username='wendy')
354
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
Explicit use of bindparam() is also common when producing UPDATE or DELETE statements that are to
be invoked multiple times, where the WHERE criterion of the statement is to change on each invocation, such
as:
stmt = users_table.update().\
where(user_table.c.name == bindparam('username')).\
values(fullname=bindparam('fullname'))
connection.execute(stmt, [
{"username": "wendy", "fullname": "Wendy Smith"},
{"username": "jack", "fullname": "Jack Jones"},
])
SQLAlchemy’s Core expression system makes wide use of bindparam() in an implicit sense. It is typical
that Python literal values passed to virtually all SQL expression functions are coerced into fixed bindparam()
constructs. For example, given a comparison operation such as:
expr = users_table.c.name == 'Wendy'
The above expression will produce a BinaryExpression contruct, where the left side is the Column object
representing the name column, and the right side is a BindParameter representing the literal value:
print(repr(expr.right))
BindParameter('%(4327771088 name)s', 'Wendy', type_=String())
The expression above will render SQL such as:
user.name = :name_1
Where the :name_1 parameter name is an anonymous name. The actual string Wendy is not in the rendered
string, but is carried along where it is later used within statement execution. If we invoke a statement like the
following:
stmt = select([users_table]).where(users_table.c.name == 'Wendy')
result = connection.execute(stmt)
We would see SQL logging output as:
SELECT "user".id, "user".name
FROM "user"
WHERE "user".name = %(name_1)s
{'name_1': 'Wendy'}
Above, we see that Wendy is passed as a parameter to the database, while the placeholder :name_1 is rendered
in the appropriate form for the target database, in this case the Postgresql database.
Similarly, bindparam() is invoked automatically when working with CRUD statements as far as the “VALUES” portion is concerned. The insert() construct produces an INSERT expression which will, at statement
execution time, generate bound placeholders based on the arguments passed, as in:
stmt = users_table.insert()
result = connection.execute(stmt, name='Wendy')
The above will produce SQL output as:
INSERT INTO "user" (name) VALUES (%(name)s)
{'name': 'Wendy'}
The Insert construct, at compilation/execution time, rendered a single bindparam() mirroring the column
name name as a result of the single name parameter we passed to the Connection.execute() method.
3.2. SQL Statements and Expressions API
355
SQLAlchemy Documentation, Release 0.8.7
Parameters
• key – the key (e.g. the name) for this bind param. Will be used in the generated SQL
statement for dialects that use named parameters. This value may be modified when part
of a compilation operation, if other BindParameter objects exist with the same key,
or if its length is too long and truncation is required.
• value – Initial value for this bind param. Will be used at statement execution time as
the value for this parameter passed to the DBAPI, if no other value is indicated to the
statement execution method for this particular parameter name. Defaults to None.
• callable_ – A callable function that takes the place of “value”. The function will
be called at statement execution time to determine the ultimate value. Used for scenarios where the actual bind value cannot be determined at the point at which the clause
construct is created, but embedded bind values are still desirable.
• type_ – A TypeEngine class or instance representing an optional datatype for this
bindparam(). If not passed, a type may be determined automatically for the bind,
based on the given value; for example, trivial Python types such as str, int, bool
may result in the String, Integer or Boolean types being autoamtically selected.
The type of a bindparam() is significant especially in that the type will apply preprocessing to the value before it is passed to the database. For example, a bindparam()
which refers to a datetime value, and is specified as holding the DateTime type, may
apply conversion needed to the value (such as stringification on SQLite) before passing
the value to the database.
• unique – if True, the key name of this BindParameter will be modified if another BindParameter of the same name already has been located within the containing expression. This flag is used generally by the internals when producing socalled “anonymous” bound expressions, it isn’t generally applicable to explicitly-named
bindparam() constructs.
• required – If True, a value is required at execution time. If not passed, it defaults to
True if neither bindparam.value or bindparam.callable were passed. If either of these parameters are present, then bindparam.required defaults to False.
Changed in version 0.8: If the required flag is not specified, it will be set automatically
to True or False depending on whether or not the value or callable parameters
were specified.
• quote – True if this parameter name requires quoting and is not currently known as a
SQLAlchemy reserved word; this currently only applies to the Oracle backend, where
bound names must sometimes be quoted.
• isoutparam – if True, the parameter should be treated like a stored procedure “OUT”
parameter. This applies to backends such as Oracle which support OUT parameters.
See also:
Bind Parameter Objects
Insert Expressions
outparam()
sqlalchemy.sql.expression.case(whens, value=None, else_=None)
Produce a CASE expression.
The CASE construct in SQL is a conditional object that acts somewhat analogously to an “if/then” construct in
other languages. It returns an instance of Case.
356
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
case() in its usual form is passed a list of “when” contructs, that is, a list of conditions and results as tuples:
from sqlalchemy import case
stmt = select([users_table]).\
where(
case(
[
(users_table.c.name == 'wendy', 'W'),
(users_table.c.name == 'jack', 'J')
],
else_='E'
)
)
The above statement will produce SQL resembling:
SELECT id, name FROM user
WHERE CASE
WHEN (name = :name_1) THEN :param_1
WHEN (name = :name_2) THEN :param_2
ELSE :param_3
END
When simple equality expressions of several values against a single parent column are needed, case() also
has a “shorthand” format used via the case.value parameter, which is passed a column expression to be
compared. In this form, the case.whens parameter is passed as a dictionary containing expressions to be
compared against keyed to result expressions. The statement below is equivalent to the preceding statement:
stmt = select([users_table]).\
where(
case(
{"wendy": "W", "jack": "J"},
value=users_table.c.name,
else_='E'
)
)
The values which are accepted as result values in case.whens as well as with case.else_ are coerced
from Python literals into bindparam() constructs. SQL expressions, e.g. ColumnElement constructs,
are accepted as well. To coerce a literal string expression into a constant expression rendered inline, use the
literal_column() construct, as in:
from sqlalchemy import case, literal_column
case(
[
(
orderline.c.qty > 100,
literal_column("'greaterthan100'")
),
(
orderline.c.qty > 10,
literal_column("'greaterthan10'")
)
],
else_=literal_column("'lessthan10'")
)
The above will render the given constants without using bound parameters for the result values (but still for the
3.2. SQL Statements and Expressions API
357
SQLAlchemy Documentation, Release 0.8.7
comparison values), as in:
CASE
WHEN (orderline.qty > :qty_1) THEN 'greaterthan100'
WHEN (orderline.qty > :qty_2) THEN 'greaterthan10'
ELSE 'lessthan10'
END
Parameters
• whens – The criteria to be compared against, case.whens accepts two different forms,
based on whether or not case.value is used.
In the first form, it accepts a list of 2-tuples; each 2-tuple consists of (<sql
expression>, <value>), where the SQL expression is a boolean expression and
“value” is a resulting value, e.g.:
case([
(users_table.c.name == 'wendy', 'W'),
(users_table.c.name == 'jack', 'J')
])
In the second form, it accepts a Python dictionary of comparison values mapped to a
resulting value; this form requires case.value to be present, and values will be compared using the == operator, e.g.:
case(
{"wendy": "W", "jack": "J"},
value=users_table.c.name
)
• value – An optional SQL expression which will be used as a fixed “comparison point”
for candidate values within a dictionary passed to case.whens.
• else_ – An optional SQL expression which will be the evaluated result of the CASE
construct if all expressions within case.whens evaluate to false. When omitted, most
databases will produce a result of NULL if none of the “when” expressions evaulate to
true.
sqlalchemy.sql.expression.cast(expression, type_, **kw)
Produce a CAST expression.
cast() returns an instance of Cast.
E.g.:
from sqlalchemy import cast, Numeric
stmt = select([
cast(product_table.c.unit_price, Numeric(10, 4))
])
The above statement will produce SQL resembling:
SELECT CAST(unit_price AS NUMERIC(10, 4)) FROM product
The cast() function performs two distinct functions when used. The first is that it renders the CAST expression within the resulting SQL string. The second is that it associates the given type (e.g. TypeEngine
358
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
class or instance) with the column expression on the Python side, which means the expression will take on the
expression operator behavior associated with that type as well as the result-row-handling behavior of the type.
An alternative to cast() is the type_coerce() function. This function performs the second task of associating an expression with a specific type, but does not render the CAST expression in SQL.
Parameters
• expression – A SQL expression, such as a ColumnElement expression or a Python
string which will be coerced into a bound literal value.
• type_ – A TypeEngine class or instance indicating the type to which the CAST should
apply.
• **kw – additional keyword arguments are unused.
Deprecated since version 0.8: **kw argument to cast() is removed in 0.9.
See also:
type_coerce() - Python-side type coercion without emitting CAST.
sqlalchemy.sql.expression.column(text, type_=None)
Produce a ColumnClause object.
The ColumnClause is a lightweight analogue to the Column class. The column() function can be invoked
with just a name alone, as in:
from sqlalchemy.sql import column
id, name = column("id"), column("name")
stmt = select([id, name]).select_from("user")
The above statement would produce SQL like:
SELECT id, name FROM user
Once constructed, column() may be used like any other SQL expression element such as within select()
constructs:
from sqlalchemy.sql import column
id, name = column("id"), column("name")
stmt = select([id, name]).select_from("user")
The text handled by column() is assumed to be handled like the name of a database column; if the string
contains mixed case, special characters, or matches a known reserved word on the target backend, the column
expression will render using the quoting behavior determined by the backend. To produce a textual SQL expression that is rendered exactly without any quoting, use literal_column() instead, or pass True as
the value of column.is_literal. Additionally, full SQL statements are best handled using the text()
construct.
column() can be used in a table-like fashion by combining it with the table() function (which is the
lightweight analogue to Table) to produce a working table construct with minimal boilerplate:
from sqlalchemy.sql import table, column
user = table("user",
column("id"),
column("name"),
column("description"),
)
3.2. SQL Statements and Expressions API
359
SQLAlchemy Documentation, Release 0.8.7
stmt = select([user.c.description]).where(user.c.name == 'wendy')
A column() / table() construct like that illustrated above can be created in an ad-hoc fashion and is not
associated with any schema.MetaData, DDL, or events, unlike its Table counterpart.
Parameters
• text – the text of the element.
• type – types.TypeEngine object which can associate this ColumnClause with
a type.
• is_literal – if True, the ColumnClause is assumed to be an exact expression that
will be delivered to the output with no quoting rules applied regardless of case sensitive settings. the literal_column() function essentially invokes column() while
passing is_literal=True.
See also:
Column
literal_column()
text()
Describing Databases with MetaData
sqlalchemy.sql.expression.collate(expression, collation)
Return the clause expression COLLATE collation.
e.g.:
collate(mycolumn, 'utf8_bin')
produces:
mycolumn COLLATE utf8_bin
sqlalchemy.sql.expression.desc(column)
Produce a descending ORDER BY clause element.
e.g.:
from sqlalchemy import desc
stmt = select([users_table]).order_by(desc(users_table.c.name))
will produce SQL as:
SELECT id, name FROM user ORDER BY name DESC
The desc() function is a standalone version of the ColumnElement.desc() method available on all SQL
expressions, e.g.:
stmt = select([users_table]).order_by(users_table.c.name.desc())
Parameters column – A ColumnElement (e.g. scalar SQL expression) with which to apply
the desc() operation.
See also:
asc()
360
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
nullsfirst()
nullslast()
Select.order_by()
sqlalchemy.sql.expression.distinct(expr)
Produce an column-expression-level unary DISTINCT clause.
This applies the DISTINCT keyword to an individual column expression, and is typically contained within an
aggregate function, as in:
from sqlalchemy import distinct, func
stmt = select([func.count(distinct(users_table.c.name))])
The above would produce an expression resembling:
SELECT COUNT(DISTINCT name) FROM user
The
distinct()
function
is
also
ColumnElement.distinct(), as in:
available
as
a
column-level
method,
e.g.
stmt = select([func.count(users_table.c.name.distinct())])
The distinct() operator is different from the Select.distinct() method of Select, which produces a SELECT statement with DISTINCT applied to the result set as a whole, e.g. a SELECT DISTINCT
expression. See that method for further information.
See also:
ColumnElement.distinct()
Select.distinct()
func
sqlalchemy.sql.expression.extract(field, expr)
Return the clause extract(field FROM expr).
sqlalchemy.sql.expression.false()
Return a False_ object, which compiles to false, or the boolean equivalent for the target dialect.
sqlalchemy.sql.expression.func = <sqlalchemy.sql.expression._FunctionGenerator object>
Generate SQL function expressions.
func is a special object instance which generates SQL functions based on name-based attributes, e.g.:
>>> print func.count(1)
count(:param_1)
The element is a column-oriented SQL element like any other, and is used in that way:
>>> print select([func.count(table.c.id)])
SELECT count(sometable.id) FROM sometable
Any name can be given to func. If the function name is unknown to SQLAlchemy, it will be rendered exactly
as is. For common SQL functions which SQLAlchemy is aware of, the name may be interpreted as a generic
function which will be compiled appropriately to the target database:
>>> print func.current_timestamp()
CURRENT_TIMESTAMP
To call functions which are present in dot-separated packages, specify them in the same manner:
3.2. SQL Statements and Expressions API
361
SQLAlchemy Documentation, Release 0.8.7
>>> print func.stats.yield_curve(5, 10)
stats.yield_curve(:yield_curve_1, :yield_curve_2)
SQLAlchemy can be made aware of the return type of functions to enable type-specific lexical and result-based
behavior. For example, to ensure that a string-based function returns a Unicode value and is similarly treated as
a string in expressions, specify Unicode as the type:
>>> print func.my_string(u'hi', type_=Unicode) + ' ' + \
... func.my_string(u'there', type_=Unicode)
my_string(:my_string_1) || :my_string_2 || my_string(:my_string_3)
The object returned by a func call is usually an instance of Function. This object meets the “column”
interface, including comparison and labeling functions. The object can also be passed the execute() method
of a Connection or Engine, where it will be wrapped inside of a SELECT statement first:
print connection.execute(func.current_timestamp()).scalar()
In a few exception cases, the func accessor will redirect a name to a built-in expression such as cast() or
extract(), as these names have well-known meaning but are not exactly the same as “functions” from a
SQLAlchemy perspective.
New in version 0.8: func can return non-function expression constructs for common quasi-functional names
like cast() and extract().
Functions which are interpreted as “generic” functions know how to calculate their return type automatically.
For a listing of known generic functions, see SQL and Generic Functions.
sqlalchemy.sql.expression.label(name, obj)
Return a Label object for the given ColumnElement.
A label changes the name of an element in the columns clause of a SELECT statement, typically via the AS SQL
keyword.
This functionality is more conveniently available via the label() method on ColumnElement.
name label name
obj a ColumnElement.
sqlalchemy.sql.expression.literal(value, type_=None)
Return a literal clause, bound to a bind parameter.
Literal clauses are created automatically when non- ClauseElement objects (such as strings, ints, dates,
etc.) are used in a comparison operation with a ColumnElement subclass, such as a Column object. Use this
function to force the generation of a literal clause, which will be created as a BindParameter with a bound
value.
Parameters
• value – the value to be bound. Can be any Python object supported by the underlying
DB-API, or is translatable via the given type argument.
• type_ – an optional TypeEngine which will provide bind-parameter translation for
this literal.
sqlalchemy.sql.expression.literal_column(text, type_=None)
Return a textual column expression, as would be in the columns clause of a SELECT statement.
The object returned supports further expressions in the same way as any other column object, including comparison, math and string operations. The type_ parameter is important to determine proper expression behavior
(such as, ‘+’ means string concatenation or numerical addition based on the type).
Parameters
362
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
• text – the text of the expression; can be any SQL expression. Quoting rules will not be
applied. To specify a column-name expression which should be subject to quoting rules,
use the column() function.
• type_ – an optional TypeEngine object which will provide result-set translation and
additional expression semantics for this column. If left as None the type will be NullType.
sqlalchemy.sql.expression.not_(clause)
Return a negation of the given clause, i.e. NOT(clause).
The ~ operator is also overloaded on all ColumnElement subclasses to produce the same result.
sqlalchemy.sql.expression.null()
Return a Null object, which compiles to NULL.
sqlalchemy.sql.expression.nullsfirst(column)
Produce the NULLS FIRST modifier for an ORDER BY expression.
nullsfirst() is intended to modify the expression produced by asc() or desc(), and indicates how
NULL values should be handled when they are encountered during ordering:
from sqlalchemy import desc, nullsfirst
stmt = select([users_table]).\
order_by(nullsfirst(desc(users_table.c.name)))
The SQL expression from the above would resemble:
SELECT id, name FROM user ORDER BY name DESC NULLS FIRST
Like asc() and desc(), nullsfirst() is typically invoked from the column expression itself using
ColumnElement.nullsfirst(), rather than as its standalone function version, as in:
stmt = select([users_table]).\
order_by(users_table.c.name.desc().nullsfirst())
See also:
asc()
desc()
nullslast()
Select.order_by()
sqlalchemy.sql.expression.nullslast(column)
Produce the NULLS LAST modifier for an ORDER BY expression.
nullslast() is intended to modify the expression produced by asc() or desc(), and indicates how
NULL values should be handled when they are encountered during ordering:
from sqlalchemy import desc, nullslast
stmt = select([users_table]).\
order_by(nullslast(desc(users_table.c.name)))
The SQL expression from the above would resemble:
SELECT id, name FROM user ORDER BY name DESC NULLS LAST
Like asc() and desc(), nullslast() is typically invoked from the column expression itself using
ColumnElement.nullslast(), rather than as its standalone function version, as in:
3.2. SQL Statements and Expressions API
363
SQLAlchemy Documentation, Release 0.8.7
stmt = select([users_table]).\
order_by(users_table.c.name.desc().nullslast())
See also:
asc()
desc()
nullsfirst()
Select.order_by()
sqlalchemy.sql.expression.or_(*clauses)
Produce a conjunction of expressions joined by OR.
E.g.:
from sqlalchemy import or_
stmt = select([users_table]).where(
or_(
users_table.c.name == 'wendy',
users_table.c.name == 'jack'
)
)
The or_() conjunction is also available using the Python | operator (though note that compound expressions
need to be parenthesized in order to function with Python operator precedence behavior):
stmt = select([users_table]).where(
(users_table.c.name == 'wendy') |
(users_table.c.name == 'jack')
)
See also:
and_()
sqlalchemy.sql.expression.outparam(key, type_=None)
Create an ‘OUT’ parameter for usage in functions (stored procedures), for databases which support them.
The outparam can be used like a regular function parameter. The “output” value will be available from the
ResultProxy object via its out_parameters attribute, which returns a dictionary containing the values.
sqlalchemy.sql.expression.over(func, partition_by=None, order_by=None)
Produce an OVER clause against a function.
Used against aggregate or so-called “window” functions, for database backends that support window functions.
E.g.:
from sqlalchemy import over
over(func.row_number(), order_by='x')
Would produce “ROW_NUMBER() OVER(ORDER BY x)”.
Parameters
• func – a FunctionElement construct, typically generated by func.
• partition_by – a column element or string, or a list of such, that will be used as the
PARTITION BY clause of the OVER construct.
364
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
• order_by – a column element or string, or a list of such, that will be used as the
ORDER BY clause of the OVER construct.
This function is also available from the func construct itself via the FunctionElement.over() method.
New in version 0.7.
sqlalchemy.sql.expression.text(text, bind=None, *args, **kwargs)
Create a SQL construct that is represented by a literal string.
E.g.:
t = text("SELECT * FROM users")
result = connection.execute(t)
The advantages text() provides over a plain string are backend-neutral support for bind parameters,
per-statement execution options, as well as bind parameter and result-column typing behavior, allowing
SQLAlchemy type constructs to play a role when executing a statement that is specified literally.
Bind parameters are specified by name, using the format :name. E.g.:
t = text("SELECT * FROM users WHERE id=:user_id")
result = connection.execute(t, user_id=12)
For SQL statements where a colon is required verbatim, as within an inline string, use a backslash to escape:
t = text("SELECT * FROM users WHERE name='\:username'")
To invoke SQLAlchemy typing logic for bind parameters, the bindparams list allows specification of
bindparam() constructs which specify the type for a given name:
t = text("SELECT id FROM users WHERE updated_at>:updated",
bindparams=[bindparam('updated', DateTime())]
)
Typing during result row processing is also an important concern. Result column types are specified using
the typemap dictionary, where the keys match the names of columns. These names are taken from what the
DBAPI returns as cursor.description:
t = text("SELECT id, name FROM users",
typemap={
'id':Integer,
'name':Unicode
}
)
The text() construct is used internally for most cases when a literal string is specified for part of a larger
query, such as within select(), update(), insert() or delete(). In those cases, the same bind
parameter syntax is applied:
s = select([users.c.id, users.c.name]).where("id=:user_id")
result = connection.execute(s, user_id=12)
Using text() explicitly usually implies the construction of a full, standalone statement.
As such, SQLAlchemy refers to it as an Executable object, and it supports the
Executable.execution_options() method. For example, a text() construct that should be subject
to “autocommit” can be set explicitly so using the Connection.execution_options.autocommit
option:
t = text("EXEC my_procedural_thing()").\
execution_options(autocommit=True)
3.2. SQL Statements and Expressions API
365
SQLAlchemy Documentation, Release 0.8.7
Note that SQLAlchemy’s usual “autocommit” behavior applies to text() constructs - that is, statements which
begin with a phrase such as INSERT, UPDATE, DELETE, or a variety of other phrases specific to certain
backends, will be eligible for autocommit if no transaction is in progress.
Parameters
• text – the text of the SQL statement to be created. use :<param> to specify bind
parameters; they will be compiled to their engine-specific format.
• autocommit – Deprecated. Use .execution_options(autocommit=<True|False>) to set
the autocommit option.
• bind – an optional connection or engine to be used for this text query.
• bindparams – a list of bindparam() instances which can be used to define the types
and/or initial values for the bind parameters within the textual statement; the keynames
of the bindparams must match those within the text of the statement. The types will be
used for pre-processing on bind values.
• typemap – a dictionary mapping the names of columns represented in the columns
clause of a SELECT statement to type objects, which will be used to perform postprocessing on columns within the result set. This argument applies to any expression
that returns result sets.
sqlalchemy.sql.expression.true()
Return a True_ object, which compiles to true, or the boolean equivalent for the target dialect.
sqlalchemy.sql.expression.tuple_(*expr)
Return a SQL tuple.
Main usage is to produce a composite IN construct:
tuple_(table.c.col1, table.c.col2).in_(
[(1, 2), (5, 12), (10, 19)]
)
Warning: The composite IN construct is not supported by all backends, and is currently known to work
on Postgresql and MySQL, but not SQLite. Unsupported backends will raise a subclass of DBAPIError
when such an expression is invoked.
sqlalchemy.sql.expression.type_coerce(expr, type_)
Associate a SQL expression with a particular type, without rendering CAST.
E.g.:
from sqlalchemy import type_coerce
stmt = select([type_coerce(log_table.date_string, StringDateTime())])
The above construct will produce SQL that is usually otherwise unaffected by the type_coerce() call:
SELECT date_string FROM log
However, when result rows are fetched, the StringDateTime type will be applied to result rows on behalf
of the date_string column.
A type that features bound-value handling will also have that behavior take effect when literal values or
bindparam() constructs are passed to type_coerce() as targets. For example, if a type implements the
TypeEngine.bind_expression() method or TypeEngine.bind_processor() method or equivalent, these functions will take effect at statement compliation/execution time when a literal value is passed, as
in:
366
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
# bound-value handling of MyStringType will be applied to the
# literal value "some string"
stmt = select([type_coerce("some string", MyStringType)])
type_coerce() is similar to the cast() function, except that it does not render the CAST expression
in the resulting statement; the bind-side behavior of type_coerce() is also not present in cast() until
SQLAlchemy version 0.9.0.
Parameters
• expression – A SQL expression, such as a ColumnElement expression or a Python
string which will be coerced into a bound literal value.
• type_ – A TypeEngine class or instance indicating the type to which the the expression is coerced.
See also:
cast()
class sqlalchemy.sql.expression.BinaryExpression(left, right, operator, type_=None,
negate=None, modifiers=None)
Bases: sqlalchemy.sql.expression.ColumnElement
Represent an expression that is LEFT <operator> RIGHT.
A BinaryExpression is generated automatically whenever two column expressions are used in a Python
binary expresion:
>>> from sqlalchemy.sql import column
>>> column('a') + column('b')
<sqlalchemy.sql.expression.BinaryExpression object at 0x101029dd0>
>>> print column('a') + column('b')
a + b
compare(other, **kw)
Compare this BinaryExpression against the given BinaryExpression.
class sqlalchemy.sql.expression.BindParameter(key, value, type_=None, unique=False,
callable_=None,
isoutparam=False,
required=False,
quote=None,
_compared_to_operator=None,
_compared_to_type=None)
Bases: sqlalchemy.sql.expression.ColumnElement
Represent a “bound expression”.
BindParameter is invoked explicitly using the bindparam() function, as in:
from sqlalchemy import bindparam
stmt = select([users_table]).\
where(users_table.c.name == bindparam('username'))
Detailed discussion of how BindParameter is used is at bindparam().
See also:
bindparam()
__init__(key, value, type_=None, unique=False, callable_=None, isoutparam=False, required=False, quote=None, _compared_to_operator=None, _compared_to_type=None)
Construct a BindParameter.
3.2. SQL Statements and Expressions API
367
SQLAlchemy Documentation, Release 0.8.7
Parameters
• key – the key for this bind param. Will be used in the generated SQL statement
for dialects that use named parameters. This value may be modified when part of a
compilation operation, if other BindParameter objects exist with the same key, or
if its length is too long and truncation is required.
• value – Initial value for this bind param. This value may be overridden by the
dictionary of parameters sent to statement compilation/execution.
• callable_ – A callable function that takes the place of “value”. The function
will be called at statement execution time to determine the ultimate value. Used for
scenarios where the actual bind value cannot be determined at the point at which the
clause construct is created, but embedded bind values are still desirable.
• type_ – A TypeEngine object that will be used to pre-process the value corresponding to this BindParameter at execution time.
• unique – if True, the key name of this BindParamClause will be modified if another
BindParameter of the same name already has been located within the containing
ClauseElement.
• quote – True if this parameter name requires quoting and is not currently known as
a SQLAlchemy reserved word; this currently only applies to the Oracle backend.
• required – a value is required at execution time.
• isoutparam – if True, the parameter should be treated like a stored procedure
“OUT” parameter.
compare(other, **kw)
Compare this BindParameter to the given clause.
effective_value
Return the value of this bound parameter, taking into account if the callable parameter was set.
The callable value will be evaluated and returned if present, else value.
class sqlalchemy.sql.expression.Case(whens, value=None, else_=None)
Bases: sqlalchemy.sql.expression.ColumnElement
Represent a CASE expression.
Case is produced using the case() factory function, as in:
from sqlalchemy import case
stmt = select([users_table]).\
where(
case(
[
(users_table.c.name == 'wendy', 'W'),
(users_table.c.name == 'jack', 'J')
],
else_='E'
)
)
Details on Case usage is at case().
See also:
case()
368
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
class sqlalchemy.sql.expression.Cast(clause, totype, **kwargs)
Bases: sqlalchemy.sql.expression.ColumnElement
Represent a CAST expression.
Cast is produced using the cast() factory function, as in:
from sqlalchemy import cast, Numeric
stmt = select([
cast(product_table.c.unit_price, Numeric(10, 4))
])
Details on Cast usage is at cast().
See also:
cast()
class sqlalchemy.sql.expression.ClauseElement
Bases: sqlalchemy.sql.visitors.Visitable
Base class for elements of a programmatically constructed SQL expression.
compare(other, **kw)
Compare this ClauseElement to the given ClauseElement.
Subclasses should override the default behavior, which is a straight identity comparison.
**kw are arguments consumed by subclass compare() methods and may be used to modify the criteria for
comparison. (see ColumnElement)
compile(bind=None, dialect=None, **kw)
Compile this SQL expression.
The return value is a Compiled object. Calling str() or unicode() on the returned value will yield
a string representation of the result. The Compiled object also can return a dictionary of bind parameter
names and values using the params accessor.
Parameters
• bind – An Engine or Connection from which a Compiled will be acquired.
This argument takes precedence over this ClauseElement‘s bound engine, if any.
• column_keys – Used for INSERT and UPDATE statements, a list of column names
which should be present in the VALUES clause of the compiled statement. If None,
all columns from the target table object are rendered.
• dialect – A Dialect instance from which a Compiled will be acquired. This argument takes precedence over the bind argument as well as this
ClauseElement‘s bound engine, if any.
• inline – Used for INSERT statements, for a dialect which does not support inline
retrieval of newly generated primary key columns, will force the expression used to
create the new primary key value to be rendered inline within the INSERT statement’s
VALUES clause. This typically refers to Sequence execution but may also refer to any
server-side default generation function associated with a primary key Column.
get_children(**kwargs)
Return immediate child elements of this ClauseElement.
This is used for visit traversal.
3.2. SQL Statements and Expressions API
369
SQLAlchemy Documentation, Release 0.8.7
**kwargs may contain flags that change the collection that is returned, for example to return a subset of
items in order to cut down on larger traversals, or to return child items from a different context (such as
schema-level collections instead of clause-level).
params(*optionaldict, **kwargs)
Return a copy with bindparam() elements replaced.
Returns a copy of this ClauseElement with bindparam() elements replaced with values taken from the
given dictionary:
>>> clause = column('x') + bindparam('foo')
>>> print clause.compile().params
{'foo':None}
>>> print clause.params({'foo':7}).compile().params
{'foo':7}
self_group(against=None)
Apply a ‘grouping’ to this ClauseElement.
This method is overridden by subclasses to return a “grouping” construct, i.e. parenthesis. In particular
it’s used by “binary” expressions to provide a grouping around themselves when placed into a larger expression, as well as by select() constructs when placed into the FROM clause of another select().
(Note that subqueries should be normally created using the Select.alias() method, as many platforms require nested SELECT statements to be named).
As expressions are composed together, the application of self_group() is automatic - end-user code
should never need to use this method directly. Note that SQLAlchemy’s clause constructs take operator
precedence into account - so parenthesis might not be needed, for example, in an expression like x OR
(y AND z) - AND takes precedence over OR.
The base self_group() method of ClauseElement just returns self.
unique_params(*optionaldict, **kwargs)
Return a copy with bindparam() elements replaced.
Same functionality as params(), except adds unique=True to affected bind parameters so that multiple
statements can be used.
class sqlalchemy.sql.expression.ClauseList(*clauses, **kwargs)
Bases: sqlalchemy.sql.expression.ClauseElement
Describe a list of clauses, separated by an operator.
By default, is comma-separated, such as a column listing.
compare(other, **kw)
Compare this ClauseList to the given ClauseList, including a comparison of all the clause items.
class sqlalchemy.sql.expression.ColumnClause(text,
selectable=None,
type_=None,
is_literal=False)
Bases: sqlalchemy.sql.expression.Immutable, sqlalchemy.sql.expression.ColumnElement
Represents a column expression from any textual string.
The ColumnClause, a lightweight analogue to the Column class, is typically invoked using the column()
function, as in:
from sqlalchemy.sql import column
id, name = column("id"), column("name")
stmt = select([id, name]).select_from("user")
The above statement would produce SQL like:
370
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
SELECT id, name FROM user
ColumnClause is the immediate superclass of the schema-specific Column object. While the Column class
has all the same capabilities as ColumnClause, the ColumnClause class is usable by itself in those cases
where behavioral requirements are limited to simple SQL expression generation. The object has none of the
associations with schema-level metadata or with execution-time behavior that Column does, so in that sense is
a “lightweight” version of Column.
Full details on ColumnClause usage is at column().
See also:
column()
Column
class sqlalchemy.sql.expression.ColumnCollection(*cols)
Bases: sqlalchemy.util._collections.OrderedProperties
An ordered dictionary that stores a list of ColumnElement instances.
Overrides the __eq__() method to produce SQL clauses between sets of correlated columns.
add(column)
Add a column to this collection.
The key attribute of the column will be used as the hash key for this dictionary.
replace(column)
add the given column to this collection, removing unaliased versions of this column as well as existing
columns with the same key.
e.g.:
t = Table('sometable', metadata, Column('col1', Integer))
t.columns.replace(Column('col1', Integer, key='columnone'))
will remove the original ‘col1’ from the collection, and add the new column under the name
‘columnname’.
Used by schema.Column to override columns during table reflection.
class sqlalchemy.sql.expression.ColumnElement
Bases: sqlalchemy.sql.expression.ClauseElement, sqlalchemy.sql.operators.ColumnOperators
Represent a column-oriented SQL expression suitable for usage in the “columns” clause, WHERE clause etc.
of a statement.
While the most familiar kind of ColumnElement is the Column object, ColumnElement serves as the
basis for any unit that may be present in a SQL expression, including the expressions themselves, SQL functions,
bound parameters, literal expressions, keywords such as NULL, etc. ColumnElement is the ultimate base class
for all such elements.
A wide variety of SQLAlchemy Core functions work at the SQL expression level, and are intended to accept
instances of ColumnElement as arguments. These functions will typically document that they accept a “SQL
expression” as an argument. What this means in terms of SQLAlchemy usually refers to an input which is either
already in the form of a ColumnElement object, or a value which can be coerced into one. The coercion rules
followed by most, but not all, SQLAlchemy Core functions with regards to SQL expressions are as follows:
•a literal Python value, such as a string, integer or floating point value, boolean, datetime, Decimal
object, or virtually any other Python object, will be coerced into a “literal bound value”. This generally
means that a bindparam() will be produced featuring the given value embedded into the construct;
the resulting BindParameter object is an instance of ColumnElement. The Python value will
3.2. SQL Statements and Expressions API
371
SQLAlchemy Documentation, Release 0.8.7
ultimately be sent to the DBAPI at execution time as a paramterized argument to the execute() or
executemany() methods, after SQLAlchemy type-specific converters (e.g. those provided by any
associated TypeEngine objects) are applied to the value.
•any special object value, typically ORM-level constructs, which feature a method called
__clause_element__(). The Core expression system looks for this method when an object of otherwise unknown type is passed to a function that is looking to coerce the argument into a ColumnElement
expression. The __clause_element__() method, if present, should return a ColumnElement instance. The primary use of __clause_element__() within SQLAlchemy is that of class-bound
attributes on ORM-mapped classes; a User class which contains a mapped attribute named .name will
have a method User.name.__clause_element__() which when invoked returns the Column
called name associated with the mapped table.
•The Python None value is typically interpreted as NULL, which in SQLAlchemy Core produces an instance of null().
A ColumnElement provides the ability to generate new ColumnElement objects using Python expressions.
This means that Python operators such as ==, != and < are overloaded to mimic SQL operations, and allow
the instantiation of further ColumnElement instances which are composed from other, more fundamental
ColumnElement objects. For example, two ColumnClause objects can be added together with the addition operator + to produce a BinaryExpression. Both ColumnClause and BinaryExpression are
subclasses of ColumnElement:
>>> from sqlalchemy.sql import column
>>> column('a') + column('b')
<sqlalchemy.sql.expression.BinaryExpression object at 0x101029dd0>
>>> print column('a') + column('b')
a + b
See also:
Column
expression.column()
__eq__(other)
inherited from the __eq__() method of ColumnOperators
Implement the == operator.
In a column context, produces the clause a = b. If the target is None, produces a IS NULL.
__init__
inherited from the __init__ attribute of object
x.__init__(...) initializes x; see help(type(x)) for signature
__le__(other)
inherited from the __le__() method of ColumnOperators
Implement the <= operator.
In a column context, produces the clause a <= b.
__lt__(other)
inherited from the __lt__() method of ColumnOperators
Implement the < operator.
In a column context, produces the clause a < b.
__ne__(other)
inherited from the __ne__() method of ColumnOperators
372
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
Implement the != operator.
In a column context, produces the clause a != b. If the target is None, produces a IS NOT NULL.
anon_label
provides a constant ‘anonymous label’ for this ColumnElement.
This is a label() expression which will be named at compile time. The same label() is returned each time
anon_label is called so that expressions can reference anon_label multiple times, producing the same label
name at compile time.
the compiler uses this function automatically at compile time for expressions that are known to be ‘unnamed’ like binary expressions and function calls.
asc()
inherited from the asc() method of ColumnOperators
Produce a asc() clause against the parent object.
base_columns
between(cleft, cright)
inherited from the between() method of ColumnOperators
Produce a between() clause against the parent object, given the lower and upper range.
bind = None
collate(collation)
inherited from the collate() method of ColumnOperators
Produce a collate() clause against the parent object, given the collation string.
comparator
compare(other, use_proxies=False, equivalents=None, **kw)
Compare this ColumnElement to another.
Special arguments understood:
Parameters
• use_proxies – when True, consider two columns that share a common base column as equivalent (i.e. shares_lineage())
• equivalents – a dictionary of columns as keys mapped to sets of columns. If
the given “other” column is present in this dictionary, if any of the columns in the
corresponding set() pass the comparison test, the result is True. This is used to expand
the comparison to other columns that may be known to be equivalent to this one via
foreign key or other criterion.
compile(bind=None, dialect=None, **kw)
inherited from the compile() method of ClauseElement
Compile this SQL expression.
The return value is a Compiled object. Calling str() or unicode() on the returned value will yield
a string representation of the result. The Compiled object also can return a dictionary of bind parameter
names and values using the params accessor.
Parameters
• bind – An Engine or Connection from which a Compiled will be acquired.
This argument takes precedence over this ClauseElement‘s bound engine, if any.
3.2. SQL Statements and Expressions API
373
SQLAlchemy Documentation, Release 0.8.7
• column_keys – Used for INSERT and UPDATE statements, a list of column names
which should be present in the VALUES clause of the compiled statement. If None,
all columns from the target table object are rendered.
• dialect – A Dialect instance from which a Compiled will be acquired. This argument takes precedence over the bind argument as well as this
ClauseElement‘s bound engine, if any.
• inline – Used for INSERT statements, for a dialect which does not support inline
retrieval of newly generated primary key columns, will force the expression used to
create the new primary key value to be rendered inline within the INSERT statement’s
VALUES clause. This typically refers to Sequence execution but may also refer to any
server-side default generation function associated with a primary key Column.
concat(other)
inherited from the concat() method of ColumnOperators
Implement the ‘concat’ operator.
In a column context, produces the clause a || b, or uses the concat() operator on MySQL.
contains(other, **kwargs)
inherited from the contains() method of ColumnOperators
Implement the ‘contains’ operator.
In a column context, produces the clause LIKE ’%<other>%’
desc()
inherited from the desc() method of ColumnOperators
Produce a desc() clause against the parent object.
distinct()
inherited from the distinct() method of ColumnOperators
Produce a distinct() clause against the parent object.
endswith(other, **kwargs)
inherited from the endswith() method of ColumnOperators
Implement the ‘endswith’ operator.
In a column context, produces the clause LIKE ’%<other>’
expression
Return a column expression.
Part of the inspection interface; returns self.
foreign_keys = []
get_children(**kwargs)
inherited from the get_children() method of ClauseElement
Return immediate child elements of this ClauseElement.
This is used for visit traversal.
**kwargs may contain flags that change the collection that is returned, for example to return a subset of
items in order to cut down on larger traversals, or to return child items from a different context (such as
schema-level collections instead of clause-level).
ilike(other, escape=None)
inherited from the ilike() method of ColumnOperators
374
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
Implement the ilike operator.
In a column context, produces the clause a ILIKE other.
E.g.:
select([sometable]).where(sometable.c.column.ilike("%foobar%"))
Parameters
• other – expression to be compared
• escape – optional escape character, renders the ESCAPE keyword, e.g.:
somecolumn.ilike("foo/%bar", escape="/")
See also:
ColumnOperators.like()
in_(other)
inherited from the in_() method of ColumnOperators
Implement the in operator.
In a column context, produces the clause a IN other. “other” may be a tuple/list of column expressions, or a select() construct.
is_(other)
inherited from the is_() method of ColumnOperators
Implement the IS operator.
Normally, IS is generated automatically when comparing to a value of None, which resolves to NULL.
However, explicit usage of IS may be desirable if comparing to boolean values on certain platforms.
New in version 0.7.9.
See also:
ColumnOperators.isnot()
is_clause_element = True
is_selectable = False
isnot(other)
inherited from the isnot() method of ColumnOperators
Implement the IS NOT operator.
Normally, IS NOT is generated automatically when comparing to a value of None, which resolves to
NULL. However, explicit usage of IS NOT may be desirable if comparing to boolean values on certain
platforms.
New in version 0.7.9.
See also:
ColumnOperators.is_()
label(name)
Produce a column label, i.e. <columnname> AS <name>.
This is a shortcut to the label() function.
3.2. SQL Statements and Expressions API
375
SQLAlchemy Documentation, Release 0.8.7
if ‘name’ is None, an anonymous label name will be generated.
like(other, escape=None)
inherited from the like() method of ColumnOperators
Implement the like operator.
In a column context, produces the clause a LIKE other.
E.g.:
select([sometable]).where(sometable.c.column.like("%foobar%"))
Parameters
• other – expression to be compared
• escape – optional escape character, renders the ESCAPE keyword, e.g.:
somecolumn.like("foo/%bar", escape="/")
See also:
ColumnOperators.ilike()
match(other, **kwargs)
inherited from the match() method of ColumnOperators
Implements a database-specific ‘match’ operator.
match() attempts to resolve to a MATCH-like function or operator provided by the backend. Examples
include:
•Postgresql - renders x @@ to_tsquery(y)
•MySQL - renders MATCH (x) AGAINST (y IN BOOLEAN MODE)
•Oracle - renders CONTAINS(x, y)
•other backends may provide special implementations.
•Backends without any special implementation will emit the operator as “MATCH”. This is compatible with SQlite, for example.
notilike(other, escape=None)
inherited from the notilike() method of ColumnOperators
implement the NOT ILIKE operator.
This is equivalent to using negation with ColumnOperators.ilike(), i.e. ~x.ilike(y).
New in version 0.8.
See also:
ColumnOperators.ilike()
notin_(other)
inherited from the notin_() method of ColumnOperators
implement the NOT IN operator.
This is equivalent to using negation with ColumnOperators.in_(), i.e. ~x.in_(y).
New in version 0.8.
See also:
376
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
ColumnOperators.in_()
notlike(other, escape=None)
inherited from the notlike() method of ColumnOperators
implement the NOT LIKE operator.
This is equivalent to using negation with ColumnOperators.like(), i.e. ~x.like(y).
New in version 0.8.
See also:
ColumnOperators.like()
nullsfirst()
inherited from the nullsfirst() method of ColumnOperators
Produce a nullsfirst() clause against the parent object.
nullslast()
inherited from the nullslast() method of ColumnOperators
Produce a nullslast() clause against the parent object.
op(opstring, precedence=0)
inherited from the op() method of Operators
produce a generic operator function.
e.g.:
somecolumn.op("*")(5)
produces:
somecolumn * 5
This function can also be used to make bitwise operators explicit. For example:
somecolumn.op('&')(0xff)
is a bitwise AND of the value in somecolumn.
Parameters
• operator – a string which will be output as the infix operator between this element
and the expression passed to the generated function.
• precedence – precedence to apply to the operator, when parenthesizing expressions. A lower number will cause the expression to be parenthesized when applied
against another operator with higher precedence. The default value of 0 is lower than
all operators except for the comma (,) and AS operators. A value of 100 will be
higher or equal to all operators, and -100 will be lower than or equal to all operators.
New in version 0.8: - added the ‘precedence’ argument.
See also:
Redefining and Creating New Operators
operate(op, *other, **kwargs)
params(*optionaldict, **kwargs)
inherited from the params() method of ClauseElement
Return a copy with bindparam() elements replaced.
3.2. SQL Statements and Expressions API
377
SQLAlchemy Documentation, Release 0.8.7
Returns a copy of this ClauseElement with bindparam() elements replaced with values taken from the
given dictionary:
>>> clause = column('x') + bindparam('foo')
>>> print clause.compile().params
{'foo':None}
>>> print clause.params({'foo':7}).compile().params
{'foo':7}
primary_key = False
proxy_set
quote = None
reverse_operate(op, other, **kwargs)
self_group(against=None)
inherited from the self_group() method of ClauseElement
Apply a ‘grouping’ to this ClauseElement.
This method is overridden by subclasses to return a “grouping” construct, i.e. parenthesis. In particular
it’s used by “binary” expressions to provide a grouping around themselves when placed into a larger expression, as well as by select() constructs when placed into the FROM clause of another select().
(Note that subqueries should be normally created using the Select.alias() method, as many platforms require nested SELECT statements to be named).
As expressions are composed together, the application of self_group() is automatic - end-user code
should never need to use this method directly. Note that SQLAlchemy’s clause constructs take operator
precedence into account - so parenthesis might not be needed, for example, in an expression like x OR
(y AND z) - AND takes precedence over OR.
The base self_group() method of ClauseElement just returns self.
shares_lineage(othercolumn)
Return True if the given ColumnElement has a common ancestor to this ColumnElement.
startswith(other, **kwargs)
inherited from the startswith() method of ColumnOperators
Implement the startwith operator.
In a column context, produces the clause LIKE ’<other>%’
supports_execution = False
timetuple = None
type
unique_params(*optionaldict, **kwargs)
inherited from the unique_params() method of ClauseElement
Return a copy with bindparam() elements replaced.
Same functionality as params(), except adds unique=True to affected bind parameters so that multiple
statements can be used.
class sqlalchemy.sql.operators.ColumnOperators
Bases: sqlalchemy.sql.operators.Operators
Defines boolean, comparison, and other operators for ColumnElement expressions.
378
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
By default, all methods call down to operate() or reverse_operate(), passing in the appropriate
operator function from the Python builtin operator module or a SQLAlchemy-specific operator function
from sqlalchemy.expression.operators. For example the __eq__ function:
def __eq__(self, other):
return self.operate(operators.eq, other)
Where operators.eq is essentially:
def eq(a, b):
return a == b
The core column expression unit ColumnElement overrides Operators.operate() and others to return
further ColumnElement constructs, so that the == operation above is replaced by a clause construct.
See also:
Redefining and Creating New Operators
TypeEngine.comparator_factory
ColumnOperators
PropComparator
__add__(other)
Implement the + operator.
In a column context, produces the clause a + b if the parent object has non-string affinity.
If the parent object has a string affinity, produces the concatenation operator, a || b - see
ColumnOperators.concat().
__and__(other)
inherited from the __and__() method of Operators
Implement the & operator.
When used with SQL expressions, results in an AND operation, equivalent to and_(), that is:
a & b
is equivalent to:
from sqlalchemy import and_
and_(a, b)
Care should be taken when using & regarding operator precedence; the & operator has the highest precedence. The operands should be enclosed in parenthesis if they contain further sub expressions:
(a == 2) & (b == 4)
__delattr__
inherited from the __delattr__ attribute of object
x.__delattr__(‘name’) <==> del x.name
__div__(other)
Implement the / operator.
In a column context, produces the clause a / b.
__eq__(other)
Implement the == operator.
In a column context, produces the clause a = b. If the target is None, produces a IS NULL.
3.2. SQL Statements and Expressions API
379
SQLAlchemy Documentation, Release 0.8.7
__format__()
inherited from the __format__() method of object
default object formatter
__ge__(other)
Implement the >= operator.
In a column context, produces the clause a >= b.
__getattribute__
inherited from the __getattribute__ attribute of object
x.__getattribute__(‘name’) <==> x.name
__getitem__(index)
Implement the [] operator.
This can be used by some database-specific types such as Postgresql ARRAY and HSTORE.
__gt__(other)
Implement the > operator.
In a column context, produces the clause a > b.
__hash__
__init__
inherited from the __init__ attribute of object
x.__init__(...) initializes x; see help(type(x)) for signature
__invert__()
inherited from the __invert__() method of Operators
Implement the ~ operator.
When used with SQL expressions, results in a NOT operation, equivalent to not_(), that is:
~a
is equivalent to:
from sqlalchemy import not_
not_(a)
__le__(other)
Implement the <= operator.
In a column context, produces the clause a <= b.
__lshift__(other)
implement the << operator.
Not used by SQLAlchemy core, this is provided for custom operator systems which want to use << as an
extension point.
__lt__(other)
Implement the < operator.
In a column context, produces the clause a < b.
__mod__(other)
Implement the % operator.
In a column context, produces the clause a % b.
380
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
__mul__(other)
Implement the * operator.
In a column context, produces the clause a * b.
__ne__(other)
Implement the != operator.
In a column context, produces the clause a != b. If the target is None, produces a IS NOT NULL.
__neg__()
Implement the - operator.
In a column context, produces the clause -a.
__new__(S, ...) → a new object with type S, a subtype of T
inherited from the __new__() method of object
__or__(other)
inherited from the __or__() method of Operators
Implement the | operator.
When used with SQL expressions, results in an OR operation, equivalent to or_(), that is:
a | b
is equivalent to:
from sqlalchemy import or_
or_(a, b)
Care should be taken when using | regarding operator precedence; the | operator has the highest precedence. The operands should be enclosed in parenthesis if they contain further sub expressions:
(a == 2) | (b == 4)
__radd__(other)
Implement the + operator in reverse.
See ColumnOperators.__add__().
__rdiv__(other)
Implement the / operator in reverse.
See ColumnOperators.__div__().
__reduce__()
inherited from the __reduce__() method of object
helper for pickle
__reduce_ex__()
inherited from the __reduce_ex__() method of object
helper for pickle
__repr__
inherited from the __repr__ attribute of object
__rmul__(other)
Implement the * operator in reverse.
See ColumnOperators.__mul__().
3.2. SQL Statements and Expressions API
381
SQLAlchemy Documentation, Release 0.8.7
__rshift__(other)
implement the >> operator.
Not used by SQLAlchemy core, this is provided for custom operator systems which want to use >> as an
extension point.
__rsub__(other)
Implement the - operator in reverse.
See ColumnOperators.__sub__().
__rtruediv__(other)
Implement the // operator in reverse.
See ColumnOperators.__truediv__().
__setattr__
inherited from the __setattr__ attribute of object
x.__setattr__(‘name’, value) <==> x.name = value
__sizeof__() → int
inherited from the __sizeof__() method of object
size of object in memory, in bytes
__str__
inherited from the __str__ attribute of object
__sub__(other)
Implement the - operator.
In a column context, produces the clause a - b.
__subclasshook__()
inherited from the __subclasshook__() method of object
Abstract classes can override this to customize issubclass().
This is invoked early on by abc.ABCMeta.__subclasscheck__(). It should return True, False or NotImplemented. If it returns NotImplemented, the normal algorithm is used. Otherwise, it overrides the normal
algorithm (and the outcome is cached).
__truediv__(other)
Implement the // operator.
In a column context, produces the clause a / b.
__weakref__
inherited from the __weakref__ attribute of Operators
list of weak references to the object (if defined)
asc()
Produce a asc() clause against the parent object.
between(cleft, cright)
Produce a between() clause against the parent object, given the lower and upper range.
collate(collation)
Produce a collate() clause against the parent object, given the collation string.
concat(other)
Implement the ‘concat’ operator.
In a column context, produces the clause a || b, or uses the concat() operator on MySQL.
382
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
contains(other, **kwargs)
Implement the ‘contains’ operator.
In a column context, produces the clause LIKE ’%<other>%’
desc()
Produce a desc() clause against the parent object.
distinct()
Produce a distinct() clause against the parent object.
endswith(other, **kwargs)
Implement the ‘endswith’ operator.
In a column context, produces the clause LIKE ’%<other>’
ilike(other, escape=None)
Implement the ilike operator.
In a column context, produces the clause a ILIKE other.
E.g.:
select([sometable]).where(sometable.c.column.ilike("%foobar%"))
Parameters
• other – expression to be compared
• escape – optional escape character, renders the ESCAPE keyword, e.g.:
somecolumn.ilike("foo/%bar", escape="/")
See also:
ColumnOperators.like()
in_(other)
Implement the in operator.
In a column context, produces the clause a IN other. “other” may be a tuple/list of column expressions, or a select() construct.
is_(other)
Implement the IS operator.
Normally, IS is generated automatically when comparing to a value of None, which resolves to NULL.
However, explicit usage of IS may be desirable if comparing to boolean values on certain platforms.
New in version 0.7.9.
See also:
ColumnOperators.isnot()
isnot(other)
Implement the IS NOT operator.
Normally, IS NOT is generated automatically when comparing to a value of None, which resolves to
NULL. However, explicit usage of IS NOT may be desirable if comparing to boolean values on certain
platforms.
New in version 0.7.9.
See also:
3.2. SQL Statements and Expressions API
383
SQLAlchemy Documentation, Release 0.8.7
ColumnOperators.is_()
like(other, escape=None)
Implement the like operator.
In a column context, produces the clause a LIKE other.
E.g.:
select([sometable]).where(sometable.c.column.like("%foobar%"))
Parameters
• other – expression to be compared
• escape – optional escape character, renders the ESCAPE keyword, e.g.:
somecolumn.like("foo/%bar", escape="/")
See also:
ColumnOperators.ilike()
match(other, **kwargs)
Implements a database-specific ‘match’ operator.
match() attempts to resolve to a MATCH-like function or operator provided by the backend. Examples
include:
•Postgresql - renders x @@ to_tsquery(y)
•MySQL - renders MATCH (x) AGAINST (y IN BOOLEAN MODE)
•Oracle - renders CONTAINS(x, y)
•other backends may provide special implementations.
•Backends without any special implementation will emit the operator as “MATCH”. This is compatible with SQlite, for example.
notilike(other, escape=None)
implement the NOT ILIKE operator.
This is equivalent to using negation with ColumnOperators.ilike(), i.e. ~x.ilike(y).
New in version 0.8.
See also:
ColumnOperators.ilike()
notin_(other)
implement the NOT IN operator.
This is equivalent to using negation with ColumnOperators.in_(), i.e. ~x.in_(y).
New in version 0.8.
See also:
ColumnOperators.in_()
notlike(other, escape=None)
implement the NOT LIKE operator.
This is equivalent to using negation with ColumnOperators.like(), i.e. ~x.like(y).
384
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
New in version 0.8.
See also:
ColumnOperators.like()
nullsfirst()
Produce a nullsfirst() clause against the parent object.
nullslast()
Produce a nullslast() clause against the parent object.
op(opstring, precedence=0)
inherited from the op() method of Operators
produce a generic operator function.
e.g.:
somecolumn.op("*")(5)
produces:
somecolumn * 5
This function can also be used to make bitwise operators explicit. For example:
somecolumn.op('&')(0xff)
is a bitwise AND of the value in somecolumn.
Parameters
• operator – a string which will be output as the infix operator between this element
and the expression passed to the generated function.
• precedence – precedence to apply to the operator, when parenthesizing expressions. A lower number will cause the expression to be parenthesized when applied
against another operator with higher precedence. The default value of 0 is lower than
all operators except for the comma (,) and AS operators. A value of 100 will be
higher or equal to all operators, and -100 will be lower than or equal to all operators.
New in version 0.8: - added the ‘precedence’ argument.
See also:
Redefining and Creating New Operators
operate(op, *other, **kwargs)
inherited from the operate() method of Operators
Operate on an argument.
This is the lowest level of operation, raises NotImplementedError by default.
Overriding this on a subclass can allow common behavior to be applied to all operations. For example,
overriding ColumnOperators to apply func.lower() to the left and right side:
class MyComparator(ColumnOperators):
def operate(self, op, other):
return op(func.lower(self), func.lower(other))
Parameters
• op – Operator callable.
3.2. SQL Statements and Expressions API
385
SQLAlchemy Documentation, Release 0.8.7
• *other – the ‘other’ side of the operation. Will be a single scalar for most operations.
• **kwargs – modifiers. These may be passed by special operators such as
ColumnOperators.contains().
reverse_operate(op, other, **kwargs)
inherited from the reverse_operate() method of Operators
Reverse operate on an argument.
Usage is the same as operate().
startswith(other, **kwargs)
Implement the startwith operator.
In a column context, produces the clause LIKE ’<other>%’
timetuple = None
Hack, allows datetime objects to be compared on the LHS.
class sqlalchemy.sql.expression.Extract(field, expr, **kwargs)
Bases: sqlalchemy.sql.expression.ColumnElement
class sqlalchemy.sql.expression.False_
Bases: sqlalchemy.sql.expression.ColumnElement
Represent the false keyword in a SQL statement.
Public constructor is the false() function.
class sqlalchemy.sql.expression.Label(name, element, type_=None)
Bases: sqlalchemy.sql.expression.ColumnElement
Represents a column label (AS).
Represent a label, as typically applied to any column-level element using the AS sql keyword.
This object is constructed from the label() module level function as well as the label() method available
on all ColumnElement subclasses.
class sqlalchemy.sql.expression.Null
Bases: sqlalchemy.sql.expression.ColumnElement
Represent the NULL keyword in a SQL statement.
Public constructor is the null() function.
class sqlalchemy.sql.expression.Over(func, partition_by=None, order_by=None)
Bases: sqlalchemy.sql.expression.ColumnElement
Represent an OVER clause.
This is a special operator against a so-called “window” function, as well as any aggregate function, which
produces results relative to the result set itself. It’s supported only by certain database backends.
class sqlalchemy.sql.expression.TextClause(text=’‘,
bind=None,
bindparams=None,
typemap=None, autocommit=None)
Bases: sqlalchemy.sql.expression.Executable, sqlalchemy.sql.expression.ClauseElement
Represent a literal SQL text fragment.
Public constructor is the text() function.
386
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
class sqlalchemy.sql.expression.Tuple(*clauses, **kw)
Bases: sqlalchemy.sql.expression.ClauseList, sqlalchemy.sql.expression.ColumnElement
Represent a SQL tuple.
__init__(*clauses, **kw)
Return a Tuple.
Main usage is to produce a composite IN construct:
from sqlalchemy import tuple_
tuple_(table.c.col1, table.c.col2).in_(
[(1, 2), (5, 12), (10, 19)]
)
Warning: The composite IN construct is not supported by all backends, and is currently known
to work on Postgresql and MySQL, but not SQLite. Unsupported backends will raise a subclass of
DBAPIError when such an expression is invoked.
class sqlalchemy.sql.expression.True_
Bases: sqlalchemy.sql.expression.ColumnElement
Represent the true keyword in a SQL statement.
Public constructor is the true() function.
class sqlalchemy.sql.operators.custom_op(opstring, precedence=0)
Represent a ‘custom’ operator.
custom_op is normally instantitated when the ColumnOperators.op() method is used to create a custom
operator callable. The class can also be used directly when programmatically constructing expressions. E.g. to
represent the “factorial” operation:
from sqlalchemy.sql import UnaryExpression
from sqlalchemy.sql import operators
from sqlalchemy import Numeric
unary = UnaryExpression(table.c.somecolumn,
modifier=operators.custom_op("!"),
type_=Numeric)
class sqlalchemy.sql.operators.Operators
Base of comparison and logical operators.
Implements base methods operate() and reverse_operate(), as well as __and__(), __or__(),
__invert__().
Usually is used via its most common subclass ColumnOperators.
__and__(other)
Implement the & operator.
When used with SQL expressions, results in an AND operation, equivalent to and_(), that is:
a & b
is equivalent to:
from sqlalchemy import and_
and_(a, b)
3.2. SQL Statements and Expressions API
387
SQLAlchemy Documentation, Release 0.8.7
Care should be taken when using & regarding operator precedence; the & operator has the highest precedence. The operands should be enclosed in parenthesis if they contain further sub expressions:
(a == 2) & (b == 4)
__invert__()
Implement the ~ operator.
When used with SQL expressions, results in a NOT operation, equivalent to not_(), that is:
~a
is equivalent to:
from sqlalchemy import not_
not_(a)
__or__(other)
Implement the | operator.
When used with SQL expressions, results in an OR operation, equivalent to or_(), that is:
a | b
is equivalent to:
from sqlalchemy import or_
or_(a, b)
Care should be taken when using | regarding operator precedence; the | operator has the highest precedence. The operands should be enclosed in parenthesis if they contain further sub expressions:
(a == 2) | (b == 4)
__weakref__
list of weak references to the object (if defined)
op(opstring, precedence=0)
produce a generic operator function.
e.g.:
somecolumn.op("*")(5)
produces:
somecolumn * 5
This function can also be used to make bitwise operators explicit. For example:
somecolumn.op('&')(0xff)
is a bitwise AND of the value in somecolumn.
Parameters
• operator – a string which will be output as the infix operator between this element
and the expression passed to the generated function.
• precedence – precedence to apply to the operator, when parenthesizing expressions. A lower number will cause the expression to be parenthesized when applied
against another operator with higher precedence. The default value of 0 is lower than
388
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
all operators except for the comma (,) and AS operators. A value of 100 will be
higher or equal to all operators, and -100 will be lower than or equal to all operators.
New in version 0.8: - added the ‘precedence’ argument.
See also:
Redefining and Creating New Operators
operate(op, *other, **kwargs)
Operate on an argument.
This is the lowest level of operation, raises NotImplementedError by default.
Overriding this on a subclass can allow common behavior to be applied to all operations. For example,
overriding ColumnOperators to apply func.lower() to the left and right side:
class MyComparator(ColumnOperators):
def operate(self, op, other):
return op(func.lower(self), func.lower(other))
Parameters
• op – Operator callable.
• *other – the ‘other’ side of the operation. Will be a single scalar for most operations.
• **kwargs – modifiers. These may be passed by special operators such as
ColumnOperators.contains().
reverse_operate(op, other, **kwargs)
Reverse operate on an argument.
Usage is the same as operate().
class sqlalchemy.sql.expression.UnaryExpression(element, operator=None, modifier=None,
type_=None, negate=None)
Bases: sqlalchemy.sql.expression.ColumnElement
Define a ‘unary’ expression.
A unary expression has a single column expression and an operator. The operator can be placed on the left
(where it is called the ‘operator’) or right (where it is called the ‘modifier’) of the column expression.
compare(other, **kw)
Compare this UnaryExpression against the given ClauseElement.
3.2.2 Selectables, Tables, FROM objects
The term “selectable” refers to any object that rows can be selected from; in SQLAlchemy, these objects descend from
FromClause and their distinguishing feature is their FromClause.c attribute, which is a namespace of all the
columns contained within the FROM clause (these elements are themselves ColumnElement subclasses).
sqlalchemy.sql.expression.alias(selectable, name=None)
Return an Alias object.
An Alias represents any FromClause with an alternate name assigned within SQL, typically using the AS
clause when generated, e.g. SELECT * FROM table AS aliasname.
Similar functionality is available via the alias() method available on all FromClause subclasses.
3.2. SQL Statements and Expressions API
389
SQLAlchemy Documentation, Release 0.8.7
When an Alias is created from a Table object, this has the effect of the table being rendered as tablename
AS aliasname in a SELECT statement.
For select() objects, the effect is that of creating a named subquery, i.e.
aliasname.
(select ...)
AS
The name parameter is optional, and provides the name to use in the rendered SQL. If blank, an “anonymous”
name will be deterministically generated at compile time. Deterministic means the name is guaranteed to be
unique against other constructs used in the same statement, and will also be the same name for each successive
compilation of the same statement object.
Parameters
• selectable – any FromClause subclass, such as a table, select statement, etc.
• name – string name to be assigned as the alias. If None, a name will be deterministically
generated at compile time.
sqlalchemy.sql.expression.except_(*selects, **kwargs)
Return an EXCEPT of multiple selectables.
The returned object is an instance of CompoundSelect.
*selects a list of Select instances.
**kwargs available keyword arguments are the same as those of select().
sqlalchemy.sql.expression.except_all(*selects, **kwargs)
Return an EXCEPT ALL of multiple selectables.
The returned object is an instance of CompoundSelect.
*selects a list of Select instances.
**kwargs available keyword arguments are the same as those of select().
sqlalchemy.sql.expression.exists(*args, **kwargs)
Return an EXISTS clause as applied to a Select object.
Calling styles are of the following forms:
# use on an existing select()
s = select([table.c.col1]).where(table.c.col2==5)
s = exists(s)
# construct a select() at once
exists(['*'], **select_arguments).where(criterion)
# columns argument is optional, generates "EXISTS (SELECT *)"
# by default.
exists().where(table.c.col2==5)
sqlalchemy.sql.expression.intersect(*selects, **kwargs)
Return an INTERSECT of multiple selectables.
The returned object is an instance of CompoundSelect.
*selects a list of Select instances.
**kwargs available keyword arguments are the same as those of select().
sqlalchemy.sql.expression.intersect_all(*selects, **kwargs)
Return an INTERSECT ALL of multiple selectables.
The returned object is an instance of CompoundSelect.
390
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
*selects a list of Select instances.
**kwargs available keyword arguments are the same as those of select().
sqlalchemy.sql.expression.join(left, right, onclause=None, isouter=False)
Produce a Join object, given two FromClause expressions.
E.g.:
j = join(user_table, address_table, user_table.c.id == address_table.c.user_id)
stmt = select([user_table]).select_from(j)
would emit SQL along the lines of:
SELECT user.id, user.name FROM user
JOIN address ON user.id = address.user_id
Similar functionality is available given any FromClause object (e.g.
FromClause.join() method.
such as a Table) using the
Parameters
• left – The left side of the join.
• right – the right side of the join; this is any FromClause object such as a Table
object, and may also be a selectable-compatible object such as an ORM-mapped class.
• onclause – a SQL expression representing the ON clause of the join. If left at None,
FromClause.join() will attempt to join the two tables based on a foreign key relationship.
• isouter – if True, render a LEFT OUTER JOIN, instead of JOIN.
See also:
FromClause.join() - method form, based on a given left side
Join - the type of object produced
sqlalchemy.sql.expression.outerjoin(left, right, onclause=None)
Return an OUTER JOIN clause element.
The returned object is an instance of Join.
Similar functionality is also available via the outerjoin() method on any FromClause.
Parameters
• left – The left side of the join.
• right – The right side of the join.
• onclause – Optional criterion for the ON clause, is derived from foreign key relationships established between left and right otherwise.
To chain joins together, use the FromClause.join() or FromClause.outerjoin() methods on the
resulting Join object.
sqlalchemy.sql.expression.select(columns=None,
**kwargs)
Returns a SELECT clause element.
whereclause=None,
from_obj=[],
Similar functionality is also available via the select() method on any FromClause.
The returned object is an instance of Select.
3.2. SQL Statements and Expressions API
391
SQLAlchemy Documentation, Release 0.8.7
All arguments which accept ClauseElement arguments also accept string arguments, which will be converted as appropriate into either text() or literal_column() constructs.
See also:
Selecting - Core Tutorial description of select().
Parameters
• columns – A list of ClauseElement objects, typically ColumnElement objects or
subclasses, which will form the columns clause of the resulting statement. For all members which are instances of Selectable, the individual ColumnElement members
of the Selectable will be added individually to the columns clause. For example,
specifying a Table instance will result in all the contained Column objects within to be
added to the columns clause.
This argument is not present on the form of select() available on Table.
• whereclause – A ClauseElement expression which will be used to form the
WHERE clause.
• from_obj – A list of ClauseElement objects which will be added to the FROM
clause of the resulting statement. Note that “from” objects are automatically located
within the columns and whereclause ClauseElements. Use this parameter to explicitly
specify “from” objects which are not automatically locatable. This could include Table
objects that aren’t otherwise present, or Join objects whose presence will supercede that
of the Table objects already located in the other clauses.
• autocommit – Deprecated. Use .execution_options(autocommit=<True|False>) to set
the autocommit option.
• bind=None – an Engine or Connection instance to which the resulting Select
object will be bound. The Select object will otherwise automatically bind to whatever Connectable instances can be located within its contained ClauseElement
members.
• correlate=True – indicates that this Select object should have its contained
FromClause elements “correlated” to an enclosing Select object. This means that
any ClauseElement instance within the “froms” collection of this Select which is
also present in the “froms” collection of an enclosing select will not be rendered in the
FROM clause of this select statement.
• distinct=False – when True, applies a DISTINCT qualifier to the columns clause
of the resulting statement.
The boolean argument may also be a column expression or list of column expressions this is a special calling form which is understood by the Postgresql dialect to render the
DISTINCT ON (<columns>) syntax.
distinct is also available via the distinct() generative method.
• for_update=False – when True, applies FOR UPDATE to the end of the resulting
statement.
Certain database dialects also support alternate values for this parameter:
– With the MySQL dialect, the value "read" translates to LOCK IN SHARE MODE.
– With the Oracle and Postgresql dialects, the value "nowait" translates to FOR
UPDATE NOWAIT.
392
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
– With the Postgresql dialect, the values “read” and "read_nowait" translate to
FOR SHARE and FOR SHARE NOWAIT, respectively.
New in version 0.7.7.
• group_by – a list of ClauseElement objects which will comprise the GROUP BY
clause of the resulting select.
• having – a ClauseElement that will comprise the HAVING clause of the resulting
select when GROUP BY is used.
• limit=None – a numerical value which usually compiles to a LIMIT expression in
the resulting select. Databases that don’t support LIMIT will attempt to provide similar
functionality.
• offset=None – a numeric value which usually compiles to an OFFSET expression in
the resulting select. Databases that don’t support OFFSET will attempt to provide similar
functionality.
• order_by – a scalar or list of ClauseElement objects which will comprise the
ORDER BY clause of the resulting select.
• use_labels=False – when True, the statement will be generated using labels for
each column in the columns clause, which qualify each column with its parent table’s
(or aliases) name so that name conflicts between columns in different tables don’t occur.
The format of the label is <tablename>_<column>. The “c” collection of the resulting
Select object will use these names as well for targeting column members.
use_labels is also available via the apply_labels() generative method.
sqlalchemy.sql.expression.subquery(alias, *args, **kwargs)
Return an Alias object derived from a Select.
name alias name
*args, **kwargs
all other arguments are delivered to the select() function.
sqlalchemy.sql.expression.table(name, *columns)
Represent a textual table clause.
The object returned is an instance of TableClause, which represents the “syntactical” portion of the schemalevel Table object. It may be used to construct lightweight table constructs.
Note that the table() function is not part of the sqlalchemy namespace. It must be imported from the
sql package:
from sqlalchemy.sql import table, column
Parameters
• name – Name of the table.
• columns – A collection of column() constructs.
See TableClause for further examples.
sqlalchemy.sql.expression.union(*selects, **kwargs)
Return a UNION of multiple selectables.
The returned object is an instance of CompoundSelect.
A similar union() method is available on all FromClause subclasses.
3.2. SQL Statements and Expressions API
393
SQLAlchemy Documentation, Release 0.8.7
*selects a list of Select instances.
**kwargs available keyword arguments are the same as those of select().
sqlalchemy.sql.expression.union_all(*selects, **kwargs)
Return a UNION ALL of multiple selectables.
The returned object is an instance of CompoundSelect.
A similar union_all() method is available on all FromClause subclasses.
*selects a list of Select instances.
**kwargs available keyword arguments are the same as those of select().
class sqlalchemy.sql.expression.Alias(selectable, name=None)
Bases: sqlalchemy.sql.expression.FromClause
Represents an table or selectable alias (AS).
Represents an alias, as typically applied to any table or sub-select within a SQL statement using the AS keyword
(or without the keyword on certain databases such as Oracle).
This object is constructed from the alias() module level function as well as the FromClause.alias()
method available on all FromClause subclasses.
alias(name=None)
inherited from the alias() method of FromClause
return an alias of this FromClause.
This is shorthand for calling:
from sqlalchemy import alias
a = alias(self, name=name)
See alias() for details.
c
inherited from the c attribute of FromClause
An alias for the columns attribute.
columns
inherited from the columns attribute of FromClause
A named-based collection of ColumnElement objects maintained by this FromClause.
The columns, or c collection, is the gateway to the construction of SQL expressions using table-bound
or other selectable-bound columns:
select([mytable]).where(mytable.c.somecolumn == 5)
compare(other, **kw)
inherited from the compare() method of ClauseElement
Compare this ClauseElement to the given ClauseElement.
Subclasses should override the default behavior, which is a straight identity comparison.
**kw are arguments consumed by subclass compare() methods and may be used to modify the criteria for
comparison. (see ColumnElement)
compile(bind=None, dialect=None, **kw)
inherited from the compile() method of ClauseElement
Compile this SQL expression.
394
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
The return value is a Compiled object. Calling str() or unicode() on the returned value will yield
a string representation of the result. The Compiled object also can return a dictionary of bind parameter
names and values using the params accessor.
Parameters
• bind – An Engine or Connection from which a Compiled will be acquired.
This argument takes precedence over this ClauseElement‘s bound engine, if any.
• column_keys – Used for INSERT and UPDATE statements, a list of column names
which should be present in the VALUES clause of the compiled statement. If None,
all columns from the target table object are rendered.
• dialect – A Dialect instance from which a Compiled will be acquired. This argument takes precedence over the bind argument as well as this
ClauseElement‘s bound engine, if any.
• inline – Used for INSERT statements, for a dialect which does not support inline
retrieval of newly generated primary key columns, will force the expression used to
create the new primary key value to be rendered inline within the INSERT statement’s
VALUES clause. This typically refers to Sequence execution but may also refer to any
server-side default generation function associated with a primary key Column.
correspond_on_equivalents(column, equivalents)
inherited from the correspond_on_equivalents() method of FromClause
Return corresponding_column for the given column, or if None search for a match in the given dictionary.
corresponding_column(column, require_embedded=False)
inherited from the corresponding_column() method of FromClause
Given a ColumnElement, return the exported ColumnElement object from this Selectable
which corresponds to that original Column via a common ancestor column.
Parameters
• column – the target ColumnElement to be matched
• require_embedded – only return corresponding columns for
the given ColumnElement, if the given ColumnElement is actually present within a sub-element of
this FromClause. Normally the column will match if it merely shares a common ancestor with one of
the exported columns of this FromClause.
count(whereclause=None, **params)
inherited from the count() method of FromClause
return a SELECT COUNT generated against this FromClause.
foreign_keys
inherited from the foreign_keys attribute of FromClause
Return the collection of ForeignKey objects which this FromClause references.
join(right, onclause=None, isouter=False)
inherited from the join() method of FromClause
Return a Join from this FromClause to another FromClause.
E.g.:
from sqlalchemy import join
j = user_table.join(address_table,
3.2. SQL Statements and Expressions API
395
SQLAlchemy Documentation, Release 0.8.7
user_table.c.id == address_table.c.user_id)
stmt = select([user_table]).select_from(j)
would emit SQL along the lines of:
SELECT user.id, user.name FROM user
JOIN address ON user.id = address.user_id
Parameters
• right – the right side of the join; this is any FromClause object such as a Table
object, and may also be a selectable-compatible object such as an ORM-mapped class.
• onclause – a SQL expression representing the ON clause of the join. If left at
None, FromClause.join() will attempt to join the two tables based on a foreign
key relationship.
• isouter – if True, render a LEFT OUTER JOIN, instead of JOIN.
See also:
join() - standalone function
Join - the type of object produced
outerjoin(right, onclause=None)
inherited from the outerjoin() method of FromClause
Return a Join from this FromClause to another FromClause, with the “isouter” flag set to True.
E.g.:
from sqlalchemy import outerjoin
j = user_table.outerjoin(address_table,
user_table.c.id == address_table.c.user_id)
The above is equivalent to:
j = user_table.join(address_table,
user_table.c.id == address_table.c.user_id, isouter=True)
Parameters
• right – the right side of the join; this is any FromClause object such as a Table
object, and may also be a selectable-compatible object such as an ORM-mapped class.
• onclause – a SQL expression representing the ON clause of the join. If left at
None, FromClause.join() will attempt to join the two tables based on a foreign
key relationship.
See also:
FromClause.join()
Join
params(*optionaldict, **kwargs)
inherited from the params() method of ClauseElement
Return a copy with bindparam() elements replaced.
396
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
Returns a copy of this ClauseElement with bindparam() elements replaced with values taken from the
given dictionary:
>>> clause = column('x') + bindparam('foo')
>>> print clause.compile().params
{'foo':None}
>>> print clause.params({'foo':7}).compile().params
{'foo':7}
primary_key
inherited from the primary_key attribute of FromClause
Return the collection of Column objects which comprise the primary key of this FromClause.
replace_selectable(old, alias)
inherited from the replace_selectable() method of FromClause
replace all occurrences of FromClause ‘old’ with the given Alias object, returning a copy of this
FromClause.
select(whereclause=None, **params)
inherited from the select() method of FromClause
return a SELECT of this FromClause.
See also:
select() - general purpose method which allows for arbitrary column lists.
self_group(against=None)
inherited from the self_group() method of ClauseElement
Apply a ‘grouping’ to this ClauseElement.
This method is overridden by subclasses to return a “grouping” construct, i.e. parenthesis. In particular
it’s used by “binary” expressions to provide a grouping around themselves when placed into a larger expression, as well as by select() constructs when placed into the FROM clause of another select().
(Note that subqueries should be normally created using the Select.alias() method, as many platforms require nested SELECT statements to be named).
As expressions are composed together, the application of self_group() is automatic - end-user code
should never need to use this method directly. Note that SQLAlchemy’s clause constructs take operator
precedence into account - so parenthesis might not be needed, for example, in an expression like x OR
(y AND z) - AND takes precedence over OR.
The base self_group() method of ClauseElement just returns self.
unique_params(*optionaldict, **kwargs)
inherited from the unique_params() method of ClauseElement
Return a copy with bindparam() elements replaced.
Same functionality as params(), except adds unique=True to affected bind parameters so that multiple
statements can be used.
class sqlalchemy.sql.expression.CompoundSelect(keyword, *selects, **kwargs)
Bases: sqlalchemy.sql.expression.SelectBase
Forms the basis of UNION, UNION ALL, and other SELECT-based set operations.
See also:
union()
union_all()
3.2. SQL Statements and Expressions API
397
SQLAlchemy Documentation, Release 0.8.7
intersect()
intersect_all()
except()
except_all()
alias(name=None)
inherited from the alias() method of FromClause
return an alias of this FromClause.
This is shorthand for calling:
from sqlalchemy import alias
a = alias(self, name=name)
See alias() for details.
append_group_by(*clauses)
inherited from the append_group_by() method of SelectBase
Append the given GROUP BY criterion applied to this selectable.
The criterion will be appended to any pre-existing GROUP BY criterion.
This is an in-place mutation method; the group_by() method is preferred, as it provides standard
method chaining.
append_order_by(*clauses)
inherited from the append_order_by() method of SelectBase
Append the given ORDER BY criterion applied to this selectable.
The criterion will be appended to any pre-existing ORDER BY criterion.
This is an in-place mutation method; the order_by() method is preferred, as it provides standard
method chaining.
apply_labels()
inherited from the apply_labels() method of SelectBase
return a new selectable with the ‘use_labels’ flag set to True.
This will result in column expressions being generated using labels against their table name, such as
“SELECT somecolumn AS tablename_somecolumn”. This allows selectables which contain multiple
FROM clauses to produce a unique set of column names regardless of name conflicts among the individual
FROM clauses.
as_scalar()
inherited from the as_scalar() method of SelectBase
return a ‘scalar’ representation of this selectable, which can be used as a column expression.
Typically, a select statement which has only one column in its columns clause is eligible to be used as a
scalar expression.
The returned object is an instance of ScalarSelect.
autocommit()
inherited from the autocommit() method of SelectBase
return a new selectable with the ‘autocommit’ flag set to
Deprecated
since
version
0.6:
autocommit()
is
Executable.execution_options() with the ‘autocommit’ flag.
398
deprecated.
Use
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
True.
c
inherited from the c attribute of FromClause
An alias for the columns attribute.
columns
inherited from the columns attribute of FromClause
A named-based collection of ColumnElement objects maintained by this FromClause.
The columns, or c collection, is the gateway to the construction of SQL expressions using table-bound
or other selectable-bound columns:
select([mytable]).where(mytable.c.somecolumn == 5)
compare(other, **kw)
inherited from the compare() method of ClauseElement
Compare this ClauseElement to the given ClauseElement.
Subclasses should override the default behavior, which is a straight identity comparison.
**kw are arguments consumed by subclass compare() methods and may be used to modify the criteria for
comparison. (see ColumnElement)
compile(bind=None, dialect=None, **kw)
inherited from the compile() method of ClauseElement
Compile this SQL expression.
The return value is a Compiled object. Calling str() or unicode() on the returned value will yield
a string representation of the result. The Compiled object also can return a dictionary of bind parameter
names and values using the params accessor.
Parameters
• bind – An Engine or Connection from which a Compiled will be acquired.
This argument takes precedence over this ClauseElement‘s bound engine, if any.
• column_keys – Used for INSERT and UPDATE statements, a list of column names
which should be present in the VALUES clause of the compiled statement. If None,
all columns from the target table object are rendered.
• dialect – A Dialect instance from which a Compiled will be acquired. This argument takes precedence over the bind argument as well as this
ClauseElement‘s bound engine, if any.
• inline – Used for INSERT statements, for a dialect which does not support inline
retrieval of newly generated primary key columns, will force the expression used to
create the new primary key value to be rendered inline within the INSERT statement’s
VALUES clause. This typically refers to Sequence execution but may also refer to any
server-side default generation function associated with a primary key Column.
correspond_on_equivalents(column, equivalents)
inherited from the correspond_on_equivalents() method of FromClause
Return corresponding_column for the given column, or if None search for a match in the given dictionary.
corresponding_column(column, require_embedded=False)
inherited from the corresponding_column() method of FromClause
Given a ColumnElement, return the exported ColumnElement object from this Selectable
which corresponds to that original Column via a common ancestor column.
3.2. SQL Statements and Expressions API
399
SQLAlchemy Documentation, Release 0.8.7
Parameters
• column – the target ColumnElement to be matched
• require_embedded – only return corresponding columns for
the given ColumnElement, if the given ColumnElement is actually present within a sub-element of
this FromClause. Normally the column will match if it merely shares a common ancestor with one of
the exported columns of this FromClause.
count(whereclause=None, **params)
inherited from the count() method of FromClause
return a SELECT COUNT generated against this FromClause.
cte(name=None, recursive=False)
inherited from the cte() method of SelectBase
Return a new CTE, or Common Table Expression instance.
Common table expressions are a SQL standard whereby SELECT statements can draw upon secondary
statements specified along with the primary statement, using a clause called “WITH”. Special semantics
regarding UNION can also be employed to allow “recursive” queries, where a SELECT statement can
draw upon the set of rows that have previously been selected.
SQLAlchemy detects CTE objects, which are treated similarly to Alias objects, as special elements to
be delivered to the FROM clause of the statement as well as to a WITH clause at the top of the statement.
New in version 0.7.6.
Parameters
• name – name given to the common table expression.
Like
_FromClause.alias(), the name can be left as None in which case an
anonymous symbol will be used at query compile time.
• recursive – if True, will render WITH RECURSIVE. A recursive common table
expression is intended to be used in conjunction with UNION ALL in order to derive
rows from those already selected.
The following examples illustrate two examples from
http://www.postgresql.org/docs/8.4/static/queries-with.html.
Postgresql’s
documentation
at
Example 1, non recursive:
from sqlalchemy import Table, Column, String, Integer, MetaData, \
select, func
metadata = MetaData()
orders = Table('orders', metadata,
Column('region', String),
Column('amount', Integer),
Column('product', String),
Column('quantity', Integer)
)
regional_sales = select([
orders.c.region,
func.sum(orders.c.amount).label('total_sales')
]).group_by(orders.c.region).cte("regional_sales")
400
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
top_regions = select([regional_sales.c.region]).\
where(
regional_sales.c.total_sales >
select([
func.sum(regional_sales.c.total_sales)/10
])
).cte("top_regions")
statement = select([
orders.c.region,
orders.c.product,
func.sum(orders.c.quantity).label("product_units"),
func.sum(orders.c.amount).label("product_sales")
]).where(orders.c.region.in_(
select([top_regions.c.region])
)).group_by(orders.c.region, orders.c.product)
result = conn.execute(statement).fetchall()
Example 2, WITH RECURSIVE:
from sqlalchemy import Table, Column, String, Integer, MetaData, \
select, func
metadata = MetaData()
parts = Table('parts', metadata,
Column('part', String),
Column('sub_part', String),
Column('quantity', Integer),
)
included_parts = select([
parts.c.sub_part,
parts.c.part,
parts.c.quantity]).\
where(parts.c.part=='our part').\
cte(recursive=True)
incl_alias = included_parts.alias()
parts_alias = parts.alias()
included_parts = included_parts.union_all(
select([
parts_alias.c.sub_part,
parts_alias.c.part,
parts_alias.c.quantity
]).
where(parts_alias.c.part==incl_alias.c.sub_part)
)
statement = select([
included_parts.c.sub_part,
func.sum(included_parts.c.quantity).
label('total_quantity')
]).\
group_by(included_parts.c.sub_part)
3.2. SQL Statements and Expressions API
401
SQLAlchemy Documentation, Release 0.8.7
result = conn.execute(statement).fetchall()
See also:
orm.query.Query.cte() - ORM version of SelectBase.cte().
description
inherited from the description attribute of FromClause
a brief description of this FromClause.
Used primarily for error message formatting.
execute(*multiparams, **params)
inherited from the execute() method of Executable
Compile and execute this Executable.
execution_options(**kw)
inherited from the execution_options() method of Executable
Set non-SQL options for the statement which take effect during execution.
Execution options can be set on a per-statement or per Connection basis. Additionally, the Engine
and ORM Query objects provide access to execution options which they in turn configure upon connections.
The execution_options() method is generative. A new instance of this statement is returned that
contains the options:
statement = select([table.c.x, table.c.y])
statement = statement.execution_options(autocommit=True)
Note that only a subset of possible execution options can be applied to a statement - these include “autocommit” and “stream_results”, but not “isolation_level” or “compiled_cache”. See
Connection.execution_options() for a full list of possible options.
See also:
Connection.execution_options()
Query.execution_options()
foreign_keys
inherited from the foreign_keys attribute of FromClause
Return the collection of ForeignKey objects which this FromClause references.
group_by(*clauses)
inherited from the group_by() method of SelectBase
return a new selectable with the given list of GROUP BY criterion applied.
The criterion will be appended to any pre-existing GROUP BY criterion.
join(right, onclause=None, isouter=False)
inherited from the join() method of FromClause
Return a Join from this FromClause to another FromClause.
E.g.:
from sqlalchemy import join
j = user_table.join(address_table,
402
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
user_table.c.id == address_table.c.user_id)
stmt = select([user_table]).select_from(j)
would emit SQL along the lines of:
SELECT user.id, user.name FROM user
JOIN address ON user.id = address.user_id
Parameters
• right – the right side of the join; this is any FromClause object such as a Table
object, and may also be a selectable-compatible object such as an ORM-mapped class.
• onclause – a SQL expression representing the ON clause of the join. If left at
None, FromClause.join() will attempt to join the two tables based on a foreign
key relationship.
• isouter – if True, render a LEFT OUTER JOIN, instead of JOIN.
See also:
join() - standalone function
Join - the type of object produced
label(name)
inherited from the label() method of SelectBase
return a ‘scalar’ representation of this selectable, embedded as a subquery with a label.
See also:
as_scalar().
limit(limit)
inherited from the limit() method of SelectBase
return a new selectable with the given LIMIT criterion applied.
offset(offset)
inherited from the offset() method of SelectBase
return a new selectable with the given OFFSET criterion applied.
order_by(*clauses)
inherited from the order_by() method of SelectBase
return a new selectable with the given list of ORDER BY criterion applied.
The criterion will be appended to any pre-existing ORDER BY criterion.
outerjoin(right, onclause=None)
inherited from the outerjoin() method of FromClause
Return a Join from this FromClause to another FromClause, with the “isouter” flag set to True.
E.g.:
from sqlalchemy import outerjoin
j = user_table.outerjoin(address_table,
user_table.c.id == address_table.c.user_id)
The above is equivalent to:
3.2. SQL Statements and Expressions API
403
SQLAlchemy Documentation, Release 0.8.7
j = user_table.join(address_table,
user_table.c.id == address_table.c.user_id, isouter=True)
Parameters
• right – the right side of the join; this is any FromClause object such as a Table
object, and may also be a selectable-compatible object such as an ORM-mapped class.
• onclause – a SQL expression representing the ON clause of the join. If left at
None, FromClause.join() will attempt to join the two tables based on a foreign
key relationship.
See also:
FromClause.join()
Join
params(*optionaldict, **kwargs)
inherited from the params() method of ClauseElement
Return a copy with bindparam() elements replaced.
Returns a copy of this ClauseElement with bindparam() elements replaced with values taken from the
given dictionary:
>>> clause = column('x') + bindparam('foo')
>>> print clause.compile().params
{'foo':None}
>>> print clause.params({'foo':7}).compile().params
{'foo':7}
primary_key
inherited from the primary_key attribute of FromClause
Return the collection of Column objects which comprise the primary key of this FromClause.
replace_selectable(old, alias)
inherited from the replace_selectable() method of FromClause
replace all occurrences of FromClause ‘old’ with the given Alias object, returning a copy of this
FromClause.
scalar(*multiparams, **params)
inherited from the scalar() method of Executable
Compile and execute this Executable, returning the result’s scalar representation.
select(whereclause=None, **params)
inherited from the select() method of FromClause
return a SELECT of this FromClause.
See also:
select() - general purpose method which allows for arbitrary column lists.
unique_params(*optionaldict, **kwargs)
inherited from the unique_params() method of ClauseElement
Return a copy with bindparam() elements replaced.
Same functionality as params(), except adds unique=True to affected bind parameters so that multiple
statements can be used.
404
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
class sqlalchemy.sql.expression.CTE(selectable, name=None, recursive=False, _cte_alias=None,
_restates=frozenset([]))
Bases: sqlalchemy.sql.expression.Alias
Represent a Common Table Expression.
The CTE object is obtained using the SelectBase.cte() method from any selectable. See that method for
complete examples.
New in version 0.7.6.
c
inherited from the c attribute of FromClause
An alias for the columns attribute.
columns
inherited from the columns attribute of FromClause
A named-based collection of ColumnElement objects maintained by this FromClause.
The columns, or c collection, is the gateway to the construction of SQL expressions using table-bound
or other selectable-bound columns:
select([mytable]).where(mytable.c.somecolumn == 5)
compare(other, **kw)
inherited from the compare() method of ClauseElement
Compare this ClauseElement to the given ClauseElement.
Subclasses should override the default behavior, which is a straight identity comparison.
**kw are arguments consumed by subclass compare() methods and may be used to modify the criteria for
comparison. (see ColumnElement)
compile(bind=None, dialect=None, **kw)
inherited from the compile() method of ClauseElement
Compile this SQL expression.
The return value is a Compiled object. Calling str() or unicode() on the returned value will yield
a string representation of the result. The Compiled object also can return a dictionary of bind parameter
names and values using the params accessor.
Parameters
• bind – An Engine or Connection from which a Compiled will be acquired.
This argument takes precedence over this ClauseElement‘s bound engine, if any.
• column_keys – Used for INSERT and UPDATE statements, a list of column names
which should be present in the VALUES clause of the compiled statement. If None,
all columns from the target table object are rendered.
• dialect – A Dialect instance from which a Compiled will be acquired. This argument takes precedence over the bind argument as well as this
ClauseElement‘s bound engine, if any.
• inline – Used for INSERT statements, for a dialect which does not support inline
retrieval of newly generated primary key columns, will force the expression used to
create the new primary key value to be rendered inline within the INSERT statement’s
VALUES clause. This typically refers to Sequence execution but may also refer to any
server-side default generation function associated with a primary key Column.
3.2. SQL Statements and Expressions API
405
SQLAlchemy Documentation, Release 0.8.7
correspond_on_equivalents(column, equivalents)
inherited from the correspond_on_equivalents() method of FromClause
Return corresponding_column for the given column, or if None search for a match in the given dictionary.
corresponding_column(column, require_embedded=False)
inherited from the corresponding_column() method of FromClause
Given a ColumnElement, return the exported ColumnElement object from this Selectable
which corresponds to that original Column via a common ancestor column.
Parameters
• column – the target ColumnElement to be matched
• require_embedded – only return corresponding columns for
the given ColumnElement, if the given ColumnElement is actually present within a sub-element of
this FromClause. Normally the column will match if it merely shares a common ancestor with one of
the exported columns of this FromClause.
count(whereclause=None, **params)
inherited from the count() method of FromClause
return a SELECT COUNT generated against this FromClause.
foreign_keys
inherited from the foreign_keys attribute of FromClause
Return the collection of ForeignKey objects which this FromClause references.
join(right, onclause=None, isouter=False)
inherited from the join() method of FromClause
Return a Join from this FromClause to another FromClause.
E.g.:
from sqlalchemy import join
j = user_table.join(address_table,
user_table.c.id == address_table.c.user_id)
stmt = select([user_table]).select_from(j)
would emit SQL along the lines of:
SELECT user.id, user.name FROM user
JOIN address ON user.id = address.user_id
Parameters
• right – the right side of the join; this is any FromClause object such as a Table
object, and may also be a selectable-compatible object such as an ORM-mapped class.
• onclause – a SQL expression representing the ON clause of the join. If left at
None, FromClause.join() will attempt to join the two tables based on a foreign
key relationship.
• isouter – if True, render a LEFT OUTER JOIN, instead of JOIN.
See also:
join() - standalone function
Join - the type of object produced
406
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
outerjoin(right, onclause=None)
inherited from the outerjoin() method of FromClause
Return a Join from this FromClause to another FromClause, with the “isouter” flag set to True.
E.g.:
from sqlalchemy import outerjoin
j = user_table.outerjoin(address_table,
user_table.c.id == address_table.c.user_id)
The above is equivalent to:
j = user_table.join(address_table,
user_table.c.id == address_table.c.user_id, isouter=True)
Parameters
• right – the right side of the join; this is any FromClause object such as a Table
object, and may also be a selectable-compatible object such as an ORM-mapped class.
• onclause – a SQL expression representing the ON clause of the join. If left at
None, FromClause.join() will attempt to join the two tables based on a foreign
key relationship.
See also:
FromClause.join()
Join
params(*optionaldict, **kwargs)
inherited from the params() method of ClauseElement
Return a copy with bindparam() elements replaced.
Returns a copy of this ClauseElement with bindparam() elements replaced with values taken from the
given dictionary:
>>> clause = column('x') + bindparam('foo')
>>> print clause.compile().params
{'foo':None}
>>> print clause.params({'foo':7}).compile().params
{'foo':7}
primary_key
inherited from the primary_key attribute of FromClause
Return the collection of Column objects which comprise the primary key of this FromClause.
replace_selectable(old, alias)
inherited from the replace_selectable() method of FromClause
replace all occurrences of FromClause ‘old’ with the given Alias object, returning a copy of this
FromClause.
select(whereclause=None, **params)
inherited from the select() method of FromClause
return a SELECT of this FromClause.
See also:
3.2. SQL Statements and Expressions API
407
SQLAlchemy Documentation, Release 0.8.7
select() - general purpose method which allows for arbitrary column lists.
self_group(against=None)
inherited from the self_group() method of ClauseElement
Apply a ‘grouping’ to this ClauseElement.
This method is overridden by subclasses to return a “grouping” construct, i.e. parenthesis. In particular
it’s used by “binary” expressions to provide a grouping around themselves when placed into a larger expression, as well as by select() constructs when placed into the FROM clause of another select().
(Note that subqueries should be normally created using the Select.alias() method, as many platforms require nested SELECT statements to be named).
As expressions are composed together, the application of self_group() is automatic - end-user code
should never need to use this method directly. Note that SQLAlchemy’s clause constructs take operator
precedence into account - so parenthesis might not be needed, for example, in an expression like x OR
(y AND z) - AND takes precedence over OR.
The base self_group() method of ClauseElement just returns self.
unique_params(*optionaldict, **kwargs)
inherited from the unique_params() method of ClauseElement
Return a copy with bindparam() elements replaced.
Same functionality as params(), except adds unique=True to affected bind parameters so that multiple
statements can be used.
class sqlalchemy.sql.expression.Executable
Bases: sqlalchemy.sql.expression.Generative
Mark a ClauseElement as supporting execution.
Executable is a superclass for all “statement” types of objects, including select(), delete(),
update(), insert(), text().
bind
Returns the Engine or Connection to which this Executable is bound, or None if none found.
This is a traversal which checks locally, then checks among the “from” clauses of associated objects until
a bound engine or connection is found.
execute(*multiparams, **params)
Compile and execute this Executable.
execution_options(**kw)
Set non-SQL options for the statement which take effect during execution.
Execution options can be set on a per-statement or per Connection basis. Additionally, the Engine
and ORM Query objects provide access to execution options which they in turn configure upon connections.
The execution_options() method is generative. A new instance of this statement is returned that
contains the options:
statement = select([table.c.x, table.c.y])
statement = statement.execution_options(autocommit=True)
Note that only a subset of possible execution options can be applied to a statement - these include “autocommit” and “stream_results”, but not “isolation_level” or “compiled_cache”. See
Connection.execution_options() for a full list of possible options.
See also:
408
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
Connection.execution_options()
Query.execution_options()
scalar(*multiparams, **params)
Compile and execute this Executable, returning the result’s scalar representation.
class sqlalchemy.sql.expression.FromClause
Bases: sqlalchemy.sql.expression.Selectable
Represent an element that can be used within the FROM clause of a SELECT statement.
The most common forms of FromClause are the Table and the select() constructs. Key features common to all FromClause objects include:
•a c collection, which provides per-name access to a collection of ColumnElement objects.
•a primary_key attribute, which is a collection of all those ColumnElement objects that indicate the
primary_key flag.
•Methods to generate various derivations of a “from” clause, including FromClause.alias(),
FromClause.join(), FromClause.select().
alias(name=None)
return an alias of this FromClause.
This is shorthand for calling:
from sqlalchemy import alias
a = alias(self, name=name)
See alias() for details.
c
An alias for the columns attribute.
columns
A named-based collection of ColumnElement objects maintained by this FromClause.
The columns, or c collection, is the gateway to the construction of SQL expressions using table-bound
or other selectable-bound columns:
select([mytable]).where(mytable.c.somecolumn == 5)
correspond_on_equivalents(column, equivalents)
Return corresponding_column for the given column, or if None search for a match in the given dictionary.
corresponding_column(column, require_embedded=False)
Given a ColumnElement, return the exported ColumnElement object from this Selectable
which corresponds to that original Column via a common ancestor column.
Parameters
• column – the target ColumnElement to be matched
• require_embedded – only return corresponding columns for
the given ColumnElement, if the given ColumnElement is actually present within a sub-element of
this FromClause. Normally the column will match if it merely shares a common ancestor with one of
the exported columns of this FromClause.
count(whereclause=None, **params)
return a SELECT COUNT generated against this FromClause.
3.2. SQL Statements and Expressions API
409
SQLAlchemy Documentation, Release 0.8.7
description
a brief description of this FromClause.
Used primarily for error message formatting.
foreign_keys
Return the collection of ForeignKey objects which this FromClause references.
is_derived_from(fromclause)
Return True if this FromClause is ‘derived’ from the given FromClause.
An example would be an Alias of a Table is derived from that Table.
join(right, onclause=None, isouter=False)
Return a Join from this FromClause to another FromClause.
E.g.:
from sqlalchemy import join
j = user_table.join(address_table,
user_table.c.id == address_table.c.user_id)
stmt = select([user_table]).select_from(j)
would emit SQL along the lines of:
SELECT user.id, user.name FROM user
JOIN address ON user.id = address.user_id
Parameters
• right – the right side of the join; this is any FromClause object such as a Table
object, and may also be a selectable-compatible object such as an ORM-mapped class.
• onclause – a SQL expression representing the ON clause of the join. If left at
None, FromClause.join() will attempt to join the two tables based on a foreign
key relationship.
• isouter – if True, render a LEFT OUTER JOIN, instead of JOIN.
See also:
join() - standalone function
Join - the type of object produced
outerjoin(right, onclause=None)
Return a Join from this FromClause to another FromClause, with the “isouter” flag set to True.
E.g.:
from sqlalchemy import outerjoin
j = user_table.outerjoin(address_table,
user_table.c.id == address_table.c.user_id)
The above is equivalent to:
j = user_table.join(address_table,
user_table.c.id == address_table.c.user_id, isouter=True)
Parameters
410
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
• right – the right side of the join; this is any FromClause object such as a Table
object, and may also be a selectable-compatible object such as an ORM-mapped class.
• onclause – a SQL expression representing the ON clause of the join. If left at
None, FromClause.join() will attempt to join the two tables based on a foreign
key relationship.
See also:
FromClause.join()
Join
primary_key
Return the collection of Column objects which comprise the primary key of this FromClause.
replace_selectable(old, alias)
replace all occurrences of FromClause ‘old’ with the given Alias object, returning a copy of this
FromClause.
select(whereclause=None, **params)
return a SELECT of this FromClause.
See also:
select() - general purpose method which allows for arbitrary column lists.
class sqlalchemy.sql.expression.Join(left, right, onclause=None, isouter=False)
Bases: sqlalchemy.sql.expression.FromClause
represent a JOIN construct between two FromClause elements.
The public constructor function for Join is the module-level join() function, as well as the join() method
available off all FromClause subclasses.
__init__(left, right, onclause=None, isouter=False)
Construct a new Join.
The usual entrypoint here is the join() function or the FromClause.join() method of any
FromClause object.
alias(name=None)
return an alias of this Join.
Used against a Join object, alias() calls the select() method first so that a subquery against a
select() construct is generated. the select() construct also has the correlate flag set to False
and will not auto-correlate inside an enclosing select() construct.
The equivalent long-hand form, given a Join object j, is:
from sqlalchemy import select, alias
j = alias(
select([j.left, j.right]).\
select_from(j).\
with_labels(True).\
correlate(False),
name=name
)
See alias() for further details on aliases.
c
inherited from the c attribute of FromClause
3.2. SQL Statements and Expressions API
411
SQLAlchemy Documentation, Release 0.8.7
An alias for the columns attribute.
columns
inherited from the columns attribute of FromClause
A named-based collection of ColumnElement objects maintained by this FromClause.
The columns, or c collection, is the gateway to the construction of SQL expressions using table-bound
or other selectable-bound columns:
select([mytable]).where(mytable.c.somecolumn == 5)
compare(other, **kw)
inherited from the compare() method of ClauseElement
Compare this ClauseElement to the given ClauseElement.
Subclasses should override the default behavior, which is a straight identity comparison.
**kw are arguments consumed by subclass compare() methods and may be used to modify the criteria for
comparison. (see ColumnElement)
compile(bind=None, dialect=None, **kw)
inherited from the compile() method of ClauseElement
Compile this SQL expression.
The return value is a Compiled object. Calling str() or unicode() on the returned value will yield
a string representation of the result. The Compiled object also can return a dictionary of bind parameter
names and values using the params accessor.
Parameters
• bind – An Engine or Connection from which a Compiled will be acquired.
This argument takes precedence over this ClauseElement‘s bound engine, if any.
• column_keys – Used for INSERT and UPDATE statements, a list of column names
which should be present in the VALUES clause of the compiled statement. If None,
all columns from the target table object are rendered.
• dialect – A Dialect instance from which a Compiled will be acquired. This argument takes precedence over the bind argument as well as this
ClauseElement‘s bound engine, if any.
• inline – Used for INSERT statements, for a dialect which does not support inline
retrieval of newly generated primary key columns, will force the expression used to
create the new primary key value to be rendered inline within the INSERT statement’s
VALUES clause. This typically refers to Sequence execution but may also refer to any
server-side default generation function associated with a primary key Column.
correspond_on_equivalents(column, equivalents)
inherited from the correspond_on_equivalents() method of FromClause
Return corresponding_column for the given column, or if None search for a match in the given dictionary.
corresponding_column(column, require_embedded=False)
inherited from the corresponding_column() method of FromClause
Given a ColumnElement, return the exported ColumnElement object from this Selectable
which corresponds to that original Column via a common ancestor column.
Parameters
• column – the target ColumnElement to be matched
412
Chapter 3. SQLAlchemy Core
SQLAlchemy Documentation, Release 0.8.7
• require_embedded – only return corresponding columns for
the given ColumnElement, if the given ColumnElement is actually present within a sub-element of
this FromClause. Normally the column will match if it merely shares a common ancestor with one of
the exported columns of this FromClause.
count(whereclause=None, **params)
inherited from the count() method of FromClause
return a SELECT COUNT generated against this FromClause.
foreign_keys
inherited from the foreign_keys attribute of FromClause
Return the collection of ForeignKey objects which this FromClause references.
join(right, onclause=None, isouter=False)
inherited from the join() method of FromClause
Return a Join from this FromClause to another FromClause.
E.g.:
from sqlalchemy import join
j = user_table.join(address_table,
user_table.c.id == address_table.c.user_id)
stmt = select([user_table]).select_from(j)
would emit SQL along the lines of:
SELECT user.id, user.name FROM user
JOIN address ON user.id = address.user_id
Parameters
• right – the right side of the join; this is any FromClause object such as a Table
object, and may also be a selectable-compatible object such as an ORM-mapped class.
• onclause – a SQL expression representing the ON clause of the join. If left at
None, FromClause.join() will attempt to join the two tables based on a foreign
key relationship.
• isouter – if True, render a LEFT OUTER JOIN, instead of JOIN.
See also:
join() - standalone function
Join - the type of object produced
outerjoin(right, onclause=None)
inherited from the outerjoin() method of FromClause
Return a Join from this FromClause to another FromClause, with the “isouter” flag set to True.
E.g.:
from sqlalchemy import outerjoin
j = user_table.outerjoin(address_table,
user_table.c.id == address_table.c.user_id)
The above is equivalent to:
3.2. SQL Statements and Expressions API
413
SQLAlchemy Documentation, Release 0.8.7
j = user_table.join(address_table,
user_table.c.id == address_table.c.user_id, isouter=True)
Parameters
• right – the right side of the join; this is any FromClause object such as a Table
object, and may also be a selectable-compatible object such as an ORM-mapped class.
• onclause – a SQL expression representing the ON clause of the join. If left at
None, FromClause.join() will attempt to join the two tables based on a foreign
key relationship.
See also:
FromClause.join()