SYNCHRONIZING DATA USING ODBC
ODBC -- Overview
ListManager may synchronize its data with other databases using ODBC (Open
DataBase Connectivity).
To use ODBC, the administrator must first install the correct ODBC 3.51 compliant driver for the datasource on the ListManager server. UNIX users should use unixODBC along with the vendor-specific ODBC 3.51-compatible driver.
Next, the server administrator must create a Database Connection in Utilities:
Administration: Server: Database: Database Connections, specifying the connection string and SQL query to be used to connect to the external database.
Once the database connection has been created, a list, site or server administrator may enable the database synchronization for a particular list in Members: Synchronization
Settings , and further specify how ListManager should handle data from the external data source, and with what frequency.
The server administrator may also specify the default frequency for database synchronization in Utilities: Administration: Server: Server Settings: Scheduled Tasks:
Synchronize Lists .
Administrators may also opt to have ListManager synchronize data before sending a mailing in Mailings: New Mailing: Recipients , to ensure that ListManager is using the most updated information before sending mail.
See Database: Add Member Column for information about how to add additional columns to the ListManager members_ database.
Important ODBC Notes
ODBC in ListManager has been tested against Oracle, Microsoft SQL Server, and
MySQL. Other databases should also work, but are not supported.
ListManager checks the entire members_ table each time it synchronizes its data; it does not only look for new or changed data.
ListManager does not check to see if there is sufficient space in the database before performing synchronizations.
Synchronization can take a long time. In our internal tests, a synchronization against a list of one million members took three hours. If you are synchronizing large amounts of data, you may opt to use methods other than ODBC to keep it in sync.
Synchronization requires more space on your ListManager database server. The amount of space depends on how much data your are synchronizing against.
Do not update the password_ field. The ListManager password_ field is encrypted.
Lyris, Inc. | 6401 Hollis Street, Suite 125 | Emeryville CA 94608 USA |
www.lyris.com
Toll free +1.800.768.2929 | International calls +1.510.844.1600å | Fax +1.510.844.1598
Page 68
ODBC on Unix
1. On the ListManager server, download, gunzip and untar UnixODBC from http://www.unixodbc.org/ . Read the included documentation carefully, and be sure you are using the correct libraries.
TIP:
Tip: Use odbcinst -j to see where UnixODBC is installed.
UnixODBC contains the odbcinst.ini and odbc.ini files, which will point to the specific driver(s) you are using.
2. On the ListManager server, download and install the correct ODBC driver for your database. Read the included documentation carefully. In our testing, we used drivers from: http://dev.mysql.com/downloads/connector/odbc/3.51.html
http://www.easysoft.com/
The driver installer should prompt you for database connection information and populate the odbc.ini file with the correct values.
3. The installer for the driver may ask whether you want to use a connection string to connect, or DSN (Data Source Name). Both contain connection information, but in different formats and in different places.
This is the value you will enter in Utilities: Administration: Server: Database: New
Database Connection: Connection String . Note: In our tests, drivers from Easysoft always use DSN.
Sample Connection Strings
MS SQL
Driver={SQL Server};Server=lmtest;Database=lmdb;Uid=sa;Pwd=pass
Oracle
DSN=DSN_Name;Server=lmtest;Uid=lmuser;Pwd=pass
MySQL For Unix:
Driver={MySQL ODBC 3.51 driver};server=lmtest;database=lmdb;uid=mysqluser;pwd=pass;
Lyris, Inc. | 6401 Hollis Street, Suite 125 | Emeryville CA 94608 USA |
www.lyris.com
Toll free +1.800.768.2929 | International calls +1.510.844.1600å | Fax +1.510.844.1598
Page 69
MySQL for Windows:
Driver={MySQL ODBC 3.51 driver};server=lmtest;database=lmdb;uid=mysqluser;pwd=pass;
Sample Connection Using DSN
If your driver uses DSN, the connection information in the odbc.ini file will be used to authenticate and connect to the database. No connection string is required; instead, indicate the Data Source Name. Example:
DSN=mysql
Sample odbc.ini File
[ODBC Data Sources]
[mysql]
Driver = /usr/lib/libmyodbc3.so
Description = MySQL on lmtest
SERVER = lmtest
USER = lmuser
Password = pass
OPTION = 3
Sample Select Query
This query is connecting to a table called info. The state field has been added to the
ListManager Members_ table. select info.email as EmailAddr_, info.status as MemberType_, info.state as state from info
Important Notes
The query may not include comments (e.g., -- select a, b, c, d, e, g from numeric)
The query must be a select query. Update, insert, and Delete queries will not work.
Troubleshooting Checklist
1. Make sure you have installed UnixODBC.
2. Make sure you have installed the correct driver.
3. Make sure you are using the correct libraries.
4. Make sure you are using the correct information to connect to the database.
5. If using DSN, be sure the odbc.ini file has the correct connection information.
Lyris, Inc. | 6401 Hollis Street, Suite 125 | Emeryville CA 94608 USA |
www.lyris.com
Toll free +1.800.768.2929 | International calls +1.510.844.1600å | Fax +1.510.844.1598
Page 70
ODBC on Windows
ODBC for Importing/Syncing from MSSQL to MSSQL
1. To open Data Sources (ODBC), click Start, point to Settings, and then click
Control Panel.
2. Double-click Administrative Tools, and then double-click Data Sources (ODBC).
3. Click the System DNS tab, and then click Add.
The linked image cannot be displayed. The file may have been moved, renamed, or deleted. Verify that the link points to the correct file and location.
4. In the Create New Data Source dialog, click SQL Server (you may have to scroll down to see it), and then click Finish.
Lyris, Inc. | 6401 Hollis Street, Suite 125 | Emeryville CA 94608 USA |
www.lyris.com
Toll free +1.800.768.2929 | International calls +1.510.844.1600å | Fax +1.510.844.1598
Page 71
The linked image cannot be displayed. The file may have been moved, renamed, or deleted. Verify that the link points to the correct file and location.
5. In the Create a New Data Source to SQL Server dialog, click the Name box, and then type the name you want. Do the same for the Description box.
Lyris, Inc. | 6401 Hollis Street, Suite 125 | Emeryville CA 94608 USA |
www.lyris.com
Toll free +1.800.768.2929 | International calls +1.510.844.1600å | Fax +1.510.844.1598
Page 72
The linked image cannot be displayed. The file may have been moved, renamed, or deleted. Verify that the link points to the correct file and location.
6. In the Server box, click the down arrow, and then select the server that hosts the database you entered in the Name box. When you are finished, click Next.
The linked image cannot be displayed. The file may have been moved, renamed, or deleted. Verify that the link points to the correct file and location.
Lyris, Inc. | 6401 Hollis Street, Suite 125 | Emeryville CA 94608 USA |
www.lyris.com
Toll free +1.800.768.2929 | International calls +1.510.844.1600å | Fax +1.510.844.1598
Page 73
7. In the next dialog, select the option With SQL Server authentication using a login ID and password entered by the User.
8. Type in your Login ID and Password, and then click Next.
The linked image cannot be displayed. The file may have been moved, renamed, or deleted. Verify that the link points to the correct file and location.
Lyris, Inc. | 6401 Hollis Street, Suite 125 | Emeryville CA 94608 USA |
www.lyris.com
Toll free +1.800.768.2929 | International calls +1.510.844.1600å | Fax +1.510.844.1598
Page 74
9. In the next dialog, make no changes. Click Next.
The linked image cannot be displayed. The file may have been moved, renamed, or deleted. Verify that the link points to the correct file and location.
Lyris, Inc. | 6401 Hollis Street, Suite 125 | Emeryville CA 94608 USA |
www.lyris.com
Toll free +1.800.768.2929 | International calls +1.510.844.1600å | Fax +1.510.844.1598
Page 75
10. In the next dialog, make no changes. Click Finish.
The linked image cannot be displayed. The file may have been moved, renamed, or deleted. Verify that the link points to the correct file and location.
Lyris, Inc. | 6401 Hollis Street, Suite 125 | Emeryville CA 94608 USA |
www.lyris.com
Toll free +1.800.768.2929 | International calls +1.510.844.1600å | Fax +1.510.844.1598
Page 76
11. A summary of your previous selections displays. To verify successful ODBC Data
Source creation, click Test Data Source.
The linked image cannot be displayed. The file may have been moved, renamed, or deleted. Verify that the link points to the correct file and location.
Lyris, Inc. | 6401 Hollis Street, Suite 125 | Emeryville CA 94608 USA |
www.lyris.com
Toll free +1.800.768.2929 | International calls +1.510.844.1600å | Fax +1.510.844.1598
Page 77
12. If you run a test and it is successful, a screen similar to the one below appears.
Click OK.
The linked image cannot be displayed. The file may have been moved, renamed, or deleted. Verify that the link points to the correct file and location.
Sample ODBC Connection Strings and Queries
MS SQL
Driver={SQL Server};Server=lmtest;Database=lmdb;Uid=sa;Pwd=pass
Oracle
DSN=DSN_Name;Server=lmtest;Uid=lmuser;Pwd=pass
MySQL For Unix:
Driver={MySQL ODBC 3.51 driver};server=lmtest;database=lmdb;uid=mysqluser;pwd=pass;
MySQL for Windows:
Driver={MySQL ODBC 3.51 driver};server=lmtest;database=lmdb;uid=mysqluser;pwd=pass;
Lyris, Inc. | 6401 Hollis Street, Suite 125 | Emeryville CA 94608 USA |
www.lyris.com
Toll free +1.800.768.2929 | International calls +1.510.844.1600å | Fax +1.510.844.1598
Page 78
Sample Connection Using DSN
If your driver uses DSN, the connection information in the odbc.ini file will be used to authenticate and connect to the database. No connection string is required; instead, indicate the Data Source Name. Example:
DSN=mysql
Sample odbc.ini File
[ODBC Data Sources]
[mysql]
Driver = /usr/lib/libmyodbc3.so
Description = MySQL on lmtest
SERVER = lmtest
USER = lmuser
Password = pass
OPTION = 3
Sample Select Query
This query is connecting to a table called info. The state field has been added to the
ListManager Members_ table. select info.email as EmailAddr_, info.status as MemberType_, info.state as state from info
Important Query Notes
You must have permission to view the table in the query. If you do not have permission, you will see the following error message when you test the database connection:
Test failed because of the following error:
If SQL query does not contain EmailAddr_ field, then it must contain both
UserNameLC_ and Domain_ fields.
The query must be a select query. Update, insert, and Delete queries will generally not work.
Please
Do not update the password_ field. The ListManager password_ field is encrypted.
Lyris, Inc. | 6401 Hollis Street, Suite 125 | Emeryville CA 94608 USA |
www.lyris.com
Toll free +1.800.768.2929 | International calls +1.510.844.1600å | Fax +1.510.844.1598
Page 79
Datatypes Supported During ODBC Synchronization
SQL Server 2003/2005 Data Types
Name binary text
Description
Fixed-length binary data with a maximum length of 8,000 bytes.
Variable-length non-Unicode data with a maximum length of 2^31 - 1
(2,147,483,647) characters.
Fixed-length non-Unicode character data with a maximum length of 8,000 char varchar characters.
Variable-length non-Unicode data with a maximum of 8,000 characters. smalldatetime Date and time data from January 1, 1900, through June 6, 2079, with an accuracy datetime of one minute.
Date and time data from January 1, 1753, through December 31, 9999, with an real decimal numeric money accuracy of three-hundredths of a second, or 3.33 milliseconds.
Floating precision number data with the following valid values: -3.40E + 38 through
-1.18E - 38, 0 and 1.18E - 38 through 3.40E + 38.
Fixed precision and scale numeric data from -10^38 +1 through 10^38 –1.
Functionally equivalent to decimal.
Monetary data values from -2^63 (-922,337,203,685,477.5808) through 2^63 - 1
(+922,337,203,685,477.5807), with accuracy to a ten-thousandth of a monetary unit. int smallint tinyint
Oracle
Integer (whole number) data from -2^31 (-2,147,483,648) through 2^31 - 1
(2,147,483,647).
Integer data from -2^15 (-32,768) through 2^15 - 1 (32,767).
Integer data from 0 through 255.
External Datatype
16 bit signed
INTEGER *
32 bit signed
INTEGER *
8 bit signed INTEGER
*
Binary LOB *
CHAR * int
C++ Type
signed short, signed signed int, signed long signed char
OCILobLocator char[n]
OCILobLocator char[7]
OCCI Type
OCCIINT
OCCIINT
OCCIINT
OCCI_SQLT_BLOB
OCCI_SQLT_AFC
OCCI_SQLT_CLOB
OCCI_SQLT_DAT
Character LOB *
DATE *
FLOAT *
FLOAT **
LONG *
NATIVE DOUBLE *
NATIVE DOUBLE ** float, double float char[n] double
Bdouble, double
OCCIFLOAT
OCCIFLOAT
OCCI_SQLT_LNG
OCCIBDOUBLE
OCCIBDOUBLE
Lyris, Inc. | 6401 Hollis Street, Suite 125 | Emeryville CA 94608 USA |
www.lyris.com
Toll free +1.800.768.2929 | International calls +1.510.844.1600å | Fax +1.510.844.1598
Page 80
External Datatype
NATIVE FLOAT *
NATIVE FLOAT ** null terminated
STRING *
NUMBER *
RAW *
UNSIGNED INT *
UNSIGNED INT **
VARCHAR2 *
C++ Type OCCI Type
float
BFloat, float
OCCIBFLOAT
OCCIBFLOAT char[n+1] OCCI_SQLT_STR unsigned char[21] unsigned char[n] unsigned int unsigned int char[n]
OCCI_SQLT_NUM
OCCI_SQLT_BIN
OCCIUNSIGNED_INT
OCCIUNSIGNED_INT
OCCI_SQLT_CHR
Lyris, Inc. | 6401 Hollis Street, Suite 125 | Emeryville CA 94608 USA |
www.lyris.com
Toll free +1.800.768.2929 | International calls +1.510.844.1600å | Fax +1.510.844.1598
Page 81
