CHAPTER 2 When to Use Data Loader
In this chapter ...
•
Considerations for
Installing Data
Loader
•
Configure Data
Loader
Data Loader complements the web-based import wizards that are accessible from the Setup menu in the online application. Refer to the following guidelines to determine which method best suits your business needs:
EDITIONS
Available in: both Salesforce
Classic and Lightning
Experience
Use Data Loader when:
•
You need to load 50,000 to 5,000,000 records. Data Loader is supported for loads of up to 5 million records. If you need to load more than 5 million records, we recommend you work with a Salesforce partner or visit the
App Exchange
for a suitable partner product.
Available in: Enterprise,
Performance, Unlimited,
Developer, and
Database.com Editions
•
You need to load into an object that is not yet supported by the import wizards.
•
You want to schedule regular data loads, such as nightly imports.
•
You want to export your data for backup purposes.
Use the import wizards when:
•
You are loading less than 50,000 records.
•
The object you need to import is supported by import wizards. To see what import wizards are available and thus what objects they support, from Setup, enter
Data Management
in the
Quick Find
box, then select Data Management.
•
You want to prevent duplicates by uploading records according to account name and site, contact email address, or lead email address.
For more information about the import wizards, see Import Data Into Salesforce .
2
When to Use Data Loader Considerations for Installing Data Loader
Considerations for Installing Data Loader
Before you download and install Data Loader, understand the system requirements, installation considerations, and login considerations. From Setup, enter
Data Loader
in the Quick Find box, then select Data Loader.
EDITIONS
System Requirements for Windows
Data Loader is signed for Windows. To use Data Loader for Windows, you need:
•
Microsoft
®
Windows
®
7, Windows
®
8, or Windows
®
10
•
120 MB of free disk space
•
256 MB of available memory
•
Java JRE 1.8 (32-bit)
Available in: both Salesforce
Classic and Lightning
Experience
Available in: Enterprise,
Performance, Unlimited,
Developer, and
Database.com Editions
USER PERMISSIONS
Note: Salesforce no longer bundles Java with the Data Loader for Windows installer.
Download and install Java on your Windows computer.
We recommend that you set the
JAVA_HOME
environment variable to the directory where the Java Runtime Environment (JRE) is installed. Doing so ensures that you can run Data
Loader in batch mode from the command line.
System Requirements for macOS
To use Data Loader for macOS, you need:
•
macOS El Capitan
•
120 MB of free disk space
•
256 MB of available memory
•
Java JRE 1.8
•
Administrator privileges on the machine
To access the page to download Data Loader:
•
Modify All Data
To use Data Loader:
•
API Enabled
AND
The appropriate user permission for the operation you are doing, for example, Create on accounts to insert new accounts
AND
Bulk API Hard Delete
(only if you configure
Data Loader to use Bulk
API to hard-delete records)
Installation Considerations
Over time, several versions of the Data Loader client application have been available for download. Some earlier versions were called
“AppExchange Data Loader” or “Sforce Data Loader.” You can run different versions at the same time on one computer. However, do not install more than one copy of the same version.
The latest version is always available in Salesforce. If you have installed the latest version and want to install it again, first remove the version on your computer.
Tip: If you experience login issues in the command line interface after upgrading to a new version of Data Loader, please try re-encrypting your password to solve the problem. For information on the password encryption utility, see
Encrypt from the
Command Line on page 20.
Note: The Data Loader command-line interface is supported for Windows only.
To make changes to the source code, download the open-source version of Data Loader from
https://github.com/forcedotcom/dataloader
.
3
When to Use Data Loader Configure Data Loader
Login Considerations
•
If your organization restricts IP addresses, logins from untrusted IPs are blocked until they’re activated. Salesforce automatically sends you an activation email that you can use to log in. The email contains a security token that you must add to the end of your password.
For example, if your password is
mypassword
, and your security token is
XXXXXXXXXX
, you must enter
mypasswordXXXXXXXXXX
to log in.
•
Data Loader version 36.0 and later supports Web Server OAuth Authentication. See OAuth Authentication for more information.
•
Data Loader version 36.0 and later supports Salesforce Communities. Communities users always log in with the OAuth option in
Data Loader. To enable OAuth for Communities, the user modifies the config.properties
file as follows.
–
Change the portion in bold in the following line to the login URL of the community. Don’t add a forward slash (/) to the end of the line.
sfdc.oauth.Production.server=
https\://login.salesforce.com
For example: sfdc.oauth.Production.server= https\://johnsmith-developer-edition.yourInstance.force.com/test
–
Change the portion in bold in the following line to the hostname of the community.
sfdc.oauth.Production.redirecturi=https\:// login.salesforce.com/services/oauth2/success
For example: sfdc.oauth.Production.redirecturi= https\:/johnsmith-developer-edition.yourInstance.force.com/services/oauth2/success
The config.properties
file is in the conf
default configuration directory, which is installed in these locations.
–
macOS:
/Applications/Data\ Loader.app/Contents/Resources/conf/
–
Windows:
%LOCALAPPDATA%\salesforce.com\Data Loader\samples\conf\
for the current user, and
C:\ProgramData\salesforce.com\Data Loader\samples\conf\
for all users
Configure Data Loader
Use the Settings menu to change the Data Loader default operation settings.
1. Open the Data Loader.
2. Select Settings > Settings.
3. Edit the fields as needed.
Field
Batch size
Description
In a single insert, update, upsert, or delete operation, records moving to or from
Salesforce are processed in increments of this size. The maximum is 200 records. We recommend a value between 50 and 100.
The maximum value is 10,000 if the
Use
Bulk API
option is selected.
EDITIONS
Available in: both Salesforce
Classic and Lightning
Experience
Available in: Enterprise,
Performance, Unlimited,
Developer, and
Database.com Editions
4
When to Use Data Loader
Field
Insert null values
Assignment rule
Server host
Reset URL on Login
Compression
Timeout
Query request size
Generate status files for exports
Read all CSVs with UTF-8 encoding
Write all CSVs with UTF-8 encoding
Use European date format
Configure Data Loader
Description
Select this option to insert blank mapped values as null
values during data operations. When you are updating records, this option instructs Data Loader to overwrite existing data in mapped fields.
This option is not available if the
Use Bulk API
option is selected. Empty field values are ignored when you update records using the Bulk API. To set a field value to null
when the
Use
Bulk API
option is selected, use a field value of
#N/A
.
Specify the ID of the assignment rule to use for inserts, updates, and upserts. This option applies to inserts, updates, and upserts on cases and leads. It also applies to updates on accounts if your organization has territory assignment rules on accounts. The assignment rule overrides
Owner
values in your CSV file.
Enter the URL of the Salesforce server with which you want to communicate. For example, if you are loading data into a sandbox, change the URL to https://test.salesforce.com
.
By default, Salesforce resets the URL after login to the one specified in
Server host
. To turn off this automatic reset, disable this option.
Compression enhances the performance of Data Loader and is turned on by default. You might want to disable compression when debugging the underlying SOAP messages. To turn off compression, enable this option.
Specify how many seconds Data Loader waits to receive a response back from the server before returning an error for the request.
In a single export or query operation, records are returned from
Salesforce in increments of this size. The maximum value is 2,000 records. Larger values may improve performance but use more memory on the client.
Select this option to generate success and error files when exporting data.
Select this option to force files to open in UTF-8 encoding, even if they were saved in a different format.
Select this option to force files to be written in UTF-8 encoding.
Select this option to support the date formats dd/MM/yyyy and dd/MM/yyyy HH:mm:ss
.
5
When to Use Data Loader Configure Data Loader
Field
Allow field truncation
Enable serial mode for Bulk API
Description
Select this option to truncate data in the following types of fields when loading that data into Salesforce: Email, Multi-select Picklist,
Phone, Picklist, Text, and Text (Encrypted).
In Data Loader versions 14.0 and earlier, values for fields of those types are truncated by Data Loader if they are too large. In Data
Loader version 15.0 and later, the load operation fails if a value is specified that is too large.
Selecting this option allows you to specify that the previous behavior, truncation, be used instead of the new behavior in
Data Loader versions 15.0 and later. This option is selected by default and has no effect in versions 14.0 and earlier.
This option is not available if the
Use Bulk API
option is selected. In that case, the load operation fails for the row if a value is specified that is too large for the field.
Allow comma as a CSV delimiter
Allow tab as a CSV delimiter
Use Bulk API
Select this option if your CSV file uses commas to delimit records.
Select this option if your CSV file uses tab characters to delimit records.
Allow other characters as CSV delimiters
Select this option if your CSV file uses a character other than a comma or tab to delimit records.
Other delimiters (enter multiple values with no separator; for example, !+?)
The characters in this field are used only if the Allow other
characters as CSV delimiters option is selected. For example, if you use the | (pipe) character to delimit data records, enter that character in this field.
Select this option to use Bulk API to insert, update, upsert, delete, and hard-delete records. Bulk API is optimized to load or delete many records asynchronously. It’s faster than the default
SOAP-based API due to parallel processing and fewer network round-trips.
Warning:
You can hard delete records when you configure Data Loader to
Use Bulk API
. Keep in mind that hard deleted records are immediately deleted and can’t be recovered from the Recycle Bin.
To use serial processing instead of parallel processing for Bulk
API, select this option. Processing in parallel can cause database contention. When contention is severe, the load can fail. Serial mode processes batches one at a time, however it can increase the processing time for a load.
This option is only available if the
Use Bulk API
option is selected.
6
When to Use Data Loader
Field
Upload Bulk API Batch as Zip File
Time Zone
Proxy host
Proxy port
Proxy username
Proxy password
Proxy NTLM domain
Start at row
Data Loader Behavior with Bulk API Enabled
Description
Select this option to use Bulk API to upload zip files containing binary attachments, such as Attachment records or Salesforce
CRM Content.
This option is only available if the
Use Bulk API
option is selected.
Select this option to specify a default time zone.
If a date value does not include a time zone, this value is used.
•
If no value is specified, the time zone of the computer where
Data Loader is installed is used.
•
If an incorrect value is entered, GMT is used as the time zone and this fact is noted in the Data Loader log.
Valid values are any time zone identifier which can be passed to the Java getTimeZone(java.lang.String)
method.
The value can be a full name such as
America/Los_Angeles
, or a custom ID such as
GMT-8:00
.
The host name of the proxy server, if applicable.
The proxy server port.
The username for proxy server authentication.
The password for proxy server authentication.
The name of the Windows domain used for NTLM authentication.
If your last operation failed, you can use this setting to begin where the last successful operation finished.
4. Click OK to save your settings.
Data Loader Behavior with Bulk API Enabled
Enabling the Bulk API in Data Loader allows you to load or delete a large number of records faster than using the default SOAP-based API. However, there are some differences in behavior in Data
Loader when you enable the Bulk API. One important difference is that it allows you to execute a hard delete if you have the permission and license. See
Configure Data Loader
on page 4.
The following settings are not available on the Settings > Settings page in Data Loader when the
Use Bulk API
option is selected:
Insert null values
This option enables Data Loader to insert blank mapped values as null
values during data operations when the Bulk API is disabled. Empty field values are ignored when you update records using the Bulk API. To set a field value to null
when the
Use Bulk API
option is selected, use a field value of
#N/A
.
EDITIONS
Available in: Salesforce
Classic and Lightning
Experience
Available in: Enterprise,
Performance, Unlimited,
Developer, and
Database.com Editions
7
When to Use Data Loader Configure the Data Loader to Use the Bulk API
Allow field truncation
This option directs Data Loader to truncate data for certain field types when the Bulk API is disabled. A load operation fails for the row if a value is specified that is too large for the field when the
Use Bulk API
option is selected.
Configure the Data Loader to Use the Bulk API
The Bulk API is optimized to load or delete a large number of records asynchronously. It is faster than the SOAP-based API due to parallel processing and fewer network round-trips. By default,
Data Loader uses the SOAP-based API to process records.
To configure Data Loader to use the Bulk API for inserting, updating, upserting, deleting, and hard deleting records:
1. Open the Data Loader.
2. Choose Settings > Settings.
3. Select the
Use Bulk API
option.
4. Click OK.
Note:
•
You can also select the
Enable serial mode for Bulk API
option.
Processing in parallel can cause database contention. When contention is severe, the load can fail. Serial mode processes batches one at a time, however it can increase the processing time for a load.
•
Caution: You can hard delete records when you configure Data Loader to
Use Bulk
API
. Keep in mind that hard deleted records are immediately deleted and can’t be recovered from the Recycle Bin.
EDITIONS
Available in: Salesforce
Classic and Lightning
Experience
Available in: Enterprise,
Performance, Unlimited,
Developer, and
Database.com Editions
8
