CHAPTER 3 Using Data Loader
In this chapter ...
•
Data Types
Supported by Data
Loader
•
Export Data
•
Define Data Loader
Field Mappings
•
Insert, Update, or
Delete Data Using
Data Loader
•
Uploading
Attachments
•
Upload Content with the Data Loader
•
Review Data Loader
Output Files
•
Data Import Limits
•
View the Data Loader
Log File
Using Data Loader, you can perform various operations which include exporting data, defining field mappings, inserting, updating, and deleting data, performing mass updates and mass deletes, uploading attachments and content, and reviewing output files.
EDITIONS
Available in: Salesforce
Classic and Lightning
Experience
Available in: Enterprise,
Performance, Unlimited,
Developer, and
Database.com Editions
9
Using Data Loader Data Types Supported by Data Loader
Data Types Supported by Data Loader
Data Loader supports the following data types:
Base64
String path to file (converts the file to a base64–encoded array). Base64 fields are only used to
insert or update attachments and Salesforce CRM Content. For more information, see Uploading
Attachments
on page 15 and Upload Content with the Data Loader on page 16.
Boolean
•
True values (case insensitive) = yes
, y
, true
, on
,
1
•
False values (case insensitive) = no
, n
, false
, off
,
0
Date Formats
We recommend you specify dates in the format
yyyy-MM-ddTHH:mm:ss.SSS+/-HHmm
:
•
yyyy
is the four-digit year
•
MM
is the two-digit month (01-12)
•
dd
is the two-digit day (01-31)
•
HH
is the two-digit hour (00-23)
•
mm
is the two-digit minute (00-59)
•
ss
is the two-digit seconds (00-59)
•
SSS
is the three-digit milliseconds (000-999)
•
+/-HHmm
is the Zulu (UTC) time zone offset
The following date formats are also supported:
•
yyyy-MM-dd'T'HH:mm:ss.SSS'Z'
•
yyyy-MM-dd'T'HH:mm:ss.SSS Pacific Standard Time
•
yyyy-MM-dd'T'HH:mm:ss.SSSPacific Standard Time
•
yyyy-MM-dd'T'HH:mm:ss.SSS PST
•
yyyy-MM-dd'T'HH:mm:ss.SSSPST
•
yyyy-MM-dd'T'HH:mm:ss.SSS GMT-08:00
•
yyyy-MM-dd'T'HH:mm:ss.SSSGMT-08:00
•
yyyy-MM-dd'T'HH:mm:ss.SSS -800
•
yyyy-MM-dd'T'HH:mm:ss.SSS-800
•
yyyy-MM-dd'T'HH:mm:ss
•
yyyy-MM-dd HH:mm:ss
•
yyyyMMdd'T'HH:mm:ss
•
yyyy-MM-dd
•
MM/dd/yyyy HH:mm:ss
•
MM/dd/yyyy
•
yyyyMMdd
Note the following tips for date formats:
EDITIONS
Available in: Salesforce
Classic and Lightning
Experience
Available in: Enterprise,
Performance, Unlimited,
Developer, and
Database.com Editions
10
Using Data Loader Export Data
•
To enable date formats that begin with the day rather than the month, select the
Use European date format
box in the Settings dialog. European date formats are dd/MM/yyyy
and dd/MM/yyyy HH:mm:ss
.
•
If your computer's locale is east of Greenwich Mean Time (GMT), we recommend that you change your computer setting to
GMT in order to avoid date adjustments when inserting or updating records.
•
Only dates within a certain range are valid. The earliest valid date is 1700-01-01T00:00:00Z GMT, or just after midnight on January
1, 1700. The latest valid date is 4000-12-31T00:00:00Z GMT, or just after midnight on December 31, 4000. These values are offset by your time zone. For example, in the Pacific time zone, the earliest valid date is 1699-12-31T16:00:00, or 4:00 PM on December
31, 1699.
Double
Standard double string
ID
A Salesforce ID is a case-sensitive 15-character or case–insensitive 18-character alphanumeric string that uniquely identifies a particular record.
Tip: To ensure data quality, make sure that all Salesforce IDs you enter in Data Loader are in the correct case.
Integer
Standard integer string
String
All valid XML strings; invalid XML characters are removed.
Export Data
You can use the Data Loader export wizard to extract data from any Salesforce object. When you export, you can choose to include (Export All) or exclude (Export) soft-deleted records.
1. Open the Data Loader.
2. Click Export or Export All. These commands can also be found in the File menu.
3. Enter your Salesforce username and password. Click Log in to log in. After your login completes successfully, click Next. (Until you log out or close the program, you will not be asked to log in again.)
If your organization restricts IP addresses, logins from untrusted IPs are blocked until they’re activated. Salesforce automatically sends you an activation email that you can use to log in. The email contains a security token that you must add to the end of your password. For example, if your password is
mypassword
, and your security token is
XXXXXXXXXX
, you must enter
mypasswordXXXXXXXXXX
to log in.
EDITIONS
Available in: both Salesforce
Classic and Lightning
Experience
Available in: Enterprise,
Performance, Unlimited,
Developer, and
Database.com Editions
USER PERMISSIONS
4. Choose an object. For example, select the Account object. If your object name does not display in the default list, check
Show all objects
to see a complete list of objects that you can access. The objects will be listed by localized label name, with developer name noted in parentheses. For object descriptions, see the
SOAP API Developer's Guide
.
To export records:
•
Read on the records
To export all records:
•
Read on the records
5. Click Browse... to select the CSV file to which the data will be exported. You can enter a new file name to create a new file or choose an existing file.
If you select an existing file, the contents of that file are replaced. Click Yes to confirm this action, or click No to choose another file.
6. Click Next.
11
Using Data Loader Export Data
7. Create a SOQL query for the data export. For example, check
Id
and
Name
in the query fields and click Finish. As you follow the next steps, you will see that the CSV viewer displays all the Account names and their IDs. SOQL is the Salesforce Object Query
Language that allows you to construct simple but powerful query strings. Similar to the SELECT command in SQL, SOQL allows you to specify the source object, a list of fields to retrieve, and conditions for selecting rows in the source object.
a. Choose the fields you want to export.
b. Optionally, select conditions to filter your data set. If you do not select any conditions, all the data to which you have read access will be returned.
c. Review the generated query and edit if necessary.
Tip: You can use a SOQL relationship query to include fields from a related object. For example:
Select Name, Pricebook2Id, Pricebook2.Name, Product2Id, Product2.ProductCode FROM
PricebookEntry WHERE IsActive = true
Or:
Select Id, LastName, Account.Name FROM Contact
When using relationship queries in Data Loader, the fully specified field names are case-sensitive. For example, using
ACCOUNT.NAME
instead of
Account.Name
does not work.
Data Loader doesn’t support nested queries or querying child objects. For example, queries similar to the following return an error:
SELECT Amount, Id, Name, (SELECT Quantity, ListPrice,
PriceBookEntry.UnitPrice, PricebookEntry.Name,
PricebookEntry.product2.Family FROM OpportunityLineItems)
FROM Opportunity
Also, Data Loader doesn’t support queries that make use of polymorphic relationships. For example, the following query results in an error:
SELECT Id, Owner.Name, Owner.Type, Owner.Id, Subject FROM Case
For more information on SOQL, see the
Force.com SOQL and SOSL Reference
.
8. Click Finish, then click Yes to confirm.
9. A progress information window reports the status of the operation.
10. After the operation completes, a confirmation window summarizes your results. Click View Extraction to view the CSV file, or click
OK to close. For more details, see Review Data Loader Output Files on page 17.
Note:
•
Data Loader currently does not support the extraction of attachments. As a workaround, we recommend that you use the weekly export feature in the online application to export attachments.
•
If you select compound fields for export in the Data Loader, they cause error messages. To export values, use individual field components.
12
Using Data Loader Define Data Loader Field Mappings
Define Data Loader Field Mappings
When you insert, delete, or update files, use the Mapping Dialog window to associate Salesforce
fields with the columns of your CSV file. For more information, see Insert, Update, or Delete Data
Using Data Loader on page 13.
1. To automatically match fields with columns, click Auto-Match Fields to Columns. The Data
Loader populates the list at the bottom of the window based on the similarity of field and column names. For a delete operation, automatic matching works only on the ID field.
EDITIONS
Available in: Salesforce
Classic and Lightning
Experience
2. To manually match fields with columns, click and drag fields from the list of Salesforce fields at the top to the list of CSV column header names at the bottom. For example, if you are inserting new Account records where your CSV file contains the names of new accounts, click and drag the
Name
field to the right of the
NAME
column header field.
Available in: Enterprise,
Performance, Unlimited,
Developer, and
Database.com Editions
3. Optionally, click Save Mapping to save this mapping for future use. Specify a name for the
SDL mapping file.
If you select an existing file, the contents of that file are replaced. Click Yes to confirm this action, or click No to choose another file.
4. Click OK to use your mapping for the current operation.
Insert, Update, or Delete Data Using Data Loader
USER PERMISSIONS
To insert records:
To update records:
To upsert records:
To delete records:
To hard delete records:
To mass delete records:
Create on the record
Edit on the record
Create or Edit on the record
Delete on the record
Delete on the record
Modify All Data
EDITIONS
Available in: both Salesforce
Classic and Lightning
Experience
Available in: Enterprise,
Performance, Unlimited,
Developer, and
Database.com Editions
Use the Data Loader wizards to add, modify, or delete records. The upsert wizard combines inserting and updating a record. If a record in your file matches an existing record, the existing record is updated with the values in your file. If no match is found, a new record is created. When you hard-delete records, the deleted records are not stored in the Recycle Bin and are eligible for deletion. For more information, see Configure Data Loader.
1. Open the Data Loader.
2. Click Insert, Update, Upsert, Delete, or Hard Delete. These commands are also listed in the File menu.
3. Enter your Salesforce username and password. To log in, click Log in. When you are logged in, click Next. (Until you log out or close the program, you are not asked to log in again.)
If your organization restricts IP addresses, logins from untrusted IPs are blocked until they’re activated. Salesforce automatically sends you an activation email that you can use to log in. The email contains a security token that you must add to the end of your password.
For example, if your password is
mypassword
, and your security token is
XXXXXXXXXX
, you must enter
mypasswordXXXXXXXXXX
to log in.
13
Using Data Loader Insert, Update, or Delete Data Using Data Loader
4. Choose an object. For example, if you are inserting Account records, select Account. If your object name does not display in the default list, select Show all objects to see a complete list of the objects that you can access. The objects are listed by localized label name, with the developer name noted in parentheses. For object descriptions, see the
Object Reference for Salesforce and Force.com
.
Note: Data Loader deletes records based on the IDs in the CSV file, not the object selected.
5. To select your CSV file, click Browse. For example, if you are inserting Account records, you could specify a CSV file called insertaccounts.csv
containing a Name column for the names of the new accounts.
6. Click Next. After the object and CSV file are initialized, click OK.
7. If you are performing an upsert, your CSV file must contain a column of ID values for matching against existing records. The column is either an external ID (a custom field with the External ID attribute) or ID (the Salesforce record ID).
a. From the dropdown list, select which field to use for matching. If the object has no external ID fields, ID is used. Click Next to continue.
b. If your file includes the external IDs of an object that has a relationship to your chosen object, enable that external ID for record matching by selecting its name from the dropdown list. If you make no selection, you can use the related object’s ID field for matching by mapping it in the next step. Click Next to continue.
8. Define how the columns in your CSV file map to Salesforce fields. To select an existing field mapping, click Choose an Existing
Map. To create or modify a map, click Create or Edit a Map. For more information, see
Define Data Loader Field Mappings
on page
13. Click Next.
9. For each operation, the Data Loader generates two unique CSV log files. One file name starts with “success,” and the other starts with “error.” Click Browse to specify a directory for these files.
10. To complete the operation, click Finish, and then click Yes to confirm. As the operation proceeds, a progress information window reports the status of the data movement.
11. To view your success or error files, click View Successes or View Errors. To close the wizard, click OK . For more information, see
Review Data Loader Output Files
on page 17.
Tip:
•
If you are updating or deleting large amounts of data, review
Perform Mass Updates and Perform Mass Deletes
for tips and best practices.
•
There is a 5-minute limit to process 100 records when the Bulk API is enabled. If it takes longer than 10 minutes to process a file, the Bulk API places the remainder of the file back in the queue for later processing. If the Bulk API continues to exceed the
10-minute limit on subsequent attempts, the file is placed back in the queue and reprocessed up to 10 times before the operation is permanently marked as failed. Even if the processing fails, some records could have completed successfully, so check the results. If you get a timeout error when loading a file, split your file into smaller files and try again.
14
Using Data Loader Perform Mass Updates
Perform Mass Updates
To update a large number of records at one time, we recommend the following steps:
1. Obtain your data by performing an export of the objects you wish to update, or by running a report. Make sure your report includes the record ID.
2. As a backup measure, save an extra copy of the generated CSV file.
3. Open your working file in a CSV editor such as Excel, and update your data.
4. Launch Data Loader and follow the update wizard. Note that matching is done according to
record ID. See Insert, Update, or Delete Data Using Data Loader
on page 13.
5. After the operation, review your success and error log files. See Review Data Loader Output
Files on page 17.
6. If you made a mistake, use the backup file to update the records to their previous values.
EDITIONS
Available in: both Salesforce
Classic and Lightning
Experience
Available in: Enterprise,
Performance, Unlimited,
Developer, and
Database.com Editions
Perform Mass Deletes
To delete a large number of records at one time using Data Loader, we recommend the following steps:
1. As a backup measure, export the records you wish to delete, being sure to select all fields. (See
Export Data on page 11.) Save an extra copy of the generated CSV file.
2. Next, export the records you wish to delete, this time using only the record ID as the desired criterion.
3. Launch the Data Loader and follow the delete or hard delete wizard. Map only the ID column.
See
Insert, Update, or Delete Data Using Data Loader on page 13.
4. After the operation, review your success and error log files. See Review Data Loader Output
Files on page 17.
EDITIONS
Available in: both Salesforce
Classic and Lightning
Experience
Available in: Enterprise,
Performance, Unlimited,
Developer, and
Database.com Editions
USER PERMISSIONS
Uploading Attachments
To mass delete records:
•
Modify All Data
You can use Data Loader to upload attachments to Salesforce. Before uploading attachments, note the following:
•
If you intend to upload via the Bulk API, verify that
Upload Bulk API Batch as Zip File
on the Settings > Settings page is enabled.
•
If you are migrating attachments from a source Salesforce organization to a target Salesforce organization, begin by requesting a data export for the source organization. On the Schedule Export page, make sure to select the
Include Attachments...
checkbox, which causes the file
Attachment.csv
to be included in your export. You can use this CSV file to upload the attachments. For more information on the export service, see Export Backup Data from Salesforce .
To upload attachments:
1. Confirm that the CSV file you intend to use for attachment importing contains the following required columns (each column represents a Salesforce field):
•
ParentId
- the Salesforce ID of the parent record.
•
Name
- the name of the attachment file, such as myattachment.jpg
.
•
Body
- the absolute path to the attachment on your local drive.
15
Using Data Loader Upload Content with the Data Loader
Ensure that the values in the
Body
column contain the full file name of the attachments as they exist on your computer. For example, if an attachment named myattachment.jpg
is located on your computer at
C:\Export
,
Body
must specify
C:\Export\myattachment.jpg
. Your CSV file might look like this:
ParentId,Name,Body
50030000000VDowAAG,attachment1.jpg,C:\Export\attachment1.gif
701300000000iNHAAY,attachment2.doc,C:\Export\files\attachment2.doc
The CSV file can also include other optional Attachment fields, such as
Description
.
2. Proceed with an insert or upsert operation; see
Insert, Update, or Delete Data Using Data Loader
on page 13. At the
Select data objects
step, make sure to select the
Show all Salesforce objects
checkbox and the
Attachment object name in the list.
Upload Content with the Data Loader
You can use Data Loader to bulk upload documents and links into libraries in Salesforce CRM
Content. Before uploading documents or links, note the following.
•
If you intend to upload via the Bulk API, verify that
Upload Bulk API Batch as Zip
File
on the Settings > Settings page is enabled.
•
When you upload a document from your local drive using Data Loader, specify the path in the
VersionData
and
PathOnClient
fields in the CSV file.
VersionData
identifies the location and extracts the format, and
PathOnClient
identifies the type of document being uploaded.
•
When you upload a link using the Data Loader, specify the URL in
ContentUrl
. Don’t use
PathOnClient
or
VersionData
to upload links.
•
You can’t export content using the Data Loader.
•
If you’re updating content that you’ve already uploaded:
EDITIONS
Available in: Salesforce
Classic and Lightning
Experience
Available in: Enterprise,
Performance, Unlimited, and Developer Editions
–
Perform the Insert function.
–
Include a
ContentDocumentId
column with an 18-character ID. Salesforce uses this information to determine that you’re updating content. When you map the
ContentDocumentId
, the updates are added to the content file. If you don’t include the ContentDocumentId, the content is treated as new, and the content file isn’t updated.
1. Create a CSV file with the following fields.
•
Title
- file name.
•
Description
- (optional) file or link description.
Note: If there are commas in the description, use double quotes around the text.
•
VersionData
- complete file path on your local drive (for uploading documents only).
Note:
Files are converted to base64 encoding on upload. This action adds approximately 30% to the file size.
•
PathOnClient
- complete file path on your local drive (for uploading documents only).
•
ContentUrl
- URL (for uploading links only).
•
OwnerId
- (optional) file owner, defaults to the user uploading the file.
•
FirstPublishLocationId
- library ID.
16
Using Data Loader Review Data Loader Output Files
•
RecordTypeId
- record type ID.
Note:
If you publish to a library that has restricted record types, specify
RecordTypeId
.
To determine the
RecordTypeId
values for your organization using Data Loader, follow the steps in
Exporting Data . The
following is a sample SOQL query:
Select Id, Name FROM RecordType WHERE SobjectType = 'ContentVersion'
To determine the
RecordTypeId
values for your organization using the AJAX Toolkit:
a. Log in to Salesforce.
b. Enter this URL in your browser: http:// instanceName.salesforce.com/soap/ajax/41.0/debugshell.html
. Enter the
instanceName
for your organization. You can see the
instanceName
in the URL field of your browser after logging in to Salesforce.
c. In the AJAX Toolkit Shell page, type: sforce.connection.describeSObject("ContentVersion")
d. Press Enter.
e. Click the arrows for recordTypeInfos
.
The
RecordTypeId
values for your organization are listed.
•
TagsCsv
- (optional) tag.
A sample CSV file is:
Title,Description,VersionData,PathOnClient,OwnerId,FirstPublishLocationId,RecordTypeId,TagsCsv testfile,"This is a test file, use for bulk upload",c:\files\testfile.pdf,c:\files\testfile.pdf,005000000000000,058700000004Cd0,012300000008o2sAQG,one
2. Upload the CSV file for the ContentVersion object (see Insert, Update, or Delete Data Using Data Loader on page 13). All documents
and links are available in the specified library.
Review Data Loader Output Files
After an import or export, Data Loader generates two CSV output files that contain the results of the operation. One file name starts with “success,” and the other starts with “error.” You can use the
Data Loader CSV file viewer to open the files.
1. Choose View > View CSV.
2. Specify the number of rows to view. Each row in the CSV file corresponds to one Salesforce record. The default is 1,000.
EDITIONS
Available in: Salesforce
Classic and Lightning
Experience
3. To view a specific CSV file, click Open CSV. To view the last success file, click Open Success.
To view the last error file, click Open Error.
4. To open the file in an external program, such as Excel, click Open in External Program.
Available in: Enterprise,
Performance, Unlimited,
Developer, and
Database.com Editions
The success file contains all the successfully loaded records. The file includes a column with the newly generated record IDs. The error file contains all the rejected records. The file has a column that describes why the load failed.
17
Using Data Loader Data Import Limits
Note: If the object you are exporting has a column named “success” or “error,” your output file columns could display incorrect information. To avoid this problem, rename the columns.
5. To return to the CSV Chooser window, click Close. To exit the window, click OK.
Note:
To generate success files when exporting data, select
Generate status files for exports
. For more information, see
Configure Data Loader on page 4.
Data Import Limits
Limits for importing data with Data Loader.
The following limits apply to data imported using Data Loader.
Only dates within a certain range are valid. The earliest valid date is 1700-01-01T00:00:00Z GMT, or just after midnight on January 1,
1700. The latest valid date is 4000-12-31T00:00:00Z GMT, or just after midnight on December 31, 4000. These values are offset by your time zone. For example, in the Pacific time zone, the earliest valid date is 1699-12-31T16:00:00, or 4:00 PM on December 31, 1699.
When using Data Loader version 28.0 and later, the maximum field size for imported CSV files is 32,000 characters.
View the Data Loader Log File
If you need to investigate a problem with Data Loader, or if requested by Salesforce Customer
Support, you can access log files that track the operations and network connections made by Data
Loader.
EDITIONS
The log file, sdl.log
, contains a detailed chronological list of Data Loader log entries. Log entries marked “INFO” are procedural items, such as logging in to and out of Salesforce. Log entries marked
“ERROR” are problems such as a submitted record missing a required field. The log file can be opened with commonly available text editor programs, such as Microsoft Notepad.
If you are using Data Loader for Windows, view the log file by entering
%TEMP%\sdl.log
in either the Run dialog or the Windows Explorer address bar.
If you are using Data Loader for Mac OSX, view the log file by opening terminal and entering
open
$TMPDIR/sdl.log
.
Available in: Salesforce
Classic and Lightning
Experience
Available in: Enterprise,
Performance, Unlimited,
Developer, and
Database.com Editions
If you are having login issues from the command line, ensure that the password provided in the configuration parameters is encrypted.
If you are having login issues from the UI, you may need to obtain a new security token.
18
