IBM Data Management TRIRIGA Application Platform User Guide

Below you will find brief information for IBM TRIRIGA Application Platform Data Management. This guide provides information about IBM TRIRIGA DataConnect and IBM TRIRIGA Data Integrator, which are tools for importing external data into the IBM TRIRIGA system. These tools can be used for initial loads of data and for batch insert/update of data on a recurring basis.

PDF

Document

IBM TRIRIGA Application Platform

Version 3 Release 4.0

Application Building for the IBM TRIRIGA

Application Platform 3:

Data Management

Note

Before using this information and the product it supports, read the information in Notices on page 59.

This edition applies to version 3, release 4, modification 0 of IBM® TRIRIGA® Application Platform and to all subsequent releases and modifications until otherwise indicated in new editions.

US Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with

IBM Corp.

ABOUT THIS GUIDE.......................................................................................................1

Conventions.................................................................................................................................................................. 1

Intended Audience....................................................................................................................................................... 1

Support ......................................................................................................................................................................... 1

1. DATACONNECT ....................................................................................................3

Glossary........................................................................................................................................................................ 3

Process Overview......................................................................................................................................................... 5

Create and Maintain Business Object Staging Tables ............................................................................................. 5

How to Identify Which Business Objects Have Staging Tables................................................................................ 6

Business Object Property: Has Staging Table .......................................................................................................... 6

Field Property – Staging Table Field......................................................................................................................... 6

Field Property – Staging Table Key .......................................................................................................................... 7

When a Business Object Is Published ....................................................................................................................... 7

When a Business Object Is Deleted........................................................................................................................... 7

Make the Job Business Object.................................................................................................................................... 7

Move Data from the External Source ........................................................................................................................ 8

Staging Tables ........................................................................................................................................................... 9

DC_JOB Table ........................................................................................................................................................ 11

Create ETL Maps and Run Mapping....................................................................................................................... 12

Use Workflow to Move Data..................................................................................................................................... 13

Workflow Tasks ...................................................................................................................................................... 14

Workflow Calls Another Workflow ........................................................................................................................ 17

Runtime Data Validation ......................................................................................................................................... 17

Status ....................................................................................................................................................................... 17

Use Temporary Data................................................................................................................................................ 17

DataConnect Agent Runs ........................................................................................................................................ 21

Workflow Processing .............................................................................................................................................. 22

Cleanup Agent ........................................................................................................................................................... 23

Possible Errors........................................................................................................................................................... 23

DataConnect Tips ...................................................................................................................................................... 27

2. DATA INTEGRATOR ...........................................................................................29

Preparation and Understanding of Field Types...................................................................................................... 29

Data Integrator Requirements ................................................................................................................................. 35

Functional Requirements......................................................................................................................................... 35

Usability Requirements ........................................................................................................................................... 36

Creating Tab-Delimited Files ................................................................................................................................... 37

Creating a Header File for Import........................................................................................................................... 38

Non-Hierarchical Module Header File .................................................................................................................... 38

Hierarchical Module Header File ............................................................................................................................ 40

Associative Upload Header File .............................................................................................................................. 42

Uploading Data .......................................................................................................................................................... 47

Manual Uploads ...................................................................................................................................................... 47

Batch Process Uploads ............................................................................................................................................ 49

Data Integrator Tips.................................................................................................................................................. 52

APPENDIX A: COLOR VALUES ..................................................................................56

INDEX............................................................................................................................57

NOTICES .......................................................................................................................59

Privacy Policy Considerations .................................................................................................................................. 60

Trademarks................................................................................................................................................................ 61

About This Guide

IBM® TRIRIGA® solutions are built on the IBM TRIRIGA Application Platform, a Web platform that provides users with unparalleled flexibility, accessibility, and scalability. This platform uses a highly scalable Java Platform, Enterprise Edition (Java EE) architecture, a data model that allows you to extend and enhance IBM TRIRIGA products or, if properly licensed, create Modules that integrate with the pre-defined logic and processes into a feature rich, workflow-driven intelligent business system.

This document is part of the Application Building for the IBM TRIRIGA Application Platform collection of user guides. The collection is intended to provide you with an understanding of the basic tools to build or customize applications running on the IBM TRIRIGA Application Platform. The following user guides are part of the Application Building for the IBM TRIRIGA Application Platform collection.



Application Building for the IBM TRIRIGA Application Platform 3



Application Building for the IBM TRIRIGA Application Platform 3: Calculations



Application Building for the IBM TRIRIGA Application Platform 3: Data Management



Application Building for the IBM TRIRIGA Application Platform 3: Performance Framework

Conventions

This document uses the following conventions.

The phrase Application Building is used as an abbreviation for the book Application Building for the

IBM TRIRIGA Application Platform 3.



Note

– A Note provides important information that you should know in addition to the standard details.



Tip – A Tip adds insightful information that may help you use the system better.



Attention – An Attention note highlights an action that may result in damage to your system or an unforeseen loss of data.

Intended Audience

This book is intended for people who are building or customizing an application that runs on the IBM

TRIRIGA Application Platform. This document assumes familiarity with and competence in the information in the Application Building for the IBM TRIRIGA Application Platform collection of user guides.

Support

IBM Software Support provides assistance with product defects, answering FAQs, and performing rediscovery. View the IBM Software Support site at www.ibm.com/support

1. DataConnect

IBM TRIRIGA DataConnect is a tool for importing external data into the IBM TRIRIGA system. It is useful for both initial loads of data and for batch insert/update of data on a recurring basis. The DataConnect solution differs from IBM TRIRIGA Connector for Business Applications and IBM TRIRIGA Data Integrator in that it employs staging tables and workflow tasks, allowing an external source to write data directly into the IBM TRIRIGA staging tables and have IBM TRIRIGA workflow process this data for insertion into native IBM TRIRIGA business objects. In addition to pushing the data, you also can store the data into the language tables (L_ tables).

The following are the basic steps in using DataConnect.



Create

a staging table for each business object that will be populated or updated from the external source.



Make

the job business object. This business object controls the instance of the integration.



Move

data from the external source into the appropriate business object staging table and the

DataConnect Job Control table.



The DataConnect agent initiates

a process (an asynchronous workflow created for the specific integration) within IBM TRIRIGA that reads records from the business object staging tables and either inserts new instances or updates existing instances while validating the input data and executing associated business logic.

Note

– The DataConnect Object Migration packages are located in the IBM TRIRIGA installation directory in the userfiles/upgrades folder. The packages are named triDataConnectJob.zip

and triDataConnectJobLog.zip and must be applied to some upgrades depending on your application version. If triDataConnectJob >

 triDataConnectJob

object is not available on your system, you need to apply the triDataConnectJob

object migration package. If triLog > triDataConnectJobLog object is not available on your system, you need to apply the triDataConnectJobLog object migration package.

DataConnect works with an 8.x Application database. After importing the triDataConnectJob

Object Migration package, the triCreatedBy field Locator will be empty. This is not a problem and should not be modified. The DataConnect Agent populates the field with the IBM TRIRIGA user from the DataConnect Job entry.

Glossary

Understanding the following terms will help you use the rest of the information in this chapter about

DataConnect.

Term

Block Task

IBM TRIRIGA Connector for

Business Applications

Definition

Iterator task, Loop task, DataConnect task, Fork, or Switch.

IBM TRIRIGA Connector for Business Applications allows external systems to communicate with IBM TRIRIGA applications. IBM TRIRIGA

Connector for Business Applications provides a Web Service interface for updating or requesting information from the IBM TRIRIGA

Term

Controlling Block Task

Correlation Number

Custom Action

Data Integrator

DataConnect Agent

DataConnect Job Business

Object

DataConnect Job Control

Table

DataConnect Job Log

Business Object

DataConnect Task

Definition

application. This Web Service interface can be any integration middleware solution that supports current Web Service standards and protocols. IBM TRIRIGA Connector for Business Applications allows external systems to insert or update information or retrieve query results from the IBM TRIRIGA applications. It uses the same business logic that is defined in the online application and validates that required fields are inserted and read-only fields are not updated. See the TRIRIGA IBM TRIRIGA Connector for Business Applications 3

Technical Specification for more information.

The block task that controls the transaction; the outermost block configured for a transaction. Currently only DataConnect supports transactions through all tasks within its scope.

Secondary key to help with sequencing or to signify child records for a specific parent.

Actions on a business object that run code for specific state transitions. These can be a Java class method or an EJB name.

DataConnect uses two custom actions to control transitions to

Completed or Failed states.

The IBM TRIRIGA tool that processes tab delimitated flat files and creates business object records. Data Integrator can transition the record. It can be run manually to process a single file or on a schedule to process files that are in a specific directory. See the

Data Integrator

chapter for more information about IBM TRIRIGA Data

Integrator.

Runs on a schedule and monitors the DataConnect Job Control table.

When there are DataConnect jobs to run, it creates a job business object and fires a workflow event to work on that job.

The business object that controls the DataConnect integration job.

Extend the as-delivered triDataConnectJob object to support your specific integration needs. The DataConnect Job business object determines the workflow to run for the job.

The DC_JOB table. Used to identify an integration job instance. Each

DC_JOB

table instance is converted into a DataConnect Job Control business object when the DataConnect Agent picks it up for processing.

The business object that controls the DataConnect integration job log. Extend the as-delivered triDataConnectJobLog object to support your specific needs. The DataConnect Job Log business object logs information about the job during the workflow run. You also can add information as desired.

The workflow task for moving data from IBM TRIRIGA staging tables into IBM TRIRIGA records. The properties of this task determine which record type is processed.

Term

ETL Maps

External Source

Integration Flag

Kettle Spoon

Staging Table

Upsert

Workflow Context

Definition

Workflows are discussed in Application Building and information about the DataConnect task can be found in the Creating Workflows chapter of Application Building.

Maps to convert/move data from source tables to different tables. In the case of DataConnect, ETL maps can be used to move data from an external source into IBM TRIRIGA staging tables and the

DataConnect Job Control table.

Data external to IBM TRIRIGA.

A workflow property. Indicates the workflow used to move data from staging tables into IBM TRIRIGA records. The Integration flag can be tested to skip steps in a workflow. This flag indicates to the

Workflow Builder that the DataConnect step should be allowed to run in this workflow.

An ETL tool that can be used to move data from an external source to the IBM TRIRIGA staging tables and the DataConnect Job Control table. Spoon can be used to create XML transforms that move data from source tables to destination tables. It is from Pentaho and is open source.

An intermediate table written to from an external system and used to populate IBM TRIRIGA business object tables.

Action to either update or insert a record depending on whether the record already exists in the IBM TRIRIGA database.

The data for the currently running workflow.

Process Overview

Before you create or update instance records with DataConnect, you must:

 Set the

Staging tables for the business objects to be influenced by the integration.



Create a

Job Control business object .

 Create a process to

move data from the external source into the tables for integration. Use ETL

maps to move the data to the DataConnect Job Control table and staging tables; however, SQL can be used.

 Have at least one

workflow to implement business rules on the data.

Create and Maintain Business Object Staging Tables

For a business object to have a staging table associated with it, the Has Staging Table property of the business object must be on. Additionally, fields within the business object must be identified as staging

table fields with their Staging Table Field property on. Publishing a business object establishes or updates the associated staging table. A business object can have only one staging table.

How to Identify Which Business Objects Have Staging Tables

Use the DataConnect managed object in the Administrator Console to see a list of the business objects with staging tables. Click Business Objects with Staging Tables. For more information about the

Administrator Console and the DataConnect managed object, see the IBM TRIRIGA Application Platform

3 Administrator Console User Guide.

To see the staging table fields for a particular entry, select the line item and click Expand Data on the

Business Object section bar. The Fields column then shows the IBM TRIRIGA name for the field, a hyphen, and the database name for the field. If the field is a key field used for Upsert, the field name has an asterisk. DataConnect smart section keys display the section name followed by the field name.

Field definitions are useful for identifying the corresponding staging table fields for inbound data.

Business Object Property: Has Staging Table

The Has Staging Table business object property identifies business objects for which the IBM TRIRIGA system maintains staging tables. If the property is on and the business object is published, a staging table is either created or updated. If the property is off and the business object is published, the publish process deletes the staging table if it exists. By default the Has Staging Table property is off.

The Data Modeling chapter in Application Building has more information about business object properties.

A staging table is created for a business object that has the Has Staging Table selected when the business object is brought in through Object Migration and Object Migration publishes the business object.

Field Property – Staging Table Field

The field property Staging Table Field identifies fields to be included in the business object staging table. If the property is on, the field is included in the staging table. Changes to this property after a staging table has been created are not reflected in the staging table until the business object is republished. By default the Staging Table Field property is off unless the field is required. The Data

Modeling and Data Types chapters in Application Building have more information about field properties.

The Staging Table Field property is supported for the following field types: Boolean, Business Object,

Classification, Color, Date, Date and Time, Duration, List, Locator, Number, Password, Text, Time,

UOM, and Url.

Only fields that are in the General section are supported to be staging table fields. Fields in a smart section can be added to the staging table.

All required fields have the Staging Table Field property on.



Note

– If you set the Staging Table Field property for a Number field and that Number field has a related UOM field defined in the business object, be sure to set the Staging

Table Field property for the related UOM field. You must also set the staging table field property for the related UOM, if the UOM that is being imported is different from the base

UOM. For example, if the base UOM for area is square feet and the UOM being imported is acres, set the staging table field for the UOM that is imported.

You can also include formatted values for Date, Date and Time, Duration, and Time fields in staging tables. If the property for these DataConnect format fields is turned on, the field is included in the staging table. The name of the field is appended with _F in the staging table. For example, if the staging table field property for the Date field named TRIDATEDA is turned on, the field TRIDATEDA_F is included in the staging table. The format of these fields is determined by the user preferences of the user. Usage of these format fields is optional. If you want to include these format fields in the staging table, you must publish the business object to apply the change.

Field Property – Staging Table Key

The field property Staging Table Key identifies fields to be used as keys to find a record via Upsert or

Update. The Insert action does not use the Staging Table Key. By default the Staging Table Key property is off. It can be selected only when the Staging Table Field property is selected. The Data

Modeling and Data Types chapters in Application Building have more information about field properties.

The Staging Table Key property is supported for the following field types: Boolean, Business Object,

Classification, Color, Date, Date and Time, Duration, List, Locator, Number, Password, Text, Time,

UOM, and Url.

When a Business Object Is Published

Selecting the Publish business object action for a business object with the Has Staging Table property on triggers the following activities.



If there are any jobs in Processing or Ready state that use the staging table and business object, the publish fails and a message is posted to the user immediately.

 Because Publish is handled by an agent and is not an immediate action, the Publish agent checks the jobs again in case a new job has started or the state of an existing job has changed in the interim. If there are any jobs in Processing or Ready state that use the staging table and business object, the system fails to publish the business object, posts a notification to the user, and writes details about why the publishing failed to the server.log.

When a Business Object Is Deleted

Deleting a business object with the Has Staging Table property on triggers the following activities.

 If there is a job in Processing state that uses the staging table and business object, the delete fails and a message is posted to the user immediately.



If there is a job in Waiting state, the business object and staging table are deleted and the state of the job is changed to Obsolete.

Make the Job Business Object

The DataConnect Job Control (DC_JOB) table manages the jobs that move data from staging tables into the IBM TRIRIGA database. A business object must exist in the triDataConnectJob module for each integration process. Creating the business object in the triDataConnectJob module is a one-time task for the integration process. The Data Modeling chapter in Application Building describes how to use the Data Modeler to create a business object.

You need a DataConnect Job Control business object for each integration process. For example, if you are populating purchase orders from an external source, you might have a cstPurchaseOrderJob business object as your DataConnect Job Control business object and the process might pull in data for the cstPurchaseOrderBody and cstPurchaseOrderLine business objects (both with the Has

Staging Table property on).

In general, the steps are:



Open the triDataConnectJob business object through the Data Modeler.

 Create a new business object in the triDataConnectJob module.

After creating the new business object, you may want to create a form and menu navigation for the business object. DataConnect ships with a base form and a base menu for the base triDataConnectJob

business object. Creating forms is described in the Building User Interfaces chapter in Application Building. Creating menu navigation is described in the Managing Records chapter in Application Building.

The state transitions for your new business object should be inherited from the base business object and should NOT be modified. The state transitions for the triDataConnectJob base business object are as follows:



Null -> triDCStartProcessing -> In-Process

 In-Process -> triDCCompleted -> Completed



In-Process -> triDCFailed -> Failed

All fields in the first section are read-only. The second section contains log records for that

DataConnect job. The log records are triDataConnectLog business object records in triLog. Click a hyperlinked line item to see the details for that event. By default the second section shows issues found during the DataConnect run. However, when you select DataConnect Runtime in the Platform

Logging managed object in the IBM TRIRIGA Administrator Console, the second section also shows positive information about records that were inserted or updated.

Logging levels for the various agents is determined by the platform logging settings and the log4j settings on the server where the agent is running. DataConnect uses two agents. One is the

DataConnect Agent, which creates the job object and provides it to the Workflow Agent asynchronously through an event. Its job is over once the job object is created and the event posted. The second agent, the Workflow Agent, runs the business logic (the workflows) and subsequently the DataConnect tasks that create and update objects in the IBM TRIRIGA database. If you want to see positive feedback in your job objects, that is, reporting on the job object even when there are no issues, you must set the DataConnect Runtime logging on the server where the Workflow Agent is running. Setting it on the server where the DataConnect Agent is running will have no effect as the Workflow Agent is the one running the workflow and creating/updating the objects from the staging tables.

Move Data from the External Source

You must provide the mechanism for moving data from the external source to the appropriate tables in the IBM TRIRIGA database (the business object staging tables and a DataConnect Job Control table).

The mechanism must create a unique job number to reference within both the DataConnect Job

Control table and the business object staging tables and may include one or more correlation and sequence numbers.

Although there is no meaning associated with the job number, it must be a unique identifier. One approach to creating the job number is to use a GUID/UUID/FUID type generator and attempt to insert the DataConnect Job Control table entry using a state of New. If the attempt succeeds, use that as the job number for the entries written to the staging tables. If it fails with a duplicate key error, generate another GUID/UUID/FUID and try again.

Use Spoon product from Kettle, similar to how IBM TRIRIGA Workplace Performance Management and

IBM TRIRIGA Real Estate Environmental Sustainability products use Spoon to create ETL mappings for

Fact Tables pulling data from IBM TRIRIGA tables.

Staging Tables

In addition to the fields you identify with the Staging Table Field property, each staging table includes the following fields.

Column Name Key Type Description

DC_JOB_NUMBER

DC_CID

NUMBER(20)

DC_SEQUENCE_ID

DC_STATE

Y NUMBER(20)

N NUMBER(4)

DC_ACTION

DC_PATH

NUMBER(4)

VARCHAR()

N A job identification number.

N A correlation identification number used for parent-child relationships coming in from staging tables.

N A sequence number for a group of entries.

N A state identifier to indicate the current state of this entry during the processing of a DataConnect Job. Values are:

0 – New

1 – Ready

2 - Processing

3 - Completed

4 - Failed

5 - Obsolete

N The action to be taken. Values are:

1 - Insert

2 - Update

4 - Upsert

Y The path (including Record Name) for the object being acted upon.

Column Name Key Type Description

DC_GUI_NAME

DC_PROJECT

Column names are determined by the

IBM TRIRIGA

Application

Platform. Use the

Administrator

Console to find the staging table name and the database name.

For an Insert, this field can indicate to the DataConnect task where in the hierarchy to place the record. Children should have the parent path in

DC_PATH

. If an object is at the top of the hierarchy or if the object is not within a hierarchy, DC_PATH should be empty.

For an Update, if no keys exist, this field should be set to the record you want to update, the IBS_SPEC.OBJECT_PATH of the record to update.

N VARCHAR(100) Y The form name you want the new or updated row to use. On insert, if this is not set, the system uses the default form for the business object if there is one.

Otherwise, the object will have no associated form. On update, if this is not set, the system does not modify the current form setting.

N VARCHAR(100) Y On Insert or Update, sets the project for the record if it is not empty. The value is what appears in the portal menu bar when a project is selected. DC_PROJECT should use the format ProjectID-

ProjectName

Only the record involved in the Insert or

Update will have its project changed. No changes occur to children of that record.

If you are inserting or updating data with a parent / child relationship, be sure to set the appropriate project for all data you want changed.

N Depends on the IBM

TRIRIGA

Application

Platform column type

Y Fields in the business object with Staging

Table Field checked (on). See the Staging

Table Field discussion in this chapter for

more information.

An additional column is added in the staging table for Date, Date and Time,

Duration, and Time fields. The name of these fields is appended with _F.

Based on the Staging Table Key property, the fields could be keys. Used by Upsert and Update.

Column Name Key Type

JOB_NAME

JOB_RUN_CTL

Description

For an Upsert, the DataConnect task generates SQL on the fly using the designated key values in the staging table to see if the record exists. If the record exists (that is, one record is found), the record is updated with the data in the staging table. If more than one record exists, the staging table row is marked Failed. If the record does not exist (that is, no records are found), a new record is inserted using the staging table data.

For an Update, if keys are defined on the staging table, the DataConnect task generates SQL on the fly using the key values in the staging table to see if the record exists. If the record exists (that is, one record is found), the record is updated with the data in the staging table. If more than one record exists, the staging table row is marked Failed. If the record does not exist (that is, no records are found), the staging table row is marked Failed.

DC_JOB Table

The DC_JOB table is filled by the external process, updated by the DataConnect Agent and the asynchronous workflows that act on it, and deleted by the Cleanup Agent. The fields in the DC_JOB table are as follows:

Column Name Key Type Description

JOB_NUMBER

JOB_TYPE

NUMBER(20)

NUMBER(4)

N VARCHAR(100)

N A job identification number.

N An identifier of the type of job. Value is:

1 - Create/Update IBM TRIRIGA records, inbound data

Y The name of the job.

N Controls concurrency. The DataConnect

Agent uses the JOB_RUN_CTL in conjunction with the JOB_NAME to throttle itself. If a Job is single threaded, no other job with that JOB_NAME will run until the running job is Completed or

Failed. Values are:

1 - single threaded

2 - multithreaded

Column Name Key Type Description

BO_NAME

USER_ID

STATE

SOURCE_SYS_ID

PROCESS_SYS_ID

CREATED_DATE

UPDATED_DATE

VARCHAR(100)

NUMBER(20)

Y The business object name for the record that will be created to represent the job.

This business object must be created in the triDataConnectJob Module.

Y User ID to use for processing. Must be a valid IBM TRIRIGA user ID (T_MYPROFILE

ID).

NUMBER(4)

VARCHAR(100)

DATETIME

0 N A state identifier to indicate the current state of the job. Values are:

0 - New

1 - Ready

2 - Processing

3 - Completed

4 - Failed

5 - Obsolete

Y Identifier for the system that created the job (used for informational reporting).

Y Identifier for the system that is processing/processed the job (used for informational reporting). You do not need to provide this information.

Date N The datetime the entry was added

(defaulted).

Y The datetime the entry was last changed.

You do not need to provide this information.

Use the Administrator Console to see the contents of the DC_JOB table. Select DataConnect Jobs in the DataConnect managed object in the Administrator Console. For more information about the

Administrator Console and the DataConnect managed object, see the IBM TRIRIGA Application Platform

3 Administrator Console User Guide.

Create ETL Maps and Run Mapping

Your implementation of a DataConnect integration determines how you move your data into the staging tables, either through an ETL tool or direct SQL statements. Somehow you move your data from the external source into the staging tables. The data must be populated in the external source before running the mapping.

If you use an ETL tool, the integration drives which ETL maps you must develop to move data from the external source to the staging tables. Use a tool to define an ETL mapping between the external source and the staging tables. One such tool is Kettle’s Spoon or you may use another tool of your choice.

When the mapping runs, it must accomplish the following:



Create a unique job number.



Create an entry in the DataConnect Job Control table (DC_JOB), setting the state to New.

 Populate the staging tables for the business objects, including any child staging tables. The entries include the job number, one or more correlation numbers, and the action. The path is required on Update actions where no keys are defined and optional for Insert actions.



Optional: Update the entry in the DataConnect Job Control table (DC_JOB), setting the state to Ready. This can be done manually through the Administrator Console by selecting the job and then the action Ready Job.

Use Workflow to Move Data

Use the workflow capabilities from IBM TRIRIGA to move data from the staging tables to the actual business object records. The exact workflow logic depends on your company business rules and the business object being filled. See Application Building for detailed descriptions of workflows and the

DataConnect task, including how they are used in IBM TRIRIGA.

Workflows for controlling DataConnect must be created in the triDataConnectJob module.

Use the DataConnect workflow task to retrieve staging records and iterate through the staged entries, creating or updating a smart record for each one. DataConnect workflow tasks can be nested to support parent-child relationships and business logic.

The DataConnect task acts in two parts.



Retrieves the set of records to work on from the appropriate staging table as defined by the

DataConnect task property settings.

 Acts as an iterator.

 Creates a new record or updates a record for each row of data from the staging table.

 Runs the body of the DataConnect task for each row. Other task steps may be positioned within the body of the DataConnect task. Please see the Creating Workflows chapter in

Application Building for more information about the DataConnect workflow task.

While inside a DataConnect workflow task, the Review Status of the task is INSERT, UPDATE, or FAILED, based on the current action of the task iteration. On exiting a DataConnect workflow task, the Status is

Completed or Failed (if any of the staging table rows failed).

The following example uses a purchase order (PO) and its PO line items for purposes of illustration.

First is the complete workflow.

Example Workflow Using DataConnect Task

Workflow Tasks

The following discusses key properties of the workflow tasks in the example above. For more details, refer to Application Building.

Workflow Builder

In the Workflow Builder, create a workflow in the triDataConnectJob module.

Start Task

Start Task

Step 1

In the workflow Start task, Name the workflow using the naming standards in the

Creating Workflows chapter in Application Building.

Step 2

Set Concurrence to Asynchronous.

Start Task

Select the Integration flag, which allows DataConnect tasks to be a part of the workflow.

Step 3

Set Module to triDataConnectJob.

Step 4

Set Object Type to the business object you created in the triDataConnectJob module.

Step 5

Set Event to SYSTEM DC PROCESS JOB. This is the DataConnect event.

Step 6

Select the Propagate Integration Status flag, which specifies to propagate the

Integration flag when calling another workflow from this one.

The triDataConnectJob-related Object Type and the Event of SYSTEM DC PROCESS JOB identify the correct workflow for the DataConnect Agent. The DataConnect Agent pulls the BO_NAME field, which should be the name of the triDataConnectJob-related Object Type from the DC_JOB entry for the job.

DataConnect Task

The DataConnect task retrieves records from the staging table and iterates through the staged entries, creating or updating a smart record for each one.

DataConnect Task

Step 1

Create the workflow body, using the DataConnect task where appropriate.

Step 2

Enter a Label and Description.

Step 3

Specify the Module and Business Object on which the DataConnect task will work.

Step 4

Make Initial State the state in which a new record should be created. The DataConnect task ignores the value of Initial State if the record is being updated. The choices in the drop-down box are the states defined for the business object.

The record is created in that state; it is not transitioned to that state. If you want the record to be transitioned, have the DataConnect task create the object in the null state and use a Trigger Action task within the DataConnect task body to transition the record to the desired state.

Step 5

Select Use Temporary Data when the DataConnect task is to use temporary data. When a DataConnect task with Use Temporary Data checked applies incoming data, the platform inserts or applies updates to temporary data instead of to the permanent

data. See Use Temporary Data

for examples and more details.

DataConnect Task

Step 6

In the Correlation section:

 In-Sequence indicates that the correlation on the staging entry and the sequence number are used to determine the order by which the records are processed. This is the default and the most common scenario unless you have DataConnect tasks one within another.



Task Step indicates that the correlation should come from an enclosing

DataConnect task and should be used in the Where clause to determine what entries to process (Job Number + Correlation Number, ordered by Sequence

Number). Using Task Step allows the DataConnect task to get the correlation number from one of the enclosing DataConnect tasks.

Step 7

Select the scope in the Transaction section.



None: The record is committed right after it is created or updated.

 Per X Iterations: A new context is created for each X iterations and committed when that number of iterations is complete.



All Iterations: A new context is created before the task starts any processing and is committed when all iterations are complete.

Trigger Action Task

The last step in the workflow should be a Trigger Action task to mark the job Completed or Failed.

 In Trigger Action Task Properties, enter Mark Job Completed in the Label field, and select

triDCCompleted for the Action field.



In the Records section, select Business Object for the Take the field and Start

(loDCPurchaseOrder) for the of Task field. Mark the Use it option and select loDCPurchaseOrder for the Object Type field.

Break Task

For transactions to rollback or commit, put a Break (or Continue) in the DataConnect body. With the

Break (or Continue) task, you can check the DataConnect task status, determine what block to go to, and decide whether to keep iterating or not (Continue versus Break) and whether to commit or rollback

(Success or Failure). The transaction setting on the DataConnect task works in conjunction with the

Break (or Continue) task settings.

The Break also has settings to control temporary data. Find more details and examples in Use

Temporary Data

If the DataConnect task is not using transactions, rows are committed with each task step. If you do not hit a Break (or Continue) that changes the processing flow and you are using transactions, a commit takes place based on the DataConnect task settings.

If the DataConnect task is using transactions and the workflow Breaks (or Continues), the transactions roll back or commit based on the Success or Failure setting on the Break. Processing continues at the scope indicated in the Break (or Continue). Current Scope means the DataConnect task you are in;

Break Flow Type means continue processing at the task after the DataConnect task; Continue Flow

Type means continue onto the next row in the staging table batch for this job; Selected Block means

Break (or Continue) at that DataConnect task.

Workflow Calls Another Workflow

If the workflow using the DataConnect task calls other workflows, those workflows may need to be updated to check the Integration setting and to change the Propagate Integration Status flag as appropriate. Analyze existing workflows that may be called by the workflow for data movement. If there are steps within the workflow that should not be performed if called from an Integration workflow, check the Integration setting with a Switch task or in the Start task. If the workflow should not propagate the Integration status, clear the Propagate Integration Status flag in the Start task.

The value of 1 for the Integration flag means the Integration Flag is on.

Runtime Data Validation

DataConnect runtime does some data validation. For example, if there is a UOM field in the staging table and its value is invalid. When the DataConnect task encounters a failed row, it stops processing and fails the DataConnect iteration (not the entire DataConnect task). The row is marked in the staging table as Failed. At this point you have a choice, whether to continue with the next iteration or to break out of the task. Place a Break into the workflow to tell the workflow how to proceed. If no Break is added, the DataConnect task will continue processing.

Specific built-in runtime data validations include:



UOMs and locator values are valid.

 Data type is valid for the field.



Form Name is valid for the object.

 Project name, if provided, is valid for the system.

You can add more data validation and cleansing. See Use Temporary Data

for details.

Status

As with most iterators, the DataConnect task remembers information from its last iteration, including status. If a workflow has two DataConnect tasks, one within the other, and you want to make sure to set the status appropriately at the end if a row fails within an inner iteration, save that row in a variable. This ensures that the status is not lost.

Use Temporary Data

Checking the Use Temporary Data property in a DataConnect task allows the workflow tasks within the

DataConnect task to process the incoming records from the staging table in session. This means the data can be manipulated before it is saved to the database. Use a Get Temp Record task to retrieve a temporary record and use a Save Permanent Record task to save the current session changes. The

Break task can remove a specific iteration from the session, clear the session completely, fail a specific staging table row, or fail all staging table rows since the last Save Permanent Record.

When Use Temporary Data is checked, as the workflow processes, the record for the DataConnect task is the permanent record (without changes from the staging table) and the record for the Get Temp

Record task is the updated or incoming values from the staging table. This enables data validation and cleansing of incoming values within the workflow and even data mapping on update if fields are blank in the staging tables.

Example of Single DataConnect Task Using Temporary Data

The following excerpt from a workflow shows a single DataConnect task processing temporary data.

Single DataConnect Task Using Temporary Data

In the DataConnect task,

 Use Temporary Data = checked

In the Break task,



Flow Type = Continue



Task Status = Success

 Break Scope = Current Scope



Discard Temporary Data = All

 Fail Staging Rows = checked, which sets All

The DataConnect task iterates through the staging table records, placing the inserted or updated records into the session. For each iteration, the workflow gets a temporary record, does something with it, and decides whether it should be kept or not.

To keep the record, save the record permanently with a Save Permanent Record task.

The Switch task fail condition uses the settings in the Break/Continue task to discard all session data, fail the staging rows used to generate the record in session, and continue to the next record in the iteration.

In this example of a single DataConnect task, setting Discard Temporary Data in the Break task to

Current

instead of All would have the same outcome.

Example of Multiple DataConnect Tasks Using Temporary Data

The following excerpt from a workflow shows a DataConnect task containing a second, child

DataConnect task. This is an extension of the purchase order body and purchase order lines example shown earlier.

Multiple DataConnect Tasks Using Temporary Data

In the outer DataConnect task:

 Use Temporary Data = checked

In the outer DataConnect task’s Break task,

 Flow Type = Continue



Task Status = Success



Break Scope = Current Scope

 Discard Temporary Data = All



Fail Staging Rows = checked, which sets All

The outer DataConnect task iterates through the POBody staging table. For each iteration, the workflow gets a temporary record, does something with it, and decides whether it should be kept or not. If there is an issue with the POBody record, the workflow executes the Break/Continue and discards all session data, fails the row, and moves on to the next POBody record.

If the POBody passes the Switch task, processing continues with the inner DataConnect task to handle the POLines for this POBody. The workflow could do one of several things based on what it finds and on the setting of the Break. You could have several Switch/Breaks and depending on your processing fail everything or just the one row.

The following shows different uses of the Break and what would happen with processing and with the session:



Fail the current POLine, remove the record it created or modified from the session, and fail that staging table row only. Processing continues on the next POLine for the POBody. These properties in the inner DataConnect task’s Break task accomplish that:

 Flow Type = Continue

 Task Status = Success

 Break Scope = Current Scope

 Discard Temporary Data = Current

 Fail Staging Rows = checked, which sets Current



Clear the current POBody iteration, remove the record it created or modified from the session, fail the POBody staging table row, clear the POLines, and fail the POLines staging table rows.

Processing continues on the next POBody. These properties in the inner DataConnect task’s

Break task accomplish that:

 Flow Type = Continue

 Task Status = Success

 Break Scope = Selected Block, Process Incoming POBody

 Discard Temporary Data = All

 Fail Staging Rows = checked, which sets All

 Nothing (no Break task in inner DataConnect).

Having the Save Permanent Record task after the end of the inner (POLine) DataConnect task ensures that the platform saves the POBody and the POLines that go with the POBody.

DataConnect Agent Runs

The DataConnect Agent checks entries in the DataConnect Job Control table and runs those in Ready state. The DATACONNECT_SLEEP_TIME property for the application server running the DataConnect

Agent controls how often the agent checks the DataConnect Job Control table in the

TRIRIGAWEB.properties

file. The IBM TRIRIGA Application Platform 3 Administrator Console User

Guide describes the Agent Manager panel. For more information about the TRIRIGAWEB.properties file, go to the IBM TRIRIGA Application Platform 3 Knowledge Center and select Installing IBM TRIRIGA

Application Platform > Configuring the platform > Properties files.

When it finds a job in the DataConnect Job Control table (DC_JOB) in the Ready state, the

DataConnect Agent takes the following steps:

 Creates a record for the record indicated in the DataConnect job entry. This record is an instance of the business object created in the triDataConnectJob module. This is the Job

Control business object.



Populates the record with information from the job entry, including job number, job type, source system ID, process system ID, create date, updated date, user, and state.



Sets the state of the DataConnect Job Control table entry to Processing.



Posts the workflow event SYSTEM DC PROCESS JOB along with the record, which fires the workflow.



If there are issues with the DataConnect Job Control table entry or with creating the Job

Control business object, the state for the DC_JOB table entry is set to Failed. The reason for failure is logged in the server.log. When the issue is fixed you can try to perform the job again from within the Administrator Console.

Workflow Processing

When a workflow with event SYSTEM DC PROCESS JOB and module and business object of your Job

Control business object type executes, the DataConnect task in the workflow performs the following steps:

This assumes the workflow was defined with the Integration property on (true).



Use the configured information to determine what type of record to process, the state to leave the new record, the correlation type, and the transaction setting. Use the record on the Start task as the source for the job number to use.



In the DataConnect tasks, get the staging table name for the record it should process and query for rows with the corresponding job number. Depending on the configuration setting for the correlation number, use the correlation number to order by or use the correlation number in a where clause to find entries with the correlation number. Get the staging table entries based on job number, correlation number, sequence number, action, and path.



For each row fetched, create or update a smart record. Put the record in the state dictated in the DataConnect task definition (it is not transitioned to that state). Change the state of the row in the staging table. Set the project and form. Depending on the Use Temporary Data flag, the changes could be permanent or made in session. Run steps defined within the iterator, creating parent-child relationships and following business logic processing.



Run the pre-create workflow on the new records. If you do not want the pre-create workflow to run, update the pre-create workflow to check the Integration flag, so when called by an

Integration workflow it is skipped.

 If there is a call to another workflow, check the setting of the Integration status to change processing based on whether the workflow was called from an Integration process.

 If there is an event, check the FROM_INTEGRATION column to determine if the Integration status is set in the workflow context.



Process the transaction based on the DataConnect task configuration setting.

Cleanup Agent

The Cleanup Agent performs automatic cleanup of DataConnect jobs. Use the Agent Manager panel in the Administrator Console to define the name of the processor running the Cleanup Agent. Set the

DC_HISTORY_RETENTION_DAYS

property in the TRIRIGAWEB.properties file. The IBM TRIRIGA

Application Platform 3 Administrator Console User Guide describes the Agent Manager panel. For more information about the TRIRIGAWEB.properties file, go to the IBM TRIRIGA Application Platform 3

Knowledge Center and select Installing IBM TRIRIGA Application Platform > Configuring the platform >

Properties files.

When a DataConnect entry in the DataConnect Job Control table is in Obsolete or Completed state and has been in that state for the number of days in the DC_HISTORY_RETENTION_DAYS property, the

Cleanup Agent:



Finds the job control number of the entry and deletes entries in the staging tables for that job control number.

 Deletes the DataConnect Job Control table (DC_JOB) entry.

The Cleanup Agent does not delete the Job Control business object entries (the business object in triDataConnectJob

). Follow your company’s business rules for managing triDataConnectJob entries that are no longer needed. You can delete them from the system directly or with a menu that you create.

Possible Errors

Error Message Where Found

A business object: XX, does not have staging table.

XX = business object name

From the DataConnect task runtime when the workflow is executing, a

WFRuntimeException

in the server.log

An error occurs while processing the staging records:

XX.

From the DataConnect task runtime when the workflow is executing

XX = staging table record fields

Cannot delete the BO, there are Pending DataConnect jobs.

User Interface

How to Fix

The staging table is missing for the XX business object. Task will fail in the workflow.

Something wrong has occurred during the DataConnect task execution. One possible issue is on update the DC_PATH did not accurately define a smart object to update.

The Processing DataConnect job needs to complete before the business object can be deleted.

Delete when the processing job is completed.

Error Message

Cannot publish the Business

Object at this time. There are

DataConnect jobs either

WAITING or PROCESSING. These must complete first before the publish can complete.

Where Found

User Interface

How to Fix

A Processing or Waiting

DataConnect job needs to be cleared out before the Publish of the object will be successful.

Both should clear out without intervention. Republish when there are no pending jobs.

DataConnect Agent - The triDataConnectJob Business

Object cannot be found. Make sure to import before using the

DataConnect Agent.

From the DataConnect Agent, a

PlatformRuntimeException in the server.log

Apply the latest triDataConnectJob

object migration package before running or using DataConnect.

DataConnect Agent - The triDataConnectJob Business

Object is not the correct version. Make sure to import before using the DataConnect

Agent.

From the DataConnect Agent, a

PlatformRuntimeException in the server.log

Apply the latest triDataConnectJob

object migration package before running or using DataConnect.

DataConnectTaskHandler - The

Staging table: XX does not contain the staging field: YY.

From the DataConnect task runtime when the workflow is executing, a warning entry in the server.log

XX = staging table name

YY = field name

DataConnectTaskHandler:

Cannot find a parent smart object whose object path is:

XX.

XX = DC_PATH from the staging table

From the DataConnect task runtime when the workflow is executing, a warning entry in the server.log

DataConnectTaskHandler: performing transition action with an invalid state, XX on the

SmartObject: YY.

From the DataConnect task runtime when the workflow is executing, a

WFRuntimeException

in the server.log

XX = state

YY = smart object ID

The staging table is missing fields that are expected. This may happen if the staging table is out of sync with the business object. Republish the business object when the DataConnect

Job is completed.

The runtime has not been able to find the parent identified in the staging table field

DC_PATH

. The smart object is created; just the parent is not set for the smart object.

Check the job’s business object state transitions to make sure it has the state transitions defined in the base triDataConnectJob

Error Message

From the DC_JOB table

XX = BO_NAME

YY = JOB_NUMBER

Where Found

The bo for the job YY, XX is not a valid Bo.

From the DataConnect Agent, an info entry in server.log

The JOB_RUN_CTL XX is not valid for job YY.

From the DC_JOB table

XX = JOB_RUN_CTL

YY = JOB_NUMBER

From the DataConnect Agent, an info entry in server.log

The JOB_TYPE XX is not valid for job YY.

From the DC_JOB table

From the DataConnect Agent, an info entry in server.log

XX = JOB_TYPE

YY = JOB_NUMBER

Unable to associate user ID BB for the job YY using the bo XX.

From the DataConnect Agent, an info entry in server.log

From the DC_JOB table

BB = USER_ID

XX = BO_NAME

YY = JOB_NUMBER

Unable to create the bo for the job YY using the bo XX.

From the DataConnect Agent, an info entry in server.log

From the DC_JOB table

XX = BO_NAME

YY = JOB_NUMBER

How to Fix

The BO_NAME column in the

DC_JOB

table for job YY is not a valid business object. Check to make sure the name is correct and is in the triDataConnectJob

module.

Fix the entry and reset the

STATE to READY (or 1).

The JOB_RUN_CTL column for job YY is invalid in the DC_JOB table. Only 0 (single threaded) or 1 (multi-threaded) are valid entries. Fix the entry and reset the STATE to READY (or 1).

The JOB_TYPE column for job

YY is invalid in the DC_JOB table. Only 1 (in-bound) is valid. Fix the entry and reset the STATE to READY (or 1).

The agent was unable use the

USER_ID

column to associate the newly created DataConnect

Job to the user. Make sure the

USER_ID

is a valid

T_MYPROFILE

spec_id. Fix the entry and reset the STATE to READY (or 1).

The agent was unable to create the DataConnect Job smart object. Check to make sure the name is correct and is in the triDataConnectJob

module.

Fix the entry and reset the

STATE to READY (or 1).

Error Message

Unable to map fields necessary for DataConnect to work properly for the job YY using the bo XX.

From the DC_JOB table

XX = BO_NAME

YY = JOB_NUMBER

Where Found

From the DataConnect Agent, an info entry in server.log

How to Fix

The agent was unable to map the necessary fields to the newly created DataConnect

Job. Check to make sure the XX business object has all the fields defined in the base triDataConnectJob

BO. Fix the entry or business object, possibly republish if the business object changed, and reset the STATE to READY (or

1).

Unable to post workflow event

SYSTEM DC PROCESS JOB for the job YY using the bo XX.

From the DC_JOB table

XX = BO_NAME

YY = JOB_NUMBER

Unable to Publish, there are existing DataConnect jobs processing or waiting for the

Business Object.

Unable to transition job to triDCStartProcessing for the job

YY using the bo XX.

From the DC_JOB table

XX = BO_NAME

YY = JOB_NUMBER

From the DataConnect Agent, an info entry in server.log

The agent was unable to post a workflow event for the newly created DataConnect Job. This could be because there is no published workflow for the

SYSTEM DC PROCESS JOB and the XX business object.

Check to make sure the workflow is published for the event and the XX business object. Fix the entry or workflow and reset the STATE to READY (or 1).

From the Publish Agent, a

PlatformRuntimeException in the server.log

The Processing or Waiting

DataConnect job needs to be cleared out before the Publish of the object will be successful.

Both should clear out without intervention. Republish when there are no pending jobs.

Check the state transitions for business object XX to make sure

XX has the state transitions defined in the base triDataConnectJob

Module triDataConnectJob.

Fix the entry or business object, possibly republish if the business object changed, and reset the STATE to READY (or

1).

DataConnect Tips

Please use the following DataConnect tips to enhance how your company uses IBM TRIRIGA

DataConnect.

Tip

Approval Task

Description

An Approval task cannot be executed from within a DataConnect task.

DataConnect job remains in

Processing state; workflow probably does not contain a

Trigger Action to move job to Failed or Completed.

If a DataConnect job remains in Processing status, check the workflow instance for the job. It is likely that the workflow has completed but there is no workflow task to transition the job to

Completed or Failed.

DC_PATH

column in the staging table can be used to set a hierarchy on insert. On update, the DC_PATH must be the record to update.

The DC_PATH column in the staging table needs to be used during an update. The DC_PATH should be set to the record you want to update. It should be the IBS_SPEC.OBJECT_PATH of the record you want to update.

DC_PATH

used with Insert can indicate to the DataConnect task where in the hierarchy to place the record. The children should have the parent’s path in the DC_PATH; with this, the DataConnect Task will set hierarchy. The objects at the top of the hierarchy or objects not in a hierarchy should have nothing in the DC_PATH.

Execution Schedule

Issues found by the

DataConnect Agent are logged to the server.log

To avoid impacting database connections and threads available to the Application Server, schedule DataConnect jobs to run during nonpeak hours.

Most of the common issues with entries in the DataConnect job table

(DC_JOB) are handled by the DataConnect Agent. The issue is logged in the server.log and the DataConnect job entry is set to Fail.

Additional information can be obtained by turning on debug logging in the DataConnect Agent.

Once the issue is resolved, the job can be retried from the

Administrator Console by using the Retry Job link.

Job Control Business Object The business object for the workflow containing the DataConnect task must be a subtype of triDataConnectJob, or in other words created in the triDataConnectJob module. If not, the system generates an error.

Number Field’s UOM If you set the Staging Table Field property for a Number field and that Number field has a related UOM field defined in the business object, be sure to set the Staging Table Field property for the related UOM field.

Tip

Staging Table Fields come from General section

Staging Table Name

Suspended Workflow

Description

Only fields that are in the General section are supported to be staging table fields at this time. If you want to pull data from other sections, use temporary fields in the business object and use the workflow engine to move the data into the appropriate non-General fields, or use a helper business object to pull all your data and use the workflow engine to move the data into the real business object.

Use the DataConnect managed object in the Administrator Console to find out the staging table name. Click Business Objects with Staging

Tables. The database field name is listed along with other information. The Expand Data link provides more information for the selected rows, such as field names in the staging table and jobs that are using the staging table.

A workflow cannot be suspended while processing a DataConnect task.

The DataConnect process can handle business object with smart section association.

If the business object in the integration has a smart section that needs to be populated through the integration, add an extra field to the business object and make it a staging table field. Populate that field in the staging table with the identifier that can be queried to find the record you want associated to the new smart object. In the workflow, Query the record and Associate it to the new smart object.

The Initial State is what the smart object is created in; it is not transitioned.

If the integration wants the smart object transitioned to the state, set the Initial State to null in the DataConnect task. Within the body of the DataConnect task, use a Trigger Action to transition the object to the desired state.

Tracking Errors You may want to create a record that is specifically used to track errors encountered during workflow validations. One approach is to create a cstDataMigrationError business object in the triIntermediate

module. Fields in this business object can include triCreatedSY (system created date time), triNameTX

[Error Type]

(stores the migration that caused the error), triLinkedRecordTX

(a locator field that links to the actual record that failed validation; remember to define the association between the error business object and any other business object being used in the validation); and triDescriptionTX [Error Message]

(holds the details of the specific error). Create a form for the data migration error record.

User Action Task

Workflow Exceptions

A User Action task cannot be executed from within a DataConnect task.

The DataConnect task fails if the workflow encounters a hard exception during workflow processing.

2. Data Integrator

IBM TRIRIGA Data Integrator is a tool for importing data into IBM TRIRIGA business objects without writing technical scripts. The Data Integrator tool gives the ability to capture groups of softcopy records of data and upload them into to the system without having to enter each data record manually into the system one at a time.

Record data can come from any source, such as a database, Excel file, or any other external system.

Regardless of your data source, Data Integrator only accepts files in a tab-delimited format (.txt).

Importing a preexisting database saves time and avoids the data integrity and quality issues inherent in manually keying records.

Data Integrator always imports data into the TRIRIGA base language tables. Import into the localized tables is not supported.

The Data Integrator can create associations between two records. The parent record must exist in the system before creating this association. Precise planning is vital when structuring your business objects in the IBM TRIRIGA system. The way you design the modules determines what data you can import and associate together.

An additional function of Data Integrator is that it can be used to do mass updates of existing records.

If a business object’s value needs to be updated across multiple existing records, use Data Integrator to alter the field values in bulk, rather than having to work through the form and perform the numerous mouse clicks required to update each record individually.

With all Data Integrator uploads, a notification is sent to the user who performed the upload. The notification informs them of the success of the upload or that errors occurred. To receive a notification, the user must have the Notifications portal section in their Home Portal.

There are a few system policies to ensure your data is properly imported. The first row of your tabdelimited file (.txt) (that is, the Header File) must contain a column for each field name in the business object for which you will be importing or updating data. To facilitate creating the Header

File, Data Integrator has a built-in tool for creating an Excel-compatible file containing the field names in the first row. This Header File becomes the upload template into which all records are entered.

There are two distinct steps in using Data Integrator:

 Create tab-delimited files (the Header File and data)



Read data from the tab-delimited files

Preparation and Understanding of Field Types

Before utilizing the Data Integrator to upload new data records, it is important to understand the use of fields presented in the IBM TRIRIGA platform. More information about field types and naming standards can be found in Application Building.

Use the table below to reference which field types can be used with Data Integrator as well as the accepted formatting of the record data for the field. Failure to follow these procedures results in a partially successful or a failed upload of the records.

If the table does not have an example in the Example column it indicates that the field type is not supported by Data Integrator; refer to the Additional Comments column for the reason.

Field Type Suffix Use Import

Format

Additional

Comments

Example

Action Button AB An onscreen image used as a clickable button.

Boolean BL

Not for use with Data

Integrator. Defined in the Form Builder only. Modification required. triReservableBL

= TRUE

Business

Object

Used to contain a value that is either

TRUE or FALSE.

FALSE = not selected

TRUE = selected

Used to associate a

Location,

Organization, or Geography with a record.

Not for use with Data

Integrator. System generated by workflow.

BOTypeName

(Business Object

Type)

FullPath (Full

Path)

RecordName

(Record Name)

Classification

Rollup

Used to contain a value selected from a defined hierarchy of values.

Exact spelling and case.

Used for rolling up sums by classification.

SysModifiedDate

Time (Update

Time)

Hierarchical Module;

Requires Header File to contain: Parent =

\Classifications and full path of child

Classification. Do not end the path with a slash (\).

A New

Classification object requires a hierarchical upload.

Not for use with Data

Integrator. Populated by the system either by workflow or by the Rollup action.

Field Type

Color

Control

Number

Date

Suffix Use

Date and Time DT

Duration DU

Used to contain a color.

Used to generate and contain unique

ID numbers.

Used to contain a date.

Used to contain a date and time combination

Import

Format

Color value with exact spelling and case.

Default date format.

Long integer in milliseconds.

Used to contain the length of a time interval.

Described as

#Years#Months

#Weeks#Days#

Hours#Minutes#

Seconds.

Expressed in the following format:

8.x - use

#w#d#h#m#s

9.x – use

#y#M#w#d#h# m#s

10.x – use

#y#M#w#d#h# m#s

Additional

Comments

Example

Enter color values using a # followed by the hexadecimal value for the color.

For example, black is

#000000

. See

Appendix A

for more information about color values.

Control Numbers are populated by the system on Publish or

Create, based on criteria specified on the business object’s

Publish tab.

Not required if

Control Number field auto-populated by workflow or set to populate by default.

Also see the tip below this table. triStartDateDA =

01/01/2006

The formats for 8.x,

9.x and 10.0 are different. Be sure to adopt the proper format for your system. triDateDT representing

08/21/2006

15:30:00 =

1156199400000 cstDurationDU representing 5 days, 6 hours, and 30 minutes

= 5d6h30m0s

Field Type

Financial

Rollup

Image

Label Only

List

Note

Number

Suffix Use

Used to contain totals from other fields involved in financial transactions.

Used to contain numbers.

Import

Format

Used to contain an image.

Used to specify a label with no corresponding field.

Exact spelling and case.

All images must already exist in

<root install folder>/userfile s/images

folder before executing the upload. Image pathing down to the filenames must match exactly. triImageIM=

//Company-

1/fileMyImageN ame.jpg

All image file names must begin with file

in order to display properly.

Not for import.

Defined in Form

Builder only.

Modification required. triMonthLI =

December

Used to select a value from a list of values.

Used to contain arbitrary length formatted text.

Note fields are not supported by Data

Integrator.

Straight numeral with decimal place setting, as necessary.

Additional

Comments

Not for use with Data

Integrator. Populated by the system either by workflow or by the Rollup action.

No commas, $, or formatting required, unless decimals exist.

Example

triGrossMeasure dNU = 12000.5

Field Type

Password

System Read

Only

Text

Suffix Use

Import

Format

Additional

Comments

Used to contain text value that can be modified but not displayed in a user interface.

Alphanumeric text.

Encrypted.

Used to access information about a record and the business object used to create it.

Populated by

IBM TRIRIGA system workflows.

Not required to be imported.

Used to contain text values.

Straight text.

Time

UOM

URL

TI Used to contain a time of day.

Upload as long integer in milliseconds.

Used to associate a unit of measure to a field or business object.

Used to contain a URL.

Exact spelling and case.

Full directory pathing, including http, ftp or \\.

Example

The only limitation is the character limit of the business object’s field. If uploading 30 characters to a

20 character field, 10 characters will be truncated. triStartTimeTI representing

9:30:35 PM =

77435000

Field Type

System

Tab::Record

Information

Section

Suffix Use Import

Format

Additional

Comments

These fields are generally populated by workflows and there is a risk of creating a workflow loop by uploading data into them (that is why they are readonly in many areas).

They include:

LocationName

(Graphics Location)

OrgName

(Organization Name)

GeographyName

(Geography Name)

Example

Locator Field A field reference to another business object record.

Requires full path for hierarchical records or the value that is displayed in the field.

Might require an associative upload if the name displayed is not unique in the result set of the query used by the locator field.



Tip – In the Mapping Properties of the Data Modeler BO Mapping tool is a place to indicate which fields contain the Publish Name of a record created from the business object. The first 100 characters of the combination of values in fields that make up a record Publish

Name uniquely identify the record among all records created from the same business object. Tab-delimited files to be processed by Data Integrator must include fields to provide unique values for the Publish Name fields. If the values in fields that make up a record’s Publish Name are provided for an existing record, Data Integrator updates the existing record with the additional field values provided in the uploaded file.

If an uploaded tab-delimited file does not provide unique record Publish Names, Data

Integrator may fail to create a record, it may create a record when it was supposed to update a record, or it may update the wrong record. If the first 100 characters specified in a tab-delimited file for a record name match a record but some of the record name after the first 100 characters does not match, the system rejects the record and no update or create occurs.



Tip – Fields tied to the sums in multiple-record smart sections specify the fields in the table should be summed. The system creates fields in the General section of the business object that auto-populate with the sums from the table. IBM TRIRIGA has used these in some business objects for summary sections and reports. They are Number fields.



Tip – The Unit of Measure value for Number fields is controlled at the business object level. You will get the default defined in the Data Modeler for that business object.



Tip – Data Integrator does not properly handle updates to records for business objects containing Publish Names that include a control number field with the Generate on Create check box selected. In this scenario Data Integrator is unable to determine that a record included in an upload file already exists in the system, causing new records to be created each time the file is uploaded.

Data Integrator Requirements

Data integrator requires that (1) the Data Import Agent in the Agent Manager of the Administrator

Console be started, and (2) the file being uploaded contains the proper publish name fields for the business object being imported. However, if an upload file does not also contain the required fields for that business object, when a user signs in and opens the record IBM TRIRIGA produces a required field error. New records being imported must also contain required field values to ensure a smooth user experience.

There are two types of requirements for Data Integrator, functional and usability. Not meeting either of these can result in undesired consequences. If the functional requirements are not met, Data

Integrator fails on upload. If usability requirements are not met, the data will be successfully uploaded but will impose problems on the user.



Tip – It is always best practice to test all Data Integrator files against a copy of a production system data before executing the actual upload or update.

Functional Requirements

Data Integrator has three functional requirements. As long as these are met to specification, the

Header File is in a compatible format.

Publish Names

All business objects must be published before users can access them. The Publish menu item is on the

Tools tab in the Data Modeler. Review the Application Building for additional information. It is important to include all fields that define a business object’s Publish Name in the data upload. Many times a name or ID field is used as the Publish Name but other fields may be used.

For example, if the Publish Name for the triPeople business object consisted of family name, given name, and ID, then Smith, John - 10025884 is the Publish Name. You can view the Publish Name definition for a business object by opening the business object in the Data Modeler and viewing the BO

Mapping properties in the Tools menu.



Attention

– The Publish Name cannot contain a Control Number field when updating existing records in IBM TRIRIGA. This causes a new record to be created instead.

35 © Copyright IBM Corporation 2011, 2014.

For Publish Names containing control number fields, the Publish Name must be redefined to a unique value that does not use a control number. Once the upload is finished, you can change the Publish

Name back to using the control number field.

Tab-Delimited File Format

If all Publish Name fields are in the Header File and spelled properly, the only additional functional requirement is that the file be output into a tab-delimited file. The file extension must be .txt, but the file name can be whatever is desired.

Hierarchical Modules/Business Objects Must Have Parent

For uploading Hierarchical records, the Parent column must contain the full path of the uploaded record’s parent. Failure to define a proper Parent path results in orphaned records that do not display in the Hierarchy window.

Usability Requirements

Usability requirements impact the user. Files may show as having uploaded successfully; however, when a user opens a record a required field many not have been populated, producing an error message. Required fields, calculations, and rollups are all instances in which usability may be an issue.

Required Fields

Although Data Integrator only requires properly spelled and case sensitive Publish Names in order to upload data, there is an additional requirement. For new records, the Data Integrator Header File must contain any required fields. If required fields are not contained in the Header File, the record uploads successfully; however, when a user first visits and opens the record they see an error message that a required field error is missing. The user then must populate all required fields before they can successfully invoke any state transition action on the record in the form. If the data upload process transitioned the record to a read-only state, the user cannot transition the record to an editable state to make entries in the required fields.

You can prevent this scenario by including all required fields in the Header File for upload. Refer to the

Form Builder in the Administration menu to identify what fields are required and append them to the

Header File.

Calculations

You will have to trigger a custom workflow to execute calculations in order to populate read-only formula fields. Do not populate read-only fields directly, but rather trigger workflow logic to execute the calculations after upload.

Rollups

You will have to trigger a custom workflow to rollup information across multiple business objects. Be sure to write a workflow that moves all the way up from the lowest child to the highest parent across all objects within a hierarchy.

Custom Workflows

Custom workflows define all upload logic against a record. Create them ad hoc and trigger them on upload. For example, a workflow may be triggered by a state transition action leaving the null state or an asynchronous workflow on an Associate event.



Attention

– Certain workflows, such as those associated with rollups or calculations, must be retired before executing a Data Integrator upload.

Creating Tab-Delimited Files

To work with tab-delimited files, you need to know a little bit about the structure of a tab-delimited file. A tab-delimited file contains rows of data. Each row of data contains one or more pieces of data.

Each piece of data is called a field.

Tab-delimited files to be read by the Data Integrator should contain the same number of fields in every row, although not every field necessarily needs a value. Before Data Integrator starts reading a tabdelimited file, the user assigns a module, business object, and form to the file. This instructs the Data

Integrator where the data in the file is to go when uploaded.

The Data Integrator uses the data fields in each row (except the first) of a tab-delimited file to create or update a record in the IBM TRIRIGA Application Platform. If Data Integrator determines that the business object assigned to the file already contains the record being uploaded, it updates that record with values from the file. If the Data Integrator determines that the record being uploaded does not already exist, it uses the business object and form associated with the file to create the record.

Data Integrator needs to know which field will contain the data from each field from a row of the tabdelimited file. Data Integrator finds out where to put data from each field by looking at the first row, called the Header, of the tab-delimited file. Data Integrator does not use the first row of a tabdelimited file as data to create or update a record. Instead, it expects the fields of the first row to contain the name of a field in the business object associated with the file. The field name identifies the field to which the data for that field goes.

The easiest way to generate this sort of tab-delimited file is to use an application that knows how to generate a tab-delimited file and can put the field names in the first row. For example, some applications allow you to export or print a report to a tab-delimited file. If the program allows you to specify the column heading for each column, that probably means that it will put the heading value in the fields of the first row. If you are working with such a program to set up generation of tab-delimited files for the Data Integrator, specify column headings that match the names of the IBM TRIRIGA fields into which you want the field values to go. After you have the column headings correctly set up to match the field names of the IBM TRIRIGA object, creating tab-delimited files for Data Integrator should be an automated process.

37 © Copyright IBM Corporation 2011, 2014.

If an application cannot put headings on a tab-delimited file, you can still use data from it with Data

Integrator. However, the process of preparing the tab-delimited file for Data Integrator requires additional work. If the program you want to import data from cannot put field names in the first row of a tab-delimited file, you can add them by editing the file. Use Microsoft Excel to edit tab-delimited files.

When you edit a tab-delimited file with Excel, Excel may ask you some questions about the file. Excel asks these questions just to be certain it understands the kind of file you are asking to edit. Excel may ask if the file is Delimited or Fixed Width. Respond to this question by selecting Delimited. Excel may then ask you to select the delimiters that the file uses. Select tab. Do not select any other delimiter or options.

After you have inserted a new first row that contains the field names, save the file as a tab-delimited text file. If you have previously saved the Excel file in its native format you may ignore any warnings that the file may contain features not compatible with the tab-delimited format. The tab-delimited file is now ready to be processed by Data Integrator.

It is possible to use Data Integrator to import data from an application that cannot create a tabdelimited file if the application can export data in a format that Excel can read. If the application can create a file that Excel can read, then read the file into Excel. If necessary, add a first row to the file that contains the appropriate field names. Then save the file as a tab-delimited text file.

If you need to edit a data file to add a first row that contains field names, it is not necessary to manually type the names of the fields. You can use the Data Integrator to create the first row.

Creating a Header File for Import

A Header File is an import template in which the first header row declares field names into which data will be imported. You can create this header in many ways. The important fact to remember is that the template field headers must match the corresponding business object field names in case and spelling and contain the Publish Name fields for the business object.

There are two types of modules in IBM TRIRIGA: Non-Hierarchical and Hierarchical. The process of creating a Header File is the same for both, except Hierarchical uploads require a Parent column containing the full path of an uploaded record’s parent record.

Non-Hierarchical Module Header File

Non-hierarchical objects are flat objects that do not have a parent-child association and are not dependent on the existence of a parent object. Examples of IBM TRIRIGA non-hierarchical business objects are triPeople, triSpecification, and triAsset. You can confirm that a module is nonhierarchical in the Data Modeler by selecting the module and reviewing the module properties.

After creating the Header File, a user only requires security access to Data Integrator in order to upload a file. As long as the steps are followed properly and the data values fit the formatting of the field, the upload will be successful. As with all Data Integrator uploads, the system sends a notification to the user about the status of the upload.

Follow the steps below to create a Header File for a non-hierarchical module using Data Integrator:

To Create a Header File for a Non-Hierarchical Module

Step 1

Navigate to Tools > Administration > Data Integrator.

Step 2

Select the module that contains the business object that the Data Integrator should associate with the file. The drop-down box shows all defined modules.

Step 3

Select the business object that the Data Integrator should associate with the file. The drop-down box contains all published business objects in the selected module.

Step 4

Select the form (GUI) to be associated with the records in the file. The drop-down box contains all published forms for the selected business object.



Attention – During upload, if there is no existing record for the selected form for a given publish name, but there is another record with the same publish name with a different defined form, the Data Integrator appends the uploaded data to the existing record even though it has a different defined form.

Step 5

Click Create Header.

Step 6

Review the Display Columns screen that appears. It contains a check box for every field in the selected business object/form.

Step 7

Select the fields that you want. As you click a check box, the system adds the name of the corresponding field to the Column Order box in the upper right.



Note

– Be sure to include all elements of the Publish Name and all required fields.

Step 8

The fields in the Column Order section are in the order in which they were selected.

The sequence in the Column Order section is the order the fields will be displayed in the Header File.

To change the sequence of fields, click the name of the field to select it and click the up arrow or down arrow until the field is in the position you want.

Step 9

When list of fields in the Column Order section is complete and the fields are in the correct sequence, click Export.

Step 10

Click Save.

Step 11

Select the destination on your local computer.

Data Integrator gives the file a csv extension, although the file is technically generated in HTML format. Change the extension to htm.

To do that: (a) Change the file name extension from csv to htm. (b) Change Save as type to All Files. (c) Click Save.

39 © Copyright IBM Corporation 2011, 2014.

To Create a Header File for a Non-Hierarchical Module

Step 12

Find and open the saved file in Excel. Notice that the field names you chose are in the first row of the form.

Step 13

Enter your data into the appropriate columns.



Note

File.

– Skip this step if you are not entering data directly into the Header



Tip – Change all cells in your spreadsheet to Text format. This allows you to enter any value and eliminates Excel’s conversion formatting of your data. To change the format of your cells, click the cell in the upper left corner (above the 1 and to the left of A), which selects the entire spreadsheet. Click Format > Cells > Text > OK.



Attention

– List and Classification values must match exactly in both spelling and case.

Step 14

Save the file as a tab-delimited txt file.



Tip – Be sure that the file name ends in txt and that the Save as type is

Text (Tab delimited)(*.txt)

Step 15

Click Yes. This keeps the file in txt format.

Hierarchical Module Header File

Hierarchical objects have parent-child associations and the children are dependent on the existence of the parent object. Examples of IBM TRIRIGA hierarchical business objects are Geography, Location, and

Organization. You can confirm that a module is hierarchical in the Data Modeler by selecting the module and reviewing the module properties. For more information about hierarchies, see the

Hierarchies chapter of Application Building.

Hierarchical modules are unique in that they require an additional field in the header row of the Data

Integrator Header File. The key word is Parent, which links the business object child record to the appropriate parent in the tree.

Every hierarchical tree has a root that is the same as the name of the module. For organization business objects the root is Organization, for location business objects the root is Location, and for geography business objects the root is Geography.

In addition to importing data, it may be necessary to import or append records that are child records to another business object (e.g., buildings are children of the location root, floors are children of buildings, and spaces are children of floors.)

On the Header File manually add a new column labeled Parent. The word Parent is one of the reserved words from IBM TRIRIGA and is never found as an option when creating the header.



Note

– Always include the full path of the parent business object, including the first backslash (\).



Note

– You must enter the full path of the Parent record for Data Integrator to find the parent and append the new child record in the indicated place in the hierarchy tree.

Follow the steps below to create a Header File for a hierarchical module using Data Integrator:

To Create a Header File for a Hierarchical Module

Step 1

Navigate to Tools > Administration > Data Integrator.

Step 2

Select the module that contains the business object that the Data Integrator should associate with the file. The drop-down box shows all defined modules.

Step 3

Select the business object that the Data Integrator should associate with the file. The drop-down box contains all published business objects in the selected module.

Step 4

Select the form (GUI) to be associated with the records in the file. The drop-down box contains all published forms for the selected business object.

Step 5

Click Create Header.

Step 6

Review the Display Columns screen that appears. It contains a check box for every field in the selected business object/form.

Step 7

Select the fields that you want. As you click a check box, the system adds the name of the corresponding field to the Column Order box in the upper right.



Note

– Be sure to include all elements of the Publish Name and all required fields.

Step 8

The fields in the Column Order section are in the order in which they were selected.

The sequence in the Column Order section is the order the fields will be displayed in the Header File.

To change the sequence of fields, click the name of the field to select it and click the up arrow or down arrow until the field is in the position you want.

Step 9

When list of fields in the Column Order section is complete and the fields are in the correct sequence, click Export.

Step 10

Click Save.

41 © Copyright IBM Corporation 2011, 2014.

To Create a Header File for a Hierarchical Module

Step 11

Select the destination on your local computer.

Data Integrator gives the file a csv extension, although the file is technically generated in HTML format. Change the extension to htm.

To do that: (a) Change the file name extension from csv to htm. (b) Change Save as type to All Files. (c) Click Save.

Step 12

Find and open the saved file in Excel. Note the field names you chose are in the first row of the form.

Step 13

Type the key word Parent in the first unoccupied cell on the first row. This column will contain the full path of the uploaded record’s parent.

Step 14

Enter your data into the appropriate columns.



Note

File.

– Skip this step if you are not entering data directly into a Header





Attention

– List and Classification values must match exactly in both spelling and case.



Attention

– Do not put a slash (\) at the end of the Parent path.

Step 15

Save the file as a tab-delimited txt file.



Tip – Be sure that the file name ends in txt and that the Save as type is

Text (Tab delimited)(*.txt)

Step 16

Click Yes. This keeps the file in txt format.

Associative Upload Header File

Importing data records with associations requires more work than hierarchical uploading. Using the

Associative Upload feature of Data Integrator allows you to associate records to each other, through which you can link people with organizations, assets with spaces, assets with people, and so forth. You can link any two data records together utilizing the association string specified.



Tip – In most cases, define the association (and the reverse association) in the Association

Manager first.

Finding Association Strings

Before you can create an upload with associations, it is very important to understand the relationship verb defined to associate two business objects. Several areas of Application Building discuss the power of associations. Every business object has several associations defined with other business objects.

Confirm the existence of association strings that are to be used in an Associative Upload with Data

Integrator. Two methods of confirming association strings are by reviewing Associations tabs on records and through the Association Manager.

Associations Tab

Most IBM TRIRIGA records have an Associations tab on their respective forms. Follow the steps below to use this Associations tab to identify the record-level associations and their corresponding association strings for a given record.

To Identify Associations via the Associations Tab

Step 1

Select Portfolio menu, Portfolio page appears.

Step 2

Step 3

Select the business object (Department) in the Organization section.

Click the name of any record in the result page.

Step 4

Step 5

Select the Associations tab in the record.

Click the module of interest.

Step 6

Click the business object of interest.

Step 7

Review the results.

On the left side, in the Records section, the system shows all records that have an association from the record open in the form. The String value, known as the

Association String, is a verb that describes the association that exists between the two records.

43 © Copyright IBM Corporation 2011, 2014.

Association Manager

Follow the steps below to use the Association Manager to confirm the business object-level association rules and their corresponding association strings:

To Confirm Association Rules via Association Manager

Step 1

Navigate to Tools > Builder Tools > Association Manager.

Step 2

Scroll the Module panel and select the radio button of the module into which Data

Integrator will upload records.

Step 3

See the Association column in the right panel.



Tip – In the right panel, use the white up and down arrows and white dot

icons at the top of a column to sort the data in the column.

You can have Data Integrator create associations between records already in the IBM TRIRIGA

Application Platform environment and the records it uploads. To arrange for this, add specific information to define each association to the tab-delimited file. The specific information identifies:

 The module that contains the business object that was used to create the record that will be associated to the records being uploaded.

 The business object that was used to create the record that will be associated to the records being uploaded.



The name of the association from the existing records to the new records.

Once the upload utilizing the association linking is complete, the two data records reflect the relationship in the Association tab. Note that the association will be created FROM the existing record defined in the Associate column TO the record being uploaded. If a business object-level association exists (that is, is in the Association Manager) between the two business objects using the association string specified in the Associate column, a corresponding association will be created from the uploaded record to the record in the Associate column.

Data Integrator cannot be used to populate a smart section and certain types of locator fields. You may need to create a custom workflow for each type of association in order to append the associated record to a smart section or a locator field. Just associating two records will not make them appear in smart sections or locator fields. An exception to this is a query section using the same association. See the

Workflow Tasks Associated with Associative Uploads

section below for an example of how to do this.

Follow the steps below to add associations to the Header File containing the records to be uploaded:

To Add Associations to the Header File

Step 1

Type the key word Associate in a new column immediately to the right of the last column of data.

Step 2

Insert two rows between the header row and the first row of data. The data now begins in row 4.

44 © Copyright IBM Corporation 2011, 2014.

To Add Associations to the Header File

Step 3

In row 2 of the Associate column, enter the fully qualified name of the business object, surrounded by two percent symbols.

For example, %%triPeople:triPeople%%. This row reflects the module and business object of the records from which an association will be made as

%%Module:BusinessObject%%

Step 4

In row 3 of the Associate column, enter an association string, surrounded by two percent symbols.

For example, %%Manages%%. This should be the association string defined from the

%%Module:BusinessObject%%

indicated in row 2 to the business object indicated for the file being uploaded.

Step 5

In each row of the spreadsheet, in the Associate column enter the name of the existing record from which the new record will be associated.

Workflow Tasks Associated with Associative Uploads

Workflow tasks are not required for associative uploads, as an associative upload can exist on its own to create an association as described above. An associative upload with a workflow is another type of

Data Integrator upload used when one of the records specified in the Associate column also needs to be appended to a smart section.

In this scenario, a temporary association string in the Associate column is used to trigger a workflow that appends the record to the correct section of the uploaded record (or maps anything else as needed) and removes the temporary association. Once mapped, the IBM TRIRIGA system automatically creates the correct association between the records in the section.



Attention

– In your workflow, use an association string that neither one of the objects currently uses. This ensures the workflow will be triggered only when the records are associated from the upload and not by any user action. Make sure you use an association string that is in the Association Types list in the List Manager. If needed, you can add your own string to this list.

The workflow should remove the temporary association created by Data Integrator and create the correct one by mapping the record into the smart section/locator field.

To Set Up Workflow Tasks Associated with Associative Uploads

Step 1

Review the Data Integrator file.

The Associate column dictates that the association of type Upload Associate will be created from the specified Organization record in the triPeople record.



Note

– The directionality of this association is important. An association is created FROM the record in the Associate column TO the record being uploaded/updated. Unless there is a business object-level association defined for the association specified in the Associate column there will not be an association created from the record being uploaded to the record indicated in the Associate column.

Step 2

Since the Data Integrator file uses an association string that does not exist in the system, you need to add it to the Association Types list in the List Manager.

Step 3

In the Manage By section, select the Association Types list.



Note

– It may take more than a minute for this list to load.

Step 4

Enter the association string you used in the Data Integrator file in the Value box.

Step 5

Click Save Entries.

Step 6

Create a new workflow in the Organization module.

The Start Task defines the trigger condition for the workflow. In this case, when an

Organization

record is associated to a triPeople record with the string of Upload

Associate, the workflow executes.



Note

– The directionality of this association is important. For example, there will only be an association (and corresponding Associate event) created FROM the Organization record TO the uploaded/updated triPeople

record in this scenario. Since there was no business objectlevel association defined for the association type specified in the Data

Integrator upload there will NOT be an association defined FROM the uploaded/updated triPeople record TO the Organization record.

Step 7

The Modify Task maps from the primary record of the workflow (that is, the business object of the Start task) to the secondary record (that is, the secondary business object from the other end of the association defined in the Start task).

In our example, we map from Organization to a triPeople record since we want to map the organization record into a smart section in the people record.

Step 8

Click Edit Map to open the Object Mapping window.

To Set Up Workflow Tasks Associated with Associative Uploads

Step 9

Click Clear All to remove all field mappings.



Note

– It may take a few moments for the mappings to clear.

Step 10

Scroll down to the triPrimaryOrganization smart section.

Step 11

Click the Search icon .

Step 12

Set the section to Source; this maps the source record (Organization) into the section of the target record (triPeople) and creates the reference and association to populate the smart section.

Step 13

Scroll up and click OK.

Step 14

Choose an Associate Records task from the New Task bar (we will change it to a De-

Associate task next).

Step 15

Change the option from ‘Associate using’ to ‘Remove the association’ and the association to Upload Associate. This task de-associates the temporary association used to trigger this workflow as specified in both the tab-delimited Data Integrator file and the Start task.

Step 16

Set the ‘Where the associated record is’ section to the Secondary BO (that is, the triPeople

record).



Note

– If you are uploading embedded records, the De-Associate task should be performed before the Modify (Map Record) task to avoid duplicate association.

Step 17

Publish your workflow. You may need to clear the server’s workflow cache if this is a

Production environment.

Uploading Data

Data can be uploaded into the IBM TRIRIGA system with Data Integrator one file at a time or in batch mode.



Note

– The name of the file imported into Data Integrator must not exceed 50 characters.

Manual Uploads

Once the Header File has been created, populated with record information, and saved as tabdelimited, you are ready to upload the file into Data Integrator. Follow the same steps regardless of the type of upload you are executing (Associative, Hierarchical, or Non-Hierarchical). The only

requirement is to ensure that the proper Module::BusinessObject::Form is selected. Failure to select the proper settings results in a failed upload or a successful record upload into the incorrect object.

The following illustrates uploading a Header File manually:

To Upload a Header File Manually

Step 1

Navigate to Tools > Administration > Data Integrator.

Step 2

Make sure the values in the Module, Business Object, and GUI drop-down boxes match the Header File. The value of Import Type should be Add.

Add

means that if a row contains values for a record’s Publish Name fields that match an existing record, the other values in the row are used to update the record. If the values a row contains for a record’s Publish Name fields do not match an existing record, data in the row is used to create a new record.

Step 3

From the drop-down box next to Action, select the action to be performed on a record after the Data Integrator creates or updates it. The drop-down box next to Action contains a list of the actions that are valid for a new record, based on the state transition family used by the selected business object.



Note

– When a record is first created it is in a special state called null.

Records in the null state disappear after the operation currently using the record is done. It is important for the Data Integrator to perform an action on newly created records to take them out of the null state. If the action does not take the record out of the null state the record will disappear immediately after it is created. All of the actions listed in the

Action drop-down box are actions that transition records out of the null state.

Step 4

If you need a workflow to launch after the Data Integrator finishes creating a record, the workflow must be launched by a sub action attached to the action indicated in the

Action drop-down box.

Step 5

The values in the drop-down box next to File Type indicate the type of files from which

Data Integrator can read data. The only value is Tab Delimited (*.txt).

Step 6

From the drop-down box next to File Char Set, select the character set in which the file being imported was written.

Step 7

Make sure the Batch Upload check box is cleared. This tells Data Integrator to read the data file you specify only once. The Batch Upload check box is the subject of the

Batch

Process Uploads section of this chapter.

Step 8

Click Browse and find and Open the Header File you created.

Step 9

Click Upload File.

Step 10

Click OK. This causes data to be read from the specified file. The system returns you to your Home Portal.

To Upload a Header File Manually

Step 11

After the process is complete, you will receive notification in the Notifications section of your Home Portal.



Attention – Always check to make sure that all records have been imported correctly.

Step 12

In this example, the imported entries can be found by selecting Portfolio on the Home

Portal, then People section, then Employee.

Batch Process Uploads

Batch uploading allows a user to automate the upload process and schedule events in which the IBM

TRIRIGA platform automatically uploads the defined file according to the scheduled settings. This is handy for automating the integration of data from an external system without requiring hard-coded scripts or integration development.



Note

– Before implementing batch uploading, run CreateBatchProcess.bat (Windows) or CreateBatchProcess.sh (Linux) scripts in the <TRIRIGA installation>\userfiles directory to create the default directories required to do the processing.

Using a batch upload is usually the best choice when you will be using the Data Integrator to periodically read data files that are all organized the same way into the same kind of records.

Setup for Batch Uploading

The TRIRIGAWEB.properties file contains settings for Batch Upload that define the paths to input, processing, output, errors, and log files. The folders specified in the BatchInputLoc,

BatchProcessLoc

, BatchOutputLoc, BatchErrorLoc, and BatchLogLoc parameters in

TRIRIGAWEB.properties

must exist and the names must match exactly. For more information about the TRIRIGAWEB.properties file, go to the IBM TRIRIGA Application Platform 3 Knowledge Center and select Installing IBM TRIRIGA Application Platform > Configuring the platform > Properties files.

Initiate Batch Uploading

To Set Up Batch Process Uploads

Step 1

Navigate to Tools > Administration > Data Integrator.

Step 2

Make sure the values in the Module, Business Object, and GUI drop-down boxes match the Header File. The value of Import Type should be Add.

Add

To Set Up Batch Process Uploads

Step 3

The primary purpose of performing the action specified in the Action field is to take newly created records out of the null state. If the action taken by the Data Integrator does not take the record out of the null state, the record disappears immediately after it is created.



Note

– When a record is first created it is in a special state called null.

Action drop-down box are actions that transition records out of the null state.

Step 4

If you need a workflow to launch after the Data Integrator finishes creating a record, the workflow must be launched by a sub action attached to the action indicated in the

Action drop-down box.

Step 5

The values in the drop-down box next to File Type indicate the type of file from which

Data Integrator can read data. The only value is Tab Delimited (*.txt).

Step 6

From the drop-down box next to File Char Set, select the character set in which the file being imported was written.

Step 7

Check the Batch Upload check box. The system displays four new fields. The fields labeled Upload Batch Data Every ___ Hours ___ Minutes control how often the Data

Integrator is scheduled to read data.

Step 8

Often it turns out that the interval you specify between times that the Data Integrator runs divides evenly into 24 hours. If this is the case, data is read from the file at the same times every day.

If you choose an interval that does not divide evenly into 24 hours, such as 5 hours and 2 minutes, data is read from the file at the same times every day. The way this works is that every day, at the time specified in the field labeled Reset clock every day at, data is scheduled to be read from the specified file. It is then rescheduled at the specified interval.

The Upload Batch Data Every field should be set to a time increment that uploads on intervals of the input time starting from the Reset clock every day at field. The default is 0 hours and 0 minutes. In the example above, the upload process will begin every day at 9:00 AM only. If we were to set it for 5 hours 0 minutes, the batch process would run starting at 9:00 AM and incrementing 5 hours for the next 24 hours; therefore it would run every day at 9:00 AM, 2:00 PM, 7:00 PM, 12:00 AM, 5:00 AM. It would then start again at 9:00 AM, iterating through the same time increments again.

To Set Up Batch Process Uploads

Step 9

Set Reset clock every day at to the time you want the batch upload process to occur.

The batch process will begin at this time every day.

If Reset clock every day at contains a value greater than 24 hours the upload will occur only once a day, at the designated Reset clock every day at time.

There is a running agent that executes about every 10 seconds. The agent iterates through all the batch process objects created and, based on the current time, runs the batch objects at the appropriate time. Since it is unknown the exact time the agent will fire, IBM TRIRIGA put a 1 minute buffer on the start time of each time increment. In the example above, if the agent fires at 9:00:10, the process will run since the system checks between 9:00 AM and 9:01 AM. In the example with a 5 hour increment, the batch files would run if the current time is between 9:00 AM - 9:01

AM, 2:00 PM - 2:01 PM, 7:00 PM - 7:01 PM, 12:00 AM - 12:01 AM, and 5:00 AM - 5:01

AM.

If the designated file is found in the appropriate location, the system processes the file accordingly and places the process time in the Last Run Time field of the Data

Upload business object.

Step 10

Enter the file name in the Filename (*.txt) field. The system expects to find this file in a directory on which the IBM TRIRIGA Application Platform runs.

Step 11

Enter an email address in Confirmation eMail. After each attempt the Data Integrator makes to batch upload a file, it sends an email to the address specified. This email is sent whether or not the upload is successful.

Step 12

Click Upload File.

Step 13

Click OK. The system schedules the upload of the file specified. It continues to schedule the specified file at the specified interval indefinitely. The system returns you to your Home Portal.

Step 14

Each time the upload process completes, the system sends an email to the address listed in Confirmation eMail.



Attention – Always check to make sure that all records have been imported correctly.

Terminate Batch Uploading

If you want to change any parameters for a batch upload, you must first end the scheduling of the batch upload and then create a new batch upload. Use the System tool to end scheduling of a batch upload.

Follow the steps below to terminate a batch upload:

To Terminate a Batch Upload

Step 1

Navigate to Tools > System Setup > System > Data Upload.

The system displays the existing records for uploads that have been or will be scheduled.

Step 2

Step 3

Click the check box next to the name of the record you want to delete.

Click Delete.

Step 4

Click OK.

Data Integrator Tips

Use the following Data Integrator tips to enhance how your company uses Data Integrator.

Tip

Associate Existing Records

Description

Once all records have been created, reuse the Header File to create an Associative upload Header File. Executing associations after all records have been created eliminates issues surrounding non-existent records.

Checking For Data Upload

Errors

Every time the Data Integrator runs and has errors, it creates a Data

Upload Error record. To find a Data Upload Error record, navigate to

Tools > System Setup > System > Data Upload Error. Scroll through the records in the result page. Click the file name hyperlink.

Confirm Via BO result page Once the data has produced a successful notification, review the data via the BO result page. For hierarchical objects, ensure the child records appear under the proper parent in the hierarchy window.

Data Integrator Cautions Data Integrator does less checking of the data than a form into which you manually enter data:



It will create or update records with missing values for required fields.



It will allow any value in a List field, even if the value is not in the list.



It does not perform requested verifications on field values.

If an upload does not explicitly set the organization and geography of the records it creates, the security access to those records may not be what is desired. You may need to include organization or geography fields in flat file records to ensure proper security access to created records.

Tip

Data Upload Records

Folder Cleanup

Managing Files for Upload

Description

Every time you use the Data Integrator to initiate the reading of data, it creates a Data Upload record. If you initiate a schedule of batch uploads, there is still just one Data Upload record.

After an upload completes, you can learn how the upload went by looking at the Data Upload record that corresponds to the upload.

The Data Upload record has a smart section that contains Data

Upload Error records. If any errors occurred, you will see them in the smart section.

To find a Data Upload record, navigate to Tools > System Setup >

System > Data Upload. Scroll through the records in the result page.

Click the file name hyperlink.

System Administration processes for Data Integrator Batch Upload should include periodic cleanup of the input and log folders.

When files are uploaded in batch mode, the upload happens without the direct participation of a person. The platform looks for the files on the computer it runs on rather than on a computer that a person is using.

To identify the computer and the directory used for batch data uploads, you may need to consult with the person who administers the IBM TRIRIGA Application Platform environment for your organization. For more information, go to the IBM TRIRIGA 10

Knowledge Center and select Administering IBM TRIRIGA.

To ensure that files are properly processed, observe the following rules regarding when and how to put a file in the directory for processing:

 Before you put a file in the directory for batch upload, first check to see if there is already a file with the same name in the directory. If there is, it means the last file with that name has not been processed. After processing a file, the file is moved to another directory.

 To ensure that a file already in the directory is correctly processed, wait for the file to disappear from the directory before putting a new file with the same name in the directory.



When it is time to put a file in the directory, do not put it in the directory by copying the file from somewhere else. The problem with copying is that the file gets put in the directory one piece at a time. This means that if a file is copied into the directory there will be a period of time when only part of the file is in the directory. If the IBM TRIRIGA Application

Platform tries to process the file when it is not all there, it may not process the file correctly.

Tip

Processing a Whole Batch

Description

Instead of copying the file into the directory, first put the file somewhere else on the same file system as the directory.

Then rename or move the file into the directory. When you rename or move a file within the same file system, no file copying takes place. Instead an entry is added to the directory that points to the file where it already is.

It is possible to have a workflow that processes all the records in the batch after the individual records have been created or updated.

When an upload of a tab-delimited file is finished, the platform performs an APPLY action on the Data Upload record associated with the upload.

As records are processed, an association is created from the Data

Upload record to the affected records. The association name is Data

Load Status Of. If you use the Association Manager to define an association with this name from the Data Load Status Of business object to the other relevant business object, a workflow launched by an APPLY action on a Data Upload record would be able to process all of the uploaded records.

Retire Formula, Rollup, and

Calculation Workflows

Since you can use a Retrieve Workflow task to retrieve records sorted in the order of their value in a particular field, you can use any field in the records to determine the order in which the workflow processes them.

In Production environments, all formula, rollup, and calculation workflows should be retired against the business object before executing the upload. This ensures the records are being created without forcing the platform to run unnecessary workflows against partial data. It also helps eliminate looping that may occur against hierarchical objects. Once the upload is successful, republish and execute the workflows against your new data set.

Run All Files During Off Peak

Hours

Depending on the amount of information being uploaded, server resources can become taxed and performance degrades. End users may notice a performance hit, which may result in false positives and your help desk being contacted.

Timing of Record Processing The IBM TRIRIGA Application Platform does not necessarily process uploaded records in the same order they appear in the tab-delimited file. It reads the data from each tab-delimited record in sequence and then uses a different thread to process the data to create or update a record.

In most cases, it does not matter in what order records are processed. If you need to process all of the records in a batch in the same order in which they are read, you will need to force the Data

Integrator to process all the records from a file with the same thread.

Tip Description

To accomplish this, set the number of threads used by the Data

Import Agent to 1. For a description of how to do this, go to the IBM

TRIRIGA Application Platform 3 Knowledge Center and select

Installing IBM TRIRIGA Application Platform.

TRIRIGAWEB.properties

For Batch Upload

The TRIRIGAWEB.properties file contains file settings for Batch

Upload, specifically for input, processing, output, errors, and log.

The files defined in TRIRIGAWEB.properties must exist and the file names must match. For more information about the

TRIRIGAWEB.properties

file, go to the IBM TRIRIGA Application

Platform 3 Knowledge Center and select Installing IBM TRIRIGA

Application Platform > Configuring the platform > Properties files.

Upload Records First

Upload Sets of Records

Do not upload a Data Integrator file with all fields and all associations in one file. Create all records for all business objects first. Once you have imported all records, begin the task of associating the records to one another.

Do not push massive amounts of data in one Data Integrator file. Use sets of 5000 records per Data Integrator file. This reduces errors related to memory and system capacity.

Uploading Images If you need to assign images to newly-uploaded records you can use the format below. Assume the image field is named triImageIM. triImageIM

//Company-1/file123.jpg

A couple of notes:



The file name must start with the word file - as in filemyimagename.jpg

or file12345.gif.



The file must be placed in the <TRIRIGA install directory>\userfiles\Company-1

directory before uploading the Data Integrator file.

APPENDIX A: COLOR VALUES

HTML colors are defined using a hexadecimal (hex) notation for the combination of Red, Green, and

Blue color values (RGB). The lowest value that can be given to one of the light sources is 0 (hex 00).

The highest value is 255 (hex FF). Hex values are written as 3 double digit numbers, starting with a # sign.

The following list shows examples of colors and their hexadecimal color values:

 Black - #000000



Red

- #FF0000



Green

- #00FF00



Blue

- #0000FF



Yellow

- #FFFF00



Turquoise

- #00FFFF



Pink

- #FF00FF



Gray

- #C0C0C0



Whhiittee

- #FFFFFF

INDEX

Add ............................................. 48, 49

Administrator Console................. 6, 8, 12, 28

All Iterations ....................................... 16

Approval Task...................................... 27

Associate ........................................... 44

Associate Existing Records....................... 52

Association ......................................... 44

Association Manager .............................. 44

Association String ............................ 43, 45

Association Types List ............................ 46

Associations Tab................................... 43

Associative Upload ................................ 42

Asterisk ...............................................6

Asynchronous ...................................... 14

Batch Upload.................... 48, 49, 50, 53, 55

Block Task ............................................3

BO_NAME ........................................... 12

Break Flow Type................................... 17

Break Task.................................16, 17, 18

Business Object Section............................6

Business Objects with Staging Tables ....... 6, 28

Calculations ........................................ 36

Checking For Data Upload Errors ............... 52

Cleanup Agent ..................................... 23

Color Values........................................ 56

Concurrence ....................................... 14

Confirm Via Manager ............................. 52

Continue Flow Type............................... 17

Control Number ................................... 36

Controlling Block Task..............................4

Correlation ......................................... 16

Correlation Number............................. 4, 9

Create Header ................................ 39, 41

CREATED DATE..................................... 12

Current Scope...................................... 17

Custom Action .......................................4

Custom Workflows ................................ 37

Data Checking ..................................... 52

Data Import Agent ........................... 35, 54

Data Integrator ......................................4

Data Integrator Cautions......................... 52

Data Load Status Of............................... 54

Data Upload Error Record................... 52, 53

Data Upload Record............................... 53

Data Validation .................................... 17

DataConnect Agent ....................... 4, 11, 21

DataConnect Job ....................................4

DataConnect Job Control Table...................4

DataConnect Jobs ................................. 12

DataConnect Log Records..........................8

DataConnect Managed Object................ 6, 12

DataConnect Runtime ..............................8

DataConnect Task ........ 3, 4, 13, 15, 22, 27, 28

DC_ACTION ...........................................9

DC_CID ............................................... 9

DC_GUI_NAME...................................... 10

DC_JOB.....................4, 8, 11, 13, 21, 23, 27

DC_JOB_NUMBER ................................... 9

DC_PATH.........................................9, 27

DC_PROJECT ....................................... 10

DC_SEQUENCE_ID................................... 9

DC_STATE............................................ 9

Delete Business Object ............................ 7

Directionality ...................................... 46

ETL Maps............................................. 5

Example of Multiple DataConnect Tasks Using

Temporary Data................................. 19

Example of Single DataConnect Task Using

Temporary Data................................. 18

Execution Schedule ............................... 27

Expand Data.....................................6, 28

Export .......................................... 39, 41

External Source..................................... 5

Field Types ......................................... 29

Fields ................................................. 6

File Char Set .................................. 48, 50

File Type....................................... 48, 50

Finding Association Strings....................... 43

Folder Cleanup..................................... 53

Fork Task ............................................ 3

General Section.................................... 28

Get Temp Record Task ........................... 17

Has Staging Table ............................ 5, 6, 7

Header File ................................38, 47, 52

Hierarchical ................................... 36, 41

IBM TRIRIGA Connector for Business

Applications ...................................... 3

Images............................................... 55

Initial State.................................... 15, 28

Initiate Batch Uploading ......................... 49

In-Sequence ........................................ 16

Insert ................................................. 9

Integration ................................15, 17, 22

Integration Flag..................................... 5

Iterator Task ........................................ 3

Job Number ................................ 9, 13, 22

JOB_NAME .......................................... 11

JOB_NUMBER....................................... 11

JOB_RUN_CTL...................................... 11

JOB_TYPE........................................... 11

Kettle Spoon ........................................ 5

Loop Task............................................ 3

Managing Files for Upload........................ 53

Mapping Properties................................ 34

Modify Task......................................... 46

Module BusinessObject ........................... 45

Module BusinessObject Form .................... 48

Multiple Record Smart Sections ................. 35

None................................................. 16

Non-Hierarchical .................................. 39

Null .................................... 15, 37, 48, 50

Number Field’s UOM .............................. 27

Object Mapping.................................... 46

Object Migration ....................................6

Parent......................................36, 40, 42

Pentaho ...............................................5

Per X Iterations .................................... 16

Platform Logging Managed Object ...............8

PROCESS_SYS_ID................................... 12

Processing .......................................... 27

Processing a Whole Batch........................ 54

Propagate Integration Status............... 15, 17

Publish BO ............................................7

Publish Business Object ............................6

Publish Name ...................................... 35

Required Fields ................................. 6, 36

Reset Clock Every Day At ........................ 51

Retire Calculation Workflows ................... 54

Retire Formula Workflows ....................... 54

Retire Rollup Workflows ......................... 54

Rollups .............................................. 37

Run Data Integrator During Off Peak Hours ... 54

Save Permanent Record Task ................... 17

Security ............................................. 52

Selected Block ..................................... 17 server.log........................................ 7, 27

Sets of Records .................................... 55

Setup for Batch Uploading ....................... 49

Smart Section Association ....................... 28

Smart Section Keys..................................6

Source............................................... 47

SOURCE_SYS_ID.................................... 12

Staging Table ........................................5

Staging Table Field .................... 6, 7, 10, 28

Staging Table Key ..............................7, 10

Staging Table Name............................... 28

Start Task...................................... 14, 46

STATE ............................................... 12

Suspended Workflow.............................. 28

Switch Task.......................................... 3

SYSTEM DC PROCESS JOB.................... 15, 22

Tab-Delimited................................. 48, 50

Tab-Delimited File ................................ 37

Tab-Delimited File Format....................... 36

Task Step ........................................... 16

Terminate Batch Uploading...................... 51

Timing of Record Processing..................... 54

Transaction......................................... 16 triDataConnectJob .......4, 8, 13, 14, 22, 23, 27 triDataConnectLog ................................. 8

Trigger Action.............................16, 27, 28 triLog................................................. 8

TRIRIGAWEB.properties .................23, 49, 55

Unit of Measure.................................... 35

Update ...........................................9, 11

UPDATED_DATE .................................... 12

Upload Batch Data Every......................... 50

Upload File .................................... 48, 51

Upload Records First.............................. 55

Uploading Images.................................. 55

Upsert ..................................... 5, 6, 9, 11

Use Temporary Data .....................15, 17, 22

User Action Task................................... 28

USER_ID ............................................. 12

Web Service ......................................... 3

Workflow ............................. 13, 22, 45, 54

Workflow Builder .................................. 14

Workflow Business Object ....................... 27

Workflow Context .................................. 5

Workflow Exceptions ............................. 28

Notices

This information was developed for products and services offered in the U.S.A.

IBM may not offer the products, services, or features discussed in this document in other countries.

Consult your local IBM representative for information about the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service.

IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not grant you any license to these patents. You can send license inquiries, in writing, to:

IBM Director of Licensing

IBM Corporation

North Castle Drive

Armonk, NY 10504-1785

U.S.A.

For license inquiries regarding double-byte character set (DBCS) information, contact the IBM

Intellectual Property Department in your country or send inquiries, in writing, to:

Intellectual Property Licensing

Legal and Intellectual Property Law

IBM Japan, Ltd.

19-21, Nihonbashi-Hakozakicho, Chuo-ku

Tokyo 103-8510, Japan

The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law:

INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT

WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED

WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you.

This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication.

IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice.

Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk.

IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you.

Licensees of this program who want to have information about it for the purpose of enabling: (i) the exchange of information between independently created programs and other programs (including this one) and (ii) the mutual use of the information which has been exchanged, should contact:

IBM Corporation

2Z4A/101

11400 Burnet Road

Austin, TX 78758

U.S.A.

Such information may be available, subject to appropriate terms and conditions, including in some cases, payment of a fee.

The licensed program described in this document and all licensed material available for it are provided by IBM under terms of the IBM Customer Agreement, IBM International Program License Agreement or any equivalent agreement between us.

Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.

All statements regarding IBM's future direction or intent are subject to change or withdrawal without notice, and represent goals and objectives only.

This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental.

Privacy Policy Considerations

IBM Software products, including software as service solutions, (“Software Offerings”) may use cookies or other technologies to collect product usage information, to help improve the end user experience, to tailor interactions with the end user or for other purposes. In many cases no personally identifiable information is collected by the Software Offerings. Some of our Software Offerings can help enable you to collect personally identifiable information. If this Software Offering uses cookies to collect personally identifiable information, specific information about this offering’s use of cookies is set forth below.

This Software Offering does not use cookies or other technologies to collect personally identifiable information.

If the configurations deployed for this Software Offering provide you as customer the ability to collect personally identifiable information from end users via cookies and other technologies, you should seek your own legal advice about any laws applicable to such data collection, including any requirements for notice and consent.

For more information about the use of various technologies, including cookies, for these purposes, see

IBM’s Privacy Policy at www.ibm.com/privacy and IBM's Online Privacy Statement at www.ibm.com/privacy/details in the section entitled “Cookies, Web Beacons and Other Technologies” and the "IBM Software Products and Software-as-a-Service Privacy Statement" at www.ibm.com/software/info/product-privacy/ .

Trademarks

IBM, the IBM logo, ibm.com, and TRIRIGA are trademarks or registered trademarks of International

Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the

Web at “ Copyright and trademark information ” at www.ibm.com/legal/copytrade.shtml

Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the

United States, other countries, or both.

Key features

Import external data into the IBM TRIRIGA system
Use for initial loads of data
Use for batch insert/update of data on a recurring basis
Provides information about IBM TRIRIGA DataConnect
Provides information about IBM TRIRIGA Data Integrator

Frequently asked questions

IBM TRIRIGA DataConnect is a tool for importing external data into the IBM TRIRIGA system. It is useful for both initial loads of data and for batch insert/update of data on a recurring basis.

IBM TRIRIGA Data Integrator is a tool for processing tab delimited flat files and creating business object records. Data Integrator can transition the record. It can be run manually to process a single file or on a schedule to process files that are in a specific directory.

IBM TRIRIGA DataConnect employs staging tables and workflow tasks, allowing an external source to write data directly into the IBM TRIRIGA staging tables and have IBM TRIRIGA workflow process this data for insertion into native IBM TRIRIGA business objects. IBM TRIRIGA Data Integrator processes tab delimited flat files and creates business object records. Data Integrator can transition the record. It can be run manually to process a single file or on a schedule to process files that are in a specific directory.

User questions

Mary T.

What are the key differences between IBM TRIRIGA DataConnect and other integration tools like IBM TRIRIGA Connector for Business Applications and IBM TRIRIGA Data Integrator?

Unlike IBM TRIRIGA Connector for Business Applications and IBM TRIRIGA Data Integrator, DataConnect utilizes staging tables and workflow tasks, enabling external sources to write data directly into IBM TRIRIGA staging tables. Then, IBM TRIRIGA workflow processes this data for insertion into native IBM TRIRIGA business objects, and also allows storing data into language tables (L_ tables).

Mary T.

How can external data be integrated into IBM TRIRIGA?

IBM TRIRIGA DataConnect is a tool for importing external data, supporting initial loads and recurring updates. It uses staging tables and workflow tasks, allowing external sources to write data directly into IBM TRIRIGA staging tables. Workflows then process this data for insertion into native IBM TRIRIGA business objects, including language tables (L_ tables).

Robert S.

How to add a field as a column in a Data Management Template?

To add a field as a column in a Data Management (DMT) template (Header File), navigate to Tools > Administration > Data Integrator. Select the module, business object, and form. Click Create Header, and in the Display Columns screen, select the fields to include. Adjust the column order using the arrows, then click Export. Save the file as an .htm file, open it in Excel, and save it as a tab-delimited .txt file.

IBM TRIRIGA Application Platform

Application Building for the IBM TRIRIGA

Application Platform 3:

Data Management

Contents

About This Guide

Conventions

Intended Audience

Support

1. DataConnect

Glossary

Process Overview

Create and Maintain Business Object Staging Tables

How to Identify Which Business Objects Have Staging Tables

Business Object Property: Has Staging Table

Field Property – Staging Table Field

Field Property – Staging Table Key

When a Business Object Is Published

When a Business Object Is Deleted

Make the Job Business Object

Move Data from the External Source

Staging Tables

DC_JOB Table

Create ETL Maps and Run Mapping

Use Workflow to Move Data

Workflow Tasks

Workflow Builder

Start Task

DataConnect Task

Trigger Action Task

Break Task

Workflow Calls Another Workflow

Runtime Data Validation

Status

Use Temporary Data

Example of Single DataConnect Task Using Temporary Data

Example of Multiple DataConnect Tasks Using Temporary Data

DataConnect Agent Runs

Workflow Processing

Cleanup Agent

Possible Errors

DataConnect Tips

2. Data Integrator

Preparation and Understanding of Field Types

Data Integrator Requirements

Functional Requirements

Publish Names

Tab-Delimited File Format

Hierarchical Modules/Business Objects Must Have Parent

Usability Requirements

Required Fields

Calculations

Rollups

Custom Workflows

Creating Tab-Delimited Files

Creating a Header File for Import

Non-Hierarchical Module Header File

Hierarchical Module Header File

Associative Upload Header File

Finding Association Strings

Associations Tab

Association Manager

Workflow Tasks Associated with Associative Uploads

Uploading Data

Manual Uploads

Batch Process Uploads

Setup for Batch Uploading

Initiate Batch Uploading

Terminate Batch Uploading

Data Integrator Tips

APPENDIX A: COLOR VALUES

INDEX

Notices

Privacy Policy Considerations

Trademarks

What is IBM TRIRIGA DataConnect?

What is IBM TRIRIGA Data Integrator?

What is the difference between IBM TRIRIGA DataConnect and IBM TRIRIGA Data Integrator?

Mary T.

Mary T.