Work with Nodes in Batch Data Transforms

Batch data transforms provide nodes that allow you to bring in data, transform it, and write the results to a target. You add nodes in the canvas. String together multiple nodes to sequentially change the flow of the data. A batch data transform can have multiple Input and Output nodes along with multiple Append, Join, Aggregate, Filter, Transform, and Update nodes.

• Input and Output Nodes: Add Data and Write to a Batch Data Transform
  Add data to a batch data transform with an input node. To save batch data transform output, write your data to one or more output nodes. In Data 360, input and output nodes can be either data lake objects or data model objects of type Profile or Other. You can’t have a data lake object input node and a data model object output node, and vice versa. Typically data aware specialists and engineers have access to data lake objects and data model objects, whereas marketers and analysts have access only to data model objects.
• Combine Rows from Different Data Sources Using the Append Node
  Use an append node to combine rows from multiple input data sources into a single result.
• Aggregate Node
  Use an Aggregate node to roll up data. You can aggregate to a higher level of granularity or roll up hierarchical data.
• Filter Node
  Use a Filter node to remove rows that you don’t need in your target. For example, you can filter case records to focus on escalated cases only. Filter values are case-sensitive.
• Join Objects in a Batch Data Transform
  On the batch data transform canvas, use a node to create a join between data objects. To make sure that data from multiple data objects is combined accurately and not duplicated, always add a key qualifier field to create a fully qualified key (FQK) in a join. To ensure that null values in a key qualifier field are matched correctly, use a null-safe join by replacing null values with null strings or empty strings.
• Join Operations
  Batch data transforms support these join operations.
• Transform Node: Transform the Data Before Loading It Into an Output
  Use a Transform node to manipulate data based on the transformations added to the node. You can add one or more transformations to each Transform node. For example, you can add transformations to concatenate two columns, standardize the formats in a date column, and detect the sentiments of comments in a feedback column. Each transformation modifies the data in a unique way. To manipulate data at multiple stages of a data transform, add a separate Transform node at each stage.
• Transformations for Batch Data Transforms
  Use transformations to perform calculations on and manipulate your data. For example, you can use a transformation to create a calculated field based on a formula. You add transformations inside a Transform node. You can string together multiple transformations to manipulate data sequentially.
• Update Node: Swap Column Values
  You can swap column values with data from another data source when key pairs match with the update node. For example, to update selected account names after a series of recent acquisitions and mergers, replace the name from an uploaded spreadsheet based on matching account IDs. To minimize processing time, change column values with matching key pairs instead of every value in the column.
• Copy and Paste a Batch Data Transform
  Use the toolbar or keyboard shortcuts to copy and paste nodes within a single batch data transform or across multiple ones. To use this feature, enable clipboard access permissions in your browser.
• Connect and Reorganize Nodes or Branches in a Batch Data Transform
  Connect disconnected nodes or branches to append data from any node to any other node in a batch data transform. You can also create a branch at an intermediate node, and append it to another path in the batch data transform.