Data Processing Engine Nodes

In the Data Processing Engine Builder, all the supported nodes are available on the Nodes tab. Nodes take input data, process the data according to its defined logic, and then produce output data that can be used in subsequent nodes.

Add a source node called the data source node to your Data Processing Engine definition to indicate the data source that contains the input data to needs to be transformed. Then, add a target node called the writeback object node, where the transformation results must be written to. Include one or more nodes called transformation nodes that specify the transformation logic that needs to be applied on the data source. For example, you can use a join node to combine two datasets, a formula node to calculate a field value, or an append node to append one dataset to another.

• Joins
  Join data from two nodes to create an output based on matching field values. Combining data into a single join node allows for further transformations, such as formulas or aggregations.
• Filters
  Use a filter node to select specific records from a source node based on your business requirements.
• Appends
  Use the Append node to combine the data from two or more nodes. Add an append node to your Data Processing Engine definition when you need to consolidate data from different data sources or datasets into a single dataset for further processing, analysis, or reporting.
• Writeback Objects
  Write back transformed data to standard and custom objects, data lake objects (DLO), data model objects (DMO), JSON, or Analytics datasets. Use the writeback object node to create, update, or upsert records with the transformed data. You can also delete records from the org using the writeback object node. Use related field mapping to write data to multiple objects in a definition.
• Formulas
  Use the formula node to create multiple formulas and store the results of each formula in a new field. Create multiple formulas in a single node and use the formula results in subsequent nodes. Use the compute relative transformation option to determine the trends of specific fields.
• Hierarchies
  Find out the hierarchy path of a field. The field's hierarchy path and the direction of the hierarchy path are determined by the parent field that you select. You can also include the field within the hierarchy path.
• Custom Nodes
  Use custom nodes to run the logic defined in a node. A custom node is related to a process type. When you add a custom node to a Data Processing Engine definition, you can select a custom node that’s available for the definition’s process type.
• Groups and Aggregates
  Use the group and aggregate node to group records by field, and aggregate the data from one or more fields.
• Slices
  Slice nodes help you retain fields that you require in subsequent nodes. You can use the multiple slice nodes to create subsets of fields from one source node. Each slice node lets you remove fields from any previously created nodes.
• Forecasts
  Use Forecast nodes to predict patterns for specific time periods in the future. A forecast node analyzes the past data points and their corresponding dates from the source data that you define in the Data Processing Engine definition. The forecast node then makes predictions about future values based on the historical data. For example, estimate the order quantity for the next four quarters based on the orders placed in the last five years. The Forecast node isn't available for definitions where the runtime is Data Cloud.
• Composite Writebacks
  Use the composite writeback node to consolidate related records from one or more writeback object nodes, so you can write back all the interrelated records together to the target object.