Filtering Records Extracted from a Salesforce Object

Add a filter to the sfdcDigest Transformation transformation to extract a subset of all records from a Salesforce object. You can filter records to reduce the number of extracted and processed records, exclude records that contain irrelevant or sensitive data, and increase dataflow performance.

A filter consists of one or more filter conditions, where each filter condition compares a field value to a value. For example, Amount >= 1000000. You can also apply SOQL functions on the field value in a filter condition, like CALENDAR_YEAR(CreatedDate) = 2011. You can add multiple filter conditions using logical operators AND, OR, and NOT. You can also use a backslash (\) to escape double quotes included in strings.

The sfdcDigest Transformation transformation extracts all records for which the filter is true. If you configured the sfdcDigest transformation for incremental extraction, the filter applies to data extracted during the incremental run only—CRM Analytics doesn't apply the filter to records that were previously loaded into the dataset. If you add an invalid filter, the dataflow fails at run time.

For each instance of sfdcDigest, you can use one of the following types of filters:

• Structured filter
• Advanced filter

Tip Are you trying to decide whether to use a filter in the sfdcDigest Transformation transformation or use a filter Transformation transformation? Use a filter Transformation transformation to filter records at any point in the dataflow. For example, you can add it after the dataflow joins two datasets. However, to reduce the number of rows processed in the dataflow and optimize dataflow performance, add the filter closest to the point at which records are extracted—when possible, add the filter in the sfdcDigest Transformation transformation.

Note Filtering records extracted from Salesforce Big Objects is supported only on primary key fields in the sfdcDigest Transformation transformation.

• Structured Filter in sfdcDigest Transformation
  You define a structured filter using JSON syntax.
• Advanced Filter in sfdcDigest Transformation
  You define an advanced filter using a Salesforce Object Query Language (SOQL) WHERE clause expression. Use an advanced filter only if you are familiar with SOQL.