CRM Connector Streaming

Batch and stream processing handle data differently. Batch mode checks the data once at scheduled intervals of 10-15 minutes, and then processes and stores it in a Data Lake Object (DLO) that you can access. Streaming mode processes data with minimal latency, and reflects changes as soon as they're made in the org.

All data streams ingest the initial data in batch mode, and if they're eligible, they move to streaming mode, which continuously processes data with minimal latency.

When you add new source fields to a data stream, or set up a new data stream, exclude batch ingested fields from the stream for faster streaming ingestion. If you prefer periodic batch mode processing, leave them in. See Create a CRM Data Stream.

The streaming process checks if a full refresh is required. The first instance of a full refresh always uses batch processing. See Data Stream Schedule in Data 360.

What's the Processing Mode of My Data Stream?

To see the procesing mode of your data stream, access the data stream's record page. The Frequency field indicates the type of processing mode.

Why Is My Data Stream Not Operating in Streaming Mode?

For streaming to take place, make sure the required permissions are set and validation checks are met. Streaming isn't available under the following conditions, which change the processing mode to batch:

• The object, object type, or derived field don't support streaming. See Object Support for CRM Data Streams.
• You include batch ingested standard or custom formula fields in your data stream. Learn how to create a Salesforce CRM cdata stream.
• Your data stream has custom entities where the deployment status is In Development instead of Deployed. See also Troubleshooting CRM Connections.
• You participate in the Filtering for Salesforce CRM Connector pilot and have defined a filter on a stream.
• You modified the Data 360 Salesforce Connector permission set by adding users. When you connect to an org, the Data 360 Salesforce Connector permission set is created and allows access to Data 360 objects and fields. This permission set is intended for the Platform Integration User. Adding other users to this permission set prevents Data 360 from assigning the necessary permissions that are needed for streaming.