You are here:
Data Stream Schedule in Data 360
Data streams can operate on different refresh schedules. Learn more about how and when these data streams update.
To see the refresh history of a data stream, access the data stream’s record page and click Refresh History.
Initiate a manual refresh of the data stream by clicking Refresh Now in the data stream’s record page.
B2C Commerce
When you create a B2C Commerce data stream using the Order Bundle, Data 360 loads 30 days of historical data. After you deploy the data stream, Data 360 pulls new data based on this schedule.
| Data Stream Name | Data Type | REFRESH MODE | REFRESH SCHEDULE |
|---|---|---|---|
BundleProduct |
Other | Full Refresh | Daily |
| GoodsProduct | Other | Full Refresh | Daily |
| MasterProduct | Other | Full Refresh | Daily |
| ProductCatalog | Other | Full Refresh | Daily |
| ProductCategory | Other | Full Refresh | Daily |
| ProductCategoryProduct | Other | Full Refresh | Daily |
| ProductOption | Other | Full Refresh | Daily |
| ProductOptionValue | Other | Full Refresh | Daily |
| ProductProductOption | Other | Full Refresh | Daily |
| SalesOrder | Engagement | Upsert | Hourly |
| SalesOrderCustomer | Profile | Full Refresh | Hourly |
Cloud Storage
Data streams that use cloud storage sources such as AWS S3 or Google Cloud Storage (GCS) are refreshed on a pre-defined schedule, or manually through the data stream record home. For more information, see Amazon S3 Storage Connector and Google Cloud Storage Connector.
Ingestion API
The Ingestion API provides a RESTful interface that supports both streaming and bulk interaction patterns for programmatically loading data into Data 360. With Bulk Ingest API you can move large amounts of data on a daily, weekly, or monthly schedule. Streaming Ingest API processes small micro-batches of records in near-real time. For information on expected latency, review Data 360 Limits and Guidelines or Customer Data Platform Limits and Guidelines.
Marketing Cloud Personalization (formerly Interaction Studio)
The Marketing Cloud Personalization connector ingests user and event data into Data 360.
- Users—This data is ingested incrementally in upsert mode. Only users who engage after the bundle is deployed are synced to Data 360.
- Standard engagement events—Data for Catalog events, Cart & CartLineItem events, and Order & OrderLineItem Events. This data is ingested incrementally in insert mode.
- Custom engagement events—This data is ingested incrementally in insert mode.
The connector is set up with a predefined bundle requiring little configuration. New Marketing Cloud Personalization data is refreshed in Data 360 hourly. Data 360 performs an incremental update of engagement data streams within 2 minutes and a full refresh of user data streams within 15 minutes.
Marketing Cloud Engagement
Data streams that use the Marketing Cloud Engagement connector are automatically scheduled to pull data either daily or hourly, depending on the type of data. No user input is required. Automation Studio handles this process. Supporting automations are scheduled dynamically, depending on when the connection was established or when the data stream was created. Don’t change the schedule after creation because it can create problems for data in transit.
| Data Type | Refresh Schedule |
|---|---|
| Standard | Daily |
| Standard Data Engagements | Hourly |
| Data Extension Full Extract | Daily |
| Data Extension Delta Extract | Hourly |
Salesforce CRM
The Salesforce CRM connector conducts full and incremental refreshes on a set schedule, requiring no action from you.
| Type | Schedule | Details |
|---|---|---|
| Full | Configurable | Periodic full refresh is disabled in new data streams, and in existing data streams within developer edition orgs, to save costs where refresh is not required. To enable periodic full refresh, you can configure its schedule by setting the full refresh interval. |
| Incremental | Every 10 minutes |
|
Formulas are refreshed during a full or incremental refresh and are recalculated if a record has changed.
To minimize unnecessary refreshes and data transfers for objects where regular refreshes are not required, perform a full or incremental refresh on demand in alignment with your business requirements. To refresh on demand, click the Refresh Now button for a specific data stream on the Data Stream tab.
You can view the refresh history on the Data Stream record home page. The refresh history includes details such as whether a full or incremental refresh occurred. If no changes have been made since the last update, the refresh history shows 0 for events.
Data streams are processed as batch or streaming. For more information, see Salesforce CRM Connector Streaming.
Mobile and Website Connectors
Mobile and Website connectors provide streaming ingestion through Engagement SDK and Salesforce Interactions SDK respectively to send data from customer-owned mobile apps and websites into Data 360. For efficiency, mobile events are queued on the device to save processing power and bandwidth.
