You are here:
Caching or Acceleration in Data Federation
Caching retrieves data from the external source and temporarily stores it in a data lake object in Data 360. When you enable acceleration while creating a data stream, you turn on caching. We recommend caching when accessing large data sets or when data changes frequently. You can refresh the cached data at your preferred frequency.
Caching is turned off by default when you create a data stream, but you can edit the data stream and enable or disable it at any time. You can incrementally refresh all objects with a datetime field as frequently as every 15 minutes. An incremental refresh updates the cached object with data that was added or changed since the last refresh, but it doesn’t update deleted data. If the refresh frequency is 24 hours or more, you can then select the time and day.
To enable incremental refresh, select Edit Data Stream after deploying a new data stream, select the desired record modified field, which denotes the timestamp for the incremental refresh, and the desired frequency. If the selected frequency is greater than or equal to 24 hours, you can then choose a desired refresh time and day. Incremental refresh currently supports upserts and doesn't support deletes. The cached object is updated with data added or changed since the last refresh. If you don't edit the data stream type, the cache refreshes fully each time.
You can also retrieve data in full refresh mode. If you opt for a full refresh, during each refresh cycle, all the existing cached data is deleted and replaced with the new dataset. To keep your data current and in sync with your data source, perform a full refresh at regular intervals. Enabling acceleration impacts the consumption of credits used for billing. When acceleration is enabled, ingesting data into the cache consumes credits in the Batch Data Pipeline usage type.
Accelerated queries consume credits at a different rate and multipliers than live queries. To learn more, see Billing Considerations for Data Federation.

