How to check the high water mark (last value) used as the criterion for Data Cloud delta extraction.

Overview of delta extraction

The methods for extracting Data Cloud data extensions are a full refresh and delta extraction (differential extraction). Of these, delta extraction is classified into the two following types.

1. Number CDC
   1. Example: Sequential numbers such as invoice numbers
2. Date & Time CDC
   1. Example: Date/time fields such as the last modified date and time

CDC (Change Data Capture) means “differential data extraction”.

Mechanism of each extraction method

Number CDC (example: Invoice numbers)

1. Data sorted in ascending order based on number type fields.
2. The difference is extracted based on the high water mark (the maximum value)
3. If new data exceeds the maximum value, the difference is sent during the next extraction.

Example: CDC (example: last modified date and time)

1. Data sorted in ascending order based on date and time type fields.
2. The difference is extracted based on the high water mark (last date and time)
3. If new data exceeds the last date and time, the difference is sent during the next extraction.

In both cases, the high water mark is updates after each extraction, and the scope of the next extraction is determined based on that value.

How to verify the high water mark

To troubleshoot delta extraction issues and understand the situation, it is important to verify the current high watermark value. This can be verified from the “Data Factory Utility” in the Marketing Cloud Engagement, Email Studio.

Verification Steps

1. Open Interaction | Data Factory Utility from the Email Studio menu.
   
2. Click any utility from the list of utilities displayed. (The same content is displayed, regardless of which option is selected.)
3. Search for the relevant data stream information using “Ctrl + F”, etc.
   The data extension name connected to the data stream is displayed in the “Table” column.

Note

• Up to 250 items can be displayed per page. Search by changing pages if you cannot find the data you are looking for.
• When creating a data stream, this utility and the related automations are automatically generated.

Main items to verify

Troubleshooting when records are not linked in delta extraction

If data is not linked to Data Cloud during delta extraction, verify that the numerical values or date/time of the target records are not below the current high water mark. Records with values equal to or below the high water mark are excluded from differential extraction.

Key Takeaways

Understanding the high water mark mechanism and knowing the current value enables efficient data linking and troubleshooting. Keep this in mind as important operational knowledge for utilizing Data Cloud .

Contributor: Watanabe Nobuyuki | Trailblazer Community Forum Ambassador

Currently working as a Marketing Cloud Engagement consultant at NAC Co., Ltd. (FPT Japan Holdings Group). He is engaged in optimization of customer communication utilizing Marketing Cloud Engagement through support for scenario design, data design, campaign operation and personalization measures centered on e-mail, LINE and app PUSH.

Posted content reflects only the opinions of the user who made the post and does not represent the views of Salesforce. Regardless of whether a user is associated with Salesforce, posts may contain or consist of products, services, information, data, content, or other materials provided by third parties (hereinafter referred to as “Third Party Materials”). Salesforce does not manage, endorse, or guarantee any Third Party Materials (including their accuracy, validity, timeliness, completeness, reliability, consistency, quality, legality, usefulness, security, or applicable intellectual property rights), and assumes no responsibility whatsoever for them. Posts provided as answers to questions submitted via message boards or forums, or posts claiming to answer questions about Salesforce or its programs, are provided for general informational purposes only and should not be relied upon as answers to specific questions (even if an answer is marked as the “best” answer or similar qualifications are granted). For answers to specific questions, be sure to contact Salesforce Support. Salesforce has no control over posted content and assumes no responsibility for the use or misuse of posted content by third parties (including its distribution).

If you have any questions, please take advantage of the knowledge and expertise available in our Trailblazer Community: 
https://trailhead.salesforce.com/trailblazer-community/feed