Loyalty Management: Data Processing Engine fails with MaxRecipeOutRowsPerMonth or recipe internal error

Identify which error you are receiving, then follow the matching cause section.

Cause 1: MaxRecipeOutRowsPerMonth exceeded

1. Navigate to Setup > Analytics > Settings > Limits Monitoring (or Setup > Data Cloud Setup > Limits Monitoring) to confirm current usage of "Output Rows Per Month".
2. Navigate to Setup > Data Processing Engine and identify the highest-volume jobs by reviewing the Monitor Workflow Services view via the BatchJob object (fields TotalInputRecordCount, TotalProcessedRecordCount, UtilisedExecutionLimit, UtilisedWritebackLimit).
3. Reduce output volume in the affected definitions:
   • Edit each Source Node to add historical data filters so older records are excluded from the read.
   • On every Writeback Object node, enable Update changed records only so DPE only writes records that have changed since the last run.
   • Remove unnecessary fields from writeback nodes by unselecting fields not required downstream.
4. If legitimate business need exceeds 10,000,000 rows per month, contact your Salesforce Account Executive to purchase the Data Pipelines Add-On license. Support cannot raise this limit without the add-on.
5. Wait for the rolling monthly window to reset, or activate the add-on, then re-run the DPE definition from Setup > Data Processing Engine > [Definition] > Run.

Cause 2: Recipe internal error (E4-0-[ID]) — stale field reference

1. Navigate to Analytics Studio > Data Manager > Recipes and open the recipe whose ID matches the failed DataflowInstance.
2. Click the Input Source node, then click Change Columns. Unselect one column and reselect it to force the recipe JSON to regenerate. Click Next, then Save and Run.
3. If the error persists, click Download recipe JSON in the Recipe UI. Open the file, search for the field name listed in the error, delete that field reference, and save the file.
4. In the Recipe UI, click Upload recipe JSON, select the edited file, click Open, then click Save and Run.

Cause 3: InvalidBatch — relationship not found on Source Node

1. Navigate to Setup > Data Processing Engine and open the failing DPE definition.
2. Locate the Source Node that references the relationship name in the error.
3. Delete the Related Object configuration on that Source Node.
4. Add the related object as a separate Source Node.
5. Add a Join Node and join the original source to the new source on the appropriate key field.
6. Save and activate the DPE definition.

Cause 4: DataflowInstance cancelled — memory limit exceeded

1. Reduce volume per run by adding filters on each Source Node so only the delta of records since the last run is processed.
2. Split the large DPE into multiple smaller DPE definitions, each scoped to a subset of records.
3. Re-run the DPE definition. If it continues to fail with an out-of-memory error, contact Salesforce Support and request a memory-limit increase — this requires internal SME action and is not customer-configurable.

Confirm the fix: Re-run the affected DPE definition from Setup > Data Processing Engine > [Definition] > Run and verify the BatchJob record reaches Status = Completed in Monitor Workflow Services without the original error in ErrorDescription.