Why are my data sync and recipe runs getting canceled?

Unless you enable priority scheduling, CRM Analytics data sync and recipe jobs use asynchronous processing. Requests are processed in a first-in, first-out order as server resources become available. The process prioritizes synchronous over asynchronous requests, so asynchronous jobs sometimes pause as server resources reprioritize them. This behavior means that recipe, dataflow, and data sync jobs have no expectations for queue or processing time and no SLA for completion.

The CRM Analytics scheduler allows any job to be simultaneously running and queued. When an earlier request is queued for a job, attempting to run that job results in the scheduling system canceling the new job request.

CRM Analytics recipes and dataflows have a concurrency limit. If two recipes perform a write operation on a dataset, they don’t run concurrently—one runs, and the other is queued. A sandbox environment is limited to a concurrency of 1, and no more than three data sync jobs can run at the same time.

A delayed job run pushes back all other queued jobs of the same type. This behavior results in a cascade of delays that eventually leads to cancellation messages. When viewing or exploring your data, you might notice your data is stale.

To avoid this issue:

• Schedule your recipe and data sync jobs with plenty of time between runs to allow for potential delays.
• Periodically review your job runs to see how long an average job takes, and update the schedule to allow for possible delays.
• Split large data sync jobs into smaller ones using additional remote or local connections to the same data source. You can set more-frequent sync schedules for smaller groups of objects that require regular updates and infrequent sync schedules for less-updated objects.
• Enable priority scheduling to automatically queue shorter or smaller runs before longer or larger ones.