You are here:
Batch Job Modes for Customizable Rollups
Cusomizable Rollups are calculated by a series of batch jobs. These batch jobs run in one of three modes: Non-Skew Mode, Skew Mode, or Incremental Mode.
The default is Non-Skew Mode, which is the most efficient mode for orgs without large data volumes or uneven data distribution (also called data skew). Learn more about data skew on the Salesforce Developers' Blog.
For orgs with data skew, NPSP automatically uses Skew Mode which runs skewed records separately so that rollup jobs can complete successfully.
In orgs with large data volumes, batch jobs may time out or take more than 24 hours to process. If this is the case, NPSP automatically uses Incremental Mode. Incremental Mode only evaluates Account and Contact records with Opportunities that were modified within a certain timeframe. It is currently only available for Contact Hard Credit and Account Hard Credit Non-Skew batch jobs.
Read on to learn more about Skew Mode, Incremental Mode, and how they work with NPSP batch jobs.
- Skew Mode
NPSP uses Skew Mode jobs when a single record has many records (more than 250 by default) related to it. - Incremental Mode
In an effort to mitigate issues in large data volume orgs, we added Incremental Mode. Incremental processing helps to reduce overall data within the Customizable Rollups batch jobs and also reduce the occurrence of a long-running batch job. - Customizable Rollups Jobs
This is a complete list of each Customizable Rollup job along with its related Apex Classes, associated NPSP Settings, and the fields it evaluates during processing. There's also a high level description of how each job works behind the scenes.
Skew Mode
NPSP uses Skew Mode jobs when a single record has many records (more than 250 by default) related to it.
You can also select Customizable Rollups: Force Skew Mode on specific Accounts or Contacts with more than 250 related records to overcome CPU timeouts or heap size limits.
Without Skew jobs, rollup calculations might fail. Customizable Rollups use Skew jobs to process these skewed records separately so that rollup jobs can complete successfully. Additionally, to keep processing within Salesforce Governor Limits, Skew Mode jobs use a Skew Mode Dispatcher that breaks the rollups into smaller groups of records (the default is 300).
Skew Mode Settings
You can turn on Skew Mode for individual Accounts or Contacts with the Customizable Rollups: Force Skew Mode field or find the Skew Mode settings in NPSP Settings, under Bulk Data Processes | Batch Process Settings.
- Skew Mode Threshold—Skew mode is used when an Account, Contact, GAU, or Recurring Donation record has this number of related Opportunities.
- Contact Skew Mode Batch Size—The number of related records, sorted by ContactId, that the batch job processes at one time.
- Account Skew Mode Batch Size—The number of related records, sorted by AccountId, that the batch job processes at one time.
- Skew Mode Dispatcher Size—The number of Accounts, Contacts, or GAUs that the batch job queries and rolls up data for at one time.
Incremental Mode
In an effort to mitigate issues in large data volume orgs, we added Incremental Mode. Incremental processing helps to reduce overall data within the Customizable Rollups batch jobs and also reduce the occurrence of a long-running batch job.
Incremental Mode only evaluates Account and Contact records with Opportunities that were modified within a certain timeframe. By default, this timeframe is the value of the largest N-Day rollup. NPSP's largest N-Day rollup by default is 2 years, but you can change it or create your own N-Day rollups.
Let's look at an example of how this works. The Contact Hard Credit job queries for Contacts listed in the Primary Contact field of an Opportunity that has been modified within the last two years (or whatever the largest N-Day timeframe is) and then processes all Contact Hard Credit rollups.
Incremental Mode is useful for:
- Orgs that have not recently imported Opportunities (Last Modified Date is over 2 years ago, by default).
- Orgs with smaller N-Day Rollups (N-Day Rollups = rolling totals — i.e. Total Gifts in Last 90 days, etc.).
- Enable Incremental Mode
Incremental mode is enabled and ready for use in orgs created from a Trial starting with NPSP version 3.155 (May 2019). If your org existed before this date and you have large data volumes, you can enable it. - Override the N-Day Value
You can override the default N-Day values if needed.
Enable Incremental Mode
Incremental mode is enabled and ready for use in orgs created from a Trial starting with NPSP version 3.155 (May 2019). If your org existed before this date and you have large data volumes, you can enable it.
- Click
, then click Setup. - In the Quick Find box, enter Custom Settings, then click Custom Settings.
- In the row for Customizable Rollup Settings, click Manage.
- Click Edit.
- Select these settings:
-
Incremental Contact Hard Credit Non-Skew—When checked, the Contact Hard Credit (Non-Skew) nightly job limits the number of Contacts selected for recalculation by including only records with at least one recently updated Opportunity.
-
Incremental Account Hard Credit Non-Skew—When checked, the Account Hard Credit (Non-Skew) nightly job limits the number of Accounts selected for recalculation by including only records with at least one recently updated Opportunity.
-
Limit Recalculated Recurring Donations—When checked, the Recurring Donation (Non-Skew) nightly job limits the number of recurring donations that are recalculated by including only records with at least one recently updated Opportunity.
-
- Save the settings.
Override the N-Day Value
You can override the default N-Day values if needed.
For example, let's say you want to filter on Opportunities with Close Dates in the last 90 days, you would enter CloseDate in the Incremental: Last N Days Field Override field and 90 in the Incremental: Last N Days Value Override field. Or if you were migrating Opportunities from a legacy system and wanted to include the last modified date of Opportunities from a custom field (called Legacy_System_Modified_Date__c) in the last 180 days, you would enter Legacy_System_Modified_Date__c in the Incremental: Last N Days Field Override field and 180 in the Incremental: Last N Days Value Override field.
Be sure to test thoroughly after making changes, as N-Day rollups may be incorrect if these fields aren't set properly. For example, if you have a rollup for Total Gifts in Last 45 Days, but you override the defaults to process Accounts/Contacts with Opportunities modified in the last 30 days, the Total Gifts in Last 45 Days rollup could be incorrect since data may have dropped off.
- Click , then click Setup.
- In the Quick Find box, enter Custom Settings, then click Custom Settings.
- In the row for Customizable Rollup Settings, click Manage.
- Click Edit.
-
To limit the Recurring Donation (non-Skew) nightly scheduled job:
-
Select Limit Recalculated Recurring Donations.
-
In RD Modified in Last Number of Days, enter the number of days after being modified that an Opportunity should be included in the Recurring Donation Rollup batch job. For example, only include Opportunities that have been modified within the last 31 days.
-
- To limit all other nightly scheduled jobs, enter values in these fields:
-
Incremental: Last N Days Field Override—Overrides the field used to filter related Opportunities to determine which Accounts and Contacts are recalculated during a nightly scheduled job. The value must be a Date field on the Opportunity object.
-
Incremental: Last N Days Value Override—Overrides the calculated number of days used to filter related Opportunities to determine which Accounts and Contacts are recalculated during a nightly scheduled job. The value must be a number.
Note If both Incremental: Last N Days Field Override and Incremental: Last N Days Value Override are populated, the field specified in Incremental: Last N Days Field Override is used to test the value in that field against the numerical range specified by Incremental: Last N Days Field Override.
-
- Save the settings.
Customizable Rollups Jobs
This is a complete list of each Customizable Rollup job along with its related Apex Classes, associated NPSP Settings, and the fields it evaluates during processing. There's also a high level description of how each job works behind the scenes.
This information is intended for use by Developers and Partners.
- Scheduled Job: NPSP 01A - Customizable Rollups - Account Hard Credit
How It Works: Query for all Accounts with Opportunities WHERE npsp__CustomizableRollups_UseSkewMode__c = false. Process Account rollups 200 records at a time and query for relevant data.Batch Job Class Key Field Related Settings CRLP_Account_BATCH On Account: Customizable Rollups: Force Skew Mode (Account.npsp__CustomizableRollups_UseSkewMode__c) Account Hard Credit Batch Size
Skew Mode Threshold
- Scheduled Job: NPSP 01B - Customizable Rollups - Account Hard Credit Skew
-
Batch Job Class Key Field Related Settings CRLP_AccountSkew_BATCH CRLP_SkewDispatcher_BATCH On Account: Customizable Rollups: Force Skew Mode (Account.npsp__CustomizableRollups_UseSkewMode__c) Account Skew Mode Batch Size
Skew Mode Threshold
Skew Mode Dispatcher Batch Size
How it Works: In batches of 300, use the Skew Dispatcher to query for Accounts WHERE npsp__CustomizeRollups_UseSkewMode__c = true. Query for Opportunities related to those Accounts, 1,000 at a time, and process rollups until all Opportunities are looped through for those Account Batches. Then, query for the next set of Accounts and their Opportunities, and repeat.
- Scheduled Job: NPSP 02A - Customizable Rollups - Contact Hard Credit
-
Batch Job Class Key Field Related Settings CRLP_Contact_BATCH On Contact: Customizable Rollups: Force Skew Mode (Contact.npsp__CustomizableRollups_UseSkewMode__c) Contact Hard Credit Batch Size
Skew Mode Threshold
How it Works: Query for all Contacts that are in the Primary Contact field (npsp__Primary_Contact__c) of an Opportunity WHERE npsp__CustomizableRollups_UseSkewMode__c = false. Process Contact rollups 200 records at a time and query for relevant data. If Always Roll Up to Primary Contact is unchecked in NPSP Settings, additionally filter for Opportunity Account's WHERE npe01__SYSTEMIsIndividual__c = True.
- Scheduled Job: NPSP 02B - Customizable Rollups - Contact Hard Credit Skew
-
Batch Job Class Key Field Related Settings CRLP_ContactSkew_BATCH
CRLP_SkewDispatcher_BATCH
On Contact: Customizable Rollups: Force Skew Mode (Contact.npsp__CustomizableRollups_UseSkewMode__c) Contact Skew Mode Batch Size
Skew Mode Threshold
Skew Mode Dispatcher Batch Size
How it Works: In batches of 300, use the Skew Dispatcher to query for Contacts WHERE npsp__CustomizableRollups_UseSkewMode__c = true. Query for Opportunities related to those Contacts (in the Primary Contact field), 1,000 at a time, and process rollups until all Opportunities are looped through for those Contact batches. Then query for the next set of Contacts and their Opportunities, and repeat. Note that if Always Roll Up to Primary Contact Role is unchecked in NPSP Settings, additionally filter on Opportunities WHERE npe01__SYSTEMIsIndividual__c = TRUE.
- Scheduled Job: NPSP 03A - Customizable Rollups - Account-level Contact Soft Credit
-
Batch Job Class Key Field Related Settings CRLP_Account_SoftCredit_BATCH On Account: Customizable Rollups: Force Skew Mode (Account.npsp__CustomizableRollups_UseSkewMode__c) Contact Soft Credit to Acct Batch Size
Skew Mode Threshold
How it Works: Query for all Accounts WHERE npsp__CustomizableRollups_UseSkewMode__c = false. Process Account rollups 200 records at a time by querying for Opportunity Contact Roles where the Contact's Account ID is in the batch of Account IDs and IsPrimary = false. If Always Roll Up to Primary Contact Role is unchecked in NPSP Settings, also add an OR filter for the Opportunity Account's npe01__SYSTEMIsIndividual__c = false. When querying for OCRs, this job will also query for relevant Partial Soft Credit records.
- Scheduled Job: NPSP 03B - Customizable Rollups - Account Soft Credit
-
Batch Job Class Key Field Related Settings CRLP_Account_AccSoftCredit_BATCH On Account: Customizable Rollups: Force Skew Mode (Account.npsp__CustomizableRollups_UseSkewMode__c) Account Soft Credit Batch Size
Skew Mode Threshold
How it Works: Query for all Accounts WHERE npsp__CustomizableRollups_UseSkewMode__c = false. Process Account rollups 200 records at a time by querying for/processing Account Soft Credit records Where the Account ID is in the batch of Account IDs.
- Scheduled Job: NPSP 03C - Customizable Rollups - Account-level Contact Soft Credit Skew
-
Batch Job Class Key Field Related Settings CRLP_AccountSkew_SoftCredit_BATCH
CRLP_SkewDispatcher_BATCH
On Account: Customizable Rollups: Force Skew Mode (Account.npsp__CustomizableRollups_UseSkewMode__c) Account Skew Mode Batch Size
Skew Mode Threshold
Skew Mode Dispatcher Batch Size
How it Works: In batches of 300, use the Skew Dispatcher to query for Accounts WHERE npsp__CustomizableRollups_UseSkewMode__c = true. Query for Opportunity Contact Roles related to those Contacts' Account IDs, 1,000 at a time, and process rollups until all Opportunity Contact Roles are looped through for those Account batches. Then query for the next set of Accounts and their Opportunity Contact Roles, and repeat. Note that if Always Roll Up to Primary Contact Role is unchecked in NPSP Settings, additionally filter on Opportunities WHERE npe01__SYSTEMIsIndividual__c = false OR IsPrimary = false. (Otherwise, we just filter on IsPrimary = false). When querying for OCRs, this job will also query for relevant Partial Soft Credit records.
- Scheduled Job: NPSP 03D - Customizable Rollups - Account Soft Credit Skew
-
Batch Job Class Key Field Related Settings CRLP_AccountSkew_AccSoftCredit_BATCH
CRLP_SkewDispatcher_BATCH
On Account: Customizable Rollups: Force Skew Mode (Account.npsp__CustomizableRollups_UseSkewMode__c) Account Skew Mode Batch Size
Skew Mode Threshold
Skew Mode Dispatcher Batch Size
How it Works: In batches of 300, use the Skew Dispatcher to query for Accounts WHERE npsp__CustomizableRollups_UseSkewMode__c = true. Query for Account Soft Credits related to those Account IDs, 1,000 at a time, and process rollups until all Account Soft Credits are looped through for those Account batches. Then query for the next set of Accounts and their Account Soft Credits, and repeat.
- Scheduled Job: NPSP 04A - Customizable Rollups - Contact Soft Credit
-
Batch Job Class Key Field Related Settings CRLP_Contact_SoftCredit_BATCH On Contact: Customizable Rollups: Force Skew Mode (Contact.npsp__CustomizableRollups_UseSkewMode__c) Contact Soft Credit Batch Size
Skew Mode Threshold
How it Works: Query for all Contacts WHERE npsp__CustomizableRollups_UseSkewMode__c = false and that are in the Contact ID of an OCR. Process Contact rollups 200 records at a time by querying for Opportunity Contact Roles where the Contact ID is in the batch of Contact IDs. Note that if Always Roll Up to Primary Contact Role is unchecked in NPSP Settings, additionally filter on Opportunities WHERE npe01__SYSTEMIsIndividual__c = false OR IsPrimary = false. (Otherwise, we just filter on IsPrimary = false). When querying for OCRs, this job will also query for relevant Partial Soft Credit records.
- Scheduled Job: NPSP 04B - Customizable Rollups - Contact Soft Credit Skew
-
Batch Job Class Key Field Related Settings CRLP_ContactSkew_SoftCredit_BATCH
CRLP_SkewDispatcher_BATCH
On Contact: Customizable Rollups: Force Skew Mode (Contact.npsp__CustomizableRollups_UseSkewMode__c) Contact Skew Mode Batch Size
Skew Mode Threshold
Skew Mode Dispatcher Batch Size
How it Works: In batches of 300, use the Skew Dispatcher to query for Contacts WHERE npsp__CustomizableRollups_UseSkewMode = true. Query for Opportunity Contact Roles related to those Contacts IDs, 1,000 at a time, and process rollups until all Opportunity Contact Roles are looped through for those Contact batches. Then query for the next set of Contacts and their Opportunity Contact Roles, and repeat. Note that if Always Roll Up to Primary Contact Role is unchecked in NPSP Settings, additionally filter on Opportunities' Accounts WHERE npe01__SYSTEMIsIndividual__c = false OR IsPrimary = false. (Otherwise, we just filter on IsPrimary = false). When querying for OCRs, this job will also query for relevant Partial Soft Credit records.
- Scheduled Job: NPSP 05 - Customizable Rollups - General Accounting Units
-
Batch Job Class Key Field Related Settings CRLP_GAU_BATCH
CRLP_SkewDispatcher_BATCH
N/A GAU Batch Size
Skew Mode Dispatcher Batch Size
How it Works: In batches of 300, use the Skew Dispatcher to query for General Accounting Units. Query for GAU Allocations related to those General Accounting Units, 1,000 at a time, and process rollups until all GAU Allocations are looped through for those General Accounting Unit batches. Then query for the next set of General Accounting Units and their GAU Allocations, and repeat.
- Scheduled Job: NPSP 06A - Customizable Rollups - Recurring Donations
-
Batch Job Class Key Field Related Settings CRLP_RD_BATCH Total Paid Installments (npe03__Total_Paid_Installments__c) Recurring Donation Batch Size
Skew Mode Threshold
Limit Recalculated Recurring Donations
RD Modified in Last Number of Days
How it Works: Query for all Recurring Donations that are in the Recurring Donation field of an Opportunity WHERE (npe03__Total_Paid_Installments__c < 250 OR npe03__Total_Paid_Installments__c = null) ; Process Recurring Donation rollups 50 records at a time and query for relevant data. Use the Limit Recalculated Recurring Donations and RD Modified in Last Number of Days settings to limit the number of recurring donations that are recalculated based on the recency of the donation.
- Scheduled Job: NPSP 06B - Customizable Rollups - Recurring Donations Skew
-
Batch Job Class Key Field Related Settings CRLP_RDSkew_BATCH
CRLP_SkewDispatcher_BATCH
Total Paid Installments (npe03__Total_Paid_Installments__c) Recurring Donation Batch Size
Skew Mode Threshold
Skew Mode Dispatcher Batch Size
How it Works: In batches of 300, use the Skew Dispatcher to query for Recurring Donations WHERE npe03__Total_Paid_Installments__c >= 250. Query for Opportunities related to those Recurring Donations, 50 at a time, and process rollups until all Opportunities are looped through for those Recurring Donation batches. Then query for the next set of Recurring Donations and their Opportunities, and repeat.
