Salesforce Big Objects Async SOQL Retirement

Salesforce is retiring and no longer supporting the Async SOQL feature of Big Objects with the Summer ’23 release. 

Async SOQL is being retired as part of the continuous improvement of the Salesforce Platform since we’re evolving away from technologies that support Async SOQL. It’s being replaced by APIs that are more familiar to customers.

What does this change mean for me? 

You must use the Bulk API or batch Apex to query or report on custom Big Objects after your org gets upgraded to the Summer ’23 release. After the release, any jobs running Async SOQL won't be available. We recommend testing your replacement of Async SOQL in Sandbox orgs before the Summer ’23 release. Review the Salesforce trust site to plan for when your org is upgraded.

Is CRM Analytics still able to load data from BigObjects?

Yes, CRM Analytics uses Bulk API requests to load the data.

What should I use to query for more than 50k records?

Both Bulk API and batch Apex can query results larger than 50k. However, Salesforce suggests using batch Apex by triggering additional batch Apex jobs.

What do I need to do if I have millions of records and my Bulk or Apex queries are timing out?

If you have many records and want to query more efficiently, you can update your index to include the fields you need to query or change how you store your data in Big Objects. If you break up the data into separate Big Objects, you can use Async SOQL to migrate data into these new objects. If you plan to migrate data using Async SOQL, make sure to migrate before it’s retired in the Summer ’23 release.

What if I decide to migrate away from Big Objects?

If you migrate to a different solution, you can use Database.deleteImmediate to delete your data. Salesforce won't delete your data. You must delete your data yourself.

What are alternatives to Async SOQL, and how to use them?

You can query Big Objects using Bulk API and batch Apex. See the following examples of alternatives for existing Async SOQL queries.

Field Audit Trail

This example shows how to query FieldHistoryArchive and analyze results in a CSV file.

Example URI

/services/data/vXX.X/jobs/query

Example Post Request

{
    "operation": "query",
    "query": "SELECT ParentId, FieldHistoryType, Field, Id, NewValue, OldValueFROM FieldHistoryArchive WHERE FieldHistoryType = ‘Account’ AND CreatedDate > LAST_MONTH"
}

Use the Get Results for a Query Job resource.

Example CURL Request

curl --include --request GET \
--header "Authorization: Bearer token" \
--header "Accept: text/csv" \
https://instance.salesforce.com/services/data/vXX.X/jobs/query/750R0000000zxr8IAA/results ?maxRecords=50000

This request results in a CSV file to examine for auditing purposes.

Customer 360 View

Batch Apex is an alternative to Async SOQL for automated processing on a Big Object or ApiEvent, ReportEvent, or ListViewEvent. This example shows how to add processing that references correlated data.

Run a batch Apex query on a Big Object, and correlate the Contact information associated with that Big Object.

Note: You can't query an unlimited number of Big Objects in Apex. Apex heap limits of 6 MB for sync and 12 MB for async apply. For example,  if you have 6000 entries at 1KB each, you can only store 6000 records in memory.

public class QueryBigObjectAndContact implements Database.Batchable<sObject> {
    private String key;
    public QueryBigObjectAndContact(String keyParam) {
        key = keyParam
    }
    
    public Iterable<SObject> start(Database.BatchableContext BC) {
        return [SELECT Big_Object_Field__c, Account__c FROM Big_Object__b WHERE Big_Object_Primary_Key__c > key LIMIT 50000]
    }


    public void execute(Database.BatchableContext bc, List<Big_Object__b> bos){
        // process the batch of Big Objects and associate them to Accounts
        Map<Id, Big_Object__b> accountIdToBigObjectMap = new Map<Id, Big_Object__b>();
        for (Big_Object__b bigObject : bos) {
            accountIdToBigObjectMap.put(bigObject.Account__c, bigObject);
            key = bigObject.Big_Object_Primary_Key__c
        }
        Map<Id, Account> = new Map<Id, Account>(
            [SELECT Id, Name, ... FROM Account WHERE Id IN :accountIdToBigObjectMap.keySet()]
        );
        for (Id accountId : accountMap.keySet()) {
            Big_Object__b bigObject = accountIdToBigObjectMap.get(accountId);
            Account account = accountMap.get(accountId);
            // perform any actions that integrate the Big Object and Account
        }
    }
    public void finish(Database.BatchableContext bc){
        // Daisy chain additional calls using the primary key of the Big Object to get around the 50k governor limit
        QueryBigObjectAndContact nextBatch = new QueryBigObjectAndContact(key);
        Database.executeBatch(nextBatch);
    }
}

Real-Time Event Monitoring

This example shows how to query and analyze an event Big Object using a field’s contents.

public class EventMatchesObject implements Database.Batchable<sObject> {
    private String lastEventDate;


    public EventMatchesObject(String lastEventDateParam) {
        lastEventDate = lastEventDateParam;
    }


    public Iterable<SObject> start(Database.BatchableContext bc) {
        return [SELECT EventDate, EventIdentifier, QueriedEntities, SourceIp, Username, UserAgent FROM ApiEvent WHERE EventDate > lastEventDate LIMIT 50000]
    }


    public void execute(Database.BatchableContext bc, List<ApiEvent> events){
        // Process this list of entities if a certain attribute matches
        for (ApiEvent event: events) {
            String objectString = 'Patent__c';
            String eventIdentifier = event.EventIdentifier;
            if (eventIdentifier.contains(objectString) {
                // Perform actions on the event that contains 'Patent__c'
            }
            lastEventDate = format(event.EventDate);
        }
    }


    public void finish(Database.BatchableContext bc){         
        // Daisy chain additional calls using EventDate or other filter fields to get around the 50k governor limit
        EventMatchesObject nextBatch = new EventMatchesObject(lastEventDate);
        Database.executeBatch(nextBatch);
    }
}

Aggregate Queries

This example shows an alternative for aggregate queries similar to the COUNT() method.

public class CountBigObjects implements Database.Batchable<sObject> {
    private Integer recordsCounted;
    private String key;


    public CountBigObjects(Integer recordsCountedParam, String keyParam) {
        recordsCounted = recordsCountedParam
        key = keyParam
    } 


    public Iterable<SObject> start(Database.BatchableContext bc) {
        return [SELECT Custom_Field__c FROM Big_Object__b LIMIT 25000]
    }


    public void execute(Database.BatchableContext bc, List<Big_Object__b> bos){
        // process the batch of Big Objects and associate them to Accounts
        Map<Id, Big_Object__b> accountIdToBigObjectMap = new Map<Id, Big_Object__b>();
        for (Big_Object__b bigObject : bos) {
            accountIdToBigObjectMap.put(bigObject.Account__c, bigObject);
        }
        Map<Id, Account> accountMap = new Map<Id, Account>(
            [SELECT Id, Name, ... FROM Account WHERE Id IN :accountIdToBigObjectMap.keySet()]
        );
        for (Id accountId : accountMap.keySet()) {
            Big_Object__b bigObject = accountIdToBigObjectMap.get(accountId);
            Account account = accountMap.get(accountId);
            // perform any actions that integrate the Big Object and Account
        }
    }


    public void finish(Database.BatchableContext bc) {         
        // Daisy chain additional calls using the primary key of the Big Object to get around the 50k governor limit
        CountBigObjects nextBatch = new CountBigObjects(recordsCounted, key);
        Database.executeBatch(nextBatch);
    }
}

If you have more questions, open a case with Support via Salesforce Help. To read about the Salesforce approach to retiring products and features, read our Product & Feature Retirement Philosophy.