Limit Ingestion of Salesforce Object File Attachments into Data 360

Options to Limit File Ingestion

Consider the following options to control which files are processed in your Search Index:

Option 1: "Veto" Files in Salesforce

How It Works:
Two user permission options can be applied to the Data 360 Connector user in Salesforce:

• Query All Files - Queries all files regardless of status
• Query Non Vetoed Files - Limits access to public, non-vetoed files only

Files can be marked as vetoed by setting the IsInternalOnly field on the ContentDocument object in Salesforce.

Implementation Requirements:

• IsInternalOnly is not a field on the UI and needs to be set programmatically (Flow, Apex, REST API, or external integration)
  • Salesforce Flow: Create a record-triggered flow to set IsInternalOnly on file creation based on criteria
  • Apex Trigger: Implement trigger logic on ContentDocument to set the field programmatically
  • External Integration: Use REST API from external systems to mark ContentDocument records as vetoed
• This setting does not affect file access for other users within Salesforce
• Only affects what the Data 360 Connector can ingest

Expected Results:
Once set to true, the file will be removed from the ContentDocument data stream/DLO in Data 360.

Use Case: Best for organizations that want to systematically exclude specific files from Data 360 ingestion based on business rules or file attributes.
Resources:

• Set Up Users to Query Salesforce Files
• ContentDocument Object Reference

Option 2: Data Space Filters

How It Works:
Search Index processing respects Data Space Filters. Files will only be processed if their linked records are included in your Data Space. You can use Data Space Filters to limit records processed during the creation of the Search Index.
Important Considerations:

• Files are still ingested into Data 360
• Files are excluded from processing for the Search Index only
• No consumption charge for using Data Space Filters
• Some limitations exist on filter logic complexity
• Formula fields can be used to create more complex logic in Data Space filters

Use Case: May serve as an alternative to Data Transforms for limiting records processed during Search Index creation, provided that the filtering logic is simple enough for Data Space Filters to support.
Resources:

• Add Filters to a Data Lake Object

Option 3: Data Transforms

How It Works:
Data Transforms allow you to filter the records from one DLO or DMO and output them filtered subset to a new DLO or DMO. A Search Index can then be created on the filtered DMO to limit the file attachments processed.
Important Considerations:

• Files are still ingested into Data 360
• Data Transforms have a consumption cost based on number of rows processed
• Incurs both additional cost, latency and management overhead

Recommendations

1. Use Option 1 (Veto Files) if you want to prevent files from entering Data 360 entirely and reduce storage usage
2. Use Option 2 (Data Space Filters) if you want files available in Data 360 but excluded from specific Search Indexes
3. Combine both options for maximum control: veto sensitive files and use Data Space Filters for business logic filtering
4. Always test your configuration in a sandbox before implementing in production
5. Monitor your consumption regularly and adjust filters as needed