You are here:
Billing Considerations for Unstructured Data and Search Index
When you use unstructured data and search index configurations, your data is stored and processed in Data 360. Use of unstructured data and search index in Data 360 impacts the consumption of credits in the following usage types.
Use Digital Wallet to check which consumption cards are active in your org, find out how many credits are available on each card, and determine how many credits have been consumed in each usage type.
Some usage types aren’t metered for certain licenses or editions, such as a Data 360 Profiles license or Agentforce 1 Edition. Check your licenses and review license documentation for more information.
Unstructured data processing features such as search index are not available for orgs operating Data 360 under the Customer Data Platform (CDP) license.
Data 360 Usage
When you use Data 360 for data collection and processing, your org consumes credits from either the Data Services consumption card or the Flex Credits consumption card. If you have any active Data Services credits, those credits are consumed first. If you run out of Data Services credits or never had any, your org consumes Flex Credits for Data 360 services. For more information, see Data Services Billable Usage Types for Data 360 and Flex Credits Usage Types for Data 360.
To check multipliers for each usage type, see the Multipliers for Data Services Rate Card or the Agentforce & Data 360 Flex Credits Rate Card. The cost of each credit is determined by your contract.
| Digital Wallet Card Consumption Card | Usage Type | Usage Type Description | Notes |
|---|---|---|---|
| Data Services | Batch Data Pipeline (External Data Pipeline) | Usage is calculated based on the number of rows batch data processed by Data 360 data streams across all connectors, with the exception of structured data ingested via the Internal Data Pipeline. | Usage is calculated based on the number of records that are ingested in batch mode by data streams. Unstructured data may be ingested through any Data 360 connector. For unstructured data files that are only referenced - and not ingested - from an external blob store, such as Amazon S3, there is no ingestion cost. |
| Data Services | Unstructured Data Processed | Usage is calculated based on the amount of unstructured data that is processed without using AI assisted content processing and extraction. For example, if the search index processes 100 PDF documents that are 1 MB each, usage is calculated as 100 MB. If the search index processes five audio/video files that are on average 100MB each, usage is calculated as 500MB. In Data 360, unstructured data may be chunked and vectorized using an embedding model. Usage is computed only once across both these activities. For example, if one 100 MB PDF document is chunked and vectorized, usage is computed as 100 MB, not as 200MB. If you create a search index that uses a data model object (DMO) as a data source and select Text data type fields from the DMO, all strings from those fields are indexed and the sum total of all bytes processed is reported as Unstructured Data Processed. |
In Data 360, unstructured data may be chunked and vectorized using an embedding model. For audio and video files, a text transcript is created before the files are chunked. Usage is computed only once across all these activities. For example, if one 100 MB PDF document is chunked and vectorized, usage is computed as 100 MB, not as 200MB. If a video file of 1GB is transcribed, chunked, and vectorized, usage is computed as 1GB. A DMO and all its file attachments are treated as a single unit for processing purposes. For example, if Data 360 processes incremental changes either to the fields on a source DMO or to a file attachment on that DMO, all file attachments are reindexed. The cost of creating a search index remains the same for vector search index and hybrid search index. |
| Flex Credits | Data 360 Unstructured Processing | Usage is calculated based on the amount of unstructured data that is processed without using AI assisted content processing and extraction. For example, if the search index processes 100 PDF documents that are 1 MB each, usage is calculated as 100 MB. If the search index processes five audio/video files that are on average 100MB each, usage is calculated as 500MB. In Data Cloud, unstructured data may be chunked and vectorized using an embedding model. Usage is computed only once across both these activities. For example, if one 100 MB PDF document is chunked and vectorized, usage is computed as 100 MB, not as 200MB.If you create a search index that uses a data model object (DMO) as a data source and select Text data type fields from the DMO, all strings from those fields are indexed and the sum total of all bytes processed is reported as Unstructured Data Processed. |
In Data 360, unstructured data may be chunked and vectorized using an embedding model. For audio and video files, a text transcript is created before the files are chunked. Usage is computed only once across all these activities. For example, if one 100 MB PDF document is chunked and vectorized, usage is computed as 100 MB, not as 200MB. If a video file of 1GB is transcribed, chunked, and vectorized, usage is computed as 1GB. A DMO and all its file attachments are treated as a single unit for processing purposes. For example, if Data 360 processes incremental changes either to the fields on a source DMO or to a file attachment on that DMO, all file attachments are reindexed. The cost of creating a search index remains the same for vector search index and hybrid search index. |
| Data Services | Intelligent Processing | Usage is calculated based on the amount of unstructured data that is processed using AI-assisted features such as LLM-based parsing, LLM-based visual data preprocessing, image processing, and Intelligent Context. | When LLM-based parsing is used, entire documents are sent to the LLM for processing, and size of all documents is reported against "Intelligent Processing” usage type. When LLM-based visual data preprocessing is used, only content that contains visual elements or tables is sent to LLM for processing. Size of all documents from which any content is sent to the LLM for processing is reported against "Intelligent Processing” usage type. The sizes of all other documents from which no content is sent to the LLM for processing are reported against “Unstructured Data Processed” usage type. The same guideline applies for documents that are uploaded and indexed in Intelligent Context. With LLM-based image processing, the entire document is sent to the LLM for processing, and size of the document is reported against "Intelligent Processing” usage type. |
| Flex Credits | Data 360 Intelligent Processing | Usage is calculated based on the amount of unstructured data that is processed using AI-assisted features such as LLM-based parsing, LLM-based visual data preprocessing, image processing, and Intelligent Context. | When LLM-based parsing is used, entire documents are sent to the LLM for processing, and size of all documents is reported against "Intelligent Processing” usage type. When LLM-based visual data preprocessing is used, only content that contains visual elements or tables is sent to LLM for processing. Size of all documents from which any content is sent to the LLM for processing is reported against " Data 360 Intelligent Processing” usage type. The sizes of all other documents from which no content is sent to the LLM for processing are reported against “ Data 360 Unstructured Data Processed” usage type. The same guideline applies for documents that are uploaded and indexed in Intelligent Context. With LLM-based image processing, the entire document is sent to the LLM for processing, and size of the document is reported against " Data 360 Intelligent Processing” usage type. |
| Data Services | Data Queries | Usage is calculated based on the number of records processed. The count of records processed depends on the structure of a query as well as other related factors such as the total number of records in the objects being queried. |
For vector search queries against unstructured data, the number of vectors in the search index are counted. For hybrid search queries against unstructured data, the number of vectors and keyword records in the search index are counted. In a typical search index, the number of keyword records is the same as the number of vectors. |
| Flex Credits | Data 360 Queries | Usage is calculated based on the number of records processed. The count of records processed depends on the structure of a query as well as other related factors such as the total number of records in the objects being queried. When an object is queried in the context of a data space, the number of rows processed is based on the total number of processed rows in the source object, not the number of rows pertaining to the particular data space |
For vector search queries against unstructured data, the number of vectors in the search index are counted. For hybrid search queries against unstructured data, the number of vectors and keyword records in the search index are counted. In a typical search index, the number of keyword records is the same as the number of vectors. |
Data 360 Storage Usage
Storage costs are determined by your contract. Use Digital Wallet to check your current usage.
| Digital Wallet Consumption Card | Usage Type | Usage Type Description | Notes |
|---|---|---|---|
| Data Storage | Storage Beyond Allocation | Usage is calculated based on the amount of storage used above the amount allocated. | Every file ingested and table created, including unstructured data lake objects (UDLO), unstructured data model objects (UDMO), CDMO, or index data model objects (index DMO), count toward Data 360 data storage, including the following.
|
Agentforce and Generative AI Usage
When your org uses an Agentforce or generative AI, it consumes Agentforce Flex Credits.
For more information, see Flex Credits Billable Usage Types
To check multipliers for each usage type, see the Agentforce & Data 360 Flex Credits Rate Card. The cost of each credit is determined by your contract.
| Digital Wallet Consumption Card | Usage Type | Usage Type Description | Notes |
|---|---|---|---|
| Flex Credits | Standard Prompts Basic Prompts Advanced Prompts |
Usage is calculated based on two factors: the number of direct requests to the LLM via the LLM gateway, and whether the gateway uses a Salesforce managed large language model. The specific category depends on the model that is used. See Large Language Model Support to find out which usage types apply. All Standard, Basic, and Advanced prompts process up to 2,000 tokens per prompt. Token usage is rounded up in 2,000-token increments. All Standard, Basic, and Advanced prompts that exceed this limit will be metered as multiple prompts, with each additional 2,000-token chunk counting as a new prompt. For example, a prompt with a total of 6,500 input and output tokens will be metered as 4 prompts. Tokens are units of data processed by the AI models. |
For search indexes that use enriched indexing, calls to the LLM to generate enriched chunks are counted as Standard Prompts. For more information see, Flex Credits Billable Usage Types. |
For more information on how usage is billed, refer to your contract or contact your account executive.
See Also
- Salesforce Help: Data 360 Billable Usage Types
- Salesforce Help: Data Services Billable Usage Types for Data 360
- Salesforce Help: Flex Credits Billable Usage Types
- Downloadable PDF: Understanding Credit Consumption: Breaking Down a Use Case
- Trailhead: Get Started with Data 360 Consumption
- Salesforce Help: Generative AI Billable Usage Types
- Salesforce Help: Metering for Agentforce and Generative AI Usage
- Salesforce Help: Flex Credits Billable Usage Types
