Loading
Salesforce now sends email only from verified domains. Read More
Salesforce Data Pipelines
Table of Contents
Filter by (0)Add
Select Filters

          No results
          No results
          Here are some search tips

          Check the spelling of your keywords.
          Use more general search terms.
          Select fewer filters to broaden your search.

            Search all of Salesforce Help
            Search all of Salesforce Help
            Data Pipelines Limits

            You are here:

            1. Salesforce Help
            2. Docs
            3. Salesforce Data Pipelines

            Data Pipelines Limits

            Review the limits for Salesforce Data Pipelines. If needed, you can purchase more licenses to increase the base limits.

            Required Editions

            Available in: Lightning Experience
            Available with Salesforce Data Pipelines, which is available for an extra cost in Enterprise, Performance, and Unlimited Editions
            Note
            Note If you purchased Data Pipelines and CRM Analytics, each has separate limits that are enforced independently. For example, datasets created by recipes created with CRM Analytics don’t affect the Data Pipelines limit on total rows for all datasets.

            This table lists Data Pipelines’ org-level limits.

            Value Limit Can Add On? Details
            Total rows in all datasets* 250 million Yes Each dataset supports up to 2 billion rows. If your org has less than 2 billion allocated rows, each dataset supports up to your org’s allocated rows.
            Rows processed 100 million per month Yes The total number of rows output by all recipes to all datasets in a month. Anytime a recipe updates a dataset, the number of rows processed increases. For example, each time a recipe updates a dataset with 1 million rows, the number of rows processed increases. After 4 updates, the number of rows processed is 4 million.
            Recipe job runtime 30 hours per month Yes Excludes queue time and data sync. All jobs started after time is exceeded are blocked until the next calendar month (GMT). Approximate 100 million rows can be processed in 30 hours, but the actual rows/hour depends on transformations applied and data complexity.
            Concurrent recipes 1 Yes. The maximum is 5.  
            Queries 1 million per day Yes  
            Assignable user licenses 3 Yes  
            Maximum file size for all CSV uploads in a rolling 24-hour period 20 GB No  
            Maximum number of recipes 1000 Yes This limit is shared between CRM Analytics and Data Pipelines if you purchased both. Both licenses add to the pool.
            Maximum amount of time each recipe can run 48 hours No  
            Maximum dependent jobs for an event-based schedule 5 No  

            *When you have both CRM Analytics and Data Pipelines licenses, the limit value stacks.

            Note
            Note The per month limit is the calendar month, and the accumulated value is reset at midnight of the first day of each month.

            Dataset Field Limits

            Value Limit
            Maximum number of fields in a dataset 5,000 (including up to 1,000 date fields)
            Maximum number of decimal places for each value in a numeric field in a dataset (overflow limit)

            17 decimal places

            When a value exceeds the maximum number of decimal places, it overflows. Both 100,000,000,000,000,000 and 10,000,000,000,000,000.0 overflow because they use more than 17 decimal places. A number also overflows if it’s greater (or less) than the maximum (or minimum) supported value. 36,028,797,018,963,968 overflows because its value is greater than 36,028,797,018,963,967. -36,028,797,018,963,968 overflows because it’s less than -36,028,797,018,963,967.

            When a number overflows, the resulting behavior in Data Pipelines is unpredictable. Sometimes Data Pipelines throws an error. Sometimes it replaces a numeric value with a null value. And sometimes mathematical calculations, such as sums or averages, return incorrect results. Occasionally, Data Pipelines handles numbers up to 19 digits without overflowing because they are within the maximum value for a 64-bit signed integer (263 - 1). But numbers of these lengths aren’t guaranteed to process.

            As a best practice, stick with numbers that are 17 decimal places or fewer. If numbers that overflow are necessary, setting lower precision and scale on the dataset containing the large numbers sometimes prevents overflow.
            Maximum value for each numeric field in a dataset, including decimal places

            36,028,797,018,963,967

            For example, if three decimal places are used, the maximum value is 36,028,797,018,963.967
            Minimum value for each numeric field in a dataset, including decimal places

            -36,028,797,018,963,968

            For example, if five decimal places are used, the minimum value is -36,028,797,018,9.63968
            Maximum number of characters in a field 32,000
            Note
            Note If your org hasn’t enabled Handle Null Numeric Values, the maximum number of decimal places for each value in a numeric field in a dataset is 16. All orgs created after Spring ’17 have Null Measure Handling enabled.

            Data Sync Limits

            Value Limit Can Add On?
            Maximum number of concurrent data sync runs 2 No
            Maximum number of objects that can be enabled for data sync, including local and remote objects 50 Yes. The maximum is 250. This limit is shared between CRM Analytics and Data Pipelines if you purchased both. Both licenses add to the pool. The 250 maximum applies to Data Pipelines only.
            Maximum amount of time each data sync job can run for local objects 48 hours No
            Maximum amount of time each data sync job can run for remote objects 12 hours No

            Data sync limits for each job:

            • NetSuite Connector
            • Zendesk Connector
            		 Up to 100,000 rows or 500 MB per object, whichever limit is reached first No

            Data sync limits for each job:

            • Amazon Athena Connector
            • Oracle Eloqua Connector
            		 Up to 10 million rows or 5 GB per object, whichever limit is reached first No

            Data sync limits for each* job:

            • AWS RDS Aurora MySQL Connector
            • AWS RDS Aurora PostgresSQL Connector
            • AWS RDS MariaDB Connector
            • AWS RDS MySQL Connector
            • AWS RDS PostgreSQL Connector
            • AWS RDS SQL Server Connector
            • Google Analytics Core Reporting V4 Connector
            • Google Cloud Spanner Connector
            • Marketo Connector (Beta)
            • Microsoft Azure Synapse Analytics Connector
            • Microsoft Dynamics CRM Connector
            • Salesforce Marketing Cloud Engagement OAuth 2.0 Connector
            • SAP HANA Cloud Connector
            		 Up to 20 million rows or 10 GB per object, whichever limit is reached first No

            Data sync limits for each job*:

            • Databricks Connector
            • Salesforce External Connector
            		 Up to 100 million rows or 50 GB per object, whichever limit is reached first Yes. The maximum is 2000 million rows or 1000 GB per object.

            Data sync limits for each job*✝:

            • Amazon Redshift Connector
            • Amazon S3 Connector
            • Google BigQuery for Legacy SQL Connector
            • Google BigQuery Standard SQL Connector
            • Heroku Postgres Connector
            • Microsoft Azure SQL Database Connector
            • Snowflake Input Connector
            		 Up to 250 million rows or 100 GB per object, whichever limit is reached first No

            *When using these connectors, Salesforce Government Cloud data is protected in transit with advanced encryption and can sync up to 10 million rows or 5 GB for each connected object, whichever limit is reached first.

            ✝When you have both CRM Analytics and Data Pipelines licenses, the higher limit of the two applies.

            Recipe Limits

            Value Limit
            Maximum number of Data Prep previews per hour per user 4,000
            Time out for data load jobs (CSV uploads, recipes, and data sync) that have been scheduled but not executed 5 minutes

            Output Limits

            When you have both CRM Analytics and Data Pipelines licenses, the data size limit value stacks per 24-hour period.

            Value Limit Can Add On?
            CSV data written to Salesforce 1 GB per day Yes
            Data written to Salesforce 10 GB per day Yes
            Data written to Tableau Hyper (Tableau Online)

            10 GB per day

            12-hour maximum amount of time each job can run.

            		 Yes
            Data written to Azure Data Lake

            10 GB per day

            12-hour maximum amount of time each job can run.

            		 Yes
            Data written to Amazon S3 and Snowflake (combined), including Snowflake Sync Out

            10 GB, or 10 million rows, of data per recipe run per output connector.

            50 GB per day (total for both). For example, if you push 40 GB of data to Snowflake, 10 GB of allotted data remains to push to Amazon S3.

            12 hour maximum amount of time each job can run.

            		 Yes. The maximum is 300 GB per day.

            Trending Data Limits

            Trended datasets count toward the overall Data Pipelines platform limits, including the total number of rows.

            Value Limit
            Maximum number of trended datasets per user 5
            Maximum number of rows per snapshot 100,000
            Maximum number of rows in the report to be trended 500,000 for admins, 100,000 for non-admins
            Maximum total number of rows in a trended dataset

            5,000,000
            Maximum monthly number of rows for all snapshots per org 40 million

            External Data Limits

            Value Limit
            Maximum file size per external data uploads 40 GB
            Maximum file size for all external data uploads in a rolling 24-hour period 50 GB
            Maximum number of external data jobs per dataset that can be run in a rolling 24-hour period* 50
            Maximum number of characters in a field 32,000
            Maximum number of fields in a record 5,000 (including up to 1,000 date fields)
            Maximum number of characters for all fields in a record 400,000

            *When you have both CRM Analytics and Data Pipelines licenses, the limit value stacks.

            Service Protection Limits

            ValueLimitCan Add On?
            Maxiumum concurrent API calls per org 20 No
            Maxiumum concurrent queries per org 5 No
            Maxiumum concurrent queries per user 5 No
            Maximum API calls per hour per user 10,000 No
            Maximum average number of values per multivalue field 100 No

            Service protection limits protect the Data Pipelines service from excessive loads.

            Queries for Data Pipelines Limits

            Value Limit Can Add On?
            Query calls per org per month 30,000 Yes
            Rows returned per query 10,000 No
            Maximum columns returned 10 No
            Concurrent queries per org 25 Yes. The maximum is 50. This limit is shared between CRM Analytics and Queries for Data Pipelines if you purchased both.
            Concurrent queries per user 5 Yes. The maximum is 10. This limit is shared between CRM Analytics and Queries for Data Pipelines if you purchased both.
            Assignable user licenses 100 Yes

            Data Pipelines enforces the API, dataset field, data sync, recipe, external data, query, and direct data limits enforced by CRM Analytics. For cases where Data Pipelines and CRM Analytics specify different limits for the same feature, the Data Pipelines limits take precedence.

            See Also

             
            Loading
            Salesforce Help | Article