Print this page

Upsert CSV with duplicate entries using Data Loader

Knowledge Article Number 000004847

What happens if I do an upsert with the Data Loader, and my CSV file contains duplicate entries (for example two records with the same ID) ?



This is down to the Batch Size:

The Data Loader processes the data by batch size, for example if the batch size is 200, it will upsert the 200 first records of the file, then the 200 next ones, etc...

You will have troubles if you have duplicate entries in the same batch, no records will be upserted.

There are two solutions:

1)Ensure that the CSV file contains no duplicates, and keep a decent batch size (i.e. 200).

2) Set the batch size to one.

-> When using the Data Loader manually: go to Settings > Batch Size (first option)

->When using the Data Loader with the command line: go to the xml file and change

<entry key="sfdc.loadBatchSize" value="200"/> To <entry key="sfdc.loadBatchSize" value="1"/> 

And in this case, for each duplicate, the last record of the file will be the winner.

However, doing such change will slow down the operation by 200, but this might be unsignificant - the best thing is to test.

promote demote