Data Loader, special characters, file encoding, and import considerations
|Knowledge Article Number||000194097|
|Description||You may find that you're seeing mixed results upon attempting to import special characters such as á, ã, ä, ç, é, í, ó, ö, ô, ú via the Data Loader. The result in Salesforce displays unexpected tokens such as � instead of appropriate characters once the data load operation is completed.|
This behavior is the result of a combination of your import file's encoding and the Data Loader settings you have selected and is typically not an unexpected issue with, or due to a changes in, the Data Loader application.
When Configuring Data Loader you'll see the "Read all CSVs with UTF-8 encoding" setting. By checking this preference you're forcing the application to attempt to render any CSV you choose for a data operation as if it has been saved with UTF-8 encoding despite whether the file may or may not be. The combination of enabling this setting and using a file which is not encoded in UTF-8 can result in the unexpected characters.
Although the Data Loader setting influences this behavior, the source of the issue is the way the file's encoding is being set upon saving the CSV file. When using Microsoft® Excel® to save a CSV file its encoding is automatically set using machine-specific ANSI encoding and not UTF-8. It is not possible to explicitly configure or manually change this CSV file encoding behavior in Windows® or Excel®.
To resolve the unexpected character import behavior you have two options:
1. Leave the "Read all CSVs with UTF-8 encoding" setting enabled
If you commonly work with files containing special characters this method is recommended to ensure all of the characters are captured properly. However, all CSV files being used with the Data Loader will have to be encoded using UTF-8. You can use a text editor program such as Notepad® to manually change the CSV's encoding to UTF-8 from ANSI so your import file is compatible with this Data Loader setting. To manually set file encoding follow the instructions outlined in the documentation Why isn’t Data Loader importing special characters?
2. Disable the "Read all CSVs with UTF-8 encoding" preference in the Data Loader
This will allow the default ANSI character encoding set by Excel® to be passed without interference from the Data Loader.
Why have I never had to change this Data Loader setting or manually set file encoding until now?
It's likely that this is due to previously supplied files already being saved with UTF-8 encoding or they may not have contained special characters. Alternatively, the way Excel® is setting file encoding on saving CSVs may have changed on your local machine.