Loading
Generate a Batch Mapping File for Externally Stored Files

You are here:

  1. Salesforce Help
  2. Docs
  3. Public Sector

Generate a Batch Mapping File for Externally Stored Files

Prepare existing files for Amazon Bedrock or Amazon Kendra by generating a mapping file that associates Salesforce files with their Amazon S3 locations. You must complete this task only if you have existing files in Amazon S3 that you'd like to search.

Required Editions

User Permissions Needed
To enable External Storage Search: Customize Application
To view permission sets: View Setup and Configuration
To assign permission sets: Assign Permission Sets
To create local External Client Apps: Create, edit, and delete External Client Apps

Before you begin, verify that you have:

  • A Salesforce org with an Amazon S3 external data source of type ContentHubAWSS3. See Set Up Semantic Search.
  • An AWS account with permission to create Lambda functions, IAM roles, and S3 buckets.
  • An Amazon S3 bucket in which to store the mapping file.

Create a Salesforce Integration User

  1. From Salesforce Setup, create an integration user.
  2. Configure the user with these values.
    Field Value
    User License Salesforce Integration
    Profile Minimum Access - API Only Integrations
  3. Assign the Salesforce API Integration permission set license to the user.
  4. Create a permission set that grants these permissions:
    • API Enabled
    • Query All Files
  5. Assign the permission set to the integration user.

For more information, see Integration Users in Salesforce Developer Documentation and Create Permission Sets.

Create an External Client App

Create a local external client app that uses the OAuth 2.0 client credentials flow. In the OAuth settings, add the Manage user data via APIs (api) scope and configure the Salesforce integration user as the run-as user.

For detailed instructions, see Create a External Client App and Configure a Client Credentials Flow.

Create the Mapping File

Deploy an AWS Lambda function that queries Salesforce for externally stored files and generates a mapping between each Salesforce ContentDocument ID and its Amazon S3 location.

Configure the function through environment variables. Attach an IAM execution role with s3:PutObject that grants permission to write the mapping file to the designated S3 bucket and send logs to Amazon CloudWatch.

For instructions, see Deploy Python Lambda functions, Using Lambda environment variables, Lambda execution role in the AWS documentation.

Here is reference Lambda code:

# 1. Authenticate to Salesforce (OAuth 2.0 Client Credentials)
token = sf_oauth_token(SF_INSTANCE_URL, SF_CLIENT_ID, SF_CLIENT_SECRET)

# 2. Query ContentVersion -> ContentDocumentId, ExternalDocumentInfo2, ExternalDataSource
soql = ("SELECT ContentDocumentId, ExternalDocumentInfo2, "
        "ExternalDataSource.NamedCredentialId, ExternalDataSource.MasterLabel "
        "FROM ContentVersion WHERE IsLatest=true AND ExternalDocumentInfo2!=null "
        "AND ExternalDataSource.Type='ContentHubAWSS3'")
records = sf_query_all(token, soql)

# 3. Resolve each Named Credential's S3 endpoint URL (Tooling API)
nc_url = {nc_id: sf_named_credential_url(token, nc_id)
          for nc_id in {r["ExternalDataSource"]["NamedCredentialId"] for r in records}}

# 4. Build the mapping keyed by the external document id
mapping = defaultdict(list)
for r in records:
    nc_id = r["ExternalDataSource"]["NamedCredentialId"]
    mapping[r["ExternalDocumentInfo2"]].append({
        "ContentDocumentId": r["ContentDocumentId"],
        "DataSourceLabel": r["ExternalDataSource"]["MasterLabel"],
        "NamedCredentialId": nc_id,
        "S3_Endpoint_URL": nc_url.get(nc_id),
    })

# 5. Upload the mapping JSON to S3
boto3.client("s3").put_object(
    Bucket=S3_BUCKET_NAME, Key=S3_FILE_NAME,
    Body=json.dumps(mapping, indent=4), ContentType="application/json")
 
Loading
Salesforce Help | Article