Geo-Aware LLM Request Routing on the Einstein Generative AI Platform

Supported Models

Geo-aware routing is available to these models, provided that the models are available in the relevant regions.

• OpenAI models (such as GPT-4 Omni). These geo-aware models use OpenAI or Azure OpenAI to service LLM requests. The same underlying model is used by OpenAI and Azure OpenAI. See Geo-Aware Routing for OpenAI and Azure OpenAI.
• Anthropic models (hosted on Amazon Bedrock)

Availability

Geo-aware routing is available to:

• AI agents
• Salesforce applications and features that use OpenAI and Anthropic models through the Einstein generative AI platform
• Customers who use OpenAI and Anthropic models through Models API or Prompt Builder.

To find out whether geo-aware routing is enabled for any specific Salesforce AI feature, refer to its documentation or contact your Salesforce account executive.

Proximity and Routing

Proximity to the nearest LLM server is determined by the region in which your Einstein generative AI platform instance is located. If you enabled the Einstein generative AI platform on or after June 13, 2024, then your Einstein generative AI platform region is the same as your Data 360 region (Data 360: Data Center Locations). Otherwise, contact your Salesforce account executive to learn where it’s provisioned.

Routing in the Models API

If you use the Einstein generative AI platform directly through Models API, then it's recommended that you use model API names for geo-aware models.

Feedback and Audit

To view where your LLM requests are being routed to, refer to Feedback and Audit features.

Feature-specific Support

To track whether an Einstein feature supports geo-aware LLM request routing, see the feature's documentation.