Geo-Aware Routing for OpenAI and Azure OpenAI
A geo-aware model, such as GPT 4 Omni, uses OpenAI or Azure OpenAI to service LLM requests. The same underlying model is used by OpenAI and Azure OpenAI.
Required Editions
| Available in: Enterprise, Performance, and Unlimited Editions with an Einstein for Sales, Einstein for Platform, Einstein for Service, Einstein 1 Service, or Einstein GPT Service add-on. To purchase add-ons, contact your Salesforce account executive. |
Requests in the United States are serviced in the United States by OpenAI, and in some cases Azure OpenAI. Requests outside the United States are routed to a supported Azure OpenAI endpoint that is closest to where your Einstein generative AI platform instance is located.
Azure OpenAI
This table shows geo-aware routing for Azure OpenAI requests for GPT 4 Omni. Each geo-aware model has its own set of model regions where model endpoints are deployed. Models other than GPT 4 Omni may not have model endpoints in all the regions listed in the table. For more information for each model, see Large Language Model Support.
The columns are:
- Salesforce Org Region: The region where a Salesforce org is hosted
- Primary Model Endpoint Region: The primary region for the Azure OpenAI endpoint
- Secondary Model Endpoint Regions: The secondary in-region model endpoints where model requests are retried if the primary model endpoint isn't responsive. If the primary and secondary endpoints aren't responsive, the request falls back to Azure OpenAI in the United States. If the request fails on each primary and secondary model endpoint and the model endpoints in the United States, an error is returned. See the Disable Fallback to the United States section after the table.
| Salesforce Org Region | Primary Model Endpoint Region | Secondary Model Endpoint Regions |
|---|---|---|
| Australia | Australia | None |
| Brazil | Brazil | None |
| Canada | Canada | None |
| France | France |
|
| Germany | Germany |
|
| India | India | None |
| Indonesia | Australia | None |
| Japan | Japan | None |
| South Korea | Japan | None |
| Singapore | Australia | None |
| Sweden | Sweden |
|
| Switzerland | Switzerland |
|
| United Kingdom | United Kingdom | None |
| United States | United States | None |
Disable Fallback to the United States
Before you can disable the final fallback to the United States, you must turn on Data 360. Data 360 is automatically provisioned as soon as a Data Cloud license is added to your Salesforce org. See Turn On Data 360.
To disable the final fallback to the United States, go to Einstein Setup and toggle Turn Off Model Provider Region Fallback. Instead of falling back to the United States, an error is returned. Requests are still tried on the primary and secondary model endpoints before an error is returned.
Batch Processing
To asynchronously generate large quantities of responses for prompt templates, you can use prompt template batch processing. These models support a global endpoint for native batch processing by Azure OpenAI.
- GPT 4 Omni
- GPT 4 Omni Mini
- GPT 4.1
For these models with global endpoints, batch requests may be processed in regions outside your org’s region, even if non-batch LLM requests are restricted to specific in-region Azure endpoints. Batch processing doesn't follow geo-aware routing rules and doesn’t provide in-region enforcement.
For more information about batch processing, see Prompt Template Batch Processing in Apex in the Agentforce Developer Guide.
