Loading
Feature degradation | Gmail Email delivery failureRead More

Remotely Closed" Error for HTTP Post Request if Streaming is Not Enabled

Publish Date: Mar 2, 2024
Resolution

SYMPTOM

Intermittently, HTTP Request processor detects "Remotely Closed" error under the following conditions:
- When the HTTP Request processor sends HTTP POST request with a large payload to target server, sometimes it gets "Remotely Closed" error
- The target server(in this example, https://test.com:443/test) resets the idle client connection after a while(for example, 10 sec)

Example) 
ERROR 2020-12-21 09:10:57,551 [[MuleRuntime].io.14: [sample]....@org.mule.service.http.impl.service.client.GrizzlyHttpClient.start:164 @65f51b91] [event: xxx] org.mule.runtime.core.internal.exception.OnErrorPropagateHandler: 
********************************************************************************
Message               : HTTP POST on resource 'https://test.com:443/test' failed: Remotely closed.
Error type            : HTTP:CONNECTIVITY
Element               : test-main/processors/2 @ test:test.xml:29 (HTTP Request to Underlying API)
Element XML           : <http:request method="POST" doc:name="Request" doc:id="xxx" config-ref="HTTP_Request_configuration" path="/test2" requestStreamingMode="NEVER"/>
 

CAUSE

This is because the HTTP Connector Post Request is not using Streaming mode.
Example) 
<http:request method="POST" doc:name="Request" doc:id="xxx" config-ref="HTTP_Request_configuration" path="/test2" requestStreamingMode="NEVER"/>
Since the Streaming is not enabled, the HTTP Connector establishes the connection first, and then sends the HTTP POST request after the entire payload becomes ready.
If the payload to be transferred is large, it takes time for the HTTP Connector to start sending the data and reaches the remote side timeout.

Scenario 1) - "Remotely Closed" with requestStreamingMode="NEVER"
1. HTTP Connector tries to send large data to the target server. First, TLS connection handshake is done successfully
2. Then the HTTP Connector will start sending the data. However, it takes some time to start sending data as the data payload is large and Streaming is disabled
3. The target server was completed TLS connection handshake and waiting for data to be transferred, but no data arrives, hence it resets the TCP connection (send RST packet back to HTTP Connector side)
4. The HTTP Connector detects "Remotely closed" and data transfer failed

Note) In such a scenario, you will see a large timestamp gap between the above 3. and 4. This sample TcpDump shows 10 seconds of gap.
User-added image

Scenario 2) - "Remotely Closed" does not happen with requestStreamingMode="ALWAYS" (or "AUTO")
1. HTTP Connector tries to send large data to the target server. First, TLS connection handshake is done successfully
2. Then the HTTP Connector will start sending the HTTP POST Request with large data payload. Since Streaming is enabled, it immediately starts sending data in small chunks
3. The target server was completed connection handshake and waiting for data to be transferred. Soon it receives the chunked data, so it does not reset the connection
4. The data transfer completes successfully
 

SOLUTION

Enable Streaming by setting requestStreamingMode="AUTO" or requestStreamingMode="ALWAYS"
Example) 
<http:request method="POST" doc:name="Request" doc:id="xxx" config-ref="HTTP_Request_configuration" path="/test2" requestStreamingMode="AUTO"/>
So that the HTTP Connector Post Request continuously sends data to the target Server, that prevents the target Server side timeout.
Knowledge Article Number

001120628

 
Loading
Salesforce Help | Article