Loading

Mule 4.11 HTTP Listener worker thread parked indefinitely on multipart/form-data upload after upgrade from Mule 4.9

Publish Date: May 27, 2026
Description

After upgrading a Mule application from runtime 4.9 to 4.10 or 4.11, flows that handle multipart/form-data uploads may have HTTP worker threads parked for hours or days when an HTTP client disconnects mid-upload. The thread dump shows the worker stuck inside MimeMultipart.parse / readFully, called from a DataWeave expression that reads payload.parts before the main flow processing.

The behaviour change is caused by Mule 4.10/4.11 switching the default HTTP service implementation from Grizzly (which buffered the request body up-front) to Netty (which streams it lazily on demand). Combined with the HTTP Listener not enforcing a read timeout while the body is still being received, a single dropped client connection can permanently leak a worker thread.

This article describes the symptoms, the diagnostic stack signature, the immediate workaround, and the permanent application-level fix.

Symptoms

- One or more HTTP Listener flows that consume multipart/form-data stop responding on a specific endpoint after the runtime is upgraded from 4.9.x to 4.10.x or 4.11.x.
- Over time the [MuleRuntime].uber.* worker pool grows; eventually the application becomes unresponsive on the affected endpoint.
- The same flow code worked correctly on Mule 4.9.
- A thread dump shows one or more workers stuck on the following stack (the thread name will reference the affected endpoint, for example "post:\<path>:multipart\form-data:<config>"):

      java.lang.Thread.State: RUNNABLE
          at sun.nio.ch.EPoll.wait(...)
          ... (Netty I/O frames) ...
          at o.m.r.a.h.utils.BlockingBidirectionalStream.read(...)
          at o.m.r.a.h.s.fileStore.FileStoreInputStreamBuffer.consumeForwardData(...)
          at jakarta.mail.internet.MimeMultipart.readFully(...)
          at jakarta.mail.internet.MimeMultipart.parse(...)
          at jakarta.mail.internet.MimeMultipart.getCount(...)
          at o.m.weave.v2.module.multipart.MultiPartReader.doRead(...)
          at o.m.weave.v2.runtime.WeaveExpressionLanguageSession.evaluate(...)
          at o.m.r.c.e.el.dataweave.DataWeaveExpressionLanguageAdaptor$1.evaluate(...)
          at o.m.r.c.i.p.simple.AddFlowVariableProcessor.process(...)

- The runtime log may also contain repeated cosmetic ERROR entries with the prefix "ForwardingToListenerHandlerReplica" and message "Connection reset". This is a separate, harmless log-noise issue that shares the same workaround.


Environment

- Mule Runtime 4.10.x or 4.11.x (verified on 4.11.4 / JDK 17.0.15)
- Default HTTP service implementation Netty (org.mule.service.http.netty)
- Application uses an HTTP Listener configured with multipart/form-data
- The flow reads payload.parts (or payload.parts.<name>.content) in a processor placed before the main transformation, for example inside a <set-variable />, <logger />, or preliminary <ee:transform />.


Root Cause
In Mule 4.9 the HTTP service was Grizzly. Grizzly buffered the entire multipart body up-front, so any DataWeave expression that referenced payload.parts found the body already materialised and returned immediately.

In Mule 4.10/4.11 the HTTP service is Netty (default). Netty streamsthe request body lazily — bytes are pulled from the socket only when the flow actually reads the payload. When the flow accesses payload.parts early (for example via "<set-variable value="#[payload.parts.binary.content]"/>"), the DataWeave evaluator calls MimeMultipart.parse() -> readFully(), which forces the entire body to be drained from the socket synchronously on the application worker thread.
 
If the HTTP client disconnects mid-upload (network blip, client abort, idle keep-alive timeout, slow client), the worker remains parked in BlockingBidirectionalStream.read waiting for bytes that never arrive. The Mule 4.11 HTTP Listener does not enforce a read timeout while the request body is being received, so the worker is never reclaimed. Repeated occurrences leak workers permanently from the runtime pool.

Reading payload.parts more than once in the same flow (for example once in <set-variable /> and again inside <ee:transform />) amplifies the impact because each access can re-trigger materialisation.

Resolution

Permanent fix (recommended)

Refactor the affected flow so the multipart payload is read exactly once, and avoid unnecessary in-memory expansion:

1. Consolidate every <set-variable /> or pre-transform read of payload.parts into a single <ee:transform> placed once at the top of the flow.

2. If only metadata is required (filename, content-type, etc.), read it from attributes.headers instead of payload.parts — header lookups do not trigger multipart materialisation.

3. If the flow uploads a file to Salesforce as a ContentVersion, pass the binary stream directly to VersionData. Do NOT  base64-encode it. Salesforce Connector v11.x accepts the binary stream as-is. Replace:

VersionData: toBase64(vars.varFileContent)

with:

VersionData: vars.varFileContent

Removing toBase64 also avoids the ~33% in-memory expansion.

4. Add an explicit connectionIdleTimeout on <http:listener-connection> (60 to 300 seconds is a reasonable range) so that any half-open or abandoned client connection is closed by the runtime regardless of which HTTP service implementation is in use.

After deploying the refactor, capture a thread dump under representative load and confirm that no workers are parked on MimeMultipart.parse / BlockingBidirectionalStream.read.

Immediate workaround

If a code change cannot be deployed immediately, force the runtime to use the legacy Grizzly HTTP service implementation by setting the following application-level property and restarting the worker(s):

      mule.http.service.implementation=GRIZZLY

This restores the Mule 4.9 buffering semantics without modifying the application code. It is a fully supported configuration but should be treated as a temporary mitigation while the permanent refactor above is planned.

Verify the property is in effect by capturing a thread dump and confirming all of the following:

- 30+ stack frames reference grizzly.framework@2.3.36-MULE-030
- Zero stack frames reference org.mule.service.http.netty
- Threads named "Grizzly-IdleTimeoutFilter-IdleCheck" and "connection-pool-delays-thread-pool" are present

Additional Resources

Related Known Issues

Known Issue — Mule 4.11 HTTP Listener does not enforce read timeout while the request body is being received. This is the runtime-level contributing factor that allows the worker thread to remain parked indefinitely.
https://help.salesforce.com/s/issue?id=a02Ka00000mRktpIAC

Knowledge Article Number

005385539

 
Loading
Salesforce Help | Article