Loading

Transferring multiple large files, in a cloudhub-deployed Mule 4 app, can result in this error: java.io.IOException: No space left on device

Publiceringsdatum: Mar 2, 2024
Lösning

SYMPTOM

You have your application deployed in CloudHub with an SFTP write and after a few files are transferred the application start raising an exception like the following:
  
06:46:28.957 11/20/2019 Worker-0 [MuleRuntime].cpuIntensive.03: [grd-rapidresponse-transfer-v1-pr].get:\RR-Upload:RapidResponse-File-Transfer-config.CPU_INTENSIVE @3c7e5ad4 ERROR
event:f93405f0-0b8a-11ea-8a26-1239931c7b53
********************************************************************************
Message : 'until-successful' retries exhausted. Last exception message was: Exception was found writing to file '/guardian/Customer.tab'
Element : org.mule.runtime.core.internal.routing.UntilSuccessful@1e07322a @ app:implementation/grd-transfer.xml:78 (Until Successful)
Element XML : <until-successful maxRetries="10" doc:name="Until Successful" doc:id="eb1870f8-2679-44a2-850e-8d13fd49df0b" millisBetweenRetries="10000">
<sftp:write doc:name="Write" doc:id="5ebd5153-12d3-423e-bb3c-8136a1e4f7a0" config-ref="SFTP_Config" path="#[vars.fname]">
<reconnect frequency="10000" count="10"></reconnect>
</sftp:write>
</until-successful>
Error type : MULE:RETRY_EXHAUSTED
Payload Type : org.mule.runtime.core.internal.streaming.bytes.ManagedCursorStreamProvider
--------------------------------------------------------------------------------
Root Exception stack trace:
java.io.IOException: No space left on device
at java.io.RandomAccessFile.writeBytes(Native Method)
at java.io.RandomAccessFile.write(RandomAccessFile.java:525)
 

CAUSE

You need to verify the following:
  •  Go to the /tmp directory in the worker and you should see files with this format 
rw-rw-r-- 1 ion-mule ion-mule    3145728 Dec  4 12:47 mule-buffer-stream-buffer-8557289976324036557.tmp
-rw-rw-r-- 1 ion-mule ion-mule    3145728 Dec  4 12:47 mule-buffer-stream-buffer--1191717355592342214.tmp
-rw-rw-r-- 1 ion-mule ion-mule    9437184 Dec  4 12:47 mule-buffer-stream-buffer-6560632003404461856.tmp
-rw-rw-r-- 1 ion-mule ion-mule   28311552 Dec  4 12:47 mule-buffer-stream-buffer--901029286358098847.tmp

 

These files are created and are consuming all the space available because each time you run the application the SFTP is going to create and save a different file that at first is going to save them in memory ( 'in memory size' ) for it reuse instead of recreating one only file. Then when there is not enough memory available it will start saving those files in the disc.


SOLUTION

1)  If these files are created and are consuming all the space available, you will have to analyze the type of the worker the client has and understand if they still need the Streaming strategy repeatable, they should analyze the possibility of upgrading the size to have more space available.

2) Increase the <In memory size> configuration and resize the worker size to have more memory available. 

3) Change the Streaming strategy to <Non repeatable stream> [Recommended if there is not a need of reusing the same files]

 
Knowledge-artikelnummer

001123070

 
Laddar
Salesforce Help | Article