Algorithm negotiation fail" Error When SFTP Connector is Trying to Connect to SFTP Server

SYMPTOM

A Mule application containing an SFTP connector fails with an exception showing a stack trace similar to the one below:

org.mule.runtime.api.connection.ConnectionException: Could not establish SFTP connection with host: 'yourHost' at port: 'yourPort' - Algorithm negotiation fail
Caused by: com.jcraft.jsch.JSchException: Algorithm negotiation fail
at com.jcraft.jsch.Session.receive_kexinit(Session.java:590)
at com.jcraft.jsch.Session.connect(Session.java:320)
at com.jcraft.jsch.Session.connect(Session.java:183)
at org.mule.extension.sftp.internal.connection.SftpClient.connect(SftpClient.java:179)
at org.mule.extension.sftp.internal.connection.SftpClient.login(SftpClient.java:161)
at org.mule.extension.sftp.internal.connection.SftpConnectionProvider.connect(SftpConnectionProvider.java:139)
at org.mule.extension.sftp.internal.connection.SftpConnectionProvider.connect(SftpConnectionProvider.java:59)

CAUSE

When a secure connection is attempted between a Client and Server through SFTP, both ends expose a list of Key exchange algorithms, Cryptographic Algorithms, Message Authentication Code (MAC) and and compression algorithms. Each of these algorithms is ordered from most secured to least secured. The algorithms are to be used to negotiate the best option to proceed with the SSH connection. If during this negotiation there is no agreement on the algorithm implementations to be used an "Algorithm negotiation fail" message will be received. 
An in-depth detail regarding this SSH Algorithm negotiation can be seen in the SSH RFC4253 section 7.1

The negotiation information can be seen in a Mule Application by enabling the SFTP Debug logs.

In the case of Mule 4.x enabling the DEBUG logs by setting the packages will suffice "How to enable verbose logging for FTP and SFTP and FTPS module", in Mule 3.x some additional modifications are required to retrieve the DEBUG logs from JSCH you'll need to contact MuleSoft Support for this.

An example of an SFTP DEBUG log showing this negotiation can be found below:

DEBUG 2021-03-17 19:57:23,944 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] org.mule.extension.sftp.internal.connection.SftpConnectionProvider: Connecting to host: 'myHost' at port: 'myPort'
DEBUG 2021-03-17 19:57:23,945 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: Connecting to myHost port myPort
DEBUG 2021-03-17 19:57:23,971 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: Connection established
DEBUG 2021-03-17 19:57:24,002 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: Remote version string: SSH-2.0-CerberusFTPServer_11.0
DEBUG 2021-03-17 19:57:24,002 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: Local version string: SSH-2.0-JSCH-0.1.54
DEBUG 2021-03-17 19:57:24,002 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: CheckCiphers: aes256-ctr,aes192-ctr,aes128-ctr,aes256-cbc,aes192-cbc,aes128-cbc,3des-ctr,arcfour,arcfour128,arcfour256
DEBUG 2021-03-17 19:57:24,006 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: aes256-ctr is not available.
DEBUG 2021-03-17 19:57:24,006 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: aes192-ctr is not available.
DEBUG 2021-03-17 19:57:24,006 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: aes256-cbc is not available.
DEBUG 2021-03-17 19:57:24,006 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: aes192-cbc is not available.
DEBUG 2021-03-17 19:57:24,006 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: CheckKexes: diffie-hellman-group14-sha1,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521
DEBUG 2021-03-17 19:57:24,075 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: CheckSignatures: ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521
DEBUG 2021-03-17 19:57:24,075 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: SSH_MSG_KEXINIT sent
DEBUG 2021-03-17 19:57:24,075 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: SSH_MSG_KEXINIT received
DEBUG 2021-03-17 19:57:24,075 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: kex: server: ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
DEBUG 2021-03-17 19:57:24,076 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: kex: server: ssh-rsa,ssh-ed25519,rsa-sha2-512,rsa-sha2-256
DEBUG 2021-03-17 19:57:24,076 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: kex: server: aes256-ctr,aes256-cbc
DEBUG 2021-03-17 19:57:24,076 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: kex: server: aes256-ctr,aes256-cbc
DEBUG 2021-03-17 19:57:24,076 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: kex: server: hmac-sha1,hmac-sha2-256,hmac-sha2-512,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-md5
DEBUG 2021-03-17 19:57:24,076 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: kex: server: hmac-sha1,hmac-sha2-256,hmac-sha2-512,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-md5
DEBUG 2021-03-17 19:57:24,076 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: kex: server: none
DEBUG 2021-03-17 19:57:24,076 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: kex: server: none
DEBUG 2021-03-17 19:57:24,076 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: kex: server: en-US
DEBUG 2021-03-17 19:57:24,076 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: kex: server: en-US
DEBUG 2021-03-17 19:57:24,076 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: kex: client: ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group14-sha1,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1
DEBUG 2021-03-17 19:57:24,076 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: kex: client: ssh-rsa,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521
DEBUG 2021-03-17 19:57:24,076 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: kex: client: aes128-ctr,aes128-cbc,3des-ctr,3des-cbc,blowfish-cbc
DEBUG 2021-03-17 19:57:24,076 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: kex: client: aes128-ctr,aes128-cbc,3des-ctr,3des-cbc,blowfish-cbc
DEBUG 2021-03-17 19:57:24,076 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: kex: client: hmac-md5,hmac-sha1,hmac-sha2-256,hmac-sha1-96,hmac-md5-96
DEBUG 2021-03-17 19:57:24,076 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: kex: client: hmac-md5,hmac-sha1,hmac-sha2-256,hmac-sha1-96,hmac-md5-96
DEBUG 2021-03-17 19:57:24,076 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: kex: client: none
DEBUG 2021-03-17 19:57:24,076 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: kex: client: none
DEBUG 2021-03-17 19:57:24,076 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: kex: client: 
DEBUG 2021-03-17 19:57:24,076 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: kex: client: 
DEBUG 2021-03-17 19:57:24,077 [[MuleRuntime].uber.08: [my-mule-application].my-mule-applicationFlow.CPU_LITE @2a316d37] [processor: my-mule-applicationFlow/processors/0; event: ] com.jcraft.jsch: Disconnecting from myHost port myPort

In this article we'll cover two scenarios:

Scenario 1: Mismatch in the Compression Algorithm.

A compression algorithm mismatch will look similar to the snippet below.

com.jcraft.jsch: kex: server: zlib

com.jcraft.jsch: kex: client: none

A Cipher Mismatch will look similar to the snippet below.

com.jcraft.jsch: kex: server: aes256-ctr,aes256-cbc

com.jcraft.jsch: kex: client: aes128-ctr,aes128-cbc,3des-ctr,3des-cbc,blowfish-cbc

SOLUTION 

The SSH versions supported by both the Server and the client (SFTP connector) is listed at the start of DEBUG logs.

Remote version string: SSH-2.0-OpenSSH_8.8
Local version string: SSH-2.0-JSCH-0.1.54

As a rule of thumb, each of the algorithm categories negotiated (Mentioned in the RFC) should have at least an implementation in common between Client and Server (and fulfill their constraints) to proceed.

RFC 4253: Algorithm Negotiation
      name-list    kex_algorithms
      name-list    server_host_key_algorithms
      name-list    encryption_algorithms_client_to_server
      name-list    encryption_algorithms_server_to_client
      name-list    mac_algorithms_client_to_server
      name-list    mac_algorithms_server_to_client
      name-list    compression_algorithms_client_to_server
      name-list    compression_algorithms_server_to_client
      name-list    languages_client_to_server
      name-list    languages_server_to_client

Scenario 1 (Compression Algorithm)

The SFTP Connector currently does not support "zlib" compression and as a consequence, the only algorithm that can agree with the server-side is "NONE". An enhancement request has been raised through our Ideas Portal  

The alternatives available at the moment are:

1. Configure the SFTP server to allow connections without compression (Offering the "NONE" algorithm as enabled). 
2. Develop a Custom connector through the Java SKD / DeviKit(Mule 3.x) or a Java program using the Java module to connect to the SFTP server in question using a library that allows this feature.

Scenario 2 (Cipher mismatch)

a. Verify that the JSCH library used supports the ciphers you intend to use

The SFTP connector uses the JSCH library to establish a connection to an SFTP server, JSCH has a set of supported Ciphers described in their documentation depending on the version used, you can see the features included in the JSCH ChangeLog. 

• In the case of Mule Runtime 4.x, the library is bundled with each SFTP connector release and you can access its version easily through the package explorer in Anypoint Studio.

Eg.

• In Mule Runtime 3.x the library is bundled with the runtime itself and you can check it by searching in the directory <MULE_HOME>/lib/opt for a .jar file starting with the prefix "jsch-".

If the library does not provide support for the ciphers in question, you'll need to enable one or more ciphers included between the supported ones by that JSCH version in your SFTP server.

IMPORTANT: Modifying the library in use is not recommended as the SFTP connector behavior has been tested and validated with the version bundled, in case of attempting this please do so in a lower environment and if possible confirm with MuleSoft Support if the JSCH version is compatible.

b. Verify if the ciphers are enabled to be used at JVM level. 

The SFTP connector uses under the hood the JSCH library to connect to an SFTP server, this library ultimately relies on JCE Java(TM) Cryptography Extension. In some JVMs, certain ciphers might not be included to be used due to control restrictions. You should verify if these limitations apply to the JVM you are using according to the Vendor documentation.

For example, in the case of Oracle JVMs, you need to add the JCE unlimited strength for it to allow certain ciphers, each Java version has its own JCE matching package and installation details.

In the case of JDK8 you can refer to:

Unlimited strength policy and installation details and unlimited strength JCE policy files.

Disclaimer: This solution provides a suggestion that should be considered in conjunction with your specific use-case and requirements and does not represent a complete solution for all circumstances.