Agent Error Messages When the Network Is Disconnected

1) Turn OFF the Network (or stop Proxy Server)

You will start seeing the following error messages.

[mule_ee.log]

WARN  2021-05-19 12:29:06,695 [pool-8-thread-1] [event: ] com.mulesoft.agent.util.client.DefaultAuthenticationProxyClient: There was an error executing a post request to the auth proxy.
WARN  2021-05-19 12:29:06,696 [pool-8-thread-1] [event: ] com.mulesoft.agent.monitoring.publisher.ingest.IngestTargetMonitorPublisher: Could not publish target metrics to Ingest, cause: UnknownHostException: data-authenticator.anypoint.mulesoft.com: nodename nor servname provided, or not known
INFO  2021-05-19 12:29:06,696 [pool-8-thread-1] [event: ] com.mulesoft.agent.buffer.BufferedHandler: Trying to retry flushing on buffer for IngestTargetMonitorPublisher. Remaining attempts: 0
ERROR 2021-05-19 12:29:06,696 [pool-8-thread-1] [event: ] com.mulesoft.agent.buffer.BufferedHandler: Buffer for IngestTargetMonitorPublisher is configured to discard messages on flush failure. Discarding 1 messages.

Notice:
- WARN/ERROR related to the network disconnection is printed
- WARN "There was an error executing a post request to the auth proxy"
- WARN "Could not publish target metrics to Ingest"
- ERROR "Buffer for IngestTargetMonitorPublisher is configured to discard messages on flush failure"
 

WARN  2021-05-19 12:30:06,684 [monitoring-application-publisher-0] [event: ] com.mulesoft.agent.util.client.DefaultAuthenticationProxyClient: There was an error executing a post request to the auth proxy.
WARN  2021-05-19 12:30:06,685 [monitoring-application-publisher-0] [event: ] com.mulesoft.agent.monitoring.publisher.ingest.IngestApplicationMonitorPublisher: Could not publish application metrics for test-app1, cause: java.util.concurrent.ExecutionException: java.net.UnknownHostException: data-authenticator.anypoint.mulesoft.com: nodename nor servname provided, or not known
WARN  2021-05-19 12:30:06,685 [pool-6-thread-1] [event: ] com.mulesoft.agent.monitoring.publisher.ingest.IngestApplicationMonitorPublisher: Some metrics for applications could not be published.
INFO  2021-05-19 12:30:06,685 [pool-6-thread-1] [event: ] com.mulesoft.agent.buffer.BufferedHandler: Trying to retry flushing on buffer for IngestApplicationMonitorPublisher. Remaining attempts: 0
ERROR 2021-05-19 12:30:06,685 [pool-6-thread-1] [event: ] com.mulesoft.agent.buffer.BufferedHandler: Buffer for IngestApplicationMonitorPublisher is configured to discard messages on flush failure. Discarding 1 messages.
WARN  2021-05-19 12:30:06,707 [pool-8-thread-1] [event: ] com.mulesoft.agent.util.client.DefaultAuthenticationProxyClient: There was an error executing a post request to the auth proxy.
WARN  2021-05-19 12:30:06,707 [pool-8-thread-1] [event: ] com.mulesoft.agent.monitoring.publisher.ingest.IngestTargetMonitorPublisher: Could not publish target metrics to Ingest, cause: UnknownHostException: data-authenticator.anypoint.mulesoft.com
INFO  2021-05-19 12:30:06,707 [pool-8-thread-1] [event: ] com.mulesoft.agent.buffer.BufferedHandler: Trying to retry flushing on buffer for IngestTargetMonitorPublisher. Remaining attempts: 0
ERROR 2021-05-19 12:30:06,707 [pool-8-thread-1] [event: ] com.mulesoft.agent.buffer.BufferedHandler: Buffer for IngestTargetMonitorPublisher is configured to discard messages on flush failure. Discarding 1 messages.

Notice:
- WARN/ERROR related to the network disconnection is printed
- WARN "Could not publish application metrics for test-app1"
- WARN "Some metrics for applications could not be published" due to the disconnection

2) Then turn ON the Network (or start again the Proxy Server)

You will start seeing the following messages.
So if you see these messages after the above ERROR/WARN, it indicates the Network/Proxy is recovered.

[mule_ee.log]

INFO  2021-05-19 12:31:41,030 [WebSocketInboundExecutor] [event: ] com.mulesoft.agent.transport.AbstractWebSocketTransport: Closing Mule Agent WebSocket - Close Code: 1006 - Close Reason: Connection was closed abnormally (that is, with no close frame being received).
INFO  2021-05-19 12:31:41,030 [WebSocketInboundExecutor] [event: ] com.mulesoft.agent.transport.AbstractWebSocketTransport: Mule Agent WebSocket closed
INFO  2021-05-19 12:31:43,858 [pool-13-thread-1] [event: ] com.mulesoft.agent.transport.WSConnection: Initializing Mule Agent WebSocket connection
INFO  2021-05-19 12:31:43,859 [pool-13-thread-1] [event: ] com.mulesoft.agent.transport.WSConnection: Mule Agent WebSocket connecting to host: runtime-manager.anypoint.mulesoft.com
INFO  2021-05-19 12:31:44,611 [WebSocketInboundExecutor] [event: ] com.mulesoft.agent.transport.AbstractWebSocketTransport: Opening Mule Agent WebSocket
INFO  2021-05-19 12:31:44,613 [WebSocketInboundExecutor] [event: ] com.mulesoft.agent.transport.AbstractWebSocketTransport: Mule Agent WebSocket opened
INFO  2021-05-19 12:31:44,614 [pool-13-thread-1] [event: ] com.mulesoft.agent.transport.WSConnection: Mule Agent WebSocket connection was initialized after: 1 attempts
INFO  2021-05-19 12:31:44,802 [WebSocketInboundExecutor] [event: ] com.mulesoft.agent.services.security.HandshakeAuthorizationService: WebSocket Client connection authorized

Notice:
- The failed communication WebSocket is closed, then WebSocket Client connection authorized
- INFO "Closing Mule Agent WebSocket - Close Code: 1006 - Close Reason: Connection was closed abnormally"
- INFO "WebSocket Client connection authorized"

Note)
The mule_ee.log is collected in a test scenario in a test server for information sharing purpose. So it is not guaranteed if you see_ the exact same log messages.

Solution

In scenarios like these, kindly contact your network administrator to understand the reason behind the disconnection of the network which causes the mule agent to disconnect from the Runtime Manager. Also please check if the Ports, IP Addresses, and Hostnames to Allow are not blocked.
https://docs.mulesoft.com/runtime-manager/rtm-agent-allowlists