With the MuleSoft Apache Kafka Connector, the error "Offset commit cannot be completed since the consumer is not part of an active group for auto partition assignment; it is likely that the consumer was kicked out of the group" is received. The below error is encountered with the Kafka Listener -
********************************************************************************
Message : Offset commit cannot be completed since the consumer is not part of an active group for auto partition assignment; it is likely that the consumer was kicked out of the group.
Element : map-and-store-flow/source @ xxxxx-stg:xxxxx.esb.xml:14 (Message listener)
Element DSL : <kafka:message-listener doc:name="Message listener" doc:id="8996ed72-xxxxxx-d516c68aae4f" config-ref="Apache_Kafka_Consumer_configuration" parallelConsumersAmount="${secure::kafka.consumers}"></kafka:message-listener>
Error type : MULE:SOURCE_RESPONSE_SEND
FlowStack :
Payload Type : java.util.List
--------------------------------------------------------------------------------
The above message was from a sample log when the microservice took a long time before committing the offset. Let's explain the context first to help you get some background information about the issue. Service A is dedicated to calling the REST API exposed by service B. Whenever A receives a message from Kafka, it calls service B's API. If B is down, A will keep trying for several more attempts. When the retry takes too long, the Kafka consumer of service A will be removed from the consumer group.
There is a component, the so-called Group Coordinator, which manages consumers/members of consumer groups. If the last poll is more than x ms and x > max.poll.interval.ms, then the Group Coordinator will believe that the consumer was dead and it will remove the consumer from the group. That's why when the retry takes longer than the max.poll.interval.ms, the consumer cannot commit the offset. Even worse, another consumer might poll the same message and falling to the same issue.
To resolve the MuleSoft Apache Kafka Connector Error "Offset commit cannot be completed", make sure that the retry will not take longer than "max.poll.interval.ms". Let us suppose there are three retry attempts and the interval is 1000 ms then the value of the "max.poll.interval.ms"
should be greater than 3*1000 ms that is more than 3000 ms.
.
001119219

We use three kinds of cookies on our websites: required, functional, and advertising. You can choose whether functional and advertising cookies apply. Click on the different cookie categories to find out more about each category and to change the default settings.
Privacy Statement
Required cookies are necessary for basic website functionality. Some examples include: session cookies needed to transmit the website, authentication cookies, and security cookies.
Functional cookies enhance functions, performance, and services on the website. Some examples include: cookies used to analyze site traffic, cookies used for market research, and cookies used to display advertising that is not directed to a particular individual.
Advertising cookies track activity across websites in order to understand a viewer’s interests, and direct them specific marketing. Some examples include: cookies used for remarketing, or interest-based advertising.