How to identify top CPU-consuming threads

Using Mule Diagnostics (Recommended)

Using support-collector

Please refer to CPU/Threads profiling with support-collector
The generated zip file using support-collector includes all the information described in the Manual Process

Manual process

This manual procedure has 5 parts: 

1) Identify that the process that is consuming the CPU is the Mule Runtime.

This can be done with any monitoring tool that your OS provides, you should see a java process with high CPU consumption.

2) Get the PID (process ID) of your Mule runtime

$ jps
31077 Jps
30636 MuleContainerBootstrap

Alternatives ways to find the PID are using the operating system tools. For example the ps command in Linux/Mac/Unix and in Windows the Task Manager's Processes tab.

3) Find the top consumer threads

3.a) Linux/MacOS Based

top -H -p <MULE_PID> -b -n 1  -o %CPU | head -n 120
ps -p <MULE_PID> -Lo pid,lwp,nlwp,ruser,pcpu,stime,etime

Notes:
* -b -n 1: one batch run (no interactive UI).
* head -n 120: first 120 lines of the top command output
* The pcpu from the ps command output is reported as a lifetime average for that thread.
How it’s computed:
(total CPU time used by the thread so far) / (total time since the thread started)

It changes slowly; short bursts of CPU get diluted over the process lifetime. A thread that just had a 100% spike for 1 second will still show a low % if the process has been running for a long time.

So ps’s pcpu = long-term average since thread start.
* The %CPU from the top command output is a recent, short-term rate.
How it’s computed:
Over the last sampling interval (usually 1–3 seconds, or your top delay), the thread’s share of CPU time in that interval is converted to a percentage. So it’s how much CPU this thread used in the last second or so.

It reflects current/recent activity. The same short 100% burst will show up as a high % during that interval.

So top’s %CPU = current/recent CPU usage (per refresh interval).

3.b) Windows Based. Using the tool Process Explorer

The Process Explorer tool visually shows the CPU usage dynamically. It is good for live analysis. If you need historical data on CPU per Thread then you can also use Windows perfmon with % Processor Time & Thread Id data counters. You can download Process Explorer from the link below: 
http://technet.microsoft.com/en-us/sysinternals/bb896653

Once you have identified the mule runtime java process, simply right click on the java.exe process and select Properties. You will see the threads tab. 
You can capture a screenshot of that view for further analysis. 

4) Take a thread dump at the same time to analyze what those threads are doing.

A thread dump is a text snapshot of the stack for every thread present in the JVM. It includes information on the exact execution point of each thread and the locks (if any) on which these threads are waiting. There are several methods to capture a Java thread dump, you can check some of these methods in section 2 of Knowledge Base Article: How to Obtain as Much Information as Possible from an Unresponsive Mule Server

5) Using the Thread Ids obtained from point 3. We can explore the threads in the thread dump to investigate what were those threads specifically doing and in which states were at that moment.