Skip to main content

 

Splunk Lantern

Virtual machines with high CPU Ready summation value

 

"CPU Ready" indicates that a virtual machine needs access to CPU resources to continue processing, but the underlying host has no remaining CPU resources to allocate. This metric can be calculated as summation or percentage.

"CPU Ready" only occurs when the host is overprovisioned on CPU. A single virtual machine with high "CPU Ready" indicates a configuration or resource problem, and the virtual machine's performance is impacted. 

Data required 

Procedure

  1. Ensure that you have installed the IT Essentials Work app to onboard VMware data and provide the various VMware entity type configurations and dashboards.
  2. Ensure that you are collecting VMware data through one or more Data Collection Nodes, which are essentially Splunk heavy forwarders with specific VMware collection configurations. 
  3. Run the following search. You can optimize it by specifying an index and adjusting the time range.
sourcetype="vmware:perf:cpu" source="VMPerf:VirtualMachine" instance=aggregated 
|stats avg(p_summation_cpu_ready_millisecond) AS avg_p_summation_cpu_ready_millisecond BY moid 
|eval is_high_sumready = if(avg_p_summation_cpu_ready_millisecond > 500, "Yes", "No") 
|fields moid avg_p_summation_cpu_ready_millisecond is_high_sumready 
|append 
    [ search index=<your index name> sourcetype="vmware:inv:hierarchy" type=VirtualMachine 
    |dedup moid 
    |eval esxi_moid = 'changeSet.runtime.host.moid' 
    |table moid esxi_moid] 
|eventstats values(esxi_moid) AS esxi_moid BY moid 
|search is_high_sumready=”Yes” 
|sort - avg_p_summation_cpu_ready_millisecond 
|table moid esxi_moid avg_p_summation_cpu_ready_millisecond is_high_sumready

Search explanation

The table provides an explanation of what each part of this search achieves. You can adjust this query based on the specifics of your environment.

Splunk Search Explanation

sourcetype="vmware:perf:cpu" 

source="VMPerf:VirtualMachine"

Search the VMware CPU performance source type and limit the search to VMware virtual machine performance data.

instance=aggregated

Return the most recent results for the performance metrics of all virtual machines. 

|stats avg(p_summation_cpu_ready_millisecond) AS avg_p_summation_cpu_ready_millisecond BY moid

Calculate the average CPU Ready summation value for each host managed object ID (MOID).

|eval is_high_sumready = if(avg_p_summation_cpu_ready_millisecond > 500, "Yes", "No")

Create the is_high_sumready field for results the average is above 500ms.

|fields moid avg_p_summation_cpu_ready_millisecond is_high_sumready

Limit the results to the fields shown.

|append 

[ search index=<your index name>     sourcetype="vmware:inv:hierarchy" type=VirtualMachine 

|dedup moid 

|eval esxi_moid = 'changeSet.runtime.host.moid' 

 fields moid esxi_moid] 

Run a subsearch that returns a unique moid along with the corresponding changeSet value. Append these results to the primary (outer) search results to correlate the hierarchy data with the performance data.

|eventstats values(esxi_moid) AS esxi_moid BY moid

Add the esxi_moid field to all virtual machine Sum Ready results. 

|search is_high_sumready=”Yes”

Return all results where the average is above 500ms.

|sort - avg_p_summation_cpu_ready_millisecond

Sort results with the largest average first.

|table moid esxi_moid avg_p_summation_cpu_ready_millisecond is_high_sumready

Display the results in a table with columns in the order shown.

Next steps

The table below shows sample results for the search. Many factors can contribute to high ready times, and referencing other relevant searches, such as ESXi hosts with high CPU Ready summation value, side-by-side on a dashboard can help you better understand the causes.  

moid esxi_moid avg_p_summation_cpu_ready_millisecond is_high_sumready

vm-199

host-20

680.6718404

Yes

vm-239

host-20

658.2372506

Yes

vm-242

host-20

633.1552106

Yes

vm-232

host-20

627.5144124

Yes

vm-213

host-20

621.9002217

Yes

Finally, you might be interested in other processes associated with the Monitoring VMware virtual machine performance use case.