Skip to main content
Splunk Lantern

ESXi hosts with high CPU Ready summation value

"CPU Ready" indicates that a virtual machine needs access to CPU resources to continue processing, but the underlying host has no remaining CPU resources to allocate. This metric can be calculated as summation or percentage. You might need to see which ESXi hosts on your network have a high CPU Ready summation value when doing the following:

Prerequisites 

In order to execute this procedure in your environment, the following data, services, or apps are required:

Example

When many virtual machines on an ESXi host have high sum ready metrics, the host might be experiencing CPU pressure. You want to monitor your network for this type of problem so you can take mitigating action.

NOTE: To optimize the search shown below, you should specify an index and a time range.

  1. Run the following search: 
sourcetype="vmware:perf:cpu" source="VMPerf:VirtualMachine"  instance=aggregated 
|stats avg(p_summation_cpu_ready_millisecond) AS avg_p_summation_cpu_ready_millisecond BY moid 
|eval is_high_sumready = if(avg_p_summation_cpu_ready_millisecond > 500, 1, 0) 
|fields moid avg_p_summation_cpu_ready_millisecond is_high_sumready 
|append 
[search index=<your index name> sourcetype="vmware:inv:hierarchy" type=VirtualMachine 
|dedup moid |eval esxi_moid = 'changeSet.runtime.host.moid' 
|fields moid esxi_moid]
|eventstats values(esxi_moid) AS esxi_moid BY moid
|search is_high_sumready=*
|stats count AS count_vms_by_host sum(is_high_sumready) AS count_high_sumready_by_host avg(avg_p_summation_cpu_ready_millisecond) AS avg_sumready_by_host BY esxi_moid
|eval perc_vms_high_sumready_by_host = round(count_high_sumready_by_host/count_vms_by_host*100, 2)
|sort - perc_vms_high_sumready_by_host

Search explanation

The table provides an explanation of what each part of this search achieves. You can adjust this query based on the specifics of your environment.

Splunk Search Explanation

sourcetype="vmware:perf:cpu" 

source="VMPerf:VirtualMachine"

Search the VMware CPU performance source type and limit the search to VMware virtual machine performance data.

instance=aggregated 

Filter by indicated value of the instance field. 

|stats avg(p_summation_cpu_ready_millisecond) AS avg_p_summation_cpu_ready_millisecond BY moid 

Calculate the average CPU Ready summation value for each host managed object ID (MOID).

|eval is_high_sumready = if(avg_p_summation_cpu_ready_millisecond > 500, 1, 0) 

Create the is_high_sumready field for results the average is above 500ms.

|fields moid avg_p_summation_cpu_ready_millisecond is_high_sumready 

Limit the results to the fields shown. 

|append [search index=<your index name> sourcetype="vmware:inv:hierarchy" type=VirtualMachine 

|dedup moid 

|eval esxi_moid = 'changeSet.runtime.host.moid' 

|fields moid esxi_moid]

Obtain the MOIDs of the ESXi hosts on which each virtual machine is running. Append the returned fields to the primary search.


 

|eventstats values(esxi_moid) AS esxi_moid BY moid

Add the esxi_moid field to all virtual machine Sum Ready results. 

|search is_high_sumready=*

Return all results where the average is above 500ms

|stats count AS count_vms_by_host sum(is_high_sumready) AS count_high_sumready_by_host avg(avg_p_summation_cpu_ready_millisecond) AS avg_sumready_by_host BY esxi_moid

Sum the total number of virtual machines with high CPU Ready summation value. Obtain the average Sum Ready for all virtual machines per each ESXi host. 

|eval perc_vms_high_sumready_by_host = round(count_high_sumready_by_host/count_vms_by_host*100, 2)

Determine the percentage of virtual machines on the host with high Sum Ready.

|sort - perc_vms_high_sumready_by_host

Sort results with the largest percent first.

Result

The table below shows sample results for the search. High values of “perc_vms_high_sumready_by_host” mean that the VM has work to do that is waiting to be scheduled. Generally, some factors to investigate are oversubscription such that too many vCPUs have been allocated from the physical (pCPU)s. Another is when there are a lot of smaller VMS alongside a larger one. Often the larger vm will be waiting for the scheduler to preempt enough vPCU’s from the small VMs to be able to run. A third factor can be limits. 

If you find a host that is experiencing the high values mentioned above, attempt to balance the workload across the cluster or look for over subscription.

esxi_moid count_vms_by_host count_high_
sumready_by_host
avg_sumready_by_host perc_vms_high_
sumready_by_host

host-20

18

14

510.4236111

77.78

host-10

17

11

466.8455882

64.71

host-11

15

0

357.2583333

0

host-26

3

0

6

0

  • Was this article helpful?