Skip to main content

 

Splunk Lantern

Windows disk drive utilization nearing capacity

You might want to disk drive is nearing capacity when doing the following:

Prerequisites 

In order to execute this procedure in your environment, the following data, services, or apps are required:

Example

Your organization's critical IT applications require space on disk to read and write to operate properly. When an application runs out of disk space, it usually results in application instability or crashing. You need a search to help detect when a disk drive is nearing capacity.

Option 1

To optimize the search shown below, you should specify an index and a time range.

  1. In Splunk Enterprise or Splunk Cloud Platform, verify that you deployed the Splunk Add-on for Microsoft Windows add-on to your search heads, indexer, and Splunk Universal Forwarders on the monitored systems. For more information, see About installing Splunk add-ons.
  2. Run the following search:
| mstats avg(LogicalDisk.%_Free_Space) AS "win_storage_free" WHERE index="<name of your metrics index>" host="<names of the hosts you want to check>" instance="<names of drives you want to check>" instance!="_Total" BY host, instance span=1m
| eval storage_used_percent=round(100-win_storage_free,2)
| eval host_dev=printf("%s:%s\\",host,instance)
| timechart max(storage_used_percent) AS storage_used_percent BY host_dev

Search explanation

The table provides an explanation of what each part of this search achieves. You can adjust this query based on the specifics of your environment.

Splunk Search Explanation
| mstats avg(LogicalDisk.%_Free_Space) AS "win_storage_free" WHERE index="<name of your metrics index>" host="<names of the hosts you want to check>" instance="<names of drives you want to check)>" instance!="_Total" BY host, instance span=1m Search metrics index(es) where perfmon disk space data is being collected and filter down to the desired host(s) to check.
| eval storage_used_percent=round(100-win_storage_free,2) Convert percent storage free to percent storage used for readability.
| eval host_dev=printf("%s:%s\\",host,instance) Create a new field that combines the host and disk drive.
| timechart max(storage_used_percent) AS storage_used_percent BY host_dev Plot the storage used for each host and disk over time.

Result

Create an alert based off this search so you can proactively manage potential stability issues. To alert when disk utilization is nearing max capacity, you can configure one of the following two recommendations:

  • Use the SPL from this procedure to configure a Core Splunk alert.
  • Build a new Vital Metric in IT Essentials Work for the desired entity type and configure vital metric alerting. Click here for an example SPL search that can be used for the vital metric search. After the vital metric has been created, configure it to alert when the disk used percentage is at or near 100.

Option 2

  1. Ensure that you have the Splunk OTEL Collector installed on the host you want to monitor.
  2. In Splunk Infrastructure Monitoring, use the following SignalFlow to search the disk.utilization streaming metric, filter down to the desired hosts and mountpoints, and summarize results by counting the total number of processes found per host.
    A = data('disk.utilization', filter=filter('host', '<name of host to check>') and filter('mountpoint', '<name of disk to check>')).publish(label='A')

Result

To alert when disk utilization is nearing capacity on the specified host(s) and mountpoint(s), use the SignalFlow from this procedure to configure a detector with an alert condition of "Resource Running Out" and alert settings of:

  • Alert when nearing: Capacity
  • Capacity: 100
  • Trigger Sensitivity: Medium