Skip to main content
Splunk Lantern

Predicting service outages before they occur

 

You can use historical KPI data and machine learning algorithms in Splunk ITSI to predict an outage 20-30 minutes before it happens. This process works best when a service has more than 5 good KPIs and more than 1 week of historical data.

The machine learning algorithm looks for recognizable/predictable KPI behavior, which comes before the service's aggregate health score changes. You can use machine learning within Splunk ITSI to build a model for the service you want to track.

Build a model

  1. Open Splunk ITSI and in the top toolbar click Configuration, then Services.
  2. In the list of services, find the service you want to track. Click the Edit drop-down box to the right of the service name, then click Predictive Analytics.
  3. On this screen you will train and test different machine learning algorithms to determine which one gives the most accurate prediction. Use the instructions on-screen to select a time, algorithm type and algorithm, and click the Train button.

clipboard_e317dc2a2ab66810750548c0720dd9a67.png

  1. After the model has run, investigate the results,  and click Save.
  2. Test out other algorithms by repeating steps 3 and 4.

Review the Predictive Analytics score and add it to a glass table

  1. Open Splunk ITSI and in the top toolbar click Dashboards, then Predictive Analytics.
  2. In the list of services, find the service you want to track, and select the recommended algorithm model.

clipboard_e48a71333654a9ac6188bf66d4e2bb568.png

  1. After you have selected a model, Splunk ITSI will calculate the future Service Health Score. Click the Cause Analysis button to review the suggested KPIs.
  2. Click the spyglass to review the SPL. Save it to a notepad to copy it into a glass table later.

clipboard_e47f77c03b7d49271389687538b43a83f.png

  1. In the Splunk ITSI top toolbar, click Glass Tables and select the glass table you'd like to add this score to.
  2. Click the Edit button and review the Database Future Health Score attributes.
  3. Select the Future Health for the service you are interested in, then under the Configuration menu, click Set up Primary Data Source.
  4. Click Create Ad-Hoc Search. Copy the previous Predictive Analytics SPL and click the Run & Save button.