Splunk IT Service Intelligence Owner's Manual
Splunk ITSI (ITSI) provides insights into IT services and infrastructure by combining data analytics, visualization, and machine learning. The insights provided by ITSI are critical to many businesses in proactively identifying faults with key IT systems and services.
Like any complex system, ITSI requires regular maintenance for it to function optimally. Just as a car needs its oil changed regularly, ITSI requires a specific set of tasks to be performed at regular intervals. The responsibility for performing these tasks rests with the owner of the individual implementation of Splunk ITSI. This may be a team of people or a single individual. This manual describes the recommended ongoing maintenance tasks that the owner of a Splunk ITSI implementation should ensure are performed to keep their implementation functional.
Maintenance schedule
Each task has a recommended schedule. The recommended frequency for a task can be anywhere from weekly to annually.
These procedures are valid as of ITSI version 4.15.
Tasks
- Event _time indexing checks. This activity involves assessing the time resolution of events as they are indexed into the Splunk platform that ITSI runs on. Source type indexing events at future or past dates beyond acceptable tolerances will be identified for remediation.
- KPI search execution check. This procedure assesses the success rate of KPI search execution to identify if any KPI searches are failing. If failures are identified, further information is collected to identify the specific problematic searches for remediation.
- KPI search lag maintenance. This procedure identifies any data source an ITSI search uses that has ingestion latency outside of the configured monitoring lag. The procedure also adjusts the monitoring lag for each search to an appropriate level.
- Adaptive threshold maintenance. This activity involves the maintenance of adaptive thresholds that are applied to service KPIs to ensure that they are appropriate and representative of good service function states.
- Duplicate entity maintenance. This activity involves the maintenance of service entities to ensure that the entity state presented in ITSI accurately reflects the service entity state of the business and can support the good function of ITSI.
Schedule
The following schedule describes the time intervals that the provided maintenance procedures are recommended to be performed at.
| Task | This task is performed at least every: | Expected duration: |
|---|---|---|
| Event _time indexing checks | Month | 15 minutes |
| KPI Search execution check | Month | 15 minutes |
| KPI Search lag maintenance | Month | 30 minutes |
| Adaptive threshold maintenance | Quarter | 45 minutes |
| Duplicate entity maintenance | Month | 30 minutes |