You’ve got data flowing in, visualizations are populated, and you’re making use of Tag Spotlight as well as business workflows. What's next? Alerting, of course. Splunk APM detectors use built-in algorithms to detect sudden spikes and historical anomalies in your Splunk APM metrics or Business Workflows.
Overview of creating service and business workflow detectors
You can dynamically monitor error rate and latency in the services you are tracing with Splunk APM as well as Business Workflows. Let’s walk through a configuration of a Splunk APM Service/Business Workflow Detector below.
So, what can you configure within a Detector? Detectors contain rules that specify:
- When the detector will be triggered, based on conditions related to the detector’s signal/metric
- The severity of the alert to be generated by the detector
- Where notifications should be sent
From there, set up your detector parameters:
- Type. Choose what type of detector to create: APM Metric or Infrastructure/Custom Metric.
- Alert Signal. Define what Service Metric or Business Workflow are you trying to alert on: Error Rate or Latency. Here you will also define the specific environment and specific service/endpoint
- Alert Condition. Define the conditions of the signal/metric in which you would like to be alerted on: Static Threshold or Sudden Change.
- Alert Settings. These settings depend on which condition is selected and will be configured at this step.
- Alert Message. Define the severity of the alert and customize the message of it. Can also link to helpful documentation to be delivered with the alert.
- Alert Recipients. Define who will receive the alert and the delivery method: email, Splunk On-Call, Slack, PagerDuty, Webhook, etc.
Now that you have the core components for Splunk APM, let’s focus on administration. It’s important for you to know how to best manage the tool in order to optimize usage throughout your organization.
You can find all of the documentation for administration related activities here, but let’s touch on a few important ones to be aware of as you get started: