Optimizing correlation searches in Enterprise Security
You have a base level of enabled correlation searches in your Splunk Enterprise Security deployment. They help you identify security events and anomalous audit trails. However, you have too many false positives and false negatives. In addition, you've added new security data and are monitoring more devices and networks now. You want to tune your enable searches to work better for your environment.
Solution
First, ensure that you have only enabled correlation searches that you need in your environment. Consider the following:
- Types of vulnerabilities or threats you have determined might exist
- The type of security operations you are focused on such as malware or intrusion detection
Then you can work on tuning thresholds, throttling, and adaptive response actions to optimize the searches. The documentation on configuring correlation searches will also help.
Correlation threshold
Some correlation searches may generate more or fewer notable events than you want. You should examine the search strings and look for comparison terms in search or Machine Learning Toolkit models, then modify as appropriate for your environment. You'll want to consider both of the following thresholds:
Numeric threshold
Some searches might be based on a simple numeric comparison, such as excessive DNS failures. These kinds of searches generally use the where command with a numeric comparison. Change the numeric value if you need to alter how frequently notable events are generated in your environment.
Conceptual threshold
The conceptual threshold uses Machine Learning Tool Kit (MLTK) functions. Here is an example macro from a brute force access behavior detected correlation search:
|`mltk_apply_upper("app:failures_by_src_count_1h","high","failure")'
The arguments are:
- Model. Name of the model for applying data and comparing against standards to find outliers (In this example,
app:failures_by_src_count_1h) - Qualitative_id. Default IDs that correspond to percentages of deviation, representing where on the distribution curve to look for outliers (In this example,
high. You might want to change this threshold tomediumorlow) - Field. Where to search for or count outliers (In this example,
failure)
Correlation search throttling
After a correlation search has been triggered, you probably don’t want it to immediately re-trigger again for the same issue. Most out-of-the-box correlation searches throttle alerts to once a day. If you want to modify this, change the Window duration.
Adaptive response actions
When a correlation search detects an issue, it can initiate one or more adaptive response actions. Adaptive responses are lists of actions to take, including:
- creating a notable event (the most common)
- setting risk
- sending email
- running scripts
- stream capture
- sending data to Splunk User Behavior Analytics
You can modify all the properties of the notable event that is created by a triggered correlation search, typically:
- Severity
- Default Owner
- Default Status
You can also add risk to the objects associated with the issue.
Next steps
If you found this article useful and want to advance your skills, Splunk Education offers a 13.5-hour, instructor-led course on administering Splunk Enterprise Security. The hands-on labs in the course will teach you how to:
- Examine how Splunk Enterprise Security functions, including data models, correlation searches, notable events, and dashboards
- Create custom correlation searches
- Customize the Investigation Workbench
- Learn how to install or upgrade Splunk Enterprise Security
- Learn the steps to setting up inputs using technology add-ons
- Fine tune Splunk Enterprise Security Global Settings
- Customize risk and configure threat intelligence
Click here for the course catalog where you can read the details about this and other Splunk Enterprise Security courses, as well as register.