Skip to main content
 
 
Splunk Lantern

Tracking assets when recovering from an incident

 

After an incident has gone through its containment and eradication stages, it's important to perform certain recovery steps. One major action an incident response team will normally perform is to either fix or rebuild affected systems to ensure they are clean and free from malware. This article discusses best practices for monitoring assets on an ongoing basis to ensure they do not become reinfected.

Fix or rebuild?

There are pros and cons to a fix or rebuild approach, and choosing the approach that's right for you will depend on your comfort with the following factors.

  Pros Cons
Fixing systems
  • Remediates specific vulnerabilities or issues on a system without completely rebuilding it, restoring from a clean backup
  • Involves less down time
  • Preserves network and log history
  • Makes tracking ownership easier
  • Possibility of overlooking areas in systems where a threat actor might hide dormant malware
  • Assets should be monitored to ensure they do not get reinfected
Rebuilding systems
  • A more comprehensive process that recreates the entire system from the ground up
  • Frequently regarded as the most secure way to recover. For more information see NIST.
  • Takes considerably longer
  • Requires a full battery of operating system and application patches
  • Might involve creating a new asset tag and computer name, making tracking more complex

How to use Splunk software to track the recovery process

The information in this article applies to Splunk Enterprise Security (ES) versions 7.x. If you have upgraded to Splunk Enterprise Security version 8.x, some terminology and steps might not apply. For additional assistance on this use case with ES 8.x, Splunk Professional Services can help.

Whether you choose a fix or a rebuild approach, you can use Splunk Enterprise Security searches to track assets on an ad-hoc basis, or Splunk SOAR playbooks to automate asset tracking.

Using the Splunk Enterprise Security Asset and Identity Framework

Having an up-to-date Asset and Identity framework in Splunk Enterprise Security helps you track the recovery process, even where assets are renamed. Tools such as Endpoint Detection and Response (EDR) systems have built-in discovery features that can be used to update the framework. Additional mechanisms, like segmentation of processes, within these tools allow for additional compliance checking.

To track an asset on an ad-hoc basis, run the following SPL in your environment:

| `assets` 
| search nt_host=<HOSTNAME> 
| fields ip,mac,nt_host 
| rename nt_host AS Asset, ip AS "IP Address", mac AS "Mac Address" 
| table Asset,"IP Address","Mac Address"

To monitor the asset, use custom dashboards or generate reports. For example, if you are concerned about the asset's outbound network activity, you can expand the search even further.

Using Splunk SOAR to run playbooks and evaluate results

Using playbooks that can automate recovery provides a simple way for security analysts to drive down the time required to get back to normal after an incident.

Playbooks unique to your organization can be developed using Splunk SOAR to evaluate any rebuilt or fixed assets. Modifying built-in playbooks such as splunk_enterprise_security_tag_assets_and_identities will provide a good foundation to build upon. The purpose of this playbook is to collect user information from Splunk Enterprise Security, then correlate the activity to host names. During the recovery phase of the incident response process the playbook will be invoked to provide the investigator new information about the rebuilt or fixed asset.

Next steps

These resources might help you understand and implement this guidance:

Splunk OnDemand Services: Use these credit-based services for direct access to Splunk technical consultants with a variety of technical services from a pre-defined catalog. Most customers have OnDemand Services per their license support plan. Engage the ODS team at ondemand@splunk.com if you would like assistance.