Skip to main content
Splunk Lantern is a nominee for Knowledge Innovation and Knowledge Management in the CXOne Customer Recognition Awards. Click here to vote for us!
Lantern Home

 

Splunk Lantern

Earning approval for automation activities in your organization

  1. Last updated
  2. Save as PDF

New customers often face challenges when implementing SOAR in their organization. Security automation is already technically complex, but it's often organizational and procedural challenges that have the most impact when implementing a SOAR platform. Being aware of these challenges and having a strategy to overcome them is just as important, if not more so, than the technical skills required to implement an effective security automation platform.

Some commonly faced issues include:

  • The typical 'all or nothing' approach leads to stalled or failed implementations. 
    Going from no existing capability straight to complex automation workflows with alert triage, investigation, determination, and response is rarely achievable and leads to immediate roadblocks that limit adoption of the platform.
  • Misconceptions about SOAR need.
    The Security Teams or SOCs that need SOAR the most often think they are too small or under-resourced to even consider implementing an automation platform. A common misconception is that only large and mature SOCs will benefit from a SOAR capability.
  • Attempting to automate poorly defined or previously non-existent processes.
    A lack of defined processes erodes the value of a SOAR platform, as you can't automate something that doesn't yet exist.
  • Gaps in key skills often undermine even a well-approached deployment.
    Security automation relies heavily on scripting and coding. However, these skills are essential for advanced playbook development, which is often underappreciated and a capability many teams lack in-house. 

This article looks at some basic steps you can take to facilitate a smooth and valuable SOAR implementation. 

Playbook implementation strategy

To smoothly implement SOAR playbooks or any other security automation, you should take a deliberate approach with a pre-planned methodology. 

Successful methodologies usually employ a staged approach to deployment, focusing on smaller achievable steps that provide immediate value to the SOC. In contrast, failed approaches usually involve trying to automate everything as a single effort, where as soon as one component stalls, the entire project halts.

The below diagram outlines a best-practices implementation methodology developed by Splunk Professional Services based on countless worldwide SOAR implementations. You can use this tried-and-tested approach as-is, or customize it to align with your organization's structure and goals.

soar-implementation-stages.png

At a high level these stages cover:

  1. Standalone Instance - Focus on becoming familiar with SOAR, the interface, containers and their properties, and how the case management / workbooks operate
  2. Splunk Integrated - Start with developing playbooks that leverage data already within Splunk to  perform automated queries and investigations
  3. SOC Tooling - Expand integration to build playbooks that leverage other security tools that sit within control of the SOC
  4. External Services - Continue to further expand to integration with external services to provides automated enrichment or analysis
  5. Enterprise Systems - Finally use the combined experience and lessons learned to integrate with larger complex enterprise systems

Underpinning these phases, also consider using a documentation-as-code approach to playbook development to streamline the documentation efforts for each stage.

Playbook design principles

As you get further along in your SOAR playbook development and implementation, you'll notice that playbook design has a lot to do with whether automation succeeds. Here are some basic principles to be aware of:

  • Detailed or low-level designs used for traditional security tools or applications do not work for a SOAR system.
    Automation is a complex area and legacy design principles do not translate across. 
  • You should use a high-level design to set goals, establish guardrails, and help align stakeholders on the automation outcomes.
    It provides the overall intent and what the playbook will do, but not every individual step or action in the process.
  • It's rarely possible to gather all requirements and permutations of the automation flow upfront during the design phase.
    Often stakeholders can't provide this information until they've seen the solution in action and can give feedback on how to adjust the automation flow to reach the desired end state.
  • You're better off spending effort prototyping the solution, iterating, and adjusting based on tangible results from the system.
    Generally speaking, security automation aligns better with a DevOps/CI-CD approach. This also reflects why Splunk SOAR stores playbooks natively within a Git-based repository, designed to support such a workflow.

In summary, the best practice approach is to establish a high-level design up-front with agreement between all stakeholders. You can then proceed with implementation, testing, and acceptance after the solution is in a working state.

Low-level design documentation and diagrams can be produced in parallel with the playbook design and updated as you make adjustments. To save on engineering resources, you could alternatively defer this documentation and complete it after you finalize the solution. 

Splunk SOAR also supports a documentation-as-code approach. Block names, descriptions, notes, tooltips, and in-line code comments all contribute to creating a living source of documentation natively within the playbook. The visual playbook editor (VPE) also provides a user-friendly view of the execution flow to further support this. See the Using a documentation-as-code approach to playbook development for more details on best practices for leveraging documentation as code within Splunk SOAR.

Playbook design principles - Case study

Let's look at some specifics of the typical legacy approach to playbook design, and contrast it with an improved method based on the above principles. 

Legacy approach to playbook development

What is supposed to happen:

  • You gather requirements and create a design.
  • The customer reviews it and, if satisfied, approves the design and locks in all playbook components.
  • You develop the playbook in line with the design.
  • You present the solution to the customer, test it, and get sign off.

This sounds good in theory, and might work in small simple environments with limited variables. However, in reality, what is more likely to occur is as follows:

  • You gather requirements, create and approve a design, and start playbook development.
  • During development and testing, you discover new environment specifics that conflict with the design.
  • You encounter new unknowns or questions that you couldn't have predicted previously.
  • The customer raises additional requirements.
  • You need to re-design, run workshops, spend more time determining the solution, and often reach deadlock.

Our experience shows that in practice, you often spend more time on design and rework than on developing the playbook itself.

Best practice approach

Instead, here is a revised approach designed to provide flexibility to overcome these challenges:

  • Enter with an enhanced focus on discovery to draw out requirements, assumptions, and potential blockers.
  • Use high-level design to capture the general flow and overall intent, but not every individual block and function.
  • Give the playbook developer freedom to innovate and develop the solution, as long as it still meets the intent from the high-level design.
  • Use a true-up activity at the end to improve the documentation to reflect what was actually built. Ideally, the developer will document the automation throughout the build process, using the principles outlined in the Using a documentation-as-code approach to playbook development page.

For more information on good playbook design and to get started with some automation, see Using SOAR automation to improve your SOC processes.

Next steps

To further drive effective automation in your organization, consider the following techniques:

  • Review existing documented SOC processes, work instructions, and SOPs, and consider how SOAR automation could improve them
  • Consult security analysts directly about their day-to-day tasks, and ask where they find themselves working on repetitive tasks or large time sinks
  • Leverage the Community Playbooks

And remember, don't discard small automations; tasks that appear minor or not worth the effort can add up to huge, tangible savings in the long run. 

You might find these additional resources helpful in implementing the guidance in this article:

  • Written by Richard Hampshire, Security Architect at Splunk and Matthew Bennett, Managing Director at Hyperion3