Reducing log volume with SPL2 Linux/Unix templates for Edge Processor and Ingest Processor

Prerequisites

Before you start using an SPL2 template to reduce log size, you should have the following:

• Splunk Cloud Platform with Splunk Edge Processor or Splunk Ingest Processor enabled
• A Splunk destination instance configured to index data after processing it through the SPL2 pipeline

Preparation

To prepare to use an SPL2 template to reduce log volume, you should do the following:

1. Install the universal forwarder and the Splunk Add-on for Unix and Linux on your *nix server.
2. Configure the add-on to enable ps logs. You can do it by modifying inputs.conf and setting disabled = 0 as below.

   /opt/splunkforwarder/etc/apps/Splunk_TA_nix/default/inputs.conf
   
   [script://./bin/ps.sh]
   interval = 30
   sourcetype = ps
   source = ps
   disabled = 0
   

3. Configure the universal forwarder to send data to Splunk Edge Processor. Navigate to Edge Processor → Select your Edge Processor instance → Actions → Configure Data Sources.
4. You will be presented with a screen similar to the one shown below. Follow the provided instructions. For more details on this process, see Splunk Docs.

   

5. If you are using this pipeline with an Edge Processor, navigate to the Source types page in your tenant and make sure that the ps source type has been defined with the following settings. For more information, see Splunk Docs.
   1. Name: ps
   2. Line breaking: ([\r\n]+)
   3. Merge lines into events: Checked
   4. Multiline event delimiter: (\s*USER)
   5. Expand Advanced settings and set Maximum lines per event to 10000

Create a pipeline using an already available template

If you already have access to the published predefined templates, you can quickly create pipelines using them.

Navigate to the Pipelines section within your tenant and choose Templates. Search for the UNIX and Linux ps logs: Reduce log size and convert to TSV format template. On the right side, you'll see the option to Create from Template. Select Edge Processor Pipeline from the list, as shown in the screenshot below.

You'll be taken to the screen shown below. The left panel displays all function names, the middle panel contains the SPL2 code, and the sourcetype is preconfigured as ps. Click Save Pipeline, enter a suitable name, and the pipeline will be created.

Apply your pipeline

After you have constructed your pipeline using the SPL2 template above, follow these instructions to apply your pipeline.

1. Test your pipeline by clicking the blue button in the top right corner of the screen.
2. Set the data destination to the appropriate admin index, as requested by your security team.
3. To save the destination, click Apply.
4. In the top right corner of the screen, click Save Pipeline.
5. You will get a popup asking about applying the pipeline on Edge Processor. Click Yes, apply.
6. Select your Splunk Edge Processor and click Save. You will see a brief message stating that your changes are being saved.

It might take a few minutes before you see the effects of your pipeline in Splunk Cloud Platform.

Verify your pipeline

You can now verify that your Splunk Edge Processor node is healthy and that the pipeline has successfully been applied.

1. To check the status of your node, click the pipeline while on the Pipelines page. A panel will open up on the right of the page, showing the status of the Splunk Edge Processor instance that this pipeline has been applied to.
2. To view more information, click the arrow next to the instance name.
3. After your pipeline has successfully applied, the status of your instance should change to healthy. To verify this, click Edge Processors in the menu on the left of the screen and refresh your web browser until your Splunk Edge Processor displays as healthy.
4. Log into the Splunk platform and open the Search & Reporting app.
5. Run the following search over the last 15 minutes to verify that you can see the events coming from this pipeline:

   index="*" sourcetype="ps"

   The result will look similar to the those in screenshot below.

   

6. Review the Useful Fields pane to see that the filtered event types are gone.

Resources

To ensure a smooth transition of transformed events into production and mitigate potential risks, it is essential to follow best practices and guidelines when using an SPL2 template. These practices should be followed to avoid any mishaps and to maintain the integrity, reliability, and efficiency of the production environment.

These additional Splunk resources might help you understand and implement this use case:

• Lantern: Following best practices for using SPL2 templates
• Lantern: Reducing Palo Alto Networks log volume with the SPL2 template
• Splunk Docs: SPL2 Search Manual
• Splunk Docs: About the Edge Processor solution
• Splunk Blog: Introducing Edge Processor: Next gen data transformation