Skip to main content
Lantern Home
 
 
Splunk Lantern

Masking IP addresses from a specific range

  1. Last updated
  2. Save as PDF

 

Your web team has asked you to mask IP addresses from your internal 10.x.x.x range in your web server data.

Solution

There are multiple ways of achieving this IP masking use case with SPL2 in Splunk Edge Processor, depending on how flexible you want your pipeline to be. Let’s look at two possible methods.

Method 1: Use eval replace

This option uses a simple eval statement to replace the IP address using a regular expression.

  1. Go to the Pipelines page, click on Create pipeline, and select New pipeline.
  2. On the pipeline menu on the Actions tab, click the pencil icon next to "Use data from $source".
  3. Change the source to the correct source type for the data you want to process in this pipeline (example: access_combined), then click Apply.
  4. In the pipeline pane, click Mask values in _raw…
  5. In the Matching regular expression field, paste the following regular expression:
    ^10\.(?:[0-9]{1,3}\.){2}[0-9]{1,3}
  6. In the Replace with field, enter [IP_REDACTED].
  7. Leave Match case selected and click Apply. An eval command is automatically added to your SPL2 statement, which should now look like this: 
    $pipeline =
| from $source
| eval _raw=replace(_raw, /^10\.(?:[0-9]{1,3}\.){2}[0-9]{1,3}/, "[IP_REDACTED]")
| into $destination;
  8. (Optional) Add a custom indexed field to your events to help you identify them in Splunk Cloud Platform by adding an eval command to your pipeline. To create a field called "participant" with the value set to your own name: 
    $pipeline =
| from $source
| eval _raw=replace(_raw, /^10\.(?:[0-9]{1,3}\.){2}[0-9]{1,3}/, "[IP_REDACTED]")
| eval participant="rlarkman" ← Update with your own name
| into $destination;
  9. Test your masking rule by clicking the blue Preview pipeline button in the top right corner of the screen. You should see a preview of your events in the center of the screen showing the IP addresses redacted.
  10. (Optional) Verify that your custom field appears by clicking the Data tab and selecting the participant check box under the FIELDS section to show the participant field in the preview pane.
  11. Return to the Actions tab and click the pencil icon next to "Append data to $destination".
  12. Set the destination to the web index and click Apply.
  13. Click Save pipeline in the top right corner of the screen.
  14. Give your pipeline a suitable name, such as access_combined_ip_mask_<yourName> or something similar.
  15. Navigate to your pipelines list by clicking on Pipelines on the top left of the page.
  16. Locate the pipeline you just created and click the three dots next to it.
  17. Select Apply/remove.
  18. Select the box next to the name of the Edge Processor you created earlier and click Save.
  19. Log in to Splunk Cloud Platform and open the Search & Reporting app.
  20. Run the following search over the last 15 minutes and verify that you now see the redacted events: 
    index=web "[IP_REDACTED]" participant="rlarkman" ← Remember to update the name!

Method 2: Use rex and cidrmatch

This option leverages the power of SPL2 to build a more complex pipeline that will give you more to adapt your IP masking in future.

  1. Go to the Pipelines page, click Create pipeline, and select New pipeline.
  2. On the pipeline menu on the Actions tab, click the pencil icon next to "Use data from $source".
  3. Change the source to access_combined then click Apply.
  4. Extract IP addresses from your web events into a new field called "ip" by adding the following rex command to your pipeline. 
    | from $source
| rex field=_raw /(?P<ip>(([0-9]{1,3}\.){3}[0-9]{1,3}))/
    Remember that Edge Processor uses RE2 syntax for regular expressions, not the PCRE syntax that is used in the Splunk platform. For more information, see Supported regular expression syntax.
  5. Check that your new field extracts correctly by clicking the blue Preview pipeline button in the top right corner of the screen and selecting the ip field on the Data tab. You should see the ip column displayed in the preview pane.
  6. To check whether the value of ip is in the 10.x.x.x range, add an eval statement and leverage the cidrmatch function to check against the 10.0.0.0/8 CIDR range. If the value matches the range ip_type will be set to internal. Otherwise, it will be set to external. 
    | from $source
| rex field=_raw /(?P<ip>(([0-9]{1,3}\.){3}[0-9]{1,3}))/
| eval ip_type=if(cidrmatch("10.0.0.0/8",ip), "internal", "external")
  7. Add a where statement to filter by only internal IPs, that is, only those that match the 10.0.0.0/8 CIDR range. 
    | from $source
| rex field=_raw /(?P<ip>(([0-9]{1,3}\.){3}[0-9]{1,3}))/
| eval ip_type=if(cidrmatch("10.0.0.0/8",ip), "internal", "external")
| where ip_type="internal"
  8. Now that you’re filtering by only the events with IPs based on the 10.0.0.0/8 range, add an eval statement and use the replace function to mask/redact the values in the ip field: 
    | from $source
| rex field=_raw /(?P<ip>(([0-9]{1,3}\.){3}[0-9]{1,3}))/
| eval ip_type=if(cidrmatch("10.0.0.0/8",ip), "internal", "external")
| where ip_type="internal"
| eval _raw=replace(_raw, ip, "[IP_REDACTED]")
  9. Click the blue Preview pipeline button in the top right corner of the screen and check that the IPs are redacted from your events.
  10. Now that you’ve masked IPs from _raw, you need to remove the ‘ip’ field you created as that field contains unmasked IP addresses. 
    | from $source
| rex field=_raw /(?P<ip>(([0-9]{1,3}\.){3}[0-9]{1,3}))/
| eval ip_type=if(cidrmatch("10.0.0.0/8",ip), "internal", "external")
| where ip_type="internal"
| eval _raw=replace(_raw, ip, "[IP_REDACTED]")
| fields -ip
  11. Finally, add a "participant" field to your events to help you identify them in Splunk Cloud Platform: 
    | from $source
| rex field=_raw /(?P<ip>(([0-9]{1,3}\.){3}[0-9]{1,3}))/
| eval ip_type=if(cidrmatch("10.0.0.0/8",ip), "internal", "external")
| where ip_type="internal"
| eval _raw=replace(_raw, ip, "[IP_REDACTED]")
| fields -ip
| eval participant="rlarkman" ← Remember to update the name!
| into $destination;
  12. Test your masking rule by clicking the blue Preview pipeline button in the top right corner of the screen. You should see a preview of your events in the center of the screen showing the IP addresses redacted.
  13. On the Actions tab, click the pencil icon next to "Append data to $destination".
  14. Set the destination to the web index and click Apply.
  15. Click Save pipeline in the top right corner of the screen.
  16. Give your pipeline a suitable name, such as access_combined_cidr_mask_<yourName> or something similar to help distinguish it from the IP masking pipeline you built in Method 1.
  17. Navigate to your pipelines list by clicking on Pipelines on the top left of the page.
  18. Locate the pipeline you just created and click the three dots next to it.
  19. Select Apply/remove to remove the previous IP masking pipeline to avoid any conflict
  20. Deselect the box next to the name of the Edge Processor you created earlier and click Save. An icon appears under the Number of Edge Processors column for your pipeline, indicating that the pipeline is being removed.
  21. Refresh the page until the icon disappears and the Number of Edge Processors reads 0 for your IP masking pipeline.
  22. Locate the CIDR masking pipeline you created and click the three dots next to your new pipeline.
  23. Select Apply/remove.
  24. Select the box next to the name of the Edge Processor you created earlier and click Save.
  25. Log in to Splunk Cloud Platform and open the Search & Reporting app.
  26. Run the following search over the last 15 minutes and verify that you now see the redacted events: 
      index=web "[IP_REDACTED]" participant="rlarkman" ← Remember to update the name!

Next steps

These additional Splunk resources might help you understand and implement this use case: