Skip to main content
Lantern Home
 
 
 
Splunk Lantern

Improving Splunk platform searches with bitwise operators

  1. Last updated
  2. Save as PDF

 

Bitwise functions in Splunk search remove the need for complicated workarounds and complex eval functions for you to manipulate fields in eval searches. They can also enable you to view extracted flags in a human-readable format. 

With Bitwise function support, Splunk SPL supports bitwise manipulation on bit arrays, which are commonly used in sources like OSes and network devices. For example, this can enable operations such as capturing a flag or applying masks on values without having to use Python workarounds.

The operators include:

  • bit_and
  • bit_or
  • bit_not
  • bit_xor
  • bit_shift_left
  • bit_shift_right

Let’s take a look at a few examples of what these functions can do.

Example one

Let’s say we want to find the number of times a set of two sensors gives us no signal. Without bitwise operators, we would need the following search:

index=summary
| table master_cluster, backup_cluster
```extract first bit from both clusters```
| eval cluster1_bit1 = substr(master_cluster, 1, 1), cluster2_bit1 = substr(backup_cluster, 1, 1)
| eval result_bit1 = if(cluster1_bit1 + cluster2_bit1 = 0, 0, 1)
```extract second bit from both clusters```
| eval cluster1_bit2 = substr(master_cluster, 2, 1), cluster2_bit2 = substr(backup_cluster, 2, 1)
| eval result_bit2 = if(cluster1_bit2 + cluster2_bit2 = 0, 0, 1)
```extract third bit from both clusters```
| eval cluster1_bit3 = substr(master_cluster, 3, 1), cluster2_bit3 = substr(backup_cluster, 3, 1)
| eval result_bit3 = if(cluster1_bit3 + cluster2_bit3 = 0, 0, 1)
```extract fourth bit from both clusters```
| eval cluster1_bit4 = substr(master_cluster, 4, 1), cluster2_bit4 = substr(backup_cluster, 4, 1)
| eval result_bit4 = if(cluster1_bit4 + cluster2_bit4 = 0, 0, 1)
| eval signal_status = result_bit1 + result_bit2 + result_bit3 + result_bit4
| table master_cluster, backup_cluster, signal_status
| where signal_status != 4
| stats count

Using bit_OR, we have a much simpler and more elegant way to get to a fairly common request.

index=summary
| table master_cluster, backup_cluster
| eval signal_status = bit_OR(tonumber(master_cluster, 2), tonumber(backup_cluster, 2))
| where signal_status != 15 ```decimal representation of "1111" is 15```
| stats count

The following video shows a comparison of running these two searches in real time. Note that the video has no sound.

Example two

Next, let’s create a simple table using the eval command. We can see we have a table with two numbers in it.  

| makeresults
| eval result1 = 2, result2 = 2

Let’s apply some bitwise operators to this table.

| makeresults 
| eval result1 = bit_shift_left(2, 1), result2 = bit_shift_right(2, 1)

In the updated search, the bit_shift_left function shifts the binary representation of the first integer in the argument, "2", to the left by the specified shift amount, "1". 

0b 010 → 0b 0 10 → 0b 100, giving us a result of 4. 

Along the same lines, the bit_shift_right function shifts the binary representation of the first integer. "2", to the right by the specified shift amount, "1".

0b 010 → 0b 010 → 0b 001, giving us a result of 1.

These results are shown in the following screenshot:

clipboard_eb26e522a2c002cfae431557558538c73.png

Example three

Bitwise operators in SPL can be nested to perform complex operations. Let’s look at an example.

Say you have a field called StreamId=0x7b1adf. You want to extract the hex values in positions 3 and 4 from the right. In this case, that would be “1a”.  This involves shifting right by 8 (every hex shift right equals four binary shift rights, and we need to eliminate the first two hex values in the right, so 2 X 4 = 8). Now we want only the last two hex digits and can discard everything else, meaning we can simply AND with “0xFF”.

To do this, you can run the following search:

| makeresults
| eval StreamId="0x7b1adf"
| eval result = bit_AND(bit_shift_right(tonumber(StreamId, 16), 8), tonumber("0xFF", 16))
| eval resultHex = tostring(result, "hex")

Bitwise functions only work with integer values, so the steps we are performing above are:

  1. Converting StreamId from hex to integer: tonumber(StreamId, 16).
  2. Shifting the converted value to the right by 8 places: bit_shift_right(tonumber(StreamId, 16), 8).
  3. Performing a bitwise AND with the integer value of “0xFF”: bit_AND(bit_shift_right(tonumber(StreamId, 16), 8), tonumber("0xFF", 16)).
  4. Finally, converting the result back to hex to see if our operation has worked: tostring(result, "hex").

The result is shown in the following screenshot:

clipboard_efee921e93a30c6b24fe9710a5a358cc7.png

Next steps

As you can see, bitwise operators can take a complex query down to just a few lines, and help you easily accomplish tasks that were previously difficult.

These additional resources might help you understand and implement this guidance: