Skip to main content
Lantern Home
 
 
Splunk Lantern

Merging common values from separate fields

  1. Last updated
  2. Save as PDF

 

​You have fields in your data that contain some commonalities. For example:

Field Value
Team A 12_357_p
Team B 12367_yzx

You want to create a third field that combines the common values in the existing fields.

Solution

You can accomplish this using a number of multivalue evaluation functions. The following search uses the two values above and returns the following value: 1237. 

| makeresults
| eval sourcefield="Team B: 12367_yzx Team A: 12_357_p"
| rex field=sourcefield "Team B: (?<f1>[^\\s]+)\\sTeam A: (?<f2>.*)"
| eval f1split=split(f1, ""), f2split=split(f2, "")
| eval f1split=mvfilter(f1split!="_"), f2split=mvfilter(f2split!="_")
| eval f1matchingf2=mvmap(f1split, if(in(f2split, f1split), f1split, null()))
| eval flattened=mvjoin(f1matchingf2, "")
| fields _time, flattened
Splunk Search Explanation
| makeresults

Create a temporary, dummy event.
| eval sourcefield="Team B: 12367_yzx Team A: 12_357_p" Create a field that contains the data you want to merge.
| rex field=sourcefield "Team B: (?<f1>[^\\s]+)\\sTeam A: (?<f2>.*)"

Extract team data into distinct fields (called f1 and f2).

This rex command creates 2 fields from 1. If you have 2 fields already in the data, omit this command.
| eval f1split=split(f1, ""), f2split=split(f2, "")

Make multi-value fields (called f1split and f2split) for each target field.

The split function uses some delimiter, such as commas or dashes, to split a string into multiple values. If you pass in a blank string, as in this example, the function will return each character of the string individually. So if your string is 15 characters long, you will get a resulting field that has 15 individual elements.
| eval f1split=mvfilter(f1split!="_"), f2split=mvfilter(f2split!="_") Filter out the underscores.
| eval f1matchingf2=mvmap(f1split, if(in(f2split, f1split), f1split, null()))

For each f1split field value, check to see if it's in f2split. If so, return the f1split value. Otherwise, do nothing. 

The mvmap command takes a multi value field, runs eval functions against each item in the array, and returns the result of each function into the field on the left side of the equals, in this case f1matchingf2. 
| eval flattened=mvjoin(f1matchingf2, "")

Flatten the multi-value field into a single string.

The mvjoin command is often is used with commas or hyphens or other padding, but in this example we want only a string of characters.
| fields _time, flattened Return only the time stamp and flattened field.

Next steps

These additional Splunk resources might help you understand and implement these recommendations: