Skip to main content
Splunk Lantern

Baseline of domains accessed by network users

Running searches to compare recent data to data collected over the past 30 days can take a long time. Splunk's lookup functionality allows you to create a cache of previously seen datapoints, enabling you to search faster.

Data required 

Proxy data

Procedure

You are concerned about employees accidentally accessing malicious domains. You want to create a baseline of commonly accessed domains that you can use to quickly compare against recent activity. 

  1. Set the time range picker to Last 30 Days. You can choose any time range you want, but 30 days is a good scope for baseline data. 
  2. Run the following search. You can optimize it by specifying an index and adjusting the time range.
tag=web url=*
| eval list="mozilla"
| `ut_parse_extended(url,list)`
| stats earliest(_time) AS earliest latest(_time) AS latest BY ut_domain
| outputlookup <previously_seen_domains.csv>

Search explanation

The table provides an explanation of what each part of this search achieves. You can adjust this query based on the specifics of your environment.

Splunk Search Explanation

tag=web

Search for fields with the "web" tag.

url=*

Return proxy data by searching for results with a value in the URL field.

| eval list="mozilla"

Search the Mozilla catalog for top level domains.

This eval function is required for the next line in the search (ut_parse_extended) to work.

| `ut_parse_extended(url,list)`

Parse the URLs based on the Mozilla top level domain list.

The punctuation in a Splunk macro is always a back tick (`), not a single quote (').

| stats earliest(_time) AS earliest latest(_time) AS latest BY ut_domain

Return the results in a table—grouped by the ut_domain field—that includes columns for the first and last time each domain was seen in the dataset.

outputlookup <previously_seen_domains.csv>

Write the results to a new CSV lookup file, here called previously_seen_domains.

Next steps

Use this lookup table when monitoring new accessed domains in order to speed up your searches.

You might also be interested in other processes associated with the Monitoring employee network traffic use case.