Skip to main content
Splunk Lantern

Baseline of domains accessed by network users

Running searches to compare recent data to data collected over the past 30 days can take a long time. Splunk's lookup functionality allows you to create a cache of previously seen datapoints, enabling you to search faster. You might need to create a lookup file of domains users on your network access often when doing the following:

Prerequisites 

In order to execute this procedure in your environment, the following data, services, or apps are required:

Example

You are concerned about employees accidentally accessing malicious domains. You want to create a baseline of commonly accessed domains that you can use to quickly compare against recent activity. 

To optimize the search shown below, you should specify an index and a time range. 

  1. Set the time range picker to Last 30 Days. You can choose any time range you want, but 30 days is a good scope for baseline data. 
  2. Run the following search:
tag=web url=*
| eval list="mozilla"
| `ut_parse_extended(url,list)`
| stats earliest(_time) AS earliest latest(_time) AS latest BY ut_domain
| outputlookup <previously_seen_domains.csv>

Search explanation

The table provides an explanation of what each part of this search achieves. You can adjust this query based on the specifics of your environment.

Splunk Search Explanation

tag=web

Search for fields with the "web" tag.

url=*

Return proxy data by searching for results with a value in the URL field.

| eval list="mozilla"

Search the Mozilla catalog for top level domains.

This eval function is required for the next line in the search (ut_parse_extended) to work.

| `ut_parse_extended(url,list)`

Parse the URLs based on the Mozilla top level domain list.

The punctuation in a Splunk macro is always a back tick (`), not a single quote (').

| stats earliest(_time) AS earliest latest(_time) AS latest BY ut_domain

Return the results in a table—grouped by the ut_domain field—that includes columns for the first and last time each domain was seen in the dataset.

outputlookup <previously_seen_domains.csv>

Write the results to a new CSV lookup file, here called previously_seen_domains.

Result

Use this lookup table when monitoring new accessed domains in order to speed up your searches.

  • Was this article helpful?