Baseline of domains accessed by network users

Running searches to compare recent data to data collected over the past 30 days can take a long time. Splunk's lookup functionality allows you to create a cache of previously seen datapoints, enabling you to search faster. You might need to create a lookup file of domains users on your network access often when doing the following:

• Monitoring employee network traffic

Prerequisites 

In order to execute this procedure in your environment, the following data, services, or apps are required:

• Proxy data
• URL Toolbox

Example

You are concerned about employees accidentally accessing malicious domains. You want to create a baseline of commonly accessed domains that you can use to quickly compare against recent activity. 

NOTE: To optimize the search shown below, you should specify an index and a time range. 

1. Set the time range picker to Last 30 Days. You can choose any time range you want, but 30 days is a good scope for baseline data. 
2. Run the following search:

tag=web url=*
| eval list="mozilla"
| `ut_parse_extended(url,list)`
| stats earliest(_time) AS earliest latest(_time) AS latest BY ut_domain
| outputlookup <previously_seen_domains.csv>

Search explanation

The table provides an explanation of what each part of this search achieves. You can adjust this query based on the specifics of your environment.

Result

Use this lookup table when monitoring new accessed domains in order to speed up your searches.