Uncommon top level domains

You might want to be able to extract uncommon top level domains from your data when doing the following:

• Monitoring employee network traffic

Prerequisites 

In order to execute this procedure in your environment, the following data, services, or apps are required:

• Firewall data
• URL Toolbox

Example

You recently started work as a Security Analyst for a company whose users often access websites that don't use the Latin alphabet. You need to be able to accurately extract all URLs from your data.

NOTE: To optimize the search shown below, you should specify an index and a time range. In addition, this sample search uses Palo Alto Networks data. You can replace this source with any other firewall data used in your organization.

1. Run the following search: 

sourcetype=pan:threat 
| stats count BY url 
| urlparser field=url listname="mozilla" mode=extended

Search explanation

The table provides an explanation of what each part of this search achieves. You can adjust this query based on the specifics of your environment.

Result

The results show the URLs visited by your users, but it doesn't separate the legitimate ones from the suspicious ones. Use your own lookup tables to help sort through the results, or use the analytic functions of the URL Toolbox app, like Shannon Entropy, to find URLs that you need to investigate.