Skip to main content
 
 
 
Splunk Lantern

Detecting Operational Technology assets communicating with external systems

 

For decades, the teams in charge of securing industrial control systems lived with a false sense of security. Firewalls and the infrequency of attacks on operational technology allowed security teams to place Operational Technology (OT) security in a separate category than overall IT security. Those days are over.

A series of high-profile, costly, and disruptive attacks have exposed weaknesses in OT, sparking a wave of new regulations aimed at bolstering security in critical industries such as electrical utilities and oil & gas pipelines. Heightened regulatory scrutiny is occurring in parallel with a steady convergence of traditional IT and OT as systems increasingly interact and overlap with a sharp rise in connectivity and data at the edge.

The answer to this problem is a unified, real-time view across an organization’s IT and OT environments. This enables better collaboration between IT, security, and operations teams, enhances security posture, improves uptime, and ensures safety and compliance.

You can use the Splunk platform to run searches, for example, to detect an OT asset communicating with external systems successfully (inbound or outbound). If you're an Splunk Enterprise Security user, you can also use the OT Security Add-On for Splunk to expand existing Splunk Enterprise Security frameworks to improve security visibility in OT environments.

How to use Splunk software for this use case

Run the following search. You can optimize it by specifying an index and adjusting the time range.

| tstats count min(_time) AS firstTime max(_time) AS lastTime 
    FROM datamodel=Network_Traffic 
    WHERE All_Traffic.action=allowed AND 
    BY All_Traffic.src All_Traffic.dest All_Traffic.dest_port All_Traffic.action
| rename All_Traffic.* AS * 
| eval is_external_destination=if(`is_external(dest)`, true, false), is_external_source=if(`is_external(src)`, true, false)
| eval is_ot_source=if(`is_ot_asset(src)`, true, false), is_ot_destination=if(`is_ot_asset(dest)`, true, false)
| search (is_ot_source=true AND is_external_destination=true) OR (is_ot_destination=true AND is_external_source=true)
| eval direction=if(is_ot_source, "outbound", "inbound")

Search explanation

The table provides an explanation of what each part of this search achieves. You can adjust this query based on the specifics of your environment.

Splunk Search Explanation

| tstats count min(_time) AS firstTime max(_time) AS lastTime

FROM datamodel=Network_Traffic

WHERE All_Traffic.action=allowed AND

BY All_Traffic.src All_Traffic.dest All_Traffic.dest_port All_Traffic.action

Search the network traffic data model for traffic being allowed.

This search could be further refined to only search network traffic that involves an OT network or OT perimeter device or to also analyze blocked connections for suspicious activity.

| rename All_Traffic.* AS *

Rename fields returned from the data model to remove the prefix of ‘All_Traffic.’

| eval is_external_destination=if(`is_external(dest)`, true, false), is_external_source=if(`is_external(src)`, true, false)

Use the Splunk Enterprise Security macro `is_external` to determine whether a source or destination is external to an organization. This section of the search can be customized based on your organization’s network setup.

| eval is_ot_source=if(`is_ot_asset(src)`, true, false), is_ot_destination=if(`is_ot_asset(dest)`, true, false)

Use the Splunk Enterprise Security macro `is_ot_asset` to test whether an asset is considered part of an OT environment. This section of the search can be customized based on your organization’s network setup.

| search (is_ot_source=true AND is_external_destination=true) OR (is_ot_destination=true AND is_external_source=true)

Tests for two different scenarios and filters down the result appropriately:

  1. The OT Asset is the destination (inbound from an external source) OR
  2. The OT Asset is the source (outbound to an external source)

| eval direction=if(is_ot_source, "outbound", "inbound")

Labels the direction of the connection as inbound (source is external) and outbound (destination is external)

Using the OT Security Add-On for Splunk

If you're an Splunk Enterprise Security user, you can use the OT Security Add-On for Splunk to expand existing Splunk Enterprise Security frameworks to improve security visibility in OT environments. This add-on provides capabilities in three primary areas:

  1. Expanded ability to ingest and monitor OT Assets
  2. Improved OT vulnerability management including support for Mitre ATT&CK for ICS
  3. Interfaces and reports to support NERC CIP audit and compliance

The OT Security Add-On for Splunk is built to enable improved integration with leading OT security technologies including inventory discovery and management systems, network monitoring and anomaly detection solutions, endpoint monitoring and patch management tools. Splunk experts have worked closely with many leading OT security vendors, including Dragos, Nozomi, Claroty and others to enable high value data access and visibility.

Dashboard examples

OT Perimeter Monitor

The OT Perimeter Monitor dashboard is designed to provide an overview of the perimeter of an OT system as defined in the get_perimeter_devices macro. It focuses on traffic traversing in and out of the security perimeter and whether it is allowed, blocked, and/or perimeter based on a combination of the interesting_ports_lookup and prohibited_traffic_lookup_by_category lookup definitions. The interesting_ports_lookup allows you to proactively identify traffic and the associated application.

For example, a user could include port TCP 502 as an interesting port and label it as the app Modbus. Using the prohibited traffic lookup file they could then specify that traffic on TCP port 502 is prohibited between an OT zone and corporate zone. Another common example is internet traffic outbound from the OT zone to another zone. Each of these lookup tables supports wildcards for fields as well.

While in many cases this traffic should be blocked automatically via the perimeter, this dashboard helps validate that these assumptions are correct. In addition, the dashboard provides changes in traffic, such as new ports or significant changes in traffic volumes.

unnamed (99).png

OT Asset Investigator - Asset Details

This dashboard provides information that is known about an asset, such as its characteristics, classifications, and physical location (when latitude and longitude are provided). In addition, the dashboard contains several high-level indicators including known vulnerabilities related to this asset, any notable events, as well as any traffic that has been detected as prohibited. Scroll down from here to view further dashboards in the OT Asset Investigator area.

unnamed (100).png

OT Asset Investigator - OT Asset Behavioral Indicators

The OT Asset Behavioral Indicators provide base metrics on the networking behavior of the selected asset and visualizes the types of communication with others hosts in the environment. This can prove important when trying to identify for example communication that should not be permitted between subnets. You can click on the subnets listed on the left to filter the network communication graph on the right-hand side.

unnamed - 2023-11-24T123334.588.png

OT Asset Investigator - Sourcetype Timeline Drilldown

The drilldown shown in the final section of the OT Asset Investigator area provides information around data sources which contain the selected asset and can help guide analysis to different data sources to investigate. By selecting a data source in the bottom panel, analysts can quickly drill down into a search showing that data source and the asset selected for ad hoc analysis.

unnamed - 2023-11-24T123422.887.png

Next steps

These resources might help you understand and implement this guidance:

Splunk OnDemand Services: Use these credit-based services for direct access to Splunk technical consultants with a variety of technical services from a pre-defined catalog. Most customers have OnDemand Services per their license support plan. Engage the ODS team at ondemand@splunk.com if you require assistance.