Skip to main content
Splunk Lantern

GCP Kubernetes cluster scan detection

A Kubernetes cluster is a set of worker machines, called nodes, that run containerized applications. A cluster usually runs multiple nodes to provide fault-tolerance and high availability. The cluster master runs the Kubernetes API server, scheduler, and core resource controllers. You might want to look for unauthenticated requests via user agent and authentication data against a GCP Kubernetes cluster when doing the following:

Prerequisites 

In order to execute this procedure in your environment, the following data, services, or apps are required:

Example

One of your concerns with using Kubernetes is unauthenticated requests. You want to know if attackers are targeting your clusters, and, if so, to get suspicious request details, such as IP address, user agent, request URI, and response status data.

To optimize the search shown below, you should specify an index and a time range.

  1. Ensure that you have configured stackdriver and set a Pub/Sub subscription to be imported to Splunk.
  2. Run the following search: 
sourcetype="google:gcp:pubsub:message" data.protoPayload.requestMetadata.callerIp!=<valid IP address>  data.protoPayload.requestMetadata.callerIp!=::1 "data.labels.authorization.k8s.io/decision"=forbid
"data.protoPayload.status.message"=PERMISSION_DENIED data.protoPayload.authenticationInfo.principalEmail="system:anonymous"
| rename data.protoPayload.requestMetadata.callerIp AS src_ip 
| stats count min(_time) AS firstTime max(_time) AS lastTime values(data.protoPayload.methodName) AS method_name values(data.protoPayload.resourceName) AS resource_name values(data.protoPayload.requestMetadata.callerSuppliedUserAgent)AS http_user_agent BY src_ip data.resource.labels.cluster_name |rename data.resource.labels.cluster_name AS cluster_name
| convert timeformat="%m/%d/%Y %H:%M:%S" ctime(lastTime)
| convert timeformat="%m/%d/%Y %H:%M:%S" ctime(firstTime)  

Search explanation

The table provides an explanation of what each part of this search achieves. You can adjust this query based on the specifics of your environment.

Splunk Search Explanation

sourcetype="google:gcp:pubsub:message" 

Search only GCP Pub/Sub messages.

data.protoPayload.requestMetadata.callerIp!=<valid IP address> 
data.protoPayload.requestMetadata.callerIp!=::1 

Exclude a legitimate IP address or range of addresses from the search.

"data.labels.authorization.k8s.io/decision"=forbid

Search for an authorization status of forbid.

"data.protoPayload.status.message"=PERMISSION_DENIED 

Search for an authentication status of PERMISSION_DENIED.

data.protoPayload.authenticationInfo.principalEmail="system:anonymous"

Search for anonymous users.

| rename data.protoPayload.requestMetadata.callerIp AS src_ip 

Rename the field as shown for better readability.

| stats count min(_time) AS firstTime max(_time) AS lastTime values(data.protoPayload.methodName) AS method_name values(data.protoPayload.resourceName) AS resource_name values(data.protoPayload.requestMetadata.callerSuppliedUserAgent)AS http_user_agent BY src_ip data.resource.labels.cluster_name 

Count the number of times the values shown occurred for each unique source IP address, user name, and user group.

| rename data.resource.labels.cluster_name AS cluster_name

Rename the fields as shown for better readability.

| convert timeformat="%m/%d/%Y %H:%M:%S" ctime(lastTime)

| convert timeformat="%m/%d/%Y %H:%M:%S" ctime(firstTime)

Convert these times into readable strings.

Result

Not all unauthenticated requests are malicious, but the strings provided in this search provide context. Analyze the following to determine what needs further investigation:

  • Source IP address reputation, geolocation, and access policy
  • Banners in the user agent for scanning tools, such as Zgrab or Nmap
  • Targeted files, directories or command strings in the request URI field 
  • A high number of forbidden, unauthorized, or failure responses

For additional information about this search, such as its applicability to common frameworks and standards, see this project on GitHub.

  • Was this article helpful?