Skip to main content
Splunk Lantern

Amazon EKS Kubernetes cluster scan detection

A Kubernetes cluster is a set of worker machines, called nodes, that run containerized applications. A cluster usually runs multiple nodes to provide fault-tolerance and high availability. The cluster master runs the Kubernetes API server, scheduler, and core resource controllers. You might want to look for unauthenticated requests via user agent and authentication data against a Kubernetes cluster in AWS when doing the following:

Prerequisites 

In order to execute this procedure in your environment, the following data, services, or apps are required:

Example

One of your concerns with using Kubernetes is unauthenticated requests. You want to know if attackers are targeting your clusters, and, if so, to get suspicious request details, such as IP address, user agent, request URI, and response status data.

NOTE: To optimize the search shown below, you should specify an index and a time range.

  1. Ensure that you have configured your CloudWatch EKS log inputs.
  2. Run the following search: 
sourcetype="aws:cloudwatchlogs:eks" "user.username"="system:anonymous" userAgent!="AWS Security Scanner" 
| rename sourceIPs{} AS src_ip 
| stats count min(_time) AS firstTime max(_time) AS lastTime values(responseStatus.reason) values(source) AS cluster_name values(responseStatus.code) values(userAgent) AS http_user_agent values(verb) values(requestURI) BY src_ip user.username user.groups{}
| convert timeformat="%m/%d/%Y %H:%M:%S" ctime(lastTime)
| convert timeformat="%m/%d/%Y %H:%M:%S" ctime(firstTime) 

Search explanation

The table provides an explanation of what each part of this search achieves. You can adjust this query based on the specifics of your environment.

Splunk Search Explanation

sourcetype="aws:cloudwatchlogs:eks"

Search only AWS EKS Kubernetes data.

"user.username"="system:anonymous" 

Search for anonymous users.

userAgent!="AWS Security Scanner"

Exclude requests from the AWS Security Scanner from the search.

| rename sourceIPs{} AS src_ip 

Rename the field as shown for Splunk processing capability.

| stats count min(_time) AS firstTime max(_time) AS lastTime values(responseStatus.reason) values(source) AS cluster_name values(responseStatus.code) values(userAgent) AS http_user_agent values(verb) values(requestURI) BY src_ip user.username user.groups{} 

Count the number of times the values shown occurred for each unique source IP address, user name, and user group.

| convert timeformat="%m/%d/%Y %H:%M:%S" ctime(lastTime)

| convert timeformat="%m/%d/%Y %H:%M:%S" ctime(firstTime)

Convert these times into readable strings.

Result

Not all unauthenticated requests are malicious, but the strings provided in this search provide context. Analyze the following to determine what needs further investigation:

  • Source IP address reputation, geolocation, and access policy
  • Banners in the user agent for scanning tools, such as Zgrab or Nmap
  • Targeted files, directories or command strings in the request URI field 
  • A high number of forbidden, unauthorized, or failure responses

For additional information about this search, such as its applicability to common frameworks and standards, see this project on GitHub.

  • Was this article helpful?