Skip to main content
Splunk Lantern

Amazon EKS cluster scan detection

A Kubernetes cluster is a set of worker machines, called nodes, that run containerized applications. A cluster usually runs multiple nodes to provide fault-tolerance and high availability. The cluster master runs the Kubernetes API server, scheduler, and core resource controllers. One of your concerns with using Kubernetes is unauthenticated requests. You want to know if attackers are targeting your clusters, and, if so, to get suspicious request details, such as IP address, user agent, request URI, and response status data.

Procedure

  1. Ensure that you have configured your CloudWatch EKS log inputs.
  2. Run the following search. You can optimize it by specifying an index and adjusting the time range.  
sourcetype="aws:cloudwatchlogs:eks" "user.username"="system:anonymous" userAgent!="AWS Security Scanner" 
| rename sourceIPs{} AS src_ip 
| stats count min(_time) AS firstTime max(_time) AS lastTime values(responseStatus.reason) values(source) AS cluster_name values(responseStatus.code) values(userAgent) AS http_user_agent values(verb) values(requestURI) BY src_ip user.username user.groups{}
| convert timeformat="%m/%d/%Y %H:%M:%S" ctime(lastTime)
| convert timeformat="%m/%d/%Y %H:%M:%S" ctime(firstTime) 

Search explanation

The table provides an explanation of what each part of this search achieves. You can adjust this query based on the specifics of your environment.

Splunk Search Explanation

sourcetype="aws:cloudwatchlogs:eks"

Search only AWS EKS Kubernetes data.

"user.username"="system:anonymous" 

Search for anonymous users.

userAgent!="AWS Security Scanner"

Exclude requests from the AWS Security Scanner from the search.

| rename sourceIPs{} AS src_ip 

Rename the field as shown for Splunk processing capability.

| stats count min(_time) AS firstTime max(_time) AS lastTime values(responseStatus.reason) values(source) AS cluster_name values(responseStatus.code) values(userAgent) AS http_user_agent values(verb) values(requestURI) BY src_ip user.username user.groups{} 

Count the number of times the values shown occurred for each unique source IP address, user name, and user group.

| convert timeformat="%m/%d/%Y %H:%M:%S" ctime(lastTime)

| convert timeformat="%m/%d/%Y %H:%M:%S" ctime(firstTime)

Convert these times into readable strings.

Next steps

Not all unauthenticated requests are malicious, but the strings provided in this search provide context. Analyze the following to determine what needs further investigation:

  • Source IP address reputation, geolocation, and access policy
  • Banners in the user agent for scanning tools, such as Zgrab or Nmap
  • Targeted files, directories or command strings in the request URI field 
  • A high number of forbidden, unauthorized, or failure responses

For additional information about this search, such as its applicability to common frameworks and standards, see this project on GitHub.

Finally, you might be interested in other processes associated with the Detecting Kubernetes scanning activity use case.