Most active Kubernetes service accounts

You might want information about activity levels of Kubernetes service accounts when doing the following:

• Monitoring Kubernetes sensitive role activities

Prerequisites 

In order to execute this procedure in your environment, you may need to first on-board the services or apps shown in the following table.

Example

The purpose of this example is to show how this procedure works in a general environment. In your environment, you can optimize the search by specifying an index, a time range, or a different data source. 

Service accounts are prime targets for attackers as they may present the opportunity to obtain credentials and move across all the resources they access. You want to investigate the most active Kubernetes service accounts for each pod to determine if their activities represent a threat. 

AWS

1. Ensure that your deployment is ingesting CloudWatch logs.
2. Run the following search: 

sourcetype="aws:cloudwatchlogs:eks" user.groups{}=system:serviceaccounts  objectRef.resource=pods 
|table sourceIPs{} user.username userAgent verb annotations.authorization.k8s.io/decision  
|top sourceIPs{} user.username verb annotations.authorization.k8s.io/decision 

Search Explanation

The table provides an explanation of what each part of this search achieves. You can adjust this query based on the specifics of your environment.

Azure

1. Ensure that you have configured Kube-Audit data diagnostics.
2. Run the following search: 

sourcetype=mscs:storage:blob:json category=kube-audit 
|spath input=properties.log 
|search user.groups{}=system:serviceaccounts* OR user.username=system.anonymous OR annotations.authorization.k8s.io/decision=allow  
|table sourceIPs{} user.username userAgent verb responseStatus.reason responseStatus.status properties.pod objectRef.namespace 
|top sourceIPs{} user.username verb responseStatus.status properties.pod objectRef.namespace 

Search Explanation

The table provides an explanation of what each part of this search achieves. You can adjust this query based on the specifics of your environment.

GCP

1. Ensure that your deployment is ingesting Pub/Sub messaging logs.
2. Run the following search:

sourcetype="google:gcp:pubsub:message"  data.protoPayload.request.spec.group{}=system:serviceaccounts 
|table src_ip src_user http_user_agent data.protoPayload.request.spec.nonResourceAttributes.verb data.labels.authorization.k8s.io/decision data.protoPayload.response.spec.resourceAttributes.resource 
|top src_ip src_user http_user_agent data.labels.authorization.k8s.io/decision data.protoPayload.response.spec.resourceAttributes.resource  

Search Explanation

The table provides an explanation of what each part of this search achieves. You can adjust this query based on the specifics of your environment.

Result

Not all service accounts interactions are malicious. Analysts must consider IP address, verb, and decision context when trying to detect maliciousness. Service accounts are usually created for specific tasks, so a number of unusual failures, deny statuses, or forbidden statuses may indicate suspicious activity. IP addresses must be restricted by security groups or allow/deny policies, unless a cluster API must be exposed. 

For additional information about this search, such as its applicability to common frameworks and standards, see this project on GitHub for AWS, Azure, or GCP.