Monitoring Kubernetes sensitive role activities – Splunk Lantern

Scenario: Kubernetes is the most used container orchestration platform. It contains sensitive roles that, if accessed by an attacker, can lead to cluster compromise. These roles can define permissions on namespaces or clusters, grant access to cluster-scoped resources and non-resource endpoints, and read secrets. You want to detect sensitive role usage within your Kubernetes clusters to verify that activity is legitimate.

How Splunk software can help

The Splunk Security Research team developed this use case to help you detect suspicious requests against Kubernetes sensitive role activities.

What you need

To succeed in implementing this use case, you need the following dependencies, resources, and information.

People

The best person to implement this use case is a threat hunter, system administrator, or security tools engineer who is familiar with operation of Kubernetes clusters and understands the authentication and authorization within cluster ecosystems. This person must understand the flow of data and be familiar with Kubernetes audit data. This person might come from your team, a Splunk partner, or Splunk OnDemand Services.

Time

Monitoring Kubernetes sensitive role activities using Splunk software can last from 24 hours to several weeks to permanently, depending on frequency of use and reliance on Kubernetes clusters in development or production.

Technologies

The following technologies, data, and integrations are useful in successfully implementing this use case:

Splunk Enterprise or Splunk Cloud
Common Information Model
Kubernetes for AWS, Azure, or GCP

How to use Splunk software for this use case

You can run many searches with Splunk software to monitor Kubernetes sensitive role activities. Depending on what information you have available, you might find it useful to identify some or all of the following:

Other steps you can take

To maximize their benefit, the how-to articles linked in the previous section likely need to tie into existing processes at your organization or become new standard processes. These processes commonly impact success with this use case:

Kubernetes best practices for securing your cluster

Related resources

If you have questions about this use case, see the Security Research team's support options on GitHub. In addition, these Splunk resources might help you understand and implement this use case:

Conf Talk: Effective strategies for monitoring Docker and Kubernetes environments
Conf Talk: Attacking and defending Kubernetes
Blog: Challenges in monitoring Kubernetes environments
Blog: Approaching Azure Kubernetes security
Tech Talk: Monitor and troubleshoot Kubernetes-based deployments

How to assess your results

Measuring impact and benefit is critical to assessing the value of monitoring Kubernetes sensitive role activities. The following are example metrics that you can track when implementing this use case:

Permission requests from unknown or unauthorized users
Sensitive role activities performed by unknown or unauthorized users
Top sources of IP addresses, users and user groups requesting or performing activities under sensitive roles