Skip to main content
Lantern Home

 

Splunk Lantern

Integrating Google Kubernetes Engine with Splunk Observability Cloud

  1. Last updated
  2. Save as PDF

 

You might have learned how to integrate Kubernetes environments with Splunk Observability Cloud, but what about integrating cloud-managed Kubernetes platforms? Let's examine how to do this with Google Kubernetes Engine (GKE).

If you want a primer on using Kubernetes Navigators in Splunk Observability Cloud before reading this article, go here first: Detect and resolve issues in a Kubernetes environment.

Why integrate GKE with Splunk Observability Cloud?

GKE dashboards provide observability metrics around infrastructure and application health for clusters and workloads from within the Google Cloud Platform (GCP) itself. So why should we integrate GKE with an observability backend?

Typically, not every piece of application infrastructure lives in the same cloud platform. In the middle of an incident when seconds matter, no one wants to navigate between platforms and remember where each discrete observability tool lives.

Instead, having everything in one unified observability platform reduces toil and time to incident resolution. With end-to-end observability all in one place, root cause analysis becomes a lot easier thanks to the ability to correlate issues impacting multiple parts of the stack. Splunk Observability Cloud provides advanced, flexible, and configurable dashboards, charts, and detectors along with support for application, service, and incident management integrations. Plus, the OpenTelemetry-native platform allows you to avoid vendor lock-in.

Observability metrics in GKE

You can monitor your applications running in Google Kubernetes Engine from GCP itself by configuring the Kubernetes Engine Monitoring setting on your cluster to System and workload logging and monitoring. You can click into your deployed workload to see the CPU, memory, and disk utilization.

clipboard_e40695647add9225f4ebfd06017a5c88b.png

Examine the observability details around the health of your application, like container errors and restarts.

clipboard_e777db6e2812cb1910ee2f1680fadf657.png

You can also go into Google Observability Monitoring and further explore GKE cluster performance.

clipboard_e72f29165cd4b0c95a1ade7964e7e64bb.png

And you can build custom dashboards and charts.

clipboard_e008f6846ee2a20caafc4197589c76f71.png

These things are all useful, but again, to unify your observability tools and improve the time to incident resolution, how can you get all of this information into a single backend observability platform?

How to integrate GKE and Splunk Observability Cloud

There are two different ways you can integrate GKE into Splunk Observability Cloud:

Follow the instructions below for the integration method that you choose.

Integrate GKE with additional GCP service integration

These steps follow the Connect to Google Cloud Platform documentation.

  1. Open the GCP UI, select the project you want to monitor, click IAM & Admin, click Service Accounts, and complete the fields to add a new Splunk Service Account.

    clipboard_eca1df583a850561722a76ba88c9be368.png

  2. After the new service account is created, edit it to create a new service account key.

    clipboard_ebc461807e8e6697ac35fc3db571881b2.png

  3. Download the key as a JSON.

    clipboard_e211932a672cac75e5a2c29725779e2bb.png

    The GCP Cloud Resource Manager API must be activated so Splunk Infrastructure Monitoring can use it to validate permissions on the service account key. Also, if you want to monitor multiple GCP projects, the above steps need to be repeated for each one.

  4. Navigate to Splunk Observability Cloud and add the Google Cloud Platform integration.

    clipboard_e5598b37615cd0b1e3f0e34352b95aad4.png

  5. Import the newly created service account key.

    clipboard_ed626ad81ccdfddc75ec147dea6571bcc.png

  6. A benefit of integrating GCP in this way is that you can sync all supported GCP services automatically. However, for our purposes, we’ll optionally refine the GCP synced services to only include GKE.

    clipboard_e05184b66691ff33e980d198fea4122da.png

  7. Now you'll see Google Cloud Platform under your actively deployed integrations.

    clipboard_ee902f748d60c1b528b52bd8da4621c7d.png

Depending on the polling interval, it might take a few minutes for your GCP metric data to populate within Splunk Observability Cloud.

Integrate GKE via Helm chart

  1. In the Data Management tab, search for available integrations for Google Kubernetes Engine and select the Google Kubernetes Engine integration to view the guided Kubernetes Monitoring integration instructions.

    clipboard_e21b5596dbcdfea95bb7ec766035bbdae.png

  2. Specify Google Cloud Platform as the provider and Google GKE as the distribution.

    clipboard_e10fab22362d19db9448a55a0a64864d1.png

  3. Follow the installation instructions.

    clipboard_e78c5f0717963fa7720d8fd6008264f52.png

  4. In the GKE UI, activate a cloud shell, and then enter the commands shown in the installation instructions within the instance.

    clipboard_e6562b3f2f91a0da9003927449bbb5d72.png

  5. You’ll notice that the helm-install splunk-otel-collector command failed because of a missing instrumentation.endpoint value. This is because the version of helm pre-installed in the Google Cloud Shell is out of date. To resolve the error, upgrade the helm version to one supported for the current Kubernetes version and rerun the commands in the Splunk Observability Cloud installation instructions.

    clipboard_e99870416e956cee924e529bdf318b2b2.png

  6. After this completes successfully, go back to Splunk Observability Cloud and see the integrated telemetry data.

    clipboard_e47886b581504146abf3ea37bde4efae0.png

From here, you can explore data like CPU and memory usage/utilization, resource capacity, container restarts, node state, and others right alongside the rest of your application and infrastructure data within Splunk Observability Cloud, even if other parts of the app run on-premises or on another public cloud provider.

Viewing observability metrics in Splunk Observability Cloud

Now that your GKE integration is complete, you can use all of the available Splunk Observability Cloud products and features to monitor your GKE environment.

From Infrastructure Monitoring, you can view your Google Cloud Platform navigators.

clipboard_e1456832a9fb1843da4cfe99fd08cb17c.png

Explore the health of your clusters.

clipboard_e8ecb05cd7441875bacbf62330759303f.png

Explore a specific cluster.

clipboard_e3f8c9fd57179a33d5d2e06d76e4aeed4.png

Monitor pod health and performance.

clipboard_e5ab735f5bf3e1698de6340622d9525b1.png

And view critical usage metrics.

clipboard_ed6a589ffe6c38c76ef359a1e644ff1a8.png

You can also create detectors and alerting rules from within your Navigators and manage them alongside the detectors for the rest of your applications and infrastructure:

clipboard_eff79b457fffd79263ac516508c91eb83.png