You have configured the Google Cloud Platform integration to connect your GCP account to Splunk Observability Cloud, but you are still experiencing some problems with your data.
Where are your metrics?
Check for GCP API call throttling
GCP has limits on the number of API calls that can be made against its endpoints. Plot the sf.org.num.gcpStackdriverClientCallCountThrottles metric in a new chart. If data appears, click Data Table beneath the chart to get details. If you are being throttled, work with GCP to increase your API limits.
Check for metric time series creation throttling
Splunk limits the number of metric time series you can create, which is 6,000 per minute or more, based on your subscription. This throttle is a funnel. New metric time series will eventually be created. You can do any of the following to understand your limits:
- Plot the sf.org.limit.metricTimeSeriesCreatedPerMinute metric to see your limit.
- Plot the sf.org.numMetricTimeSeriesCreated metric to see the number of MTS created.
- Plot the sf.org.numThrottledMetricTimeSeriesCreateCallsByToken metric to see number of creations throttled.
Check your active metric time series limits
Splunk limits the number of active metric time series you can have, based on your subscription. New metric time series will not be created until you are under your limit. You can do any of the following to understand your limits:
- Plot the sf.org.limit.activeTimeSeries metric to see your limit.
- Plot the sf.org.numActiveTimeSeries metric to see your number of active MTS.
- Plot the sf.org.numLimitedMetricTimeSeriesCreateCalls metric to see whether new MTS creations are limited.
These additional Splunk resources might help you understand and implement these recommendations:
- Splunk Docs: Connect to GCP