Skip to main content
Do you build apps on Splunk or are a Splunk admin? If so, we want to hear from you. Help shape the future of Splunk and win a $35 gift card!

Take the Survey
Lantern Home
 
 
Splunk Lantern

Understanding workload pricing in Splunk Cloud Platform

  1. Last updated
  2. Save as PDF

 

A critical challenge that many data practitioners and IT administrators face today is that while some of their data is of extremely high value, they often do not know the value of the rest of their data until they actually need it. Along with varying values of data, it is also difficult to predict when high volumes of data may be needed, as business requirements can change at any given moment. Finally, in many use cases, the value of the platform is driven more by search rather than the creation of a data repository.

Instead of being metered on data you ingest into the Splunk Cloud Platform, an optimal pricing approach should align with the value you generate. Such an approach gives you the freedom to put more data into the Splunk Cloud Platform without limiting your data volume and to use that data when it actually adds value to your organization.

Solution: Workload pricing

Workloads are activities in Splunk Cloud Platform such as searching, investigating, monitoring, machine learning, data streaming, data indexing, and data processing that require compute resources. Workload Pricing aligns what you pay with the types of workloads running on Splunk Cloud Platform. This pricing model makes it economical to bring more of your less-frequently-searched data into Splunk Cloud Platform. You gain visibility into your workload utilization, and you are charged based on your anticipated workloads rather than the data volume ingested. While you will still purchase a fixed capacity of workloads, this model allows you to choose whether you want to optimize for ingest or search, depending on how your needs fluctuate during your business cycle or even during a given day.

The following video provides a quick overview of how workload pricing works.

Splunk Virtual Compute

Splunk Cloud Platform workloads are measured with Splunk Virtual Compute (SVCs) units. An SVC is a unit of compute and related resources that provides a consistent level of search and ingest equal to the SVC benchmark. This Splunk-created unit ensures that SVCs continue to provide the same or better levels of performance as underlying infrastructure or software configurations evolve. Splunk Cloud Platform captures SVC utilization measurements for each machine every few seconds. You can view your hourly SVC utilization anytime in the Cloud Monitoring Console. The total number of SVCs you need is the maximum compute resources anticipated for your peak demands. Thus, this model involves the purchase of a fixed capacity of SVCs.

Workload Management

Because the Workload Pricing model puts you in control, you can drive more value out of the Splunk Cloud Platform by optimizing your system and usage patterns. This is enabled by Workload Management, a rich set of capabilities that provide fine-grained, rules-based control of resource prioritization. This rule-based framework lets you set rules that automatically manage your system resources in the following ways:

  • You can prioritize critical searches and manage workloads during peak/off-peak times using schedule-based rules.
  • You can place searches in different pools and also provide granular access controls to certain users, so they have the ability to choose their own workload pools.
  • You can track utilization and fine-tune the resource allocations through rich monitoring capabilities.

For more information on workloads, see How workload management works.

Cloud Monitoring Console

Visibility of resource utilization also comes in the form of Splunk's Cloud Monitoring Console (CMC), which provides full visibility into resource utilization, as well as detailed information on charge-back needs for large enterprises. The CMC includes prebuilt views of both search and ingest health. Every item represented in the CMC is an aspect of Splunk Cloud Platform that you can control. The License Usage > Workload dashboard shows you all the following metrics:

  • Peak SVC usage within selected time granularity (1-hour, 15-min, 5-min)
  • Peak SVC usage as a % of provisioned SVC by tier (search head and indexer) within selected time granularity (1-hour, 15-min, 5-min)
  • Peak SVC usage per hour split by consumer (ingestion, search, and shared services)
  • (Search seconds, SVC usage)* per hour by search type
  • (Search Seconds, SVC usage)* per hour by top 10 (apps, searches, users)*
  • Dispatched and skipped search count per hour
  • Peak SVC usage per hour by ingestion source (index, sourcetype)*
  • Hourly rate of ingestion

*These are variable selections that can be changed within the panel.

For more information, see Introduction to the Cloud Monitoring Console.

Implement workload pricing in your Splunk environment

To get started:

  1. Ask yourself these questions to decide whether workload pricing is right for you:
    • Do you have medium and low value data that is not searched as frequently as your high value data?
    • Would you like more flexibility and control in determining how your license capacity is used between indexing and search?
    • Do you have the time, resources, and infrastructure to manage workloads so that you can get the best value?
  2. If it is the right solution, purchase Splunk Cloud Platform based on Workload Pricing.
    • If you are an existing customer, work with your sales team to appropriately size the number of SVCs.
    • If you are a new customer, see the following section: Estimate your workloads.
  3. Choose your storage option. Storage blocks are the number of terabytes of storage required to meet your data retention policies. It is driven by both the volume and duration for which storage is needed. For example, customers that expect to add a high volume of data will need more storage. In addition, you have the option to differentiate your storage between active searchable storage and archive storage - to retain data for option value in case you may need to search at a later date. You can subscribe to storage upfront based on estimates, then true-up annually to account for the variability of ingest.
  4. Optionally, add Premium Solutions Licenses (Premium Apps) a-la carte.

Estimate your workloads

You can use Splunk’s sizing calculator to calculate the number of SVCs you will need based on how efficiently you believe you can operate Splunk Cloud Platform.

The following table provides some common estimates. The stated volume is not guaranteed. Talk to your sales representative for assistance with your unique environment.

Workload Type/Data Use Case Description GB/day per SVC for each Use Case
Compliance Storage Compliance data is written once and almost never searched. This data is stored for compliance and retention reasons only. 35-45+
Data Lake
(Exploration / Use Case Development)		 Data with unknown/unrealized perceived value. This data is typically indexed and forgotten or very rarely used, and searches against this data are not expected to be highly performant. 25-35+
Basic Reporting This data is used for fixed scheduled reporting and/or view only dashboards. This data is infrequently searched or utilized. 20-30
Ad-hoc Investigation Data with few fields or used for ad-hoc searching. Low touch data is typically searched a few times a day or more and is used in interactive investigations. 15-25

Continuous Monitoring

 High value data is typically used proactively in live or near real time background searches. This data is typically extremely high value and used often for security, IT, and business operation intelligence. 10-20
Premium Solution - ES or ITSI
(Low workload)		 Splunk Premium Solutions provide continuous monitoring and investigation capabilities to improve security risk posture and maintain business service availability and reliability. These premium applications use the most system resources. 10-15
Premium Solution - ES or ITSI
(High workload)		 Splunk Premium Solutions provide continuous monitoring and investigation capabilities to improve security risk posture and maintain business service availability and reliability. These premium applications use the most system resources. 5-10

Become more efficient with SVC utilization

You can start preparing now to benefit from workload pricing by using the following tips to become more efficient with SVC utilization.

Improve search profiles:

  • Search frequency. Review how often searches are running
  • Search density. Review how many data sources and how wide a time range your searches run against

Improve SVC utilization of searches:

  • Review unscoped data models and all-time searches
  • Review skipped searches and adjust the frequency or scheduling
  • Ensure that scheduled searches are evenly distributed and not skewed
  • Review long time running searches and optimize the SPL
  • Disable unused scheduled searches
  • Remove unused apps and technology add-ons

Free up capacity:

  • Enforce search best practices. Splunk allows admins to “block” bad searches from executing. Bad searches increase concurrent search loads and require more compute.
  • Spread-out scheduled searches. One of the easiest methods to reduce concurrent searches is to spread them out.
  • Focus on summary indexes. Searches against summary indexes are up to 100x faster than similar ad-hoc searches. Up-front administration planning can enable functional, basic dashboards.

If you're already using workload pricing, see Monitor current SVC usage of your workload-based subscription for more help optimizing your workloads.