Managing Splunk Cloud Platform knowledge objects

Knowledge objects in the Splunk platform are the building blocks on which collected data transforms to actionable information. Knowledge objects can be anything from field extractions, tags, and event types to saved searches, reports, dashboards, and alerts. These objects help in interpreting and visualizing raw data, turning it into meaningful insights.

Knowledge objects can be delivered via uploaded apps or or users can add and update them through the GUI. The Splunk best practice recommendation is to use apps. There are a number of reasons for this:

• change control with layered environments (you can transfer objects among your Dev, Test, and Prod environments)
• quality, predictability, and consistency among your Splunk implementations
• establishment of a baseline, which enables restoration of knowledge objects should anything happen to your deployment

Protection against accidental deletions is possibly the more useful reason to use apps. When you delivery knowledge objects through apps, you have a repository for backup, in case anything happens to your Splunk Cloud Platform environment. For example, if an employee decides to delete an important dashboard, you can restore it quickly, copying the JSON or XML. Though any local customizations or changes might be lost (as they are not part of the repository app), the default configuration will always exist in the app. Although app-based delivery requires control and change management, it might be a preferable often to simply hoping accidental deletions never happen.

In Splunk Cloud Platform, apps are managed through the GUI or through the Admin Config Service (ACS). There are safeguards in place that help make up for the lack of a controlled app repository that Splunk Enterprise customers might use.

• Only apps that have passed AppInspect can be uploaded.
• Default app content cannot be changed in the GUI.
• Enhanced protection against unintended deletions, since there is no command line access with Splunk Cloud Platform

The challenges with using apps in Splunk Cloud Platform are the following:

• To find out what is in an app or to update it, you can only see the default information without going through Cloud Support for additional assistance.
• Alert actions, custom commands, and custom inputs in an app sometimes prompt a manual inspection from AppInspect because it doesn't know how to really get into scripts. This can add a few days to the AppInspect approval process.
• Individual users can override the baseline. They can clone or change a dashboard, and then use their own local version. Then, if the central Dev team updates the dashboard, the user might never see it.
• If you have the Victoria experience, apps get installed on all search heads, instead of select ones. This means that you should disable scheduled content; otherwise, it runs on all search heads. Then you have to go through the process of re-enabling scheduled content on the specific servers or search heads that you want it to run on.
• Managing roles in Splunk Cloud Platform via app is difficult as any change to a role using the GUI will update system/local/authorize.conf.