Using AI to auto-schematize raw log data into CIM-compliant add-ons

Prerequisites

• Splunk Cloud Platform access. Guided Onboarding with Auto-schema is available on Splunk Cloud Platform. The Data Management app it sits in comes preinstalled, so there is nothing extra to set up before you start.

Data required

• A sample of raw log events. Provide a plain text file (up to 30 MB) containing events from the source you want to onboard. The more representative your sample, with enough variety to cover the event types you care about, the better the grouping and field-mapping results.
• Know where the data will land. Know whether you need the data schematized at search time in Splunk Cloud Platform, or at ingest time for an external data lake. Auto-schematization asks this upfront because the answer determines what kind of artifact it generates: a TA for search-time schema-on-read or an SPL2 pipeline for ingest-time schema-on-write.
• (Optional) A source type or source name. For example, cisco:asa:syslog or source::/var/log/app.log. If you leave this blank, the workflow suggests one based on your data. In that case, configure your data input to use the suggested source type after the add-on is generated.

You do not need to know in advance which CIM data model your events map to. That's one of the things the workflow determines for you.

How to use Splunk software for this use case

The first time you open Guided Onboarding with Auto-schema, you'll see a one-time AI consent prompt. It explains that Auto-schema uses large language models and that Splunk might process data relevant to the workflow, like your sample log data and onboarding answers. To continue, review and acknowledge the prompt, which includes agreeing to the Splunk General Terms and Data Security & Privacy Commitment.

1. Answer the general questions

   Start a new scenario from Onboard data > Schematize custom data. On the General questions step, answer the following:

   • Where the data is going: Splunk Cloud Platform search-time schema-on-read or an external data lake's ingest-time schema-on-write.
   • The source type or source you'd like to use.
   • A name for the scenario so you can find it again later.

   These answers shape what the workflow generates, so it is worth getting the destination right before you proceed. After filling in the details, click Next.

   

2. Add your sample data

   On the Add sample data step, paste or upload the raw events for the source you're onboarding. Auto-schema accepts .log, .txt, .json, .csv, and .xml files. If your file is large, only the first 30,000 characters appear as a preview, but the entire file is uploaded and processed.

   Before moving on, confirm that your sample covers every event type you care about. The grouping and mapping steps that follow can only work with what you provide here.

   

3. Review the AI event grouping

   Moving to Select data groups starts the first AI processing stage. You can expand the live reasoning trace to watch it work: it splits your raw sample into individual events, validates the line-breaking rule it inferred, clusters similar events into distinct groups, and filters down to the groups that are CIM-relevant. When reasoning completes, the workflow presents the identified groups and asks you to select which ones you want to proceed with.

   

4. Select add-ons or data models for each group

   On Select add-ons/data models, the workflow presents each data group alongside the CIM data models it thinks fit best. For example, an audit log's network-change events might match the CIM Change.Network_Changes dataset. For each candidate you can see how many of its required fields were detected in your data and drill into matched versus missing fields with an events preview. Select the best-fitting model per group.

   If a group's data matches an already-published, Splunk-supported add-on—for example, Palo Alto Networks—that add-on is surfaced right in the group selection, with a link to its Splunkbase page. Using the existing supported add-on is the recommended approach whenever it meets your onboarding requirements, rather than generating a new one. If you install both a generated add-on and an existing supported add-on for the same data source, standard Splunk add-on precedence rules apply and the two can conflict, affecting source type handling, field extractions, and CIM normalization.

   If none of the suggested CIM models fit a group, or if you only need meaningful fields extracted without CIM alignment, choose Custom schema. This path derives a schema from the event's own structure and generates field extractions for it, but skips CIM mapping entirely.

   

5. Refine the generated field mappings

   The Refine step is where the workflow generates the actual field mappings, that is, your raw fields matched to CIM field names. Expand the reasoning trace to see it decide on extraction settings (for example, native JSON parsing through KV_MODE=json when your sample data is JSON) before adding the FIELDALIAS and EVAL logic needed for CIM semantics.

   

   After mapping completes, a sample data preview shows your events with the mapped fields laid out side by side, flagged as required, recommended, or missing, so you can spot problems at a glance.

   If something needs adjusting, the Auto-schema assistant panel lets you request changes in plain language. For example, you could ask it to derive dest_port from the host field when present and fall back to a default value otherwise. To do this, select the assistant icon in the top right to open the chat experience. The assistant updates the mapping and refreshes the sample preview so you can verify the fix.

   You cannot edit generated regular expressions directly in the UI. Ask the Auto-schema assistant to update them for you instead.

   

6. Review and download the generated add-on

   The final Review artifacts step shows the complete generated Technology Add-on: props.conf with the line-breaking, timestamp, and FIELDALIAS/EVAL stanzas for each source type, plus transforms.conf, eventtypes.conf, tags.conf, default.meta, app.conf, and reference documentation (README.md, known_issues.md). Review the configuration inline, then download the full add-on package.

   

   Any mapping corrections you requested in the Refine step are already reflected in the artifacts. If you need further changes after artifacts are generated, ask the Auto-schema assistant to update and regenerate them, as you cannot edit the files directly in the UI at this stage.

   After you click Done, the scenario appears as completed in your Scenarios list, alongside the source types it produced. This is useful if you're onboarding several sources and want to track them all in one place.

   

Next steps

After your add-on is installed, verify that the schematized data lands as expected by running a CIM-compliant search against the new source type, for example, a tstats search against the data model you mapped to. If you're onboarding several related sources, repeat this workflow for each source. Each scenario is scoped independently, so mappings for one source won't affect another. Each Splunk Cloud Platform environment can start up to 400 Auto-schema scenarios per month, so plan larger rollouts accordingly.

Now that you know how to use Auto-schema to generate CIM-compliant Technology Add-ons from raw log samples, you can try the process yourself with the interactive Schematize Custom Data walkthrough, which steps through this same flow from onboarding questions and sample events to recommended CIM mappings and generated deployment artifacts.

You might also find the following resources useful:

• Splunk Lantern Article: Normalizing values to a common field name with the Common Information Model
• Splunk Lantern Article: Configuring new source types
• Splunk Lantern Article: Onboarding useable and purposeful security data
• Splunk Lantern Article: Leveraging LLM reasoning and ML capabilities for Jira alert investigations
• Splunk Lantern Article: Leveraging Splunk MCP and AI for enhanced IT operations and security investigations
• Splunk Help: Overview of the Splunk Common Information Model
• Splunk Help: Installing Splunk add-ons
• Splunkbase: Splunk Common Information Model (CIM)

• Splunk OnDemand Services: Use these credit-based services for direct access to Splunk technical consultants with a variety of technical services from a pre-defined catalog. Most customers have OnDemand Services per their Success Plan. Engage the ODS team at ondemand@cisco.com if you would like assistance.