Skip to main content
Lantern Home

 

Splunk Lantern

Investigating security incidents across Amazon S3 and Snowflake with federated search

  1. Last updated
  2. Save as PDF

This article shows how federated search can provide 360-degree security visibility across a distributed data estate. By combining Splunk-indexed authentication data, long-term firewall and AWS telemetry in Amazon S3, and business context in Snowflake, analysts can investigate threats end to end without moving or duplicating data.

Three investigation examples in this series demonstrate how you can identify password spraying activity, trace attacker behavior, determine targeted identities, assess affected customers and financial exposure, and uncover ownership or compliance gaps. The result is faster threat hunting, richer incident context, reduced data movement, and more actionable response, all of which advance the Cisco Data Fabric vision.

Use the links below to jump straight to the investigation examples, or read on to find out more about their applicability, learn more about federated search, and find reference architecture and prerequisites for implementing the investigation examples in your own environment.

How federated search in Splunk Cloud Platform helps with this use case

Security evidence rarely sits in one place. Operational logs might remain in an index for only a few days. Cloud audit trails might be retained in Amazon S3 for months. The records that explain who an account belongs to, what data a customer owns, or business related KPI metrics data might live in Snowflake, where the security analyst does not normally have a warehouse login.

Federated Search for Snowflake (FS-Snowflake) lets analysts search Snowflake datasets directly from the Splunk Cloud Platform search experience using SPL2. When you combine it with Federated Search for Amazon S3 (FS-S3), you can correlate recent activity indexed in Splunk Cloud Platform with long-retention cloud telemetry and structured business context, without moving, duplicating, or re-ingesting the underlying data.

This series presents a reference investigation pattern, with three worked examples in separate articles. The examples use a small set of Linux authentication logs, AWS CloudTrail and VPC Flow Logs stored in Amazon S3, and Snowflake datasets containing HR, customer, export-manifest, and revenue information. Adapt the dataset names, field names, time ranges, and business data to your environment.

This pattern is a good fit when you:

  • Retain high-volume security, observability, or audit data in Amazon S3 or Snowflake.
  • Want to enrich data insights by correlating data distributed across multiple silos.
  • Need end-to-end security visibility across the entire data estate, including S3, Snowflake, Azure, DDSS, and other datastores with Federated Search support.
  • Want analysts to work from the Splunk Cloud Platform without granting direct warehouse access.
  • Need to enrich alerts with identity, asset, customer, financial, or compliance context.
  • Want to avoid full re-ingestion of historical data solely to make it searchable.
  • Need to return compact, aggregated evidence to the SOC, privacy, legal, or executive audience.
  • Have data owners who need to keep their existing Snowflake tables, views, governance, and retention policies.

By following this series, you'll learn how to:

  • Correlate a live identity signal in an index with 90 days of AWS history in Amazon S3 and ownership data in Snowflake.
  • Turn a successful password spray into a scoped breach report with named users, affected customers, financial exposure, and regulatory follow-up.
  • Use Snowflake as the authority for accountability, including cases where the absence of an owner record is itself the finding.
  • Push filtering, aggregation, and enrichment toward the system where the data lives so the Splunk platform receives a focused result set.
  • Design federated datasets and searches that support long-lookback threat hunting without re-ingesting an entire historical dataset into hot storage in the Splunk platform.

Why federated search matters

Federated search is useful when historical data is valuable but doesn't need to remain in hot storage. It can help you:

  • Avoid full re-ingestion or repeated rehydration of historical datasets.
  • Preserve Snowflake as the system of record for business and identity data.
  • Reduce context switching during incident response.
  • Limit the amount of sensitive customer or employee data returned to the search head by aggregating in Snowflake.
  • Extend investigations beyond the retention window of an index in Splunk Cloud Platform.

The exact cost and performance outcome depends on the connection, warehouse, table design, permissions, and query shape in your environment.

What Federated Search for Amazon S3 provides

Federated Search for Amazon S3 performs searches directly over datasets stored in Amazon S3 buckets and retrieves the results in Splunk Cloud Platform for correlation, enrichment, and analysis. This gives teams access to historical S3 data without the cost and complexity of restoring the data into Splunk Cloud Platform before they can investigate it.

FS-S3 uses two configuration objects:

  • An Amazon S3 connection, which securely authenticates the link between Splunk Cloud Platform and the federated dataset. You can reuse a connection across multiple datasets.
  • An Amazon S3 dataset, which identifies the searchable data at an Amazon S3 location and is associated with one connection.

For federated searching, each S3 dataset must have a data catalog that defines the dataset's columns, schema, and partitions. Depending on your design, the dataset can use an AWS Glue Data Catalog table, an Apache Iceberg REST catalog, or a Splunk-native catalog. The examples in this series use the static dataset based on JSON schema.

FS-S3 is the historical telemetry path used in these examples. It supplies the CloudTrail and VPC Flow Logs that are no longer in the short-retention index in Splunk Cloud Platform. FS-Snowflake supplies the structured identity, customer, revenue, and ownership context used to interpret those events.

What Federated Search for Snowflake provides

Federated Search for Snowflake provides a unified search experience in Splunk Cloud Platform for Snowflake-hosted datasets. After an administrator configures the connection and registers the datasets, users can query those datasets with SPL2 and correlate the results with data indexed in Splunk Cloud Platform and S3-federated data.

FS-Snowflake provides the following capabilities: 

  • A guided Snowflake connection and dataset configuration experience in the Data Management app.
  • Snowflake account authentication and warehouse selection.
  • Search from the search bar in Splunk Cloud Platform using SPL2.
  • Cross-source enrichment between indexed data and Snowflake tables or views.
  • Query translation and pushdown of filters, aggregations, and other operations where appropriate, which helps reduce unnecessary data transfer.
  • Improved SPL2 execution behavior, error handling, and reliability for federated searches.

This means security, observability, compliance, and business teams can keep their existing data architecture while using the Splunk platform as the investigation and decision-making surface.

Using federated search during investigations

During an investigation, the first question is often easy to ask but hard to answer: what happened? The harder questions are usually who was responsible, what was affected, and what should you do next.

Those questions cross data boundaries. A host authentication event can identify a username and source address, but not whether the account belongs to an analyst or a privileged administrator. CloudTrail can show that the identity created an access key, but not whether that identity is owned, active, or expected to perform that action. A customer export manifest can identify the records behind an object, but not the balance, revenue, geography, or regulatory scope associated with those records.

Without federation, analysts commonly have to switch tools, request access to a warehouse, build a temporary data pipeline, or wait for data to be rehydrated into Splunk Cloud Platform. Those steps add delay and might create additional copies of sensitive data.

FS-Snowflake addresses this gap by making Snowflake data searchable from Splunk Cloud Platform while Snowflake remains the system of record. Combined with FS-S3, analysts can use one SPL2 workflow to move from a recent signal to historical evidence and business impact.

By using FS-Snowflake and FS-S3 in Splunk Cloud Platform, organizations can turn distributed data into a single investigation workflow:

  • Security teams get historical and business context without a second analyst login.
  • Incident commanders get quantified scope instead of a list of suspicious events.
  • Privacy and legal teams get country-level and financial summaries while sensitive row-level data stays governed in Snowflake.
  • Cloud and platform teams can identify unowned identities and unexplained infrastructure across a longer lookback.
  • Data owners preserve Snowflake as the system of record and avoid unnecessary replication in Splunk Cloud Platform.

Reference architecture

This series uses two federated data paths and one native index.

Layer Example data Role in the investigation
Splunk Cloud Platform linux_secure Recent SSH and sudo activity; the starting point for live signals.
Amazon S3 s3_cloudtrail 90 days of AWS API activity, including IAM and EC2 actions.
Amazon S3 s3_vpc_flowlogs 90 days of network-flow activity, including rejected scans and mining-pool traffic.
Snowflake sf_employee_status Account ownership, department, role, clearance, and employment status.
Snowflake sf_export_manifest The customer records contained in each exported S3 object.
Snowflake sf_customer Customer country, segment, account balance, and customer key.
Snowflake sf_customer_revenue Annual revenue and order count per customer.

The physical locations remain separate. The Splunk Cloud Platform supplies the search surface, Amazon S3 supplies long-retention telemetry, and Snowflake supplies structured context and business authority.

The examples in this series assume the following dataset names and fields. Use the exact identifiers and casing from your Snowflake environment.

Dataset Example fields Purpose
linux_secure _time, host, _raw Seven days of Linux SSH and sudo activity in an index in the Splunk platform.
s3_cloudtrail event_time, userIdentity, sourceIPAddress, eventName, eventSource, errorCode, awsRegion, objectKey, bucketName Cloud API activity and S3 object access retained in Amazon S3.
s3_vpc_flowlogs flow_start, srcaddr, dstaddr, srcport, dstport, action, bytes Network connection attempts and accepted flows retained in Amazon S3.
sf_employee_status OS_ACCOUNT, FULL_NAME, DEPARTMENT, ROLE, CLEARANCE_LEVEL, EMPLOYMENT_STATUS Identity and employment authority in Snowflake.
sf_export_manifest OBJECT_KEY, C_CUSTKEY Mapping between exported objects and customer records.
sf_customer C_CUSTKEY, COUNTRY, C_MKTSEGMENT, C_ACCTBAL Customer and account context.
sf_customer_revenue C_CUSTKEY, ANNUAL_REVENUE, ORDER_COUNT Business-impact context.

Prerequisites

Complete the following preparation steps:

  1. Configure a Snowflake connection in the Data Management app. Follow the Snowflake connection documentation and the Federated Search for Amazon S3 overview for the S3 datasets.
  2. Register each Snowflake table or view as a federated dataset.
  3. Supply the Snowflake programmatic access token (PAT) directly in the connection form.
  4. Confirm that the Snowflake network policy allows the IP addresses for the AWS region of the Splunk Cloud Platform stack. The allow list is keyed to the AWS region of the Splunk Cloud Platform stack, not the Snowflake account's region. A mismatch can surface as Snowflake error 390422 INCOMING_REQUEST_BLOCKED, reported by Splunk Cloud Platform as an authorization failure.
  5. Declare the time fields for the S3 datasets: event_time and flow_start, both using %Y-%m-%dT%H:%M:%SZ. Leave the time field unchecked for the Snowflake datasets in this example.
  6. Switch the search bar in Splunk Cloud Platform to SPL2.
  7. Set the time picker wide enough to cover the investigation. The time picker applies to the two S3 datasets that declare a time field, and is intersected with any earliest constraint in the search. The Snowflake datasets don't use the time picker in this example.
  8. Confirm that the analyst's role includes the federated datasets.

For the FS-S3 portion of these examples, use supported file formats such as CSV, JSON, or Parquet and confirm that the S3 dataset is backed by a supported catalog. FS-S3 is available for Splunk Cloud Platform deployments in AWS regions; confirm the deployment and data governance requirements for your environment.

Use role-based access controls to expose only the Snowflake fields and federated datasets each audience needs. Customer, revenue, HR, and clearance fields in particular might require separate access policies.

Investigation examples in this series

The following three investigations are independent examples, but they share the same operating model: start with a search query in the Splunk Cloud Platform, extend the lookback into Amazon S3 when the relevant history is no longer in the index, enrich the result with authoritative identity, customer, ownership, or revenue data in Snowflake, aggregate close to the source whenever possible, and end with an action such as contain an account, scope an incident, notify the right stakeholders, or remediate an ownership gap. Each example builds on the reference architecture and prerequisites in this article.

What these investigations demonstrate

Investigation Splunk platform provides Amazon S3 provides Snowflake provides Decision enabled
Identity takeover Recent host login and refused sudo actions 90 days of IAM activity from the same source Human owner, role, clearance, and employment status Contain a likely takeover and assess privilege mismatch.
Successful password spray Authentication outcomes and target accounts Network sweep and object-access history Employee context, export membership, customer impact, and revenue Scope a breach and route privacy, legal, and business response.
Unowned compute Change-run evidence Provisioning history and mining-pool traffic Identity ownership authority Shut down suspicious infrastructure and remediate accountability gaps.

The common outcome is not just a broader search. It's a more useful decision: which identity to contain, which records to notify on, which infrastructure to isolate, and which control to fix so the same gap doesn't recur.

Applying the patterns shown in this series

Query-design guidance for federated searches

Use these practices when adapting the examples:

  • Filter early on high-signal fields such as identity, source address, event name, object key, date, or region.
  • Aggregate inside Snowflake or Amazon S3 before joining when the underlying dataset is large.
  • Return only fields needed for the current decision, especially when data includes customer or employee information.
  • Use a bounded lookback such as earliest=-90d for historical pivots; avoid scanning an entire warehouse table without a business question.
  • Align filters with the remote table's physical design. Date, region, application, action, and severity are often useful partition or clustering dimensions.
  • Check identifier casing. Snowflake object names can be case-sensitive, especially when quoted identifiers are used.
  • Treat the time picker and earliest constraints deliberately. In these examples, S3 datasets use declared time fields while Snowflake datasets are queried without a time field.
  • Validate the federated dataset with a small sample before building a complex join.

Recommended adoption plan

  • Choose one investigation: start with an alert or recurring question that requires identity, customer, or historical context.
  • Register the minimum datasets: onboard only the S3 and Snowflake datasets needed to answer that question.
  • Validate access and schema: run a small sample search and confirm field names, casing, time behavior, and role-based access.
  • Build a decision-oriented search: filter and aggregate before joining; return the fields needed to take action.
  • Operationalize the outcome: convert the investigation into a saved search, dashboard panel, detection, audit report, or response runbook.
  • Measure the result: track time to scope, number of tool switches, data movement avoided, and the percentage of identities or assets with authoritative ownership.

Next steps

Contact your Splunk account team for availability, enablement, and configuration guidance for Splunk Cloud Platform 10.5.x in AWS Commercial regions.

Continue with the three investigation examples in this series, or explore the following resources for implementing the guidance in this article:

  • Written by Nithin Krishna Reghunathan (Staff Engineering Product Manager) and Nikita Voitov (Partner Solutions Engineer)
  • Splunk, A Cisco Company