Skip to main content
Lantern Home

 

Splunk Lantern

AWS EBS volumes without a current snapshot

  1. Last updated
  2. Save as PDF

 

Snapshots are backups of an EBS volume that you can use to recover from problems or data loss. While it might be desirable to not take a snapshot of a volume, you should evaluate any volume without a recent snapshot because it might be at risk for data loss. You want a search to help you do that.  

Data required 

AWS description data

Procedure

  1. Configure the Splunk Add-on for Amazon Web Services.
  2. Ensure that your deployment is ingesting AWS data through one of the following methods:
    • Pulling the data from Splunk via AWS APIs. At small scale, pull via the AWS APIs will work fine.
    • Pushing the data from AWS into Splunk via Lambda/Firehose to Splunk HTTP event collector. As the size and scale of either your AWS accounts or the amount of data to be collected grows, pushing data from AWS into Splunk is the easier and more scalable method.
  3. Run the following search. You can optimize it by specifying an index and adjusting the time range.
sourcetype="aws:description" region="*" source="*:ec2_volumes" earliest=-1h 
|dedup id sortby -_time 
|rename "attach_data.instance_id" AS instanceId 
|fields account_id, id, region, instanceId, size, status, type 
|join type=left id 
    [ search index=* sourcetype="aws:description" region="*" source="*:ebs_snapshots" 
    |dedup id sortby -_time 
    |rename id AS snapshotId, status AS snapshotStatus 
    |rename volume_id AS id 
    |fields id, snapshotId, snapshotStatus, start_time] 
|eval snapTime=strptime(start_time,"%Y-%m-%dT%T") 
|eval diff=round(((now() - snapTime) / 86400),0) 
|eval insight=case((NOT (diff>0 AND diff<30)),"No Recent Snapshot")
|search insight!=null 
| table account_id id region size type status insight

Search explanation

The table provides an explanation of what each part of this search achieves. You can adjust this query based on the specifics of your environment.

Splunk Search Explanation

sourcetype="aws:description" region="*" source="*:ec2_volumes" 

Search only your EC2 volumes and filter by description data for all regions.

earliest=-1h

Search in the last one hour.

|dedup id sortby -_time 

Remove duplicate instances of ID and sort the remaining results with the most recent instances first. 

|rename "attach_data.instance_id" AS instanceId 

Rename the field as shown for better readability.

|fields account_id, id, region, instanceId, size, status, type

Return only the fields shown.

|join type=left id

Join the results from the main search of ec2 volumes found with the secondary search of the ebs snapshots found (in the next line) where the id (the join key) matches in both searches. 

[ search sourcetype="aws:description" region="*" source="*:ebs_snapshots" 

|dedup id sortby -_time 

|rename id AS snapshotId, status AS snapshotStatus 

|rename volume_id AS id 

|fields id, snapshotId, snapshotStatus, start_time]

Start a subsearch for matching ids (volume_id) that will be joined or added to the result set of the main search. 

|eval snapTime=strptime(start_time,"%Y-%m-%dT%T")

Convert the start_time of the snapshot to UNIX epoch time, which is in seconds. 

|eval diff=round(((now() - snapTime) / 86400),0)

Calculate the difference between now and when the snap was taken, convert seconds to days, and then round to a whole number to obtain the number of days since the snap was taken. 

|eval insight=case((NOT (diff>0 AND diff<30)),"No Recent Snapshot")

Determine if the snapshot is outside the 0-30 day range, and return the string if so. Otherwise, set insight to null. 

|search insight=!null

Eliminate results where the insight field is null.

|table account_id id region size type status insight

Display the results in a table with columns in the order shown.

Next steps

Sample results for this search are shown in the table below. The table below shows the volumes that have snapshots that are older than 30 days. This insight could be used to decide which volumes need a recent snapshot or which volumes could be archived and deleted. The lack of a snapshot could be caused by many things, one of which is that the infrastructure is no longer being used. 

account_id id region size type status insight

63605715280

vol-c9831616

ap-southeast-1

80

standard

available

No Recent Snapshot

63605715280

vol-be20b6aa

ap-southeast-1

80

gp2

in-use

No Recent Snapshot

63605715280

vol-c8980101

ap-southeast-1

80

standard

available

No Recent Snapshot

63605715280

vol-c992c7c1

ap-southeast-1

80

standard

available

No Recent Snapshot

Using AWS makes setting up infrastructure easy but can lead to inefficiency and wasted money when resources go unused. Inventory data such as usage, age and location can be used to find efficiencies, which are important in a cloud environment due to usage fees. This type of information can help you maintain infrastructure in the most cost effective way.

The Splunk App for AWS yields the same results on recent snapshots through an interactive dashboard. It also includes an evaluation for severity. In the app, navigate to the top menu and select Insights > EBS Insights > Insights Filter > No Snapshot (30 days). 

Finally, you might be interested in other processes associated with the Managing an Amazon Web Services environment use case.