Skip to main content
Splunk Lantern

Monitoring AWS EC2 for unusual modifications

You are an Amazon Web Services (AWS) admin who manages access to AWS resources and services across your organization. As part of your role, you need to identify unusual changes to your AWS Elastic Compute Cloud (EC2) instances that may indicate malicious activity, for example, by looking for modifications to your EC2 instances by previously unseen users. Adversaries commonly aim to infiltrate a cloud instance and make modifications, then secure access to your infrastructure and hide their activities. It's important for you to look for changes that may indicate that your environment has been compromised.

These searches can help you detect the presence of a threat by monitoring for EC2 instances that have been created or changed - either by users who have never previously performed these activities or by known users who modify or create instances in a way that has not been done before.

How to use Splunk software for this use case

  • Some commands, parameters, and field names in the searches below may need to be adjusted to match your environment.
  • To optimize the searches, you should specify an index and a time range when appropriate. 
  • Install the AWS App for Splunk (version 5.1.0 or later) and Splunk Add-on for AWS (version 4.4.0 or later), then configure your CloudTrail inputs. Some searches also require configuration of description inputs.

Support searches

► Previously seen EC2 modifications by user

This search builds a table of previously seen Amazon Resource Names (ARNs) that have launched a EC2 instance.

| search (errorCode=success sourcetype=aws:cloudtrail (eventName=AssociateAddress OR eventName=AssociateIamInstanceProfile OR eventName=AttachClassicLinkVpc OR eventName=AttachNetworkInterface OR eventName=AttachVolume OR eventName=BundleInstance OR eventName=DetachClassicLinkVpc OR eventName=DetachVolume OR eventName=GetConsoleOutput OR eventName=GetConsoleScreenshot OR eventName=ModifyInstanceAttribute OR eventName=ModifyInstancePlacement OR eventName=MonitorInstances OR eventName=RebootInstances OR eventName=ResetInstanceAttribute OR eventName=StartInstances OR eventName=StopInstances OR eventName=TerminateInstances OR eventName=UnmonitorInstances)) 
| spath output=arn userIdentity.arn 
| stats earliest(_time) AS firstTime latest(_time) AS lastTime BY arn 
| outputlookup previously_seen_ec2_modifications_by_user 
| stats count

Detection searches

► EC2 instance modified with previously unseen user

This search works best when you run the Previously seen EC2 modifications by user support search once to create a history of previously seen ARNs.

This search looks for EC2 instances being modified by users who have not previously modified them.

The subsearch returns the ARNs of all successful EC2 instance modifications within the last hour and then appends the historical data in the lookup file to those results. The search then recalculates the firstTime and lastTime field for each ARN and returns only those ARNs that have first been seen in the past hour. This is combined with the main search to return the time, user, and instance ID of those systems.

False positives can occur from this search since it's possible that a new user will start to modify EC2 instances when they haven't before for any number of reasons. Verify with the user that is modifying instances that this is the intended behavior.

search sourcetype=aws:cloudtrail (eventName=AssociateAddress OR eventName=AssociateIamInstanceProfile OR eventName=AttachClassicLinkVpc OR eventName=AttachNetworkInterface OR eventName=AttachVolume OR eventName=BundleInstance OR eventName=DetachClassicLinkVpc OR eventName=DetachVolume OR eventName=GetConsoleOutput OR eventName=GetConsoleScreenshot OR eventName=ModifyInstanceAttribute OR eventName=ModifyInstancePlacement OR eventName=MonitorInstances OR eventName=RebootInstances OR eventName=ResetInstanceAttribute OR eventName=StartInstances OR eventName=StopInstances OR eventName=TerminateInstances OR eventName=UnmonitorInstances) [search sourcetype=aws:cloudtrail (eventName=AssociateAddress OR eventName=AssociateIamInstanceProfile OR eventName=AttachClassicLinkVpc OR eventName=AttachNetworkInterface OR eventName=AttachVolume OR eventName=BundleInstance OR eventName=DetachClassicLinkVpc OR eventName=DetachVolume OR eventName=GetConsoleOutput OR eventName=GetConsoleScreenshot OR eventName=ModifyInstanceAttribute OR eventName=ModifyInstancePlacement OR eventName=MonitorInstances OR eventName=RebootInstances OR eventName=ResetInstanceAttribute OR eventName=StartInstances OR eventName=StopInstances OR eventName=TerminateInstances OR eventName=UnmonitorInstances) errorCode=success 
| stats earliest(_time) AS firstTime latest(_time) AS lastTime BY userIdentity.arn 
| rename userIdentity.arn AS arn 
| inputlookup append=t previously_seen_ec2_modifications_by_user 
| stats min(firstTime) AS firstTime, max(lastTime) AS lastTime BY arn 
| outputlookup [previously_seen_ec2_modifications_by_user]
| eval newUser=if(firstTime >= relative_time(now(), "-1h@h"), 1, 0) 
| where newUser=1 
| convert timeformat="%m/%d/%Y %H:%M:%S" ctime(firstTime) 
| convert timeformat="%m/%d/%Y %H:%M:%S" ctime(lastTime) 
| rename arn AS userIdentity.arn 
| table userIdentity.arn] 
| spath output=dest responseElements.instancesSet.items{}.instanceId 
| spath output=user userIdentity.arn

Contextual searches

► EC2 instance details by instanceId

In order to implement this search, you must configure your AWS description inputs.

This search queries AWS description logs and returns all the information about a specific instance via the instanceId field.

| search sourcetype="aws:description" source="*:ec2_instances"
| dedup id sortby -_time 
| search id={instanceId} 
| spath output=tags path=tags 
| eval tags=mvzip(key,value," = "), ip_address=if((ip_address == "null"),private_ip_address,ip_address) 
| table id, tags.Name, aws_account_id, placement, instance_type, key_name, ip_address, launch_time, state, vpc_id, subnet_id, tags 
| rename aws_account_id AS "Account ID", id AS ID, instance_type AS Type, ip_address AS "IP Address", key_name AS "Key Pair", launch_time AS "Launch Time", placement AS "Availability Zone", state AS State, subnet_id AS Subnet, "tags.Name" AS Name, vpc_id AS VPC

Investigative searches

► AWS investigate user activities by ARN

This search lists all the logged CloudTrail activities by a specific user ARN and creates a table that contains the source of the user, the region of the activity, the name and type of the event, the action taken, and all the user's identity information.

| search sourcetype=aws:cloudtrail userIdentity.arn={arn} 
| table _time userIdentity.type userIdentity.userName userIdentity.arn aws_account_id src awsRegion eventName eventType

Next steps

The content in this article comes from Splunk Enterprise Security (ES). As a Splunk premium security solution, ES solves a wide range of security analytics and operations use cases including continuous security monitoring, advanced threat detection, compliance, incident investigation, forensics and incident response. Splunk ES delivers an end-to-end view of an organization's security posture with flexible investigations, unmatched performance, and the most flexible deployment options offered in the cloud, on-premises, or hybrid deployment models. If you have questions about this use case, see the Security Research team's support options on GitHub.

In addition, Splunk Enterprise Security provides a number of other searches to help reinforce your Cloud Security posture which are reproduced here, including:

Need technical help? Explore our customer success resources to find education and training, engage experts through OnDemand services, view support options, and more.