Managing an Amazon Web Services environment

Scenario: To reduce the expenses of buying, owning, and maintaining physical data centers and servers, your organization has converted most of its infrastructure to cloud infrastructure on Amazon Web Services. This means you have whole new data types to secure and monitor. You need to learn the new services and the metrics that help you run your applications in the AWS cloud efficiently. You can continue to use your Splunk deployment to manage all components of your infrastructure and provide you with necessary information and alerts from AWS.

How Splunk software can help

You can use Splunk software to manage all the different components of Amazon Web Services, including elastic cloud compute instances, elastic load balancer instances, virtual private clouds, and elastic block store volumes. You can also monitor user behavior on these systems.

What you need 

To succeed in implementing this use case, you need the following dependencies, resources, and information.

People

The best person to implement this use case is an AWS administrator who is familiar with the in scope AWS services. This person might come from your team, a Splunk partner, or Splunk OnDemand Services.

Time

Maintaining AWS systems using Splunk software can last up to a day or more to on board the data as there can be a lot of configuration details to set up on the AWS side. 

Technologies 

The following technologies, data, and integrations are useful in successfully implementing this use case: 

• Splunk Enterprise or Splunk Cloud
• Data sources onboarded
  • Cloudwatch
  • Vpcflow
  • Access logs(elb, s3 cloudfront)
  • Descriptions
  • Aws:config
  • Cloudtrail
  • Aws:inspector
  • aws:sqs
• Splunk Add-on for Amazon Web Services
• Splunk AWS Project Trumpet  (Helpful for high volume AWS source)

How to use Splunk software for this use case

You can run many searches with Splunk software to maintain your AWS environment. Depending on what information you have available, you might find it useful to identify some or all of the following: 

• Current AWS elastic compute cloud instances
• Current AWS elastic load balancer instances
• Current AWS virtual private cloud infrastructure
• Current AWS elastic block store volumes
• Health of AWS elastic load balancers
• Unattached AWS elastic block store volumes
• AWS EBS volumes without a current snapshot
• Users who haven't accessed AWS for an extended time
• Resources with non-compliant AWS configuration rules
• Changes made to AWS cloud infrastructure

Other steps you can take

To maximize their benefit, the how-to articles linked in the previous section likely need to tie into existing processes at your organization or become new standard processes. These processes commonly impact success with this use case: 

• Capacity planning and resource optimization to keep operating expenses as low as possible
• Provisioning and automation groups using orchestrators such as Kubernetes, Ansible, Puppet, and AWS CloudFormation 
• Security and compliance will consume and compliment data within this use case. 

Related resources 

These additional Splunk resources might help you understand and implement this use case:

• Blog: Six top metrics to monitor in AWS EBS
• Blog: 12 top things to monitor in Amazon EC2
• Conf Talk: Capacity planning and cost containment with AWS
• Conf Talk: How to save money monitoring, managing, and securing your cloud using the Splunk App for AWS
• White Paper: Getting Data Into (GDI) Splunk from AWS 
• Tool: Splunk AWS Project Trumpet 

How to assess your results

Measuring impact and benefit is critical to assessing the value of IT operations. The following are example metrics that can be useful to monitor when implementing this use case:

• Number of deprovisioned resources that are found to be idle 
• Cost savings from deprovisioning and optimization 
• Mean time to problem resolution