Skip to main content
Splunk Lantern

Azure load balancers with no healthy instances

 

Azure Load Balancers provide high availability by distributing incoming traffic among healthy virtual machines (VMs). An Azure Load Balancer health probe monitors a given port on each VM and only distributes traffic to operational VMs. As an administrator, you need to run a search to ensure that there are enough healthy VMs supporting the traffic requests.

Data required 

Microsoft Azure data

Procedure

  1. Configure the Microsoft Azure Add-on for Splunk.
  2. Run the following search. You can optimize it by specifying an index and adjusting the time range.
sourcetype=azure:metrics metric_name=VipAvailability 
| mvexpand host 
| search host="/subscription*" 
| rex field=host "(?<resource>[^\/]*)\Z" 
| dedup host sortby - _time 
| eval {metric_name}=average 
| where VipAvailability=0 
| table resource VipAvailability namespace subscription_id
| rename resource AS LB VipAvailability AS Availability namespace AS Namespace subscription_id AS Subscription

Search explanation

The table provides an explanation of what each part of this search achieves. You can adjust this query based on the specifics of your environment.

Splunk Search Explanation
sourcetype=azure:metrics metric_name=VipAvailability Search only Azure metric VipAvailability data. 
| mvexpand host Expand values of the host field into separate events.
| search host="/subscription*" Filter to a subset of hosts.
| rex field=host "(?<resource>[^\/]*)\Z" Extract the resource name from the host field.
| dedup host sortby - _time Return only one record per host or resource.
| eval {metric_name}=average Create a new field where the VipAvailability metric equals the average field value.
| where VipAvailability=0 Identify when the VipAvailability metric is 0.
| table resource VipAvailability namespace subscription_id Display the host, VipAvailability, namespace and subscription_id fields.
| rename resource AS LB VipAvailability AS Availability namespace AS Namespace subscription_id AS Subscription Improve field name readability.

Next steps

If there are no healthy VMs supporting the traffic requests, there's likely a problem to investigate.

You might also be interested in other processes associated with the Managing Azure cloud infrastructure use case.