Managing Cisco IOS devices

Data required  

Cisco IOS

How to use Splunk software for this use case

This use case is best deployed using the IT Essentials Learn app, a free application that helps you find specific procedures that fit your environment, learn how they work, deploy them successfully, and measure your success. By deploying the prebuilt searches available in IT Essentials Learn or using the sample searches below and adapting them to your environment, you will be able to do the following:

• Detect duplicate IP addresses. IP addresses uniquely identify each and every network device and should never be duplicated. However, networking misconfigurations and circumstances sometimes cause IP addresses to be duplicated across devices, which then leads to unpredictable system behavior. You can search your Cisco IOS device logs to detect the presence of duplicate IP addresses and emit a message when found.

  index=* sourcetype=cisco:ios "DUPADDR"
  | stats count values(src_mac) AS macs values(src_interface) AS interfaces BY host src_ip

• Identify and investigate duplex mismatches. Duplex mismatches occur when two physically connected devices have been configured in different duplex modes. A Cisco IOS device can detect a duplex mismatch between it and another device and emits a message when found. You can identify duplex mismatches and investigate their cause.

  index=* sourcetype=cisco:ios "duplex mismatch"
  | eval src = host.": ".src_interface, cdp_neighbor=cdp_neighbor.": ".dest_interface
  | stats count BY src cdp_neighbor

• Identify and investigate high temperature alarms. Overheating of any electronic device can lead to performance problems and device failures. Cisco IOS devices emit a message when various temperature sensors exceed preconfigured thresholds. You can identify and investigate any device currently reporting high temperature alarms.

  index=* sourcetype=cisco:ios alarm temperature
  | rex "(?<mnemonic>%.*:) (?<module>\S+) reported (?<severity>\S+) (?<alarm>\S+).*"
  | stats sparkline(count,15m) AS trend count values(alarm) AS alarms values(severity) AS severity BY host module
  | table host module alarms severity trend count

• Identify hosts with highest log volume. As network devices operate and route traffic, critical status information is regularly emitted via syslog. Hosts that produce large volumes of syslog data can indicate a highly used device. However, the volume of syslog messages can also rise dramatically because of a network issue or misconfiguration. You can identify which hosts are producing the largest volumes of syslog data and review the syslog messages to ensure the device is operating as expected. 

  index=* sourcetype=cisco:ios
  | stats count sparkline(count,15m) AS trend BY host
  | sort - count
  | head 10
  | table host trend count

• Identify port flapping. Port flapping is a situation in which a physical interface on the switch continually goes up and down, three or more times a second for at least 10 seconds. Common causes for port flapping are bad, unsupported, or non-standard cable or other link synchronization issues. The cause for port flapping can be intermittent or permanent. You can identify when it happens on your network so you can investigate and resolve the problem.

  index=* eventtype="cisco_ios-port_down" OR eventtype="cisco_ios-port_up" product IN (IOS) index IN (*) 
  | eval port_state=if(vendor_action="up",1,0)
  | stats sparkline(sum(port_state),15m) as trend count, latest(vendor_action) AS current_port_status BY host,src_interface 
  | eventstats sum(count) AS host_total BY host
  | sort -host_total -count
  | fields -host_total
  | table host,src_interface,trend,current_port_status,count

• Monitor error trends and network instability. As network devices operate and route traffic, critical status information including errors, warnings, and other signs of network instability is regularly emitted via syslog, SNMP, and other networking data sources. It's not uncommon to see a regular stream of error events. However, sudden increases in the volume of errors or a rise in error volumes over time might be a sign of a problem with the network and should be monitored and investigated.

  index="<NETWORK-INDEX>" (drop* OR timeout OR linkdown OR critical OR major)
  | timechart span=5m count

Next steps

To maximize their benefit, the how-to articles linked in the previous section likely need to tie into existing processes at your organization or become new standard processes. These processes commonly impact success with this use case: 

• Log collection with syslog
• Integration of Splunk dashboards and reports into the Network Operations Center (NOC) 

Measuring impact and benefit is critical to assessing the value of IT operations. The following are example metrics that can be useful to monitor when implementing this use case:

• Reduction of mean time to problem resolution
• Reduction in network related ticks submitted by end users 

You might also be interested in the following Splunks apps:

• Cisco Networks App for Splunk Enterprise
• IT Essentials Work