Gaining better visibility into your third-party APM solutions

Glass table: 3rd Party APM Overview

The 3rd Party APM Overview glass table delivers rolled-up insights across your APM vendors for a summarized view. Summaries for overall health along with both End User Experience Health and Application Performance Health are included. In addition, six KPIs are included:

• Apdex score to help you assess user satisfaction
• Requests per minute
• Response time
• Availability
• Error rate
• Health

This glass table provides a visual overview of everything going on, across all vendors, and with a single click on any of these results you can access more detail.

In the example below, there is an error rate of 55%, but from this dashboard alone it's not immediately clear what's causing this error rate. To find this information, you can click on it to go to the Service Analyzer.

The Service Analyzer shows the specific services that contribute to the metric seen in the Dashboard. From the Service Analyzer, you can see the individual KPIs that contribute to that error rate. In the example below, Dynatrace is showing a high error rate. Clicking on Dynatrace brings up a list of the services associated with it where you can immediately see that the Checkout service is experiencing a 63.57% error rate.

Using the 3rd Party APM Overview Glass Table in this way lets you pinpoint specific issues in only two clicks.

Glass table: 3rd Party APM All-In-One

The All-In-One glass table shows overall health, along with each of the six total KPIs - six for Application Performance, and a subset three in End User Experience. It also shows results summarized by each of the three legacy third-party APM vendors, and a total for each of the KPIs.

With this view, you are able to see the status of each of the KPIs for the overall, application performance, and end user experience areas in a simple and summarized view. Like other glass tables, you can click on any of the results to get more detail.

Glass table: 3rd Party APM Online Retail

This glass table gives you an out-of-the-box solution specifically designed for online retail settings. You can use the Global Time Range and Global Refresh Rate menus at the top of the page to filter the results that displays.

The glass table is divided into 5 major sections. The first section shows the same scores as the All-In-One table. In the upper-right are four critical online service KPIs. Lastly, the bottom sections shows a left-to-right workflow of a user moving from browsing, to cart, to purchase. It highlights several of the key services and their KPIs throughout the online retail experience. You can click any of these sections to see more.

Custom glass table example

It's easy to create new glass tables or clone existing ones, then edit the services and KPIs shown within them, to create glass tables that are relevant to your organization. The example below shows a clone of the retail-focused glass table but updated with metrics applicable to a financial services scenario, with a flow from login to transaction to execution of those transactions. Setting up a new glass table from a clone of the retail-focused glass table can take as little as 30 minutes.

How to use the Service Analyzer

Within the Service Analyzer tree view, as shown in the example below, you can clearly see each of the 12 services and also see how the underlying KPIs are reporting. The third-party APM solution is shown at the top. Underneath that you can see services for application performance, as well as services for end user experience. The colors used indicate the health of each of the underlying services. Under APM_End_User_Experience, for example, you can see APM_EUEM_Response_Time, where EUEM stands for End User Experience Management. This service is shown in orange and with an exclamation mark to indicate that the service is not performing optimally and that a problem of high severity is occurring.

 

In the example below, clicking into the service shows some normal and some low severity KPIs associated with the service. These are based on thresholds that have been set. The data that these KPIs are drawing from is based on Dynatrace, and AppDynamics as well as New Relic. The KPI marked low here is reporting a 1682 millisecond delay. Since this KPI is associated with the end user experience, this indicates that the end user customer is waiting 1.6 seconds for something.

 

Clicking the low severity KPI brings up detail of the entities associated with the KPI. In the example below, most of the entities are reporting in as normal, but one entity associated with the checkout service is high severity, with an associated 15.6 second wait.

Clicking into the checkout service that's experiencing delays brings you to the Event Data Search so you can investigate further. If you need to access the raw data, you can click Show as raw text to see that. In the top-right corner, you can open the results in search and click the icon above to return further entity information.

The entity information you can see in the example below is specific to the APM entity type. You can see 18 KPIs associated with the service and all the results associated with those. 

 

Clicking through the Service Analyzer in this way provides you with the overall view you need to understand how your services are performing on average. It also lets you click down into the detail for deep investigation into what factors influence overall service health.

Vendor-specific dashboards

The Content Pack for Third-party APM contains several dashboards specific to AppDynamics, New Relic, and Dynatrace. The dashboards included are:

• AppDynamics Applications
• AppDynamics Incidents 
• AppDynamics Overview
• Dynatrace Application Overview
• Dynatrace Host Investigator
• Dynatrace Host Overview
• Dynatrace Process Group Instance Investigator
• Dynatrace Service Overview
• Dynatrace Trend Analysis
• New Relic Alerts
• New Relic Insights Data
• New Relic Key Transactions
• New Relic Mobile Applications
• New Relic Web Applications

The example below shows the AppDynamics Overview dashboard. In this dashboard, you can filter by time period, controllers, or different applications or services. The dashboard uses predictive analytics to show the KPI ranges that you would expect to see for these vendors.

Event Data Search dashboard

The Event Data Search dashboard displays the 100 most recent log events associated with an entity for the last 60 minutes. The dashboard provides a high-level overview of entity performance across your whole environment, regardless of the entity type you associated with the entity. You can add to, modify, or delete the preconfigured APM entity type.

From the Event Data Search you can also click Application Overview to access the Application Overview dashboard.

Application Overview dashboard

The Application Overview dashboard provides you with information that can be filtered to a specific service or vendor, including:

• Application availability as well as the error rate percentage.
• Application performance, both throughput and response time.
• End user performance, both throughput and response time.
• A comparison of end user response time and application response time. You can use this to see the differences between the two and identify gaps that might require further investigation.
• Apdex score for application performance and the trend in score changes over time.
• Apdex score for end user experience and the trend in score changes over time.
• A list of searchable events. You could search these events for a term that might indicate performance issues, for example the health status "orange", which then shows events you might want to investigate.

Infrastructure Overview dashboard

You can also find events of interest through the Infrastructure Overview dashboard. Click through to the dashboard from the Splunk ITSI toolbar and select the entity you are interested in investigating.

Clicking on the returned entity takes you through to the Event Data search where you can access more information about the entity and view KPIs associated with it.