Deciding on automatic versus manual instrumentation

Automatic instrumentation

Automatic instrumentation is a run-time automated process that automatically identifies the frameworks and libraries that are in use. It instruments those libraries to capture tracing, without making any code changes. If your library is supported by auto-instrumentation, helpful operational data can be obtained. For example, you might want to know when:

• a Python application made an HTTP request to another service
• a Java application called a MySQL database to create a new user

Auto-instrumentation covers most cases and adds Splunk APM instrumentation to your application or service. However, you might want to use manual instrumentation if any of the following apply to you:

• auto-instrumentation does not capture relevant distributed traces
• there is missing metadata relevant to your business logic
• you don't have expected level of tracing data

Manual (custom) instrumentation

With manual instrumentation, you modify application code to selectively capture trace spans in interesting areas. You can also add relevant span metadata tags around blocks of code that you want to instrument. Manually adding spans in this way gives you flexibility of capturing what you need and allows for capturing custom metadata. This method is the only way to add instrumentation and capture detail in first-party business logic.

Hybrid instrumentation

You can mix automatic and manual instrumentation. Even when your library is supported by auto-instrumentation, manual instrumentation can be added to get more insight. Auto instrumentation data can be augmented by the addition of manually captured spans and tags that add more detail.

An example of the kind of information that could be captured by a hybrid approach could be where a Java application makes a call to a MySQL database to create a new user (auto), and the new user belongs to a certain customer (manual).