Monitoring electronic data interchange transmission and acknowledgement
Electronic Data Interchange (EDI) is a standardized way to exchange business documents electronically between organizations. EDI plays a critical role in ensuring that data flows seamlessly across various stakeholders—suppliers, manufacturers, logistics providers, and retailers—without manual intervention. It is a core technology for automating supply chain processes, enabling organizations to quickly and securely exchange important documents such as purchase orders (850), invoices (810), shipment notifications (856), and functional acknowledgments (997) between systems. This automation helps reduce errors, speed up processes, and improve the overall efficiency of the supply chain.
Effective EDI communication depends on accurate exchanges and timely acknowledgments between suppliers, manufacturers, and customers. When an EDI message is sent, the receiving system must respond with an acknowledgment, confirming that the transmission was successful and the document has been recognized. Additionally, monitoring the acknowledgment process (for example, receiving 997 functional acknowledgments) confirms that the recipient successfully received the transmitted EDI documents. Failures or delays in these transmissions can disrupt procurement, inventory management, and fulfillment processes, leading to operational delays, stock shortages, or delayed payments. All of this ultimately impacts both operational efficiency and customer satisfaction.
By continuously monitoring EDI transmissions and acknowledgments, businesses can proactively identify and resolve issues, ensuring smooth communication between systems and uninterrupted supply chain operations.
Prerequisites
The following software is required for this use case.
- The Splunk platform. Mainly used for initial data processing, analysis and alerting.
- Optional. Splunk ITSI. For further expanding the analytics, leverage the ITSI thresholding and anomaly detection capability.
- Optional. PowerConnect for SAP Solutions. If the supply chain operational transactions needs to be integrated from SAP ERP, this app is necessary.
The following data types from supply chain operational transactions are required inputs for accomplishing this use case.
- EDI X12, EDIFACT. Standard supply chain transactions data.
- Optional. API Data. Supply chain transactions data gathered through APIs.
- Optional. SAP iDocs. iDocs data collected using SAP Powerconnect for Splunk.
The EDI transmissions and acknowledgement process
Successful transmission and acknowledgment of EDI documents are crucial to maintaining an efficient supply chain.
Several types of EDI documents are commonly used throughout the supply chain, each serving a specific function:
- Purchase Orders (EDI 850): Used to place orders with suppliers.
- Order Acknowledgments (EDI 855): Confirms that the supplier has received the purchase order.
- Advance Ship Notices (EDI 856): Provides details about shipments that are en route.
- Invoices (EDI 810): Outlines the billing details for the goods or services provided.
- Functional Acknowledgement (997): Confirms that these documents have been received and processed successfully.
EDI 977 is an especially important part of this process, because without receiving these acknowledgments, businesses are left in uncertainty about whether critical documents have been delivered, risking delays in order fulfillment, inventory management, and payment processing.
Creating KPIs for EDI transmission status
What to monitor
It's important to track a few key performance indicators (KPIs) to ensure EDI messages are processed correctly.
- Monitoring acknowledgments. Track the receipt of acknowledgment documents (997) within a specified time frame after sending each transaction.
- Transmission success rates. Ensure EDI documents such as POs (850), ASNs (856), and Invoices (810) are successfully transmitted to their respective recipients.
- Transmission errors and retry rates. Look for EDI documents that fail to transmit or are stuck in retry loops. Lost or failed EDI transactions can lead to operational inefficiencies, such as missed orders, delayed shipments, or inventory discrepancies.
- Acknowledgment timing. Monitor delayed acknowledgments.
- Finding repeated failure patterns. Look for patterns of repeated transmission or acknowledgment errors might indicate a systemic issue.
Delays or failures in transmitting or receiving EDI documents can cause significant disruptions in the supply chain, leading to procurement delays, missed deliveries, and potential financial penalties.
Building KPIs in the Splunk platform
This section explains the base for any search to build visual KPIs to track key EDI transmission metrics. As you click into each of linked searches in the section above, you'll see the following components:
First, searching for events with sourcetype="edi:x12"
will retrieve all EDI transactions, including both the actual transactions and their corresponding acknowledgments.
index=supply_chain_edi sourcetype="edi:x12" | table _time edi_ack_status edi_buyer edi_code edi_code_ack edi_cont_num edi_date edi_flag edi_requestor edi_responder edi_seller edi_sequence edi_time edi_tr_id edi_type
Next, use the eval
command to create a field called edi_ack_status_combo
, which associates acknowledgment events with the corresponding EDI transactions. The result of this evaluation will look like 997-214-A
, which indicates that 997 (the acknowledgment) corresponds to 214 (the Transportation Carrier Shipment Status Message).
Additionally, the edi_code_groupby
field is evaluated to group the actual EDI request with its acknowledgment. In the example above, this would be 214. The purpose of this grouping is to combine the request and acknowledgment events, allowing for easier filtering and analysis. For instance, you can quickly search for requests that lack an acknowledgment.
index=supply_chain_edi sourcetype="edi:x12" | table _time edi_ack_status edi_buyer edi_code edi_code_ack edi_cont_num edi_date edi_flag edi_requestor edi_responder edi_seller edi_sequence edi_time edi_tr_id edi_type | eval edi_ack_status_combo=edi_code+"-"+edi_code_ack+"-"+edi_ack_status | eval edi_code_groupby=if(isnull(edi_code_ack), edi_code, edi_code_ack)
With the proper evaluation conditions in place, we can now merge EDI requests with their corresponding acknowledgments using the stats
command.
index=supply_chain_edi sourcetype="edi:x12" | table _time edi_ack_status edi_buyer edi_code edi_code_ack edi_cont_num edi_date edi_flag edi_requestor edi_responder edi_seller edi_sequence edi_time edi_tr_id edi_type | eval edi_ack_status_combo=edi_code+"-"+edi_code_ack+"-"+edi_ack_status | eval edi_code_groupby=if(isnull(edi_code_ack), edi_code, edi_code_ack) | stats last(_time) AS _time, last(edi_requestor) AS edi_requestor, last(edi_responder) AS edi_responder, first(edi_code) AS edi_type, list(edi_code) AS edi_code, last(edi_ack_status) AS edi_ack_status BY edi_tr_id edi_code_groupby
The result of the stats
command provides an excellent starting point for various analyses, particularly in identifying issues related to missing acknowledgments, which could disrupt critical physical processes. For example, by searching through the results of the stats command, you can easily identify processed events that lack an edi_ack_status field
, which indicates that the acknowledgment event hasn't been merged with the request event.
Search explanations
Splunk search | Explanation |
---|---|
index=supply_chain_edi sourcetype="edi:x12" |
Select EDI X12 data by selecting sourcetype of edi:x12 . |
| table _time edi_ack_status edi_buyer edi_code edi_code_ack edi_cont_num edi_date edi_flag edi_requestor edi_responder edi_seller edi_sequence edi_time edi_tr_id edi_type |
Format the relevant fields into a table. |
| eval edi_ack_status_combo=edi_code+"-"+edi_code_ack+"-"+edi_ack_status |
Create a new evaluated field edi_ack_status_combo that combines edi_code , edi_code_ack , edi_ack_status field. Use another eval command to create edi_code_groupby field to join all EDI events with EDI 997 acknowledgment events. |
| stats last(_time) AS _time, last(edi_requestor) AS edi_requestor, last(edi_responder) AS edi_responder, first(edi_code) AS edi_type, list(edi_code) AS edi_code, last(edi_ack_status) AS edi_ack_status BY edi_tr_id edi_code_groupby |
Join all EDI events with EDI 997 (acknowledgment) events into a single event. A successfully joined event will have an EDI Events with null values in |
Summary
Monitoring your EDI transmissions and acknowledgments ensures operational efficiency and helps prevent costly delays in procurement and fulfillment. By setting up real-time alerts and dashboards, you can maintain visibility into the health of your supply chain and act on issues as soon as they arise.
Now you know how to calculate key performance indicators (KPIs) to monitor EDI transmission and acknowledgment, which are crucial for assessing the health of your EDI transaction systems. These KPIs can also help identify potential infrastructure issues that might hinder your organization’s ability to process critical EDI information across your supply chain partners.
Next steps
With the knowledge you've gained, consider incorporating the KPIs discussed in this template into your essential supply chain system health monitoring dashboard.
- Go to Dashboards > Create New Dashboard.
- Name your dashboard (for example, EDI Transmission Monitoring).
- Add panels to visualize the key metrics covered in this use case.
- Customize your visuals: Use bar charts, gauges, and time charts to represent the data clearly. You can also set thresholds for acceptable transmission success rates or acknowledgment timing.
As a next step, follow the Solution Accelerator for Supply Chain Optimization to learn how to ingest the EDI transaction data set. After the data is collected, ensure that the appropriate TAs are applied to extract the default EDI fields for analysis.
Splunk also offers a turnkey professional services solution to monitor the reliability and health of your supply chain systems. To learn more about engaging our professional services, contact our sales team.