Constructing an API test JSON payload for alerting on external dependencies

About the JavaScript engine

• Splunk Synthetic Monitoring only provides the V8 engine for running scripts. It doesn’t provide a Node.js runtime environment, so it doesn’t support Node.js features. For example, there is no support for file system access using fs, HTTP requests using http, event loops using setTimeout or similar functions, or modules such as require or module.exports.
• If you expect an API to return null values you will need to handle them appropriately in your JavaScript before parsing.

GET from GitHub API

Here you'll use the githubstatus.com API as your first API endpoint to gather data from. This endpoint includes an indicator - one of none, minor, major, or critical, as well as a human description of the blended component status. Examples of the blended status include "All Systems Operational", "Partial System Outage", and "Major Service Outage". You'll use these two indicators to construct a JSON payload.

The GitHub API returns a body that looks like this:

 { 
    "page": 
    { 
        "id": "kctbh9vrtdwd", 
        "name": "GitHub", 
        "url": "https://www.githubstatus.com", 
        "time_zone": "Etc/UTC", 
        "updated_at": "2025-04-10T09:51:56.432Z" 
    }, 
    "status": 
    { 
        "indicator": "none", 
        "description": "All Systems Operational" 
    } 
} 

After you receive this response, use the Validation portion of your request setup with a JavaScript extraction to extract the response body to a custom variable custom.response that you can use with your JavaScript to parse out the useful information.

javascript
var apiResponse = JSON.parse(custom["response"]); 
  
function createGaugeEntry(apiResponse) { 
    var value = (apiResponse.status.indicator === "none") ? "0" : "1"; 
  
    var gaugeEntry = { 
        "gauge": [ 
            { 
                "metric": "github.status", 
                "dimensions": { 
                    "description": apiResponse.status.description, 
                    "indicator": apiResponse.status.indicator 
                }, 
                "value": value 
            } 
        ] 
    }; 
    return gaugeEntry; 
} 
var gaugeEntry = createGaugeEntry(apiResponse); 
custom.payload_github = gaugeEntry; 

Your JavaScript (as seen in the screenshot below) will:

1. Parse the custom.response variable as JSON.
2. Create a function to establish from your status.indicator if you should set the value of your metric to 1 (meaning a status other than none) or 0.
3. Create a JSON payload to match the format expected by your datapoint ingest API where you will send your constructed metric.
4. Use the fields from the parsed JSON response you received to update the dimensions for description and indicator included with your metric along with the 1 or 0 value you will chart and alert on in Splunk Observability Cloud.
5. Output the constructed JSON as another custom variable called custom.payload_github.
6. Check that the payload you constructed is not empty.

GET from CloudFlare API

Next, you'll do almost the same thing with the cloudflarestatus.com API. This endpoint includes the same indicators and descriptions as the githubstatus.com API. Again, you'll use these two indicators to construct a JSON payload.

The CloudFlare API returns a body that looks like this:

{ 
    "page": 
    { 
        "id": "yh6f0r4529hb", 
        "name": "Cloudflare", 
        "url": "http://www.cloudflarestatus.com", 
        "time_zone": "Etc/UTC", 
        "updated_at": "2025-04-10T15:02:38.563Z" 
    }, 
    "status": 
    { 
        "indicator": "minor", 
        "description": "Minor Service Outage" 
    } 
}  

After you receive this response, do the same validation as detailed in the previous step to extract the response body to a custom variable custom.response that you can use with your JavaScript to parse out the useful information into the variable custom.payload_cloudflare.

javascript
var apiResponse = JSON.parse(custom["response"]); 
  
function createGaugeEntry(apiResponse) { 
    var value = (apiResponse.status.indicator === "none") ? "0" : "1"; 
  
    var gaugeEntry = { 
        "gauge": [ 
            { 
                "metric": "cloudflare.status", 
                "dimensions": { 
                    "description": apiResponse.status.description, 
                    "indicator": apiResponse.status.indicator 
                }, 
                "value": value 
            } 
        ] 
    }; 
  
    return gaugeEntry; 
} 
  
var gaugeEntry = createGaugeEntry(apiResponse); 
  
custom.payload_cloudflare = gaugeEntry; 

Finally, you'll check that the payload you constructed is not empty.

With your two custom payload variables created, you can move on to combining them and sending your final request to create your metric in Splunk Observability Cloud.

Creating a global variable for the API ingest token

To send your metric for the status of GitHub and CloudFlare into Splunk Observability Cloud, you'll need to create a global variable in Splunk Synthetic Monitoring that contains a valid Splunk Observability Cloud ingest token.

In the Splunk Synthetic Monitoring interface, click the gear icon and select Global variables.

Click the Create Variable button and enter the following to create a variable:

• Variable type: Environment
• Variable name: env.o11y-ingest-token (or your own name)
• Variable value: <Your valid Splunk Observability Cloud ingest token>
• Conceal value: Click to redact this value in the UI and in requests

You will reference this variable in your POST request.

POST to Splunk Observability Cloud datapoint ingest

The final stage of the Splunk Synthetic Monitoring API test is to take your two JSON payloads and combine them into a single payload you can send on to the datapoint ingest API.

You'll create a POST request to https://ingest.us1.signalfx.com/v2/datapoint and in the Setup step, use JavaScript to combine these payloads.

That JavaScript takes the two custom variables custom.payload_github and custom.payload_cloudflare and passes them through a function that iterates through each of the custom payloads and adds their data to a new combined data array. A final custom.payload variable is returned that you'll use as the payload body for your final POST. That JavaScript is included here:

javascript
var gaugePayloads = [ 
    JSON.parse(custom["payload_github"]), 
    JSON.parse(custom["payload_cloudflare"]) 
]; 
  
function combineMultipleGaugeEntries(payloads) { 
    var combinedGaugeArray = []; 
     
    for (var i = 0; i < payloads.length; i++) { 
        combinedGaugeArray = combinedGaugeArray.concat(payloads[i].gauge); 
    } 
  
    return { "gauge": combinedGaugeArray }; 
} 
  
var combinedGaugeEntry = combineMultipleGaugeEntries(gaugePayloads); 
  
custom.payload = combinedGaugeEntry; 

• In the Request, use the URL for datapoint ingest https://ingest.us1.signalfx.com/v2/datapoint and set the type to POST.
• For Payload body, use templating to include your payload {{custom.payload}}.
• Include two required request headers for your X-SF-Token, which references the environment variable you created in Creating a global variable for the API ingest token:
  • X-SF-Token: {{env.org_ingest_token}}
  • Content-Type: application/json
• Click Try Now at the top of the test to verify that your test works. You want it to return a 2XX response from the ingest API.
• If you get any 400 errors, verify that your tokens are correct.

Charting your metrics

By charting cloudflare.status and github.status you can now easily chart when your external dependencies are experiencing issues. This then becomes a useful signal to alert on or use as context in aggregated alerting. No one wants to be paged when external dependencies like code repositories, CDN, or other third-party dependencies are down.