Splunk Cloud Integration

  • 4 minute(s) read
Prev Next

Introduction

From infrastructure management to data compliance, Splunk Cloud is built to scale to your data analytics needs, ranging from GBs to PBs and beyond. Read more about Splunk cloud here. This integration with Splunk Cloud enables you to push performance and alert data the moment it iss recorded. It relies on Catchpoints two push APIs: Test Data Webhook, and Alert Webhook. Test Data Webhook API pushes each test performance metric after every run when it is enabled. Alert Webhook API pushes alert data the moment an alert trigged. The data is sent via HTTP Event Collector (HEC) in Splunk over Secure HTTP (HTTPS). HEC uses a token-based authentication model. You can generate a token and then configure a logging library or HTTP client with the token to send data to HEC in a specific format.

Prerequisites

  • Splunk Cloud account.

Installation and Configuration

**Splunk Setup

Create an Event Collector token

To use HEC, you must configure at least one token. On Splunk Cloud instances, the platform distributes the token across the deployment. The token is not ready for use until distribution has completed.

  1. Click Settings > Data Inputs.
  2. Click HTTP Event Collector.
  3. Select New Token.
  4. In the Name field, enter a name for the token.
  5. (Optional) In the Source name override field, enter a name for a source to be assigned to events that this endpoint generates.
  6. (Optional) In the Description field, enter a description for the input.
  7. Click Next.
  8. (Optional) Make edits to source type and confirm the index where you want HEC events to be stored. See Modify input settings.
  9. Click Review.
  10. Verify the settings are as per your use case.
  11. Click Submit. Otherwise, click < to make changes.
  12. Copy the token value that Splunk Web displays and paste it into another document for reference later.
    step_8.png

URI for HEC.

The standard form for the HEC URI in Splunk Cloud is as follows:

<protocol>://http-inputs-<host>:<port>/<endpoint>
Attributes of URI:

  • <protocol> is https.
  • <host> is the Splunk Cloud instance that runs HEC, pre-pend the hostname with 'http-inputs-'.
  • <port> is the HEC port number. use '443'.
  • <endpoint> is the HEC endpoint, use '/services/collector'.

For example, if the hostname for your Splunk cloud is catchpoint.splunkcloud.com the format for the HEC URI in Splunk Cloud will be:
https://http-inputs-catchpont.splunkcloud.com:443/services/collector

Create Events indexes:

  1. In Splunk Web, navigate to Settings > Indexes and click New.
  2. To create a new index, enter: Name for the index: catchpoint-webhooks. Index data type: Events. App: Search and Reporting.
  3. Click Save

Catchpoint Setup:

To get Catchpoint data into Splunk, login into the Catchpoint Portal and go to Settings > API.

  1. In the Test Data Webhook add the Splunk cloud HEC endpoint.
  2. Select Template.
  3. Click Add New from the drop-down.
  4. Enter a name.
  5. Select JSON under format.
  6. Paste template content(Testdata-Webhook-Template.txt) from the attachment found along with this KB.
  7. Next for authentication with the Splunk server add the Authorization headers in Key -> value format. Key: Authorization. Value: Splunk <Event – Collector -Token>.
    webhook_setup.png
  8. Click Save.

Note: Test Data Webhook feature should be enabled for each test to send metrics under the test properties page.

To get Catchpoint Aert data into Splunk, login into the Catchpoint Portal and go to Settings > API.

  1. In the Alert Webhook enter the Alert Webhook name according to your preference.
  2. Set status to Active
  3. For URL, add the Splunk cloud HEC endpoint.
  4. Select Template.
  5. Click Add New from the drop-down.
  6. Enter a name.
  7. Select JSON under format.
  8. Paste template content(Alert-Webhook-Template.txt) from the attachment found along with this KB.
  9. Click Save.
  10. Next for authentication with the Splunk server add the Authorization headers in Key -> value format: Key: Authorization Value: Splunk <Event – Collector -Token>.
    alert_settings.png 11. Click Save.

Implementation

Importing Dashboards:

  1. From Splunk home page under Apps select Search & Reporting
  2. Select Dashboards and click Create New Dashboard.
  3. Add title as Catchpoint - Overview. ID should be automatically assigned.
  4. Provide required permissions to the dashboard and select create dashboard.
  5. Now select Source and copy the contents of catchpoint_overview.xml file and select save.
  6. Repeat the same steps for Catchpoint - Recent Errors, Catchpoint - Test Times, Catchpoint - Response Size, and Catchpoint - Alerts.

Note: Make sure all the Dashboard names are the same as mentioned above for the drill-downs to work as expected.

Result

The catchpoint Overview Dashboard is your central location for the Catchpoint tests in your account. View at-a-glance information surrounding your recent Errors. The Tests widget lets you search for and quickly access your test data.

The Test Time dashboard focuses on displaying how much time was spent loading resources. It plots the metrics over time making it easier to identify trends.

The Response size dashboard plots the amount of data downloaded when loading each resource. This highlights the amount of content and the header's download size over time.

The Errors page lists all the errors encountered by tests. This page makes it easy to view the top issues as well as narrow down on problems to identify commonality between failures for any given test or group of tests.

The Alerts page allows you to view the history of all the alerts reported by Catchpoint.

Sample Query to Plot DNS, Connect, Load, Send, SSL, Wait

index="catchpoint-webhooks" | spath output=Connect path=Summary.Timing.Connect | spath output=DNS path=Summary.Timing.Dns | spath output=Load path=Summary.Timing.Load | spath output=Send path=Summary.Timing.Send | spath output=Ssl path=Summary.Timing.Ssl | spath output=Wait path=Summary.Timing.Wait | spath output=Client path=Summary.Timing.Client | spath output=TestName path=TestDetail.Name | spath output=NodeName path=NodeName | spath output=TestRuntime path=TestRuntime | eval Dimensions=TestName + "#" + NodeName | regex NodeName="^$NodeName$$"| regex TestName="^$TestName$$"| timechart cont=false span=1m latest(Connect) as Connect latest(DNS) as DNS latest(Load) as Load latest(Send) as Send latest(Ssl) as Ssl latest(Wait) as Wait latest(Client) as Client by Dimensions

Catchpoint-Recent_Errors.xml

Catchpoint-Response_Size.xml

Catchpoint - Test Times.xml

Catchpoint-Overview.xml

Catchpoint-Alerts.xml

Alert-Webhook-Template.txt

Testdata-Webhook-Template.txt