Demisto Add-on for Splunk
Supporting add-on for Cortex XSOAR. This application enables you to push incidents from Splunk into Cortex XSOAR, according to configurable trigger parameters.
The add-on is open source. Source code is available at: https://github.com/demisto/splunk-app.
Prerequisites#
- Splunk version 8.0 or later
Test on a local Splunk Environment#
Run the following command to create a Splunk Docker container (replace the ***** with any 8-character password, containing letters and digits):
When the command is finished executing, the Splunk environment will be available at http://localhost:8000.
Installation of the Add-on#
Download Demisto Add-on for Splunk from Splunkbase.
After initializing the container, open your local Splunk environment.
Go to "Manage Apps" → Install app from file → upload the latest version of Demisto Add-on for Splunk. Note: if a version of the app already exists, select the "Upgrade app" checkbox.
Restart Splunk and login again.
Configuration#
In order to use the add-on and create incidents in XSOAR, you must complete the setup of the application. After you install the add-on, click "Launch app" and provide the following:
- Create a XSOAR instance:
Under XSOAR Instances tab, press the "Add" button. Choose an instance name, and fill the XSOAR server URL (including port if needed) and the API key fields. The API key is used for authorization with XSOAR. To generate this parameter, login to Cortex XSOAR and click on Settings → Integration → API Keys.
NOTE: When using XSOAR v8 and above, and Advanced API key should be generated and be inserted as follows: <API_KEY>$<API_AUTH_ID>
Set up proxy settings (optional):
Under the Proxy tab, select the "Enable" checkbox and provide all of the necessary proxy parameters.Choose log level (optional):
By default, the logging level is "INFO". You can change the logging level to "DEBUG" if needed.Additional Settings (optional):
- If you have an SSL certificate, provide its full path under the "Location to Certificate" field. By default, "Validate SSL" is enabled.
- If you would like to extend the incident creation request timeout, provide the desired timeout under the "Timeout Value" field. By default, timeout value is 10 seconds.
You must restart Splunk in order to apply changes in the configuration settings.
Connectivity Test - Create a Custom Alert Action from Saved Searches#
Upload data to Splunk (any small PDF, CSV, or YML file).
When the file is uploaded, click "Start Searching" and save the search as an Alert (on the top-right corner).
Complete the Alert settings:
- Title
- Permissions – Shared in App
- Alert type – Run on Cron Schedule
- Cron Expression – * * * * * (every 1 minute)
Press "Add Actions" and choose Create XSOAR Incident, from which you can setup the alert incident details:
- Name - name of the alert
- Time Occurred - time when alert was triggered
- XSOAR Server (if "Send Alert to all the servers" is unchecked)
- Type – incident type in XSOAR
- Custom Fields - A comma-separated 'key:value' custom fields pairs
- Labels – a comma-separated values to populate the labels field
- Severity – the alert severity
- Details – "details" field of the incident
Go to the XSOAR server and wait for incidents to be ingested (one for each event in Splunk).
Note: Saved Alerts can be found under Search & Reporting → Alerts.
Create an Adaptive Response Action from Splunk Enterprise Security#
Click "Incident Review". For Splunk ES App users, all of the incidents are reported in the "Incident Review" Dashboard.
Click the "Actions" drop-down button for the incident that needs to be reported to Cortex XSOAR.
Click "Run Adaptive Response Actions".
Click "Add New Response Action".
Select Create XSOAR Incident from the list of Actions.
Fill in the details that need to be sent out to Cortex XSOAR (as described in the Connectivity Test. When finished, click "Run".
Splunk will display a message indicating that the action has been dispatched.
Back in the main screen of the Incident List table, Click the "I" to list incidents in detail and track actions that took place when the incident occurred in Splunk.
Check the "Adaptive Response" section under details. It should include a response such as "XSOAR Custom Alert Action" with a status.
Click "XSOAR Custom Alert Action" to drill down to more details on the alert action that was called along with the detailed response received from Cortex XSOAR.
Troubleshooting#
We recommend checking the following logs as an initial step in troubleshooting:
- Splunk issues:
var/log/splunk/splunkd.log - Add-on issues:
var/log/splunk/create_xsoar_incident_modalert.log
Common Issues#
Splunk Events are not Created in XSOAR#
In cases where after associating Create XSOAR Incident with saved searches or correlation searches using automated invocation or using ad-hoc invocation from Splunk-ES incident review dashboard, Incidents are not getting created into the Cortex XSOAR, the following criteria should be checked.
Check the configuration page. You may need to restart the add-on in order for the modifications to take effect.
Check the Saved Search trigger conditions. Splunk might be suppressing the events and not creating an incident in Cortex XSOAR.
Test the network connectivity in between applications to ensure that there are no connectivity issues. You may try one of the following:
Test the API key generated in XSOAR.
Connection Timeout Errors#
In case of a connection timeout error as in the above log:
- Locate the
SavedSplunkerlog facility. - Increase its verbosity to ERROR level.
- If the logs include the following:
WARN SavedSplunker - Reached maximum amount of time allowed to spend in per-result alerts for savedsearch_id="SAVEDSEARCH_ID"Increase the max_per_result_alerts_time value in thelimits.conffile.
Incident Created with Incorrect Fields#
If incidents being created in XSOAR but with incorrect fields, refer to the Connectivity Test section for an understanding of the fields which are associated with create incident actions.
Too Many Incidents Created in XSOAR#
Splunk has its own trigger conditions using the number of incidents which get pushed to XSOAR controlled.
Refer to Splunk Documentation to understand Splunk Trigger Conditions to limit the incidents. Also, it is possible to throttle alerts in Splunk after the saved search returned positive results, in order to suppress more incident creation.
Playbooks not getting invoked on Cortex XSOAR#
Cortex XSOAR allows users to set up different types of incidents and the user can associate different playbooks the each type of incident. If after pushing incidents from Splunk, the appropriate playbooks are not being invoked, you should check the following:
Check if the type of incident which is getting pushed from Splunk has a playbook associated with the Type in XSOAR.
Check if there are no case differences in the Type created in XSOAR and the type which is being pushed along with incident information in Splunk.
SSL Certificates Issues#
If you don’t use a certificate, make sure the "Validate SSL" checkbox is not selected.
If the you have a self-signed certificate, you must add it to the Splunk server first, and then set the path to it in the Add-on configuration page.
In the case of a self-signed certificate, make sure that the whole certificate chain exist. Each of root, intermediate, and client certificate is required.
When checking SSL certificate validity by pinging, ping to the server domain. SSL certificates are signed on server domain rather than its IP.
Splunk Common Information Model (CIM) Issues#
If Splunk CIM is installed, make sure its version is compatible with the installed Splunk version. A common error caused by a version incompatibility might look as follows: