SplunkPy
Use the SplunkPy integration to fetch events (logs) from within Cortex XSOAR, push events from Cortex XSOAR to SplunkPy, and fetch SplunkPy ES notable events as Cortex XSOAR incidents.
This integration was integrated and tested with Splunk v7.2.
#
Use Cases- Query Splunk for events.
- Create a new event in Splunk.
- Get results of a search that was executed in Splunk.
#
Configure SplunkPy on Cortex XSOAR- Navigate to Settings > Integrations > Servers & Services.
- Search for SplunkPy.
- Click Add instance to create and configure a new integration instance.
Parameter | Description | Required |
---|---|---|
host | The host name to the server, including the scheme (x.x.x.x). | True |
authentication | The username used for authentication. | True |
port | The port affiliated with the server. | True |
fetchQuery | The notable events ES query to be fetched. | False |
fetch_limit | The limit of incidents to fetch. The maximum is 200 (It is recommended to fetch less than 50). | False |
isFetch | The incidents fetched. | False |
incidentType | The incident type. | False |
proxy | Runs the integration instance using the proxy server (HTTP or HTTPS) that you defined in the server configuration. | False |
timezone | The timezone of the Splunk server (in minutes). For example, GMT is gmt +3, set +180 (set this only if it is different than the Cortex XSOAR server). This is relevant only for fetching notable events. | False |
parseNotableEventsRaw | Parses the raw part of notable events. | False |
replaceKeys | Replace with Underscore in Incident Fields | False |
extractFields | The CSV fields that will be parsed out of _raw notable events. | False |
useSplunkTime | Uses the Splunk clock time for the fetch. | False |
unsecure | When selected, certificates are not checked. (not secure) | False |
earliest_fetch_time_fieldname | The earliest time to fetch (the name of the Splunk field whose value defines the query's earliest time to fetch). | False |
latest_fetch_time_fieldname | The latest time to fetch (the name of the Splunk field whose value defines the query's latest time to fetch). | False |
app | The context of the application's namespace. | False |
hec_token | The HEC token (HTTP Event Collector). | False |
hec_url | The HEC URL. For example, https://localhost:8088. | False |
fetch_time | The first timestamp to fetch in \<number>\<time unit> format. For example, "12 hours", "7 days", "3 months", "1 year". | False |
use_requests_handler | Use Python requests handler | False |
type_field | Used only for Mapping with the Select Schema option. The name of the field that contains the type of the event or alert. The default value is "source", which is a good option for Notable Events, however you may choose any custom field that suits the need. | False |
use_cim | Use this option to get the mapping fields by Splunk CIM. See https://docs.splunk.com/Documentation/CIM/4.18.0/User/Overview for more info. | False |
The (!) Earliest time to fetch
and Latest time to fetch
are search parameters options. The search uses All Time
as the default time range when you run a search from the CLI. Time ranges can be specified using one of the CLI search parameters, such as earliest_time
, index_earliest
, or latest_time
.
- Click Test to validate the URLs, token, and connection.
Note: To use a Splunk Cloud instance, contact Splunk support to request API access. Use a non-SAML account to access the API.
#
Configure Splunk to Produce Alerts for SplunkPyIt is recommended that Splunk is configured to produce basic alerts that the SplunkPy integration can ingest, by creating a summary index in which alerts are stored. The SplunkPy integration can then query that index for incident ingestion. It is not recommended to use the Cortex XSOAR application with Splunk for routine event consumption because this method is not able to be monitored and is not scalable.
- Create a summary index in Splunk. For more information, click here.
- Build a query to return relevant alerts.
- Identify the fields list from the Splunk query and save it to a local file.
- Define a search macro to capture the fields list that you saved locally. For more information, click here.
Use the following naming convention: (demistofields{type}).
- Define a scheduled search, the results of which are stored in the summary index. For more information about scheduling searches, click here.
- In the Summary indexing section, select the summary index, and enter the {key:value} pair for Cortex XSOAR classification.
- Configure the incident type in Cortex XSOAR by navigating to Settings > Advanced > Incident Types.
- Navigate to Settings > Integrations > Classification & Mapping, and drag the value to the appropriate incident type.
- Click the Edit mapping link to map the Splunk fields to Cortex XSOAR.
- (Optional) Create custom fields.
- Build a playbook and assign it as the default for this incident type.
#
Mapping fetched incidents using Select SchemaThis integration supports the Select Schema
feature of XSOAR 6.0 by providing the get-mapping-fields
command.
When creating a new field Mapping for fetched incidents, the Pull Instances
option retrieves current alerts which can be clicked to visually map fields.
The Select Schema
option retrieves possible objects, even if they are not the next objects to be fetched, or have not been triggered in the past 24 hours.
This enables you to map fields for an incident without having to generate a new alert or incident just for the sake of mapping.
The get-mapping-fields
command can be executed in the Playground to test and review the list of sample objects that are returned under the current configuration.
To use this feature, you must set several integration instance parameters:
Fetch notable events ES query
- The query used for fetching new incidents.Select Schema
will run a modified version of this query to get the object samples, so it is important to have the correct query here.Event Type Field
- The name of the field that contains the type of the event or alert. The default value issource
which forNotable Events
will contains the rule name. However you may choose any custom field that suits this purpose.First fetch timestamp
- The time scope of objects to be pulled. You may choose to go back further in time to include samples for alert types that haven't triggered recently - so long as your Splunk server can handle the more intensive Search Job involved.
#
Mapping Splunk CIM fields using Select SchemaThis integration supports the Select Schema
feature of XSOAR 6.0 by providing the get-mapping-fields
command.
When creating a new field Mapping for fetched incidents, the Pull Instances
option retrieves current alerts which can be clicked to visually map fields.
If the user has configured the Use CIM Schemas for Mapping
parameter then the Select Schema
option retrieves fields based on Splunk CIM.
For more information see: https://docs.splunk.com/Documentation/CIM/4.18.0/User/Overview
The CIM mapping fields implemented in this integration are of 4.18.0 version.
#
CommandsYou can execute these commands from the Cortex XSOAR CLI, as part of an automation, or in a playbook. After you successfully execute a command, a DBot message appears in the War Room with the command details.
#
Get resultsReturns the results of a previous Splunk search. This command can be used in conjunction with the splunk-job-create
command.
#
Base Commandsplunk-results
#
InputArgument Name | Description | Required |
---|---|---|
sid | The ID of the search for which to return results. | Required |
limit | The maximum number of returned results per search. To retrieve all results, enter "0" (Not recommended). | Optional |
#
Context OutputThere is no context output for this command.
#
Command Example!splunk-results sid="1566221331.1186" limit="200"
#
Search for eventsSearches Splunk for events.
#
Base Commandsplunk-search
#
InputArgument Name | Description | Required |
---|---|---|
query | The Splunk search language string to execute. For example, "index=* | head 3". | Required |
earliest_time | Specifies the earliest time in the time range to search. The time string can be a UTC time (with fractional seconds), a relative time specifier (to now), or a formatted time string. The default is 1 week ago, in the format "-7d". You can also specify time in the format: 2014-06-19T12:00:00.000-07:00". | Optional |
latest_time | Specifies the latest time in the time range to search. The time string can be a UTC time (with fractional seconds), a relative time specifier (to now), or a formatted time string. For example: "2014-06-19T12:00:00.000-07:00" or "-3d" (for time 3 days before now). | Optional |
event_limit | The maximum number of events to return. The default is 100. If "0" is selected, all results are returned. | Optional |
app | The string that contains the application namespace in which to restrict searches. | Optional |
batch_limit | The maximum number of returned results to process at a time. For example, if 100 results are returned, and you specify a batch_limit of 10, the results will be processed 10 at a time over 10 iterations. This does not affect the search or the context and outputs returned. In some cases, specifying a batch_size enhances search performance. If you think that the search execution is suboptimal, it is recommended to try several batch_size values to determine which works best for your search. The default is 25,000. | Optional |
update_context | Determines whether the results will be entered into the context. | Optional |
#
Context OutputPath | Type | Description |
---|---|---|
Splunk.Result | Unknown | The results of the Splunk search. The results are a JSON array, in which each item is a Splunk event. |
#
Command Example!splunk-search query="* | head 3" earliest_time="-1000d"
#
Human Readable Output#
Splunk Search results for query: * | head 3_bkt | _cd | _indextime | _kv | _raw | _serial | _si | _sourcetype | _time | host | index | linecount | source | sourcetype | splunk_server |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
main~445~66D21DF4-F4FD-4886-A986-82E72ADCBFE9 | 445:897774 | 1585462906 | 1 | InsertedAt="2020-03-29 06:21:43"; EventID="837005"; EventType="Application control"; Action="None"; ComputerName="ACME-code-007"; ComputerDomain="DOMAIN"; ComputerIPAddress="127.0.0.1"; EventTime="2020-03-29 06:21:43"; EventTypeID="5"; Name="LogMeIn"; EventName="LogMeIn"; UserName=""; ActionID="6"; ScanTypeID="200"; ScanType="Unknown"; SubTypeID="23"; SubType="Remote management tool"; GroupName="";\u003cbr\u003e | 2 | ip-172-31-44-193, main | sophos:appcontrol | 2020-03-28T23:21:43.000-07:00 | 127.0.0.1 | main | 2 | eventgen | sophos:appcontrol | ip-172-31-44-193 |
#
Create eventCreates a new event in Splunk.
#
Base Commandsplunk-submit-event
#
InputArgument Name | Description | Required |
---|---|---|
index | The Splunk index to which to push the data. Run the splunk-get-indexes command to get all of the indexes. | Required |
data | The new event data to push. Can be, any string. | Required |
sourcetype | The event source type. | Required |
host | The event host. Can be, "Local" or "120.0.0.1". | Required |
#
Context OutputThere is no context output for this command.
#
Command Example!splunk-submit-event index="main" data="test" sourcetype="demisto-ci" host="localhost"
#
Human Readable Output#
Print all index namesPrints all Splunk index names.
#
Base Commandsplunk-get-indexes
#
InputThere are no input arguments for this command.
#
Context OutputThere is no context output for this command.
#
Command Example !splunk-get-indexes extend-context="indexes="
#
Human Readable Output#
Update notable eventsUpdate an existing notable event in Splunk ES.
#
Base Commandsplunk-notable-event-edit
#
InputArgument Name | Description | Required |
---|---|---|
eventIDs | The comma-separated list of event IDs of notable events. | Required |
owner | The Splunk user to assign to the notable event. | Optional |
comment | The comment to add to the notable event. | Required |
urgency | The urgency of the notable event. | Optional |
status | The notable event status. Can be 0 - 5, where 0 - Unassigned, 1 - Assigned, 2 - In Progress, 3 - Pending, 4 - Resolved, 5 - Closed. | Optional |
#
Context OutputThere is no context output for this command.
#
Command Example !splunk-notable-event-edit eventIDs=66D21DF4-F4FD-4886-A986-82E72ADCBFE9@@notable@@a045b8acc3ec93c2c74a2b18c2caabf4 comment="Demisto"
#
Human Readable Output#
Create a new jobCreates a new search job in Splunk.
#
Base Commandsplunk-job-create
#
InputArgument Name | Description | Required |
---|---|---|
query | The Splunk search language string to execute. For example, "index=* | head 3". | Required |
app | The string that contains the application namespace in which to restrict searches. | Optional |
#
Context OutputPath | Type | Description |
---|---|---|
Splunk.Job | Unknown | The SID of the created job. |
#
Command Example!splunk-job-create query="index=* | head 3"
#
Context Example#
Human Readable Output#
Parse an eventParses the raw part of the event.
#
Base Commandsplunk-parse-raw
#
InputArgument Name | Description | Required |
---|---|---|
raw | The raw data of the Splunk event (string). | Optional |
#
Context OutputPath | Type | Description |
---|---|---|
Splunk.Raw.Parsed | unknown | The raw event data (parsed). |
#
Command Example!splunk-parse-raw
#
Submit an eventSends events to an HTTP event collector using the Splunk platform JSON event protocol.
#
Base Commandsplunk-submit-event-hec
#
InputArgument Name | Description | Required |
---|---|---|
event | The event payload key-value. An example string: "event": "Access log test message.". | Required |
fields | Fields for indexing that do not occur in the event payload itself. Accepts multiple, comma separated, fields. | Optional |
index | The index name. | Optional |
host | The hostname. | Optional |
source_type | The user-defined event source type. | Optional |
source | The user-defined event source. | Optional |
time | The epoch-formatted time. | Optional |
#
Context OutputThere is no context output for this command.
#
Command Example!splunk-submit-event-hec event="something happened" fields="severity: INFO, category: test, test1" source_type=access source="/var/log/access.log"
#
Human Readable OutputThe event was sent successfully to Splunk.
#
Get job statusReturns the status of a job.
#
Base Commandsplunk-job-status
#
InputArgument Name | Description | Required |
---|---|---|
sid | The ID of the job for which to get the status. | Required |
#
Context OutputPath | Type | Description |
---|---|---|
Splunk.JobStatus.CID | Unknown | The ID of the job. |
Splunk.JobStatus.Status | Unknown | The status of the job. |
#
Command Example!splunk-job-status sid=1234.5667
#
Context Example#
Human Readable Output#
Get Mapping FieldsGets one sample alert per alert type. Used only for creating a mapping with Select Schema
.
#
Base Commandget-mapping-fields
#
InputThere are no input arguments for this command.
#
Context OutputThere is no context output for this command.
#
Command Example!get-mapping-fields using="SplunkPy_7.2" raw-response="true"
#
Human Readable Output{ "Splunk": { "CollectionList": [ "autofocus_tags", "files" ] } }
{ "Splunk": { "KVstoreData": { "demisto_store": [ { "_key": "5f4e2e9c097d9e6749453536", "_user": "nobody", "addr": "0.0.0.0" } ] } } }
{ "Splunk": { "KVstoreData": { "demisto_store": [ { "_key": "5f4e2e9c097d9e6749453536", "_user": "nobody", "addr": "0.0.0.0" } ] } } }