Elasticsearch v2

Use the Elasticsearch v2 integration to query and search indexes using the Lucene syntax.

Use Cases

• Query and search indexes.
• Calculate query and search scores based on accuracy of results.

Additional Information

The Elasticsearch v2 integration supports Elasticsearch 6.0.0 and later.

Strings are queried using the Lucene syntax. For more information about the Lucene syntax, see: https://www.elastic.co/guide/en/elasticsearch/reference/7.3/query-dsl-query-string-query.html#query-string-syntax

For more information about request response fields, see: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-body.html#request-body-search-explain

For more information about type mapping, see: https://www.elastic.co/guide/en/elasticsearch/reference/7.x/mapping.html#mapping-type

Note: Not all fields can be sorted in Elasticsearch. The fields are used to sort the results table. The supported result types are boolean, numeric, date, and keyword fields.

Configure Elasticsearch v2 on Demisto

1. Navigate to Settings > Integrations > Servers & Services .
2. Search for Elasticsearch v2.
3. Click Add instance to create and configure a new integration instance.
   • Name : a textual name for the integration instance.
   • Server URL : the Elasticsearch server to which the integration connects.
   • Username and password : to log in to the server.
4. (Optional) Select the Fetch Incidents box and input the additional parameters.
   • A CSV list from which to fetch incidents.
   • The query by which to fetch incidents (Lucene syntax).
   • The index time field (for sorting sort and limiting data).
   • The time format as kept in Elasticsearch.
   • The first fetch timestamp.
   • The number of results returned in each fetch.

     Selecting the Fetch Incidents checkbox makes the additional parameters above mandatory.

5. Click Test to validate the new instance.

Commands

You can execute these commands from the Demisto CLI, as part of an automation, or in a playbook. After you successfully execute a command, a DBot message appears in the War Room with the command details.

1. Query an index: es-search
2. Searches an index: search

1. Query an index

Queries an index.

Base Command

es-search

Input

Argument Name	Description	Required
index	The index in which to perform a search.	Required
query	The string to query. Strings are queried using the Lucene syntax.	Required
fields	A CSV list of the fields of a document to fetch. Leaving the fields empty fetches the entire document.	Optional
explain	Calculates an explanation of a score for a query. Default is "false". For example, "value:1.6943597".	Optional
page	The number of the page from which to start a search. The default is "0".	Optional
size	The number of documents displayed per page. Can be "1" to "10,000". The default is "100".	Optional
sort-field	The field by which to sort the results table. The supported result types are boolean, numeric, date, and keyword fields. Keyword fields require the doc_values parameter to be set to "true" from the Elasticsearch server.	Optional
sort-order	The order by which to sort the results table. The results tables can only be sorted if a sort-field is defined.	Optional

Context Output

Path	Type	Description
Elasticsearch.Search.Results._index	String	The index to which the document belongs.
Elasticsearch.Search.Results._id	String	The ID of the document.
Elasticsearch.Search.Results._type	String	The mapping type of the document.
Elasticsearch.Search.max_score	Number	The maximum relevance score of a query.
Elasticsearch.Search.Query	String	The query performed in the search.
Elasticsearch.Search.total.value	Number	The number of search results.
Elasticsearch.Search.Results._score	Number	The relevance score of the search result.
Elasticsearch.Search.Index	String	The index in which the search was performed.
Elasticsearch.Search.Server	String	The server on which the search was performed.
Elasticsearch.Search.timed_out	Boolean	Whether the search stopped due to a time-out.
Elasticsearch.Search.took	Number	The time in milliseconds taken for the search to complete.
Elasticsearch.Search.Page	Number	The number of the page from which the search started
Elasticsearch.Search.Size	Number	The maximum amount of scores that a search can return.

Command Example

!es-search query="Date:* AND name:incident" index=users fields=name,nums sort-field=Date sort-order=desc size=2

Human Readable Output

2. Search an index

Searches an index.

Base Command

search

Input

Argument Name	Description	Required
index	The index in which to perform a search.	Required
query	The string to query. Strings are queried using the Lucene syntax.	Required
fields	A CSV list of the fields of a document to fetch. Leaving the fields empty fetches the entire document.	Optional
explain	Calculates an explanation of a score for a query. Default is "false". For example, "value:1.6943597".	Optional
page	The number of the page from which to start a search. The default is "0".	Optional
size	The number of documents displayed per page. Can be "1" to "10,000". The default is "100".	Optional
sort-field	The field by which to sort the results table. The supported result types are boolean, numeric, date, and keyword fields. Keyword fields require the doc_values parameter to be set to "true" from the Elasticsearch server.	Optional
sort-order	The order by which to sort the results table. The results tables can only be sorted if a sort-field is defined.	Optional

Context Output

Path	Type	Description
Elasticsearch.Search.Results._index	String	The index to which the document belongs.
Elasticsearch.Search.Results._id	String	The ID of the document.
Elasticsearch.Search.Results._type	String	The mapping type of the document.
Elasticsearch.Search.max_score	Number	The maximum relevance score of a query.
Elasticsearch.Search.Query	String	The query performed in the search.
Elasticsearch.Search.total.value	Number	The number of search results.
Elasticsearch.Search.Results._score	Number	The relevance score of the search result.
Elasticsearch.Search.Index	String	The index in which the search was performed.
Elasticsearch.Search.Server	String	The server on which the search was performed.
Elasticsearch.Search.timed_out	Boolean	Whether the search stopped due to a time-out.
Elasticsearch.Search.took	Number	The time in milliseconds taken for the search to complete.
Elasticsearch.Search.Page	Number	The number of the page from which the search started.
Elasticsearch.Search.Size	Number	The maximum amount of scores that a search can return.

Command Example

!search query="Date:* AND name:incident" index=users fields=name,nums sort-field=Date sort-order=desc size=2

Human Readable Output

Troubleshooting

For more information about the correct time format, see http://strftime.org/ .

Schema Mapping

Cortex XSOAR version 6.0 introduces an improved classification & mapping experience, which includes fetching schema data.

Elasticsearch v2 integration supports fetching the schema of indexes that are set in the Index from which to fetch incidents integration parameter, thereby enabling mapping fields per index.

Setup schema mapping

1. Navigate to Settings > Integrations > Classification & Mapping .
2. Create a new mapper and edit an existing one
3. Set Get data to Select schema and Select instance to the Elasticsearch v2 integration instance to fetch from.
4. Map fields according to the fetched index schema.