Skip to main content

Elasticsearch v2

Use the Elasticsearch v2 integration to query and search indexes using the Lucene syntax.

Use Cases

  • Query and search indexes.
  • Calculate query and search scores based on accuracy of results.

Additional Information

The Elasticsearch v2 integration supports Elasticsearch 6.0.0 and later.

Strings are queried using the Lucene syntax. For more information about the Lucene syntax, see: https://www.elastic.co/guide/en/elasticsearch/reference/7.3/query-dsl-query-string-query.html#query-string-syntax

For more information about request response fields, see: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-body.html#request-body-search-explain

For more information about type mapping, see: https://www.elastic.co/guide/en/elasticsearch/reference/7.x/mapping.html#mapping-type

Note: Not all fields can be sorted in Elasticsearch. The fields are used to sort the results table. The supported result types are boolean, numeric, date, and keyword fields.

Configure Elasticsearch v2 on Demisto

  1. Navigate to Settings > Integrations > Servers & Services .
  2. Search for Elasticsearch v2.
  3. Click Add instance to create and configure a new integration instance.
    • Name : a textual name for the integration instance.
    • Server URL : the Elasticsearch server to which the integration connects.
    • Username and password : to log in to the server.
  4. (Optional) Select the Fetch Incidents box and input the additional parameters.
    • A CSV list from which to fetch incidents.
    • The query by which to fetch incidents (Lucene syntax).
    • The index time field (for sorting sort and limiting data).
    • The time format as kept in Elasticsearch.
    • The first fetch timestamp.
    • The number of results returned in each fetch.

      Selecting the Fetch Incidents checkbox makes the additional parameters above mandatory.

  5. Click Test to validate the new instance.

Commands

You can execute these commands from the Demisto CLI, as part of an automation, or in a playbook. After you successfully execute a command, a DBot message appears in the War Room with the command details.

  1. Query an index: es-search
  2. Searches an index: search

1. Query an index

Queries an index.

Base Command

es-search

Input
Argument Name Description Required
index The index in which to perform a search. Required
query The string to query. Strings are queried using the Lucene syntax. Required
fields A CSV list of the fields of a document to fetch. Leaving the fields empty fetches the entire document. Optional
explain Calculates an explanation of a score for a query. Default is "false". For example, "value:1.6943597". Optional
page The number of the page from which to start a search. The default is "0". Optional
size The number of documents displayed per page. Can be "1" to "10,000". The default is "100". Optional
sort-field The field by which to sort the results table. The supported result types are boolean, numeric, date, and keyword fields. Keyword fields require the doc_values parameter to be set to "true" from the Elasticsearch server. Optional
sort-order The order by which to sort the results table. The results tables can only be sorted if a sort-field is defined. Optional

Context Output
Path Type Description
Elasticsearch.Search.Results._index String The index to which the document belongs.
Elasticsearch.Search.Results._id String The ID of the document.
Elasticsearch.Search.Results._type String The mapping type of the document.
Elasticsearch.Search.max_score Number The maximum relevance score of a query.
Elasticsearch.Search.Query String The query performed in the search.
Elasticsearch.Search.total.value Number The number of search results.
Elasticsearch.Search.Results._score Number The relevance score of the search result.
Elasticsearch.Search.Index String The index in which the search was performed.
Elasticsearch.Search.Server String The server on which the search was performed.
Elasticsearch.Search.timed_out Boolean Whether the search stopped due to a time-out.
Elasticsearch.Search.took Number The time in milliseconds taken for the search to complete.
Elasticsearch.Search.Page Number The number of the page from which the search started
Elasticsearch.Search.Size Number The maximum amount of scores that a search can return.

Command Example 
!es-search query="Date:* AND name:incident" index=users fields=name,nums sort-field=Date sort-order=desc size=2
Human Readable Output

1.png

2. Search an index

Searches an index.

Base Command

search

Input
Argument Name Description Required
index The index in which to perform a search. Required
query The string to query. Strings are queried using the Lucene syntax. Required
fields A CSV list of the fields of a document to fetch. Leaving the fields empty fetches the entire document. Optional
explain Calculates an explanation of a score for a query. Default is "false". For example, "value:1.6943597". Optional
page The number of the page from which to start a search. The default is "0". Optional
size The number of documents displayed per page. Can be "1" to "10,000". The default is "100". Optional
sort-field The field by which to sort the results table. The supported result types are boolean, numeric, date, and keyword fields. Keyword fields require the doc_values parameter to be set to "true" from the Elasticsearch server. Optional
sort-order The order by which to sort the results table. The results tables can only be sorted if a sort-field is defined. Optional

Context Output
Path Type Description
Elasticsearch.Search.Results._index String The index to which the document belongs.
Elasticsearch.Search.Results._id String The ID of the document.
Elasticsearch.Search.Results._type String The mapping type of the document.
Elasticsearch.Search.max_score Number The maximum relevance score of a query.
Elasticsearch.Search.Query String The query performed in the search.
Elasticsearch.Search.total.value Number The number of search results.
Elasticsearch.Search.Results._score Number The relevance score of the search result.
Elasticsearch.Search.Index String The index in which the search was performed.
Elasticsearch.Search.Server String The server on which the search was performed.
Elasticsearch.Search.timed_out Boolean Whether the search stopped due to a time-out.
Elasticsearch.Search.took Number The time in milliseconds taken for the search to complete.
Elasticsearch.Search.Page Number The number of the page from which the search started.
Elasticsearch.Search.Size Number The maximum amount of scores that a search can return.

Command Example 
!search query="Date:* AND name:incident" index=users fields=name,nums sort-field=Date sort-order=desc size=2
Human Readable Output

1.png

Troubleshooting

For more information about the correct time format, see http://strftime.org/ .

Schema Mapping

Cortex XSOAR version 6.0 introduces an improved classification & mapping experience, which includes fetching schema data.

Elasticsearch v2 integration supports fetching the schema of indexes that are set in the Index from which to fetch incidents integration parameter, thereby enabling mapping fields per index.

Setup schema mapping

  1. Navigate to Settings > Integrations > Classification & Mapping .
  2. Create a new mapper and edit an existing one
  3. Set Get data to Select schema and Select instance to the Elasticsearch v2 integration instance to fetch from.
  4. Map fields according to the fetched index schema.
Edit this page
Report an Issue