Cortex XSOAR for Developers (Formerly Demisto)

Devo v2

Overview

The Devo_v2 Integration with enhanced functionality and data structures. This integration was integrated and tested with version 6.0+ Devo. Devo is a generic log management solution which can also act as an advanced SIEM. Users are able to query petabytes of data in a fraction of the time that other traditional time series databases can't.

Use Cases

  • Ingest all user defined alerts from Devo into Demisto
  • Query any data source available on the Devo.
  • Run needle in haystack multi-table queries for threat hunting incidents.
  • Write results back to Devo as searchable records or alerts.
  • Write new entries into lookup tables to be used in synthesis tables (ALPHA)

Prerequisites

  • Active Devo account and domain.
  • OAuth token with the *.** permissions.
  • Writer TLS Certificate, Key, and Chain if writing back to Devo.

Get your Demisto OAuth Token

  1. Login to your Devo domain with a user with the ability to create security credentials.
  2. Navigate to Administration > Credentials > Authentication Tokens.
  3. If a token for Demisto has not already been created, Click CREATE NEW TOKEN
  • Create the Token with *.** table permissions as an apiv2 token.
  1. Note the generated Token

Get your Demisto Writer Credentials

  1. Login to your Devo domain with a user with the ability to create security credentials.
  2. Navigate to Administration > Credentials > X.509 Certificates.
  3. Click NEW CERTIFICATE if you do not already have a set of keys for Demisto.
  4. Download the following files:
  • Certificate
  • Private Key
  • CHAIN CA

Configure Devo_v2 on Demisto

  1. Navigate to Settings > Integrations > Servers & Services.
  2. Search for Devo_v2
  3. Click Add instance to create and configure a new integration instance.
    • Name: a textual name for the integration instance.
    • Query Server Endpoint (e.g. https://apiv2-us.devo.com/search/query)
    • Oauth Token (Preferred method)
    • Writer relay to connect to (e.g. us.elb.relay.logtrust.net) Optional
    • Writer JSON credentials Optional
    {
        "key": string,
        "crt": string,
        "chain": string
    }
    • Devo base domain Optional
    • Use system proxy settings Optional
    • Fetch incidents Optional
    • Incident type Optional
    • Fetch incidents alert filter (Same filters for get-alerts) Optional
    {
        "type": <"AND" | "OR">,
        "filters" : [
          {"key": <String Devo Column Name>, "operator": <Devo Linq Operator>, "value": <string>},
          {"key": <String Devo Column Name>, "operator": <Devo Linq Operator>, "value": <string>},
          ...
          {"key": <String Devo Column Name>, "operator": <Devo Linq Operator>, "value": <string>}
        ]
    }
    • Deduplication parameters JSON if required. SEE README Optional
    {
        "cooldown": <int seconds cooldown for each type of alert>
    }
  4. Click Test to validate the URLs, token, and connection.

Fetched Incidents Data

Fetched incidents data will resemble closely to that of the data you get back from the devo-get-alerts command. The format is as follows. The keyN in the main body will be the columns that you used to define your alert in Devo.

{
  "devo.metadata.alert": {
    "eventdate" :  string,
    "alertHost" :  string,
    "domain" :  string,
    "priority" :  string,
    "context" :  string,
    "category" :  string,
    "status" :  string,
    "alertId" :  string,
    "srcIp" :  string,
    "srcPort" :  string,
    "srcHost" :  string,
    "dstIp" :  string,
    "dstPort" :  string,
    "dstHost" :  string,
    "application" :  string,
    "engine" :  string
  },
  <key0>: <value0>,
  <key1>: <value1>,
  ...
  <keyN>: <valueN>
}

Currently the only data that is fetchable in Devo are the alerts that users have defined in the platform.

Commands

You can execute these commands from the Demisto CLI, as part of an automation, or in a playbook. After you successfully execute a command, a DBot message appears in the War Room with the command details. 1. devo-run-query 2. devo-get-alerts 3. devo-multi-table-query 4. devo-write-to-table 5. devo-write-to-lookup-table

1. devo-run-query

Queries Devo based on linq query.

Please refer to to the Devo documentation for building a query with LINQ HERE

Required Permissions

A Demisto instance configured with the correct OAuth token that has permission to query the target tables

Base Command

devo-run-query

Input
Argument NameDescriptionRequired
queryA LINQ Query to runRequired
fromStart datetime for specified query. Unix timestamp in seconds expected (Decimal milliseconds okay)Required
toEnd datetime for specified query. Unix timestamp in seconds expected (Decimal milliseconds okay)Optional
writeToContextWhether to write results to context or notOptional

#####from and to time note: This integration allows for the following formats. Note that when from and to times are both given that they must be the same given format.

  • When from is a date range such as "1 day", "30 minute", etc... to is not needed and will be ignored even if given.
  • Unix timestamps in millis and seconds are accepted.
  • Datetime strings in the format '%Y-%m-%dT%H:%M:%S' are accepted.
  • Python datetime objects are accepted as well.
  • Unsupported formats will error out.
Context Output
PathTypeDescription
Devo.QueryResultsunknownList of dictionary of results
Devo.QueryLinkunknownLink back to Devo table for executed query
Command Example
!devo-run-query query="from siem.logtrust.web.activity select *" from=1576845233.193244 to=1576845293.193244
Human Readable Output

Devo run query results

eventdateleveldomainuseridusernamesessionidcorrelationIdsrcHostsrcPortserverHostserverPorttypemethodurlheadersparamsrefereruserAgentlocalecontentLengthresponseLengthresponseTimeresultresourceInfoerrorInfocountryregioncityisporg
2019-10-23T17:18:29.784000INFOhelloworld988409ce-3955-44a8-bcbb-b613bc8d9f8ejohn.doe@devo.com22671FE384D9FDF20E9BFFD7F44699711.2.3.445590us.devo.com8080GEThttps://us.devo.com/alerts/alertsGlobe.json{origin:app.custom.tsAnomalyDetectionDev,serialNumber:ad475065-b0ef-4bbe-a620-a6dcd0874629,}https://us.devo.com/welcomeMozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36en_US01247OKUSNJSecaucusPpman Services SrlM247 Ltd New Jersey
2019-10-23T17:18:29.800000INFOhelloworld988409ce-3955-44a8-bcbb-b613bc8d9f8ejohn.doe@devo.com22671FE384D9FDF20E9BFFD7F44699711.2.3.445588us.devo.com8080GEThttps://us.devo.com/domain/notification.json{origin:app.custom.tsAnomalyDetectionDev,serialNumber:ad475065-b0ef-4bbe-a620-a6dcd0874629,}https://us.devo.com/welcomeMozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36en_US011924OKUSNJSecaucusPpman Services SrlM247 Ltd New Jersey
2019-10-23T17:18:59.780000INFOhelloworld988409ce-3955-44a8-bcbb-b613bc8d9f8ejohn.doe@devo.com22671FE384D9FDF20E9BFFD7F44699711.2.3.445816us.devo.com8080GEThttps://us.devo.com/alerts/alertsGlobe.json{origin:app.custom.tsAnomalyDetectionDev,serialNumber:ad475065-b0ef-4bbe-a620-a6dcd0874629,}https://us.devo.com/welcomeMozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36en_US01247OKUSNJSecaucusPpman Services SrlM247 Ltd New Jersey
2019-10-23T17:18:59.799000INFOhelloworld988409ce-3955-44a8-bcbb-b613bc8d9f8ejohn.doe@devo.com22671FE384D9FDF20E9BFFD7F44699711.2.3.445814us.devo.com8080GEThttps://us.devo.com/domain/notification.json{origin:app.custom.tsAnomalyDetectionDev,serialNumber:ad475065-b0ef-4bbe-a620-a6dcd0874629,}https://us.devo.com/welcomeMozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36en_US011925OKUSNJSecaucusPpman Services SrlM247 Ltd New Jersey
2019-10-23T17:19:29.777000INFOhelloworld988409ce-3955-44a8-bcbb-b613bc8d9f8ejohn.doe@devo.com22671FE384D9FDF20E9BFFD7F44699711.2.3.446096us.devo.com8080GEThttps://us.devo.com/alerts/alertsGlobe.json{origin:app.custom.tsAnomalyDetectionDev,serialNumber:ad475065-b0ef-4bbe-a620-a6dcd0874629,}https://us.devo.com/welcomeMozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36en_US01247OKUSNJSecaucusPpman Services SrlM247 Ltd New Jersey
DevoTableLink
Devo Direct Link

2. devo-get-alerts

Queries alerts in the specified timeframe.

Alerts are based off the table siem.logtrust.alert.info found in your Devo account. Please refer to this table for a list of columns you can filter off of. Also please refer back to the LINQ documentation for operations that are allowed.

Required Permissions

Requires a Devo OAuth token that has read permission on siem.logtrust.alert.info table

Base Command

devo-get-alerts

Input
Argument NameDescriptionRequired
fromStart datetime for alerts to fetchRequired
toEnd datetime for alerts to fetchOptional
filterskey value filter to apply to retrieve specified alerts. refer to docsOptional
writeToContextwrite results to context or notOptional

#####from and to time note: This integration allows for the following formats. Note that when from and to times are both given that they must be the same given format.

  • When from is a date range such as "1 day", "30 minute", etc... to is not needed and will be ignored even if given.
  • Unix timestamps in millis and seconds are accepted.
  • Datetime strings in the format '%Y-%m-%dT%H:%M:%S' are accepted.
  • Python datetime objects are accepted as well.
  • Unsupported formats will error out.
Context Output
PathTypeDescription
Devo.AlertsResultslist of dictionariesList of dictionary alerts in specified time range
Devo.QueryLinkstringLink back to Devo table for executed query
Command Example
!devo-get-alerts from=1576845233.193244 to=1576845293.193244
Human Readable Output

Devo get alerts results

eventdatealertHostdomainprioritycontextcategorystatusalertIdsrcIpsrcPortsrcHostdstIpdstPortdstHostprotocolusernameapplicationengineextraData
2019-10-23T18:18:07.320000backofficehelloworld5.0my.alert.helloworld.simultaneous_loginmy.context46715552pilot.my.alert.helloworld.simultaneous_loginduration_seconds: 30.142
cluster: -
prev_timestamp: 2019-10-23+18:17:29.652
instance: -
distance: 294.76
level: info
city: Secaucus
srcHost: 1.2.3.4
prev_city: Waltham
format: output_qs9n126lnvh
prev_geolocation: 42°23'49.925537109375"N+71°14'36.2420654296875"W
message: 0,9,31,49,69,77,123,136,149,156,204,217,231<>ANOMALOUSjohn.doe@devo.com294.755774516937950.008372777777777778Secaucus40°47'15.36529541015625"N+74°3'35.9912109375"W15718546797941.2.3.4Waltham42°23'49.925537109375"N+71°14'36.2420654296875"W157185464965250.204.142.130
eventdate: 2019-10-23+18:18:02.087
prev_srcHost: 50.204.142.130
duration: 0.008372777777777778
indices: 0,9,31,49,69,77,123,136,149,156,204,217,231
payload: ANOMALOUSjohn.doe@devo.com294.755774516937950.008372777777777778Secaucus40°47'15.36529541015625"N+74°3'35.9912109375"W15718546797941.2.3.4Waltham42°23'49.925537109375"N+71°14'36.2420654296875"W157185464965250.204.142.130
state: ANOMALOUS
category: modelserverdev
facility: user
username: john.doe@devo.com
geolocation: 40°47'15.36529541015625"N+74°3'35.9912109375"W
timestamp: 2019-10-23+18:17:59.794
DevoTableLink
Devo Direct Link

3. devo-multi-table-query

Queries multiple tables for a given token and returns relevant results.

This method is used for when you do not know which columns a specified search token will show up in (Needle in a haystack search) Thus querying all columns for the search token and returning a union of the given tables.

Required Permissions

A Demisto instance configured with the correct OAuth token that has permission to query the target tables

Base Command

devo-multi-table-query

Input
Argument NameDescriptionRequired
tablesList of table names to check for searchTokenRequired
searchTokenString that you wish to search for in given tables in any columnRequired
fromStart time in seconds unix timestampRequired
toEnd time in seconds unix timestampOptional
writeToContextwrite results to context or notOptional

#####from and to time note: This integration allows for the following formats. Note that when from and to times are both given that they must be the same given format.

  • When from is a date range such as "1 day", "30 minute", etc... to is not needed and will be ignored even if given.
  • Unix timestamps in millis and seconds are accepted.
  • Datetime strings in the format '%Y-%m-%dT%H:%M:%S' are accepted.
  • Python datetime objects are accepted as well.
  • Unsupported formats will error out.
Context Output
PathTypeDescription
Devo.MultiResultslist of dictionariesList of dictionary results
Command Example
!devo-multi-table-query tables='["siem.logtrust.web.activity", "siem.logtrust.web.navigation"]' searchToken="john@doe.com" from=1576845233.193244 to=1576845293.193244
Human Readable Output

Devo multi-query results

ispserverPortsrcPortresponseTimeheaderseventdatecorrelationIduserEmailresponseLengthmessageresultmethodtypeurluseridlevelrefererusernameregionuserAgentsessionidresourceInfocontentLengthorgdomainsrcHostcityparamsserverHosterrorInfosectionactionorigincountrylocale
Amazon.com8080335222019-09-18T07:58:39.691000john@doe.com0https://us.devo.com/alerts/view.json400d338d-c9a6-4930-90a5-357937f3e735https://us.devo.com/welcomeVAMozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/1.2.3.4 Safari/537.368723DEE4B38F1056BC738760B5E79FD3Amazon.comhelloworld1.2.3.4Ashburnus.devo.comalertindexundefinedUS
Amazon.com8080335322019-09-18T07:58:40.789000john@doe.com0https://us.devo.com/generic/storedSearchAction.streamjson400d338d-c9a6-4930-90a5-357937f3e735https://us.devo.com/welcomeVAMozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/1.2.3.4 Safari/537.368723DEE4B38F1056BC738760B5E79FD3Amazon.comhelloworld1.2.3.4Ashburnus.devo.comstored_continuum_searchcreateundefinedUS
Amazon.com8080335382019-09-18T07:58:40.801000john@doe.com0https://us.devo.com/generic/storedSearchAction.streamjson400d338d-c9a6-4930-90a5-357937f3e735https://us.devo.com/welcomeVAMozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/1.2.3.4 Safari/537.368723DEE4B38F1056BC738760B5E79FD3Amazon.comhelloworld1.2.3.4Ashburnus.devo.comstored_continuum_searchcreateundefinedUS
Amazon.com8080335742019-09-18T07:58:41.685000john@doe.comUserDomain: UserDomain[id: 2942, domain: 6ab72601-e982-4694-8ce6-3d526047f8a5/helloworld, roles: null, logged: 2019-09-18 04:32:58.0, status: 0, creation date: 2018-11-05 14:23:44.0, update date: 2019-09-18 04:32:58.0]|0https://us.devo.com/lxcWidgets/lxcWidget.json400d338d-c9a6-4930-90a5-357937f3e735https://us.devo.com/welcomeVAMozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/1.2.3.4 Safari/537.368723DEE4B38F1056BC738760B5E79FD3Amazon.comhelloworld1.2.3.4Ashburnus.devo.comlxc_widgetsindexundefinedUS
Comcast Cable808037094452019-09-18T08:08:21.593000124OKGEThttps://us.devo.com/alerts/alertsGlobe.json400d338d-c9a6-4930-90a5-357937f3e735INFOhttps://us.devo.com/welcomejohn@doe.comCAMozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/1.2.3.4 Safari/537.368723DEE4B38F1056BC738760B5E79FD30Comcast Cablehelloworld1.2.3.4San Francisco{origin:menu.alerts,serialNumber:b181cf08-14e0-49c2-826b-e4ff36afaa84,}us.devo.comUSen_US
Comcast Cable808037092782019-09-18T08:08:21.625000119OKGEThttps://us.devo.com/domain/notification.json400d338d-c9a6-4930-90a5-357937f3e735INFOhttps://us.devo.com/welcomejohn@doe.comCAMozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/1.2.3.4 Safari/537.368723DEE4B38F1056BC738760B5E79FD30Comcast Cablehelloworld1.2.3.4San Francisco{origin:menu.alerts,serialNumber:b181cf08-14e0-49c2-826b-e4ff36afaa84,}us.devo.comUSen_US
Comcast Cable808037196102019-09-18T08:08:51.563000124OKGEThttps://us.devo.com/alerts/alertsGlobe.json400d338d-c9a6-4930-90a5-357937f3e735INFOhttps://us.devo.com/welcomejohn@doe.comCAMozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/1.2.3.4 Safari/537.368723DEE4B38F1056BC738760B5E79FD30Comcast Cablehelloworld1.2.3.4San Francisco{origin:menu.alerts,serialNumber:b181cf08-14e0-49c2-826b-e4ff36afaa84,}us.devo.comUSen_US
Comcast Cable808037194332019-09-18T08:08:51.583000119OKGEThttps://us.devo.com/domain/notification.json400d338d-c9a6-4930-90a5-357937f3e735INFOhttps://us.devo.com/welcomejohn@doe.comCAMozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/1.2.3.4 Safari/537.368723DEE4B38F1056BC738760B5E79FD30Comcast Cablehelloworld1.2.3.4San Francisco{origin:menu.alerts,serialNumber:b181cf08-14e0-49c2-826b-e4ff36afaa84,}us.devo.comUSen_US

4. devo-write-to-table

Write records to a specified Devo table

The records written to the table should all be of the same JSON format and to the same table. We currently do not support writing to multiple tables in a single operation.

For more information on the way we write to a table please refer to this documentation found HERE

Required Permissions

A Demisto instance configured with the correct write JSON credentials

Base Command

devo-write-to-table

Input
Argument NameDescriptionRequired
tableNameTable name to write toRequired
recordsRecords to write to given tableNameRequired
Context Output
PathTypeDescription
Devo.RecordsWrittenintCount of records written to Devo
Devo.LinqQuerystringLinq query that is to be used to see your data in Devo
Devo.QueryLinkstringLink back to Devo table for executed query
Command Example
!devo-write-to-table tableName="my.app.demisto.test" records='[{"hello": "world"}, {"hello": "demisto"}]'
Human Readable Output

Entries to load into Devo

hello
world
demisto

Link to Devo Query

DevoTableLink
Devo Direct Link

5. devo-write-to-lookup-table

Writes a record to a given lookup table

For more information on lookup tables please refer to documentation found HERE. We can add extra records with incremental lookup additions. Please refer to our Python SDK for more information on how we are adding in extra lookup information found HERE

Required Permissions

A Demisto instance configured with the correct write JSON credentials

Base Command

devo-write-to-lookup-table

Input
Argument NameDescriptionRequired
lookupTableNameLookup table name you are trying to write toRequired
headersHeaders of records to upload. Order sensitive.Optional
recordsLookup table records to insertRequired
Context Output
PathTypeDescription
Devo.RecordsWrittenintCount of records written to Devo
Command Example
!devo-write-to-lookup-table lookupTableName="lookup123" headers='["foo", "bar", "baz"]' records='[{"key": "foo1", "values": ["foo1", "bar1", "baz1"]}]'
Human Readable Output

N/A

Additional Information

Youtube Video Demo (Click Image, Will redirect to youtube)

Devo-Demisto Plugin Demo

Known Limitations

  • Currently the lookup table functionality is in Alpha. Please use at your own risk as behavior is still not fully stable.
  • It is up to the user to make sure your demisto instance can handle the amount of data returned by a query.
Edit this page
Report an Issue