Skip to main content

RSA NetWitness Packets and Logs

This Integration is part of the RSA NetWitness Packets and Logs Pack.#

Overview

Use the RSA NetWitness Logs and Packets integration to collect and analyze network data.

This API enables additional low-level functionality for managing various system components accessible on Cortex XSOAR. Each command has several options that analyze and query NetWitness P&L packets, sessions, and logs. You can use the results as tasks in playbooks for automated analysis, or view them in the War Room.

This integration was integrated and tested with NetWitness v11.1.0.0.

Use Cases

  • Collect network data in real time for analysis.
  • Dump information for backup and duplication.

Prerequisites

Verify that you satisfy these requirements in RSA.

  • Obtain RSA NetWitness credentials.
  • Enable the RESTful API that ships with RSA Security Analytics.
    • Configure rest/config/enabled to on. This is the default.
    • The default port for communication is the default port + 100 (for example, 50105 for a Concentrator). You can change the port by setting the /rest/config/port parameter. SSL is controlled by the setting in /sys/config/ssl .

Configure RSA NetWitness Logs and Packets on Cortex XSOAR

  1. Navigate to Settings > Integrations > Servers & Services.
  2. Search for RSA NetWitness Packets and Logs.
  3. Click Add instance to create and configure a new integration.
    • Name : a textual name for the integration instance.
    • Appliance IP / Hostname
    • Appliance Port: Logs(50102), Packets(50104), Concentrator (50105), Broker (50103)
    • Credentials: NetWitness user credentials
    • Secure (over HTTPS) - wheter to send requests over SSL
    • Use system proxy settings
    • Expiration time
  4. Click Test to validate the URLs, token, and connection.

Commands

You can execute these commands from the Cortex XSOAR CLI, as part of an automation, or in a playbook. After you successfully execute a command, a DBot message appears in the War Room with the command details.

  1. Search pattern matches: netwitness-msearch
  2. Search: netwitness-search
  3. Query: netwitness-query
  4. Stream packets: netwitness-packets
  5. Get meta ID range: nw-sdk-session
  6. Get packet content: nw-sdk-content
  7. Get summary: nw-sdk-summary
  8. Get value count: nw-sdk-values
  9. Dump information: nw-database-dump

Search pattern matches

Search for pattern matches in multiple sessions or packets.

Base Command

netwitness-msearch

Input
Parameter Type Description
sessions string Search within these session ID ranges
packets string Search within these packet ID ranges
search string

String to search for. Can be a simple string or a query. For regular expressions, change the word “keyword” to “regex”.

Example: "select hit, pretext, posttext where keyword=’netwitness’ sp ci ds". sp = Search Packets ci = Case Insensitive ds = Decode Sessions (for example, convert email attachments before searching)

where string

Filter criteria for which sessions to search

limit uint64 Maximum number of sessions to search in this instance
flags string

Comma-separated list of one or more flag values.

Example: regex, sp, sm, si, ci, pre, post, ds

concentratorIP integer

Use the instance that is mapped to this concentrator IP

concentratorPort integer

Port to use with concentratorIP. 50105 (SSL/NON-SSL) for concentrator, 50103 (SSL/NON-SSL) for broker.

useSSL boolean

Send request in HTTPS instead of HTTP. By default the parameter is set to True but if your appliance is not configured to use SSL you can change it here.

Context Data
Path Description
NetWitness.SearchHits.hit All matching search results in NetWitness

Command Example

!netwitness-msearch search="ab" limit="10" flags="sp,ci"

Raw Output 
[  
   {  
      "flags":1074200578,
      "results":{  
         "fields":[  
            {  
               "count":0,
               "flags":0,
               "format":65,
               "group":1,
               "id1":1,
               "id2":1,
               "type":"hit",
               "value":"ab"
            }
         ],
         "id1":1,
         "id2":1
      }
   },
   {  
      "flags":1074200578,
      "results":{  
         "fields":[  
            {  
               "count":0,
               "flags":0,
               "format":65,
               "group":2,
               "id1":2,
               "id2":2,
               "type":"hit",
               "value":"ab"
            }
         ],
         "id1":2,
         "id2":2
      }
   },
   {  
      "flags":1074200578,
      "results":{  
         "fields":[  
            {  
               "count":0,
               "flags":0,
               "format":65,
               "group":10,
               "id1":10,
               "id2":10,
               "type":"hit",
               "value":"ab"
            }
         ],
         "id1":1,
         "id2":1
      }
   },
   {  
      "flags":1074200578,
      "results":{  
         "fields":[  
            {  
               "count":0,
               "flags":0,
               "format":65,
               "group":3,
               "id1":3,
               "id2":3,
               "type":"hit",
               "value":"ab"
            }
         ],
         "id1":3,
         "id2":3
      }
   },
   {  
      "flags":1074200578,
      "results":{  
         "fields":[  
            {  
               "count":0,
               "flags":0,
               "format":65,
               "group":4,
               "id1":4,
               "id2":4,
               "type":"hit",
               "value":"ab"
            }
         ],
         "id1":4,
         "id2":4
      }
   },
   {  
      "flags":1074200578,
      "results":{  
         "fields":[  
            {  
               "count":0,
               "flags":0,
               "format":65,
               "group":7,
               "id1":7,
               "id2":7,
               "type":"hit",
               "value":"ab"
            }
         ],
         "id1":7,
         "id2":7
      }
   },
   {  
      "flags":1074200578,
      "results":{  
         "fields":[  
            {  
               "count":0,
               "flags":0,
               "format":65,
               "group":8,
               "id1":8,
               "id2":8,
               "type":"hit",
               "value":"ab"
            }
         ],
         "id1":8,
         "id2":8
      }
   },
   {  
      "flags":1074200577,
      "results":{  
         "fields":[  

         ],
         "id1":11,
         "id2":10
      }
   }
]
War Room Output

Search

Deprecated. See Search Pattern Matches.

Base Command

netwitness-search

Input
Parameter Type Description
sessions string Search within these session ID ranges
size

uint32

Maximum number of entries to return (range: 1-1677721)

search string

String to search for

concentratorIP string

Use the instance that is mapped to this concentrator IP

concentratorPort uint32

Port to use with concentratorIP. 50105 (SSL/NON-SSL) for concentrator, 50103 (SSL/NON-SSL) for broker.

useSSL boolean

Send request in HTTPS instead of HTTP. By default the parameter is set to True but if your appliance is not configured to use SSL you can change it here.

Context Data
Path Description
NetWitness.SearchHits All matching search results in NetWitness

Query

Performs a query against the meta database.

Base Command

netwitness-query

Input
Parameter Type Description
id1 uint64

First Meta ID (to run the query from most recent to oldest meta, make id1 larger than id2)

id2 uint64

Final Meta ID

size uint32

Maximum number of entries to return

Default: All results

query string

Query string to use

flags string

Flags to use for query. Can be a number (bitwise mask) or comma-separated values like query-log

threshold uint64

Stops processing results after the threshold is reached (useful with select aggregate functions)

Default: No threshold

lastHours uint64

Filter for date and time

lastMinutes uint64

Filter by last minute

concentratorIP string

Use the instance that is mapped to this concentrator address

concentratorPort uint32

Port to use with concentratorIP. 50105 (SSL/NON-SSL) for concentrator, 50103 (SSL/NON-SSL) for broker.

useSSL boolean

Send request in HTTPS instead of HTTP. By default the parameter is set to True but if your appliance is not configured to use SSL you can change it here.

Context Data
Path Description
NetWitness.Events.ipSrc Source IP
NetWitness.Events.aliasIp Destination IP
NetWitness.Events.aliasHost Hostname or FQDN
NetWitness.Events.ipDst IP Destination of event
NetWitness.Events.userSrc Source user
NetWitness.Events.userDst Destination user
NetWitness.Events.username Includes any available username metadata (source and destination) as defined by the user. This is an optional field which only appears if configured by the end user.
NetWitness.Events.service The service that created the session (if available)
NetWitness.Events.sessionid The NW Specific field that contains an NW specific ‘serial number’ of the Session - allows a user to specify a session to retrieve from a Concentrator
NetWitness.Events.deviceType System type that generated a log message (for example, SQUID, Windows, rhlinux)
NetWitness.Events.direction Traffic flow direction (for example, inbound/outbound)
NetWitness.Events.did Specific decoder a session was captured on
NetWitness.Events.ioc Endpoint specific metadata that describes malware on an endpoint

Command Example

!netwitness-query size="5" query="select*where alias.host contains 'skimresources'"

Raw Output 
{  
   "flags":1074200577,
   "results":{  
      "fields":[  
         {  
            "count":0,
            "flags":0,
            "format":8,
            "group":13,
            "id1":504,
            "id2":504,
            "type":"sessionid",
            "value":"13"
         },
         {  
            "count":0,
            "flags":0,
            "format":32,
            "group":13,
            "id1":505,
            "id2":505,
            "type":"time",
            "value":1372882420
         },
         {  
            "count":0,
            "flags":0,
            "format":6,
            "group":13,
            "id1":506,
            "id2":506,
            "type":"size",
            "value":"16452"
         },
         {  
            "count":0,
            "flags":0,
            "format":6,
            "group":13,
            "id1":507,
            "id2":507,
            "type":"payload",
            "value":"13590"
         },
         {  
            "count":0,
            "flags":0,
            "format":2,
            "group":13,
            "id1":508,
            "id2":508,
            "type":"medium",
            "value":"1"
         }
      ],
      "id1":509,
      "id2":3938
   }
}
War Room Output

More Query Command Examples:

!netwitness-query size="5" query="select*where device.class contains 'Anti Virus'"

!netwitness-query size="5" query="select*where sessionid=1831740465537"


Stream packets

Streams packets back based on the input parameters provided.

Base Command

netwitness-packets

Input
Parameter Type Description
op string

The operation to perform (start, cancel, processed)

{enum-one:start|cancel|processed}

sessions string

A comma-separated list of session IDs or session ID ranges (#-#) whose packets will be streamed back

packets string

A comma-separated list of packet IDs or session IDs (#&#) which will be streamed back

where string

The sessions to stream back

time1 date-time

Starting time (UTC) for when matching packets are streamed back ("2010-Apr-20 09:00:00")

time2 date-time

Ending time (UTC) for when matching packets are streamed back ("2010-Apr-20 10:00:00")

flags uint32 Additional flags as defined by the NwPackets SDK function
concentratorIP string Use the instance that is mapped to this concentrator address
concentratorPort uint32

Port to use with concentratorIP. 50105 (SSL/NON-SSL) for concentrator, 50103 (SSL/NON-SSL) for broker.

useSSL boolean

Send request in HTTPS instead of HTTP. By default the parameter is set to True but if your appliance is not configured to use SSL you can change it here.

render string

Render type

Examples: pcap, logs, logs (csv), logs (xml), or logs (json)

renderToContext boolean

If renderToContext is set to True and the render type is set to logs (json), the command will not create a file with the output. It will output to the context data.

Context Data

There is no context data for this command.

Command Example

!netwitness-packets sessions="1,2,3,4,5"

Raw Output

There is no raw output for this command.

War Room Output

Get meta ID range

Returns the meta ID range for the session range.

Basic Command

nw-sdk-session

Input
Parameter Type Description
id1 uint64

Starting session ID

id2 uint64

Ending session ID

concentratorIP string

Use the instance that is mapped to this concentrator IP

concentratorPort uint32

Port to use with concentratorIP. 50105 (SSL/NON-SSL) for concentrator, 50103 (SSL/NON-SSL) for broker.

useSSL boolean

Send request in HTTPS instead of HTTP. By default the parameter is set to True but if your appliance is not configured to use SSL you can change it here.

Context Data

There is no context data for this command.

Command Example

!nw-sdk-session id1=1 id2=10

Raw Output 
{  
   "flags":1073938433,
   "params":{  
      "field1":"1",
      "field2":"334",
      "id1":"1",
      "id2":"10"
   }
}
War Room Output

Get packet content

Returns the packet content for a session.

Base Command

nw-sdk-content

Input
Parameter Type Description
concentratorIP string

Use the instance that is mapped to this concentrator IP

concentratorPort uint32

Port to use with concentratorIP. 50105 (SSL/NON-SSL) for concentrator, 50103 (SSL/NON-SSL) for broker.

useSSL boolean

Send request in HTTPS instead of HTTP. By default the parameter is set to True but if your appliance is not configured to use SSL you can change it here.

session uint64

Session ID to return.

renderType string

Render type for content (defined in NwSDK.h)

OR

One of the following render types:

text, hex, packets, web, mail, raw, rtp, voip, meta, im, files or pcap

maxSize uint64

Maximum size of the session returned (bytes). Zero is unlimited.

Default is zero.

base64 binary

If 1, encode the multipart/mixed response as base64

Default is binary

flags string

Flags for content rendering, see the SDK header file NwContent flags parameter for more info (Optional)

includeFileTypes string

Semicolon-separated list of file extensions to extract

Example: .ext1;.ext2

excludeFileTypes string

Semicolon-separated list of file extensions to exclude

Example: .ext1;.ext2

renameFileTypes string

Renames certain files from one or more extensions to another

Example: renameFileTypes=.download|.octet-stream|.program|.exe;.jpeg|.jpg

fileExt string

Extension of the saved file

where string

The sessions to stream back

subdirFileTypes string

Groups files into the specified directory

Example: documents=.doc;.xls;.pdf; images=.png;.gif;.jpg;.jpeg;

maxDirSize uint32

Deletes oldest files, when this value is exceeded by top-level directory

renderFlags uint32

Bitwise mask to control options, (see NwContentFlags in NwSDK.h)

(Optional)

Context Data

There is no context data for this command.

Command Example

!nw-sdk-content session="23" renderType="mail" fileExt=".zip"

Raw Output

There is no raw output for this command.

War Room Output

Get summary

Returns summary information from the databases.

Base Command

nw-sdk-summary

Input
Parameter Type Description
flags string

SDK flags - can be a number (bitwise mask) or comma-separated values like default or ignore-cache.

concentratorIP string

Use the instance that is mapped to this concentrator IP

concentratorPort uint32

Port to use with concentratorIP. 50105 (SSL/NON-SSL) for concentrator, 50103 (SSL/NON-SSL) for broker.

useSSL boolean

Send request in HTTPS instead of HTTP. By default the parameter is set to True but if your appliance is not configured to use SSL you can change it here.

Context Data

There is no context data for this command.

Command Example

!nw-sdk-summary

Raw Output 
{  
   "flags":1073872897,
   "string":"mid1=1 mid2=3938 msize=106496 mmax=142947249029 pid1=0 pid2=0 psize=0 pmax=0 time1=336000 time2=1522264920 ptime1=0 ptime2=0 sid1=1 sid2=65 ssize=4096 smax=142947249029 stotalsize=12787062 isize=195854392 memt=8371580928 memu=3604697088 memp=213417984 hostname=NWAPPLIANCE14623 version=11.1.0.0"
}
War Room Output

Get value count

Performs a value count query and returns the matching values in a report.

Basic Command

nw-sdk-values

Input
Parameter Type Description
id1 uint64

Starting meta ID

id2 uint64

Ending meta ID

size uint32

Max number of entries to return

{range:1 to 1677721}

flags string

The flags to use for values. Can be a number (bitwise mask) or comma-separated values (for example, sessions, size, packets, aggregate, sort-total, sort-value, order-ascending, order-descending, ignore-cache, clear-cache, query-log or database-scan)

threshold uint64

Query optimization to stop processing large session counts

fieldName

Field to retrieve values for

where string

Filter criteria for the values

aggregateFunction string

The meta field to aggregate in the aggregateFunction

min string

Limits the returned values to values higher than this

max string

Limits the returned values to values lower than this

concentratorIP string

Use the instance that is mapped to this concentrator IP

concentratorPort uint32

Port to use with concentratorIP. 50105 (SSL/NON-SSL) for concentrator, 50103 (SSL/NON-SSL) for broker.

useSSL boolean

Send request in HTTPS instead of HTTP. By default the parameter is set to True but if your appliance is not configured to use SSL you can change it here.

Context Data
Path Description
NetWitness.Values.ipSrc Source IP
NetWitness.Values.aliasIp Destination IP
NetWitness.Values.aliasHost Hostname or FQDN
NetWitness.Values.ipDst IP Destination of the event
NetWitness.Values.userSrc Source user
NetWitness.Values.userDst Destination user
NetWitness.Values.username Includes any available username meta – both source and destination – as defined by the user. This is an optional field, which only appears if configured by the end user.
NetWitness.Values.service The service that created the session (if available)
NetWitness.Values.sessionid The NW Specific field that contains an NW specific ‘serial number’ of the session. Allows a user to specify a session to retrieve from a Concentrator
NetWitness.Values.deviceType The type of system that generated a log message (ex. SQUID / Windows / rhlinux)
NetWitness.Values.direction Traffic flow direction (for example, inbound/outbound)
NetWitness.Values.did Specific decoder a session was captured on
NetWitness.Values.ioc Endpoint specific meta which describes malware on an endpoint

Command Example

!nw-sdk-values size="10" fieldName="client"

Raw Output 
{  
   "flags":1074200577,
   "results":{  
      "fields":[  
         {  
            "count":1,
            "flags":0,
            "format":65,
            "group":0,
            "id1":25,
            "id2":25,
            "type":"client",
            "value":"opera mail/12.11"
         },
         {  
            "count":5,
            "flags":0,
            "format":65,
            "group":0,
            "id1":13,
            "id2":63,
            "type":"client",
            "value":"mozilla/5.0"
         },
         {  
            "count":31,
            "flags":0,
            "format":65,
            "group":0,
            "id1":14,
            "id2":61,
            "type":"client",
            "value":"mozilla/4.0"
         },
         {  
            "count":2,
            "flags":0,
            "format":65,
            "group":0,
            "id1":5,
            "id2":6,
            "type":"client",
            "value":"e1e8d428-5bf1-4323-8808-d138a039102f"
         }
      ],
      "id1":0,
      "id2":0
   }
}
War Room Output

Dump information

Dumps information out of the database in nwd formatted files.

Base Command

nw-database-dump

Input
Parameter Type Description
session uint64

ID of the session to dump

type string

Dump type

{enum-one:db|nwd}

source string

Types of data to dump

Default is all data

{enum-any:s|m|p}

verbose boolean

Default is false

If true - dumps more information

file string

Filename to use for NWD type

Default: sessionid .nwd

concentratorIP string

Use the instance that is mapped to this concentrator IP

concentratorPort uint32

Port to use with concentratorIP. 50105 (SSL/NON-SSL) for concentrator, 50103 (SSL/NON-SSL) for broker.

useSSL boolean

Send request in HTTPS instead of HTTP. By default the parameter is set to True but if your appliance is not configured to use SSL you can change it here.

Context Data

There is no context data for this command.

Command Example

!nw-database-dump session="1" source="m"

Raw Output 
{  
   "flags":1074003969,
   "params":[  
      {  
         "MetaArray":"31",
         "dbFile":"/var/netwitness/concentrator/metadb/meta-000000001.nwmdb"
      },
      {  
         "meta.id":"1",
         "session":"1",
         "type":"sessionid",
         "value":"1"
      },
      {  
         "meta.id":"2",
         "session":"1",
         "type":"time",
         "value":"2018-Mar-27 16:05:59"
      },
      {  
         "meta.id":"3",
         "session":"1",
         "type":"size",
         "value":"1468"
      },
      {  
         "meta.id":"4",
         "session":"1",
         "type":"medium",
         "value":"32"
      },
      {  
         "meta.id":"5",
         "session":"1",
         "type":"device.type",
         "value":"rsaflow"
      },
      {  
         "meta.id":"6",
         "session":"1",
         "type":"device.class",
         "value":"Switch"
      },
      {  
         "meta.id":"7",
         "session":"1",
         "type":"alias.host",
         "value":"xxx.xxx.x.xxx"
      },
      {  
         "meta.id":"8",
         "session":"1",
         "type":"version",
         "value":"9"
      },
      {  
         "meta.id":"9",
         "session":"1",
         "type":"event.type",
         "value":"flowdata"
      },
      {  
         "meta.id":"10",
         "session":"1",
         "type":"event.desc",
         "value":"flowdata"
      },
      {  
         "meta.id":"11",
         "session":"1",
         "type":"ip.src",
         "value":"xxx.xxx.x.xxx"
      },
      {  
         "meta.id":"12",
         "session":"1",
         "type":"netname",
         "value":"private src"
      },
      {  
         "meta.id":"13",
         "session":"1",
         "type":"ip.dst",
         "value":"xxx.xxx.x.xxx"
      },
      {  
         "meta.id":"14",
         "session":"1",
         "type":"netname",
         "value":"other dst"
      },
      {  
         "meta.id":"15",
         "session":"1",
         "type":"direction",
         "value":"outbound"
      },
      {  
         "meta.id":"16",
         "session":"1",
         "type":"ip.proto",
         "value":"6"
      },
      {  
         "meta.id":"17",
         "session":"1",
         "type":"ip.dstport",
         "value":"80"
      },
      {  
         "meta.id":"18",
         "session":"1",
         "type":"tcp.flags",
         "value":"24"
      },
      {  
         "meta.id":"19",
         "session":"1",
         "type":"bytes.src",
         "value":"10"
      },
      {  
         "meta.id":"20",
         "session":"1",
         "type":"eth.src",
         "value":"xxx.xxx.x.xxx"
      },
      {  
         "meta.id":"21",
         "session":"1",
         "type":"eth.dst",
         "value":"xxx.xxx.x.xxx"
      },
      {  
         "meta.id":"22",
         "session":"1",
         "type":"event.name",
         "value":"flowdata"
      },
      {  
         "meta.id":"23",
         "session":"1",
         "type":"direction",
         "value":"inbound"
      },
      {  
         "meta.id":"24",
         "session":"1",
         "type":"event.time",
         "value":"2006-Sep-07 02:42:10"
      },
      {  
         "meta.id":"25",
         "session":"1",
         "type":"bytes",
         "value":"307"
      },
      {  
         "meta.id":"26",
         "session":"1",
         "type":"msg.id",
         "value":"rsaflow"
      },
      {  
         "meta.id":"27",
         "session":"1",
         "type":"event.cat.name",
         "value":"Other.Default"
      },
      {  
         "meta.id":"28",
         "session":"1",
         "type":"device.disc",
         "value":"100"
      },
      {  
         "meta.id":"29",
         "session":"1",
         "type":"sourcefile",
         "value":"drivebyDL_1-1.log"
      },
      {  
         "meta.id":"30",
         "session":"1",
         "type":"did",
         "value":"nwappliance27515"
      },
      {  
         "meta.id":"31",
         "session":"1",
         "type":"rid",
         "value":"1"
      }
   ]
}
War Room Output

Error Responses

Error What to do
No response generated from commands

Make sure the appliance's IP or port are correct.

401 Unauthorized

Make sure the NetWitness user credentials are correct.

400 Bad Request

Make sure the command was sent to the right appliance, and that a compatible argument is attached

404 Not Found

Make sure the appliance supports the executed command

Troubleshooting

When executing the !netwitness-query command and you want to use the sessionid RSA NetWitness meta key as part of the query that is passed to the query argument, please note that for any given session that Decoders, Concentrators, and Brokers appliances all maintain Session IDs which may be different. You can read more about using Session IDs here .

Edit this page
Report an Issue