Asset Enrichment

The basic steps to import an asset are as follows:

These steps are described in detail below.

You need to create a new asset source or configure an existing source before you can import assets. If you use an existing source, you can edit it by clicking the ellipsis on the source, then selecting Edit from the dropdown.

To create a new asset source:

Configuration options are different based on the source type. Configuration for LDAP and Active Directory follow the same options, while Microsoft 365 configuration is a different set of options. Skip to the section below for the type of source you are configuring.

After setting up the asset connection, you can create multiple queries to import specific subsets of data. For example, you can import "just the admin users" or "just the laptops." This filtering is achieved by defining one or more import mapping configurations that determine which assets are imported from the parent source and how entries in the source map to imported assets in the Security Data Lake asset schema.

After an asset connection is established, you have access to all existing mapping configurations. You can then continue with the configuration wizard to define mapping configurations.

A mapping configuration allows you to define the specific assets that you want to import from a source. It also gives you the option to determine what default values are applied.

For example, you can have an LDAP server with two different mappings. One mapping is configured to select only admin users and has an admin category as well as a high priority category. When you import admin users via this mapping, all of the assets have the same priority and category.

You can set up another mapping that selects general users and uses a medium or low priority. You can set up a high priority mapping for accounting machines and a low priority mapping for user laptops. Configuration parameters can be applied to machine assets and user assets.

You can also configure an asset source sync interval on the mapping configuration page.

Mappings can also be edited or deleted on the Sources page by selecting an asset source to reveal carousel cards that represent each mapping.

When both connection and mapping configurations are saved, you can initiate an asset import. Click the Actions button, then select Import from the dropdown. This action pulls the targeted entries from the asset source, maps them based on the mapping configuration, and creates assets in Security Data Lake that can be viewed from the Assets page.

Click a source on the Sources page to view or edit its configuration.

When you import or sync an asset, it's name remains the same as in the source. Any subsequent imports or syncs match existing assets by name and update all of the details from the backend. This way, the asset remains constant and searching is unaffected.

The asset source sync functionality performs the same actions as asset import. An update in the source is automatically reflected in Security Data Lake via the asset sync. This can be either an update in the source or removal of an asset or assets. All changes made in the source are reflected in Security Data Lake.

You can schedule the import of assets by defining an interval. Asset source sync is available for all mappings listed under a source. Select a source to view the available mappings.

To enable asset source sync:

The default sync interval is in hours and can be modified.

Security Data Lake includes two roles specific to managing or working with assets:

The following permissions are included in the above roles and are required for the management of asset sources and mappings as follows:

The Asset Reader role includes only the asset: read permission, while Asset Manager includes all of these permissions.

You can create a new asset through the Security Data Lake Security user interface manually. To do so:

The Config menu found on the Assets page gives you the ability to manage asset priorities and categories. Each asset can be assigned a priority and multiple categories.

This pipeline rule populates the associated_assets and associated_asset_categories fields on the message using GIM schema fields. The associated assets field will be an array with the ID of each asset that matches a field on the message to an asset field of the same type. The associated_assets field will determine which assets are displayed on the expanded log message on the search page. The associated_asset_categories field will be an array that includes all associated asset categories.

This pipeline rule takes a lookup_type and value parameter. lookup_type can be either name, ip, mac, or hostname. The value field is the value used to look up the specified type. The rule assumes that the lookup will be unique, so if multiple assets happen to match the lookup, only one will be returned. If the lookup has a match, it will return a map with the following structure:

{
  "id": "string"
  "name": "string",
  "priority": number,
  "category": ["string", "array"],
  "details": {
    "type": "machine",
    "description": "string",
    "owner": "string",
    "ip_addresses": ["string", "array"],
    "mac_addresses": ["string", "array"],
    "hostnames": ["string", "array"],
    "custom_fields": Map
  }
}

The custom_fields map structure will depend on the custom fields defined for a given asset. Each entry will have a string key and an array of values that are either strings, dates, or numbers.

These fields can then be used to enrich the message in a more targeted way than the set_associated_assets rule. For example, to look up an asset by the source_ip field and then set fields on the message based on the asset returned would look similar to:

rule "machine_asset_lookup"
when
    has_field("source_ip")
then
    let asset = machine_asset_lookup(lookup_type:"ip", value:to_string($message.source_ip));
    let details = asset.details;
    set_field("asset_id", asset.id);
    set_field("asset_name", asset.name);
    set_field("asset_description", details.description);
    set_field("asset_ips", details.ip_addresses);
    set_field("asset_macs", details.mac_addresses);
    set_field("asset_hostnames", details.hostnames);
end

This rule will update IP addresses and hostnames of existing machine assets. The parameters are:

rule "machine_asset_update"
when
    true
then
    machine_asset_update(lookup_type:"mac", lookup_value:"AA:BB", ip_addresses:"10.0.0.0");
end

Using this rule on DHCP logs, for example, can keep existing assets up to date based on incoming logs.

rule "machine_asset_update"
when
    true
then
    machine_asset_update(lookup_type:"mac", lookup_value:$message.mac, ip_addresses:to_string($message.new_ip));
end

This pipeline rule is used to look up a user asset and enrich log messages with user asset data. The rule takes a lookup_type and value parameter. lookup_type can be either name, username, user_id, or email. The value field is the value to be used in looking up the specified type. The rule assumes that the lookup will be unique, so if multiple assets happen to match the lookup, only one will be returned. If the lookup has a match, it will return a map with the following structure:

{
  "id": "string"
  "name": "string",
  "priority": number,
  "category": ["string", "array"],
  "details": {
    "type": "user",
    "description": "string",
    "username": "string",
    "user_ids": ["string", "array"],
    "email_addresses": ["string", "array"],
    "first_name": "string",
    "last_name": "string"
  }
}

These fields can then be used to enrich the message in a more targeted way than the set_associated_assets rule. For example, looking up an asset by the username field and then set fields on the message based on the asset returned would look similar to:

rule "user_asset_lookup"
when
    has_field(“username”)
then
    let asset = user_asset_lookup(lookup_type:"username", value:"username");
    let details = asset.details;
    set_field("asset_id", asset.id);
    set_field("asset_name", asset.name);
    set_field("asset_type", details.type);
    set_field("asset_username", details.username);
    set_field("asset_user_ids", details.user_ids);
    set_field("asset_emails", details.email_addresses);
end

When the Illuminate pack is activated, Security Data Lake automatically runs the set_associated_assets pipeline rule for all messages.

However, you have the option to specifically apply the set_associated_assets pipeline rule for a subset of logs. In this case, you would not enable the Illuminate assets processing pack but rather would implement the set_associated_assets pipeline function by putting the set_associated_assets in a pipeline rule, adding it to a pipeline, setting filters, and assigning it to selected streams.

This functionality is customizable to the extent that you can either apply it to all logs by enabling the Illuminate processing pack or to a subset of logs by manually configuring and applying the rule. Refer to our documentation on pipelines for more information on creating pipeline rules.

Once a message has been enriched with the associated_assets fields, those fields can be displayed in the expanded log view of an individual message. The details of each associated asset can then be expanded as well.

In addition to viewing the asset in search results, you also have the capability to add an asset to the search query and to pivot into any log message associated with that asset.

This capability allows you to view an asset in search results and transition to exploring additional logs related to that specific asset for further investigation. For instance, if a log message identifies a finance department's computer as an asset, you can pivot and access all logs associated with that machine, allowing for a deeper delve into its activities.

To pivot to an asset search, click the Add to query button for a particular asset to view all log messages for that asset. Note that the asset ID is then added to the search query for the associated_assets field.

On the Assets page, you can search for assets. This functionality allows you to create a search query based on asset information. You can search for assets individually or in bulk.

To search for an individual asset, click the ellipsis for the selected asset and choose the Search for asset option.

To perform bulk searches, choose the assets you want to search, then click on the Bulk Actions button and search for the selected assets.

Vulnerability scan data applied to machine assets helps provide a more complete view of potential threats in your environment. Vulnerability data can be useful in the following areas:

For examples of this integration, see the documentation on risk scoring use cases.

Before you can include vulnerability scan data in Security Data Lake, you must have your own third-party scanner set up and connected to Security Data Lake. Security Data Lake does not perform scans but instead imports scan data from your configured scanner or scanners.

You can use the following types of vulnerability scanners with Security Data Lake:

You can add multiple scanners for each type so that you can have scan data focused on different areas of your network or types of scans. See the setup information for each scanner type for complete details.

The Vulnerability Scanners tab on the Assets page lists all the vulnerability scanners you have defined for your environment. Click a scanner to view its detail page that displays the scanner's settings and connection information.

From the detail screen, click Import Scans to perform a manual import of new scan data for the selected scanner. To update scanner settings, click Edit Connection.

If you select the check boxes for one or more scanners, the Bulk Actions menu becomes available with the following actions: