Illuminate Enrichments
Illuminate functionality includes enriching events with additional data that can help contextualize events. This is done throughout Illuminate, both in the core functionality and in many of the Illuminate processing packs. This article focuses on the enrichments added and used by Illuminate core.
Some of the lookups provided in Illuminate can be customized by the user, as detailed explained in the descriptions of the lookups where it applies.
Severity Mapping Enrichments
Event logs and alerts will often have a severity assigned. Sometimes this event is provided as a plain text severity, and sometimes it is provided as a numeric value. Severity mapping lookups are used to ensure that, where either the text or numeric value is the only value defined, that corresponding value will be defined.
The severity mapping lookups included with Illuminate core:
illuminate-mapping-alert-severity-to-level: When a message includes an
alert_severityvalue but not analert_severity_levelfield, this mapping is used to define the level value.illuminate-mapping-level-to-alert-severity: When a message includes an
alert_severity_levelnumeric value but not analert_severityvalue, this mapping is used to define the severity value.illuminate-mapping-event-severity-to-level: When a message includes an
event_severityvalue but not anevent_severity_levelfield, this mapping is used to define the level value.illuminate-mapping-event-level-to-severity: When a message includes an
event_severity_levelnumeric value but not anevent_severityvalue, this mapping is used to define the severity value.
Warning
These lookups should not be customized.
GIM Data Enrichment
This lookup takes the gim_event_type_code value and defines the following fields:
gim_event_classgim_event_categorygim_event_subcategorygim_event_type
Warning
The GIM event data mapping lookup is core_gim_data_lookup. This lookup should not be customized.
Network Range Enrichment Lookup
Illuminate provides the ability to define category fields for source, destination, and host systems defined in logs. This enrichment will examine the fields source_ip, destination_ip, and host_ip. If the IP address in those fields is in a range that has been added to the network ranges lookup, then any category values defined in that lookup will be assigned to the fields source_category, destination_category, or host_category, respectively.
After installing Security Data Lake Illuminate and enabling any content pack, there will be a lookup added with a lookup adapter titled core_networks_adapter. You can add entries to this lookup adapter and provide one or more category values that will be added to incoming logs.
Defining Categories for Hosts Based On Network Ranges
Warning
Use caution when adding lookup table customization entries for which you are modifying the intended lookup table adapter.
Navigate to Enterprise > Illuminate in the Security Data Lake interface.
Click on the Customization tab located on the top left-hand side of the Illuminate page. Here you will see a list of Illuminate lookup adapters.
Locate the lookup adapter named
core_networks_lookupand select the Edit button for the adapter, located to the right of the adapter entry. You will see an entry box with the key column titled ip range and a value column titled ip category data.Add a CIDR-notation formatted IP range (both IPv4 and IPv6 are supported) in the ip range column.
Add one or more category values, separating each category value with a pipe character (
|), in the ip category data column.
We recommend using only alphanumeric characters and replacing spaces with a replacement character, such as an underscore (_), to simplify searching for these category values.
HTTP Data Enrichment Adapters
These lookups will modify some common HTTP fields with useful data. There are two lookup adapters in this group:
lookup_http_classlookup_http_response
Illuminate will detect when the http_response_code field is defined and enrich these events with two fields:
http_response_class: The response class is based upon the first digit of the HTTP response (RFC2616#6.1.1).http_response: The response phrase corresponds with the response code (RFC2616#6.1.1 and RFC6585).
Network Protocol Enrichments
These lookups will enrich events based on the presence of fields associated with network-related event logs.
Network Protocols
Some network event sources will provide a numeric value representing the IANA-registered protocol number (which should be assigned to the field network_iana_number) but not the protocol name. This enrichment, using a lookup with the lookup adapter named core_iana_to_network_transport, will identify when only network_iana_number is defined and use it to determine the protocol name, which will be assigned to the field network_transport.
Reserved/Non-Routable Address Identification
This lookup will inspect the key fields:
source_ipdestination_iphost_ip
When any of these fields exist, a lookup with the adapter titled core_ip_processing_reserved_ip_ranges_adapter will be used to identify the use of any reserved IP address ranges. The reserved ranges used are:
Range(s) | Tag Suffixes |
|---|---|
192.168.0.0/16, 172.16.0.0/12, 10.0.0.0/8, RFC1918 |
|
127.0.0.0/8, RFC1122: See section 3.2.1.3 regarding loopback addressing |
|
169.254.0.0/16, RFC3927: IPv4 link-local addressing (A.K.A. APIPA addressing) |
|
224.0.0.0/4, RFC3171: IPv4 Multicast addressing |
|
fe80::/10, ff00::/8, ::1/128, ::/128, ::FFFF:0:0/96, RFC4291: See section 2.4, IPv6 Address Architecture |
|
fc00::/7, RFC4193: See section 3.1, IPv6 unique local unicast addressing |
|
2002::/16, RFC3056: IPv6 encapsulation of IPv4 |
|
2001::/32, RFC4380: Teredo |
|
192.0.2.0/24, 198.51.100.0/24, 203.0.113.0/24, RFC5737: IPv4 reserved address block for documentation |
|
198.18.0.0/15, RFC2544: IPv4 reserved addresses for benchmark testing |
|
2001:db8::/32, RFC3849: IPv6 reserved address block for documentation |
|
2001:10::/28, RFC4843: Orchid routing |
|
Illuminate will detect when an IP in one of the key fields is in scope for any of the identified ranges and add values to the gim_tags field. The values added have some common values and some that are specific to the ranges, and these will be prefixed with the context of the key field for which Illuminate has identified. For example, if the value source_ip value is the IP address 192.0.2.10 , Illuminate will add the following values to the gim_tags field:
source_reserved_ipsource_is_illegalsource_rfc5737