Investigations
The following article exclusively pertains to a Security Data Lake Security feature or functionality. Security Data Lake Security is a part of the Security Data Lake centralized log management platform and requires a separate license. Contact the Security Data Lake Sales team for more information on this product.
Security Data Lake Security Investigations provide a solution for analysts to quickly gather and analyze data in real-time, allowing you to view the full context of an issue or threat without spending hours trawling through logs.
With Security Data Lake Investigations, you can:
Create investigations based on timelines, data sets, events, and alerts.
Associate events with investigations
See associated data points grouped in a single location.
Create and reuse investigations to save time and effort.
Quickly narrow down results.
Collaborate with multiple users on investigations by sharing, assigning, and notifying assignees.
How It Works
Investigations are essentially a collection of items within Security Data Lake comprising dashboards, logs, searches, and events (collectively called “evidence”) that are all grouped in one place. You can create a new investigation, update an investigation, and delete or archive investigations.
There are three primary menus on the Investigations page:
Open: Displays all open investigations.
Archived: Displays all archived investigations.
Config: Configuration options for managing priorities and status settings.
Default Priority and Status
On the Config tab, you can designate a default priority and status for new investigations. Hovering to the left of the Edit button on any priority or status will reveal a Set Default button. Clicking this button will designate that priority or status as the default option when creating a new investigation.
Create a New Investigation
The following instructions detail how to manually create a new investigation with Security Data Lake. To automatically generate a new investigation based on an alert from a defined event, see the related documentation.
On the Investigations page, click the New Investigation button, and a window appears, prompting you to fill out details of a new investigation:
Name: Provide a unique name for your new investigation.
Assign To: Select other users or teams to which you can assign the new investigation. Alternatively, you can assign an investigation to yourself. (Note that only users with the Admin, Investigations Manager, or Investigations Reader roles are able to be assigned to investigations.)
Priority: By default, there are four priority types: Low, Medium, High, and Critical. However, you can edit the priority settings via the Config tab on the Investigations page. You can create new priority types, determine the order in which the priorities should rank, and delete as needed.
Status: By default, there are four statuses: Open, Investigating, Closed, and False Positive. You can also edit the status settings via the Config tab on the investigations page. You can create new status types or delete existing ones.
Notes: Here, you can collect ideas, thoughts, and notes connected to an investigation to share with others.
Fill in the details and click Confirm to create a new investigation. The newly created investigation will appear on the Investigations page, and you can start adding evidence to it as you navigate through your data.
Add Evidence to an Investigation
After creating a new investigation, the next step is to add evidence to it. You can add dashboards, logs, saved searches, and events, including events that result from anomaly detection, as evidence entities.
This section details how to add evidence to an investigation manually; however, you may also opt to add evidence to an existing investigation automatically based on an alert created from a defined event. For more information on this process, see Investigations and Alerts.
Saved Searches
You can add saved searches as evidence to investigations directly from the saved search by clicking on the ellipsis icon to the right of the search bar and opting either to add to the active investigation or to another existing investigation.
Note
Both absolute time ranges and relative time ranges may be selected upon saving your search. See the section on relative time ranges for more details.
Dashboards
Dashboards are helpful for visualizing data and understanding trends over time, which can be relevant to any investigation. You can add dashboards as evidence directly from the Dashboards page itself by clicking on the More drop-down action button and selecting either to add to the active investigation or another investigation.
Note that both absolute time ranges and relative time ranges may be displayed in widgets as with saved searches. See the section on relative time ranges for more details.
Logs
Individual logs can often be useful as key pieces of source data for an investigation. Any relevant log can be directly added as evidence by clicking on a search result and from the Investigations sub-menu, selecting either to add to the active investigation or another investigation.
Events
Alerts and events are often the most important pieces of evidence for an investigation. For instance, if an alert for several unsuccessful login attempts is triggered, then you can jump right into an investigation by adding the events as evidence directly from the Events tab under Security Events > Alerts.
Note
Events triggered by your configured anomaly detectors are also available to add as evidence.
Depending on your index rotation and archiving configuration, older logs and events can be removed from an investigation. In order to ensure pieces of evidence attached to investigations are available even after their source data is gone, we created two streams:
All Investigation events: This stream includes all investigation events across all investigations.
All Investigation messages: This stream includes all investigation messages across all investigations.
Any log and event evidence added is duplicated in these streams to preserve them.
Associate Assets with Investigations
Associated assets are pulled from log and event evidence when retrieving individual investigations. Those assets are returned with the investigations in an associated_assets field.
View Investigations on a Timeline
The investigations timeline functionality outlines key events and messages that are part of an investigation. It provides a chronological record to track progress and findings. To view investigation details on a timeline, toggle to the Timeline view found on each individual investigation page.
This view helps you visualize and understand the environment of an alert. For example, you can choose to see events that happened during the past week to gain insight into an ongoing investigation. You may widen the timeframe you wish to see, or shorten it to focus on a specific period. To zoom into any evidence in the timeline, click and drag your cursor to mark the period. The timeline range can be reset by clicking on the Reset range button found in the top right corner.
Any messages and events that are related to the investigation are presented in the timeline. Each dot represents an evidence card. When you click on a dot, the related card is highlighted. You can filter the evidence displayed in the timeline to show either messages, events or both. The dots are a darker color when there is more than one piece of evidence that relates to that time period. A highlighted dot means that the Event card is in view.
The evidence cards found below the timeline include details related to log messages and events that are added to an investigation as evidence. Click on the drop down arrow found in the right upper corner of each card to reveal detailed information about the selected piece of data. Details such as message or event field values are displayed in these cards.
The evidence cards offer a Replay Search functionality for events. This provides a view of all messages related to the search. With message evidence cards, you can click Permalink to bring up a detailed message view.
Click Show Similar to display messages and events with similar fields. This filter may be reset by clicking Reset, which is located in the top right corner of the timeline widget.
Assign Investigations
When you have added pieces of evidence to an investigation, you can assign the investigation to other users or teams. Investigations can be assigned at the point of creating a new investigation or subsequently by editing an existing investigation. You may also choose to assign an investigation to yourself via the Assign to drop down menu.
Enable Investigation Assignment Email Alerts
Email alerts can be assigned to investigations. This functionality is enabled by default for existing installations. To disable this setting:
Navigate to the Config tab on the Investigations page.
Locate the Enable Investigation assignment email notifications capsule button and toggle to disable this setting.
When enabled, email alerts will be sent to both the user and the assigned team(s) involved in the investigation. These alerts are triggered immediately upon assignment.
Note
This feature requires that the transport_email_server configuration setting (server.conf) is properly configured to allow your Security Data Lake instance to send emails. If these parameters are not configured properly, the email will not be sent.
Update Investigation Status
Let’s say you have concluded an investigation and you are ready to close the investigation. From your Investigations page:
Click on the ellipsis located to the right of the desired investigation.
Click Edit.
Click on the ellipses found to the right of the investigation's title.
Click Edit.
Select Closed to update the status selection in the modal.
This ends an open investigation. However, closed investigations are still editable and can be assigned to users or teams.
To fully close an investigation, making the investigation inaccessible, you must archive the investigation:
Click the vertical ellipsis to the right of the selected investigation.
From the resulting drop-down menu options, choose Archive.
This action removes the investigation from the Open tab to the Archived tab. At this point you cannot make any edits to the archived investigation.
If an investigation was archived in error:
Click the ellipsis to the right of an investigation in the Archived tab.
Select Restore.
You can also bulk restore archived investigations by selecting the check boxes for the archived investigations from the Archived tab. Then click the Bulk Actions button drop-down, and select Restore. This restores the archived investigations into the Open tab.
Perform Bulk Actions
You can perform bulk archiving, assigning, and deleting tasks in the Investigations menu.
To perform bulk actions, navigate to the Investigations page, select the investigations by clicking on the checkbox, and click the Bulk Actions drop-down to assign, archive, or delete multiple investigations.
Compose Investigation Report by AI
Investigations include an AI-powered reporting feature that analyzes submitted events and logs to generate a detailed report, including key findings and recommended defensive actions. For the AI-generated report to be produced, the investigation must contain at least three logs, and your Security Data Lake environment must have access to the public internet.
Warning
This reporting feature involves the transmission of select logs and data to a third-party AI service. This service is not directly managed or controlled by Security Data Lake. Review the full terms and conditions for this feature before you enable its use in Security Data Lake!
To create a new AI report:
Navigate to the Security Data Lake Security interface and select the Investigations tab.
Locate the investigation you wish to summarize and click AI Report. (Note that this button will only be available if you have met the minimum requirement of three attached logs to the investigation.) If this is the first report you have generated, you will be prompted to review the Terms and Conditions for this feature.
Review this disclaimer carefully and determine if you wish to use the feature as indicated.
Once you proceed the investigation report will appear in the resultant window for your review. You may then copy the text of the report by selecting Copy Report or download a text file of the report by selecting Download Report.
Roles
Security Data Lake includes two roles in user permissions related to investigations:
Investigations Manager: With this role, you have full control over investigations.
Investigations Reader: With this role, you have read access to investigations only.
Warning
Please check your permissions settings to ensure that users and teams have the required access to investigations and any additional permissions for evidence entities. For instance, if an investigation contains a dashboard as evidence, the assignee cannot view or add to the dashboard without permission to view dashboards, even if they can view/edit the investigation itself.
Determine Relative Time Or Absolute Time Ranges
Often searches are conducted using a relative time line, such as "everything from the last hour." Considering this, when you decide to include a saved search as evidence to an investigation, Security Data Lake will now prompt you to either convert the logged times to absolute time or maintain the relative search. Opting to convert to relative time will result in the system duplicating your saved search, preserving the absolute time parameters, and labeling the saved search as Absolute Time. This allows you to utilize either search as necessary.
Use Markdown in Investigation Notes
Investigation notes have a custom markdown editor. The custom markdown editor significantly enhances the note-taking experience by allowing you to structure and format your notes using features such as headers, bullet points, tables, numbered lists, code snippets, and hyperlinks.
The markdown editor preview feature enables you to view the final output, allowing you to rectify errors and discrepancies before saving.
Investigations and Alerts
You can associate an alert attached to an event definition with Security Data Lake's Investigations module, allowing new investigations to be created based off of alerts or evidence to be added to existing investigations when an alert is fired. This workflow is configured through the Event Definitions menu. For more information on events and defining event definitions, see the documentation on events and event definitions.
Create a New Investigation from Alert Notification
To generate a new investigation whenever an event is triggered, it is necessary to create an event definition. An event definition consists of conditions that, once met, fire an alert. Users can then be notified of this alert via various notification types supported by Security Data Lake, such as Slack, MS Teams, Email, and other provided alert options.
Navigate to the Alerts tab in Security Data Lake.
Click on the Event Definitions sub-menu tab.
At the right hand corner of the resulting page, click on the Create event definition button.
You may now create the new event definition as needed.
In the Notifications dialogue, click on the Add Notification button and select Create New Notification from the resulting drop-down options. For further details on Security Data Lake notifications, see the corresponding documentation.
Provide a title and a description for the notification and select the Create Investigation Notification option from the Notification Type drop-down options.
Click on the check box option to "Create a New Investigation for Every Alert."
Upon clicking the check box, a message prompt appears. Read the prompt and confirm to proceed.
Warning
Generating an investigation for each triggered alert can easily lead to an overwhelming list of investigations. Please proceed with caution.
To manage automatically generated investigations, you can specify the search and execution run times in the event conditions dialogue. For instance, you can specify time intervals, such as every 30 minutes or every 1 hour, for the "Search within the last" and "Execute search every" fields. This will limit the number of investigations created automatically whenever the alert is triggered. You also have the flexibility to customize the event limit based on your preferences.
Additionally, you have the option to disable the automatic execution of event definitions and manually enable them when necessary from the Events Definitionspage.
Complete the process with the following fields:
Assign Investigation To: The assignee for any new investigation.
Investigation Priority: The priority for the new investigation (Critical, High, Medium, or Low).
Investigation Status: Assign a status to the new investigation by selecting from the available options in the drop-down menu.
You can test the notification by clicking on the Execute Test Notification button and selectDone or Next to proceed to the summary dialogue.
The Summary dialogue provides a summary of all imputed configurations. Click on the Create event definition button at the bottom right of the page, which concludes the process flow for creating a new investigation for every alert triggered by this event definition.
Now for every time the condition set for the event definition is met, an alert is triggered, and this will automatically create a new investigation.
The event definition you have created is now also enabled and can be further managed from the Event Definitions page just as any investigation generated from the alert is added to your list of investigations.
Add Every Alert to an Existing Investigation
You can additionally configure Security Data Lake to add an event as evidence to an existing investigation whenever an alert for that event is triggered. This way, you can seamlessly integrate events into your ongoing investigations without creating separate investigations for each alert.
Navigate to the Alerts tab in Security Data Lake.
Click on the Event Definitions sub-menu tab.
At the right hand corner of the resulting page, click on the Create event definition button.
You may now create the new event definition as needed.
In the Notifications dialogue, click on the Add Notification button and select Create New Notification from the resulting drop-down options. For further details on Security Data Lake notifications, see the corresponding documentation.
Do NOT select the Create a New Investigation for Every Alert check box.
Provide a title and a description for the notification and select the Create Investigation Notification option from the Notification Type drop-down options. The following investigation configuration options will appear.
Investigation: Choose the investigation to which you would like to add the evidence.
Assign Investigation To: The assignee for any new investigation.
Investigation Priority: The priority for the new investigation (Critical, High, Medium, or Low).
Investigation Status: Assign a status to the new investigation by selecting from the available options in the drop-down menu.
You can test the notification by clicking on the Execute Test Notification button and select Done or Next to proceed to the summary dialogue.
The Summary dialogue provides a summary of all imputed configurations. Click on the Create event definition button at the bottom right of the page, and this concludes the process flow for adding an event as evidence to an existing investigation.