Using Log-Based Intrusion Detection with CloudPassage Halo

Using Log-Based Intrusion Detection with CloudPassage Halo
Log-Based Intrusion Detection
Setup Guide
About Halo Log-Based intrusion Detection
How It Works
Integrating Log-Based Intrusion Detection with SIEM Tools
Performing Log-Based Intrusion Detection Through the API
Setting Up and Running Log-Based Intrusion Detection
Creating a Log-Based Intrusion Detection Policy
Managing LIDS Policies
Addressing LIDS Issues and Events
View LIDS Event Details
Act on LIDS Issues and Events
Policy Design Tips and LIDS Best Practices
About Halo Log-Based Intrusion Detection
The CloudPassage Halo log-based intrusion detection system (LIDS) is a Halo security module that allows you to
monitor server log files for events of interest, and receive alerts when such events occur. The module is available to
Halo users on both Linux and Windows platforms.
When enabled and configured, this module detects selected important events that are recorded in any number of
system or application log files on any of your servers. If you also enable Halo alerting, you can receive near-real-time
alerts when the highest-priority events are logged.
Event detection in this feature is policy-driven—the events that are to be considered indicators of intrusion are
specified (by event ID or by event-message text pattern) in a policy that is assigned to a server group that is being
monitored for intrusion. Every detected event occurrence is logged as a Halo security event that can be viewed in the
Halo portal, retrieved through the Halo API, and exported to third-party tools for further analysis. The policy also
specifies which events are to generate Halo alerts.
A key advantage of log-based intrusion detection is its light impact. Because only specific, high-value events are
logged into Halo, the massive gathering, storage, and analysis of voluminous events from hundreds to thousands of
log files is avoided.
With log-based intrusion detection you can continually monitor the security of all of your server systems and
applications, and be certain that you will be notified whenever specific events of critical importance occur anywhere in
your server infrastructure.
How It Works
Halo's log-based intrusion detection system leverages Halo's built-in distributed scanning architecture and policybased security analytics to detect and report on the most recent events of interest soon after they are written to any
of the log files that you specify, on any sets of servers that you want.
To specify which events should be monitored, you create a log-based intrusion detection policy. Like other Halo
policies, it consists of rules that are applied to an object being scanned on a server. In the following illustration of a
text-based Linux rule, each rule specifies (1) the path to the log file to scan, (2) the event message or ID to look for
(specified with a search pattern), and (3) whether to send a Halo alert when this event is detected.
In addition to text-based rules, LIDS rules for Windows also enable you to specify Channel-based rules that specify a
Windows event channel type and event ID.
To specify which servers to monitor, you assign the log-based intrusion detection policy to one or more server groups.
Note: The scanning frequency for Halo log-based intrusion detection is fixed at 5 minutes. This frequency
provides near-real-time reporting and alerting on events, without negatively impacting the performance of
your servers.
Also, to prevent "event overload" during a scan, Halo reports a maximum of fifty events (rule matches) for any
one policy rule in a given log file. Additional events that match that rule are ignored for that scan.
All occurrences of log events that you specify are saved as Halo events (that include the complete original event
message, whether text or XML), so that you can search for and view them on the Security Events History page of the
Halo portal.
To perform deeper analyses on these events, especially in relation to other events across your installations that might
not be monitored by Halo log-based intrusion detection, you may wish to integrate these Halo events into whatever
log-management and analysis or SIEM solutions your organization uses, as described next.
Integrating Log-Based Intrusion Detection with SIEM tools
If your organization already uses log-management, log-analysis or SIEM tools such as Splunk, Sumo Logic, ArcSight,
or RSA enVision, you can leverage their power by integrating Halo log-based intrusion detection with them.
By automatically extracting event data from Halo and feeding it into your SIEM solution, you'll gain the advantages of
both types of systems: Halo log-based intrusion detection will alert you directly and immediately to the occurrence of
events of critical importance, and then your log-analysis tool can evaluate the relationships among those events and
any others that may be occurring anywhere in your network, perhaps uncovering additional evidence of intrusion or
To perform the integration, you can develop your own scripts. For information, in the CloudPassage Halo REST API
Guide see Log-Based Intrusion Detection Policies. For tools created for this purpose, see the Halo Toolbox on
GitHub. For example code and instructions for integrating with Splunk, Sumo Logic, ArcSight and other systems, in
Github see the Halo Event Connector Script - Python.
Performing Log-Based Intrusion Detection Through the API
The log-based intrusion detection portion of the Halo REST API includes calls that allow you to develop or extend an
application to manipulate log-based intrusion detection policies. You'll be able to automate the creation, assignment,
and management of your policies from within your own software tools.
The API includes methods to perform the following tasks:
List log-based intrusion detection policies
Get a single log-based intrusion detection policy
Create a new log-based intrusion detection policy
Delete a log-based intrusion detection policy
Update a log-based intrusion detection policy
Assign a log-based intrusion detection policy to a server group
Remove a log-based intrusion detection policy from a server group
Beyond these tasks, you can also use the Halo API to pass log-based intrusion detection events to a logmanagement or SIEM system, as noted in the previous section.
For details on the above methods, and for complete information on using the CloudPassage API, in the Halo REST
API Developer Guide see Log-Based Intrusion Detection Policies.
Setting Up and Running LIDS
Halo's Log-Based Intrusion Detection (LIDS) is simple to enable and configure. The following steps provide an
overview of the tasks you'll need to perform to put LIDS to work in your environment.
1. Verify that log-based intrusion detection is enabled.
Verify that the Enable Automatic Scanning check box for "Log-Based Intrusion Detection" check box is
selected then click Save.
For information about Scanner Settings, in the Halo Operations Guide see Scanner Settings.
2. Create a log-based intrusion detection policy.
If you are just getting started with log-based intrusion detection policies, it may be simplest to clone then
customize one of the example log-based intrusion detection policy templates.
For information about creating a new policy or customizing a cloned policy, see Creating a Log-Based Intrusion
Detection Policy.
3. Assign the policy to a server group.
After saving your intrusion detection policy, you must assign it to one or more Halo server groups. In the Halo
Operations Guide see Assign Policies to a Group.
If you have not yet installed Halo agents or created server groups, in the Halo Operations Guide see Setting Up
Server Groups.
4. Assign an alert profile to the server group.
It is not required as part of setting up LIDS, but you can choose to create an alert profile at this time.
For details, in the Halo Operations Guide see Create Alert Profiles.
5. View and act on LIDS issues and events.
After a log-based intrusion detection scan has completed, you can view details about the issues and events
detected by that scan as follows:
On the Environment screen, or at the server or server group level, view the Issue Details Sidebar. In the Halo
Operations Guide see View Issue Details.
On the Security Events History page, view the details of LIDS events. See View LIDS Event Details.
To act on LIDS issues and events, see Act on LIDS Issues and Events.
Creating a Log-Based Intrusion Detection Policy
The most efficient way to create a new LIDS Policy is to use an existing LIDS policy or policy template as the starting
point for the new policy. You then assign a unique name to the policy, customize it as required, and save the new
1. Display the Log-based Intrusion Detection Policies page. In the Halo portal click Policies, then choose Policies
> Log-based IDS Policies.
2. Choose the starting point for the new policy.
For information about all policy creation options, see Managing LIDS Policies.
3. Type the Name and Description of the new policy, then add, remove, or modify the new policy's rules.
In both Linux and Windows rules:
You specify a path to the log file to examine and a search pattern to identify a specific log.
The search pattern's syntax is very similar to regular expressions. For information, in the Halo Operations
Guide see Search Expression Syntax.
Important: You cannot specify a search expression that will match every event in the log file. That is, you
cannot use expressions such as "*", ".", or ".*".
Note: If you are conducting log-based intrusion detection scans of firewall logs, you can improve the
detection capability by insert identifying strings into individual firewall policy rules. See Use Log
Prefixes for Firewall Events for details.
Linux rules identify events by searching for text patterns in the event message, like this:
Windows rules for Event Channel-based events use a mandatory event ID plus an optional text
In this type of LIDS rule, Halo uses the Event ID to detect specific types of events. The Search
Expression field is not required. However, you can add a search expression to the rule to limit matches of a
numbered event type to, for example, specific user names, server names or IP addresses. For example, the
rule shown above limits matches of events of type 4722 to those that also include the string Guest.
Windows rules for non-Event Channel events use a text pattern only, similar to Linux rules:
When specifying the path of a Windows Log file you can use the Windows system environment variable
%envVariable%partialPath. (For example, %SYSTEMDRIVE%\Windows\system32).
For suggestions about designing a LIDS policy's rules, see Policy Design Tips, below.
4. Enable the rule and configure its alert settings.
Select Active to enable Halo to use the rule. If the rule's Active check box is cleared, Halo will not detect the
events specified in the rule.
Select Alert to enable Halo to create an alert whenever the rule is matched.
Select Critical to identify the alert as critical.
5. Click Save Policy to save the policy.
Managing LIDS Policies
As your security requirements grow and change, you will also want to use Halo's policy management options to
manage the policies that are or are not included in the list of active policies.
This section describes Halo's policy actions. To perform most of the following actions, you choose an active policy,
policy template, or retired policy from a list and click its Action button. Then you select an action from the drop-down
Policy Manipulation Actions:
Export a Policy
To export a policy from Halo, select "Export" from the Actions dropdown list.
Halo saves the policy's settings as a JSON-formatted file. You can securely archive the policy file, share the policy
with other Halo users, or re-import it at a later time.
Import a Policy
To import a policy into Halo, click the Import Policy button above the policy list.
On the Import page, click Choose File, then navigate to and select the desired JSON-formatted policy file to import.
If the import is successful, the imported policy appears in your list of active policies.
Retire a Policy
To retire a policy, select "Retire" from the Actions dropdown list.
Retiring a policy removes it from Halo's list of active policies and adds it to the list of retired policies. Retired policies
are available for later use if they are unretired.
Unretire a Policy
To unretire a policy, in the Retired Policies page select "Unretire" from the Actions dropdown list.
Halo moves the policy from the retired policies list to the active policies list. It is once again available for editing or
assigning to server groups.
Delete a Policy
To delete a policy, select "Delete" from the Actions dropdown list, then in the confirmation dialog click OK.
Halo permanently removes the policy. It no longer appears in the Active Policies page and cannot be retrieved.
Note: The only way you can recover a deleted policy is to have exported it first, so that you can re-import the
exported file.
Policy Creation and Editing Actions:
Clone a Policy or Template
To clone a policy or policy template, select "Clone" from the Actions dropdown list.
Halo creates a copy of the policy, adds the word Copy to the policy's name, and places it in the Active Policies list.
You often can use the cloned template as-is, or you may wish to use it as the starting point for a custom policy. In
that case, create a unique name and description for the new policy, then customize its rules.
Note that Halo will not permit you to save a cloned policy if it does not have a unique name.
Create a New Policy
To create a new policy, click the Add New Policy button above the policy list.
On the new Policy page, Create a unique name and description for the policy. Initially, the policy is empty; add rules
as desired.
Halo will not permit you to save a new policy until you assign the policy a unique name.
Edit a Policy
To edit an active policy, select "Edit" from the Actions dropdown list.
The Edit Policy page opens, on which you can change the policy's name, description, and rules. When you save it,
the updated policy appears on the Active policies page.
Addressing LIDS Issues and Events
To accurately assess the level of risk associated with a given LIDS event, you need to examine the event's details.
View LIDS Event Details
To view the log-based intrusion detection policy violations Halo has detected, in the Event Types list choose "Logbased intrusion detection rule matched" then click Search.
By default, Halo lists the log-based intrusion detection events captured in the past 24 hours. For information about
Security Events History options, in the Halo Operations Guide, see View the Security Events History
For each displayed event, the page indicates the event's criticality, lists its date-time of occurrence, indicates which
policy rule was matched, in the source: area provides a link to the policy associated with the LIDS event, and shows
full text (or XML) of the event message.
To view the log entry that produced the event, in the list of LIDS events, click an event's More Details link
To respond to a LIDS event, see Act On LIDS Issues and Events, below.
Act on LIDS Issues and Events
Use LIDS issue or event details to assess the security implications of an event and take appropriate action.:
If a LIDS event that is not a security concern occurs regularly, edit the rule associated with the event.
In an event's Event Type / Details column, click the source: link to open the policy for editing. Remove, disable,
or modify the policy's rules to generate a more useful event or suppress the event entirely.
Contact the personnel involved with the event to determine whether the event is a security concern.
Use your organization's log-management or SIEM capabilities to investigate the cause of the issue.
Notify your security team of the issue,
Immediately quarantine the server involved and launch an incident response investigation.
Policy Design Tips and LIDS Best Practices
Halo includes three example LIDS policy templates: one for RPM-style Linux, one for Debian-style Linux, and and
one for Windows. You can use the templates as examples or starting points for your own policies, or you can create
your own policies from scratch.
General Guidelines
In designing or customizing your policies, keep these concepts in mind:
Log-based intrusion detection is designed to scan log files and detect and identify strong indicators of compromise.
It is not useful for collecting large numbers of non-critical events.
Design your policy to identify events that indicate that a real security violation might have occurred, and avoid
"noisy" events that do not need to be investigated. Careful design of your search patterns can go a long way
toward eliminating unimportant events.
The search expression in a rule can be the exact text you are looking for, or it can be an expression with a syntax
similar to that used in regular expressions. For information, in the Halo Operations Guide see Search Expression
Remember that searches of the log files are case-sensitive. For example, if the phrase "Event Channel" appears in
your search expression, you must capitalize it correctly or it will not be found.
Select the Alert check box only for the highest-priority events that must be dealt with immediately. Overwhelming
your email inbox with non-time-critical events will only make it more difficult to identify and deal with the truly highpriority ones. You can always review all detected events (both alertable and non-alertable) in the Halo portal.
Clear the Active flag for a rule if you want to temporarily disable it but not delete it permanently.
For a Windows Event Channel-based rule, you select the Event Channel name from a drop-down list that includes
all accepted channels. The most common Event Channel names are "Security", "Application", and "System".
Windows Event Channel-based policy rules use the search expression differently than do Windows or Linux textbased rules. In a text-based rule, the search expression is the only means for identifying the desired event. In an
Event Channel rule, the event ID is sufficient to identify the event, meaning that the search expression is not
required. However, the expression can be very useful for further filtering events of that ID to, for example, include
only a specific user, group, IP address, and so on.
Use Effective Search Patterns
The following examples from the built-in policy templates illustrate how you can use search patterns effectively to finetune the events reported.
Authentication failure for root. This Linux rule detects failed root logins in the file var/log/auth.log.
The search expression is "Failed password for root", which will match on a login failure of any user with root
Password reset on Administrator. This Windows Event Channel rule detects a password change for the
Administrator user on a host. The Event Channel ID (4724) specifies that it is a password reset event, and the
search expression "TargetUserName.>Administrator" will match the portion of the XML event text that identifies the
user, when the user name is "Administrator".
Use Log Prefixes for Firewall Events
Halo Linux firewall policies include a feature that can greatly aid in fine-grained detection of firewall events. For each
rule in a firewall policy, you can specify a log prefix, a text string that will be appended to the beginning of the event
message that is logged whenever that firewall rule is matched. (Logging for that rule must be enabled, or else there
will be no event for the prefix to be appended to.)
Craft each log prefix to uniquely identify the particular rule whose match you want to be detected when Halo scans
the firewall log file. For example, suppose you add the prefix "Inbound Drop:" to the inbound default-drop rule in your
firewall policy, and you add the prefix "Outbound Drop:" to the outbound default-drop rule:
If either of the default-drop rules is executed, its prefix appears in the logged event message:
Note: For improved readability in the log entry, you may wish to visually separate the prefix from the rest of the
log message. In that case, include a trailing space at the end of the prefix so that its last word doesn't run
into the first word of the message.
Finally, define a pair of log-based intrusion detection policy rules that will match the log prefixes:
Your log-based intrusion detection scans of the firewall log file will then easily pick up any inbound or outbound
default-drop events, without the need for you to create potentially highly complicated regular-expression search
For information, in Workload Firewall Management, see Create and Assign a Firewall Policy.
Copyright ©2016 CloudPassage Inc. All rights reserved. CloudPassage ® and Halo ® are registered trademarks of CloudPassage, Inc.
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF