OSSEC Host-Based Intrusion Detection Guide

by Rory Bray, Daniel Cid, Andrew Hay

Overview

This book is the definitive guide on the OSSEC Host-based Intrusion Detection system and frankly, to really use OSSEC you are going to need a definitive guide. Documentation has been available since the start of the OSSEC project but, due to time constraints, no formal book has been created to outline the various features and functions of the OSSEC product. This has left very important and powerful features of the product undocumented...until now! The book you are holding will show you how to install and configure OSSEC on the operating system of your choice and provide detailed examples to help prevent and mitigate attacks on your systems.
-- Stephen Northcutt
OSSEC determines if a host has been compromised in this manner by taking the equivalent of a picture of the host machine in its original, unaltered state. This ?picture? captures the most relevant information about that machine's configuration. OSSEC saves this ?picture? and then constantly compares it to the current state of that machine to identify anything that may have changed from the original configuration. Now, many of these changes are necessary, harmless, and authorized, such as a system administrator installing a new software upgrade, patch, or application. But, then there are the not-so-harmless changes, like the installation of a rootkit, trojan horse, or virus. Differentiating between the harmless and the not-so-harmless changes determines whether the system administrator or security professional is managing a secure, efficient network or a compromised network which might be funneling credit card numbers out to phishing gangs or storing massive amounts of pornography creating significant liability for that organization.
Separating the wheat from the chaff is by no means an easy task. Hence the need for this book. The book is co-authored by Daniel Cid, who is the founder and lead developer of the freely available OSSEC host-based IDS. As such, readers can be certain they are reading the most accurate, timely, and insightful information on OSSEC.

• Get Started with OSSEC
Get an overview of the features of OSSEC including commonly used terminology, pre-install preparation, and deployment considerations.
• Follow Steb-by-Step Installation Instructions
Walk through the installation process for the "local?, “agent?, and "server" install types on some of the most popular operating systems available.
• Master Configuration
Learn the basic configuration options for your install type and learn how to monitor log files, receive remote messages, configure email notification, and configure alert levels.
• Work With Rules
Extract key information from logs using decoders and how you can leverage rules to alert you of strange occurrences on your network.
• Understand System Integrity Check and Rootkit Detection
Monitor binary executable files, system configuration files, and the Microsoft Windows registry.
• Configure Active Response
Configure the active response actions you want and bind the actions to specific rules and sequence of events.
• Use the OSSEC Web User Interface
Install, configure, and use the community-developed, open source web interface available for OSSEC.
• Play in the OSSEC VMware Environment Sandbox
Use the OSSEC HIDS VMware Guest image on the companion DVD to implement what you have learned in a sandbox-style environment.
• Dig Deep into Data Log Mining
Take the “high art? of log analysis to the next level by breaking the dependence on the lists of strings or patterns to look for in the logs.

Product Details

ISBN-13:	9780080558776
Publisher:	Elsevier Science
Publication date:	04/09/2008
Sold by:	Barnes & Noble
Format:	eBook
Pages:	416
File size:	7 MB

About the Author

Rory Bray is senior software engineer at Q1 Labs Inc. with years of experience developing Internet and security related services. In addition to being a long-time advocate of Open Source software, Rory has developed a strong interest in network security and secure development practices. Rory has a diverse background which includes embedded development, web application design, software architecture, security consulting and technical editing. This broad range of experience provides a unique perspective on security solutions.

Daniel Cid is the creator and main developer of the OSSEC HIDS (Open Source Security Host Intrusion Detection System). Daniel has been working in the security area for many years, with a special interest in intrusion detection, log analysis and secure development. He is currently working at Q1 Labs Inc. as a software engineer. In the past, he worked at Sourcefire, NIH and Opensolutions. Daniel holds several industry certifications including the CCNP, GCIH, CISSP.

Andrew leads a team of software developers at Q1 Labs Inc. integrating 3rd party event and vulnerability data into QRadar, their flagship network security management solution. Prior to joining Q1 Labs, Andrew was CEO and co-founder of Koteas Corporation, a leading provider of end to end security and privacy solutions for government and enterprise. His resume also includes such organizations as Nokia Enterprise Solutions, Nortel Networks, and Magma Communications, a division of Primus. Andrew is a strong advocate of security training, certification programs, and public awareness initiatives. He also holds several industry certifications including the CCNA, CCSA, CCSE, CCSE NGX, CCSE Plus, Security+, GCIA, GCIH, SSP-MPA, SSP-CNSA, NSA, RHCT, and RHCE.

Read an Excerpt

OSSEC Host-Based Intrusion Detection Guide

---

By Andrew Hay Daniel Cid Rory Bray

Syngress Publishing, Inc.

Copyright © 2008 Elsevier, Inc.
All right reserved.
ISBN: 978-0-08-055877-6

---

Chapter One

Getting Started with OSSEC

Solutions in this chapter:

* Introducing Intrusion Detection

* Introducing OSSEC

* Planning Your Deployment

* Identifying OSSEC Pre-installation Considerations

  Summary

  Solutions Fast Track

  Frequently Asked Questions

Introduction

It's 8:15 p.m. on a Friday night in a tiny apartment building in Seoul, South Korea. Byung-Soon, an expert in the electronic theft of corporate information, is exploiting a well-known Internet Information Services (IIS) vulnerability on an American Web server in San Francisco, California. After spending weeks of careful reconnaissance against servers in his target's DMZ, he has finally found a way in through a well-known Microsoft IIS 6.0 vulnerability that, left unpatched, has provided him full access to the server. The target is a medium-sized consultancy firm that is known to do business with a large defense contractor who designs, among other things, ballistic missiles for sale to the United States military.

Byung-Soon begins searching through the various Web directories on the server and notices that an intranet site has been set up so that consultants within the firm can log their hours for work performed at the defense contractor. He downloads the index page for the intranet site and includes some malicious JavaScript that, when run, connects to a previously exploited system in India and downloads his rootkit. The rootkit, invisible to the user and other system processes, acts as a key-logger to record user keystrokes and enables Byung-Soon to connect directly to the compromised host through an encrypted remote access connection. After modifying the index page, he uploads his modified copy, removes any log entries generated by his actions, and heads out for a late dinner. In four hours, when the consultants start their day, Byung-Soon's plan begins.

Bob, a senior consultant assigned to the defense contract, logs in to the company intranet to start his day. Although this is the most boring part of his day, he knows he must keep an accurate count of the hours spent on this project so that his company, and of course he, gets paid. The process goes like clockwork, as it does every day, and Bob, like several of his coworkers, unwittingly installs Byung-Soon's rootkit. When he finishes with the intranet page, he launches Eclipse, the development platform the defense contractor uses for development of software, and starts working. The rootkit records all of Bob's keystrokes, including usernames, passwords, and server information, as it is designed to do. At random intervals throughout the day, Byung-Soon's rootkit sends out snippets of logged information to a collection of previously exploited servers located all over the world.

On Monday, Byung-Soon wakes up in his tiny apartment in Seoul and decides to check on his progress. He logs in to an exploited box at a university in Italy and executes a script to pull all the pieces of collected information together. He then pulls the compiled information down to another server in Warsaw, Poland and starts parsing the information for keywords provided by his employer. Luckily, the developers provide extensive comments within their code so Byung-Soon's script is able to easily identify the target code. The code belongs to Bob Johnson, one of the contractors whose system has a certain rootkit installed. Byung-Soon decides that it is time to connect to this system and finish the job he was hired to do.

This story, although fictional, is entirely possible and might be happening to your organization right now. By adding a host-based intrusion detection system (HIDS) to your servers and workstations, this embarrassing and potentially dangerous scenario, can be completely avoided. If an HIDS solution was installed on the compromised Web server, the remote access connection, file changes, and removal of the logs to cover Byung-Soon's tracks could have been logged, and potentially blocked, depending on the type of HIDS. If each client machine had an HIDS solution installed, the rootkit download, installation, and communications could have also been logged and blocked.

Introducing Intrusion Detection

Have you ever wondered what was happening on your network at any given time? What about the type of traffic trying to get to a server on your network? Intrusion detection is the act of detecting events that have been deemed inappropriate or unwelcome by the business, organizational unit, department, or group. This can be anything from the emailing of company secrets to a competitor, to malicious attacks from a host on the Internet, to the viewing of inappropriate Web content during your lunch break.

Intrusion detection can be performed manually, by inspecting network traffic and logs from access resources, or automatically, using tools. A tools used to automate the processing of intrusion-related information is typically classified as an intrusion detection system (IDS).

Before understanding how the Open Source Security (OSSEC) host intrusion detection system (HIDS) works, we should first review the differences between an HIDS and a network intrusion detection system (NIDS).

Network Intrusion Detection

When you hear the term "intrusion detection system," or "IDS," you probably think of an NIDS. Network intrusion detection systems have become widely used over the past decade because of the impressive capability to provide a granular view of what is happening on your network. The NIDS monitors network traffic using a network interface card (NIC) that is directly connected into your network. The monitoring can be implemented by connecting your NIC to a HUB (Figure 1.1), which allows you to monitor all traffic that crosses the hub; connecting to a SPAN port on a switch (Figure 1.2), which mirrors the traffic seen on another port of the switch; or connecting to a network tap (Figure 1.3), which is an inline device that sits between two interfaces and mirrors the traffic that passes between devices.

NIDS is typically deployed to passively monitor a sensitive segment of your network, such as a DMZ off the firewall where your corporate Web servers are located (Figure 1.4) or monitoring connections to an internal database that holds your customer credit card information (Figure 1.5). This monitoring allows you to passively watch all communications between your server and the systems attempting to access it.

A signature or pattern is used to match specific events, such as an attack attempt, to traffic seen on your network. If the traffic seen on your network matches your defined IDS signature, an alert is generated. An alert can also trigger an action, such as logging the alert to a file, sending an email to someone with details of the alert, or following an action to address this alert, such as adding a firewall rule to block the traffic on another device.

An NIDS is a powerful monitoring system for your network traffic, but there are some things to remember before deploying one:

* What do you do if well-known NIDS evasion techniques are used to bypass your NIDS and signatures? Common NIDS evasion techniques such as fragmentation attacks, session splicing, and even denial-of-service (DoS) attacks can be used to bypass your NIDS, rendering it useless. * What do you do if the communications between hosts are encrypted? With an NIDS you are passively monitoring traffic and do not have the ability to look into an encrypted packet. * What do you do if an attack is used against your server, but it is encrypted? Your carefully designed signatures would be unable to catch the attacks that your NIDS is deployed to protect against.

Tuning your NIDS to detect or account for these types of attacks will go a long way to help you focus your time on actual incidents instead of chasing down false positives. Each NIDS must be tuned for the network segment it is monitoring. Remember that most NIDS solutions take a top-down approach to comparing traffic against your signature set. Reducing the number of rules in your deployed signature set reduces processor and memory usage on your NIDS solution. If the DMZ your NIDS is deployed on doesn't contain any Web servers, you probably do not need to include signatures to detect Web server attacks.

Attackers are becoming adept at sidestepping an NIDS, which is why an HIDS is now a necessary safeguard to supplement your current NIDS deployments. Detecting these attacks at the final destination allow you to mitigate the previously mentioned NIDS headaches.

Host-Based Intrusion Detection

An HIDS detects events on a server or workstation and can generate alerts similar to an NIDS. An HIDS, however, is able to inspect the full communications stream. NIDS evasion techniques, such as fragmentation attacks or session splicing, do not apply because the HIDS is able to inspect the fully recombined session as it is presented to the operating system. Encrypted communications can be monitored because your HIDS inspection can look at the traffic before it is encrypted. This means that HIDS signatures will still be able to match against common attacks and not be blinded by encryption. An HIDS is also capable of performing additional system level checks that only IDS software installed on a host machine can do, such as file integrity checking, registry monitoring, log analysis, rootkit detection, and active response.

File Integrity Checking

Every file on an operating system generates a unique digital fingerprint, also known as a cryptographic hash. This fingerprint is generated based on the name and contents of the file (Figure 1.6). An HIDS can monitor important files to detect changes in this fingerprint when someone, or something, modifies the contents of the file or replaces the file with a completely different version of the file.

Registry Monitoring

The system registry is a directory listing of all hardware and software settings, operating system configurations, and users, groups, and preferences on a Microsoft Windows system. Changes made by users and administrators to the system are recorded in the system registry keys so that the changes are saved when the user logs out or the system is rebooted. The registry also allows you to look at how the system kernel interacts with hardware and software.

An HIDS can watch for these changes to important registry keys to ensure that a user or application isn't installing a new or modifying an existing program with malicious intent. For example, a password management utility can be replaced with a modified executable and the registry key changed to point to the malicious copy (Figure 1.7).

Rootkit Detection

A rootkit is a program developed to gain covert control over an operating system while hiding from and interacting with the system on which it is installed. An installed rootkit can hide services, processes, ports, files, directories, and registry keys from the rest of the operating system and from the user.

Active Response

Active response allows you to automatically execute commands or responses when a specific event or set of events is triggered. For example, look at Figure 1.8. An attacker launches an attack against your organization's mail server (1). The attack then passes through your firewall (2), and finally, transparently, passes by your deployed network tap that inspects all traffic destined for your mail server (3). Your NIDS happens to have a signature for this particular attack. The NIDS active response service sends a command to your firewall (4) to reset the attacker's session and place a rule blocking that host. When the attacker, whose connection has been reset, tries to initiate the attack again (5), the attacker is blocked.

The benefits of active response are enormous, but also risky. For example, legitimate traffic might generate a false positive and block a legitimate user/host if the rules are poorly designed. If an attacker knows that your HIDS blocks a certain traffic signature, the attacker could spoof IP addresses of critical servers in your infrastructure to deny you access. This is essentially a DoS attack that prevents your host from interacting with that IP address.

(Continues...)

---

Excerpted from OSSEC Host-Based Intrusion Detection Guide by Andrew Hay Daniel Cid Rory Bray Copyright © 2008 by Elsevier, Inc.. Excerpted by permission of Syngress Publishing, Inc.. All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.

---

Table of Contents

Chapter 1: Introduction This chapter will introduce you to the OSSEC project, its history, and its goals.
Chapter 2: Getting Started With OSSEC This chapter provides an overview of the features of OSSEC including commonly used terminology, pre-install preparation, and deployment considerations.
Chapter 3: Installation This chapter walks through the installation process for the "local" and "server" install types, including the Windows and Unix agent, and techniques to automate multiple agents installations.
Chapter 4: Configuration This chapter discusses the post-install configuration of OSSEC. Within this chapter you learn how to monitor log files, remote messages, email notification, alerting levels, etc.
Chapter 5: Working With Log Analysis - Decoders This chapter shows you how to extract key information from logs using decoders.
Chapter 6: Working With Log Analysis - Rule Files This chapter discusses how you can leverage rules for various devices and how to write your own rules. It will include examples on how to parse atomic and composite rules, how to keep state between messages, remove false positives and tune it appropriately.
Chapter 7: Configuring System Integrity Check This chapter explains the system integrity check features of OSSEC including monitoring of the binary executable files, system configuration files, and even the Windows registry.
Chapter 8: Rootkit Detection This chapter explains the rootkit detection capabilities of OSSEC on Unix and its configuration.
Chapter 9: Policy Enforcement This chapter explains the policy enforcement capabilities of OSSEC, explaining how to perform host-based system auditing and application monitoring.
Chapter 10: Active Response Configuration This chapter explains how to configure the active response actions you want to configure as well as how to bind the actions to specific rules or events.
Chapter 11: Integration and Advanced Configuration This chapter explains previously undocumented features, advanced configuration topics, and integration with third-party products.
Chapter 12: Using the Web interface This chapter explains how to install and use the community developed, open source web interface, that is available for OSSEC.
Appendix A: The Importance of Log Analysis

What People are Saying About This

From the Publisher

OSSEC is the most commonly used host intrusion detection software. This is the only book on the product and it is co-authored by Daniel Cid, founder and lead developer of OSSEC