Provide steps to log file preparation in security onion.

Question

Because log file normalization is important, log analysis tools often include log normalization features.
Tools that do not include such features often rely on plugins for log normalization and preparation. The
goal of these plugins is to allow log analysis tools to normalize and prepare the received log files for
tool consumption.
The Security Onion appliance relies on a number of tools to provide log analysis services. ELSA, Bro,
Snort and SGUIL are arguably the most used tools.
ELSA (Enterprise Log Search and Archive) is a solution to achieve the following:
? Normalize, store, and index logs at unlimited volumes and rates.
? Provide a simple and clean search interface and API.
? Provide an infrastructure for alerting, reporting and sharing logs.
? Control user actions with local or LDAP/AD-based permissions.
? Plugin system for taking actions with logs.
? Exist as a completely free and open-source project.

Bro is a framework designed to analyze network traffic and generate event logs based on it. Upon net-
work traffic analysis, Bro creates logs describing events such as the following:

? TCP/UDP/ICMP network connections
? DNS activity
? FTP activity
? HTTPS requests and replies
? SSL/TLS handshakes
Snort and SGUIL
Snort is an IDS that relies on pre-defined rules to flag potentially harmful traffic. Snort looks into all
portions of network packets (headers and payload), looking for patterns defined in its rules. When
found, Snort takes the action defined in the same rule.
SGUIL provides a graphical interface for Snort logs and alerts, allowing a security analyst to pivot from
SGUIL into other tools for more information. For example, if a potentially malicious packet is sent to
the organization web server and Snort raised an alert about it, SGUIL will list that alert. The analyst can
then right-click that alert to search the ELSA or Bro databases for a better understanding of the event.
Note: The directory listing may be different than the sample output shown below.

Answer 1

Step 1. Switch to Security Onion.

Launch the Security Onion VM from VirtualBox’s Dashboard (username: analyst / pass-
word: cyberops). The CyberOps Workstation VM can be closed to free up memory in the

host computer for this part of the lab.
Step 2. ELSA Logs
a. Open a terminal window in the Security Onion VM. Access to the applications menu is

shown in the following screenshot:


b. You can also right-click the Desktop > Open Terminal Here, as shown in the following

screenshot:


c. ELSA logs can be found under the /nsm/elsa/data/elsa/log/ directory. Change the

directory using the following command:
```
analyst@SecOnion:~/Desktop$ cd /nsm/elsa/data/elsa/log
analyst@SecOnion:/nsm/elsa/data/elsa/log$
```
d. Use the ls –l command to list the files:
```
analyst@SecOnion:/nsm/elsa/data/elsa/log$ ls -l
total 99112
total 169528
-rw-rw---- 1 www-data sphinxsearch 56629174 Aug 18 14:15 node.log
-rw-rw---- 1 www-data sphinxsearch 6547557 Aug 3 07:34 node.log.1.gz
-rw-rw---- 1 www-data sphinxsearch 7014600 Jul 17 07:34 node.log.2.gz
-rw-rw---- 1 www-data sphinxsearch 6102122 Jul 13 07:34 node.log.3.gz
-rw-rw---- 1 www-data sphinxsearch 4655874 Jul 8 07:35 node.log.4.gz
-rw-rw---- 1 www-data sphinxsearch 6523029 Aug 18 14:15 query.log
-rw-rw---- 1 www-data sphinxsearch 53479942 Aug 18 14:15 searchd.log
-rw-rw---- 1 www-data sphinxsearch 32613665 Aug 18 14:15 web.log
analyst@SecOnion:/nsm/elsa/data/elsa/log$
```

Step 3. Bro Logs in Security Onion
a. Bro logs are stored at /nsm/bro/logs/. As usual with Linux systems, log files are rotated
based on the date, renamed and stored on the disk. The current log files can be found

under the current directory. From the terminal window, change directory using the fol-
lowing command:
```
analyst@SecOnion:/nsm/elsa/data/elsa/log$ cd /nsm/bro/logs/current
analyst@SecOnion:/nsm/logs/current$
```
b. Use the ls -l command to see all the log files generated by Bro:
```
analyst@SecOnion:/nsm/bro/logs/current$ ls -l
total 100
-rw-rw-r-- 1 sguil sguil 368 Aug 18 14:02 capture_loss.log
-rw-rw-r-- 1 sguil sguil 46031 Aug 18 14:16 communication.log
-rw-rw-r-- 1 sguil sguil 2133 Aug 18 14:03 conn.log
-rw-rw-r-- 1 sguil sguil 2028 Aug 18 14:16 stats.log
-rw-rw-r-- 1 sguil sguil 40 Aug 18 14:00 stderr.log
-rw-rw-r-- 1 sguil sguil 188 Aug 18 13:46 stdout.log
analyst@SecOnion:/nsm/bro/logs/current$
```

Step 4. Snort Logs
a. Snort logs can be found at /nsm/sensor_data/. Change directory as follows:
```
analyst@SecOnion:/nsm/bro/logs/current$ cd /nsm/sensor_data
analyst@SecOnion:/nsm/sensor_data$
```
b. Use the ls -l command to see all the log files generated by Snort.
```
analyst@SecOnion:/nsm/sensor_data$ ls -l
total 16
drwxrwxr-x 7 sguil sguil 4096 Jun 19 23:16 seconion-eth0
drwxrwxr-x 7 sguil sguil 4096 Jun 19 23:16 seconion-eth1
drwxrwxr-x 7 sguil sguil 4096 Jun 19 23:16 seconion-eth2
drwxrwxr-x 5 sguil sguil 4096 Jun 19 23:08 seconion-eth3
analyst@SecOnion:/nsm/sensor_data$
```
c. Notice that Security Onion separates files based on the interface. Because the Security

Onion VM image has four interfaces, four directories are kept. Use the ls –l seconion-
eth0 command to see the files generated by the ethernet 0 interface.
```
analyst@SecOnion:/nsm/sensor_data$ ls -l seconion-eth0/
total 52
drwxrwxr-x 2 sguil sguil 4096 Jun 19 23:09 argus
drwxrwxr-x 10 sguil sguil 4096 Jul 7 12:09 dailylogs
drwxrwxr-x 2 sguil sguil 4096 Jun 19 23:08 portscans
drwxrwxr-x 2 sguil sguil 4096 Jun 19 23:08 sancp
drwxr-xr-x 2 sguil sguil 4096 Jul 7 12:12 snort-1
-rw-r--r-- 1 sguil sguil 27566 Jul 7 12:12 snort-1.stats
-rw-r--r-- 1 root root 0 Jun 19 23:08 snort.stats
analyst@SecOnion:/nsm/sensor_data$
```
Step 5. Various Logs
a. While the /nsm/ directory stores some logs files, more specific log files can be found
under /var/log/nsm/. Change directory and use the ls -l command to see all the log
files in the directory.
```
analyst@SecOnion:/nsm/sensor_data$ cd /var/log/nsm/
analyst@SecOnion:/var/log/nsm$ ls -l
total 8364
-rw-r--r-- 1 sguil sguil 4 Aug 18 14:56 eth0-packets.log
-rw-r--r-- 1 sguil sguil 4 Aug 18 14:56 eth1-packets.log
-rw-r--r-- 1 sguil sguil 4 Aug 18 14:56 eth2-packets.log
-rw-r--r-- 1 sguil sguil 182 Aug 18 13:46 ossec_agent.log
-rw-r--r-- 1 sguil sguil 202 Jul 11 12:02 ossec_agent.
log.20170711120202
-rw-r--r-- 1 sguil sguil 202 Jul 13 12:02 ossec_agent.
log.20170713120201
-rw-r--r-- 1 sguil sguil 202 Jul 14 12:02 ossec_agent.
log.20170714120201
-rw-r--r-- 1 sguil sguil 202 Jul 15 12:02 ossec_agent.
log.20170715120202
-rw-r--r-- 1 sguil sguil 249 Jul 16 12:02 ossec_agent.
log.20170716120201
-rw-r--r-- 1 sguil sguil 202 Jul 17 12:02 ossec_agent.
log.20170717120202
-rw-r--r-- 1 sguil sguil 202 Jul 28 12:02 ossec_agent.
log.20170728120202
-rw-r--r-- 1 sguil sguil 202 Aug 2 12:02 ossec_agent.
log.20170802120201
-rw-r--r-- 1 sguil sguil 202 Aug 3 12:02 ossec_agent.
log.20170803120202
-rw-r--r-- 1 sguil sguil 202 Aug 4 12:02 ossec_agent.
log.20170804120201
-rw-r--r-- 1 sguil sguil 42002 Aug 4 07:33 pulledpork.log
drwxr-xr-x 2 sguil sguil 4096 Aug 18 13:46 seconion-eth0
drwxr-xr-x 2 sguil sguil 4096 Aug 18 13:47 seconion-eth1
drwxr-xr-x 2 sguil sguil 4096 Aug 18 13:47 seconion-eth2
drwxr-xr-x 2 sguil sguil 4096 Jun 19 23:08 securityonion

-rw-r--r-- 1 sguil sguil 1647 Jun 19 23:09 securityonion-elsa-con-
fig.log

-rw-r--r-- 1 sguil sguil 7708106 Aug 18 14:56 sensor-clean.log
-rw-r--r-- 1 sguil sguil 1603 Aug 4 00:00 sensor-newday-argus.log
-rw-r--r-- 1 sguil sguil 1603 Aug 4 00:00 sensor-newday-http-agent.
log
-rw-r--r-- 1 sguil sguil 8875 Aug 4 00:00 sensor-newday-pcap.log
-rw-r--r-- 1 sguil sguil 53163 Aug 4 05:01 sguil-db-purge.log
-rw-r--r-- 1 sguil sguil 369738 Aug 4 07:33 sid_changes.log
-rw-r--r-- 1 sguil sguil 22598 Aug 8 01:35 so-bro-cron.log
drwxrwxr-x 2 sguil securityonion 4096 Jun 19 23:09 so-elsa
-rw------- 1 sguil sguil 7535 Jun 19 23:09 sosetup.log
-rw-r--r-- 1 sguil sguil 14046 Jun 19 23:09 sosetup_salt_call.log
-rw-r--r-- 1 sguil sguil 63208 Jun 19 23:09 sphinx_initialization.log
-rw-r--r-- 1 sguil sguil 81 Aug 18 14:55 squert-ip2c-5min.log
-rw-r--r-- 1 sguil sguil 1079 Jul 16 06:26 squert-ip2c.log
-rw-r--r-- 1 sguil sguil 125964 Aug 18 14:54 watchdog.log
analyst@SecOnion:/var/log/nsm$
```
Notice that the directory shown above also contains logs used by secondary tools such
as OSSEC, Pulledpork, Sphinx, and Squert.
b. Take some time to Google these secondary tools and answer the questions below:

For each one of the tools listed above, describe the function, importance, and place-
ment in the security analyst workflow.

____________________________________________________________________________
____________________________________________________________________________
____________________________________________________________________________
____________________________________________________________________________
____________________________________________________________________________
____________________________________________________________________________
____________________________________________________________________________
____________________________________________________________________________
Sphinx is an open source search engine and is used by ELSA to provide search
capabilities.
Pulledpork is a Snort rule manage system. It facilitates Snort rules updating. Outdated
Snort rules makes the entire system useless.
OSSEC is a system used to normalize and concentrate local system logs. When
deployed throughout the organization, OSSEC allows an analyst to have a clear picture
of what is happening in the systems.
Squert is a visual tool that attempts to provide additional context to events through the
use of metadata, time series representations, and weighted and logically grouped result
sets.