What is it that people call network firewalls?

So I have been reading about firewalls today. It can be said that it is a tool, piece of software that controls the flow of traffic between networks. They sit at the border between networks, acting as a gateway that makes decisions about the kind of traffic that should be allowed or denied access. This decision is usually done based on the TCP/IP characteristics. They also log which connections were blocked. These blocked activities are usually generated by malicious hackers running automated tools, port scanning or malwares.

Now we know the basic idea of what firewall does, lets look at different functions a firewall carries out. Here we will look at the following five functions.

  1. Simple Packet filtering
  2. Stateful Packet filtering
  3. Network Address Translation
  4. Intrusion Prevention System
  5. Logging

Different types of Firewalls based on functionality

Packet Filters

This is the most basic functions of firewall device. These devices inspect ingress and egress transmissions to see if their properties meet the rule set, if they meet the rules they are allowed or denied access based on action specified in the rule. Simple packet filtering operation is stateless in their operation, meaning they do not have the capability to associate multiple request or handle sessions.

Stateless packet filters are susceptible to most exploits that take advantage of the TCP/IP protocol stack. For example, many packet filters are unable to detect spoofed IP addresses. More over, some packet filters can filter packets that are fragmented, even though they are legitimate requests. (Firewalls blocking fragmented packets is a common VPN interoperabitliy issues).

The table below shows a simple rule-set for a packet filtering firewall. It is important to know that rules are applied from the top, that is when a rule that match is found, that rules is applied immediately.

Action Source Address Destination Address Protocol Source Port Destination Port Control Bit
Allow Inside Outside TCP Any 80 Any
Allow Outside Inside TCP 80 > 1023 ACK
Deny All All All All All All

The above rule-set allows the users to browse internet.

Stateful Firewalls

This type of devices improve upon the packet filters, by have an additional component, a states table. Stateful firewalls intercepts the packets at the network layer and inspect them to see if they are allowed by the existing firewall rule, while also keep a track of the packet in the state table. The state table properties vary by vendors, but most of them maintains the following properties. Source address, source port, destination address, destination port, connection state and timeout period.

Three major connection state for TCP traffic are

  • Connection establishment
  • Usage
  • Termination

For stateless protocols such as UDP, which do not have a formal process of intialize, establish and termination, their state cannot be monitored at the transport layer. For such protocols, stateful firewalls are able to track the source and destination addresses and port numbers. Hence a state entry is added with a state to allow traffic, but a timeout field is added so the entry can be deleted when timeout occurs.

Application Layer Firewalls

These are the third generation firewalls that work at the application layers. The main component of this type of firewalls is stateful protocol analysis. This improves on the basic stateful inspection described above by adding a basic intrusion detection technology, i.e. an inspection engine that analyses protocols at the application layer to compare benign activities against the traffic activities. This technology is also sometimes referred as Deep Packet Inspection (DPI), as the device is now looking in to the contents of the packet, not just the header as in the previous types of firewalls. Firewalls with both stateful inspection and stateful protocol analysis capabilities are not fully-fledged IDPS, which usually have more capabilities.

Proxy Firewalls

An Application Proxy gateway is a feature of modern firewalls, that combines the lower layer access control with upper layer functionality. These firewalls contains a proxy agent that acts as an intermediary between two host that wish to communicate. This stops direct connection between them, each connection resulting in two separate connection. The proxy is transparent to the host. The proxy agent interfaces directly with the firewall rule-set to determine whether a given instance of network traffic should be allowed to transit the firewall.

Application-proxy gateways are different than the application firewalls in two ways. Firstly they can offer a higher level of security for specific applications, because it prevents direct communications between two hosts. And secondly, they can decrypt the packets, examine them and re-encrypt before sending the to the destination.

The main drawback of this type of firewall is that it take time reading and interpreting the packets.

What is a buffer overflow?


Buffer overflow happens when more data is written to a buffer, then it is allocated to hold. This extra data is translated to memory direction numbers, which is executed by a computer. A buffer overflow could crash the application, and also could be exploited by an attacker to execute any command they want.

Types of Buffer Overflow

  • Stack Overflow
  • Heap Overflow

Prevention Mechanisms

  • Use higher level programming languages that are strongly typed and disallow direct memory access
  • Canary (stack cookie) is a data pattern that is written to the stack just before EIP register (EIP contains the address of the next instruction)
  • Address Space Layout Randomization (ASLR)
  • Data Execution Protection
  • Stack Canary/Cookie

Principles of a secure network design


As internet keeps on growing, letting more people to come online, the number of network security breaches are increasing as well. To guard against these security breaches a good secure network design needs to be in place. This article will discuss some of theses principles of a securely designed network. Below is the five principle that will be discussed.

  1. Defense in depth
  2. Compartmentalization
  3. Principle of Least Privilege
  4. Weakest Link in the chain
  5. Accountability and Traceability

Defense in depth

This is a term that includes several of the principles of network design. The large number of components that builds up a network and its complexity makes the network vulnerable in many areas, its hard to identify which one has the weak security features built in. Hence, it is desirable to build a network, with many layers of security giving a more depth of security against the weakness in different components.

The following three rules apply to defense in depth strategy.

  1. Defense in multiple places: This principle states that network could be attack from multiple points (insiders and outsiders), so security devices should be deployed at different points in the network, to prevent attacks that pass through one device.
  2. Build layered defenses: More than one security layer should be built, to stop attackers from reaching their target. For example nested firewalls, couple with IDS/IPS could be deployed at outer and inner boundaries.
  3. Use robust components: This rule states that the security devices should be deployed based on the value of what you are protecting and threat at the point of application.


IT system resources of different sensitivity levels(i.e., different risk tolerance values and threatsusceptibility) should be located in different security zones, be it physically or logically.

With proper placement and configurations of firewalls help create secure architectures by dividing the network infrastructure into security zones and controlling communication between them.

Principle of Least Privilege

This rule states that access of systems or should be restricted to minimum user or administrators. This also covers the physical access to the domain, access to the systems, applications and web services as well. In addition to this, control of external user access is also covered. An extension of this rule is the “Need-to-know” rule, which states the information should be given to the people who absolutely need to know it.

Weakest Link in the chain

The security of the IT system depends on the least secured element of the system. A layered approach to network design with weaker and least protected assets residing in a separated domain mitigates the existent of these weakest link (Humans are considered to be the weakest link in information security architectures).

 Accountability and Traceability

This principle states that the security risks will exist, we need to do our best to manage and mitigate the risk as soon as the risk appears.

Hence, the the network architecture should provide means of tracking the users, attackers and administrator actions. This principle translate to functions such as auditing, event management and monitoring and forensics.

Correlating Network Intrusion Detection Alerts

Abstract:  With the increase in number of components that makes up a computer network, and increase in attack types, large amounts of low level alerts are being generated by an Intrusion detection systems (IDS). It is difficult to make sense of these alerts by themselves, the manual task of finding the relationship among them would take huge time. Hence, came the need of Alert Correlation systems.

Keywords: Intrusion Detection System, Alert Correlation, Alert

1. Introduction

An intrusion detection system (IDS) is an automated software that is capable of scanning all the activities that occur in the network. There are 3 main methodologies used by these systems; signature based, anomaly based, and stateful protocol analysis. When an event occurs in a network, the IDS analyse the event using one of the above mentioned methodology, to determine if that event is malicious or not. If it is found to be malicious, an alert is generated that is passed to the system administrators.

Due to increase in the number of automated systems, that tries to infiltrate a network (such as nmap, Nessus), a huge amount alerts may be generated by an IDS. The number alerts increases, when multiple IDS sensors are used at different points in the network. In addition to this, IDS lack the capabilities to identify the relationships among different alerts. As a result, it is time consuming and challenging to analyse all the generated alerts.

To fulfil this gap, alert correlation systems were introduced. Paper (1) define alert correlation as a conceptual interpretation of multiple alerts such that new meanings are assigned to these alerts. The alert correlation system tries to identify the relationships among the alerts, to give a high level picture of what is happening.

This paper is written to give an introduction to what alert correlation is, to identify the basic building blocks of these systems and to identify the categories of alert correlation algorithms. In this regard, section 2, looks at the fundamental operations of these systems. While the section 3, looks at different algorithms that have been designed to correlate alerts.

2. Fundamental Correlation Operations

The different types of alert correlation can be identified based on the correlation operation that is executed on alerts. According to the paper (2), these operations could be divided into 7 types. They are compression, filtering, selective suppression, thresholding, modification, generalization, enrichment and specialization.

2.1 Compression

Compression operation consists of detecting the number of same alerts, which occurred in a given time frame and generating a new alert, which includes the count of each alert type. This method loses some data, such as the time at which the alerts occurred. If it is lossless, then the operation is referred as aggregation (2). For example, if alert a, b, c occurred 2,3,1 times respectively within a time frame, the compression process will generate a new alert d, which will include the number of time each alert occurred.

2.2 Filtering operation

Filtering is the process of filtering out alerts based on certain parameters. This operation may lead to many losses as well. Eg. If a filter is in place to filter out alerts with the attribute ‘c’, if any alerts with attribute ‘a’ and ‘c’, or ‘c’ and ‘d’ may be filtered out. Only alerts that contain attribute ‘c’ will pass through.

2.3 Selective Suppression

Selective suppression operation consists of discarding of alerts based on a predefined criteria. The criteria may involve discarding of an alert if another type of alert is present, or due to a priority setting. For example, alert type 1 may be discarded if alert type 3 is present.

2.4 Thresholding

This operation generates an alert if a particular type of alerts is generated beyond a certain threshold value within a certain time period. This method may result in huge loss of information, as many alerts may not be reported to the analyst, if they do not meet the threshold value.

 2.5 Modification

Modification correlation operation works by which, depending on a context, an alert may be replaced by another one. This approach may result in loss of information as well, if the modification is irreversible. An example of this process, may include the alert priority could be modified due to the presence of a certain condition.

2.6 Generalization

Generalization is the operation by which a certain group of alerts are replaced by another alert, if they meet certain characteristics. For example, many alerts may be generated during nmap scanning, a new alert may be created by correlation process, which says ‘nmap scanning took place.’

2.7 Specialization

This is the opposite operation of generalization, by which multiple alerts are generated based on an alert using a deductive type reasoning (2). These alerts may be specialised based on data from the database and configurations. For example, if alert type 5 occurs, system knows that for that type of alert to occur alert type 6, and alert type 7 should have been present, hence the system generates new alerts for alert type 6 and 7.

2.8 Enrichment

Enrichment operation consists of generating an alert which contains additional information based on previous data. For example, if an alert type 7 occurs, system adds additional information such as a list of possible attack types.

3. Alert Correlation Algorithms 

Alert correlation algorithms can be categorised into 3 main types, based on the characteristics. They are similarity based, knowledge based and statistical based correlation algorithms.

Similarity based algorithms, works by detecting the similarities between alerts. These similarities are determined by a rule set which is defined by the analyst. The second category, knowledge based correlation, requires a database of knowledge consisting of fixed patterns and alert histories in the correlating process. The statistical-based approach requires the least amount of knowledge. These algorithms can detect alert that occurs outside the norm of the normal alerts. They are also referred as an anomaly based alert correlation.

The following sub-section explains each of the category separately and highlight some of the works done by researchers in each category.

3.1 Similarity based correlation

These algorithms works on the basis that there has to be a factor that would define a particular set of alerts. If this defining factor is found in an alert or a cluster of alerts (meta-alert), in a given period of time, these alerts are merged to form a new alert or a new meta-alert is formed.

This group of correlation algorithms can be subdivided in to three types. They are simple rules, hierarchal rules and machine learning. (3) Simple rules category works on the basis that rules need to be designed expressing relationships among the attributes of alerts. The second sub category, find alerts, that identifies the drawbacks of network architecture. This type of algorithms are needed as network issues are persistent, and require special handling or should be discarded. Finally, the machine learning algorithm works by automatically building models with special parameters, which could be used to identify an attack type from a group alerts.

Simple rules

In the paper (4) simple rule based Emerald product is discussed. These algorithms work by defining a rule set which tries to match the relationships among the attributes of alerts. These rules are evaluated among alerts to identify the relationships among them. Hence, these systems does not require a large database of knowledge.

Hierarchical Rules

This type algorithms forms hierarchical abstraction levels, and these levels are used to make decisions about security events. This type of algorithms work on the premise that an alert occurs due to some reason also referred as “root cause”. The paper (5) argues that 90% of alerts generated by an Intrusion detection system are due to a root-cause. And these alerts are persistent and should be removed as it floods the IDS, hiding more imminent threats on the network.

Machine Learning

The last sub category of similarity based correlation is machine learning. These algorithms generate comparison factors automatically during training process. Machine learning algorithms could be of supervised or unsupervised. For a supervised machine learning algorithm, a set of clustered alerts should be given during the training process, using which the algorithms builds models based on a set of parameters specified by the supervisor. On the other hand, for an unsupervised algorithm, the perimeters are decided by the system, during the training process.

3.2 Knowledge based correlation

This type of algorithms requires a knowledge base in order to work. These can be further divided based into two categories based on the knowledge it requires. They are scenario based and pre/post condition based.

Scenario based algorithms are based on the idea that the attacker must follow specific actions for a success of an attack. The second type of correlation, works by defining a set of pre-requisites and the expected result for a particular attacks. This type of algorithm lacks alert correlation against unknown attack types.

Scenario based

These types of algorithms are used to identify multi-step attacks. Scenario based algorithm assume that for a particular attack type to occur, the attacker will follow a known sequence of steps.

Paper (6) introduce an approach which automatically analyse multiple alerts generated by an IDS, and group them based on the likelihood of them belonging to a particular attack scenario. They achieved this by using self-organizing maps and unsupervised clustering algorithms.

Pre/post condition based

This category of algorithms are implemented by defining a knowledge base, which include a prerequisite for an alert to occur, such as network configurations and structure, and post conditions that follows it. With the help of this knowledge base, the systems are capable of identifying the alerts which belongs to groups, hence reducing the number of alerts shown to the security analyst.

3.3 Statistical based correlation

These algorithms works on the idea that some attacks will have similar statistical attributes and can be categorized based on these attributes. These types of algorithms store causal relationships between different incidents and analyses there occurred frequencies in the system education period using previous data statistical analysis and then attack steps are generated (3).

Purely statistical based approach does not require a knowledge base of attack types. But according to paper (3) these algorithms yield good result only in a specific domain, otherwise results in a high error rate.

4. Comparison

Below summarizes the comparison of algorithms done on paper (3). According to the results obtained by them, a complete alert correlation system needs a hybrid approach, which include algorithms addressing all concerns.

Characteristic Similarity Knowledge Statistical
Combining alerts from various sensors Yes Yes No
Prior knowledge Yes Yes No
Detect false alert Yes Yes Guessing
Detecting multi-staged attacks Hardly Yes Guessing
New attacks Yes No Yes
Error rate Average Low high

5. Summary

The state of art intrusion detection systems are capable of identifying almost all possible attack types. Due to increase in the number of attacks, these systems generates a huge amount of alerts, which are over whelming and time consuming for an analyst to analyse. In order to address these issues, alert correlation systems are needed. This paper summarized fundamental correlation approaches used by these systems. Moreover, different categories of alert correlation algorithm was introduced and discussed.

This paper looked at several techniques that have already been developed in this area, each addressing different concerns in alert correlation systems. I hope this paper gives a better understanding of alert correlation systems and would be useful in furthering the knowledge of this topic.

6. References

  1. A Survey on IDS Alerts Processing Techniques. SAFAA O. AL-MAMORY, HONG LI ZHANG. s.l. : 6th WSEAS International Conference on Information Security and Privacy, 2007.
  2. Fabien Pouget, Marc Dacier. Alert Correlation: Review of the state of the art. 2003.
  3. Seyed Ali Mirheidari, Sajjad Arshad, Rasool Jalili. Alert Correlation Algorithms: A Survey and Taxonomy. 2013.
  4. Skinner, Alfonso Valdes and Keith. Probabilistic Alert Correlation. 2001.
  5. Julisch, Klaus. Clustering Intrusion Detection Alarms to Support. s.l. : ACM Journal, 2002.
  6. Multistep Attack Detection and Alert Correlation in Intrusion Detection Systems. Fabio Manganiello, Mirco Marchetti, and Michele Colajanni. 2011. Vol. 200.