Intrusion Detection & Firewalls

Firewalls and intrusion detection systems are the two primary network-layer defenses that every production environment relies on. A firewall is your gatekeeper — it actively decides what traffic may enter or leave. An IDS is your surveillance camera — it watches traffic and raises an alarm when something looks wrong. Understanding both, and knowing where each falls short, is essential for designing real-world defenses.

Intruders

Before building defenses, consider who you are stopping. Intruders range from individual hackers and criminal organizations to opportunists seeking community reputation. Even a "benign" intruder who is merely curious consumes resources, may stumble on sensitive data, and creates legal liability — so every unauthorized access must be stopped.

Intrusion Detection Systems (IDS)

An intrusion detection system is a security service that monitors and analyzes system events to identify suspicious activity. It does not actively block — that is the job of an intrusion prevention system (IPS) or firewall. Three deployment models exist:

Deployment What it watches Typical data sources
HIDS (Host-based) A single host Login records, file-access logs, syscall sequences, registry changes
NIDS (Network-based) Traffic across many hosts on a network segment Raw packets on the wire (passive copy)
Hybrid IDS Both Combines host agents with a central network sensor

Host-Based IDS (HIDS)

A HIDS runs as an agent on each machine and monitors:

When HIDS agents are deployed on every host in an organization, their data can be forwarded to a central facility for correlation — the precursor to modern SIEM.

Network-Based IDS (NIDS)

A NIDS receives a passive copy of network traffic (via a network tap or port mirroring) and analyzes it across multiple OSI layers:

A NIDS performs deep packet inspection and is typically stateful, maintaining connection context so it can detect attacks spread across multiple packets. Because the NIDS is passive, it adds no forwarding latency — but it cannot block on its own.

Contrast with a firewall: A firewall sits inline — all traffic flows through it. A NIDS sits off to the side, receiving a copy. Active filtering adds overhead; passive monitoring does not.

Canonical tool — Snort: Snort is a widely used open-source NIDS. A Snort rule has the form:

action protocol src_ip src_port -> dst_ip dst_port (options)

Example rule that alerts on any ICMP traffic entering the home network:

alert icmp any any -> $HOME_NET any (msg:"ICMP test"; sid:1000001; rev:1;)

To install and run Snort on Ubuntu:

sudo apt install snort
sudo vi /etc/snort/rules/local.rules   # add your rules
sudo snort -A console -i eno1 -u snort -g snort -c /etc/snort/snort.conf

Challenges and Attacks on NIDS

Running a NIDS at scale is hard:

Attackers exploit these weaknesses through:

Honeypots

A honeypot is a decoy system intentionally made attractive to attackers. It has no legitimate traffic, so any interaction is suspicious. Honeypots serve dual purposes: they alert the defender to active probing, and they collect intelligence on attacker tools and techniques. In network diagrams, honeypots are often placed in the DMZ — visible from the internet but isolated from internal systems.

Signature-Based vs. Anomaly-Based Detection

IDS engines use two fundamental detection strategies:

Strategy How it works Strengths Weaknesses
Signature-based Matches traffic/events against a database of known attack patterns Low false-positive rate for known attacks; easy to audit Blind to zero-day attacks; rules must be kept updated
Anomaly-based Builds a statistical model of "normal" and flags deviations Can catch novel attacks High false-positive rate; baseline must be carefully trained

The Base-Rate Fallacy

Anomaly-based IDS raises a critical statistical concern. Suppose an IDS is 99% accurate (1% false-positive rate) and real attacks represent 0.1% of all traffic. Then for every 100,000 events:

The analyst sees roughly 10 false alarms for every real alert. Tuning the false-positive rate down is therefore the dominant operational challenge of any IDS deployment.

Firewalls

A firewall is a network device or service that monitors and filters traffic between a private network and the public internet. It enforces a collection of security rules that control incoming (ingress) and outgoing (egress) traffic flows.

Firewall Types

Type Decision basis Layer inspected State maintained
Packet filter Each packet independently Network / Transport (IP, TCP/UDP headers) None (stateless)
Stateful firewall Connection state formed from related packets Network / Transport Yes — connection table
Application / Proxy firewall Full application-layer content Application (HTTP, DNS, …) Yes — full session proxy

Packet filter: Makes an allow/deny decision on each individual packet in isolation using the 5-tuple (source IP, destination IP, source port, destination port, protocol). Fast and simple, but cannot distinguish a legitimate reply from an unsolicited inbound packet.

Stateful firewall: Remembers which outbound connections were initiated by the protected network and automatically permits the matching reply packets. The diagram from class shows a UDP exchange where a reply with matching (SA=172.16.3.4, DA=192.168.51.50, SP=1525, DP=3264) is allowed because it matches a tracked outbound flow, while a reply to a different destination port is silently dropped.

Application/Proxy firewall: Acts as a full man-in-the-middle — the client connects to the proxy, the proxy inspects the request at the application layer, and only then forwards it to the real server. Offers the deepest inspection but highest latency and complexity.

Default-Deny vs. Default-Permit

The default-deny (allowlist) posture blocks everything and explicitly permits only known-good traffic. This is the security best practice: the attack surface is bounded by what you explicitly allow. Default-permit (denylist) allows everything and blocks known-bad traffic; it is easier to deploy but the unknown attack surface is unbounded.

Network DMZ and Segmentation

A DMZ (demilitarized zone) is a network segment that sits between an external firewall (internet-facing) and an internal firewall (protecting the corporate LAN). Public-facing servers (web, mail, DNS) live in the DMZ — they can be reached from the internet but are isolated from sensitive internal systems. Internal users can reach DMZ servers, but a compromised DMZ server cannot directly reach internal hosts.

Firewalls in Linux: iptables / Netfilter

On Linux, the kernel subsystem Netfilter intercepts packets at five hook points in the network stack (NF_INET_PRE_ROUTING, NF_INET_LOCAL_IN, NF_INET_FORWARD, NF_INET_LOCAL_OUT, NF_INET_POST_ROUTING). The user-space tool iptables (frontend to the kernel's Xtables) configures these hooks via tables and chains:

Table Chains Purpose
filter INPUT, FORWARD, OUTPUT Packet allow/deny decisions
nat PREROUTING, INPUT, OUTPUT, POSTROUTING Source/destination address translation
mangle All five chains Arbitrary packet content modification

The general iptables rule format is:

iptables [-t filter] -A INPUT <match-criteria> -j <target>

Common targets: ACCEPT, DROP, RETURN, LOG. Example rules:

# Block all traffic from a specific host to an internal subnet
sudo iptables -A INPUT -s 192.168.30.6 -d 192.168.1.0/24 -j DROP

# Open SSH and HTTP inbound
sudo iptables -A INPUT -p tcp --dport 22 -j ACCEPT
sudo iptables -A INPUT -p tcp --dport 80 -j ACCEPT

# Allow all outgoing TCP
sudo iptables -A OUTPUT -p tcp -j ACCEPT

ufw (UncomplicatedFirewall) is a simpler wrapper around iptables used widely on Ubuntu desktops and servers.

IDS vs. IPS

IDS IPS
Position in traffic path Out-of-band (passive copy) Inline (traffic flows through it)
Action on detection Alert / log Alert + actively block/drop
Latency added None Some
Example Snort in IDS mode Snort in inline IPS mode, firewall with IDS integration

A next-generation firewall (NGFW) combines stateful inspection with IPS capabilities and deep application awareness into a single device.

SIEM and Log Aggregation

When HIDS agents on many hosts send events to a central facility, the result is the foundation of a Security Information and Event Management (SIEM) system. A SIEM correlates events across firewalls, IDS sensors, servers, and endpoints to detect multi-stage attacks that no single sensor would catch in isolation. Examples include Splunk and the ELK stack (Elasticsearch, Logstash, Kibana).

Key Takeaways

Practice

  1. What is the key difference between a Network-based IDS (NIDS) and a firewall in terms of how they are positioned in the network?
  2. A stateful firewall allows a reply UDP packet from 172.16.3.4:1525 back to 192.168.51.50:3264 but drops a second reply to port 2049. Why?
  3. Why is a default-deny (allowlist) firewall policy considered more secure than a default-permit (denylist) policy?
  4. Consider this Snort rule: alert tcp any any -> $HOME_NET 22 (msg:"SSH scan"; flags:S; threshold:type threshold,track by_src,count 10,seconds 60; sid:1000010;) What does this rule do?
  5. Which firewall type terminates the client connection, inspects the full application-layer request, and then opens a new connection to the real server?
  6. An anomaly-based IDS has a false-positive rate of 1% and real attacks constitute 0.1% of all events. Approximately how many false alarms occur for every true positive alert?
  7. An attacker splits a single exploit payload across many tiny IP fragments. What IDS evasion technique is this, and why can it fool a NIDS?
  8. In Linux, what is the relationship between Netfilter, iptables, and Xtables?
  9. Explain the purpose of a DMZ (demilitarized zone) in a network architecture and why it improves security compared to placing public-facing servers directly on the internal LAN.