Intrusion Detection & Firewalls
Firewalls and intrusion detection systems are the two primary network-layer defenses that every production environment relies on. A firewall is your gatekeeper — it actively decides what traffic may enter or leave. An IDS is your surveillance camera — it watches traffic and raises an alarm when something looks wrong. Understanding both, and knowing where each falls short, is essential for designing real-world defenses.
Intruders
Before building defenses, consider who you are stopping. Intruders range from individual hackers and criminal organizations to opportunists seeking community reputation. Even a "benign" intruder who is merely curious consumes resources, may stumble on sensitive data, and creates legal liability — so every unauthorized access must be stopped.
Intrusion Detection Systems (IDS)
An intrusion detection system is a security service that monitors and analyzes system events to identify suspicious activity. It does not actively block — that is the job of an intrusion prevention system (IPS) or firewall. Three deployment models exist:
| Deployment | What it watches | Typical data sources |
|---|---|---|
| HIDS (Host-based) | A single host | Login records, file-access logs, syscall sequences, registry changes |
| NIDS (Network-based) | Traffic across many hosts on a network segment | Raw packets on the wire (passive copy) |
| Hybrid IDS | Both | Combines host agents with a central network sensor |
Host-Based IDS (HIDS)
A HIDS runs as an agent on each machine and monitors:
- Login and session activity — failed login counts, unusual login times, login locations
- Program execution activity — execution denials, resource utilization spikes, abnormal syscall sequences
- File access activity — unusual frequency of file reads/writes, attempts to copy system binaries, direct device access
When HIDS agents are deployed on every host in an organization, their data can be forwarded to a central facility for correlation — the precursor to modern SIEM.
Network-Based IDS (NIDS)
A NIDS receives a passive copy of network traffic (via a network tap or port mirroring) and analyzes it across multiple OSI layers:
- Network layer — IP/ICMP protocol violations, spoofed source addresses, illegal header values
- Transport layer — unusual TCP/UDP fragmentation, port scans, SYN floods
- Application layer — malformed DNS/HTTP/DHCP requests, unauthorized remote login attempts
A NIDS performs deep packet inspection and is typically stateful, maintaining connection context so it can detect attacks spread across multiple packets. Because the NIDS is passive, it adds no forwarding latency — but it cannot block on its own.
Contrast with a firewall: A firewall sits inline — all traffic flows through it. A NIDS sits off to the side, receiving a copy. Active filtering adds overhead; passive monitoring does not.
Canonical tool — Snort: Snort is a widely used open-source NIDS. A Snort rule has the form:
action protocol src_ip src_port -> dst_ip dst_port (options)
Example rule that alerts on any ICMP traffic entering the home network:
alert icmp any any -> $HOME_NET any (msg:"ICMP test"; sid:1000001; rev:1;)
To install and run Snort on Ubuntu:
sudo apt install snort
sudo vi /etc/snort/rules/local.rules # add your rules
sudo snort -A console -i eno1 -u snort -g snort -c /etc/snort/snort.conf
Challenges and Attacks on NIDS
Running a NIDS at scale is hard:
- Volume — a single sensor may see gigabits per second
- State tracking — maintaining per-connection state for thousands of simultaneous flows
- Resilience — the NIDS itself is a high-value target
Attackers exploit these weaknesses through:
- Algorithmic complexity attacks — crafting traffic that forces worst-case processing
- Evasion attacks — IP fragmentation or protocol ambiguity exploits disagreements between how the NIDS and the end host reassemble packets, so the NIDS sees harmless fragments while the host sees a complete exploit payload
- Encryption — TLS/HTTPS tunneling hides payload content from a NIDS that cannot decrypt it; modern NIDS often requires TLS inspection proxies
- Stealthy port scanning — slow, distributed scans stay below threshold-based detection
Honeypots
A honeypot is a decoy system intentionally made attractive to attackers. It has no legitimate traffic, so any interaction is suspicious. Honeypots serve dual purposes: they alert the defender to active probing, and they collect intelligence on attacker tools and techniques. In network diagrams, honeypots are often placed in the DMZ — visible from the internet but isolated from internal systems.
Signature-Based vs. Anomaly-Based Detection
IDS engines use two fundamental detection strategies:
| Strategy | How it works | Strengths | Weaknesses |
|---|---|---|---|
| Signature-based | Matches traffic/events against a database of known attack patterns | Low false-positive rate for known attacks; easy to audit | Blind to zero-day attacks; rules must be kept updated |
| Anomaly-based | Builds a statistical model of "normal" and flags deviations | Can catch novel attacks | High false-positive rate; baseline must be carefully trained |
The Base-Rate Fallacy
Anomaly-based IDS raises a critical statistical concern. Suppose an IDS is 99% accurate (1% false-positive rate) and real attacks represent 0.1% of all traffic. Then for every 100,000 events:
- ~100 are true attacks → ~99 detected correctly
- ~99,900 are benign → ~999 flagged as false positives
The analyst sees roughly 10 false alarms for every real alert. Tuning the false-positive rate down is therefore the dominant operational challenge of any IDS deployment.
Firewalls
A firewall is a network device or service that monitors and filters traffic between a private network and the public internet. It enforces a collection of security rules that control incoming (ingress) and outgoing (egress) traffic flows.
Firewall Types
| Type | Decision basis | Layer inspected | State maintained |
|---|---|---|---|
| Packet filter | Each packet independently | Network / Transport (IP, TCP/UDP headers) | None (stateless) |
| Stateful firewall | Connection state formed from related packets | Network / Transport | Yes — connection table |
| Application / Proxy firewall | Full application-layer content | Application (HTTP, DNS, …) | Yes — full session proxy |
Packet filter: Makes an allow/deny decision on each individual packet in isolation using the 5-tuple (source IP, destination IP, source port, destination port, protocol). Fast and simple, but cannot distinguish a legitimate reply from an unsolicited inbound packet.
Stateful firewall: Remembers which outbound connections were initiated by the protected network and automatically permits the matching reply packets. The diagram from class shows a UDP exchange where a reply with matching (SA=172.16.3.4, DA=192.168.51.50, SP=1525, DP=3264) is allowed because it matches a tracked outbound flow, while a reply to a different destination port is silently dropped.
Application/Proxy firewall: Acts as a full man-in-the-middle — the client connects to the proxy, the proxy inspects the request at the application layer, and only then forwards it to the real server. Offers the deepest inspection but highest latency and complexity.
Default-Deny vs. Default-Permit
The default-deny (allowlist) posture blocks everything and explicitly permits only known-good traffic. This is the security best practice: the attack surface is bounded by what you explicitly allow. Default-permit (denylist) allows everything and blocks known-bad traffic; it is easier to deploy but the unknown attack surface is unbounded.
Network DMZ and Segmentation
A DMZ (demilitarized zone) is a network segment that sits between an external firewall (internet-facing) and an internal firewall (protecting the corporate LAN). Public-facing servers (web, mail, DNS) live in the DMZ — they can be reached from the internet but are isolated from sensitive internal systems. Internal users can reach DMZ servers, but a compromised DMZ server cannot directly reach internal hosts.
Firewalls in Linux: iptables / Netfilter
On Linux, the kernel subsystem Netfilter intercepts packets at five hook points in the network stack (NF_INET_PRE_ROUTING, NF_INET_LOCAL_IN, NF_INET_FORWARD, NF_INET_LOCAL_OUT, NF_INET_POST_ROUTING). The user-space tool iptables (frontend to the kernel's Xtables) configures these hooks via tables and chains:
| Table | Chains | Purpose |
|---|---|---|
filter |
INPUT, FORWARD, OUTPUT | Packet allow/deny decisions |
nat |
PREROUTING, INPUT, OUTPUT, POSTROUTING | Source/destination address translation |
mangle |
All five chains | Arbitrary packet content modification |
The general iptables rule format is:
iptables [-t filter] -A INPUT <match-criteria> -j <target>
Common targets: ACCEPT, DROP, RETURN, LOG. Example rules:
# Block all traffic from a specific host to an internal subnet
sudo iptables -A INPUT -s 192.168.30.6 -d 192.168.1.0/24 -j DROP
# Open SSH and HTTP inbound
sudo iptables -A INPUT -p tcp --dport 22 -j ACCEPT
sudo iptables -A INPUT -p tcp --dport 80 -j ACCEPT
# Allow all outgoing TCP
sudo iptables -A OUTPUT -p tcp -j ACCEPT
ufw (UncomplicatedFirewall) is a simpler wrapper around iptables used widely on Ubuntu desktops and servers.
IDS vs. IPS
| IDS | IPS | |
|---|---|---|
| Position in traffic path | Out-of-band (passive copy) | Inline (traffic flows through it) |
| Action on detection | Alert / log | Alert + actively block/drop |
| Latency added | None | Some |
| Example | Snort in IDS mode | Snort in inline IPS mode, firewall with IDS integration |
A next-generation firewall (NGFW) combines stateful inspection with IPS capabilities and deep application awareness into a single device.
SIEM and Log Aggregation
When HIDS agents on many hosts send events to a central facility, the result is the foundation of a Security Information and Event Management (SIEM) system. A SIEM correlates events across firewalls, IDS sensors, servers, and endpoints to detect multi-stage attacks that no single sensor would catch in isolation. Examples include Splunk and the ELK stack (Elasticsearch, Logstash, Kibana).
Key Takeaways
- A NIDS watches a passive copy of network traffic across multiple hosts; a HIDS monitors events on a single machine; a hybrid IDS combines both.
- Signature-based detection has low false-positive rates for known attacks but misses zero-days; anomaly-based detection can catch novel attacks but suffers from the base-rate fallacy — high false-positive volume in practice.
- Firewalls come in three generations: packet filter (stateless, per-packet), stateful (tracks connection context), and application/proxy (full content inspection).
- Default-deny is the secure default: block everything, permit only what is explicitly needed.
- A DMZ places public-facing servers in a network segment isolated from the internal LAN, limiting the blast radius of a compromise.
- Evasion techniques — IP fragmentation, encryption, slow scans — exploit gaps between what the IDS sees and what the end host processes.
- On Linux, Netfilter/iptables is the kernel mechanism behind packet filtering;
ufwprovides a friendlier interface to the same engine.