Table of Contents
This report on configuring an OpenWRT router aims to address a gap in the existing documentation. I am no networking expert and I have been learning OpenWRT along the way, but I do understand the OSI model, TCP/IP, CIDR notation, Linux Netfilter, and the essentials of network design. OpenWRT documentation describes the specifics of the firmware and how to use it in sub-enterprise environments, but I could not find end-to-end guidance for the only slightly more complicated full solution I was planning:
multiple public IP addresses
multiple VLANs (one per IP address)
DMZs in front of server hosts
Fully configurable firewalling in front of the DMZs
While I had set up similar networks on DD-WRT using iptables directly, little of this experience was transferable. This is because OpenWRT employs its own abstraction layers that are best not resisted. Besides, the task is inherently complex; while doing my research, I ran across Cisco-trained engineers struggling with the very same questions in their own world. (Possibly they were not the sharpest knives in the Cisco box, but who am I to judge?)
My router is a Turris Omnia. Pricey but good. As a project of the Czech Internet authority, it is better supported and more hacker-friendly than any commercial router I'm aware of. It contains three network interfaces – one for the WAN and two available to the internal switch. It runs OpenWRT and offers automatic updates. Best of all, in my experience, it comes with the Btrfs file system installed, which allows snapshots and back-ups through its Schnapps utility.
My network is a /29 subnet comprised of eight contiguous IP addresses. Deducting the [unusable] network address, the gateway address, and the broadcast address, I have five addresses available for use. One, of course, is the router's IP address.
Table 1.1. Subnet details
CIDR notation | x.x.x.224/29 | |
Netmask | 255.255.255.248 | |
IP addresses | x.x.x.224 | network |
x.x.x.225 | gateway | |
x.x.x.226 | router | |
x.x.x.227 | unused | |
x.x.x.228 | unused | |
x.x.x.229 | host A | |
x.x.x.230 | host B | |
x.x.x.231 | broadcast |
In the sections that follow, I will work through all the levels of configuration, from hardware to firewall. I used the LuCi graphical interface intermittently, but mostly I'll refer to the two text configuration files that define most of the network configuration for OpenWRT:
/etc/config/network
/etc/config/firewall
At the end, I provide the complete (sanitized) text of the finished files from my working system.
The illustration below is based on the official Turris Omnia architectural diagram, modified to show my particular setup. Here is a key to the hardware:
SoC is the router's system-on-a-chip.
Switch-chip is the router's internal network switch.
SoC and Switch-chip are hardwired together over switch ports 5 and 6 and hardware interfaces eth0 and eth1.
Switch ports 0 – 4 are hardwired to corresponding RJ-45 jacks LAN0 – LAN4.
Hardware interface eth1 is hardwired to the RJ-45 WAN jack.
Connection paths inside Switch-chip are programmable. This allows mutually isolated virtual LANs (VLANs) to be defined for each of my discrete services:
VLAN1 (orange) — LAN4 connects to an internal network, my LAN, over a dumb switch for Ethernet and also over bridged Wi-Fi.
VLAN2 (green) — LAN2 connects to an internal network, DMZ_229, which contains a server known to the Internet as x.x.x.229.
VLAN3 (purple): — LAN3 connects to an internal network, DMZ_230, which contains a server known to the Internet as x.x.x.230.
The gray ports are unallocated.
With help from Configuring internal VLANs - Omnia, I defined the VLANs as follows in /etc/config/network:
Table 1.2. VLAN definitions
VLAN1 | VLAN2 | VLAN3 |
---|---|---|
config switch_vlan option device 'switch0' option vlan '1' option ports '4 6' list comment 'vlan1: Default LAN' |
config switch_vlan option device 'switch0' option vlan '2' option ports '2 5t' list comment 'vlan2: DMZ_229' |
config switch_vlan option device 'switch0' option vlan '3' option ports '3 5t' list comment 'vlan3: DMZ_230' |
Server hosts x.x.x.229 and x.x.x.230, both alone on their respective VLANs, share eth0 / port 5 for access to the Internet. The "5t" in VLAN2 and VLAN3 means that switch port 5 is shared by two traffic streams, which are distinguished by tagging the packets with destination headers. This tagging setup can be configured "intuitively" in LuCi, but I find it easier to understand in the bare text configuration.
Meanwhile, the eth2 / port 6 pair carries all of the traffic passing in or out of the LAN.
I had a lot of trouble figuring out the interfaces. In a normal configuration, with one public IP address, there is a single WAN interface and a single LAN behind it. With a /29 or other exotic subnet, that original WAN is still there, but additional WAN interfaces must also be defined. The OpenWRT documentation Using multiple public IPs on wan interface shows how to do this.
The additional interfaces are not quite the equals of the original WAN, which has the IP of the router itself. Once I had everything set up and working, I did speed tests between various interfaces and found that the latency increase across (for example) WAN and WAN_229 is just as big as that between WAN_229 and DMZ_229. In other words, there is NAT happening behind the scenes for those extra public IPs, and that means this whole design has double-NATing built into it. I found that you can also assign a public IP directly to a host plugged into a router LAN port, but then you have a true old-style DMZ without any firewalling, and that is not what I want. Thus the cost of convenient firewalling to protect a server host is double NAT.
Once all of that is settled, the next problem is gateways. The OpenWRT documentation assumes that there is single WAN and also single LAN that everything downstream plugs into, including servers. With a multi LAN/DMZ setup, what gateways do the DMZs use, and what gateways do the DMZ and LAN hosts use?
Experiment proved that the original and subsidiary WAN interfaces all need to use the same public gateway – x.x.x.225 – and their associated respective DMZ networks use it, too. The hosts residing in those internal networks, however, use gateways specific to their networks, as shown in the /etc/config/network configuration blocks:
Table 1.3. interfaces
/29 IP ADDRESS | EXTERNAL | INTERNAL | HOSTS |
---|---|---|---|
x.x.x.226 |
config interface 'wan' config interface 'lan' option ifname 'eth1' option ifname 'eth2' 192.168.1.0/24 option proto 'static' option proto 'static' Static or DHCP option ipaddr 'x.x.x.226' option ipaddr '192.168.1.1' option netmask '255.255.255.248' option netmask '255.255.255.0' option gateway 'x.x.x.225' –––––––––––––> option gateway ' x.x.x.225' ––––––––> gateway: 192.168.1.1 option broadcast 'x.x.x.231' option broadcast '192.168.1.255' option dns '192.168.1.1 8.8.8.8' option force_link '1' option ip6assign '60' option type 'bridge' | ||
x.x.x.229 |
config interface 'wan_229' config interface 'dmz_229' option ifname 'eth1' option ifname 'eth0.2' iface eth0 inet static option proto 'static' option proto 'static' address 192.168.2.2 option ipaddr 'x.x.x.229' option ipaddr '192.168.2.1' broadcast 192.168.2.255 option netmask '255.255.255.248' option netmask '255.255.255.0' netmask 255.255.255.255 option gateway 'x.x.x.225' –––––––––––––> option gateway 'x.x.x.225' –––––––––> gateway 192.168.2.1 | ||
x.x.x.230 |
config interface 'wan_230' config interface 'dmz_230' option ifname 'eth1' option ifname 'eth0.3' iface eth0 inet static option proto 'static' option proto 'static' address 192.168.3.3 option ipaddr 'x.x.x.230' option ipaddr '192.168.3.1' broadcast 192.168.3.255 option netmask '255.255.255.248' option netmask '255.255.255.0' netmask 255.255.255.255 option gateway 'x.x.x.225' –––––––––––––> option gateway 'x.x.x.225' –––––––––> gateway 192.168.3.1 |
Note how the LAN and DMZs make use of the unusual hardware in the Turris Omnia. On VLAN1, the dedicated ETH2 interface connects the LAN to the WAN over internal port 6. On VLANs 2 and 3, ETH0 connects the DMZs to subsidiary WAN addresses over shared internal port 5. The notation "eth0.2" indicates "VLAN2 over ETH0," and "eth0.3" indicates "VLAN3 over ETH0."
The final mystery of configuring this router lay in the firewall. OpenWRT's firewall is built around zones and forwards between them, but I don't really understand the behaviors that I observed through experiment. A zone was configured by default for the LAN and WAN, so I configured corresponding zones for my four new interfaces. The key is to assign the right network to the right zone.
What is less clear is what the optimal input/output/forward settings should be. I started out by setting everything to ACCEPT so that at least everything would work, and then backed away from that by progressively resetting parameters to DROP until something broke. As it turns out, almost nothing here needs to be accepting packets.
Here is the minimal configuration that worked, anyway.
Table 1.4. Firewall zones
/29 IP ADDRESS | WAN ZONES | INTERNAL ZONES |
---|---|---|
x.x.x.226 |
config zone option name 'wan' option input 'DROP' option output 'ACCEPT' option forward 'DROP' option masq '1' option mtu_fix '1' option network 'wan wan6' |
config zone option name 'lan' option input 'ACCEPT' option output 'DROP' option forward 'DROP' option network 'lan' |
x.x.x.229 |
config zone option name 'wan_229' option network 'wan_229' option masq '1' option input 'DROP' option output 'DROP' option forward 'DROP' |
config zone option name 'dmz_229' option network 'dmz_229' option input 'DROP' option output 'DROP' option forward 'DROP' |
x.x.x.230 |
config zone option name 'wan_230' option network 'wan_230' option masq '1' option input 'DROP' option output 'DROP' option forward 'DROP' |
config zone option name 'dmz_230' option network 'dmz_230' option input 'DROP' option output 'DROP' option forward 'DROP' |
I used LuCi set up zone forwards since it was relatively intuitive, and again pared away the forwards that turned out not to be needed. Below are the surviving blocks in /etc/config/firewall, all of them for outbound traffic.
Table 1.5. Firewall forwards
Outbound LAN → WAN |
config forwarding option dest 'wan' option src 'lan' |
Outbound DMZ_229 → WAN_229 |
config forwarding option dest 'wan_229' option src 'dmz_229' |
Outbound DMZ_230 → WAN_230 |
config forwarding option dest 'wan_230' option src 'dmz_230' |
And finally, the firewall redirects. My starting point was again the OpenWRT documentation Using multiple public IPs on wan interface. That example prescribes one SNAT block and one unrestricted DNAT block for each of the extra public IPs.
The SNAT blocks provide IP masquerading, so that outgoing packets identify themselves
as coming from a non-default public address (with the default being x.x.x.226). The DNAT
blocks redirect traffic to internal hosts based on the public IP destinations they
carry. I have three internal networks, not one, so the src_dip
and src_ip
options route the packets as needed. I dropped the
lines option src 'lan'
from each SNAT block because they made no
sense in the new context and turned out to be unnecessary. The option dest
'wan'
lines really are needed.
I dropped entirely the wide-open DNAT blocks in the example, substituting port-specific redirects that I worked out through LuCi, which has pretty intuitive descriptive language for these rules, at least once you get one right the first time.
Table 1.6. Firewall redirects
x.x.x.226 (etc.) | x.x.x.229 (etc.) | x.x.x.230 (etc.) | |
---|---|---|---|
Outbound (SNAT) I.e., masquerading |
N/A |
config redirect option name 'snat_229' option src_dip 'x.x.x.229' option dest 'wan' option proto 'all' option target 'SNAT' option src_ip '192.168.2.2' |
config redirect option name 'snat_230' option src_dip 'x.x.x.230' option dest 'wan' option proto 'all' option target 'SNAT' option src_ip '192.168.3.3' |
Inbound (DNAT) |
N/A |
config redirect option target 'DNAT' option dest 'dmz_229' option src 'wan' option proto 'tcp' option src_dport '22' option dest_port '22' option name 'HostA SSH' option dest_ip '192.168.2.2' option src_dip 'x.x.x.229' config redirect option target 'DNAT' option dest 'dmz_229' option src 'wan' option proto 'tcp udp' option src_dport '53' option dest_port '53' option name 'HostA DNS' option dest_ip '192.168.2.2' option src_dip 'x.x.x.229' Etc. |
config redirect option target 'DNAT' option dest 'dmz_230' option src 'wan' option proto 'tcp' option src_dport '80' option dest_port '80' option name 'HostB HTTP' option dest_ip '192.168.3.3' option src_dip 'x.x.x.230' config redirect option target 'DNAT' option dest 'dmz_230' option src 'wan' option proto 'tcp' option src_dport '443' option dest_port '443' option name 'HostB HTTPS' option dest_ip '192.168.3.3' option src_dip 'x.x.x.230' Etc. |
These are the configuration files that I ended up with. I do not doubt that lint remains to be gathered, to say nothing of bad ideas. Maybe someone can suggest improvements.
Table 1.7. A sanitized version of the final product
/etc/config/network | /etc/config/firewall |
---|---|
config interface 'loopback' option ifname 'lo' option proto 'static' option ipaddr '127.0.0.1' option netmask '255.0.0.0' config globals 'globals' option ula_prefix '0123:4567:89ab::/48' config interface 'lan' option force_link '1' option type 'bridge' option proto 'static' option netmask '255.255.255.0' option ip6assign '60' option ipaddr '192.168.1.1' option gateway 'x.x.x.225' option broadcast '192.168.1.255' option ifname 'eth2' config interface 'dmz_229' option ifname 'eth0.2' option proto 'static' option ipaddr '192.168.2.1' option netmask '255.255.255.0' option gateway 'x.x.x.225' config interface 'dmz_230' option ifname 'eth0.3' option proto 'static' option ipaddr '192.168.3.1' option netmask '255.255.255.0' option gateway 'x.x.x.225' config interface 'wan' option ifname 'eth1' option proto 'static' option dns '192.168.1.1 8.8.8.8' option ipaddr 'x.x.x.226' option netmask '255.255.255.248' option gateway 'x.x.x.225' option broadcast 'x.x.x.231' config interface 'wan_229' option proto 'static' option ipaddr 'x.x.x.229' option netmask '255.255.255.248' option gateway 'x.x.x.225' option ifname 'eth1' config interface 'wan_230' option proto 'static' option ipaddr 'x.x.x.230' option netmask '255.255.255.248' option gateway 'x.x.x.225' option ifname 'eth1' config interface 'wan6' option ifname '@wan' option proto 'dhcpv6' option noserverunicast '1' config switch option name 'switch0' option reset '1' option enable_vlan '1' config switch_vlan option device 'switch0' option vlan '1' option ports '4 6' list comment 'vlan1: LAN 226' config switch_vlan option device 'switch0' option vlan '2' option ports '2 5t' list comment 'vlan2: DMZ_229' config switch_vlan option device 'switch0' option vlan '3' option ports '3 5t' list comment 'vlan3: DMZ_230' |
config redirect option target 'DNAT' option dest 'dmz_229' option proto 'tcp' option src_dport '22' option dest_port '22' option name 'HostA SSH' option dest_ip '192.168.2.2' option src 'wan' option src_dip 'x.x.x.229' config redirect option target 'DNAT' option dest 'dmz_229' option proto 'tcp udp' option src_dport '53' option dest_port '53' option name 'HostA DNS' option dest_ip '192.168.2.2' option src 'wan' option src_dip 'x.x.x.229' config redirect option target 'DNAT' option dest 'dmz_229' option proto 'tcp' option src_dport '80' option dest_port '80' option name 'HostA HTTP' option dest_ip '192.168.2.2' option src 'wan' option src_dip 'x.x.x.229' config redirect option target 'DNAT' option dest 'dmz_229' option proto 'tcp' option src_dport '443' option dest_port '443' option name 'HostA HTTPS' option dest_ip '192.168.2.2' option src 'wan' option src_dip 'x.x.x.229' config redirect option target 'DNAT' option proto 'tcp' option src_dport '22' option dest_port '22' option name 'HostB SSH' option dest_ip '192.168.3.3' option dest 'dmz_230' option src 'wan' option src_dip 'x.x.x.230' config redirect option target 'DNAT' option proto 'tcp' option src_dport '80' option dest_port '80' option name 'HostB HTTP' option dest_ip '192.168.3.3' option dest 'dmz_230' option src 'wan' option src_dip 'x.x.x.230' config redirect option target 'DNAT' option proto 'tcp' option src_dport '443' option dest_port '443' option name 'HostB HTTPS' option dest_ip '192.168.3.3' option dest 'dmz_230' option src_dip 'x.x.x.230' option src 'wan' config defaults option syn_flood '1' option input 'DROP' option forward 'DROP' option output 'DROP' config rule option name 'Allow-DHCP-Renew' option src 'wan' option proto 'udp' option dest_port '68' option target 'ACCEPT' option family 'ipv4' config rule option name 'Allow-Ping' option src 'wan' option proto 'icmp' option icmp_type 'echo-request' option family 'ipv4' option target 'ACCEPT' config rule option name 'Allow-IGMP' option src 'wan' option proto 'igmp' option family 'ipv4' option target 'ACCEPT' config rule option name 'Allow-DHCPv6' option src 'wan' option proto 'udp' option src_ip 'fe80::/10' option src_port '547' option dest_ip 'fe80::/10' option dest_port '546' option family 'ipv6' option target 'ACCEPT' config rule option name 'Allow-MLD' option src 'wan' option proto 'icmp' option src_ip 'fe80::/10' list icmp_type '130/0' list icmp_type '131/0' list icmp_type '132/0' list icmp_type '143/0' option family 'ipv6' option target 'ACCEPT' config rule option name 'Allow-ICMPv6-Input' option src 'wan' option proto 'icmp' list icmp_type 'echo-request' list icmp_type 'echo-reply' list icmp_type 'destination-unreachable' list icmp_type 'packet-too-big' list icmp_type 'time-exceeded' list icmp_type 'bad-header' list icmp_type 'unknown-header-type' list icmp_type 'router-solicitation' list icmp_type 'neighbour-solicitation' list icmp_type 'router-advertisement' list icmp_type 'neighbour-advertisement' option limit '1000/sec' option family 'ipv6' option target 'ACCEPT' config rule option name 'Allow-ICMPv6-Forward' option src 'wan' option dest '*' option proto 'icmp' list icmp_type 'echo-request' list icmp_type 'echo-reply' list icmp_type 'destination-unreachable' list icmp_type 'packet-too-big' list icmp_type 'time-exceeded' list icmp_type 'bad-header' list icmp_type 'unknown-header-type' option limit '1000/sec' option family 'ipv6' option target 'ACCEPT' config include option path '/etc/firewall.user' config include option path '/usr/share/firewall/turris' option reload '1' config include option path '/etc/firewall.d/with_reload/firewall.include.sh' option reload '1' config include option path '/etc/firewall.d/without_reload/firewall.include.sh' option reload '0' config rule option src 'wan' option dest 'lan' option proto 'esp' option target 'ACCEPT' config rule option src 'wan' option dest 'lan' option dest_port '500' option proto 'udp' option target 'ACCEPT' config include 'miniupnpd' option type 'script' option path '/usr/share/miniupnpd/firewall.include' option family 'any' option reload '1' config redirect option name 'snat_229' option src 'lan' option src_dip 'x.x.x.229' option dest 'wan' option proto 'all' option target 'SNAT' option src_ip '192.168.2.2' config redirect option name 'snat_230' option src 'lan' option src_dip 'x.x.x.230' option dest 'wan' option proto 'all' option target 'SNAT' option src_ip '192.168.3.3' config zone option name 'lan' option input 'ACCEPT' option output 'ACCEPT' option forward 'ACCEPT' option network 'lan' config zone option name 'wan' option input 'DROP' option output 'ACCEPT' option forward 'DROP' option masq '1' option mtu_fix '1' option network 'wan wan6' config zone option name 'wan_229' option network 'wan_229' option masq '1' option input 'DROP' option output 'DROP' option forward 'DROP' config zone option name 'wan_230' option network 'wan_230' option masq '1' option input 'DROP' option output 'DROP' option forward 'DROP' config zone option name 'dmz_229' option network 'dmz_229' option input 'DROP' option output 'DROP' option forward 'DROP' config zone option name 'dmz_230' option network 'dmz_230' option input 'DROP' option output 'DROP' option forward 'DROP' config forwarding option dest 'wan_229' option src 'dmz_229' config forwarding option dest 'wan_230' option src 'dmz_230' config forwarding option dest 'dmz_229' option src 'lan' config forwarding option dest 'dmz_230' option src 'lan' config forwarding option dest 'wan' option src 'lan' |