Linux Router Home Network
I have a fairly complicated network at home... at least for a home network. I've done a lot of research and pulled together several pieces of knowledge, and wanted to share my setup for those of you who are doing the same.
Topology
(Apologies for my visual skills, this was done quickly)
Overview
I have a linux PC running debian acting as a router. The PC is setup primarily as a server with a few extra NIC cards on it (represented via the enp interfaces). I keep wifi on one, and wired on the other.. which will be nice for the day I need to filter wireless traffic more.
The machine also has a fair number of qemu-based and lxc-based virtual machines that have their own veth interfaces.
Lastly, the server runs two different VPNs (two different use cases). One is a layer-2 VPN (openvpn tap), and the other is layer 3 (openvpn tun).
Interfaces
All the internal interfaces and VMs are bridged to br0. br0 uses the network 192.168.8.0/24,
though I may switch that to a /20 at some point as IoT takes over my house.
vpn0 is not part of the bridge since it's layer 3, so it has its own network 192.168.10.0/24. Rules
are installed so that the systems can talk across subnets, and the local route is pushed downstream via
openvpn config.
Lastly, enp1s0 is the external interface that connects directly to the modem (not a router!).
I try to keep the rest of the machine pretty bare-bones and run everything else on lxc. The
one exception I do have is nginx which does reverse proxying.
Tooling
Most of the tooling comes fairly standard: iptables, ip, ss, (ifconfig and netstat for legacy systems) etc.
miniupnpd to enable upnp on the network. Setup is not covered by this, but it should be as easy as setting it up on the right subnets.
I use monitorix and vnstat to monitor the interfaces.
tcpdump can be useful for debugging, along with ping, iftop, and mtr.
netfilter-persistent to save the configuration and restore them on reset.
dnsmasq to act both as a DHCP server (assign IPs) and DNS relay. dnsmasq is bound to br0 only. It will
also provide a clean method to resolve machine names on your internal network. I chose to add a suffix pseudo-TLD, but
you don't need to.
Allowing Forwarding
When setting this up, make sure to uncomment net.ipv4.ip_forward=1 in /etc/sysctl.conf, and run sysctl -p to reload, otherwise
your interfaces won't be able to talk to each other.
Configuration
Here are my core configs, with some added comments to help out.
Interface Configs: /etc/network/interfaces
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
auto enp1s0
iface enp1s0 inet dhcp # Use DHCP here to acquire IP from ISP
name WAN Interface
hwaddress ether 01:23:24:56:67 # SPECIAL NOTE: I had to clone the MAC of my old router so that my ISP would connect to it. You may or may not have to do the same
auto enp4s6
iface enp4s6 inet manual
auto enp3s0
iface enp3s0 inet manual
# Bridge
auto br0
iface br0 inet static # Static IP. Will use dnsmasq to assign to downstream clients
bridge_ports enp4s6 enp3s0 # The other interfaces self-add per their configuration
address 192.168.8.1
netmask 255.255.255.0
bridge_stp off # You may have to use STP if connecting multiple routers
bridge_maxwait 0
bridge_fd 0
Router iptables
Put this script wherever, and run it as root to apply it:
WARNING: Having local access (or pseudo-local-access) to the OS is critical. A typo or bad code can result it no longer having access to the server's network.
NOTE: This script does not really account for IPv6 (matter of fact, it blocks all IPv6 traffic except for localhost). Should you need it,
you'll likely have to replicate a lot of the config for ip6tables.
#/bin/bash
set -e
# Flush all existing rules
echo "Flushing..."
iptables -F INPUT
iptables -F OUTPUT
iptables -F FORWARD
iptables -F -t nat
ip6tables -F INPUT
ip6tables -F OUTPUT
ip6tables -F FORWARD
# Set up interface defaults
echo "Setting defaults..."
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -P OUTPUT ACCEPT # NOTE: As personal preference, I allow all output from this machine globally. In a more restricted environment you might not
ip6tables -P INPUT DROP
ip6tables -P OUTPUT DROP
ip6tables -P FORWARD DROP
##############################################
# enable NAT
echo "Setting up NAT..."
# The main NAT that will mask the external address, but allow established connections
iptables -t nat -A POSTROUTING -o enp1s0 -j MASQUERADE
iptables -A INPUT -i enp1s0 -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -i enp1s0 -p udp --dport 67:68 --sport 67:68 -j ACCEPT # Allow incoming DHCP data so we can establish/renew our external IP
iptables -A FORWARD -i enp1s0 -m state --state ESTABLISHED,RELATED -j ACCEPT
# Allow all inner communication of trusted br0
iptables -A FORWARD -i br0 -j ACCEPT
iptables -A INPUT -i br0 -j ACCEPT
# Some firewall protection. Protects the internal network from receiving forged packets from the outside
iptables -A INPUT -i enp1s0 -d 192.168.0.0/16 -j DROP
iptables -A INPUT -i enp1s0 -s 192.168.0.0/16 -j DROP
# Allow all localhost traffic (trafic to self)
echo "Setting up lo..."
iptables -A INPUT -i lo -j ACCEPT
ip6tables -A INPUT -i lo -j ACCEPT
ip6tables -A OUTPUT -o lo -j ACCEPT
# Bridging subnets for vpn0
# openvpn will set up the routes for me
iptables -A FORWARD -i vpn0 -j ACCEPT
iptables -A FORWARD -o vpn0 -j ACCEPT
iptables -A INPUT -i vpn0 -j ACCEPT
##############################################
# LOCAL Network rules
echo "Setting up input..."
IPIN="iptables -A INPUT -i enp1s0 -j ACCEPT"
$IPIN -p icmp
#SSH
$IPIN -p tcp --dport 22 --syn
#HTTP
$IPIN -p tcp --dport 88 --syn
$IPIN -p tcp --dport 443 --syn
# Other things here I've redacted
##############################################
# PORT forwarding
echo "Port forwarding..."
# Some nice helper functions:
function tcp_forward() {
INPORT=$1
IP=$2
FWDPORT=${3:-$INPORT}
echo " Forwarding TCP $INPORT -> $IP:$FWDPORT"
iptables -t nat -A PREROUTING -i enp1s0 -p tcp --dport $INPORT -j DNAT --to-destination $IP:${FWDPORT/:/-}
iptables -A FORWARD -p tcp -d $IP --dport $FWDPORT -i enp1s0 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT
}
function udp_forward() {
INPORT=$1
IP=$2
FWDPORT=${3:-$INPORT}
echo " Forwarding UDP $INPORT -> $IP:$FWDPORT"
iptables -t nat -A PREROUTING -i enp1s0 -p udp --dport $INPORT -j DNAT --to-destination $IP:${FWDPORT/:/-}
iptables -A FORWARD -p udp -d $IP --dport $FWDPORT -i enp1s0 -j ACCEPT
}
# Some examples: (Modify for your preference)
# tcp_forward 3483:5839 192.168.8.111
# udp_forward 2222:3481 192.168.8.103
####################################################
# save it all for reboot
echo "Saving to netfilter..."
service netfilter-persistent save
# Need to reset some services once new config as been applied
# systemctl restart monitorix
# systemctl restart miniupnpd
Dnsmasq Config: /etc/dnsmasq.conf
I won't post the entire file, but the changes I've made from the default config:
# Add .lan to all machines on the network
domain=lan
# Specify IP range
dhcp-range=192.168.8.100,192.168.8.200,12h
# Reserve some IPs
dhcp-host=12:34:56:78:91:21,192.168.8.105
# Use custom downstream DNS config files
server=8.8.8.8
server=8.8.4.4
# Don't use config from /etc/resolv.conf for domains (provided by ISP)
no-poll
no-resolv
# Override interface to listen on
interface=br0
Summary
I hope this provides a good stepping stone for you. Good luck!