Gentoo as a router - Robin Hallabro-Kokko

After trying out pfSense, OPNsense and VyOS and not being entirely happy about either of them I’ve decided to install my good old favorite GNU/Linux distribution Gentoo on my router.

The router is a PCEngine APU2C2 bought from TekLager. I’ll try to format this more as a reference than a complete guide.

1. Installation media

wget https://sourceforge.net/projects/systemrescuecd/files/latest/download
isohybrid systemrescuecd-x86-*iso
dd if=systemrescuecd-x86-*iso of=/dev/sda

2. Booting

Insert the SD card and start the router. To get the serial port terminal working correctly you may pick the option:

Then select Standard 64bit kernel (rescue64) with more choice and SystemRescueCD with a console in 800x600 and press TAB to edit the options. From the prompt the option video=800x600 needs to be removed. And the following options need to be added:

console=ttyS0,115200 text

It should now say:

linux rescue64 initrd=initram.igz console=ttyS0,115200 text

Press Ctrl-X to boot.

3. Partitioning

/dev/sda gets partitioned into two pieces: 1 for /boot and 1 for rootfs. No UEFI partition needed because MBR will be used.

Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00000000

Device     Boot  Start      End  Sectors  Size Id Type
/dev/sda1  *      6144   415743   409600  200M 83 Linux
/dev/sda2       415744 31277055 30861312 14.7G 83 Linux

Set up the file systems:

mkfs.vfat /dev/sda1
mkfs.ext4 /dev/sda2

And mount them:

mkdir /mnt/{boot,rootfs}
mount /dev/sda2 /mnt/rootfs
mkdir /mnt/rootfs/boot
mount /dev/sda1 /mnt/rootfs/boot

4. Chroot

Grab the stage 3 tarball from http://distfiles.gentoo.org/releases/amd64/autobuilds/current-stage3-amd64/ and extract it. I use the nomultilib versions because no 32 bit applications are going to be used.

wget http://distfiles.gentoo.org/releases/amd64/autobuilds/current-stage3-amd64/stage3-amd64-nomultilib-20180830T214502Z.tar.xz
tar xpf stage3-amd64-nomultilib-20180830T214502Z.tar.xz
rm -f stage3-amd64-nomultilib-20180830T214502Z.tar.xz

mount -o bind /dev /mnt/rootfs/dev
mount -t proc none /mnt/rootfs/proc
mount -o bind /sys /mnt/rootfs/sys
cp /etc/resolv.conf /mnt/rootfs/etc
chroot /mnt/rootfs /bin/bash

5. Portage

My make.conf looks like this:

CFLAGS="-O2 -pipe -march=native"
CHOST="x86_64-pc-linux-gnu"
PORTDIR="/usr/portage"
DISTDIR="/usr/portage/distfiles"
CPU_FLAGS_X86="aes avx f16c mmx mmxext pclmul popcnt sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3"
USE="-dri -fortran -ipv6 -multilib"
ACCEPT_KEYWORDS="~amd64"
GRUB_PLATFORMS="efi-64"
FEATURES="noinfo nodoc noman"

6. Kernel

emerge gentoo-sources

The kernel configuration for 4.18.5-gentoo can be found here in case anyone wants to use it.

7. inittab

A serial console needs to be spawned during boot. Under the SERIAL CONSOLES section there should be a line saying:

# SERIAL CONSOLES
s0:12345:respawn:/sbin/agetty -L 115200 ttyS0 vt100

A small optimization can be made in inittab by not spawning any local TTYs. Remove or comment the lines in the TERMINALS section so it looks like this:

# TERMINALS
#x1:12345:respawn:/sbin/agetty 38400 console linux
#c1:12345:respawn:/sbin/agetty 38400 tty1 linux
#c2:2345:respawn:/sbin/agetty 38400 tty2 linux
#c3:2345:respawn:/sbin/agetty 38400 tty3 linux
#c4:2345:respawn:/sbin/agetty 38400 tty4 linux
#c5:2345:respawn:/sbin/agetty 38400 tty5 linux
#c6:2345:respawn:/sbin/agetty 38400 tty6 linux

8. Networking

Udev rules

Create udev rules to name the interfaces properly. It will be much easier to deal with them this way.

cat /etc/udev/rules.d/70-persistent-net.rules
SUBSYSTEM=="net", ACTION=="add", ATTR{address}=="redacted", NAME="wan0"
SUBSYSTEM=="net", ACTION=="add", ATTR{address}=="redacted", NAME="lan0"
SUBSYSTEM=="net", ACTION=="add", ATTR{address}=="redacted", NAME="lan1"
SUBSYSTEM=="net", ACTION=="add", ATTR{address}=="redacted", NAME="wlan0

Create a bridge

This will let the router acts as a switch on interfaces lan0 and lan1.

emerge net-misc/bridge-utils

cat /etc/conf.d/net
config_wan0="dhcp"

bridge_br0="lan0 lan1"
config_br0="192.168.0.1 netmask 255.255.255.0"
routes_br0="default via 192.168.0.1"

bridge_forward_delay_br0=0
bridge_hello_time_br0=1000

hostapd

As the name suggests hostapd sets up an access point.

emerge net-wireless/hostapd

For now I only use the 2.4GHz band.

cat /etc/hostapd/hostapd.conf
interface=wlan0
bridge=br0

logger_syslog=-1
logger_syslog_level=0
logger_stdout=-1
logger_stdout_level=2

ctrl_interface=/var/run/hostapd
ctrl_interface_group=0

ssid=redacted
country_code=redacted

ieee80211d=1
ieee80211h=1
ieee80211n=1
ieee80211ac=1

hw_mode=g
channel=10

auth_algs=1
wpa=2
wpa_key_mgmt=WPA-PSK
rsn_pairwise=CCMP
wpa_passphrase=redacted

driver=nl80211

nftables + Wireguard

Added 2019-03-07.

The following nftables config will set up will route all outgoing traffic through with the exception of traffic marked by wg-quick (see the init scripts provided),

Make sure to set vpn_port, fileserver_ip and port_to_forward which I have redacted.

# http://kangran.su/~nnz/pub/nf-doc/nftables/nft.html
# http://wiki.nftables.org/wiki-nftables/index.php/Main_Page

define external = wan0
define internal = br0
define vpn_out = wg0
define vpn_in = wg1
define vpn_port = <redacted>
define fileserver_ip = <redacted>
define port_to_forward = <redacted>

flush ruleset

table firewall {

	set blacklist {
		type ipv4_addr
	}

	set tcp_open_ports {
		type inet_service
	}

	set udp_open_ports {
		type inet_service
		elements = {
			$vpn_port
		}
	}

	chain outgoing-vpn {
		udp dport $vpn_port counter accept
	}

	chain incoming {
		type filter hook input priority 0

		# established/related connections
		ct state established,related accept

		# invalid connections
		ct state invalid drop

		# bad tcp -> avoid network scanning:
		tcp flags & (fin|syn) == (fin|syn) drop
		tcp flags & (syn|rst) == (syn|rst) drop
		tcp flags & (fin|syn|rst|psh|ack|urg) < (fin) drop
		tcp flags & (fin|syn|rst|psh|ack|urg) == (fin|psh|urg) drop

		# no ping floods:
		ip protocol icmp limit rate 10/second accept
		ip protocol icmp drop

		# drop connections from blacklisted addresses
		ip saddr @blacklist drop

		# accept input from loopback and internal interfaces
		iif { lo, $internal, $vpn_in } accept

		udp sport bootps udp dport bootpc counter accept

		# allow open tcp port
		tcp dport @tcp_open_ports accept

		# allow open udp ports
		udp dport @udp_open_ports accept

		reject
	}

	chain forwarding {
		type filter hook forward priority 0
		iif $external oif $internal ct state established,related accept
		iif $internal oif $external accept
		iif $internal oif $vpn_in accept
	}

	chain outgoing {
		type filter hook output priority 0
	}
}

table nat {
	map tcp_forwarding {
		type inet_service : ipv4_addr
	}

	map udp_forwarding {
		type inet_service : ipv4_addr
	}

	chain prerouting {
		type nat hook prerouting priority -100;
		iifname wg0 tcp dport $port_to_forward dnat $fileserver_ip
	}

	chain postrouting {
		type nat hook postrouting priority 100;
		policy accept;

		oifname { $external, $vpn_out, $vpn_in } masquerade
	}
}

Here are the Wireguard init scripts for reference. The two most important commands in the scripts are:

Make sure the fwmark matches on both interfaces in order to mark the traffic for nftables.
The MTU need to match on both interfaces or packets might be dropped when routing from wg1 (the “inbound” interface) to wg0 (the “outbound” interface).

Here’s /etc/init.d/wg0:

#!/sbin/openrc-run
# Copyright 1999-2018 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2

name="Wireguard"
description="Starts a given wireguard tunnel."
command=/usr/bin/wg-quick
command_args="${wireguard_args}"
interface=wg0

depend() {
    after net
}

start() {
    ebegin "Starting Wireguard client"
    $command up $interface
    eend $?
}

stop() {
    ebegin "Stopping Wireguard client"
    $command down $interface
    eend $?
}

And /etc/init.d/wg1(which is basically the same but adds the MTU and fwmark settings):

#!/sbin/openrc-run
# Copyright 1999-2018 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2

name="Wireguard"
description="Starts a given wireguard tunnel."
command=/usr/bin/wg-quick
command_args="${wireguard_args}"
wg_command=/usr/bin/wg
interface=wg1

depend() {
    after net
}

start() {
    ebegin "Starting Wireguard server"
    $command up $interface
    eend $?

    ebegin "Marking outgoing traffic"
    $wg_command set $interface fwmark <redacted>
    eend $?

    ebegin "Matching mtu value to wg0 interface"
    ifconfig wg1 mtu 1420
    eend $?
}

stop() {
    ebegin "Stopping Wireguard server"
    $command down $interface
    eend $?
}

Stuff I left out

Tons of important instructions.

1. Installation media#

2. Booting#

3. Partitioning#

4. Chroot#

5. Portage#

6. Kernel#

7. inittab#

8. Networking#

Udev rules#

Create a bridge#

hostapd#

nftables + Wireguard#

Stuff I left out#

Sources#