The reduced attack surface area and ease of deployment of unikernels pair very well with the simplicity and security of the WireGuard protocol. We can combine the two and create a VPN gateway to give us secure access to a private network. Many people may be familiar with the WireGuard support built into the Linux kernel, but instead of reimplementing the protocol in Nanos, we can just use one of the well-known userspace implementations of WireGuard such as wireguard-go or, as we'll see here, boringtun, written in Rust.
Our objective is to create a vm that will listen for WireGuard connections and then forward that traffic to the cloud's private network. Our private network is subnet 10.240.0.0/16 and we'll make our VPN subnet 10.0.0.0/24. Since our vm has to do IP forwarding, we'll need to compile a version of Nanos with IP forwarding support enabled. Don't worry, this is actually very easy! Check the Nanos README to make sure you have the required prerequisites installed before building. We'll have to make one small change to enable IP forwarding in LWIP and then we'll have our special kernel ready to go.
~/wg$ git clone https://github.com/nanovms/nanos && cd nanos
Cloning into 'nanos'...
remote: Enumerating objects: 21976, done.
remote: Counting objects: 100% (2724/2724), done.
remote: Compressing objects: 100% (957/957), done.
remote: Total 21976 (delta 1966), reused 2411 (delta 1758), pack-reused 19252
Receiving objects: 100% (21976/21976), 6.59 MiB | 21.63 MiB/s, done.
Resolving deltas: 100% (16881/16881), done.
~/wg/nanos$ echo '#define IP_FORWARD 1' >> src/net/lwipopts.h
~/wg/nanos$ make
The next thing we have to do is get a version of boringtun that will work for our unikernel vm. If you are already familiar with WireGuard, you are aware of using the command wg or wg-quick to configure it. This command connects to the WireGuard instance (in kernel or userspace) that is listening on a UNIX socket and uses a special configuration protocol in order to set keys and peers. Since this configuration happens via a second process and our unikernel only runs one, what do we do? We use a patch and build a version of boringtun that can read the configuration protocol from a file rather than from a second process. You can download the patch here that adds a new argument -c <config> to read a file of configuration protocol commands on startup. Make sure you have a Rust environment installed and then you can compile this modified boringtun very easily. Note that if you aren't building boringtun on Linux, you will need to cross-compile it for 64-bit Linux so that it can run on Nanos.
~/wg$ curl -sO https://nanovms.com/static/dev/boringtun.diff
~/wg$ git clone https://github.com/cloudflare/boringtun && cd boringtun
Cloning into 'boringtun'...
remote: Enumerating objects: 756, done.
remote: Counting objects: 100% (44/44), done.
remote: Compressing objects: 100% (38/38), done.
remote: Total 756 (delta 14), reused 19 (delta 5), pack-reused 712
Receiving objects: 100% (756/756), 783.00 KiB | 7.75 MiB/s, done.
Resolving deltas: 100% (482/482), done.
~/wg/boringtun$ patch -p1 < ../boringtun.diff
patching file src/device/mod.rs
patching file src/main.rs
~/wg/boringtun$ cargo build --bin boringtun --release
With our binaries now ready, we can create the config files needed for the image. Let's first create the wireguard config file. Note that this file will be different from the one you usually use with the wg or wg-quick commands. It follows the WireGuard configuration protocol, so we have to convert the traditional file to this protocol format. We can do that with a clever use of netcat (or nc). Let's take this configuration for the VPN server, listening on port 51820 with two peers that can connect to it. The VPN server will be 10.0.0.1, so these two peers will be 10.0.0.2 and 10.0.0.3. Do not use these keys below, be sure to generate your own keys!
[Interface]
PrivateKey = OKrznnAWZhskqHMx9F0JD46pMZWWEkeWqdyFK1PGVXI=
ListenPort = 51820
[Peer]
PublicKey = 6SCWZuT/kO8Q2C6L3ynL2K583YcVuG7FwoMiH7xrpFs=
AllowedIps = 10.0.0.2/32
[Peer]
PublicKey = 5DApKFqCrqDE2/PMR+3xIMHFwGBdguq2XK7KaK0zy1U=
AllowedIps = 10.0.0.3/32
~/wg$ ./wg-convert nanoswg.conf > boringtun.conf
~/wg$ cat boringtun.conf
set=1
private_key=38aaf39e7016661b24a87331f45d090f8ea9319596124796a9dc852b53c65572
listen_port=51820
fwmark=0
replace_peers=true
public_key=e9209666e4ff90ef10d82e8bdf29cbd8ae7cdd8715b86ec5c283221fbc6ba45b
replace_allowed_ips=true
allowed_ip=10.0.0.2/32
public_key=e43029285a82aea0c4dbf3cc47edf120c1c5c0605d82eab65caeca68ad33cb55
replace_allowed_ips=true
allowed_ip=10.0.0.3/32
Finally, we'll create the config.json file that ops needs to create our vm image. See the example below. We specify a few arguments for boring tun:
- -f for running in foreground
- -v debug to get debug messages printed to the console
- -c boringtun.conf to read our configuration file on startup
- --disable-drop-privileges yes because there aren't privileges to drop
- wg2 which will be the name of our tunnel interface
{
"Args": ["-f", "-v", "debug", "-c", "boringtun.conf", "--disable-drop-privileges", "yes", "wg2"],
"Files": ["boringtun.conf"],
"Dirs": ["var"],
"RunConfig": {
"Klibs": ["tun"],
"UDPPorts": ["51820"]
},
"Boot": "nanos/output/platform/pc/boot/boot.img",
"Kernel": "nanos/output/platform/pc/bin/kernel.img",
"ManifestPassthrough": {
"tun": {
"wg": {
"ipaddress": "10.0.0.1",
"netmask": "255.255.255.0",
"up": "true"
}
}
},
"CloudConfig": {
"ProjectID": "my-cloud",
"Zone": "us-west2-a",
"BucketName": "my-bucket"
}
}
At last we can create our image.
~/wg$ ls
boringtun boringtun.diff nanos var
boringtun.conf config.json nanoswg.conf wg-convert
~/wg$ ops image create boringtun/target/release/boringtun -c config.json -t gcp
Image creation started. Monitoring operation operation-1519128474355-5ca694bb32624-30387e93-2914fd45.
.................
Operation operation-1519128474355-5ca694bb32624-30387e93-2914fd45 completed successfully.
Image creation succeeded boringtun.
gcp image 'boringtun' created...
We now need to create our instance from the image. Usually we do this with ops instance create but since our vm is going to forward traffic across networks we must enable IP forwarding in GCP on the instance at creation time. We will do this by manually creating an instance from our image with IP forwarding enabled. We must also add a network tag to allow UDP port 51820 or else the vm will not receive the WireGuard requests. The final piece of setup is that we must add a route to the private network to send VPN traffic (10.0.0.0/24) back through our gateway instance for two-way communication with the clients outside.
Once the gateway is set up and the cloud private network configured, all that is left is to configure our local WireGuard peer to connect to the gateway and we have access to our private network through a Nanos gateway running boringtun!
~$ cat utun2.conf
[Interface]
PrivateKey = 6G5U9fOiA5qVbWMlJ0wx05RiJhfvFj+b2Diit+EKW2o=
Address = 10.0.0.3/32
[Peer]
PublicKey = hOtvMDcRRJJttd8xETZ/17W+gn3BbHfKpLQVpmEJ6zk=
Endpoint = 34.35.212.58:51820
AllowedIPs = 10.0.0.0/24,10.240.0.0/16
~$ sudo wg-quick up ./utun2.conf
[#] wireguard-go utun
[+] Interface for utun2 is utun3
[#] wg setconf utun3 /dev/fd/63
[#] ifconfig utun3 inet 10.0.0.3/32 10.0.0.3 alias
[#] ifconfig utun3 up
[#] route -q -n add -inet 10.0.0.0/24 -interface utun3
[#] route -q -n add -inet 10.240.0.0/16 -interface utun3
[+] Backgrounding route monitor
~$ ping -c 4 10.240.0.2
PING 10.240.0.2 (10.240.0.2): 56 data bytes
64 bytes from 10.240.0.2: icmp_seq=0 ttl=63 time=106.790 ms
64 bytes from 10.240.0.2: icmp_seq=1 ttl=63 time=70.320 ms
64 bytes from 10.240.0.2: icmp_seq=2 ttl=63 time=65.156 ms
64 bytes from 10.240.0.2: icmp_seq=3 ttl=63 time=65.344 ms
--- 10.240.0.2 ping statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 65.156/76.903/106.790/17.379 ms
With a little bit of effort you can run a userspace WireGuard on the Nanos unikernel, combining the security of WireGuard and unikernels to make a secure VPN gateway. Note that we have IP masquerading on the roadmap, which will vastly simplify vm instance creation by eliminating the need to make special routing rules as well as get rid of the hassle of setting IP forwarding on the vm. As Nanos continues to evolve and improve running this application and others like it will only get easier!
