Today's article is about working with external volumes for your unikernel instance. Many applications won't use this feature but there are also situations where this makes sense.
The most obvious time when you would want to attach a volume to your unikernel instance is when you are working with a database. Your database image probably doesn't change that much but many databases can grow very large and it makes no sense to keep the same data volume in your base image.
The second use-case is when you have a base image, say a webserver, and you want to package additional files or configuation as a build step. For instance some companies will rotate certificates every few hours in the day to protect access to various services and this rotation is usually done out-of-band of deploys. Now unikernel deploys for small webservers are typically fairly fast but the ability to put your configuration on a separate partition and re-mount the volume on the fly every few hours is definitely enticing.
Ok, let's start with the code. For this example we have a simple little go webserver that implements a root filesystem filewalker. We've declared that there is a separate partition called 'mnt' in the code but it is non-existent right now.
import (
"fmt"
"io/ioutil"
"net/http"
"os"
"path/filepath"
)
func printDir() {
err := filepath.Walk("/",
func(path string, info os.FileInfo, err error) error {
if err != nil {
return err
}
fmt.Println(path, info.Size())
return nil
})
if err != nil {
fmt.Println(err)
}
}
func main() {
printDir()
b, err := ioutil.ReadFile("/mnt/bob.txt")
if err != nil {
fmt.Println(err)
}
fmt.Println(string(b))
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
printDir()
b, err := ioutil.ReadFile("/mnt/bob.txt")
if err != nil {
fmt.Println(err)
}
fmt.Println(string(b))
fmt.Fprintf(w, "Welcome to my website!")
})
fs := http.FileServer(http.Dir("static/"))
http.Handle("/static/", http.StripPrefix("/static/", fs))
go func() {
err = http.ListenAndServe(":80", nil)
if err != nil {
fmt.Println(err)
}
}()
http.ListenAndServe(":8080", nil)
}
➜ g2 ops run -p 8080 g2
booting /Users/eyberg/.ops/images/g2.img ...
assigned: 10.0.2.15
/ 0
/dev 0
/dev/null 0
/dev/urandom 0
/etc 0
/etc/passwd 33
/etc/resolv.conf 18
/etc/ssl 0
/etc/ssl/certs 0
/etc/ssl/certs/ca-certificates.crt 207436
/g2 7533614
/lib 0
/lib/x86_64-linux-gnu 0
/lib/x86_64-linux-gnu/libnss_dns.so.2 26936
/proc 0
/proc/self 0
/proc/self/exe 0
/proc/self/maps 0
/proc/sys 0
/proc/sys/kernel 0
/proc/sys/kernel/hostname 7
/sys 0
/sys/devices 0
/sys/devices/system 0
/sys/devices/system/cpu 0
/sys/devices/system/cpu/cpu0 0
/sys/devices/system/cpu/online 0
open /mnt/bob.txt: no such file or directory
Creating a Volume
Let's create a simple volume with one file in it - bob.txt.
mkdir mnt
echo "Hi - I'm a text file" > mnt/bob.txt
➜ g2 ops volume create mnt -d mnt
2020/12/08 11:12:35 volume: mnt created with UUID 04c56e4a-5b8b-512c-eaa3-b82b4cd46d9e and label mnt
➜ g2 ops volume list
+--------------------------------------+------+--------+-----------+-------------------------------------------------------------------------+---------+----------+
| UUID | NAME | STATUS | SIZE (GB) | LOCATION | CREATED | ATTACHED |
+--------------------------------------+------+--------+-----------+-------------------------------------------------------------------------+---------+----------+
| 04c56e4a-5b8b-512c-eaa3-b82b4cd46d9e | mnt | | 1.6 MB | /Users/eyberg/.ops/volumes/mnt:04c56e4a-5b8b-512c-eaa3-b82b4cd46d9e.raw | | |
+--------------------------------------+------+--------+-----------+-------------------------------------------------------------------------+---------+----------+
Attaching a Volume
You can attach a volume to an instance that is expecting one. So that means when we create the image we'll want to pass any mount points with the volume label and mount path - this is loosely similar to how something in /etc/fstab would work. Ran locally you can test with 'ops run' but you can pass the same '--mounts' flag when issuing 'ops image create' for images ran on Google or AWS. Let's try it out locally:
➜ g2 ops run -p 8080 g2 --mounts mnt:/mnt
booting /Users/eyberg/.ops/images/g2.img ...
assigned: 10.0.2.15
/ 0
/dev 0
/dev/null 0
/dev/urandom 0
/etc 0
/etc/passwd 33
/etc/resolv.conf 18
/etc/ssl 0
/etc/ssl/certs 0
/etc/ssl/certs/ca-certificates.crt 207436
/g2 7533614
/lib 0
/lib/x86_64-linux-gnu 0
/lib/x86_64-linux-gnu/libnss_dns.so.2 26936
/mnt 0
/mnt/bob.txt 21
/proc 0
/proc/self 0
/proc/self/exe 0
/proc/self/maps 0
/proc/sys 0
/proc/sys/kernel 0
/proc/sys/kernel/hostname 7
/sys 0
/sys/devices 0
/sys/devices/system 0
/sys/devices/system/cpu 0
/sys/devices/system/cpu/cpu0 0
/sys/devices/system/cpu/online 0
Hi - I'm a text file
Cool! It works! Now let's edit the file.
echo "New text has come to light." > mnt/bob.txt
➜ g2 ops volume create mnt2 -d mnt
2020/12/08 11:19:41 volume: mnt2 created with UUID f82da0e3-3980-ddd8-5720-e1b320e21371 and label mnt2
Keep in mind we are creating a *new* volume with new contents and then re-attaching the volume to the instance.
➜ g2 ops run -p 8080 g2 --mounts mnt2:/mnt
booting /Users/eyberg/.ops/images/g2.img ...
assigned: 10.0.2.15
/ 0
/dev 0
/dev/null 0
/dev/urandom 0
/etc 0
/etc/passwd 33
/etc/resolv.conf 18
/etc/ssl 0
/etc/ssl/certs 0
/etc/ssl/certs/ca-certificates.crt 207436
/g2 7533614
/lib 0
/lib/x86_64-linux-gnu 0
/lib/x86_64-linux-gnu/libnss_dns.so.2 26936
/mnt 0
/mnt/bob.txt 28
/proc 0
/proc/self 0
/proc/self/exe 0
/proc/self/maps 0
/proc/sys 0
/proc/sys/kernel 0
/proc/sys/kernel/hostname 7
/sys 0
/sys/devices 0
/sys/devices/system 0
/sys/devices/system/cpu 0
/sys/devices/system/cpu/cpu0 0
/sys/devices/system/cpu/online 0
New text has come to light.
If you are attaching the volume to an instance on Google or AWS you'd use the attach command:
ops volume attach g2 mnt mnt2 -t gcp -c config.jsonSimilarly, you can detach as well:
ops volume detach g2 mnt -t gcp -c config.jsonWhat's really great about unikernel volumes when working on AWS or Google is that this is all managed for you by the cloud provider of choice. There is no duplicate storage layer you have to manage like you do in container land. Now you know the basics of mounting external volumes into your unikernel images.
