Custom Zig Klibs for the Nanos Unikernel

While we've got quite a few zig unikernels, I've been getting interested in zig more and more lately and wanted to make a klib that can utilize zig code.

Nanos klibs are a way to provide binary integration to your unikernels without it having to be in the same language. You can think of them like plugins. You'll see this pop up in things like checking into an IMDS server on the various clouds or as an apm agent or providing syslog funtionality.

You might want to make your own klib to create a tighter integration to an existing service we don't support. You might also want to use a different language than c to do this. Good news - you can. Today we're going to do this from zig.

The easiest way to get started on a custom klib is to copy an existing klib and drill it down to nothing like so.

Easy Custom Klib in C

#include <kernel.h>

int init(status_handler complete)
{
    rprintf("Test from custom klib\n");
    return KLIB_INIT_OK;
}

We touch up the makefile:

KLIBS= \
        azure \
+       bob \
        cloud_init \
        cloudwatch \
        digitalocean \
@@ -25,6 +26,9 @@ SRCS-azure= \
        $(KLIB_DIR)/azure.c \
        $(KLIB_DIR)/azure_diagnostics.c \

+SRCS-bob= \
+       $(KLIB_DIR)/bob.c \
+

Now you should be able to add code to your new bob klib.

Calling Zig from a C Klib:

Now let's start adding some Zig. Zig makes it relatively easy to call into c or vice-versa. The zig crew have also spent a bunch of time making their toolchain excellent, to the extent that some companies use the toolchain itself versus the language for various tasks.

We're also going to build this out of the nanos tree but you still need a nanos tree to build it with - at least for this example. One thing I'll call out is that most of the c <> zig tutorials out there will capitlize the 'callconv(.C)' but, as of this writing you want those to be 'callconv(.c)'.

We named the last one bob, let's name this one tim:

const std = @import("std");

pub export fn add(a: c_int, b: c_int) callconv(.c) c_int {
    return a + b;
}

We go ahead and compile this with freestanding-none.

#!/bin/sh

zig build-lib math.zig \
-target x86_64-freestanding-none \
-O ReleaseSafe \
 --name math

cp libmath.a ../.

Now we can call this from c:

#include <kernel.h>

extern int add(int a, int b);

int init(status_handler complete) {
  rprintf("Test from tim\n");

  int result = add(5, 7);

  rprintf("result is %d\n", result);

  if (result == 12) {
     rprintf("that is correct\n");
  }

  result = add(4, 4);

  if (result == 8) {
     rprintf("that is also correct\n");
  }

  return KLIB_INIT_OK;
}

Being built out of tree we still want access to various sources and we want to ensure we are building it under the same constraints so just set your NANOS_DIR to wherever you have your nanos code:

NANOS_DIR = /home/eyberg/go/src/github.com/nanovms/nanos

KERNEL=   $(NANOS_DIR)/bin/kernel.img

CFLAGS = -nostdinc -fno-builtin -O3 \
          -fPIC \
          -DBUILD_VDSO \
         -I$(NANOS_DIR)/src \
         -I$(NANOS_DIR)/src/kernel \
         -I$(NANOS_DIR)/src/unix \
         -I$(NANOS_DIR)/src/runtime \
         -I$(NANOS_DIR)/src/unix_process/ \
         -I$(NANOS_DIR)/output/platform/pc/ \
         -I$(NANOS_DIR)/src/x86_64

LDFLAGS = -static -shared -Bsymbolic -nostdlib -T$(NANOS_DIR)/src/x86_64/klib.lds

SRCS = tim.c
OBJ = tim.klib

all: $(OBJ)

$(OBJ): $(SRCS)
  $(CC) $(CFLAGS) -c $(SRCS) -o $(OBJ)
  $(LD) $(LDFLAGS) $(OBJ) libmath.a
  cp a.out ~/.ops/0.1.54/klibs/tim

clean:
  rm -f $(OBJ)
  rm -rf a.out
  rm -rf tim.klib

Now we can simply adjust our klib include in ops and run it.

{
  "Klibs": ["tim"]
}

Pure Zig Klib

Ok, how about a pure zig klib? In this example we build this out of tree and don't even require a tree, however, keep in mind at this point it'll be harder to access other existing code in nanos. It's something to think about because most operations you would do in a normal user program aren't going to work here and you might have to write a bunch of stuff from scratch.

First off, let's adjust our build script a bit:

#!/bin/sh

zig build-obj zim.zig \
  -target x86_64-freestanding-none \
  -O ReleaseSmall \
  -fPIC

ld -shared -nostdlib -T klib.ld zim.o -o zim.so -z max-page-size=4096 -z execstack

cp zim.so ~/.ops/0.1.54/klibs/zim

You'll notice that we are also using a custom linker script. We actually were using a linker script in the previous example as well but this one is tuned for use with the zig toolchain to play nicely with nanos.

ENTRY(init)

SECTIONS
{
    . = 0x0;

    .text : {
        *(.text .text.*)
    }

    .rodata : {
        *(.rodata .rodata.*)
    }

    .hash          : { *(.hash) }
    .gnu.hash      : { *(.gnu.hash) }
    .dynsym        : { *(.dynsym) }
    .dynstr        : { *(.dynstr) }
    .dynamic       : { *(.dynamic) }

    .data : {
        *(.data .data.*)
    }

    .bss : {
        *(.bss .bss.*)
    }

    /DISCARD/ : {
        *.note*
        *.comment
        *.eh_frame*
    }
}

You'll have to forgive me if my zig is poor as I haven't written much of it before. Unlike in the previous examples where we were in-tree and had access to rprintf (and numerous other functions) here we are writing directly to the serial console. Note: this is an extremely inefficient operation to do so this is merely an example.

const std = @import("std");

const klib_get_sym = ?*const fn ([*c]const u8) callconv(.c) ?*anyopaque;
const klib_add_sym = ?*const fn ([*c]const u8, ?*anyopaque) callconv(.c) void;

fn outb(port: u16, value: u8) void {
    asm volatile ("outb %[value], %[port]"
        :
        : [value] "{al}" (value),
          [port] "{dx}" (port),
    );
}

export fn init(md: ?*anyopaque, get_sym: klib_get_sym, add_sym: klib_add_sym) callconv(.c) ?*anyopaque {
    _ = md;
    _ = get_sym;
    _ = add_sym;

    const msg = "hello from zig\n";
    const COM1_PORT: u16 = 0x3f8;

    for (msg) |char| {
        outb(COM1_PORT, char);
    }

    return null;
}

Klibs aren't really designed for your day to day application needs. In general you should just run your application code inside your actual application but for more advanced situations, especially where you might need binary plugin like behavior across a fleet of unikernels these come in handy and now you know how you can start extending them with zig.

On the flip side if you have a need for something like this but you don't want to do it yourself the NanoVMs team can do this for you. Reach out to learn how we can support your company and we can make it happen.