CrossBuilding Unikernels on Mac without Docker or Vagrant

Many developers develop on Macs but deploy their software to Linux servers. This is a problem since Macs use the Mach-o file format and Linux uses ELF - but it goes deeper than that. Most applications (like 99%) are dynamically linked and every extension out there for any interpreted language is usually dynamically loaded as a shared library. So you can't just pick up your software on a mac and shove it onto a linux system. This is arguably one of the positive points of using something like Docker although it never seems to be stated this way. It is usually stated as the "work on my machine" problem which doesn't really provide clarity into why.

OPS has been able to build, deploy, and run Nanos unikernels on Macs for a long time now, however for languages or applications where it was hard to cross-compile, end-users would resort to running things in a vm and copying things down or running them in a docker container and importing them (which on a Mac means it is in a vm regardless).

We're excited to share that we now have initial cross-build support for those end-users that need/want it and was built specifically for those on Macs. Want to try it out? This is hot off the PR merge so if you aren't reading this in the future you'll need to build OPS from source.

Let's walk through a real world example. Node works fine on a mac through the various packages the ops registry has but what if you want to run the redis fast driver? Since it is written in c we're going to have to compile it, and then load it into the node runtime and since we are on a mac that means cross-compilation. Probably not something you personally want to do if all you want to do is use the driver.

ops env install

apt-get install -y nodejs make gcc g++
npm install redis-fast-driver --save

{
  "Dirs": ["node_modules"],
  "Program": "/usr/bin/node"
}

There is more that gets extracted out but ops is smart enough to do the legwork on that.

Now we build it and set the target root to our working directory:

ops env build build.txt -c config.json -r .

You'll notice that after it builds your local directory will now have the artifacts that are necessary to run your unikernel locally outside of the linux guest including the node_modules and the various libraries that were linked:

➜  redis-test ls
build.txt     config.json   config_1.json example.js    lib
lib64         node          node_modules  usr

➜  redis-test tree lib*
lib
└── x86_64-linux-gnu
    ├── libc.so.6
    ├── libdl.so.2
    ├── libgcc_s.so.1
    ├── libm.so.6
    ├── libpthread.so.0
    └── libstdc++.so.6
lib64
└── ld-linux-x86-64.so.2

1 directory, 7 files

Notice that we didn't just grab eveything from the vm - we wouldn't even need a full chroot because we're not actually running in linux. We cross-build in linux but run in nanos. Now if we wish to run the new node application *outside* of the guest linux vm in the guest nanos unikernel we can with this config:

{
  "Dirs": ["node_modules"],
  "Files": ["/lib/x86_64-linux-gnu/libstdc++.so.6", "example.js"],
  "Program": "/usr/bin/node",
  "Args": ["example.js"]
}

You can use this example to test. Just set the host as 10.0.0.2, for user-mode, and use the 'redis-fast-driver' that is commented out.

➜  redis-test ops run usr/bin/node -c config_1.json -r .

booting /Users/eyberg/.ops/images/node.img ...
en1: assigned 10.0.2.15
en1: assigned FE80::FC65:67FF:FEF2:F10D
=================================================
===
Start test: PING command 2500 times

And that my friends, is how you cross-compile a Linux program to a Nanos unikernel on a Mac without Docker or Vagrant.