Create ext2 filesystem from a directory

This might interest/disgust/shock/scare(?!) people on this list:

$ time ./nbdkit --filter=partition -U - linuxdisk . partition=1 --run 'qemu-img convert $nbd /var/tmp/ext2fs.img'

real   0m1.314s
user   0m0.424s
sys    0m0.889s

$ ls -lh /var/tmp/ext2fs.img 
-rw-r--r--. 1 rjones rjones 351M Feb 19 07:44 /var/tmp/ext2fs.img

Code here:

https://github.com/rwmjones/nbdkit/commit/6e7908c828e60f082d84d866070fe8406e6f2b04

Rich.

Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v

For those of us who aren't really familiar with nbdkit, what does this do?

- Ted

There's quite a lot packed into a small space ...

Firstly nbdkit is an NBD server which is unique in having plugins to implement different data sources. In this case I have written a plugin called "linuxdisk" which creates a GPT-partitioned ext2-formatted disk image from a local directory ("." in this case).

nbdkit can serve over IP or Unix domain sockets, including randomly created Unix domain sockets ("-U -").

nbdkit can also run other commands and exit when those commands have finished (we call this "captive nbdkit"). In this case we ask nbdkit to run a qemu-img command and then exit when qemu-img finishes.

"qemu-img convert $nbd /var/tmp/ext2fs.img" connects to the private Unix domain socket (nbdkit sets $nbd to a suitable string to make this happen). It then copies the data out of that NBD socket to a local file, optionally doing a format conversion although not in this case.

Finally nbdkit has a concept of filters which can be placed on top of plugins to select parts of the data. Remember that the "linuxdisk" plugin creates GPT-partitioned ext2 disk images. However we want only the naked filesystem. Therefore we place the partition filter on top to pick out and serve only the partition content, which is the naked ext2 filesystem.

I recently did a talk about nbdkit if you want to find out more:
https://rwmj.wordpress.com/2019/02/04/video-take-your-loop-mounts-to-the-next-level-with-nbdkit/

Here's another fun thing you can do with the linuxdisk plugin:
https://rwmj.wordpress.com/2019/02/19/nbdkit-linuxdisk-plugin/

Hope that helps!

Rich.

Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top

How does this compare to "mke2fs -d <source_dir>"?

Cheers, Andreas

Wow, good question. I was completely unaware of this option until now, but it'll make the implementation massively simpler. (We still need the linuxdisk plugin.)

Rich.

Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/