Posted on by dwm

First ZFS pool now on kiva

I finally got around to creating the first ZFS pool on my new-to-me server (kiva). At the moment, this particular pool is for backups of other machines.

I am using the 4TB HGST Deskstar drive I bought a little bit ago, and a 4TB HGST Deskstar NAS I bought today. Once installed in hot-swap bays, they showed up as da1 and da2.

I created the GPT partitioning scheme on each:

# gpart create -s gpt da1
# gpart create -s gpt da2

I created a partition on each, leaving 2 gigabytes of space unused. It’s not uncommon for a replacement drive to have slightly less space, and I don’t want to be trapped in a jam if one of the drives fails and I need to use a different type of drive as a replacement. 2 gigabytes seems like a lot of space, but in the grand scheme of this ZFS host, it’s nothing. On FreeBSD, there is no performance penalty for using partitions for ZFS versus using whole disks. This allows me to wait to buy a replacement disk, which means I don’t have spare disks sitting around with their warranty period ticking away without the drives being used. I would always have spare drives on hand in a production environment, but at home it makes sense (especially for backups) to wait for a drive to have some trouble before purchasing its replacement. 4TB drives are readily available locally.

# gpart add -t freebsd-zfs -l gpzfs1_0 -b1M -s3724G da1
# gpart add -t freebsd-zfs -l gpzfs1_1 -b1M -s3724G da2

So now I see:

% gpart show da1
=>        34    7814037101  da1  GPT  (3.6T)
          34          2014       - free -  (1.0M)
        2048    7809794048    1  freebsd-zfs  (3.6T)
  7809796096       4241039       - free -  (2.0G)

% gpart show da2
=>        34    7814037101  da2  GPT  (3.6T)
          34          2014       - free -  (1.0M)
        2048    7809794048    1  freebsd-zfs  (3.6T)
  7809796096       4241039       - free -  (2.0G)

I created the pool:

# zpool create zfs1 mirror /dev/gpt/gpzfs1_0 /dev/gpt/gpzfs1_1

I created my filesystem heirarchy. For now I only need my backups mount point. Since FreeBSD now has lz4_compress enabled by default, I can use lz4 compression. lz4 is considerably faster than lzjb, especially on incompressible data.

# zfs create -o compression=lz4 zfs1/backups

And since I had not yet enabled ZFS on kiva, I added to /boot/loader.conf:

zfs_load="YES"

And added to /etc/rc.conf:

zfs_enable="YES"

After copying over 38 gigabytes of backups from another host, I have this:

% zpool list -v
NAME               SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
zfs1              3.62T  23.9G  3.60T         -     0%     0%  1.00x  ONLINE  -
  mirror          3.62T  23.9G  3.60T         -     0%     0%
    gpt/gpzfs1_0      -      -      -         -      -      -
    gpt/gpzfs1_1      -      -      -         -      -      -

lz4 compression yielded a 37% reduction in disk space for these backups. That’s quite reasonable.

A friend asked me why I was using a mirror. The simple answer is that it’s more reliable than raidzN, and more easily expanded. This machine has 12 hot-swap drive bays, and I don’t expect to need all of them anytime soon (if ever). While a raidzN is more space-efficient, it’s not easily expanded and when one drive from a batch fails, others are often not far behind. Resilvering a raidzN is hard on all of the drives involved, and it’s not uncommon to have another disk fail during a resilvering. Resilvering a raidzN is slower than resilvering a mirror, and array performance suffers dramatically during resilvering of a raidzN. If/when I need to add more space to the pool, I can simply buy two more drives and add another mirror to the pool.

It’s worth noting that ZFS is not a substitute for backups. Here I am using ZFS to store backups of other machines, and it’s very useful for this use case.

Leave a Reply