Enhancing our ZFS support on Ubuntu 19.10 – an introduction

Ubuntu has supported ZFS as an option for some time. We started with a file-based ZFS pool on Ubuntu 15.10, then delivered it as a FS container in 16.04, and  recommended it for the fastest and most reliable container experience on LXD.

We have also created some dedicated tutorials for users who want to become more familiar with ZFS concepts, like on basic layouts and taking snapshots.

To do all this, we are using the excellent ZFS On Linux implementation which has a vibrant and active upstream community. This is built as a kernel module and therefore, no DKMS is involved.  

Three years ago we spent time looking at the licensing which applies to the Linux kernel and to ZFS.  Our conclusions are that we are acting within the rights granted and in compliance with the terms on both licenses.

By working towards adding support for ZFS as the root file system, we will bring the benefits of ZFS to Ubuntu users through an easy to use interface and automated operations, abstracting some of the complexity while still allowing flexibility for power users.

ZFS & Ubuntu 19.10

We announced 6 months ago that support for deploying Ubuntu root on ZFS with MAAS was available as an experimental feature.

So, what’s new for Ubuntu 19.10 (Eoan Ermine)?  As has already been reported, spotted in our weekly team report on Ubuntu discourse, we are going to enhance ZFS on root support in the coming cycles. Ubuntu 19.10 is a first round towards that goal.

We want to support ZFS on root as an experimental installer option, initially for desktop, but keeping the layout extensible for server later on. The desktop will be the first beneficiary in Ubuntu 19.10. Note the use of the term ‘experimental’ though! As we want to have the dataset layout right and we know a file system is very crucial as it’s responsible for all your data, we don’t want to encourage people to use it on production systems yet, or at least, not without regular backups. The option will be highlighted as such – you are now warned! However, feel free to play with it and pass on feedback. 

What’s already in Eoan?

The worked started several weeks ago and Eoan already has some nice improvements concerning ZFS:

• We are shipping ZFS On Linux version 0.8.1, with features like native encryption, trimming support, checkpoints, raw encrypted zfs transmissions, project accounting and quota and a lot of performance enhancements. You can see more about 0.8 and 0.8.1 released on the ZOL project release page directly.
• We backported (and will continue to backport) some post-release upstream fixes as they fit, to provide the best user experience and reliability.
• We added a new support in the GRUB menu. A small preview is available below and a more detailed blog post will be presented later on.

Any existing ZFS on root user will automatically get those benefits as soon as they update to Ubuntu 19.10.

Incoming work with zsys

The goal here is to make some of the ZFS basic and advanced concepts easily accessible and transparent to anyone, like providing automated snapshots, an easy way to rollback, offline instant updates, easy backup support and so on. This is to make a solid and robust system which is configured correctly by default. This work therefore focuses on not requiring a deep understanding of ZFS, but still being to make use of the advanced features.

However, we are aware that some system administrators are very passionate about the file systems and want to be in control. This is why we designed our system in such a way that it can cope with manual tweaking, is easy to understand for people having some know-how on ZFS, and still remains very flexible.

Finally, we want to provide some best practices in terms of the ZFS dataset layout for various needs. For example, a daily desktop user will need reliability and for it to be easy to revert to a stable state, while a system administrator will want to optimise, tweak heavily and have persistent datasets, even when rolling back his/her server operating system.

For this, we are developing a new user space daemon, named zsys. This will cooperate with GRUB (but is not limited to it) and ZFS on Linux initramfs to give advanced features we’ll describe later on. Our goal is to upstream as much as possible to GRUB and ZOL project maintainers when things are solid enough.

The whole current progress and what’s up next is accessible via our public card project against the Ubuntu GitHub organisation via this link.

We didn’t do it alone

Thanks to early press coverage, we got in touch with Richard Laager who maintains the upstream HOWTO on root on ZFS for Ubuntu. After some back and forth on the draft specification, we came to some good conclusions and he slightly modified the HOWTO to be more compatible with our plans.

Similarly, Marcin Skadetailrbek got in touch as well and wants to bring ZSYS to Fedora. However, some adjustments will be needed, making this is a longer-term project.

Both have been commenting and contributing on a specification (Work In Progress) that we wrote and a copy is accessible here for those who want to see what’s cooking in the longer term plan. Thanks to them for their very useful input!

More work ahead

As you can see, the future of ZFS as root on Ubuntu is bright. We still have a lot to tackle and 19.10 will be only the beginning of the journey. However, the path forward is exciting and we hope to be able to bring something fresh and unique to ZFS users.

More blog posts will follow to shed more light on these enhancements and report as to our status.

Join the discussion via the dedicated Ubuntu discourse thread.