System Installation
Optional: wipe solid-state drives with the generic tool blkdiscard, to clean previous partition tables and improve performance.
All content will be irrevocably destroyed:
for i in ${DISK}; do blkdiscard -f $i & done wait
This is a quick operation and should be completed under one minute.
For other device specific methods, see Memory cell clearing
Partition the disks. See Overview for details:
for i in ${DISK}; do sgdisk --zap-all $i sgdisk -n1:1M:+${INST_PARTSIZE_ESP}G -t1:EF00 $i sgdisk -n2:0:+${INST_PARTSIZE_BPOOL}G -t2:BE00 $i if [ "${INST_PARTSIZE_SWAP}" != "" ]; then sgdisk -n4:0:+${INST_PARTSIZE_SWAP}G -t4:8200 $i fi if [ "${INST_PARTSIZE_RPOOL}" = "" ]; then sgdisk -n3:0:0 -t3:BF00 $i else sgdisk -n3:0:+${INST_PARTSIZE_RPOOL}G -t3:BF00 $i fi sgdisk -a1 -n5:24K:+1000K -t5:EF02 $i done
Create boot pool:
disk_num=0; for i in $DISK; do disk_num=$(( $disk_num + 1 )); done if [ $disk_num -gt 1 ]; then INST_VDEV_BPOOL=mirror; fi zpool create \ -o compatibility=grub2 \ -o ashift=12 \ -o autotrim=on \ -O acltype=posixacl \ -O canmount=off \ -O compression=lz4 \ -O devices=off \ -O normalization=formD \ -O relatime=on \ -O xattr=sa \ -O mountpoint=/boot \ -R /mnt \ bpool_$INST_UUID \ $INST_VDEV_BPOOL \ $(for i in ${DISK}; do printf "$i-part2 "; done)
You should not need to customize any of the options for the boot pool.
GRUB does not support all of the zpool features. See
spa_feature_names
in grub-core/fs/zfs/zfs.c. This step creates a separate boot pool for/boot
with the features limited to only those that GRUB supports, allowing the root pool to use any/all features.Features enabled with
-o compatibility=grub2
can be seen here.Create root pool:
zpool create \ -o ashift=12 \ -o autotrim=on \ -R /mnt \ -O acltype=posixacl \ -O canmount=off \ -O compression=zstd \ -O dnodesize=auto \ -O normalization=formD \ -O relatime=on \ -O xattr=sa \ -O mountpoint=/ \ rpool_$INST_UUID \ $INST_VDEV \ $(for i in ${DISK}; do printf "$i-part3 "; done)
Notes:
The use of
ashift=12
is recommended here because many drives today have 4 KiB (or larger) physical sectors, even though they present 512 B logical sectors. Also, a future replacement drive may have 4 KiB physical sectors (in which caseashift=12
is desirable) or 4 KiB logical sectors (in which caseashift=12
is required).Setting
-O acltype=posixacl
enables POSIX ACLs globally. If you do not want this, remove that option, but later add-o acltype=posixacl
(note: lowercase “o”) to thezfs create
for/var/log
, as journald requires ACLsSetting
normalization=formD
eliminates some corner cases relating to UTF-8 filename normalization. It also impliesutf8only=on
, which means that only UTF-8 filenames are allowed. If you care to support non-UTF-8 filenames, do not use this option. For a discussion of why requiring UTF-8 filenames may be a bad idea, see The problems with enforced UTF-8 only filenames.recordsize
is unset (leaving it at the default of 128 KiB). If you want to tune it (e.g.-o recordsize=1M
), see these various blog posts.Setting
relatime=on
is a middle ground between classic POSIXatime
behavior (with its significant performance impact) andatime=off
(which provides the best performance by completely disabling atime updates). Since Linux 2.6.30,relatime
has been the default for other filesystems. See RedHat’s documentation for further information.Setting
xattr=sa
vastly improves the performance of extended attributes. Inside ZFS, extended attributes are used to implement POSIX ACLs. Extended attributes can also be used by user-space applications. They are used by some desktop GUI applications. They can be used by Samba to store Windows ACLs and DOS attributes; they are required for a Samba Active Directory domain controller. Note thatxattr=sa
is Linux-specific. If you move yourxattr=sa
pool to another OpenZFS implementation besides ZFS-on-Linux, extended attributes will not be readable (though your data will be). If portability of extended attributes is important to you, omit the-O xattr=sa
above. Even if you do not wantxattr=sa
for the whole pool, it is probably fine to use it for/var/log
.Make sure to include the
-part3
portion of the drive path. If you forget that, you are specifying the whole disk, which ZFS will then re-partition, and you will lose the bootloader partition(s).
This section implements dataset layout as described in overview.
Create root system container:
Unencrypted:
zfs create \ -o canmount=off \ -o mountpoint=none \ rpool_$INST_UUID/$INST_ID
Encrypted:
Pick a strong password. Once compromised, changing password will not keep your data safe. See
zfs-change-key(8)
for more info:zfs create \ -o canmount=off \ -o mountpoint=none \ -o encryption=on \ -o keylocation=prompt \ -o keyformat=passphrase \ rpool_$INST_UUID/$INST_ID
Create other system datasets:
zfs create -o canmount=off -o mountpoint=none bpool_$INST_UUID/$INST_ID zfs create -o canmount=off -o mountpoint=none bpool_$INST_UUID/$INST_ID/BOOT zfs create -o canmount=off -o mountpoint=none rpool_$INST_UUID/$INST_ID/ROOT zfs create -o canmount=off -o mountpoint=none rpool_$INST_UUID/$INST_ID/DATA zfs create -o mountpoint=/boot -o canmount=noauto bpool_$INST_UUID/$INST_ID/BOOT/default zfs create -o mountpoint=/ -o canmount=off rpool_$INST_UUID/$INST_ID/DATA/default zfs create -o mountpoint=/ -o canmount=noauto rpool_$INST_UUID/$INST_ID/ROOT/default zfs mount rpool_$INST_UUID/$INST_ID/ROOT/default zfs mount bpool_$INST_UUID/$INST_ID/BOOT/default for i in {usr,var,var/lib}; do zfs create -o canmount=off rpool_$INST_UUID/$INST_ID/DATA/default/$i done for i in {home,root,srv,usr/local,var/log,var/spool}; do zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/$i done chmod 750 /mnt/root
Format and mount ESP:
for i in ${DISK}; do mkfs.vfat -n EFI ${i}-part1 mkdir -p /mnt/boot/efis/${i##*/}-part1 mount -t vfat ${i}-part1 /mnt/boot/efis/${i##*/}-part1 done mkdir -p /mnt/boot/efi mount -t vfat ${INST_PRIMARY_DISK}-part1 /mnt/boot/efi
Create separate user dataset at
/home/User
, dateset name can be changed later:zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/home/User
If needed, snapshot, rollback and other related permissions can be delegated to the user later.
Create optional program data datasets to omit data from rollback:
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/games zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/www # for GNOME zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/AccountsService # for Docker zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/docker # for NFS zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/nfs # for LXC zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/lxc # for LibVirt zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/libvirt ##other application # zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/$name
Add other datasets when needed, such as PostgreSQL.
Install base packages:
dnf --installroot=/mnt --releasever=${INST_FEDORA_VER} -y install \ https://zfsonlinux.org/fedora/zfs-release.fc${INST_FEDORA_VER}.noarch.rpm \ @core grub2-efi-x64 grub2-pc-modules grub2-efi-x64-modules shim-x64 efibootmgr cryptsetup \ kernel kernel-devel python3-dnf-plugin-post-transaction-actions
Install ZFS:
dnf --installroot=/mnt -y install zfs zfs-dracut
Optional: enable boot environment support and dnf integration:
dnf --installroot=/mnt copr enable -y m0p/bieaz dnf --installroot=/mnt install -y bieaz python3-dnf-plugin-rozb3
If multi-disk setup is used, enable multi-disk support inside
/mnt/etc/bieaz.cfg
.