Message ID: ZFS-8000-ER
ZFS Errata #1
Type: |
Compatibility |
Severity: |
Moderate |
Description: |
The ZFS pool contains an on-disk format incompatibility. |
Automated Response: |
No automated response will be taken. |
Impact: |
Until the pool is scrubbed using OpenZFS version 0.6.3 or newer the pool may not be imported by older versions of OpenZFS or other ZFS implementations. |
Suggested Action for System Administrator
The pool contains an on-disk format incompatibility. Affected pools must be imported and scrubbed using the current version of ZFS. This will return the pool to a state in which it may be imported by other implementations. This errata only impacts compatibility between ZFS versions, no user data is at risk as result of this erratum.
# zpool status -x
pool: test
state: ONLINE
status: Errata #1 detected.
action: To correct the issue run 'zpool scrub'.
see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-ER
scan: none requested
config:
NAME STATE READ WRITE CKSUM
test ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
vdev0 ONLINE 0 0 0
vdev1 ONLINE 0 0 0
vdev2 ONLINE 0 0 0
vdev3 ONLINE 0 0 0
errors: No known data errors
# zpool scrub test
# zpool status -x
all pools are healthy
ZFS Errata #2
Type: |
Compatibility |
Severity: |
Moderate |
Description: |
The ZFS packages were updated while an asynchronous destroy was in progress and the pool contains an on-disk format incompatibility. |
Automated Response: |
No automated response will be taken. |
Impact: |
The pool cannot be imported until the issue is corrected. |
Suggested Action for System Administrator
Affected pools must be reverted to the previous ZFS version where they can be correctly imported. Once imported, all asynchronous destroy operations must be allowed to complete. The ZFS packages may then be updated and the pool can be imported cleanly by the newer software.
# zpool import
pool: test
id: 1165955789558693437
state: ONLINE
status: Errata #2 detected.
action: The pool cannot be imported with this version of ZFS due to
an active asynchronous destroy. Revert to an earlier version
and allow the destroy to complete before updating.
see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-ER
config:
test ONLINE
raidz1-0 ONLINE
vdev0 ONLINE
vdev1 ONLINE
vdev2 ONLINE
vdev3 ONLINE
Revert to previous ZFS version, import the pool, then wait for the
freeing property to drop to zero. This indicates that all
outstanding asynchronous destroys have completed.
# zpool get freeing
NAME PROPERTY VALUE SOURCE
test freeing 0 default
The ZFS packages may be now be updated and the pool imported. The on-disk format incompatibility can now be corrected online as described in Errata #1.
ZFS Errata #3
Type: |
Compatibility |
Severity: |
Moderate |
Description: |
An encrypted dataset contains an on-disk format incompatibility. |
Automated Response: |
No automated response will be taken. |
Impact: |
Encrypted datasets created before the ZFS packages were updated cannot be mounted or opened for write. The errata impacts the ability of ZFS to correctly perform raw sends, so this functionality has been disabled for these datasets. |
Suggested Action for System Administrator
System administrators with affected pools will need to recreate any
encrypted datasets created before the new version of ZFS was used.
This can be accomplished by using zfs send and zfs receive.
Note, however, that backups can NOT be done with a raw zfs send -w,
since this would preserve the on-disk incompatibility.
Alternatively, system administrators can use conventional tools to
back up data to new encrypted datasets. The new version of ZFS will
prevent new data from being written to the impacted datasets, but
they can still be mounted read-only.
# zpool status
pool: test
id: 1165955789558693437
state: ONLINE
status: Errata #3 detected.
action: To correct the issue backup existing encrypted datasets to new
encrypted datasets and destroy the old ones.
see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-ER
config:
test ONLINE
raidz1-0 ONLINE
vdev0 ONLINE
vdev1 ONLINE
vdev2 ONLINE
vdev3 ONLINE
Import the pool and backup any existing encrypted datasets to new
datasets. To ensure the new datasets are re-encrypted, be sure to
receive them below an encryption root or use zfs receive -o
encryption=on, then destroy the source dataset.
# zfs send test/crypt1@snap1 | zfs receive -o encryption=on -o keyformat=passphrase -o keylocation=file:///path/to/keyfile test/newcrypt1
# zfs send -I test/crypt1@snap1 test/crypt1@snap5 | zfs receive test/newcrypt1
# zfs destroy -R test/crypt1
New datasets can be mounted read-write and used normally. The errata
will be cleared upon reimporting the pool and the alert will only be
shown again if another dataset is found with the errata. To ensure
that all datasets are on the new version reimport the pool, load all
keys, mount all encrypted datasets, and check zpool status.
# zpool export test
# zpool import test
# zfs load-key -a
Enter passphrase for 'test/crypt1':
1 / 1 key(s) successfully loaded
# zfs mount -a
# zpool status -x
all pools are healthy
ZFS Errata #4
Type: |
Compatibility |
Severity: |
Moderate |
Description: |
An encrypted dataset contains an on-disk format incompatibility. |
Automated Response: |
No automated response will be taken. |
Impact: |
Encrypted datasets created before the ZFS packages
were updated cannot be backed up via a raw send to
an updated system. These datasets also cannot
receive additional snapshots. New encrypted
datasets cannot be created until the
|
Suggested Action for System Administrator
First, system administrators with affected pools will need to enable
the bookmark_v2 feature on their pools. Enabling this feature
will prevent this pool from being imported by previous versions of
the ZFS software after any new bookmarks are created (including
read-only imports). If the pool contains no encrypted datasets, this
is the only step required. If there are existing encrypted datasets,
administrators will then need to back these datasets up. This can be
done in several ways. Non-raw zfs send and zfs receive can be
used as per usual, as can traditional backup tools. Raw receives of
existing encrypted datasets and raw receives into existing encrypted
datasets are currently disabled because ZFS is not able to guarantee
that the stream and the existing dataset came from a consistent
source. This check can be disabled which will allow ZFS to receive
these streams anyway. Note that this can result in datasets with data
that cannot be accessed due to authentication errors if raw and
non-raw receives are mixed over the course of several incremental
backups. To disable this restriction, set the
zfs_disable_ivset_guid_check module parameter to 1. Streams
received this way (as well as any received before the upgrade) will
need to be manually checked by reading the data to ensure they are
not corrupted. Note that zpool scrub cannot be used for this
purpose because the scrub does not check the cryptographic
authentication codes. For more information on this issue, please
refer to the zfs man page section on zfs receive which describes
the restrictions on raw sends.
# zpool status
pool: test
state: ONLINE
status: Errata #4 detected.
Existing encrypted datasets contain an on-disk incompatibility
which needs to be corrected.
action: To correct the issue enable the bookmark_v2 feature and backup
any existing encrypted datasets to new encrypted datasets and
destroy the old ones. If this pool does not contain any
encrypted datasets, simply enable the bookmark_v2 feature.
see: http://openzfs.github.io/openzfs-docs/msg/ZFS-8000-ER
scan: none requested
config:
NAME STATE READ WRITE CKSUM
test ONLINE 0 0 0
/root/vdev0 ONLINE 0 0 0
errors: No known data errors
Import the pool and enable the bookmark_v2 feature. Then backup
any existing encrypted datasets to new datasets. This can be done
with traditional tools or via zfs send. Raw sends will require
that the zfs_disable_ivset_guid_check is set to 1 on the receive
side. Once this is done, the original datasets should be destroyed.
# zpool set feature@bookmark_v2=enabled test
# echo 1 > /sys/module/zfs/parameters/zfs_disable_ivset_guid_check
# zfs send -Rw test/crypt1@snap1 | zfs receive test/newcrypt1
# zfs send -I test/crypt1@snap1 test/crypt1@snap5 | zfs receive test/newcrypt1
# zfs destroy -R test/crypt1
# echo 0 > /sys/module/zfs/parameters/zfs_disable_ivset_guid_check
The errata will be cleared upon reimporting the pool and the alert
will only be shown again if another dataset is found with the errata.
To check that all datasets are fixed, perform a zfs list -t all,
and check zpool status once it is completed.
# zpool export test
# zpool import test
# zpool scrub # wait for completion
# zpool status -x
all pools are healthy