Learning ZFS on Root Setup for Raspberry Pi 4B.
Learning ZFS on Root Setup for Raspberry Pi 4B.
Author - Shahriar Shovon
The following property can be set at creation time and import time:
- altroot
- Alternate root directory. If set, this directory is prepended to any mount points within the pool. This can be used when examining an unknown pool where the mount points cannot be trusted, or in an alternate boot environment, where the typical paths are not valid. altroot is not a persistent property. It is valid only while the system is up. Setting altroot defaults to using cachefile=none, though this may be overridden using an explicit setting.
The following properties can be set at creation time and import
time, and later changed with the zpool
set
command:
-
- bootfs=(unset)|pool[/dataset]
- Identifies the default bootable dataset for the root pool. This property is expected to be set mainly by the installation and upgrade programs. Not all Linux distribution boot processes use the bootfs property.
https://openzfs.github.io/openzfs-docs/man/master/8/index.html Index of zpool and zfs commands
https://openzfs.github.io/openzfs-docs/man/master/8/zpool-create.8.html Zpool create command
https://openzfs.github.io/openzfs-docs/man/master/7/zpool-features.7.html Zpool features
https://openzfs.github.io/openzfs-docs/man/master/8/zpool.8.html
zpool create -O compress=lz4 -O atime=off -o altroot=/arm64poolalt -m none arm64pool /dev/gpt/rootzfs
zpool import -R /mnt -N -f arm64pool
reddit example:
zpool destroy zpi || true
zpool create -O compress=lz4 -O atime=off -o altroot=/zpialt -m none zpi da1s2
Manipulating a pool from the rescue system
https://klarasystems.com/articles/manipulating-a-pool-from-the-rescue-system/
First, import the pool with an ‘altroot’ (a path prepended to all of the mountpoints, so as not to mount over top of the live system you are using). We also set the “do not automatically mount filesystems” flag, because we need to manually mount the boot environment.
# zpool import -R /mnt -N -f zroot
Once that completes, confirm you can see all of your datasets:
# mount -t zfs zroot/ROOT/default /mnt
If you are not sure what is currently the default boot environment, you can use:
# zpool get bootfs zroot
And it will return the name of the current default boot environment. Once you have mounted the root directory, you can mount the rest of the ZFS datasets:
# zfs mount -a
Now your broken system is mounted with a prefix of /mnt, so rc.conf will be in /mnt/etc/rc.conf, and you can edit files as required to repair your system.
Then just reboot without the USB/CD image connected, and your system should boot as normal.
July 22, 2023
arm64pool/ROOT/default /mnt
arm64pool/tmp /mnt/tmp
arm64pool/usr/src /mnt/usr/src
arm64pool/var/mail /mnt/var/mail
arm64pool/usr/ports /mnt/usr/ports
arm64pool/var/audit /mnt/var/audit
arm64pool/var/log /mnt/var/log
arm64pool/var/tmp /mnt/var/tmp
arm64pool/var/crash /mnt/var/crash
arm64pool/home /mnt/arm64pool/home
USEFUL ZFS, ZPOOL commands:
Beginner's guide to ZFS. Part 16: Importing and Exporting Pools
Let's take a look into ZFS and zpools on FreeBSD GaryH Tech
OUTPUT from my 1 TB arm64pool, zpool
usb_msc_auto_quirk: UQ_MSC_NO_PREVENT_ALLOW set for USB mass storage device ULT-Best Best USB Device (0x152d:0x0562)
umass1 on uhub0
umass1: <ULT-Best Best USB Device, class 0/0, rev 3.10/41.03, addr 5> on usbus0
umass1: SCSI over Bulk-Only; quirks = 0x8100
umass1:1:1: Attached to scbus1
da1 at umass-sim1 bus 1 scbus1 target 0 lun 0
da1: <Samsung SSD 870 QVO 1TB 4103> Fixed Direct Access SPC-4 SCSI device
da1: Serial Number 333457EBD0F2
da1: 400.000MB/s transfers
da1: 953869MB (1953525168 512 byte sectors)
da1: quirks=0x2<NO_6_BYTE>
Sep 18 09:13:02 generic login[1449]: ROOT LOGIN (root) ON ttyv0
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
ugen0.6: <ULT-Best Best USB Device> at usbus0 (disconnected)
umass1: at uhub0, port 3, addr 5 (disconnected)
da1 at umass-sim1 bus 1 scbus1 target 0 lun 0
da1: <Samsung SSD 870 QVO 1TB 4103> s/n 333457EBD0F2 detached
(da1:umass-sim1:1:0:0): Periph destroyed
umass1: detached
usb_msc_auto_quirk: UQ_MSC_NO_GETMAXLUN set for USB mass storage device ULT-Best Best USB Device (0x152d:0x0562)
usb_msc_auto_quirk: UQ_MSC_NO_PREVENT_ALLOW set for USB mass storage device ULT-Best Best USB Device (0x152d:0x0562)
ugen0.6: <ULT-Best Best USB Device> at usbus0
umass1 on uhub0
umass1: <ULT-Best Best USB Device, class 0/0, rev 3.10/41.03, addr 5> on usbus0
umass1: SCSI over Bulk-Only; quirks = 0x8100
umass1:1:1: Attached to scbus1
da1 at umass-sim1 bus 1 scbus1 target 0 lun 0
da1: <Samsung SSD 870 QVO 1TB 4103> Fixed Direct Access SPC-4 SCSI device
da1: Serial Number 333457EBD0F2
da1: 400.000MB/s transfers
da1: 953869MB (1953525168 512 byte sectors)
da1: quirks=0x2<NO_6_BYTE>
root@generic:~ # zfs list
no datasets available
root@generic:~ # zpool list
no pools available
root@generic:~ # zpool import
pool: arm64pool
id: 9016811727597087920
state: ONLINE
status: Some supported features are not enabled on the pool.
(Note that they may be intentionally disabled if the
'compatibility' property is set.)
action: The pool can be imported using its name or numeric identifier, though
some features will not be available without an explicit 'zpool upgrade'.
config:
arm64pool ONLINE
gpt/zfs ONLINE
root@generic:~ #
pool: arm64pool
state: ONLINE
status: Some supported and requested features are not enabled on the pool.
The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
the pool may no longer be accessible by software that does not support
the features. See zpool-features(7) for details.
config:
NAME STATE READ WRITE CKSUM
arm64pool ONLINE 0 0 0
gpt/zfs ONLINE 0 0 0
errors: No known data errors
root@generic:~ #
you have mail
/dev/ufs/rootfs on / (ufs, local, soft-updates)
devfs on /dev (devfs)
/dev/msdosfs/EFI on /boot/efi (msdosfs, local, noatime)
tmpfs on /tmp (tmpfs, local)
arm64pool/tmp on /tmp (zfs, local, noatime, nosuid, nfsv4acls)
arm64pool/var/mail on /var/mail (zfs, local, nfsv4acls)
arm64pool/usr/src on /usr/src (zfs, local, noatime, nfsv4acls)
arm64pool/var/tmp on /var/tmp (zfs, local, noatime, nosuid, nfsv4acls)
arm64pool/var/crash on /var/crash (zfs, local, noatime, noexec, nosuid, nfsv4acls)
arm64pool/var/audit on /var/audit (zfs, local, noatime, noexec, nosuid, nfsv4acls)
arm64pool/var/log on /var/log (zfs, local, noatime, noexec, nosuid, nfsv4acls)
arm64pool/usr/ports on /usr/ports (zfs, local, noatime, nosuid, nfsv4acls)
arm64pool/poudriere on /arm64pool/poudriere (zfs, local, noatime, nfsv4acls)
arm64pool/home on /arm64pool/home (zfs, local, noatime, nfsv4acls)
arm64pool/poudriere/ports/ghostbsd_ports on /usr/local/poudriere/ports/ghostbsd_ports (zfs, local, noatime, nfsv4acls)
arm64pool/poudriere/jails/ghostbsd-13-2-aarch64 on /usr/local/poudriere/jails/ghostbsd-13-2-aarch64 (zfs, local, noatime, nfsv4acls)
root@generic:~ # zfs mount
arm64pool/tmp /tmp
arm64pool/var/mail /var/mail
arm64pool/usr/src /usr/src
arm64pool/var/tmp /var/tmp
arm64pool/var/crash /var/crash
arm64pool/var/audit /var/audit
arm64pool/var/log /var/log
arm64pool/usr/ports /usr/ports
arm64pool/poudriere /arm64pool/poudriere
arm64pool/home /arm64pool/home
arm64pool/poudriere/ports/ghostbsd_ports /usr/local/poudriere/ports/ghostbsd_ports
arm64pool/poudriere/jails/ghostbsd-13-2-aarch64 /usr/local/poudriere/jails/ghostbsd-13-2-aarch64
root@generic:~ #
root@generic:~ # zfs get canmount,mounted,mountpoint
NAME PROPERTY VALUE SOURCE
arm64pool canmount off local
arm64pool mounted no -
arm64pool mountpoint /arm64pool local
arm64pool/ROOT canmount on default
arm64pool/ROOT mounted no -
arm64pool/ROOT mountpoint none local
arm64pool/ROOT/default canmount noauto local
arm64pool/ROOT/default mounted no -
arm64pool/ROOT/default mountpoint / local
arm64pool/ROOT/default@1970-01-01-00:18:15-0 canmount - -
arm64pool/ROOT/default@1970-01-01-00:18:15-0 mounted - -
arm64pool/ROOT/default@1970-01-01-00:18:15-0 mountpoint - -
arm64pool/ROOT/freds_first canmount noauto local
arm64pool/ROOT/freds_first mounted no -
arm64pool/ROOT/freds_first mountpoint / local
arm64pool/home canmount on default
arm64pool/home mounted yes -
arm64pool/home mountpoint /arm64pool/home inherited from arm64pool
arm64pool/poudriere canmount on default
arm64pool/poudriere mounted yes -
arm64pool/poudriere mountpoint /arm64pool/poudriere inherited from arm64pool
arm64pool/poudriere/jails canmount on default
arm64pool/poudriere/jails mounted no -
arm64pool/poudriere/jails mountpoint /arm64pool/poudriere/jails inherited from arm64pool
arm64pool/poudriere/jails/ghostbsd-13-2-aarch64 canmount on default
arm64pool/poudriere/jails/ghostbsd-13-2-aarch64 mounted yes -
arm64pool/poudriere/jails/ghostbsd-13-2-aarch64 mountpoint /usr/local/poudriere/jails/ghostbsd-13-2-aarch64 local
arm64pool/poudriere/jails/ghostbsd-13-2-aarch64@clean canmount - -
arm64pool/poudriere/jails/ghostbsd-13-2-aarch64@clean mounted - -
arm64pool/poudriere/jails/ghostbsd-13-2-aarch64@clean mountpoint - -
arm64pool/poudriere/ports canmount on default
arm64pool/poudriere/ports mounted no -
arm64pool/poudriere/ports mountpoint /arm64pool/poudriere/ports inherited from arm64pool
arm64pool/poudriere/ports/ghostbsd_ports canmount on default
arm64pool/poudriere/ports/ghostbsd_ports mounted yes -
arm64pool/poudriere/ports/ghostbsd_ports mountpoint /usr/local/poudriere/ports/ghostbsd_ports local
arm64pool/tmp canmount on default
arm64pool/tmp mounted yes -
arm64pool/tmp mountpoint /tmp local
arm64pool/usr canmount off local
arm64pool/usr mounted no -
arm64pool/usr mountpoint /usr local
arm64pool/usr/ports canmount on default
arm64pool/usr/ports mounted yes -
arm64pool/usr/ports mountpoint /usr/ports inherited from arm64pool/usr
arm64pool/usr/src canmount on default
arm64pool/usr/src mounted yes -
arm64pool/usr/src mountpoint /usr/src inherited from arm64pool/usr
arm64pool/var canmount off local
arm64pool/var mounted no -
arm64pool/var mountpoint /var local
arm64pool/var/audit canmount on default
arm64pool/var/audit mounted yes -
arm64pool/var/audit mountpoint /var/audit inherited from arm64pool/var
arm64pool/var/crash canmount on default
arm64pool/var/crash mounted yes -
arm64pool/var/crash mountpoint /var/crash inherited from arm64pool/var
arm64pool/var/log canmount on default
arm64pool/var/log mounted yes -
arm64pool/var/log mountpoint /var/log inherited from arm64pool/var
arm64pool/var/mail canmount on default
arm64pool/var/mail mounted yes -
arm64pool/var/mail mountpoint /var/mail inherited from arm64pool/var
arm64pool/var/tmp canmount on default
arm64pool/var/tmp mounted yes -
arm64pool/var/tmp mountpoint /var/tmp inherited from arm64pool/var
root@generic:~ #
https://docs.oracle.com/cd/E19253-01/819-5461/gazru/index.html
Exporting a ZFS Storage Pool
To export a pool, use the zpool export command. For example:
# zpool export tank |
The command attempts to unmount any mounted file systems within the pool before continuing. If any of the file systems fail to unmount, you can forcefully unmount them by using the -f option. For example:
# zpool export tank cannot unmount '/export/home/eschrock': Device busy # zpool export -f tank |
After this command is executed, the pool tank is no longer visible on the system.
If devices are unavailable at the time of export, the devices cannot be identified as cleanly exported. If one of these devices is later attached to a system without any of the working devices, it appears as “potentially active.”
If ZFS volumes are in use in the pool, the pool cannot be exported, even with the -f option. To export a pool with a ZFS volume, first ensure that all consumers of the volume are no longer active.Example of exporting a zpool
root@generic:~ # zfs list
no datasets available
zpool root@generic:~ # zpool list
no pools available
Example of zpool import -R /mnt/arm64zfs -N -f arm64pool
total 8
drwxr-xr-x 2 root wheel 512 Jul 9 05:50 espfat32
drwxr-xr-x 2 root wheel 512 Jul 9 05:51 freebsd14_ufs
root@generic:~ # gpart show -lp
=> 63 30261185 da0 MBR (14G)
63 1985 - free - (993K)
2048 102400 da0s1 (null) [active] (50M)
104448 30156800 da0s2 (null) (14G)
=> 0 30156800 da0s2 BSD (14G)
0 128 - free - (64K)
128 27125632 da0s2a (null) (13G)
27125760 3031040 da0s2b (null) (1.4G)
root@generic:~ # zfs list
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
no datasets available
root@generic:~ # zpool list
no pools available
root@generic:~ # zpool import
no pools available to import
root@generic:~ # usb_msc_auto_quirk: UQ_MSC_NO_GETMAXLUN set for USB mass storage device ULT-Best Best USB Device (0x152d:0x0562)
usb_msc_auto_quirk: UQ_MSC_NO_PREVENT_ALLOW set for USB mass storage device ULT-Best Best USB Device (0x152d:0x0562)
ugen0.6: <ULT-Best Best USB Device> at usbus0
umass1 on uhub0
umass1: <ULT-Best Best USB Device, class 0/0, rev 3.10/41.03, addr 5> on usbus0
umass1: SCSI over Bulk-Only; quirks = 0x8100
umass1:1:1: Attached to scbus1
da1 at umass-sim1 bus 1 scbus1 target 0 lun 0
da1: <Samsung SSD 870 QVO 1TB 4103> Fixed Direct Access SPC-4 SCSI device
da1: Serial Number 333457EBD0F2
da1: 400.000MB/s transfers
da1: 953869MB (1953525168 512 byte sectors)
da1: quirks=0x2<NO_6_BYTE>
root@generic:~ # zfs list
no datasets available
root@generic:~ # zpool import -R /mnt/arm64zfs -N -f arm64pool
root@generic:~ # ls /mnt
efibootarm esp rootfs zfs
root@generic:~ # mount
/dev/ufs/rootfs on / (ufs, local, soft-updates)
devfs on /dev (devfs)
/dev/msdosfs/EFI on /boot/efi (msdosfs, local, noatime)
tmpfs on /tmp (tmpfs, local)
root@generic:~ # df -h -t
df: option requires an argument -- t
usage: df [-b | -g | -H | -h | -k | -m | -P] [-acilnT] [-t type] [-,]
[file | filesystem ...]
root@generic:~ # df -h
Filesystem Size Used Avail Capacity Mounted on
/dev/ufs/rootfs 13G 2.9G 8.7G 25% /
devfs 1.0K 0B 1.0K 0% /dev
/dev/msdosfs/EFI 50M 25M 25M 51% /boot/efi
tmpfs 8.9G 4.0K 8.9G 0% /tmp
root@generic:~ # zpool list
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
arm64pool 832G 49.7G 782G - - 1% 5% 1.00x ONLINE /mnt/arm64zfs
root@generic:~ #
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
arm64pool 832G 49.7G 782G - - 1% 5% 1.00x ONLINE /mnt/arm64zfs
root@generic:~ # zfs list
NAME USED AVAIL REFER MOUNTPOINT
arm64pool 49.7G 757G 96K /mnt/arm64zfs/arm64pool
arm64pool/ROOT 23.5G 757G 96K none
arm64pool/ROOT/default 23.5G 757G 22.7G /mnt/arm64zfs
arm64pool/ROOT/freds_first 8K 757G 982M /mnt/arm64zfs
arm64pool/home 96K 757G 96K /mnt/arm64zfs/arm64pool/home
arm64pool/poudriere 5.21G 757G 104K /mnt/arm64zfs/arm64pool/poudriere
arm64pool/poudriere/jails 2.14G 757G 96K /mnt/arm64zfs/arm64pool/poudriere/jails
arm64pool/poudriere/jails/ghostbsd-13-2-aarch64 2.14G 757G 2.14G /mnt/arm64zfs/usr/local/poudriere/jails/ghostbsd-13-2-aarch64
arm64pool/poudriere/ports 3.07G 757G 96K /mnt/arm64zfs/arm64pool/poudriere/ports
arm64pool/poudriere/ports/ghostbsd_ports 3.07G 757G 3.07G /mnt/arm64zfs/usr/local/poudriere/ports/ghostbsd_ports
arm64pool/tmp 236K 757G 236K /mnt/arm64zfs/tmp
arm64pool/usr 21.0G 757G 96K /mnt/arm64zfs/usr
arm64pool/usr/ports 19.8G 757G 19.8G /mnt/arm64zfs/usr/ports
arm64pool/usr/src 1.21G 757G 1.21G /mnt/arm64zfs/usr/src
arm64pool/var 1.06M 757G 96K /mnt/arm64zfs/var
arm64pool/var/audit 96K 757G 96K /mnt/arm64zfs/var/audit
arm64pool/var/crash 96K 757G 96K /mnt/arm64zfs/var/crash
arm64pool/var/log 508K 757G 508K /mnt/arm64zfs/var/log
arm64pool/var/mail 192K 757G 192K /mnt/arm64zfs/var/mail
arm64pool/var/tmp 96K 757G 96K /mnt/arm64zfs/var/tmp
Example of importing a zpool to an alternate mount point for servicing
-R
root- Sets the cachefile property to none and the altroot property to root.
arm64pool 832G 49.7G 782G - - 1% 5% 1.00x ONLINE /mnt
root@generic:/mnt # ls -l
total 16
drwxr-xr-x 2 root wheel 512 Jul 3 21:39 efibootarm
drwxr-xr-x 2 root wheel 512 Sep 18 07:32 esp
drwxr-xr-x 2 root wheel 512 Sep 18 07:40 rootfs
drwxr-xr-x 2 root wheel 512 Sep 18 09:15 zfs
mkdir /mnt/arm64zfs
mount -t zfs arm64pool/ROOT/default /mnt/arm64zfs
root@generic:~ # chroot /mnt/arm64zfs /bin/sh
root@generic:/ # pwd
/
root@generic:/ # ls
.cshrc COPYRIGHT bin dev etc libexec net root tmp
.profile arm64pool boot efibootarm home media proc sbin usr
.snap arm64zfs d,rfosrootfs entropy lib mnt rescue test1.txt var
root@generic:/ # cd /usr
root@generic:/usr # ls
bin lib libexec patch_vchiq_D37878_diff.txt share
ghostbsd-src lib32 local ports src
include libdata obj sbin tests
root@generic:/usr # cd src
Comments
Post a Comment