Tuesday, May 12, 2009

ZFS root with gmirror

One of the people who influenced me to with BSD and specifically with FreeBSD is Jimmy B. Lim. I also met him online way way back in the 90's with Team FDN's core group Error 404. His words of challenge always echoes in my mind whenever I am faced with despair in front of my b0x... "Mag windows ka na lang!" (Just use Windows!) haha!

This approach with ZFS and mirroring has raised a question in my head with my first glance on his post from the mailing list. Why use gmirror when you can already use ZFS to mirror hard disk drives? I asked him this during one of our YM sessions and his answer is quite simple. This approach is much cleaner and with the least administrative effort. You don't need to manually edit your fstab and do other uncessary actions with your box :)

Let's get our hands dirty and configure our gmirror root[0]!



1. Install 2 identical hard drive on your box
2. Install FreeBSD-7.1
3. Partition your hard drive:
ad0s1 6GB (for 2GB RAM and this will be change if you want more
swap if your RAM is more than 4GB)
ad0s2
4. Make ad0s1a 2GB (just enough for bootdir, I know this is kinda big,
but it is much better to have spare than to have few space)
5. Make ad0s1b 4GB (twice the size of 2GB RAM)
6. Make ad0s2d as your ZFS partition
7. Partition your second drive (don't for get to make it bootable of
course) according the first drive. e.g. ad1)
8. Install FreeBSD-7.1
9. Configure gmirror root[0]
a. Setup gmirror on the second drive
gmirror label -v -n -b round-robin gm0 /dev/ad1s1
gmirror load
b. Format the gmirror partition and mount to /mnt
newfs -U /dev/mirror/gm0a
mount /dev/mirror/gm0a /mnt
c. Copy the / to newly created gmirror
dump -L -0 -f- / | (cd /mnt && restore -r -v -f-)
d. Setup the fstab for gmirror
vi /mnt/etc/fstab
# Device Mountpoint FStype Options
Dump Pass#
/dev/mirror/gm0b none swap sw
0 0
/dev/mirror/gm0a / ufs rw
1 1
e. Force to boot to the second drive and load gmirror
echo 'geom_mirror_load="YES"' >>/mnt/boot/loader.conf
echo ad1 | sed -e
's;^\([^0-9]*\)\([0-9][0-9]*\)$;1:\1(\2,a)/boot/loader;' \
>/boot.config
f. Reboot and cross your fingers
g. Once everything goes as we expected, insert the first drive on
the gmirror
gmirror configure -a gm0
gmirror insert -p 1 gm0 /dev/ad0s1
h. From this point the gmirror is now rebuilding the first drive
gmirror status
i. Once the rebuild is finished, reboot the server
10. Set up ZFS root[1]
mount -w /
zpool create tank /dev/ad0s2d
zfs set mountpoint=none tank
zfs create tank/root
zfs create tank/usr
zfs create tank/var
zfs create tank/tmp
zfs create tank/home
zfs set mountpoint=/tank tank/root
zfs set mountpoint=/tank/usr tank/usr
zfs set mountpoint=/tank/var tank/var
zfs set mountpoint=/tank/tmp tank/tmp
zfs set mountpoint=/tank/home tank/home

11. Check zfs settings
df -h
zfs list

12. Enable ZFS on boot
echo 'zfs_enable="YES"' >> /etc/rc.conf

13. Copy / to zfs root

13. Copy / to ZFS root
find -x / | cpio -pmd /tank

14. Remove /tank/boot to load gmirror boot
rm -rf /tank/boot
mkdir /tank/bootdir
cd /tank
ln -s bootdir/boot boot

15. Load ZFS on boot time
echo 'zfs_load="YES"' >> /boot/loader.conf
echo 'vfs.root.mountfrom="zfs:tank/root"' >> /boot/loader.conf

16. Edit fstab
/dev/mirror/gm0a /bootdir ufs
rw 1 1

17. Setup the mounting points of ZFS
zfs set mountpoint=/tmp tank/tmp
zfs set mountpoint=/usr tank/usr
zfs set mountpoint=/var tank/var
zfs set mountpoint=/home tank/home

18. Set the mountpoint legacy
cd /
zfs set mountpoint=legacy tank/root

19. Reboot

20. If everything goes as we expected, chech the ZFS root
df -h

21. Setup the secondary drive to mirror ZFS
zpool attach tank ad0s2d ad1s2d
zpool status

Thanks Jim!

"Now What?"