Recently, I upgraded my FreeNAS server from 3 TB drives to 8 TB drives after getting a great deal on the new ones. However, the resilver of my RAID-Z2 array was slow—clocking in at about 40 MB/s.
After doing some research, I discovered some kernel options that allowed the resilver to get up to about 70 MB/s. They can be set on FreeBSD or FreeNAS with the following commands:
sysctl vfs.zfs.top_maxinflight=128
sysctl vfs.zfs.resilver_min_time_ms=6000
sysctl vfs.zfs.resilver_delay=0
The first option increases the queue depth for top-level ZFS devices. In my case, I have one RAID-Z2 with eight drives, so it makes sense to allow more than the default of 32 in-flight operations across eight disks.
The second allows ZFS to spend more time resilvering before it flushes pending transactions.
The last option gives resilver I/O the same priority as any user interation with the filesystem. This is fine in my case because I want the resilver to complete as quickly as possible at the expense of slightly slower user interaction.
Of course, you may have to adjust these depending on your system. Don’t forget to set them to their previous values once you’re done with your resilver.