Some time ago a feature was added to use sane default zvol blocksize based on pool topology .

Upon investigation it seems 3 x 5 wide raidz1 still gets a 16k volblocksize recommendation.

root@truenas[~]# midclt call pool.dataset.recommended_zvol_blocksize tank
16K
root@truenas[~]# zpool status tank
pool: tank
state: ONLINE
scan: scrub repaired 0B in 00:02:38 with 0 errors on Sun Jun 6 00:02:38 2021
config:
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
gptid/77101c31-a85e-11eb-a65a-3cecef0d68cc ONLINE 0 0 0
gptid/77548b0a-a85e-11eb-a65a-3cecef0d68cc ONLINE 0 0 0
gptid/778d1181-a85e-11eb-a65a-3cecef0d68cc ONLINE 0 0 0
gptid/78b23826-a85e-11eb-a65a-3cecef0d68cc ONLINE 0 0 0
gptid/791328d4-a85e-11eb-a65a-3cecef0d68cc ONLINE 0 0 0
raidz1-1 ONLINE 0 0 0
gptid/78305f6b-a85e-11eb-a65a-3cecef0d68cc ONLINE 0 0 0
gptid/78e97fd3-a85e-11eb-a65a-3cecef0d68cc ONLINE 0 0 0
gptid/7b709c9f-a85e-11eb-a65a-3cecef0d68cc ONLINE 0 0 0
gptid/7c3085a9-a85e-11eb-a65a-3cecef0d68cc ONLINE 0 0 0
gptid/7ce57e33-a85e-11eb-a65a-3cecef0d68cc ONLINE 0 0 0
raidz1-2 ONLINE 0 0 0
gptid/7b4377e1-a85e-11eb-a65a-3cecef0d68cc ONLINE 0 0 0
gptid/7b92df42-a85e-11eb-a65a-3cecef0d68cc ONLINE 0 0 0
gptid/7c0de90c-a85e-11eb-a65a-3cecef0d68cc ONLINE 0 0 0
gptid/7c8b256c-a85e-11eb-a65a-3cecef0d68cc ONLINE 0 0 0
gptid/7ca4639c-a85e-11eb-a65a-3cecef0d68cc ONLINE 0 0 0
spares
gptid/7d2e704d-a85e-11eb-a65a-3cecef0d68cc AVAIL
errors: No known data errors
root@truenas[~]#

The result is that adding 2 TiB zvols will report space used as 2.47 TiB. The same can be reproduce with 2 GiB zvols they will be reported as larger. Additionally non optimal volblocksize may have impacts on performance so it would be good to recommend an optimal volblocksize if we can strategically do this in a reasonable way.

This explanation came from Alexander:

" I don't remember exactly that math in ZFS, but it is clear that 5-wide RAIDZ1 can't efficiently store 16K ZVOL blocks. Theoretically 16KB is 4 block, that after adding parity should use exactly 5 disks, but RAIDZ also padds all allocations to be multiple of parity disks plus 1, so for RAIDZ1 it will be rounding up to next 2, so to the useful 5 4KB blocks it will add another 4KB of padding. For 128KB ZVOL blocks that padding effect should be much lower (if any), plus compression should be much more efficient and much less suffer from padding."

Upon his investigation he see's it does seem to work if a 6-wide raidz it should start proposing a 32KB volblocksize after his review of the code.

" Hmm. As I see now, the implemented code indeed proposes the 16KB volblocksize for 5-wide raidz1. As I see, starting from 6-wide it should already start proposing 32KB. So it is not as much of bug as an attempt to be safe when changing things. Though we may tune the math to be more aggressive for RAIDZ and even more aggressive for DRAID once we support it."

My understanding is that we may need to tune the math for 12.0-u5 or future release if we can. Until then perhaps we should reference this ticket in known issues for release notes so that users with raidz layouts know they should review volblocksize to make sure it is optimal for their pool layout? After this is sorted, and we know what the criteria is to test the feature I would like to be pinged to make sure we have proper tests implemented to cover this feature.