The patch is included into upcoming OpenZFS 2.1.6. If it will be possible, it will be included into SCALE 22.12-BETA2, if not – following RC1.

This PR should make ZFS prefer space efficiency to some performance in case like that: .

In product briefs I see that those NVMes are read-optimized, and specs measure performance and endurance in terms of 64KB blocks. So it may be that they indeed prefer writes of that size. If that is true, then ZFS and TrueNAS Core did exactly what they were made to do. With the sysctl above you can override that, if you think it is right, but then I suppose write endurance or performance may suffer, but I have no idea how much. If that will be write-once-read-many storage, then you should be fine, since ZFS writes data sequentially, when possible, but otherwise once your pool get fragmented with time performance may suffer badly.

It was surprising first, but I think I have an answer. I see that kernel reports “stripe size” AKA physical sector size for those NVMes of 64KB. Seeing that ZFS happily increases its ashift value to its maximum of 16 (64KB), that makes RAIDZ2 extremely space inefficient, that is why you probably see so small projected capacity. The question is why the NVMe driver in kernel reports such a big stripe size.

Could you please provide outputs of nvmecontrol identify nvme1, nvmecontrol identify nvme1ns1 , nvmecontrol identify -xv nvme1 and nvmecontrol identify -xv nvme1ns1 commands? I suppose the devices report Preferred Write Granularity or Preferred Write Alignment of that big size.

To workaround the issue you should be able to set sysctl vfs.zfs.max_auto_ashift=12 before the pool creation.

Thank you for your report, !

This issue ticket is now in the queue for review. An Engineering representative will update with further details or questions in the near future.