Truenas 12 Can't Mirror - Error: "can only attach to mirrors and top-level disks"
Description
Problem/Justification
Impact
SmartDraw Connector
Katalon Manual Tests (BETA)
Activity
Alexander Motin December 15, 2020 at 2:58 PM
The only thing to fix here would be more accurate error messages. Those are coming from upstream OpenZFS, so the issue could be reported there. We are very limited on time now, sorry.
Alexander Motin December 14, 2020 at 9:20 PM
I've reproduced this issue on my test system with two different devices. Addition of `-o ashift=9` into the `zpool attach` command allowed the attachment to succeed, but resulted in `block size: 512B configured, 8192B native` message in `zpool status`.
Alexander Motin December 14, 2020 at 9:09 PMEdited
Looking into the code I see that "can only attach to mirrors and top-level disks" message is in fact reported on many other reasons when disk can't be added to the pool (ENOTSUP). In this particular case I bet the problem is in physical sector size of devices, reported as 4KB by the SSDs and not reported by the USB stick and probably defaulted to 512. You can not add into mirror with existing vdev with ashift=9 vdev with ashift=12. It should probably be possible to force ashift=9 for the added vdev, since 512e disks support smaller I/Os, but they will work suboptimal. So instead of the new SSDs addition into the mirror I would recommend you to backup config and reinstall from scratch onto the new devices.
John Moberly December 14, 2020 at 5:02 PM
@clinderwell I added mine when I added my comment. @Bonnie Follweiler mine is available. Just let me know if you need anything else.
clinderwell December 14, 2020 at 2:09 PM
@Bonnie Follweiler - I can try this later. I ended up just making an entire SSD a boot drive and don't have a comparable SSD to run this again on. I can likely do this with the USB Drive I was trying to use before but need a few days.
@John Moberly/@Keynaan - Any chance you have this debug that you can easily pull?
Details here - https://www.truenas.com/community/threads/cant-mirror-error-can-only-attach-to-mirrors-and-top-level-disks.89276/#post-618298
I've been a longtime user of using a good USB drive as my boot device. I'm trying to move off that to an SSD.
I was intending to mirror that boot pool over to the SSD but keep running into issues.
I keep trying to set up a mirror of my freenas-boot but it keeps giving me the error 'can only attach to mirrors and top-level disks'. I've tried via GUI and CLI.
Three users in the forum were able to replicate