Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Cannot import ZFS pool after shutdown - I/O error despite ONLINE status

Description

This issue already happened a month ago on another pool (hdd_mirror) under the exact same conditions, which resulted in total data loss.

System Info

  • ZFS version: zfs-2.2.99-1, zfs-kmod-2.2.99-1

  • Affected pool: ssd_mirror (2-way SSD mirror)

Diagnostic & Troubleshooting Steps

  • zpool import detects the pool as ONLINE, with both devices ONLINE.

  • Attempted import with -f, -F, -FX, -o readonly=on, -o cachefile=none, under multiple combinations → all fail neverending

  • Disks are fully visible via /dev/disk/by-id/

  • Pool appears under zpool import -d /dev/disk/by-id, status is ONLINE

  • No meaningful output in dmesg or /var/log related to ZFS or disk errors

Example Output of zpool import:

pool: ssd_mirror id: 11584508092520712974 state: ONLINE status: Some supported features are not enabled on the pool. action: The pool can be imported using its name or numeric identifier. config: ssd_mirror ONLINE mirror-0 ONLINE wwn-0x5f8db4c042401374-part1 ONLINE wwn-0x5f8db4c042401407-part1 ONLINE

Problem/Justification

After a clean shutdown using the command `/sbin/shutdown -h now`, my ZFS pool `ssd_mirror` became completely unimportable upon reboot.

Impact

None

Attachments

2
  • 10 Apr 2025, 07:13 AM
  • 10 Apr 2025, 07:13 AM

Activity

Show:

Alexander Motin13 hours ago

I don’t know yet how can it be, but could you at least attach debug after you managed to import the pool?

Julien4 days ago

Yes, after a full scrub and SMART test, there are no errors reported. However, if I reboot the system, the pool is automatically exported and shows as offline. And become unimportable again

Alexander Motinlast week

After that, I reimported the pool and performed a long test scan.

Do you mean you run full pool scrub and it went clean without any errors, or you are talking about something else?

Julienlast week

Thank you for your help!

I managed to reimport the pool by deactivating the metadata verification using the following commands:

sudo sh -c 'echo 0 > /sys/module/zfs/parameters/spa_load_verify_metadata' sudo sh -c 'echo 0 > /sys/module/zfs/parameters/spa_load_verify_data'

After that, I reimported the pool and performed a long test scan.

I then reactivated the metadata verification using these commands:

sudo sh -c 'echo 1 > /sys/module/zfs/parameters/spa_load_verify_metadata' sudo sh -c 'echo 1 > /sys/module/zfs/parameters/spa_load_verify_data'

However, upon rebooting, the pool is exported, and I cannot reimport it without repeating the previous steps. I encounter the same error and receive the alert "Pool ssd_mirror state is OFFLINE: None."

Alexander Motinlast week
Edited

During pool import ZFS scrubs last few TXGs of written data and metadata to make sure the pool is sane and will not freeze immediately or cause even bigger corruption. And it fails the import if detects metadata corruption:

1744002451 ffff960646d63280 spa_misc.c:2381:spa_import_progress_set_notes_impl(): 'ssd_mirror' Verifying pool data 1744002451 ffff960646d63280 spa_misc.c:429:spa_load_note(): spa_load(ssd_mirror, config trusted): spa_load_verify found 11 metadata errors and 0 data errors 1744002451 ffff960646d63280 spa_misc.c:415:spa_load_failed(): spa_load(ssd_mirror, config trusted): FAILED: spa_load_verify failed [error=5]

You may skip the check if you set spa_load_verify_metadata and spa_load_verify_data module parameters to zero, but you might just see it hanging or failing a moment later. Your chances to evacuate the data may be higher if you import the pool read-only.

Considering this is a desktop-grade system and that as you say it is not the first time, I’d recommend you to run good long memory test and do other maintenance, considering this system is at least 13 years old.

Details

Assignee

Reporter

Labels

Components

Fix versions

Affects versions

Priority

More fields

Katalon Platform

Created last week
Updated 13 hours ago