Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Thanks for using the TrueNAS Community Edition issue tracker! TrueNAS Enterprise users receive direct support for their reports from our support portal.

cant disconnect pool

Description

I had tons of reads by k3s-server on my backup pool, after not finding what could cause it, i tried to disconnect the pool, to get an error msg by k3s, but instead I got this error msg:

[EFAULT] Failed to close encrypted /dev/dm-1 device mapping : device-mapper: remove ioctl on md126 failed: Device or resource busy device-mapper: remove ioctl on md126 failed: Device or resource busy device-mapper: remove ioctl on md126 failed: Device or resource busy device-mapper: remove ioctl on md126 failed: Device or resource busy device-mapper: remove ioctl on md126 failed: Device or resource busy device-mapper: remove ioctl on md126 failed: Device or resource busy device-mapper: remove ioctl on md126 failed: Device or resource busy device-mapper: remove ioctl on md126 failed: Device or resource busy device-mapper: remove ioctl on md126 failed: Device or resource busy device-mapper: remove ioctl on md126 failed: Device or resource busy device-mapper: remove ioctl on md126 failed: Device or resource busy device-mapper: remove ioctl on md126 failed: Device or resource busy device-mapper: remove ioctl on md126 failed: Device or resource busy device-mapper: remove ioctl on md126 failed: Device or resource busy device-mapper: remove ioctl on md126 failed: Device or resource busy device-mapper: remove ioctl on md126 failed: Device or resource busy device-mapper: remove ioctl on md126 failed: Device or resource busy device-mapper: remove ioctl on md126 failed: Device or resource busy device-mapper: remove ioctl on md126 failed: Device or resource busy device-mapper: remove ioctl on md126 failed: Device or resource busy device-mapper: remove ioctl on md126 failed: Device or resource busy device-mapper: remove ioctl on md126 failed: Device or resource busy device-mapper: remove ioctl on md126 failed: Device or resource busy device-mapper: remove ioctl on md126 failed: Device or resource busy device-mapper: remove ioctl on md126 failed: Device or resource busy Device /dev/dm-1 is still in use.
remove_circle_outlineMore info...
Error: Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/middlewared/job.py", line 382, in run
await self.future
File "/usr/lib/python3/dist-packages/middlewared/job.py", line 418, in __run_body
rv = await self.method(*([self] + args))
File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1263, in nf
return await func(*args, **kwargs)
File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1131, in nf
res = await f(*args, **kwargs)
File "/usr/lib/python3/dist-packages/middlewared/plugins/pool.py", line 1633, in export
await self.middleware.call('disk.swaps_remove_disks', disks, {'configure_swap': False})
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1310, in call
return await self._call(
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1267, in _call
return await methodobj(*prepared_call.args)
File "/usr/lib/python3/dist-packages/middlewared/service.py", line 50, in l_fn
return await fn(*args, **kwargs)
File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1263, in nf
return await func(*args, **kwargs)
File "/usr/lib/python3/dist-packages/middlewared/plugins/disk_/swap_remove.py", line 27, in swaps_remove_disks
return await self.swaps_remove_disks_unlocked(disks, options)
File "/usr/lib/python3/dist-packages/middlewared/plugins/disk_/swap_remove.py", line 56, in swaps_remove_disks_unlocked
await self.middleware.call(
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1310, in call
return await self._call(
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1267, in _call
return await methodobj(*prepared_call.args)
File "/usr/lib/python3/dist-packages/middlewared/plugins/disk_/encryption_linux.py", line 11, in remove_encryption
raise CallError(f'Failed to close encrypted {device} device mapping : {cp.stderr}')
middlewared.service_exception.CallError: [EFAULT] Failed to close encrypted /dev/dm-1 device mapping : device-mapper: remove ioctl on md126 failed: Device or resource busy
device-mapper: remove ioctl on md126 failed: Device or resource busy
device-mapper: remove ioctl on md126 failed: Device or resource busy
device-mapper: remove ioctl on md126 failed: Device or resource busy
device-mapper: remove ioctl on md126 failed: Device or resource busy
device-mapper: remove ioctl on md126 failed: Device or resource busy
device-mapper: remove ioctl on md126 failed: Device or resource busy
device-mapper: remove ioctl on md126 failed: Device or resource busy
device-mapper: remove ioctl on md126 failed: Device or resource busy
device-mapper: remove ioctl on md126 failed: Device or resource busy
device-mapper: remove ioctl on md126 failed: Device or resource busy
device-mapper: remove ioctl on md126 failed: Device or resource busy
device-mapper: remove ioctl on md126 failed: Device or resource busy
device-mapper: remove ioctl on md126 failed: Device or resource busy
device-mapper: remove ioctl on md126 failed: Device or resource busy
device-mapper: remove ioctl on md126 failed: Device or resource busy
device-mapper: remove ioctl on md126 failed: Device or resource busy
device-mapper: remove ioctl on md126 failed: Device or resource busy
device-mapper: remove ioctl on md126 failed: Device or resource busy
device-mapper: remove ioctl on md126 failed: Device or resource busy
device-mapper: remove ioctl on md126 failed: Device or resource busy
device-mapper: remove ioctl on md126 failed: Device or resource busy
device-mapper: remove ioctl on md126 failed: Device or resource busy
device-mapper: remove ioctl on md126 failed: Device or resource busy
device-mapper: remove ioctl on md126 failed: Device or resource busy
Device /dev/dm-1 is still in use.

I didn't create anything encrypted on the whole system.

Problem/Justification

None

Impact

None

duplicates

Activity

Show:

Walter Zimmermann December 2, 2021 at 9:38 AM

I have imported and disconnected the pool five times in a row, while the NAS was fully operational and under normal load,
every time without any problems and the result of:

Export/Disconnect Pool

Successfully exported/disconnected 'archive'

Muhammad Rehan December 2, 2021 at 9:13 AM

I see the file descriptors issue was resolved for you @ NAS-113384. Can you please confirm if you're still facing this? If yes, please provide the latest debug. Thanks!

Walter Zimmermann November 28, 2021 at 11:54 AM

okay, lets see how it goes with the next release then.

 

thx slightly smiling face

Muhammad Rehan November 26, 2021 at 1:48 PM
Edited

, digging into your attached logs further, I have found out a lot of errors like:

File "/usr/lib/python3.9/asyncio/selector_events.py", line 164, in _accept_connection File "/usr/lib/python3.9/socket.py", line 293, in accept OSError: [Errno 24] Too many open files

There are decent chances that this might be happening due to an on-going issue with file descriptors that 's handling. I think we can circle-back on this once that issue fixed to make sure that issue does not have influence on this one..

Walter Zimmermann November 25, 2021 at 10:50 PM

as I ve just told at https://jira.ixsystems.com/browse/NAS-113384 that we can do a TeamViewer session,
we can also go for one.

 
I am operating at CET, if you use Signal, we could make it on the fly.

otherwise: now, Friday (tomorrow) and Sunday (if you work on Sundays?), next week Monday

and Tuesday before noon, Wednesday the whole day are good

Duplicate
Pinned fields
Click on the next to a field label to start pinning.

Details

Assignee

Reporter

Labels

Original estimate

Time remaining

30m

Components

Fix versions

Affects versions

Priority

Katalon Platform

Created November 22, 2021 at 3:53 PM
Updated July 6, 2022 at 8:56 PM
Resolved December 2, 2021 at 10:06 AM
Configure