Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Thanks for using the TrueNAS Community Edition issue tracker! TrueNAS Enterprise users receive direct support for their reports from our support portal.

Apps fail to migrate when upgrading to Electric Eel

Description

Steps: I'm running 24.04.2.5. Three apps are installed & working correctly: syncthing, emby, and frigate. All three have been updated to the most recent version available. Next, I upgrade to 24.10.0.2.

Expected Result: After the upgrade, all three apps should have been migrated and are up and running.

Actual Result: Only one of my apps (frigate) migrated successfully. The other two (syncthing, emby) do not appear in the list of Applications after the upgrade. When looking at /var/log/app_migrations.log it's clear that there is some kind of issue when interpreting the config files:

% cat app_migrations.log
[2024/12/15 21:04:02] (DEBUG) app_migrations.migrate():235 - Migration details for 'system-update--2024-12-16_03:00:37' backup on 'isildur' pool
[2024/12/15 21:04:02] (DEBUG) app_migrations.migrate():241 - 'syncthing' app failed to migrate successfully: 'Failed to migrate config: Failed to migrate config: Traceback (most recent call last):\n File "/mnt/.ix-apps/truenas_catalog/trains/stable/syncthing/1.1.1/migrations/migrate_from_kubernetes", line 51, in <module>\n print(yaml.dump(migrate(yaml.safe_load(f.read()))))\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/mnt/.ix-apps/truenas_catalog/trains/stable/syncthing/1.1.1/migrations/migrate_from_kubernetes", line 19, in migrate\n "additional_envs": config["syncthingConfig"].get("additionalEnvs", []),\n ~~~~~~^^^^^^^^^^^^^^^^^^\nKeyError: \'syncthingConfig\'\n'
[2024/12/15 21:04:02] (DEBUG) app_migrations.migrate():241 - 'emby' app failed to migrate successfully: 'Failed to migrate config: Failed to migrate config: Traceback (most recent call last):\n File "/mnt/.ix-apps/truenas_catalog/trains/stable/emby/1.2.5/migrations/migrate_from_kubernetes", line 49, in <module>\n print(yaml.dump(migrate(yaml.safe_load(f.read()))))\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/mnt/.ix-apps/truenas_catalog/trains/stable/emby/1.2.5/migrations/migrate_from_kubernetes", line 18, in migrate\n "additional_envs": config["embyConfig"].get("additionalEnvs", []),\n ~~~~~~^^^^^^^^^^^^^\nKeyError: \'embyConfig\'\n'
[2024/12/15 21:04:02] (DEBUG) app_migrations.migrate():241 - 'frigate' app failed to migrate successfully: 'App with same name is already installed'

Session ID: 3f0904bb-7b91-1ffc-ffaf-42014ddfc8be

Environment

None

Activity

Show:

Muhammad February 9, 2025 at 10:06 PM

you would have to manually trigger the migration again as it has already executed once i.e

midclt call --job k8s_to_docker.list_backups k8s | jq

First inspect which backup you would like to restore and then

midclt call --job midclt call --job k8s_to_docker.migrate poolname '{"backup_name": "name_)here"}'

You can skip the last argument if you want to just use the last/latest k8s backup to be restored.

For clarity the fix was merged to Fangtooth, so only execute it in Fangtooth (we have BETA coming soon for it)

ShannonC February 9, 2025 at 3:19 AM

So, was this fixed somehow? Should I do something?

ShannonC January 12, 2025 at 7:31 PM

After sudoing, I ran this:

k3s kubectl get secrets --all-namespaces | grep "sh.helm.release"

And got this:

ix-emby sh.helm.release.v1.emby.v9 helm.sh/release.v1 1 623d ix-emby sh.helm.release.v1.emby.v10 helm.sh/release.v1 1 576d ix-frigate sh.helm.release.v1.frigate.v1 helm.sh/release.v1 1 426d ix-frigate sh.helm.release.v1.frigate.v2 helm.sh/release.v1 1 388d ix-emby sh.helm.release.v1.emby.v11 helm.sh/release.v1 1 388d ix-frigate sh.helm.release.v1.frigate.v3 helm.sh/release.v1 1 271d ix-syncthing sh.helm.release.v1.syncthing.v11 helm.sh/release.v1 1 364d ix-syncthing sh.helm.release.v1.syncthing.v12 helm.sh/release.v1 1 48d ix-emby sh.helm.release.v1.emby.v13 helm.sh/release.v1 1 48d ix-syncthing sh.helm.release.v1.syncthing.v13 helm.sh/release.v1 1 48d ix-syncthing sh.helm.release.v1.syncthing.v14 helm.sh/release.v1 1 27d ix-emby sh.helm.release.v1.emby.v17 helm.sh/release.v1 1 3d16h ix-frigate sh.helm.release.v1.frigate.v5 helm.sh/release.v1 1 3d16h ix-syncthing sh.helm.release.v1.syncthing.v16 helm.sh/release.v1 1 3d16h

I would have run some helm commands for you too, but I’m not sure how to use it in the context of k3s.

Stavros January 10, 2025 at 4:34 PM
Edited


Can you jump into shell in dragonfish and run this? What’s the output?

kubectl get secrets --all-namespaces | grep "sh.helm.release

ShannonC January 9, 2025 at 3:19 AM
Edited

Ok, weird… here’s what happened:

  1. Deleted the Electric Eel boot environment created by the previous upgrade

  2. Updated Syncthing

  3. Clicked “Edit” on each app, one by one.

    1. made no changes to any of the config

    2. clicked save (or update or whatever)

    3. each one re-deployed even though I didn’t make any changes

  4. Upgraded to Electric Eel again

This time, Syncthing shows up in the list… but Emby still does not.

Contents of /var/log/app_migrations.log:

[2025/01/08 21:06:46] (DEBUG) app_migrations.migrate():250 - Migration details for 'system-update--2025-01-09_03:01:38' backup on 'isildur' pool [2025/01/08 21:06:46] (DEBUG) app_migrations.migrate():253 - 'syncthing' app migrated successfully [2025/01/08 21:06:46] (DEBUG) app_migrations.migrate():256 - 'emby' app failed to migrate successfully: 'Failed to migrate config: Failed to migrate config: Traceback (most recent call last):\n File "/mnt/.ix-apps/truenas_catalog/trains/stable/emby/1.2.11/migrations/migrate_from_kubernetes", line 49, in <module>\n print(yaml.dump(migrate(yaml.safe_load(f.read()))))\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/mnt/.ix-apps/truenas_catalog/trains/stable/emby/1.2.11/migrations/migrate_from_kubernetes", line 18, in migrate\n "additional_envs": config["embyConfig"].get("additionalEnvs", []),\n ~~~~~~^^^^^^^^^^^^^^\nKeyError: \'embyConfig\'\n' [2025/01/08 21:06:46] (DEBUG) app_migrations.migrate():256 - 'frigate' app failed to migrate successfully: 'App with same name is already installed'

Certainly seems that something’s not quite right… Apparently it is capable of working correctly, but something is preventing it…

Done

Details

Assignee

Reporter

Labels

Time remaining

0m

Priority

Katalon Platform

Created December 16, 2024 at 3:24 AM
Updated February 9, 2025 at 10:06 PM
Resolved January 22, 2025 at 2:51 PM

Flag notifications