App Catalog Sync is Too Slow
Description
Problem/Justification
Impact
Activity
ksimm1 January 20, 2022 at 1:11 PMEdited
@Waqar ix-applications is on mirrored SATA SSD pool separate from media. Last time we had a teamviewer session it benchmarked at 500MB/s read, which you said wasn't terrible.
Oh i see, you ran a sync on TRUECHARTS, so thats where it shows "updating catalog repository" and "reading catalog information."
Still over 10minutes for me though:
❯ time cli -c 'app catalog sync TRUECHARTS'
[5%] Updating catalog repository...
[15%] Reading catalog information...[100%] Synced 'TRUECHARTS' catalog...
<null>
user=16.12s system=0.86s cpu=2% total=13:47.18
Waqar January 20, 2022 at 12:43 PM
Can you copy paste the following command and share it's output ?
time cli -c 'app catalog sync TRUECHARTS'
Waqar January 20, 2022 at 12:39 PM
@ksimm1 it depends on where ix-applications is hosted, what read/write times we are getting and how much load your system can take on generally speaking and if there's something else as well going on at the time wrt middleware. I tested on a different VM i had and i am seeing 4 minutes there.
We can schedule a team viewer session to see what is taking so much time for you.
ksimm1 January 20, 2022 at 12:19 PMEdited
@Waqar It takes 10+ minutes to complete every single time I hit refresh button. It hasn't been anywhere close to three minutes for me. That sounds OK to you?
(Also less important, but why does the output for the same command look different for you than me? I dont have any 'Updating catalog repository' step at 5% and 'Reading catalog information' at 15%)
The app catalog process shouldn't take ~13mins, even if there are dozens of apps.
To Reproduce:
1. Add the TrueCharts catalog
2. Click Refresh All or run cli -c 'app catalog sync_all'
To look at the duration:
❯ time cli -c 'app catalog sync_all'
[0%] Syncing OFFICIAL catalog...
[50%] Syncing TRUECHARTS catalog...
[100%] Syncing TRUECHARTS catalog...
user=15.89s system=0.98s cpu=2% total=13:40.30