Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Thanks for using the TrueNAS Community Edition issue tracker! TrueNAS Enterprise users receive direct support for their reports from our support portal.

Issues

Select view

Select search mode

 
1 of 1

Cloud Sync Task can no-longer work for onedrive due to missing parameters for --checkers and --tpslimit

Complete

Description

syncing to onedrive with rclone has always been hit and miss (microsoft throttling) and fiddling with --checkers, --tpslimit and --transfers has always been required to have a successful sync.

Due to not being able to enter these settings into a cloud sync task, any sync with onedrive (that checks more than a few thousand file) will get throttled and normally not even finish.

My work around is to run rclone from a custom bash script and set the suggested settings for onedrive sync to --checkers=1 and --tpslimits=10

reference: https://forum.rclone.org/t/throttling-of-rclone-with-sharepoint-saga-continues/29874

Problem/Justification

None

Impact

None

Details

Assignee

Reporter

Labels

Impact

High

Original estimate

Time remaining

0m

Components

Fix versions

Affects versions

Priority

Katalon Platform

Created October 7, 2022 at 2:24 PM
Updated February 27, 2025 at 10:10 PM
Resolved October 11, 2022 at 1:40 PM

Activity

Show:

William GryzbowskiOctober 10, 2022 at 8:01 PM

as a workaround you can set these custom args via API (or CLI in SCALE).

Michelle JohnsonOctober 7, 2022 at 7:33 PM

Thank you for your respons, !

This issue ticket is in the queue for review. An Engineering representative will update with further details or questions in the near future.

DENNY VANDEMAELEOctober 7, 2022 at 6:43 PM
Edited

It’s a defect in core in that the required parameters cannot be set in a Cloud Sync Task to meet the requirements of syncing a large quantity of files to/from onedrive, so a debug does not apply here since the GUI doesn’t have a place to enter these parameters and will not show in a debug.

There are a large quantity of threads on rclone forums through the years that suggest settings for getting the sync to work with onedrive… the newer the post, the more stringent the settings need to be to make it work (due to microsoft ever changing and likely dynamic load capicity). All of these threads suggest using variations of the following parameters at minimum: tpslimits, checkers and occasionally transfers.

I’ve tried a few dozen settings over the years, and only the current suggestion will work now: --checkers=1 and --tpslimit=10
These settings cannot be set in the Cloud Sync Task, but testing by using a custom shell script works successfully.

So effectively a user cannot successfully use a Cloud Sync Task if the cloud provider is onedrive and there are a large number of files.

Michelle JohnsonOctober 7, 2022 at 5:26 PM

Thank you for your report, !

First, to help us correctly direct your issue, are you reporting a function in CORE that you believe no longer works properly due to a defect in CORE? Or are you requesting a feature change that addresses this Microsoft issue?

Second, please provide a debug file if you are reporting a bug issue in CORE. All tickets are private to protect personal and private information.

To generate a debug file on TrueNAS CORE, log in to the TrueNAS web interface, go to System > Advanced, then click Save Debug and wait for the file to download to your local system.

DENNY VANDEMAELEOctober 7, 2022 at 2:28 PM
Edited

clarifying why I stated “no-longer work for onedrive”. I had several cloud sync tasks that had few enough files pulling from onedrive that were working up until the last couple of months… when microsoft apparently tweaked their limits again… so impossible to sync without --checkers=1 (and perhaps --tpslimit=10).