Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Rclone 1.49.4 memory usage

Description

I believe the latest version on rclone (v1.49.4) in FreeNAS 11.3-RC2/RC1 has a memory leak. It only seems to popup if the process runs for >24 hours or so.

For the past two weeks or so of running 11.3-RC{1,2}, rclone tasks have been killed a few times from running out of swap (messages:Jan 21 19:36:53 mars kernel: pid 44711 (rclone), jid 0, uid 0, was killed: out of swap space). The machine is a very lightly loaded with 64GB of memory and 10GB of swap.

To give you a little bit of background of my setup, I backup to google drive and am limited by Google's 750GB per-day limits. I believe what is happening is after I hit the daily upload limit, rclone continues to try and leaks memory during the retry.

Attached is a screenshot that shows memory usage, the spike in free memory is once the rclone task was killed.

Problem/Justification

None

Impact

None

SmartDraw Connector

Katalon Manual Tests (BETA)

Activity

Show:

Vladimir Vinogradenko February 4, 2020 at 1:51 AM

cloud_sync.py changes are unrelated to memory leak, it's just something else I've noted and fixed while inspecting your logs.

This is the binary extracted from our Nightly builds: https://download.freenas.org/12.0/MASTER/latest/x64/

You can also try an official binary, it should work too.

piers saye February 3, 2020 at 6:05 PM

Thanks, Former user. It looks like you've changed src/middlewared/middlewared/plugins/cloud_sync.py  so unclear how a new rclone binary would make a difference?  

I appreciate this is your corporate Jira but I'm not really happy pulling a random binary onto my system - is the binary you suggest testing one of the official rclone releases

https://github.com/rclone/rclone/releases

that i can at least check hashes for?

If not, no stress - I'll wait until U1.  Thank you for your efforts.

 

Vladimir Vinogradenko February 3, 2020 at 3:56 PM

this will only be fixed in 11.3-U1

Meanwhile you can test the fix by putting attached rclone binary to /usr/local/bin/rclone

Bug Clerk February 3, 2020 at 2:45 PM

piers saye February 1, 2020 at 7:24 PM

This is still happening in 11.3 Release. 

Complete

Details

Assignee

Reporter

Labels

Fix versions

Affects versions

Priority

More fields

Katalon Platform

Created January 22, 2020 at 12:52 AM
Updated July 1, 2022 at 4:46 PM
Resolved February 3, 2020 at 2:45 PM