Tdrive in unionfs not work correctly | PlexGuide.com

Tdrive in unionfs not work correctly

  • Stop using Chrome! Download the Brave Browser via >>> [Brave.com]
    It's a forked version of Chrome with native ad-blockers and Google's spyware stripped out! Download for Mac, Windows, Android, and Linux!
Welcome to the PlexGuide.com
Serving the Community since 2016!
Register Now

PlexPower

Experienced
Original poster
Mar 18, 2019
52
15
Hi all.

First: Each time i go in PG Clone. He re-install everytime rclone. I never do a update manually.

Second: My Tdrive in unionfs do not update correctly. See the picture:

In Sonarr:
1605517451073.png

in Ftp /mnt/unionfs/my tdrive:
1605517538996.png

In Drive File Stream:
1605517719204.png

When i do: sudo systemctl restart tdrive. Everything come back but for shot time (6-12 hrs). Is happen when he upload a full season. He upload correctly but after that...
I have 2 others Tdrive mounted whit this command and work good:

The only big change i do is install radarr4k and sonarr4k.

Thx for your help.
 

Attachments

Last edited:

Edrock200

MVP
Staff
Nov 17, 2019
768
275
It seems like maybe your cache isn't updating but not sure. What is your rclone env variables in pg set to? Also upgrade to latest rclone:
Code:
curl https://rclone.org/install.sh | sudo bash
 

PlexPower

Experienced
Original poster
Mar 18, 2019
52
15
It seems like maybe your cache isn't updating but not sure. What is your rclone env variables in pg set to? Also upgrade to latest rclone:
Code:
curl https://rclone.org/install.sh | sudo bash
Thank for help us (my friend have same probleme).

I never change the settings.
⏫ Upload Settings Default Current
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[1] BW Limit [1000M]
[2] Drive-Chunk-Size 64M [128M]
[3] Transfers 8 [8]
[4] Max-Transfer 750G [750G]

⏬ Download Settings Default Current
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[5] Buffer-Size 16M [16M]
[6] VFS-Read-Chunk-Size 64M [64M]
[7] VFS-Read-Chunk-Size-Limit 1024M [1024M]

🔄 VFS Cache Mode Settings Default Current
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[8] VFS-Cache-Mode writes [writes]
[9] VFS-Cache-Max-Age 1h [1h]
[10] VFS-Cache-Max-Size 100G [off]

🔣 Misc Settings Default Current
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[11] Dir-Cache-Time 5m [5m]
[12] Log-Level NOTICE [NOTICE]
[13] User Agent [rclone/v1.48]

Do i really need update rclone? Never have probleme before. No Api ban. The only change is (and the problem come in same time):
Install Sonarr4k Radarr4k
Download/upload 4tb each day.

But if you think that can help? I can try.

Thx again.
 

PlexPower

Experienced
Original poster
Mar 18, 2019
52
15
4tb upload? How many sa's do you have?
Last week. 10. And I noticed this problem. i changed to 6 because i always worked with 6 and always worked. But the problem is still there.

Yesterday. I deleted the plex autoscan and the .service. For see if something changes.

Any idea? Thx again.
 

Edrock200

MVP
Staff
Nov 17, 2019
768
275
Last week. 10. And I noticed this problem. i changed to 6 because i always worked with 6 and always worked. But the problem is still there.

Yesterday. I deleted the plex autoscan and the .service. For see if something changes.

Any idea? Thx again.
That's very odd. Are u positive all 10 service accounts have manager level rights on the drive itself, and not at the folder level (new feature by google). I've heard several people report uploading issues in the last week. So it's possible that google is throttling uploads. You also have your upload speed (option 1) set to unlimited. You may want to try throttling the upload speed a bit, maybe 50MBs (500mbs) or half a 1gbs' line rate. Fully pegged I've found a 1gbs connection can do approximately 8tb/day, realistically 6 from my testing, which means you are uploading about 2/3 of the day and near full line rate.

One thing you may want to try is creating a new project on Google and generating keys/sa's from that. Sometimes google will flag projects/SA's that are generating too much activity.

What I can't figure out is why your unionfs file list is showing a different list from your file stream client. That seems to indicate your mount is not refreshing properly, which is usually indicative of api limits or rclone settings. Since it worked fine before, rclone settings doesn't seem likely.
 

luizfeliperc

Citizen
Nov 15, 2018
8
0
I am facing similar problem, regardless of the FULL or Writes cache mode. I use a folder inside the TD where I make subtitle changes using Raidrive / DFS or Browser. When I rename a caption to pt.srt and I will check it in the assembly, it keeps both, the old one and the one that was renamed. It is not updating the folder, and it only works if I leave the service.

1605636358995.png
1605636370834.png

If I delete the caption ".srt" it will delete both.

I was experiencing problems with rclone version 1.53.1 (which now appears to be the default when installing PG) and also with 1.53.2

Any idea?
 

PlexPower

Experienced
Original poster
Mar 18, 2019
52
15
That's very odd. Are u positive all 10 service accounts have manager level rights on the drive itself, and not at the folder level (new feature by google). I've heard several people report uploading issues in the last week. So it's possible that google is throttling uploads. You also have your upload speed (option 1) set to unlimited. You may want to try throttling the upload speed a bit, maybe 50MBs (500mbs) or half a 1gbs' line rate. Fully pegged I've found a 1gbs connection can do approximately 8tb/day, realistically 6 from my testing, which means you are uploading about 2/3 of the day and near full line rate.

One thing you may want to try is creating a new project on Google and generating keys/sa's from that. Sometimes google will flag projects/SA's that are generating too much activity.

What I can't figure out is why your unionfs file list is showing a different list from your file stream client. That seems to indicate your mount is not refreshing properly, which is usually indicative of api limits or rclone settings. Since it worked fine before, rclone settings doesn't seem likely.
Thx for reply

1605635897585.png

Right click on Shared Drive -> Manage Members. Before "Contributor" and i change for "Content manager"
1605636481936.png

My sister call me last night and tell me the video is on buffering and she do a Speedtest via Pg Speedtest:
1605636695885.png

I change:
[1] BW Limit [75M]
[2] Drive-Chunk-Size 64M [128M]
[3] Transfers 8 [4]
[4] Max-Transfer 750G [750G]

And now:
1605636763862.png

About Api limits::ROFLMAO: Im big fan of Skip Intro and Chapter Thumbnails. I KNOW! API LIMITS DUDE!!! But. For my defense. Before Sonarr4k/Radarr4k. Never hit the limits whit:
Skip Intro and Chapter Thumbnails-> as a scheduled task and when media is added
Scheduled Tasks-> 9pm to 4am-> All option set-> ON (Whit: Update all libraries during maintenance)
And i have over 500tb in 3 Shared Drive. 6hrs for scan everything.

But maybe. With 3-4tb download everyday. I thing i hit the limits even though PGUI has no warning.
So yesterday morning. I delete Plex Autoscan (Just for try because is a new thing for us) and set Off Skip Intro and Chapter Thumbnails and Update all libraries during maintenance.
This morning i dont see any problem and i do a: sudo systemctl restart tdrive just after the Plex Backup for a good morning start (2hours after the California reset Api Ban).
If everything run good. Tomorow moning (Last day of my 26tb download whit Sonarr4k (just before 16tb whit Radarr4k)). I will re-install Plex Autoscan and see what happen whit the last 3.5tb 4k video (NO Skip Intro and Chapter Thumbnails).

So everything above that for say. Yes i thing is the Skip Intro and Chapter Thumbnails. Will See in 48hrs and sorry for all this text.

BTW: Thx Edrock200 for your help. Is always good receive help.
 

luizfeliperc

Citizen
Nov 15, 2018
8
0
I don't know what may be happening. I upgraded to rclone version to 1.53.2 and am using full mode. Below is my mounting of the tdrive.

[Service]
Type=notify
EnvironmentFile=/opt/appdata/plexguide/rclone.env
ExecStart=/usr/bin/rclone mount tdrive: /mnt/tdrive \
--config=/opt/appdata/plexguide/rclone.conf \
--log-file=/var/plexguide/logs/rclone-tdrive.log \
--log-level INFO \
--uid=1000 --gid=1000 --umask=002 \
--allow-other \
--dir-cache-time 24h \
--tpslimit 10 \
--user-agent HoltzFlix \
--drive-skip-gdocs \
--poll-interval 60s \
--vfs-cache-mode full \
--cache-dir=/mnt/cache \
--vfs-cache-max-age 720h \
--vfs-cache-max-size 100G \
--vfs-read-chunk-size 16M \
--buffer-size=8M \
--use-mmap \
--vfs-case-insensitive
ExecStop=/bin/fusermount -uz /mnt/tdrive > /dev/null
TimeoutSec=60
User=0
Group=0

1605646763739.png

1605646777564.png
1605646790554.png

I just did a test creating a new file by raidrive in the folder and in the assembly it was recognized but the caption is still there, the "two"

1605646917613.png


Does this have anything to do with the cache?
 

Edrock200

MVP
Staff
Nov 17, 2019
768
275
I am facing similar problem, regardless of the FULL or Writes cache mode. I use a folder inside the TD where I make subtitle changes using Raidrive / DFS or Browser. When I rename a caption to pt.srt and I will check it in the assembly, it keeps both, the old one and the one that was renamed. It is not updating the folder, and it only works if I leave the service.

View attachment 5036
View attachment 5037

If I delete the caption ".srt" it will delete both.

I was experiencing problems with rclone version 1.53.1 (which now appears to be the default when installing PG) and also with 1.53.2

Any idea?
Edit ur service file. E.g.
sudo nano /etc/systemd/system/tdrive.service
Add
--poll-interval=1m --vfs-cache-poll-interval=1m
Save
sudo systemctl daemon-reload
sudo systemctl restart tdrive
 

PlexPower

Experienced
Original poster
Mar 18, 2019
52
15
That's very odd. Are u positive all 10 service accounts have manager level rights on the drive itself, and not at the folder level (new feature by google). I've heard several people report uploading issues in the last week. So it's possible that google is throttling uploads. You also have your upload speed (option 1) set to unlimited. You may want to try throttling the upload speed a bit, maybe 50MBs (500mbs) or half a 1gbs' line rate. Fully pegged I've found a 1gbs connection can do approximately 8tb/day, realistically 6 from my testing, which means you are uploading about 2/3 of the day and near full line rate.

One thing you may want to try is creating a new project on Google and generating keys/sa's from that. Sometimes google will flag projects/SA's that are generating too much activity.

What I can't figure out is why your unionfs file list is showing a different list from your file stream client. That seems to indicate your mount is not refreshing properly, which is usually indicative of api limits or rclone settings. Since it worked fine before, rclone settings doesn't seem likely.
Hi again. After few test. The problem is not fix.

New Project. No daily scan.No intro.No chapter. Just the minimum. Nothing change.

i delete everything on my server and fresh install. Nothing change.

Update rclone this morning. After 1:30hes. Missing video and whit restart tdrive. Everything come back.

Pls help me. 😭
 

luizfeliperc

Citizen
Nov 15, 2018
8
0
 

PlexPower

Experienced
Original poster
Mar 18, 2019
52
15
Thx for reply but if i understand you. I just need install rclone beta? Because i dont use the « Full » thing.

Thx for reply me.
 

luizfeliperc

Citizen
Nov 15, 2018
8
0
[QUOTE = "PlexPower, postagem: 45391, membro: 7875"]
Obrigado pela resposta, mas se eu entendi. Eu só preciso instalar o rclone beta? Porque eu não uso a coisa «Full».

Obrigado por me responder.
[/ CITAR]

For the tests that I did the problem occurs in both FULL and Writes. For now I'm using beta 1.54 and I haven't had any problems yet.
 

PlexPower

Experienced
Original poster
Mar 18, 2019
52
15
[QUOTE = "PlexPower, postagem: 45391, membro: 7875"]
Obrigado pela resposta, mas se eu entendi. Eu só preciso instalar o rclone beta? Porque eu não uso a coisa «Full».

Obrigado por me responder.
[/ CITAR]

For the tests that I did the problem occurs in both FULL and Writes. For now I'm using beta 1.54 and I haven't had any problems yet.
Thx for reply. Im very bad whit linux. Can you give me the exact commands line for update rclone whit beta version? Like a baby. Thx
 

Edrock200

MVP
Staff
Nov 17, 2019
768
275
[QUOTE = "PlexPower, postagem: 45391, membro: 7875"]
Obrigado pela resposta, mas se eu entendi. Eu só preciso instalar o rclone beta? Porque eu não uso a coisa «Full».

Obrigado por me responder.
[/ CITAR]

For the tests that I did the problem occurs in both FULL and Writes. For now I'm using beta 1.54 and I haven't had any problems yet.
I haven't had this issue w 1.48, or 1.53
 

PlexPower

Experienced
Original poster
Mar 18, 2019
52
15
I haven't had this issue w 1.48, or 1.53
Me its only 2-3 weeks that I have this problem. But when you do a new installation. He do not install rclone 1.48 but 1.53.1 and I think is that version have a problem. I did a little test with v1.54.0-beta.4903.211b08f77 but tomorrow I will hurt it to see if it will work better.
 

Recommend NewsGroups

      Up To a 58% Discount!

Trending