Rclone with Gdrive on LibreElec | PlexGuide.com

Rclone with Gdrive on LibreElec

  • Stop using Chrome! Download the Brave Browser via >>> [Brave.com]
    It's a forked version of Chrome with native ad-blockers and Google's spyware stripped out! Download for Mac, Windows, Android, and Linux!
Welcome to the PlexGuide.com
Serving the Community since 2016!
Register Now

SirTranceALoT

Governer
Original poster
Staff
Jun 14, 2018
319
147
Last week, someone asked if this was possible. After looking into a couple of things besides LibreElec (xbian), I did manage to get rclone running on LibreElec. Why would one want to do that? Well, in LibreElec (KODI) you can use a plugin called PlexKodiConnect. What it does is let Kodi connect to your plex server and sync your library directily into Kodi instead of having to install the plex for kodi plugin or having to run a version of plex media player on your system. If you have a kodi based live TV system or want to use an IP TV service, that is actually a great asset. Now why the RClone mount? Simple. PlexKodiConnect has the ability to use the plex database but, stream your files from your local folders, as long as they are the same as the ones used in Plex. OK, but using LIbreElec, those paths can not be the same. Well no but there's a way around that. PlexKodiConnect (PKC) can substitute paths. Why would you benefit? Well I read about a lot of people having issues with buffering and speed. Since you'd be streaming from a local mounted gdrive, you'd bypass peering issues caused by your server location. Accessing and buffering of files would go a lot faster. The downside to this is, that you can not use things like transcoding anymore. That being sayd, it takes a huge part off the load off your pelx server. Now, as this little guide is for LibreElec systems, this does work for Kodi on iwndows, Mac or LInux aswell. I am not going to cover how you need to set up PKC. I'll do a different tutorial on that. This is simply for mouning rclone.

Now, LibreElect (LE), isn't a standard distro so you can not install packages to it directly and most of the filesystem is read-only. Lucky RClone doesn't need any dependencies and you can run it straight away. What you do need to consider is that some things are going to be in different paths. Like the path to your rclone mount wouldn't be /mnt/gdrive since there's no /mount available but /var/media. Now ok, that doesn't correspond with the paths I got on the server. Wll no worries. PKC can solve that with substituting your paths, as I mentioned before.

Fiçrst thing you need to do, is make sure the SSH service is enabled in LibreElec. That allows you to login to your system remotely with an SSH client like Putty if you'd be using windows.

1. Login to your LibreElec device. You can find the IP-address to connect to in your KODI under system information. Username and password are standard:
Username: root
Password: libreelec
Onece you logged in, you can change that password to your liking by typing
2. switch your directory just to be sure. You don't want to end up in a read-only filesystem
cd /storage
3. download the latest build of rclone. Make sure you have the right version for your device architecture. Mine is a Raspberry PI 3b 32 bit. So in my case it's
4. Unzip this file
unzip rclone-v1.45-linux-arm.zip
You should end up with the following directory
/storage/rclone-v1.45-linux-arm
Remember this path or note it down

5. Run rclone config to configure your mount.
./storage/rclone-v1.45-linux-arm/rclone config
make sure you name your remote gdrive like you used to do in the plexguide config. The reason for that is that for the service to mount your drive automatically, I used the plexguide one with some little changes.

6. now if you were to test if everything works, just issue the following command.
./storage/rclone-v1.45-linux-arm/rclone mount gdrive: /var/media
Open a second SSH clinet, log in and browse to the /var/media folder to see if it's populated with your gdrive contents.

7. Now lets create the service shall we? run the following command
nano /run/systemd/system/gdrive.service
Paste in the following code
Code:
#!/bin/bash
#
# GitHub:
https://github.com/Admin9705/PlexGuide.com-The-Awesome-Plex-Server
# Author:   FlickerRate
# URL:      https://plexguide.com
#
# PlexGuide Copyright (C) 2018 PlexGuide.com
# Licensed under GNU General Public License v3.0 GPL-3 (in short)
#
#   You may copy, distribute and modify the software as long as you
track
#   changes/dates in source files. Any modifications to our software
#   including (via compiler) GPL-licensed code must also be made
available
#   under the GPL along with build & install instructions.
#
#################################################################################
[Unit]
Description=GDrive Daemon
After=multi-user.target

[Service]
Type=simple
User=0
Group=0
ExecStart=/storage/rclone-v1.45-linux-arm/rclone --allow-non-empty --allow-other mount gdrive: /var/media \
                      --uid=1000 --gid=1000 \
                      --size-only --dir-cache-time=2m \
                      --vfs-read-chunk-size=96M --vfs-cache-max-age 675h \
                      --vfs-read-chunk-size-limit=1G \
                      --buffer-size=48M --syslog --umask 002 --log-level INFO \
                      --config /storage/.config/rclone/rclone.conf
ExecStop=/bin/fusermount -uz /var/media
TimeoutStopSec=20
KillMode=process
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target
press CTRL+o and press ENTER to save the file. Press CTRL+X to quite nano.

8. Now only thing left to do is start the service.
systemctl start gdrive.service
If all went well and you browse to your /var/media folder, you should see your gdrive contents.

That's it. You can now populate your library in LE straight from this mount or install PKC and substitue your paths.
 

tonycfromit

Experienced
Staff
Mar 16, 2018
85
40
Last week, someone asked if this was possible. After looking into a couple of things besides LibreElec (xbian), I did manage to get rclone running on LibreElec. Why would one want to do that? Well, in LibreElec (KODI) you can use a plugin called PlexKodiConnect. What it does is let Kodi connect to your plex server and sync your library directily into Kodi instead of having to install the plex for kodi plugin or having to run a version of plex media player on your system. If you have a kodi based live TV system or want to use an IP TV service, that is actually a great asset. Now why the RClone mount? Simple. PlexKodiConnect has the ability to use the plex database but, stream your files from your local folders, as long as they are the same as the ones used in Plex. OK, but using LIbreElec, those paths can not be the same. Well no but there's a way around that. PlexKodiConnect (PKC) can substitute paths. Why would you benefit? Well I read about a lot of people having issues with buffering and speed. Since you'd be streaming from a local mounted gdrive, you'd bypass peering issues caused by your server location. Accessing and buffering of files would go a lot faster. The downside to this is, that you can not use things like transcoding anymore. That being sayd, it takes a huge part off the load off your pelx server. Now, as this little guide is for LibreElec systems, this does work for Kodi on iwndows, Mac or LInux aswell. I am not going to cover how you need to set up PKC. I'll do a different tutorial on that. This is simply for mouning rclone.

... important steps redacted for reply message length ...

If all went well and you browse to your /var/media folder, you should see your gdrive contents.

That's it. You can now populate your library in LE straight from this mount or install PKC and substitue your paths.
Thank you so much for this!

Fantastic info! This is (almost) what I was looking for myself but with a few changes.

Would it be possible to set it up so that I:

- Create G drive connector but as **RWX** and make it so that files added look like they were added by my Google account owner regardless of who on my local network added the files, like a translater that makes all added files added by "[email protected]" instead of "Localdomain\LocalGuy" so that I don't run into permissions issues.​
- Create T drive connector that has the same settings as the G Drive setup in the above step.​
- Fold up both the G Drive and the T Drive into a UnionFS setup while keeping RWE permissions (good for adding metadata changes, folder changes and adding one or two files here and there, never to go over 1GB per day.​

If I can do this and run it off of my QNAP, I can use my QNAP as a central clearinghouse for media data going in both directions in and out of my local network and then I could use the VPN setup to make the whole thing run in a nice and secure tunnel to my Hetzner server, my levels of paranoia will drop about 90 percent while still allowing my systems non-VPN internet access through split-tunneling for non-media related stuff. It would be GRAVY if I could then cache with my QNAP to cut down on actual files in transit, but that is a topic for a different thread, I think.

That is what I am going for... Thoughts?
 

SirTranceALoT

Governer
Original poster
Staff
Jun 14, 2018
319
147
Well technically rclone behaves like rclone you'd install on your linux machine so tdrive and even encrypted mounts are possible. I would need to look into unionfs though. As it comes to permissions you're fine like you would be with any other rclone install on your server. I have it on multiple ones and reading and writing to them goes fine as for updates to the mounts. I don't own a QNAP NAS so can't try it out but as it seems a lot of people are busy with that. This for example https://forum.rclone.org/t/qnap-nas-rclone-tutorial-work-in-progress/3225
I suspect unionfs type setups are perfectly possible in QNAP so maybe a combo of both systems?
 

tonycfromit

Experienced
Staff
Mar 16, 2018
85
40
Well technically rclone behaves like rclone you'd install on your linux machine so tdrive and even encrypted mounts are possible. I would need to look into unionfs though. As it comes to permissions you're fine like you would be with any other rclone install on your server. I have it on multiple ones and reading and writing to them goes fine as for updates to the mounts. I don't own a QNAP NAS so can't try it out but as it seems a lot of people are busy with that. This for example https://forum.rclone.org/t/qnap-nas-rclone-tutorial-work-in-progress/3225
I suspect unionfs type setups are perfectly possible in QNAP so maybe a combo of both systems?
I agree with you in that I think if I can get the G,T and UnionFS (that one seems the hardest) then the built-in Docker will let me do enough to make a hybrid setup that meets my example. I have mapped rclone drives on it before, but never with unionFS, so hopefully that one part will not be too difficult. That, a mnt and a opt folder and I should be in business to run a docker setup similar enough to share packages and backup systems with PG if I am lucky.
 

SirTranceALoT

Governer
Original poster
Staff
Jun 14, 2018
319
147
The problem with a system like QNAP is that they are in a way limited to the functionality they deliver out of the box, at least when it comes to disk management. The thing that comes close to unionfs on a qnap is the storage pool. Those systems are based on RAID technology. I would also love to see those systems having unionfs out of the box because it is so flexible. This is actually why I retained myself from buying such a system and went for the remote server route and gdrive. Now servers do not need to be remote. If I were to build a storage at home, I would buy a simple computer with enough drive bays, install a linux OS on that and configure a unionfs from the different drives. I would litarly have a NAS that could virtually do anything. Then again, that's just me. You wouldn't need a heavy duty system CPU wise. Memory that's a different thing. You can build that for almost the same price of a decent NAS. Also, second hand prices for QNAP systems are quite high depending on the model you have. Maybe it's worth thinking of doing a switch?
 
  • Like
Reactions: 1 user

FullCycle

Active
Feb 22, 2018
46
13
Hi guys,

Thanks for this. I literally just setup my new Vero4k+ box with OSMC with Pg. Well had to modify the Pg or start scripts to remove calls to install docker, traefik, portainer & watchtower. Plus had to alter the rclone install role so that it targeted the 1.5 arm64 version.

Then Pg just worked to install gdrive & tdrive with phmove & unionfs. I setup plexkodiconnect's to substitute /unionfs/ from my remote Plex docker with /many/unionfs/ on the local system. And let it sync the database overnight.

So far, it is working, sort of, well.

This issue I am having is with initial buffering when selecting something to watch. It seems to take such a long time for some reason.... This is after the box has been on for a while... I think after a fresh boot, it is more responsive, but I need to test more.

After it has done the initial buffering before play starts, playback is then completely solid.

Does anyone have any idea's I might be able to tweak my mount services possibly for a smoother experience on the device directly?

The other option is to have a dedicated rclone cache NFS share on my local LAN, but then I have to use an extra system and hop...

Another option is to use an external HDD, and have the gdrive / tdrive mounts configured as cache mounts on the external drive?
 
  • Like
Reactions: 1 users

SirTranceALoT

Governer
Original poster
Staff
Jun 14, 2018
319
147
Nice to see you got this working too. The buffering issue in kodi here is actually quite bareable. I'll see if I kan make a short video to show you how much exactly. Kodi and buffering have always been an issue and I believe there's a little workaround for it but I have to look it up. Strange thing is, it does work fine with IPTV streams though. I wonder why that is.
 

FullCycle

Active
Feb 22, 2018
46
13
Okay, I think I am just about there now. I first tried example 1 from here: https://kodi.wiki/view/HOW-TO:Modify_the_video_cache

This worked great, a little pause at the start, but not too much. The I looked at the progress bar while the movie was playing. In 7 mins the whole movie had passed through the rclone VFS transfer process & was all cached in Kodi to the local emmc.

However, I believe emmc is a type of flash memory, and so this method is likely to decrease the device's lifespan.

So, currently trying out example 5 from the same link above. This will use the RAM, rather than kill the flash. Kodi will be trying to cache as fast as rclone will let it, rather than not fast enough as before. My device has 2G RAM, so after looking at free RAM space with htop, while the last movie was playing, I have set it to use 1.25 G of that RAM for the play cache.

I have also tweaked my rclone mount settings, so that the max read chunk size is 512M, half a little under half of the buffer size. Initial read chunk size is 128M, so not too low for when the Kodi buffer is full and it starts with the small chunks again. Rclone buffer is set to 256M.

Will see how it goes...
 

SirTranceALoT

Governer
Original poster
Staff
Jun 14, 2018
319
147
Thnx! I know what to mess with too the upcomming week.
 

FullCycle

Active
Feb 22, 2018
46
13
Not working again for me now....

Endless buffer at start of play. Going to try Kodi buffermode 3, which is no buffer, and try and rely on rclone buffer only next.
 

SirTranceALoT

Governer
Original poster
Staff
Jun 14, 2018
319
147
oh bummer. I so hope we get this working. Then we can make a nice little build of it and share that bastard. The quest for the perfect mdeiacenter continues.
 
  • Like
Reactions: 1 user

FullCycle

Active
Feb 22, 2018
46
13
Okay, got it a bit more reliable, not perfect yet though.

I tried the Kodi disk cache again last night by setting memory size to 0. It worked good again, until I got an hour into a film, space ran out and it froze up the system.

I have now done some more tweaks:
Increased rclone mount buffers on both gdrive & tdrive to 512MB.
Reduced Kodi RAM cache size to 100MB (I think I read somewhere that this gets split between audio & video, 50MB for each. Therefore actually loads of read ahead for the audio...)
Set the readfactor to 2, much lower than before. This means it will only try to fill the Kodi cache at 2x the media birate, rather than slam the rclone buffer too much. I could also try this at 1...

A little pause at first still, no where near as much as before, but then good playback all the way through.
 
  • Like
Reactions: 1 user

SirTranceALoT

Governer
Original poster
Staff
Jun 14, 2018
319
147
Alright, testing this tonight. I'll report back.
 
  • Like
Reactions: 1 user

FullCycle

Active
Feb 22, 2018
46
13
Actually, some of the changes I made seemed it improve things, but have still been having trouble with the delay at the start of play. It seems it take too long to inially fill the buffer after opening the file and trying to start to play it.

I think I have found a potential solution to this though...

I found a system setting related the the auto refresh rate switching, but not in the same place.
With expert level settings on, and go to system -> display, there is an option to add a pause/delay before play, but after the system has changed the output refresh rate to match the media.

I have set this to at least 7 seconds, and it seems to open the file, but wait until play. Rclone then starts its buffering, and Plex then can immediately have enough buffer when it goes to play slightly later.
 

SirTranceALoT

Governer
Original poster
Staff
Jun 14, 2018
319
147
The delay at startup could also have something to do with rclone. I noticed when you mount without any flags, it is accessible straight away but using the various flags, it needs to sync the directory structure at least. This is very noticable when I do an rsync for example for testing. Or, an ls -la when something has changed. on the gdrive. If that mount was accessable in the same way as a regular filesystem, this would all go smooth.
 
  • Like
Reactions: 1 user

FullCycle

Active
Feb 22, 2018
46
13
Yep, this is repeatable and working for me!

I have reduced my delay after refresh change to 4 seconds. Any lower and it gets stuck it the impossible buffer black hole.

I have been tweaking other settings too, with VFS read chunk size set as 128M as a minimum value. Currently got the rclone buffer set to 1G, but I think this is overkill as my RAM in htop was only just hitting this high.

I will try to lower the rclone buffer to 256M or 128M & increase the Kodi readfactor value to a maximum, with a healthy size Kodi cache memorysize. See if this let's my reduce the startup delay below 4secs.

Worse cast senarios too, with higher bitrate media, could increase the Kodi cache size some more and maybe increase the start delay so a value is safe.

Have only been trying with HD TV so far. So highbitrate HD movies next. Once stable, hopefully, will have to get some UHD movies to test... With Kodi downconvert output for now.

Gonna try these Kodi cache settings next:

<advancedsettings>
<cache>
<buffermode>1</buffermode>
<memorysize>4289999999</memorysize>
<readfactor>9999999999</readfactor>
</cache>
</advancedsettings>
 
  • Like
Reactions: 1 user

FullCycle

Active
Feb 22, 2018
46
13
Yeah, I did read about some settings in rclone, if you increase the dir-cache time and run a recursive find on the entire structure it can potentially speed things up. That adds about 30mins to my start up time though.

Could be something to try again once we're sure things are stable and good playback start times are repeatable with manual interventions.
 
  • Like
Reactions: 1 user

SirTranceALoT

Governer
Original poster
Staff
Jun 14, 2018
319
147
We need to look at this in two parts actually. The Kodi part and the rclone part. I was thinking. What if we tweak kodi in such a way that it starts playing local media, on the fly. I just tested and that too needs to buffer. So if we got that running, then we know things on the kodi end are good. Then we fiddle with the rclone settings.
 
  • Like
Reactions: 1 user

SirTranceALoT

Governer
Original poster
Staff
Jun 14, 2018
319
147
OK so I actually tested with local media and that fires up in a second. The problem is with rclone.
 
  • Like
Reactions: 1 user

Recommend NewsGroups

      Up To a 58% Discount!

Trending