Discussion [Request]: Multi-Team-Drive and rclone Crypt Support for SuperTransfer

Assists Greatly with Development Costs
Status
Not open for further replies.

YipYup

Junior Member
Original poster
Local time
6:55 AM
Apr 28, 2018
8
3
@flicker-rate @Admin9705

I'd like to see support for (1)multiple team drives and (2)rclone crypt added to the SuperTransfer script suite.

(#1) Multi-Team-Drive as-a-Target Support. (Not to be confused with multi-account or Multi-GDSA transfer).
Right now I've got three team drives, one for each type of media I have. I'd be awesome if SuperTransfer could be refactored to add support for multiple targets depending on the source directory. IE, /media/audio could be user-configuration mapped to TeamDrive A; /media/television could be user-configuration mapped to TeamDrive B; /media/movies could be user-configuration mapped to TeamDrive C. Taking the principle further, I anticipate soon moving even more so in this direction, as the drive limitations of 250,000 files per team drive are very limiting. I may go the route of a single Team Drive per alphabetical letter for each type of media. I would love to have a way to define multiple local paths to remote team drives.

For example:

/media/tv/0-9 > TeamDrive-TV-0-9:
/media/tv/A > TeamDrive-TV-A:
/media/tv/B > TeamDrive-TV-B:
/media/tv/C > TeamDrive-TV-C:
/media/tv/D > TeamDrive-TV-D:
...

/media/audio/0-9 > TeamDrive-Audio-0-9:
/media/audio/A > TeamDrive-Audio-A:
/media/audio/B > TeamDrive-Audio-B:
/media/audio/C > TeamDrive-Audio-C:
/media/audio/D > TeamDrive-Audio-D:
...

/media/movies/0-9 > TeamDrive-Movie-0-9
/media/movies/A > TeamDrive-Movie-A
/media/movies/B > TeamDrive-Movie-B
/media/movies/C > TeamDrive-Movie-C
/media/movies/D > TeamDrive-Movie-D
...

(#2) I use rclone crypt for my team drives. I would love to have the multitransfer work with my crypt configurations. It would be ideal to have the super transfer read my crypt configuration from rclone, and be able to assist in utilizing that in conjunction with the multi-GDSA transfers.

Then supertransfer could do it's GDSA account load balancing (based on API and BW usage for the rolling 24 hour period), using all available accounts to get media to the correct user mapped target Team Drive.
 

Deiteq

Administrator
Project Manager
Local time
2:55 PM
Jan 19, 2018
283
111
Yes I have worked on the encryption side of rclone but nothing to do with supertransfer or team drives!:geek:
Never used either...sorry buddy, but feel free to look into it further and push something to our github or make a wiki for it if you figure anything out?;)
You clearly are knowledgeable and we welcome people like yourself to the project!:)
 

Admin9705

Administrator
Project Manager
Local time
8:55 AM
Jan 17, 2018
4,760
1,814
I fully agree. We are still hashing out the unencrypted version; once that's done and all is working right; the encrypted piece will be easier to develop.
 
  • Like
Reactions: Deiteq

bdunn

Junior Member
Local time
7:55 AM
Oct 24, 2018
18
5
This would be a great feature given the limited number of files that one Team Drive can hold, especially when you factor in Audiobooks which can be one file or 500 files.

If anyone knows of a good way to process hundreds of folders containing multiple Audiobook files into a single (or at least fewer) Audiobook file(s) that would be an even better solution. I can do this with Audiobook Binder on MacOS but I’ve not found anything to do the same in Linux. Anyone have any ideas or thoughts about that?
 
M

MrDoob

Guest
400.000 files is max on " one " TDrive !!
but you can move the audio books over the webpanel to GDrive !!

GDrive can hold more then 100.000.000 files or 1PB !!
 

bdunn

Junior Member
Local time
7:55 AM
Oct 24, 2018
18
5
It will be me. :geek:

Actually that may not be the case now as I’m working on a method to use FFMPEG to combine all the Audiobook files into one file each. Just have to work out the meta like chapter marks, etc. Anyone happen to already know how this is done?

They must be continuing to optimize Team Folders as the limit on the number of files was once 250k and now 400k. I know that limit is a serious issue for many companies adopting G-Suite completely as simple archive folders routinely exceed 4-5 million files.
 

Admin9705

Administrator
Project Manager
Local time
8:55 AM
Jan 17, 2018
4,760
1,814
ya it's not an issue right now. it requires an insane amount of coding and this will not happen until then :D closing this (very old thread about ST2) lol :D
 
  • Like
Reactions: MrDoob
Assists Greatly with Development Costs
Status
Not open for further replies.

Similar threads


Maintenance Donations

Recommend NewsGroups

      Up To a 58% Discount!

Trending