PG Blitz v2

Admin9705

Administrator
Project Manager
Jan 17, 2018
4,685
1,989
113
Reactions
1,989 0 0
#1
PG Blitz v2
NOTE:  When updating for a newer version of blitz beta, you must redeploy it for the changes to go into effect!
V 2.7 (Merged into 8.0.15)

Added clean folders execution for /{hd}/pgblitz/upload after upload is done to catch more folders (outside of the cron job)
Added bandwidth controls for PGBlitz

Been requested feature for awhile




V2.6

Fixed the cron for properly clean command. Just run PG and it will work automatically

V2.5

UnionFS only used /mnt/move.  Fix made is for 2nd HD users.  When deploying pgblitz, it will mount /(hdpath)/move so that is a new fix.

 V2.4

Added Clone Clean. Cleans empty folders every hour
Sped up PG Moves sleep time

V2.3

BlitzUI removed; does not work with new version
Remove pglog command

replace with ‘blitz’ command which shows current transfer as shown in picture below


files are now...
Continue reading...
 
Last edited:

thepj

PG Master Sergeant
Tech Lead
Nov 13, 2018
91
57
18
Arizona, USA
Reactions
57 0 0
#2
I can validate this is making the uploads not only go faster, but nzbget +sonarr/radarr seems to be moving faster as well, and not waiting as long for the files. Looks like it is doing more in parallel as well.
 

Admin9705

Administrator
Project Manager
Jan 17, 2018
4,685
1,989
113
Reactions
1,989 0 0
#3
I can validate this is making the uploads not only go faster, but nzbget +sonarr/radarr seems to be moving faster as well, and not waiting as long for the files. Looks like it is doing more in parallel as well.
Good feedback. I'm working it some more. Keep reporting. Goal is to replace the other one soon as possible.
 
Assists Greatly with Development Costs

bubbadk

PG Captain
Tech Lead
Mar 18, 2018
334
93
28
44
Denmark
Reactions
93 0 0
#4
How much will I have to start over with if I switch to v8. All I need is plex and tautulli to work. The rest can I setup again 😊
 

Admin9705

Administrator
Project Manager
Jan 17, 2018
4,685
1,989
113
Reactions
1,989 0 0
#5
so bubbadk, you can backup plex and restore. in update 8.0.11, i enable the unionfs path, so your plex will work. tautulli... that shouldnt' be an issue at all. i just started. i took my sonarr and did mass path change from /unionfs/tv to /mnt/unionfs/tv and that was easy.
 

bubbadk

PG Captain
Tech Lead
Mar 18, 2018
334
93
28
44
Denmark
Reactions
93 0 0
#6
Hehe then it's good I'm in cozy little Denmark :LOL:
I will tell you when I make the change friend :coffee:(y)
 
Assists Greatly with Development Costs

Simpleton

PG Sergeant
Dedicated Sponsor
PG Donor
Sep 25, 2018
41
10
8
42
Denmark
Reactions
10 0 0
#7
observations on pgblitz v2.3 beta:

move of files from /mnt/move to /mnt/pgblitz/upload seems very very slow.

When uploads starts, speed looks very good. Maxing out my line. Looks like it transfers 16 files in parallel, but I only see one rclone process. Smooth.

However, uploads being suspended while moving chunks to /mnt/pgblitz/upload could be a significant performace impact. Especially while moves are so slow (they don't seem to get moved by "mv"?

Good job guys! Keep up the good work!
 

Admin9705

Administrator
Project Manager
Jan 17, 2018
4,685
1,989
113
Reactions
1,989 0 0
#8
observations on pgblitz v2.3 beta:

move of files from /mnt/move to /mnt/pgblitz/upload seems very very slow.

When uploads starts, speed looks very good. Maxing out my line. Looks like it transfers 16 files in parallel, but I only see one rclone process. Smooth.

However, uploads being suspended while moving chunks to /mnt/pgblitz/upload could be a significant performace impact. Especially while moves are so slow (they don't seem to get moved by "mv"?

Good job guys! Keep up the good work!
ah ok, so confused on the last part. thanks for the update :D

ok think I know, so what happens is files are being moved, then uploaded, then moved, then uploaded. this is to ensure there are no lost files and etc, including better logging. so technically, nothing is suspended. it's just the order it works.
 
Last edited:

Simpleton

PG Sergeant
Dedicated Sponsor
PG Donor
Sep 25, 2018
41
10
8
42
Denmark
Reactions
10 0 0
#9
Further observations on pgblitz 2.3 beta:
local move to /mnt/pgblitz/upload was 112GB (I guess that it tries to hit around 100G and not 100G exactly).
Then upload started.
When upload was finished "blitz" command reported:
2019/01/08 13:55:13 INFO :
Transferred: 68.905G / 68.905 GBytes, 100%, 30.683 MBytes/s, ETA 0s
Errors: 0
Checks: 175 / 175, 100%
Transferred: 175 / 175, 100%
Elapsed time: 38m19.6s
Some strange things:
1. /pgblitz/upload still contains a lot of (now empty" directories)
in /mnt/tcrypt/ the same directories exists, but they are also empty on upload side :/ (not good!) looks like they are all empty (unfortunately du -sk * does not seem to work on /mnt/tcrypt)
2. no "new" 100G chunk is moved to upload, so upload stops after the first 100G (which is actually good, as it looks like data integrity is NOT ok)

EDIT: Turns out a new move DID start. It was just so slow and the wait for it to start was so long, that I thought it had stopped.
EDIT: also, it it not strange that the first upload chunk was 112G but the blitz report said 68.905G and then stopped?
 
Last edited:

Admin9705

Administrator
Project Manager
Jan 17, 2018
4,685
1,989
113
Reactions
1,989 0 0
#10
Further observations on pgblitz 2.3 beta:
local move to /mnt/pgblitz/upload was 112GB (I guess that it tries to hit around 100G and not 100G exactly).
Then upload started.
When upload was finished "blitz" command reported:
2019/01/08 13:55:13 INFO :
Transferred: 68.905G / 68.905 GBytes, 100%, 30.683 MBytes/s, ETA 0s
Errors: 0
Checks: 175 / 175, 100%
Transferred: 175 / 175, 100%
Elapsed time: 38m19.6s
Some strange things:
1. /pgblitz/upload still contains a lot of (now empty" directories)
in /mnt/tcrypt/ the same directories exists, but they are also empty on upload side :/ (not good!) looks like they are all empty (unfortunately du -sk * does not seem to work on /mnt/tcrypt)
2. no "new" 100G chunk is moved to upload, so upload stops after the first 100G (which is actually good, as it looks like data integrity is NOT ok)
ya i haven't put a piece for the empty directories, that easy to fix (was put in originally for /mnt/move) i'll patch later. keep typing blitz and you'll notice the size shift. i don't understand why rclone doesnt push it all out, breaks it up.
 

Simpleton

PG Sergeant
Dedicated Sponsor
PG Donor
Sep 25, 2018
41
10
8
42
Denmark
Reactions
10 0 0
#11
what is the safest way to stop pgblitz, hopefully without loosing any further data?
will systemctl stop pgblitz do it? Will I then loose any ongoing transfers?
 

Simpleton

PG Sergeant
Dedicated Sponsor
PG Donor
Sep 25, 2018
41
10
8
42
Denmark
Reactions
10 0 0
#12
pgblitz local move to /mnt/pgblitz/upload this time was 115G.
Then It started upload.
blitz status showed 65.732G to transfer (exactly half of what was moved to upload?
an IMMIDIATE du -sh * on /mnt/pgblitz/upload now shows 65G remaining (where do the other 65G go while they are uploading? They are still ongoing. )
 

Admin9705

Administrator
Project Manager
Jan 17, 2018
4,685
1,989
113
Reactions
1,989 0 0
#13
no nothing is lost. because it just moves it. you can do systemctl stop pgblitz, check /mnt/move for the rest of the data. no commands are place to delete any data
 

Simpleton

PG Sergeant
Dedicated Sponsor
PG Donor
Sep 25, 2018
41
10
8
42
Denmark
Reactions
10 0 0
#14
no nothing is lost. because it just moves it. you can do systemctl stop pgblitz, check /mnt/move for the rest of the data.
Thanx :)

I've put my datamigration on halt, pending pgblitz to get out of beta :) too risky for me.
 

Admin9705

Administrator
Project Manager
Jan 17, 2018
4,685
1,989
113
Reactions
1,989 0 0
#15
@Simpleton try

df -h /mnt

see if your disk size looks right by what you have. the rclone logs are not putting out all the correct information, but yet all my contents are uploaded.
 

Simpleton

PG Sergeant
Dedicated Sponsor
PG Donor
Sep 25, 2018
41
10
8
42
Denmark
Reactions
10 0 0
#15
Looks like it might be down to missing logging as you say.

Can't go by df -h /mnt as my /mnt/move is a mounted 13TB NFS filesystem from the old server.
I stopped pgblitz with systemctl stop bgblitz, but files are still turning up in /mnt/tdrive/ and there is still an rclone process running.
Guess I am struggeling with understanding the workflow.

As I understood it, it was like this:
/mnt/move contains files to be uploaded
100G chunks (approx) are taken from /mnt/move by pgblitz and moved (but how? with rclone?) locally to /mnt/pgblitz/upload
rclone upload from /mnt/pgblitz/upload is then started (but how? because it looks like the files are gone in an instant, but not all of them. Approx 65-69GB chunks.)
then when upload is done, an new cycle is started.
 

Admin9705

Administrator
Project Manager
Jan 17, 2018
4,685
1,989
113
Reactions
1,989 0 0
#16
it attempts to move 100gb into /mnt/pgblitz/upload, then uploads. why the logs kept shifting, don't know, but I'll look into. but if you let it run, it does all show up. I have to dig into what Rclone's output does
 
Last edited:

sparkyowns

PG First Class
Dec 21, 2018
2
0
1
Reactions
0 0 0
#17
with blitz 2.4 this started happening. can't find the files on any of the directories to manually delete them. has cause NZBGet to stop working because of a full drive. blitz 2.4.jpg
 

Admin9705

Administrator
Project Manager
Jan 17, 2018
4,685
1,989
113
Reactions
1,989 0 0
#18
@sparky it takes 60 minutes for empty folders to go away, but i'm going to rewrite the log to exempt the notices, tracking already. thanks for the report.
 

skijbal

PG Sergeant
Oct 4, 2018
40
9
8
Reactions
9 0 0
#19
How often does blitz run? What’s the threshold for uploading? Does it upload immediately after download? I had 139GB waiting to upload but it wouldn’t take off.
 

Admin9705

Administrator
Project Manager
Jan 17, 2018
4,685
1,989
113
Reactions
1,989 0 0
#20
every second when idol and not uploading. no after hit's /mnt/move. which version? if using regular, use the beta edition. fixes those stuck issues. beta v2.6 blitz out.