Discussion - Dedicated Feeder for Wholesaleinternet Server | PlexGuide.com

Discussion Dedicated Feeder for Wholesaleinternet Server

  • Stop using Chrome! Download the Brave Browser via >>> [Brave.com]
    It's a forked version of Chrome with native ad-blockers and Google's spyware stripped out! Download for Mac, Windows, Android, and Linux!
Welcome to the PlexGuide.com
Serving the Community since 2016!
Register Now

jrwillis177

Citizen+
Original poster
Apr 25, 2019
18
4
I currently use a wholesaleinternet server for 4-5 concurrent players, 1 direct play the rest transcode. All clients are located in SoCal, the plex server in Texas. This works no problem, and for around $30 a month is a deal. I'd been using GCE as a feeder for about a year now, and outside of the every other month redeploy, it's been no hassle. Recently however, I've had two GCE instances terminated for cryptomining, presumably because of frequent downloading.

I believe I'd like to switch to a feeder with less resetup hassle. I've got the full new account and redeploy down to like an hour and a half but this with the other issue is just becoming annoying. This server only needs to be able to run sabnzbd, sonarr, radarr and ombi, that's it. Preferably as fast as possible download speed, and decently fast processing speed so that the drives don't fill up.
Post automatically merged:

Also to preempt, I've tried downloading on the wholesaleinternet server, however it takes too much over head away from Plex.
 
Last edited:

Datamonkeh

Data Hoarding Primate
Project Manager
Donor
Donor
Jan 20, 2018
850
388
$30 tells us nothing about the actual specs of the server, in general you would limit download speed and upload speed on a combined box as well as potentially playing with IO/CPU options. $30 likely doesn’t buy you enough resources not to care, especially if it’s a mechanical drive set-up.
 

jrwillis177

Citizen+
Original poster
Apr 25, 2019
18
4
? I'm not trying to change my current server or its function. Currently it does exactly what I want it to do. What I'm looking for is a secondary server that is able to solely replace GCE. Meaning this second server (hetzner?) needs only enough hardware to cap out download speed/transfer speed, about 500-750gb ssd or pref NVME, and a strong enough cpu JUST to run Sabnzbd. That's all I need from it. It doesn't matter where it's based, so long as it's capable of quickly and reliably rclone transferring to my tdrive.
 

urbanbob

Citizen
Feb 16, 2020
10
2
Do you plan for this server to be in constant use. Uploading and downloading? My secondary server finds maybe a couple of episodes and movies a day. If its a netflix series an entire season but it isn't in constant use. You likely dont need a dedicated for this and could probably get away with a smaller instance.

btw if you were termed for crypto mining then you were probably maxing out your CPU 100% of the time. They don't like that on instances as it is a shared resource enviroment
 
Last edited:

Edrock200

MVP
Staff
Nov 17, 2019
549
195
Imo you dont need ssd/nvme for a feeder. If anything you want more hdd space for processing/seeding overhead. Just set one up for a friend at hetzner. For ~$25 he got a 4770, 2x2tb hdd, 32gb ram. Its running 104 containers (he has a ton of automated categories fed by multiple ombi front ends, requestrrs, *arrs, etc) and ita chugging along fine. With 7200rpm disks in raid0 they can do 300MB/s (3gbs) no issues, so with a 1g ethernet connection maxed out up/down, the drives will not be a bottleneck. If you use sabznzbd, make sure the direct decompress option is enabled that allows each rar to decompress as its downloaded vs once the entire set is dowloaded to minimize massive disk i/o storms.

If space isnt a concern you can probably find a similar spec with ssd without issue.

Hetzner pull gets nearly 100% line rate @ 1g and push to google 70-80% line rate (700-800mbs).
 

jrwillis177

Citizen+
Original poster
Apr 25, 2019
18
4
@Edrock200 that sounds perfect. I would prefer more space and if the raid setup with hhd can max the bandwidth of a 1g line @ hetzner for $25 a month I want to go that route. Could you pm a link on where I can find more info on that specific server config? I for sure do not need that many containers, as I would likely use pgblitz front end to setup the feed server through the gce/hetzner option on the main menu.

Also to answer the question on usage. My server is the primary way I and my FIL and my brother access TV, as well as two close friends. So we add seasons very regularly. I also have a watch list that automatically grabs 4k Hdr remux rips with a second radarr setup, which works great with my rpi4 local plex server pointed @tdrive.
 

Edrock200

MVP
Staff
Nov 17, 2019
549
195
@Edrock200 that sounds perfect. I would prefer more space and if the raid setup with hhd can max the bandwidth of a 1g line @ hetzner for $25 a month I want to go that route. Could you pm a link on where I can find more info on that specific server config? I for sure do not need that many containers, as I would likely use pgblitz front end to setup the feed server through the gce/hetzner option on the main menu.

Also to answer the question on usage. My server is the primary way I and my FIL and my brother access TV, as well as two close friends. So we add seasons very regularly. I also have a watch list that automatically grabs 4k Hdr remux rips with a second radarr setup, which works great with my rpi4 local plex server pointed @tdrive.
Unfortunately hetzner won't let me link direct to it, but go here:
Search for 4770 for one similar to I got. 3770 would be ok too. You can also move the sliders to see all offerings with at least 2hdds, then set the HDD size.
 

jrwillis177

Citizen+
Original poster
Apr 25, 2019
18
4
@Edrock200 Thanks, I ordered a 4770. Could you point me to a proper setup instruction? The wiki is in a... weird state right now and I have 0 experience with Hetzner itself.
 

Edrock200

MVP
Staff
Nov 17, 2019
549
195
Basically just go to the hetzner robot control panel and select rescue tab, then select to boot to rescue. Make note of the root password then go to reboot tab and reboot host. Ssh into host and type installimage. Chose Ubuntu 18.04 lts/minimal. At the config screen set raid to enabled and set raid level to 0, you can set your hostname here too if desired. F10 to save and deploy the image.

Once it comes up ssh in with the root account,
adduser username
(Where username is your desired username)
Set pw
Enter through the rest
usermod -aG sudo username
usermod -aG video username
Logout, ssh in with your newly created username
sudo apt update -y && sudo apt dist-upgrade -y
sudo apt install wget && sudo apt install git
sudo reboot
ssh back in with username
Code:
sudo apt install wget -y && sudo rm -rf /opt/pgstage && sudo wget -qO- https://raw.githubusercontent.com/PGBlitz/Install/v8.5/install.sh | sudo -H sh
then
sudo pg
When asked for install version put in 8.7.5
When asked for node name name whatever you like
Once PG menu pops up just follow the steps in order starting with 1. Steps 2 and 3 are optional but recommended. Step 5 is where you can install your apps. In the options menu I recommend toggling the docker swampiness option. If you will use as a media server you can also do the hetzner igpu enable. Chime in here if you get stuck/need help.
 
  • Like
Reactions: 1 user

jrwillis177

Citizen+
Original poster
Apr 25, 2019
18
4
@Edrock200 Alright, everything with the Hetzner server is setup. I setup a Raid 0. I updated and installed pgblitz, connected my tdrive and installed my apps, then restored the backups from my previous GCE instance (I've done this probably 10 times in the past year). Everything works out perfectly. Went to cloudflare and updated my IP to the Hetzner Box.
Then I went to setup traefik. It's [email protected] the portainer validation check. I can manually access the domain through my browser but for whatever reason, it is just freezing @ the validation check. It's sat here for 15 minutes. I've tried closing pg and retrying, reinputing my cloudflare info, and restarting the Hetzner server. Same issue. I'm kind of at a loss, because clearly it's working as I can access Portainer, but it wont pass the validation checks, meaning all my apps aren't being rebuilt with the domain name. (meaning they're all Radarr.NOT-SET.com or whatever the default is). I'mnot sure how to fix this.



UPDATE
Weird, I always setup pgblitz through root login. I fixed this by just adding sudo pg to launch pgblitz.
Post automatically merged:

I'm only seeing about 80MB/s. I was expecting closer to 100. Is there some tuning that I should do to get higher download speeds from my usenet server?
 
Last edited:

Edrock200

MVP
Staff
Nov 17, 2019
549
195
@Edrock200 Alright, everything with the Hetzner server is setup. I setup a Raid 0. I updated and installed pgblitz, connected my tdrive and installed my apps, then restored the backups from my previous GCE instance (I've done this probably 10 times in the past year). Everything works out perfectly. Went to cloudflare and updated my IP to the Hetzner Box.
Then I went to setup traefik. It's [email protected] the portainer validation check. I can manually access the domain through my browser but for whatever reason, it is just freezing @ the validation check. It's sat here for 15 minutes. I've tried closing pg and retrying, reinputing my cloudflare info, and restarting the Hetzner server. Same issue. I'm kind of at a loss, because clearly it's working as I can access Portainer, but it wont pass the validation checks, meaning all my apps aren't being rebuilt with the domain name. (meaning they're all Radarr.NOT-SET.com or whatever the default is). I'mnot sure how to fix this.



UPDATE
Weird, I always setup pgblitz through root login. I fixed this by just adding sudo pg to launch pgblitz.
Post automatically merged:

I'm only seeing about 80MB/s. I was expecting closer to 100. Is there some tuning that I should do to get higher download speeds from my usenet server?
Glad you got past validation check issue.

80MB/s is pretty good. That's 800mbs, you'll rarely ever get 100% line rate out of a gig circuit because there are so many factors, protocol overhead, peering, etc.

You can play with a few options for tuning - make sure you have set your usenet server to the european one, and play with the # of connections. Too many or too few can negatively impact speeds. Try 5 connection, then 10, then 20, etc. until you find the sweet spot. This is of course dependent on your usenet provider being able to deliver that rate and assuming they are not throttling/etc.

Also, after playing with sabz for a little while I have found the best combo of settings to avoid disk i/o storms is to ENABLE direct unpack as well as "pause downloads on post processing." This doesn't seem to slow down the queue unless an item needs repaired, in which case, you'll want the queue to pause to prevent a backlog.
 
  • Like
Reactions: 1 user

jrwillis177

Citizen+
Original poster
Apr 25, 2019
18
4
Seems like it's chugging along just fine. Thanks for all the help. $31 a month for 0 headache seems like a no brainer to me. I think now, I need to find a better solution for a local rclone mounted plex server for my 4k rwmuxes, as even hard wired on a 1G connection, my rPi4 is struggling with direct playing them to my Nvidia shield on files >60ish mbps
Post automatically merged:

I'm going to look into seeing about running the rclone mount directly on the Nvidia Shield Pro, and then using one of the 4K kodi forks to see if the Shields hardware is capable of seamless playback. For my 4K I don't need a pretty plex interface if it simply works.
 

Edrock200

MVP
Staff
Nov 17, 2019
549
195
Seems like it's chugging along just fine. Thanks for all the help. $31 a month for 0 headache seems like a no brainer to me. I think now, I need to find a better solution for a local rclone mounted plex server for my 4k rwmuxes, as even hard wired on a 1G connection, my rPi4 is struggling with direct playing them to my Nvidia shield on files >60ish mbps
Post automatically merged:

I'm going to look into seeing about running the rclone mount directly on the Nvidia Shield Pro, and then using one of the 4K kodi forks to see if the Shields hardware is capable of seamless playback. For my 4K I don't need a pretty plex interface if it simply works.
The shield runs plex server. So if you get an rclone mount working, that might work.

If you have a desktop you leave on, you can try rclone for windows. I just added hyper-v to my win10 pro desktop and did a small pg vm on it. Since I can directstream throughtout the house, it doesn't need to be a workhorse, just a relay essentially.
 

Datamonkeh

Data Hoarding Primate
Project Manager
Donor
Donor
Jan 20, 2018
850
388
It's very rare a Pi makes sense as a Plex server, even though the 4 is a big step up (no shared network/USB bus finally), they really aren't suited to the task, especially not for 4K. Depending on where int he US you are, you *may* be able to run Plex on the Hetzner box and stream 4K, but no guarantees as a REMUX can top out at 100MB/s and that's on the edge of what any gigabit server will push out, let alone one half way round the world. Other than that I know rclone for Android is a thing, but i've never needed to use it.
 

jrwillis177

Citizen+
Original poster
Apr 25, 2019
18
4
I mean the rpi4 is 100% capable of probably 80% of remuxes. It's just the massive 80 mbps bitrate files are too much for it to handle bandwidth wise, but honestly still watchable, just buffers every 3-4 minutes. I know that the Shield can function as a plex server, however I do not believe you can actually use it as a server and also watch directly on it at the same time.
I do NOT like the idea of using an always on Windows PC. Also, I very seriously doubt the Hetzner box will have a stronger/faster connection that my WholeSaleInternet server, and that one struggles @ 35-40mbps bitrate files.
Post automatically merged:

Also, I think your numbers may be mixed up as far as required connection speed for REMUX files. For a standard 45-55 mbps bitrate Remux, a Gigabit connection is MORE THAN enough. I generally go by wanting about twice the bandwidth in mbps for the clients connection. My rpi4 can stream from my tdrive @ 260-280mbps, just shy of my cap of 300mbps internet speed. I have long suspected that my largest source of issues is not the rpi4, but my internet connection strength, as I notice the issues more when you would suspect congestion on the network. My isp recently contacted me letting me know they are now offering up to 500mbps. I think I may try that too.
 

Edrock200

MVP
Staff
Nov 17, 2019
549
195
I mean the rpi4 is 100% capable of probably 80% of remuxes. It's just the massive 80 mbps bitrate files are too much for it to handle bandwidth wise, but honestly still watchable, just buffers every 3-4 minutes. I know that the Shield can function as a plex server, however I do not believe you can actually use it as a server and also watch directly on it at the same time.
I do NOT like the idea of using an always on Windows PC. Also, I very seriously doubt the Hetzner box will have a stronger/faster connection that my WholeSaleInternet server, and that one struggles @ 35-40mbps bitrate files.
I can stream 80mbs from my hetzner box. I get about 600-700mbs downstream from them, but I use cloudflare CDN. If you're not using it, I highly recommend checking it out (even though they took out half the internet today.) ;-)
 

Datamonkeh

Data Hoarding Primate
Project Manager
Donor
Donor
Jan 20, 2018
850
388
I mean the rpi4 is 100% capable of probably 80% of remuxes. It's just the massive 80 mbps bitrate files are too much for it to handle bandwidth wise, but honestly still watchable, just buffers every 3-4 minutes. I know that the Shield can function as a plex server, however I do not believe you can actually use it as a server and also watch directly on it at the same time.
I do NOT like the idea of using an always on Windows PC. Also, I very seriously doubt the Hetzner box will have a stronger/faster connection that my WholeSaleInternet server, and that one struggles @ 35-40mbps bitrate files.
Pretty sure the shield will function as both server and client - it would be useless if it didn't, 'watchable' and 'buffers every 3-4 minutes' are mutually exclusive terms for me, also unless 2-3 of the dev team in the US are mistaken, 4K streaming works from Hetzner, but feel free to test and report back - the US is a big place and ISP routing/peering can make a big difference.
 

jrwillis177

Citizen+
Original poster
Apr 25, 2019
18
4
I'm sorry, I read back my messages and realized I did not state that the content I am talking about is all UHD/HDR remuxes. I'm not sure if that makes a difference. With that said, if it still remains true that people are streaming 4K UHD content from Hetzner (mines in Germany), then I will for SURE check that out.

With that said @ Edrock200 are you saying my local rpi4 could possibly benefit from using cloudflare CDN? If so, that could clear up a bit of congestion/routing issues I maybe experiencing. My main Plex server from wholesale IS using cloudflare cdn through pg. Obviously as rpi4 is ARM based, I cannot use plexguide, so I'll have to figure out a way to do that with a rpi4.
 

Edrock200

MVP
Staff
Nov 17, 2019
549
195
:eek:
I'm sorry, I read back my messages and realized I did not state that the content I am talking about is all UHD/HDR remuxes. I'm not sure if that makes a difference. With that said, if it still remains true that people are streaming 4K UHD content from Hetzner (mines in Germany), then I will for SURE check that out.

With that said @ Edrock200 are you saying my local rpi4 could possibly benefit from using cloudflare CDN? If so, that could clear up a bit of congestion/routing issues I maybe experiencing. My main Plex server from wholesale IS using cloudflare cdn through pg. Obviously as rpi4 is ARM based, I cannot use plexguide, so I'll have to figure out a way to do that with a rpi4.
I thought you meant you were having issues with remux HD from wholesale to your pi4. Not sure about uhd to pi4 as I have little experience with them. Did you install the speedtest container to wholesale? I'll send you some links for comparison via DM.
 

jrwillis177

Citizen+
Original poster
Apr 25, 2019
18
4
Hmm. I think I haven't been clear about my setup. Prior to hetzner, my setup was as follows

Wholesale = Plex media server for 1080p content, no 4K, light transcoding for about 5 people. PGBlitz to TDrive.
GCE = Feeder for the wholesale server through pgblitz. Downloader only with sabnzbd, radarr, sonarr, ombi. Also a second radarr installation for a secondary 4K UHD library.
RPI4 = Local device setup with rclone to the TDrive with exact same config from PGBLITZ(literally copied it and editted where needed to make it work).
Also runs a Local Plex Server only pointed at the 4K HDR content.

The wholesale server and the rpi4 are completely independent from each other. The only thing they share is that they both have an rclone mount to the tdrive.
 

Recommend NewsGroups

      Up To a 58% Discount!

Trending