Storage Solutions Giving up on Jellyfin server, 2nd HDD Failed in 1 year time, what am I doing wrong?

raksrules

raksrules

Elite
I want to explain the events chronologically so it is easier to understand for everyone what's going on and can even help me identifying where I am lacking or what mistakes I am doing.

  1. In June 2021, I bought a 5TB WD Passport External HDD to use with my Raspberry Pi so I can use Jellyfin media server. At that time, the setup was that the HDD was straighaway plugged into the RPi4's USB3.0 Port. Additionally, all the torrent downloads were happening on this HDD directly. Basically, it was barebone setup -> RPi4 + 5TB External HDD.
  2. This setup was working fine and torrents were downloading fine. My HDD was spinning almost 24x7 due to this. But one problem I had was when streaming content locally, my videos would buffer and stutter and only way to fix was to kill torrent client so my streaming and torrents both don't fight for the HDD. Mind you, there was no transcoding happening. It was always direct play.
  3. Then in June 2022, out of the blue, my HDD went bad and I lost all data. It was not getting detected anywhere and I don't remember fully but I think there was that knocking sound. Also, I did not make much attempts to retrieve data. Neither Rpi4 nor windows machine and not even MAC would detect the HDD. I gave up and got it replaced through WDD.
  4. Then in June 2022, I started again from step 1 but this time, I changed few things...
    1. I bought a powered USB hub and plugged HDD into that rather than in RPi4 directly.
    2. I also brought a middleman in the form of a MicroSD card, explained later why.
  5. Now the setup was that torrent downloading was happening to MicroSD card directly and not on HDD. After download, Sonarr/Radarr would transfer the data to HDD in appropriate folder.
  6. This way, the HDD would only spin when transfer of data would happen or when I would stream something. I know that HDD was NOT SPINNING except these times because whenever I started to stream something fresh (as in not streamed for last few hours), the video would take time to load but subsequent episodes would start instantaneously.
  7. I thought I found perfect setup, until yesterday when the HDD stopped getting detected in RPi4 and now I have it plugged into my windows machine and using R-STUDIO app to recover data (thankfully no knocking sound right now). It appears that many of the stuff is recoverable and I can see entire file structure. Windows doesn't show the drive letter (may be because it is EXT4 formatted?) but the recover softwares can see it fine but do tell me there is issue with drive. Raspberry Pi (using dietpi) cannot see the HDD anymore, its drive manager cannot see and neither does "df -h" command.

Screenshot of crystal disk info. Not sure what the yellow indicator is (at bottom) but google seems to say it is hardware issue. Also, when I hover over the CAUTION, the count was 88 yesterday and 99 today but below it shows 63, which was 58 or so yesterday so definitely it is increasing. The power on hours I think is 375 days. So even though that may make it seem like HDD was on for last one year (which it was), I feel it wasn't spinning always.
jellyfin hdd.JPG



As per R-STUDIO too the same is the situation but as you can see, the file structure is preserved and I am right now recovering whatever I can.
rstudio.JPG




Here are my questions...

  1. What am I doing wrong here that my HDD is going bad in one year? Am I taxing the disk too much and these types of HDDs aren't meant for the usage I am using them for? Is heat a factor? I don't have these in enclosed space, in fact it gets good amount of air flow.
  2. For this HDD, is the best course of action to attempt and recover whatever I can and even try to fix if it works else just RMA? Is it even fixable at this point?
  3. What is the Warranty on WD Passport HDD? I bought original one in June 2021 which was RMAed in June 2022 and this one shows warranty till July 2025. So will WDD take warranty based on my original drive's warranty period or this replacement one because I think usual warranty is 3 years?
  4. Is there anything different I can do with the replacement HDD I get from WD after I RMA this one? Is me using a ODD Number HDD causing this? I had read people saying a 5TB may be more susceptible to failure than say a 4 or 6 TB (assuming it exists).
  5. I do not store tv shows if I have watched them already. I am not a data hoarder. For my use case, should I just give up on media server strategy and just download content on need basis and watch, erase and repeat? Because now when I think of it, I don't need the HDD (for streaming) to be On 24x7. I hardly see stuff for 1-2 hours per day from the HDD and that too when it is some daily sort of show that I have downloaded (like masterchef australia). If I do this, should I stop using HDD and instead have say a 128 GB MicroSD card on which I download torrents and completely do away with HDD?
 
I think the external hdd's are not ventilated or cooled properly, that makes them hard for use for 24x7 usage. I use RPi4 but only for plexamp. But for movies and all, a cheap DIY system running the Xpenology can do wonders. Yea, it might take some more amount that a simple hdd on a RPi4 but, it comes with its own advantages. I also use Asustor 4 bay nas but the Xpenology system has more power than any NAS out there. Now for your answers:
1) hdd's might be getting hot due to 24x7 use.
2) Better RMA, also OVM has the plugin that lets you put the hdd's on sleep, you can try that.
3) Warranty is from the initial hdd onwards, any RMA units warranty still applies from original purchase date.
4) not sure
5) Always delete the movies/tv shows that you are sure may not watch them again in an year. Now with good broadband speeds, its better to redownload them instead of keeping them and adding storage...
 
Is your Pi and USB hub connected to a UPS ? If not, do you face frequent power outages ?
I would suggest going for a RMA.
I have an old 1 TB WD passport connected to my router for past one year which runs 24x7. So far no issues.
Also I would suggest moving your downloading off the SD card as they are more prone to failure.
 
ibose said:
Is your Pi and USB hub connected to a UPS ? If not, do you face frequent power outages ?
I would suggest going for a RMA.
I have an old 1 TB WD passport connected to my router for past one year which runs 24x7. So far no issues.
Also I would suggest moving your downloading off the SD card as they are more prone to failure.
Click to expand...

Ironically, the MicroSD has not failed. And that is used for downloading purpose. Also no UPS but then I stay in Mumbai so we hardly ever get power loss. It is like once in a blue moon. So power failure is not an issue I feel.
 
What i see is you are trying to correlate something with something. Because two HDDs went bad, you have started doubting your setup which is normal. Let's think it out logically.

1. The pi is the problem device. Was the pi brand new when you attached the 5TB drive to it or have you attached different drives to the pi if it's not new?

2. It is entirely possible that you got bad hard drives both times. WD HDDs are low end to me atleast (i know there are hundreds of people here who will swear by them being good). I've had more failed drives of WD than Seagate, Maxtor, Toshiba, Verbatim combined.

What's happened is history. If there is even one month left for warranty to expire, the RMA center will take it and give a similar drive. I've had drives running for years torrenting and nothing happened to them and one of those drives dying after just a few months of just leaving them in a box unused. So don't think too much about this. Mechanical drives will fail for no reason (to me) suddenly. I know there must be a reason technically but it's beyond my scope.
 
6pack said:
2. It is entirely possible that you got bad hard drives both times. WD HDDs are low end to me atleast (i know there are hundreds of people here who will swear by them being good). I've had more failed drives of WD than Seagate, Maxtor, Toshiba, Verbatim combined.
Click to expand...
A great insight into hard drive reliability is looking at Backblaze's blog posts and reports. They're one of the few places that uses hard drives on an industrial scale and also releases their data publicly. Although you have to keep in mind that they're generally using drives from the mid and high end ranges. This is my favorite blog post of theirs (not directly related to reliability).
 
Running Jellyfin on a tablet but with the external HDD connected to a router that spins it down on inactivity. The downloads happen on the SSD of the tablet which itself is from 2015 and still running fine while the downloaded file is then transferred to the HDD which has been in use since 2019. There is no enforced spinning of the drive beyond the file transfer and usage, so the issue is with the device the HDD is connected to.

My previous 2TB WD HDD failed with this setup after 4-5 years which is par for the course. The 5 TB one with more densely packed platters is known to fail much earlier.

babhishek said:
just get a server online, spend the amount on an annual plan.
:)
Click to expand...
Can't tell you the number of times the local setup provided entertainment when the net was down. Wouldn't really like to take that online.
 
babhishek said:
just get a server online, spend the amount on an annual plan.
:)
Click to expand...
I understand but local is local. Right now I feel that only option is to just keeping RMAing this until I can and also explore in DietPi to see if I can spin down HDD when not in use. I guess that still counts as "Power On" hours right?
t3chg33k said:
Running Jellyfin on a tablet but with the external HDD connected to a router that spins it down on inactivity. The downloads happen on the SSD of the tablet which itself is from 2015 and still running fine while the downloaded file is then transferred to the HDD which has been in use since 2019. There is no enforced spinning of the drive beyond the file transfer and usage, so the issue is with the device the HDD is connected to.

My previous 2TB WD HDD failed with this setup after 4-5 years which is par for the course. The 5 TB one with more densely packed platters is known to fail much earlier.


Can't tell you the number of times the local setup provided entertainment when the net was down. Wouldn't really like to take that online.
Click to expand...

You need to explain me how. Like is the tablet being used as SERVER? My ASUS router does have USB3 port but I have no idea how to download stuff on it. But I think DLNA/uPnP would work to view content if it had videos.
Do you manually transfer files to HDD over network or some automated way?
 
Last edited:
raksrules said:
Also no UPS but then I stay in Mumbai so we hardly ever get power loss.
Click to expand...
Get a UPS anyway. I have a similar setup while living in Mumbai. I got a cheap Luminous UPS. It runs my router, Pi, and Synology for barely 20 minutes, but that's enough time to shut it down gracefully.

We get around 1 power cut every 3 weeks or so (averaging out). If it's the same in your area, then that's like 16 power cuts a year when your HDD is just knocked out. That's too many.

Keep an eye on the temps, if your HDD is constantly running too hot, you might want to improve the cooling.

Also look up hdparm utility, it allows you to put hard drives to sleep when not in use.

Finally, you could consider getting a smart plug and schedule your system to shut down at night so that it's not running for 6-8 hours a day. Then the smart plug switches off. When it switches back on, your Pi will switch on with it.
 
raksrules said:
You need to explain me how. Like is the tablet being used as SERVER? My ASUS router does have USB3 port but I have no idea how to download stuff on it. But I think DLNA/uPnP would work to view content if it had videos.
Do you manually transfer files to HDD over network or some automated way?
Click to expand...
I assumed they had a Windows tablet, but from some quick googling it seems you can run Jellyfin in Termux (linux environment for Android).
If your router has a USB3 port, it probably has some facility in the UI to set up a file share. Although I would recommend installing OpenWRT on your router and using that, as it's generally more reliable and more capable than stock firmware.
If you use Sonarr or Radarr to automate the downloading of media, they can also handle moving downloaded files to the correct location. I don't think you can run those on a tablet though.
You can definitely run them on your Raspberry Pi though.
Edit: Of course, you can also set up shell scripts to move files automatically when the downloads are complete. Transmission supports it, I would imagine most torrent clients would. That said, IIRC you can also configure separate incomplete-complete folders in Transmission and it will copy the files for you.
 
The external drives are not built for 24x7 operations, If you want to attach a HDD to RPi then get an internal hard drive that is rated for 24x7 operations and a external case, WD Red's for example are rated for 24x7 so are Seagate Exos.
I currently have a 6TB Seagate Exos plugged into a Tiny PC that runs Unraid and multiple dockers including JellyFin, the system is on 24x7 and only consumes around 36W. the Hard drive spin down is handled by Unraid. I have an internal 2TB NVME SSD on the machine that handles the majority of reads/writes, e.g., Camera recordings and downloads etc. and only long-term data is stored or moved to the HDD. So in 24 hours, my hard drive is only spinning if either I am backing or moving stuff to it or I am accessing media from the Jellyfin.
 
gourav said:
Get a UPS anyway. I have a similar setup while living in Mumbai. I got a cheap Luminous UPS. It runs my router, Pi, and Synology for barely 20 minutes, but that's enough time to shut it down gracefully.

We get around 1 power cut every 3 weeks or so (averaging out). If it's the same in your area, then that's like 16 power cuts a year when your HDD is just knocked out. That's too many.

Keep an eye on the temps, if your HDD is constantly running too hot, you might want to improve the cooling.

Also look up hdparm utility, it allows you to put hard drives to sleep when not in use.

Finally, you could consider getting a smart plug and schedule your system to shut down at night so that it's not running for 6-8 hours a day. Then the smart plug switches off. When it switches back on, your Pi will switch on with it.
Click to expand...

For me, the power cut is like once a year.
I have definitely considered the smartplug and scheduling the Pi to shut off at night.I guess if the Pi is shut but the HDD connected to USB Hub is not, still HDD would have spun down right?
 
raksrules said:
For me, the power cut is like once a year.
I have definitely considered the smartplug and scheduling the Pi to shut off at night.I guess if the Pi is shut but the HDD connected to USB Hub is not, still HDD would have spun down right?
Click to expand...
Yup, the HDD will spin down if the host is off.
 
t3chg33k said:
Running Jellyfin on a tablet but with the external HDD connected to a router that spins it down on inactivity. The downloads happen on the SSD of the tablet which itself is from 2015 and still running fine while the downloaded file is then transferred to the HDD which has been in use since 2019. There is no enforced spinning of the drive beyond the file transfer and usage, so the issue is with the device the HDD is connected to.

My previous 2TB WD HDD failed with this setup after 4-5 years which is par for the course. The 5 TB one with more densely packed platters is known to fail much earlier.


Can't tell you the number of times the local setup provided entertainment when the net was down. Wouldn't really like to take that online.
Click to expand...

you can always download 4 hrs of entertainment.
else get a better connection amigo :p
 
raksrules said:
ASUS router does have USB3 port
Click to expand...
What is the model ? You may want to check if it supports Merlin firmware. If yes, you can download and install a torrent package like transmission or aria2c. You will need to do some research.
The only bit you would need to watch is if your wifi gets impacted as sometimes usb3 usage interferes with it.
 
ibose said:
What is the model ? You may want to check if it supports Merlin firmware. If yes, you can download and install a torrent package like transmission or aria2c. You will need to do some research.
The only bit you would need to watch is if your wifi gets impacted as sometimes usb3 usage interferes with it.
Click to expand...
I will be honest, I won't do it. I am just too scared. Mine is ASUS RT68 or something which is Tmobile branded so had a slightly tweaked firmware (appearance wise it is same). This is my primary router and I don't want to play with it.
rudiv said:
Yup, the HDD will spin down if the host is off.
Click to expand...
Now need to find a plug which is not expensive. Any good brands out there with associated app where I can turn on and off at set times?
 
Back
Top