Storage Solutions Giving up on Jellyfin server, 2nd HDD Failed in 1 year time, what am I doing wrong?

[gourav]

Any good brands out there with associated app where I can turn on and off at set times?

Most smart plugs will support timers. If they support Alexa/Google home, then you can even do the scheduling using Alexa/Google home app on your phone (you don't need an alexa or nest device).

I'm using one from Amazon, which is ridiculously expensive (I got it on a sale), so won't recommend and another from realme, which was ridiculously cheap, but no longer available. Would recommend go with Wipro and use the Smart Life app instead of Wipro's app. Wipro app is very bad and their devices are basically rebranded Smart Life devices, so perfect compatibility with Smart Life.

 

[t3chg33k]

I understand but local is local. Right now I feel that only option is to just keeping RMAing this until I can and also explore in DietPi to see if I can spin down HDD when not in use. I guess that still counts as "Power On" hours right?


You need to explain me how. Like is the tablet being used as SERVER? My ASUS router does have USB3 port but I have no idea how to download stuff on it. But I think DLNA/uPnP would work to view content if it had videos.
Do you manually transfer files to HDD over network or some automated way?

I am still using the AC68U with Merlin which is in its 8th year now and have the HDD connected to the USB 3 port. It has a built-in option to wind down the HDD when idle.

Transferring from the device to the HDD can be done using qBittorrent and its option as to where to move the completed files.

 

[raksrules]

I am still using the AC68U with Merlin which is in its 8th year now and have the HDD connected to the USB 3 port. It has a built-in option to wind down the HDD when idle.

Transferring from the device to the HDD can be done using qBittorrent and its option as to where to move the completed files.

Yes my router too is bought in 2015 (from tmobile in US) and still works so great that I am surprised and has great range too.

 

[rsaeon]

Ironically, the MicroSD has not failed. And that is used for downloading purpose.

SD card corruption was resolved with the Pi4, so it shouldn't be an issue these days.

The external drives are not built for 24x7 operations, If you want to attach a HDD to RPi then get an internal hard drive that is rated for 24x7 operations and a external case, WD Red's for example are rated for 24x7 so are Seagate Exos.

I barely trust the reliability of input devices with USB so it's always a surprise to me that the younger (raspberry pi) generation rely so heavily on USB for data intensive tasks like storage and video capture. USB was never designed to be a stable, consistent connection — It was meant for convenience above everything else.

That's not to say smaller embedded devices shouldn't be trusted, I have a pogoplug from a decade ago that works well as a low power nas with a drive connected to the SATA port. It's also running Syncthing (which is CPU intensive when adding new files) to keep a tertiary backup of various configuration files (templates, shell scripts) that I may need later. Uptime has been fantastic.

PogoPlug Pro - Discussion Thread

^ Ok. Will try that.

techenclave.com

What am I doing wrong here that my HDD is going bad in one year?

At the very least, I'd invest in a good quality third party external enclosure/dock and use that with a laptop or desktop drive. Nearly everyone I know who has used a branded external drive (WD, Seagate, etc) has had failures within two years. I've had good experiences with the 2.5" type-c Orico enclosures for intensive sequential writes/reads with SSDs but not anything long term with spinning discs.

 

[raksrules]

An Update:

So yesterday full day I spent with the drive connected to my windows laptop and using the R-Studio software to try and backup as much as I could and do few other things. Posting below each point....

• Some folders I was able to retrieve without any errors. I selected the option which said "Do not recover files with bad sectors" which meant that if a file was physically where there were bad sectors, the file itself won't be retrieved completely. So some files and folders I was able to recover 100% while in some cases I encountered issues like "Access denied" or "Could not read". Gave up eventually.
• I even attempted to delete the volume through disk management in windows but for some reason it gave "Access denied". No clue why. All other options on right clicking the drive in disk management were greyed out. I was wanting to format the drive but could not.
• Since the drive is EXT4 formatted, windows cannot show the drive in explorer. So I decided to plug the HDD back in my RPi4 so I could try to see if something else I can do. To my surprise, the drive was detected and mounted fine.
• I unmounted the drive and I ran the fsck command and it said drive is CLEAN. This was more surprising. I even played video using jellyfin and it worked fine. Right now, I am streaming content on my tablet over SFTP using nPlayer on my android device and will keep a tab on it.

I still will be approaching this with caution and will expect the drive to die anytime without notice. I will just use it until it dies.

 

[n1r0]

1. Keeping a HDD spinning continuosly is better than having it spool up when data is accessed, and then spool down after a few minutes if idle. The power cycle Start/Stop count seems to increase chances of disk failure from what I've seen. My oldest HDD that I used for OS would never idle timeout and keep spinning, whereas those that are used less frequently seem to develop problems.

2. External HDDs can override their idle timeouts even if you have disabled it in OS.

3. Heat is probably the #1 killer of electronic components. 2.5" HDDs seem to have barely any vents for air circulation.

 

[altair21]

Using external HDDs is your issue, aside from cooling issues, these are SMR drives, to say in roughly and largely incorrectly, SMR drives pack data more efficiently but in return for more storage and cheaper costs, random Read/write and even sustained read/write performance goes down and HDDs arent known for their IO anyways, what you need is a CMR drive which mostly comes in enterprise HDDs, something like a Seagate Exo will work perfectly for you if the number of torrents arent huge (torrents take up a lot of random IO plus ram which pi itself will choke on), otherwise I would recommend to do a raid setup for your storage and a mini pc build if you are a heavy user. because both your pi and the HDD itself arent built for this kind of usage

 

[raksrules]

Using external HDDs is your issue, aside from cooling issues, these are SMR drives, to say in roughly and largely incorrectly, SMR drives pack data more efficiently but in return for more storage and cheaper costs, random Read/write and even sustained read/write performance goes down and HDDs arent known for their IO anyways, what you need is a CMR drive which mostly comes in enterprise HDDs, something like a Seagate Exo will work perfectly for you if the number of torrents arent huge (torrents take up a lot of random IO plus ram which pi itself will choke on), otherwise I would recommend to do a raid setup for your storage and a mini pc build if you are a heavy user. because both your pi and the HDD itself arent built for this kind of usage

I agree with you about the external HDD not being best for the purpose. It is at such times that I evaluate if I really need all this setup or I can do without it. Right now, I feel the only option is to get this RMAed and use replacement drive until I am not longer able to RMA and then do away with my jellyfin server setup completely.
I will check on CMR HDDs and such but I feel it is not worth investing so much as there is only so much as I watch at one time and I can always download on need basis so even a 128 GB MicroSD Card will suffice as I will delete seasons and shows immediately after watching.
There is like 5% of what I download I watch on TV, rest 95% is on my tablet so I can even download directly on tablet's MicroSD card with torrent client.

If I do cost benefit analysis, spending 10K or more on a drive will not be a good decision I feel.

 

[altair21]

I agree with you about the external HDD not being best for the purpose. It is at such times that I evaluate if I really need all this setup or I can do without it. Right now, I feel the only option is to get this RMAed and use replacement drive until I am not longer able to RMA and then do away with my jellyfin server setup completely.
I will check on CMR HDDs and such but I feel it is not worth investing so much as there is only so much as I watch at one time and I can always download on need basis so even a 128 GB MicroSD Card will suffice as I will delete seasons and shows immediately after watching.
There is like 5% of what I download I watch on TV, rest 95% is on my tablet so I can even download directly on tablet's MicroSD card with torrent client.

If I do cost benefit analysis, spending 10K or more on a drive will not be a good decision I feel.

I mean yea, Exos are expensive af specially in India, but yea if you are using arrs for PTs then it makes sense to use it but for public trackers its not worth it, agreed. if you have a PC/laptop at home, it might make sense for you to host jellyfin on it and run the server whenever you wanna watch something, and if jellyfin is your priority and if you are fine paying 400-500Rs a month, get a seedbox and use jellyfin on it

 

[t3chg33k]

Honesty though been running the external HDD setup for a decade now and I am only on my second drive. The issue is simply how things have been setup.

 

[raksrules]

Honesty though been running the external HDD setup for a decade now and I am only on my second drive. The issue is simply how things have been setup.

So what different setup would you recommend for me?

 

[t3chg33k]

So what different setup would you recommend for me?

Probably have a small solid state drive as the workhorse for read/write operations and the HDD for storage and playback.

 

[raksrules]

Probably have a small solid state drive as the workhorse for read/write operations and the HDD for storage and playback.

But isn't that my setup already? Instead of SSD, it is MicroSD card. My torrents (for last one year) are never downloaded on HDD, it is only done on MicroSD card and transferred to HDD automatically.
The HDD is only for storage and playback.

 

[enthusiast29]

1TB internal HDDs were around 3K INR mark IIRC. Is it worth the risk now to buy those instead of the cheap 1TB TLC SSDs which come around 4K INR or less?
At least SSDs are more reliable and shock proof.

 

[altair21]

But isn't that my setup already? Instead of SSD, it is MicroSD card. My torrents (for last one year) are never downloaded on HDD, it is only done on MicroSD card and transferred to HDD automatically.
The HDD is only for storage and playback.

all I can say is, you just lost the silicon lottery, I have seen peeps (including me rocking HDDs for more than a decade) and some who lost theirs with a year. Your setup sounds good since you are not really abusing the HDD but shit happens, also jellyfin transcodes a lot unless you have disabled it, so that might have led to increased read/writes for the hdd

 

[raksrules]

all I can say is, you just lost the silicon lottery, I have seen peeps (including me rocking HDDs for more than a decade) and some who lost theirs with a year. Your setup sounds good since you are not really abusing the HDD but shit happens, also jellyfin transcodes a lot unless you have disabled it, so that might have led to increased read/writes for the hdd

No transcoding. I have disabled transcoding completely and I use apps like VLC/MX Player on fire stick and nPlayer on android tablet and Infuse on iOS and direct play videos.
Also, right now, some how the HDD is working with Pi again. I connected and it is detected fine (haven't formatted or anything) and my downloads and playback is happening fine too. But I am monitoring the situation and always expecting the HDD to die anytime.

 

[dvader]

USB HDD for 24x7 operations, that's a no no. I tried a similar setup years back but eventually, it required way more oversight than I had time. There are multiple things that can probably go wrong,

1. Your RPi USB itself might be the issue.
2. Almost all external drives I have had in the past (15-20) had ata-passthrough issues in Linux, especially the ARM kernels, I would shuck the devices just to get the SATA interface. Happened mostly with Seagate drives.
3. How are you dissipating heat, external drives are not meant to be plugged 24x7.
4. Configure your torrent client to limit read/write speeds. This helps a lot when you stream media.

Cloud VPS/VMs/Dedicated hosts are another option but it's gonna cost, you can get a budget NAS for that price (I mean when you combine the yearly cost). You can head on to lowendtalk, lowendbox, lowendspirit and get yourself a storage box. However, budget hosts would be unreliable too. If you have an old PC lying around that can serve as NAS too.

 

[vivek.krishnan]

I am using a RPI4 with external drive for eons unknown, but its a powered drive. 100% inside its a enterprise drive for sure due to capacity.

When I was using with a USB powered HDD had same issues. Ditched that for powered HDD and never looked back.

It does spin down when not in use, and takes a fraction of few seconds

Several times - 6-8 times a year - power loss has occurred but so far has worked flawlessly.

 

[babhishek]

I am using a RPI4 with external drive for eons unknown, but its a powered drive. 100% inside its a enterprise drive for sure due to capacity.

When I was using with a USB powered HDD had same issues. Ditched that for powered HDD and never looked back.

It does spin down when not in use, and takes a fraction of few seconds

Several times - 6-8 times a year - power loss has occurred but so far has worked flawlessly.

agree
he needs a DAS or a NAS

honestly with a pi jellyfin is also redundant, upnp is enough for his usecase!


my recommendation is still online server at the end!

 

[raksrules]

USB HDD for 24x7 operations, that's a no no. I tried a similar setup years back but eventually, it required way more oversight than I had time. There are multiple things that can probably go wrong,

1. Your RPi USB itself might be the issue.
2. Almost all external drives I have had in the past (15-20) had ata-passthrough issues in Linux, especially the ARM kernels, I would shuck the devices just to get the SATA interface. Happened mostly with Seagate drives.
3. How are you dissipating heat, external drives are not meant to be plugged 24x7.
4. Configure your torrent client to limit read/write speeds. This helps a lot when you stream media.

Cloud VPS/VMs/Dedicated hosts are another option but it's gonna cost, you can get a budget NAS for that price (I mean when you combine the yearly cost). You can head on to lowendtalk, lowendbox, lowendspirit and get yourself a storage box. However, budget hosts would be unreliable too. If you have an old PC lying around that can serve as NAS too.

To answer your questions...

1. Not sure if my drive is bad or not right now as for now, it is working fine with my Pi and i did the fsck command it says drive is CLEAN. Not sure it means all ok. Also I found that the 3A Power adapter has gone bad. The green light kept blinking and LAN lights won't come on. I replaced it with another wall adapter which outputs 2.4A so hopefully it will help.
2. no reply as I don't understand this.
3. no specific way to dissipate the heat. But the entire setup is in a area where there is lot of airflow, not in some enclosed space.
4. Torrent client is not even writing on HDD, it writes on MicroSD card and the data is transferred to HDD once download is complete. The HDD is only used for storage and spins only when transfer happens or when I stream something.

I am using a RPI4 with external drive for eons unknown, but its a powered drive. 100% inside its a enterprise drive for sure due to capacity.

When I was using with a USB powered HDD had same issues. Ditched that for powered HDD and never looked back.

It does spin down when not in use, and takes a fraction of few seconds

Several times - 6-8 times a year - power loss has occurred but so far has worked flawlessly.

What drive you have? I guess 3.5mm? Can you tell me name or amazon link or something?

agree
he needs a DAS or a NAS

honestly with a pi jellyfin is also redundant, upnp is enough for his usecase!


my recommendation is still online server at the end!

I understand UPNP is enough for which I don't even need to Pi, I can do from my router directly. Just that jellyfin can have trakt sync and such features and also remembering the location where I stopped, what is upcoming etc.
Also my brother and his wife stay in same house so I have separate account for him in jellyfin and same for my kids so only kids content they can see.