What am I doing wrong? 2nd external HDD gone bad in last 1 year (running 24x7)

raksrules

raksrules

Elite
I am using RPi4 with an external HDD. Last year, sometime in June 2021, my Seagate external HDD went bad. This HDD was connected to the Pi and running 24x7. I am not sure if there was any spin down or so when data is not being written or read.
I had RMAed that HDD and started using that for other purposes and ordered a WD Passport 5TB drive from tatacliq in July 2021. It has been used with my RPi4 since 06th July 2021 and today out of blue, it too seems to have corrupted. I connected it to a mac and windows laptop and in both, the drive doesn't appear and clicking sound comes.
On my RPI4, I did run (i think) "sudo -l fdisk" and after lot of time, the drive was listed but file system was "Microsoft basic file" something, no clue what is. I expected EXT4.

Now, I have initiated RMA of this less than a year old WD Passport. I want to understand the root cause here...

  1. Is using an external 2.5 Inch external HDD a bad idea for something that is needed to run 24x7? Especially torrent downloading and accessing downloaded media over wifi?
  2. If yes, then what other option should I go for?
  3. Will a powered 3.5 Inch external HDD be better bet than a 2.5 Inch one?
  4. If yes to above question, which one to buy?
 
First and foremost - not all external HDDs are meant for running 24x7. If you really need to run the HDDs 24x7, you should either go for Backup+ range - they have larger/bigger enclosure for heat dissipation and it serves the HDDs. In my observation - heat played major role in higher wear and tear of HDDs.

My recommendation will be to go with docking station and keep it ventilated or get Backup+ drive and get smaller fan at the bottom, this way you will have portability and still can run 24x7 these disks.
 
In addition to what @tech.monk mentioned I would add that it is critical that the RPI is powered properly so that appropriate power can be distributed to the peripherals including the HDD. It is better to use a higher rated power adapter than the minimum needed as the power draw tends to vary based on the running load and the number of peripherals you connect. In that sense, getting a external drive with separate power supply is always better.
Having said that, I have a 1 TB WD 2.5 inch connected to a Rpi 2b for more than 1.5 years now and it is working fine. The Pi is connected via a UPS though as it also hosts the Adguard Home install.
 
tech.monk said:
First and foremost - not all external HDDs are meant for running 24x7. If you really need to run the HDDs 24x7, you should either go for Backup+ range - they have larger/bigger enclosure for heat dissipation and it serves the HDDs. In my observation - heat played major role in higher wear and tear of HDDs.

My recommendation will be to go with docking station and keep it ventilated or get Backup+ drive and get smaller fan at the bottom, this way you will have portability and still can run 24x7 these disks.
Click to expand...

I am ok with a powered HDD. Can you suggest some (minimum 4TB HDD preferred) can I look into for my needs? I am not sure if I will buy immediately as once this WD one is RMAed, I will use the replacement one but still will explore other options.
@tech.monk

You suggesting a docking station that can host a 3.5 Inch or 2.5 Inch drive in it and I can power a 3.5 Inch drive using this right? Here is an example.


You think this will help as HDD is sort of exposed so more ventilation and more heat dissipation?

if yes then what HDD should I buy to use with docking station?
 
Last edited:
For your use case(running 24x7), you should use a SSD. If you must use a HDD buy a NAS rated hard drive with external enclosure. All external HDDs are rated for running 8X7 not 24x7 . If it is a SMR drive inside, and you use it for thousands of small file(1KB to 10MB) writes everyday, it significantly reduces the lifespan of the drive.
 
Can anyone tell me if what I am planning to do will help me? Because I don't intend to buy a new drive for now and will work with the replacement drive that I get from WD.

I intend to do this..

  1. Attach a 32GB / 64GB MicroSD Card which I already have with me and this will be primary place where torrents will be downloaded. I even have 7-8 16GB Pendrives that are not so good (slow and all) and were given to my brother from office (company name on it etc).
  2. I use public torrents 99% of time so once download is complete, I move the files to the HDD (i guess qbittorrent can do this). Also at this point, torrent seeding is paused / stopped.
  3. This will free up space from the MicroSD Card / Pen Drive.
  4. This way, writes only happen to the HDD when full episodes or seasons are downloaded and all other times, on need basis, only reads happen when streaming content.
  5. This may also help me with an issue I used to face earlier when say a torrent is being downloaded, since the HDD is being written, if I attempted to stream, many a time, stream would stutter / buffer very badly. I feel this may also have been reason for failure of my HDD. Just speculating.
 
raksrules said:
I am ok with a powered HDD. Can you suggest some (minimum 4TB HDD preferred) can I look into for my needs? I am not sure if I will buy immediately as once this WD one is RMAed, I will use the replacement one but still will explore other options.
@tech.monk

You suggesting a docking station that can host a 3.5 Inch or 2.5 Inch drive in it and I can power a 3.5 Inch drive using this right? Here is an example.


You think this will help as HDD is sort of exposed so more ventilation and more heat dissipation?

if yes then what HDD should I buy to use with docking station?
Click to expand...
I use PiBox and recently fellow forum member tested them (since you're also in Mumbai - I can lend you for few days to get the experience) if all works Okay - you can switch to docking station based setup with proper ventilation. This will help with any desktop based drive to survive longer.
 
raksrules said:
Can anyone tell me if what I am planning to do will help me? Because I don't intend to buy a new drive for now and will work with the replacement drive that I get from WD.

I intend to do this..

  1. Attach a 32GB / 64GB MicroSD Card which I already have with me and this will be primary place where torrents will be downloaded. I even have 7-8 16GB Pendrives that are not so good (slow and all) and were given to my brother from office (company name on it etc).
  2. I use public torrents 99% of time so once download is complete, I move the files to the HDD (i guess qbittorrent can do this). Also at this point, torrent seeding is paused / stopped.
  3. This will free up space from the MicroSD Card / Pen Drive.
  4. This way, writes only happen to the HDD when full episodes or seasons are downloaded and all other times, on need basis, only reads happen when streaming content.
  5. This may also help me with an issue I used to face earlier when say a torrent is being downloaded, since the HDD is being written, if I attempted to stream, many a time, stream would stutter / buffer very badly. I feel this may also have been reason for failure of my HDD. Just speculating.
Click to expand...
Actually you are better off writing directly to the HDD rather than the SD Card since the card will not survive frequent R/Ws.
In your last point you are implying that the buffering happens when you stream a file different than the one you are downloading or seeding ?
 
ibose said:
Actually you are better off writing directly to the HDD rather than the SD Card since the card will not survive frequent R/Ws.
In your last point you are implying that the buffering happens when you stream a file different than the one you are downloading or seeding ?
Click to expand...

Yes different file. I am sure it happens because qbittorrent is downloading content or uploading something at higher speeds (many leechers downloading from me) while I am streaming (totally different file). If there are stalled torrents or someone is not uploading from me or so, then no issues come.

I also see that in Raspberry Pi, the load reaches 7-8 (red color) when this happens and the only way for me to come out of this is to pause all torrents and wait for load to come down well under 5.

The reason I told I can use this pen drive as intermediatory is because I have many of them which are serving no purpose and I can leverage for this. Also, aim is to have writes happen to HDD only in bulk whenever a torrent is downloaded fully and this means, I think HDD can spin down when no writes are happening for extended time.
Obviously my thinking may be wrong.
avinandan012 said:
For your use case(running 24x7), you should use a SSD. If you must use a HDD buy a NAS rated hard drive with external enclosure. All external HDDs are rated for running 8X7 not 24x7 . If it is a SMR drive inside, and you use it for thousands of small file(1KB to 10MB) writes everyday, it significantly reduces the lifespan of the drive.
Click to expand...
Isn't frequent writes to SSD bad? I am not sure but I had read somewhere.
 
Last edited:
SSDs are made with different types of NAND cells which have finite number of P/E cycle. There are types of SSDs(SLC type) which have very high endurance(e.g. Intel Optane), for these frequent writes are not that bad. Read this. See the program/erase(P/E) cycle rating of SLC/MLC vs others.
I have a MLC drive from 2014 which is being used as Windows boot drive and have a TLC drive from 2016 used similarly. The MLC drive still shows remaining life as 99% where as TLC drive is at 84%.

Similarly there are different recording technologies available in HDDs too. Mainly CMR vs SMR. SMR drives have very low write endurance compared to CMR drives. Most cheap external drives these days have SMR drives. Though SMR drives provides very high capacity/₹ compared to CMR drives. SMR drives are good for writing huge files, very bad at handling small size file writes.
 
Last edited:
When I first got a router with USB port, I used a 2.5" HDD with it for a basic NAS setup. I used that setup for over 2 years. It didn't run 24x7, I used to often switch off the router when I was at work, since I was living alone back then. My HDD survived that phase, but started giving issues later.

What you say about writing torrents to SD card might work for the HDD, but will destroy the SD card very quickly.

If and when you do want to buy a new drive, you can go for Seagate Backup Plus Hub or something similar. They mostly have NAS or enterprise hard drives inside, even though they don't specify. So are much better for 24x7 workloads.

Otherwise, get an Orico 2.5" enclosure and a Crucial MX500 SSD and use this as your caching drive rather than the SD card or pen drives.
 
avinandan012 said:
Similarly there are different recording technologies available in HDDs too. Mainly CMR vs SMR. SMR drives have very low write endurance compared to CMR drives. Most cheap external drives these days have SMR drives. Though SMR drives provides very high capacity/₹ compared to CMR drives. SMR drives are good for writing huge files, very bad at handling small size file writes.
Click to expand...

Thanks. I am not very knowledgable on CMR/SMR and now have some fair idea. As you told, SMR drives which most external HDDs may have are good for huge files, in that case, if for my torrent downloads, I have a caching system, either a pen drive / Microsd card like I am using right now or say a smaller SSD in future can serve well? Because in that case, all writes to HDD will only happen once torrent has downloaded so it is kind of bulk writes occasionally instead of small small files like torrents downloading does.
gourav said:
When I first got a router with USB port, I used a 2.5" HDD with it for a basic NAS setup. I used that setup for over 2 years. It didn't run 24x7, I used to often switch off the router when I was at work, since I was living alone back then. My HDD survived that phase, but started giving issues later.

What you say about writing torrents to SD card might work for the HDD, but will destroy the SD card very quickly.

If and when you do want to buy a new drive, you can go for Seagate Backup Plus Hub or something similar. They mostly have NAS or enterprise hard drives inside, even though they don't specify. So are much better for 24x7 workloads.

Otherwise, get an Orico 2.5" enclosure and a Crucial MX500 SSD and use this as your caching drive rather than the SD card or pen drives.
Click to expand...

Thanks. I will consider external SSD or internal SSD + enclosure in near future. For now, even with full understanding that torrent writes will destroy SD Card / pen drive, I still will go ahead because these pen drives (16GB) are not like some great brand ones, they are the ones which people give at conferences or booths of some company for free and has their company name on it etc. Also the SD Card I have is actual SD Card, no MicroSD from when i owned a digital camera like 7-8 years bac so I won't worry about their failures honestly as at this point, these things are just lying in a box at my home and will even go bad probably without be doing anything.
 
Last edited:
If you use pen drives which are disposable for you, then you don't have to worry about anything. But if you're using your SD card, then when that stops working, you'll have to rebuild the system from scratch. So it's better to use pen drive and only use SD card when the torrent is 15 GB or more.

Also tweak the settings of the torrent client to reduce writes to drive.
 
for torrents, the file may be big but it is fragmented in 100s of small chunks, so for a SMR drive it may mean it is writing 100s of small chunks instead of a single big write.

gourav said:
If you use pen drives which are disposable for you, then you don't have to worry about anything. But if you're using your SD card, then when that stops working, you'll have to rebuild the system from scratch. So it's better to use pen drive and only use SD card when the torrent is 15 GB or more.

Also tweak the settings of the torrent client to reduce writes to drive.
Click to expand...
@OP for torrenting disposable pen drives are your best bet.
 
Last edited:
Back
Top