Suitable method for Backup of Data

[Psycho_McCrazy]

While I did read a few threads about this, my needs may be a bit different and thus I am creating my thread.

Backed-up data I believe may fall into four categories:
Absolutely Necessary > Necessary > Good to Have > I Live Luxuriously

I currently have close to ~3TB of Data

• A fair bit of it is Games (Installed) and Steam Game Backups - these will definitely fall into the "I Live Luxuriously" category.
• Then there are a lot of media files (movies, series, music etc.) - These will be a part of the "Good to Have" category, since most of the stuff is now on subscription services, and in better image quality levels. Even if the data is lost for some reason, it won't be the end of the world.
• Personal photos, Videos, and all are memories - and definitely "Necessary". Data loss would be bad.
• Last and most important I guess are certain financial and asset related documents, which are sensitive as well as "Absolutely Necessary" to backup. Any data loss or leak may have stark consequences.

As it turns out however, all my data is currently in the "I'm living dangerously" category.
No backups, and stored on HDDs that are up-to 15 year old!

I wish to change that, and am thinking that I'll create a tiered system.
Critical data would be stored on the actively accessed storage, as well as two local back-ups (and maybe a cloud backup).
Non-Critical, personal data (photos etc.) would be stored on the active access storage, as well as one local back-up copy.
Media can be kept on the active access storage as well as partially backed up on one local copy (assorted important ones).
Game Installers can just be left on the active access storage...

My current thought is have the following system:

Storage Type	Critical Data	Personal Data	Media etc.	Game Stuff
Active Storage Disks	Keep and use as usual	Keep and use as usual	Keep and use as usual	Keep and use as usual
Tier 1 Backup	Backup every Week	Backup Every Month	Sorted Backup / Month
Tier 2 Backup	Backup every Month
Cloud Backup	Maybe? Monthly?

The question therefore is to find suitable hardware (this being the computer hardware forum) for each storage tier:
Active Storage Disks:
Current HDD's in a NAS (including old ones, replace when fails)

Tier 1 Backup:
Option-1: Additional, dedicated HDDs in that same NAS
Option-2: External HDD

Tier 2 Backup (probably limited storage required):
Option-1: Separate External HDD
Option-2: Burnable media, dated backups

Please suggest suitable methods from the above options, or from beyond these as well....

 

[b.life]

Good Evening,
To be honest with you no two solution are the same, each persons requirements are going to be different as such I would recommend you to look at others and customize it for your requirements this way you will have total control and full understanding of the system. Said that I do have some questions before we can comeup with a solution.

Technical Section
1. How often does the Critical/Personal Data change?
2. How often do you access the Critial/Personal data?
3. Regarding the photos do you plan/already run a service(Lychee/Piwigo/etc) to access them?
4. Do you have any plans regarding data integrity verification? (Data stored on disks will degrade overtime so the disks has to be replaced or data refreshed)

Personal Section
1. Other than you, will there be anyone else responsible for maintaining the backup?
2. Other than you, Is there anyone else who you turst with personal info(critical or otherwise) who is also computer literate enough to maintain/recover/access the backup system?

Few points that I have picked up over the years are.
1. Critical data needs to be backed up daily(incremental), and weekly full backup
2. Non critical data gets backed up every week(incremental) and monthly full backup

From your current setup(I assumed you do have a NAS)
For Tire 1: It is generally recommended to create a new server(NAS) for first layer of backup. Reason being if your current Active storage does down(motherboard,ram,psu,etc) then your tire 1 also goes down. So it is recommended to create a cheap nas(like a low power cpu/mb/psu) with lots of HDD
For Tire 2: Burnable media is nice but I want to remaind you that technology changes(DVD drives are almost gone in less then 15 years i think) the storage medium which you choose must be future proof, meaning you should be able to access the data even if the current access mechanism(CD Drive, DVD Drive, Blueray Drive) becomes obsolete.
For Cloud: If your ISP provides one take it, they are your first point of contact for internet and you can easly reach them, than the ones provided by MNC companies. I would suggest you to strongly consider keeping your critical/personal date in an encrypted format in the cloud. Reason being they will handle the harddisk failure, technology change etc so you dont need to worry about them.

Regards

Edit 1: Spelling correction

 

[krsamy]

I am using Microsoft onedrive which gives 1TB of cloud storage free along with office 365 subscription that costs about 4300 per year. It can be shared with 5 users and you can invite yourselves to use that extra space but I have never done that.

 

[reborn]

I have been looking into this in some detail recently for backing up my files. Please note that I am still researching this topic, the following may not be entirely correct / up to date. I was primarily looking at backing up to OneDrive since I already have a subscription.

There are two threats to my data as I see it:

1. What if my HDD crashes or I delete some files by mistake?
2. What if I get infected by ransomware?

The first one is relatively easy to handle. Backup data to a NAS or an external HDD with something like Veeam community. If you want cloud backup, you can use something like rclone / restic / Duplicacy. From what I read, these three seem to be the best supported / production ready software. There's also Kopia / Duplicati / Borg. All of them support encrypted backup.

Do note that sync is not backup - just syncing your files to OneDrive or using something like Cryptomator to sync your files won't work as backup . If you delete or overwrite a file by mistake OneDrive will happily modify the cloud copy. I think OneDrive provides file history of 1 month for some file types, but not entirely sure.

The second scenario is a lot harder to handle. A ransomware which sits quietly encrypting your data and you don't notice it for say 6 months or 1 year is a nightmare scenario for backups. OneDrive has built in ransomware protection, but I am not sure how good it is. They also only provide 1 month of history, so if their protection doesn't trigger and you don't notice the ransomware for 1 month or go on a vacation and don't check your email, your files are gone. Since I intend to upload only encrypted files to OneDrive, it is even less likely that their ransomware protection will trigger - its like choosing between two random data files at their end.

If you back your data to a local NAS share or external HDD, ransomware can easily encrypt those. If you use a cloud backup software to upload your backup, a smart ransomware can steal your credentials from your cloud backup software and log into your cloud storage and delete or encrypt your backups. I believe ransomware already does the former, and while I haven't heard of any ransomware doing the latter, its not hard to imagine a ransomware doing so once cloud backup becomes more popular. So currently as it stands, a cloud backup is probably sufficient to protect against ransomware, but for your most valuable data, you may want to go beyond that.

There are some mitigation techniques. You can run your backup software as a different user - you can't access it's files but it can access yours readonly. That way if your account gets compromised the ransomware cannot steal the cloud login credentials from your backup software. Windows inherently supports this through the Backup Operator user group. However if you are running as admin or the ransomware is able to obtain admin credentials then again its game over. You can also run the backup software on a separate computer, say a Raspberry Pi or your NAS, and have the backup software log into your PC and access your files. This is called a pull backup, and it is the most secure form of backup. In case you use a NAS and cloud backup both for 3-2-1 backup strategy, the cloud backup software should run on your NAS. Hopefully your NAS as a dedicated machine will be less vulnerable to ransomware infection.

Ideally what is needed is a append-only data storage location, be it a cloud or local server. Dedicated cloud backup services provide this, but OneDrive does not. Restic in server mode supports this in case you want to roll one locally.

Anyways, this is a long wall of text, and my research is still work in progress. TLDR is you absolutely need to plan for ransomware attacks when deciding on your backup strategy, not just HDD crash.

 

[b.life]

@reborn I believe could storage providers are providing immutable storage options these days, I am not sure if onedrive provides one but if they do then availing it will be a good thing.

I will detail my setup for the immutable storeage backup and hope it will give you an idea for yours.
Step 1: Data from phones(photos,videos)(incremental)(sms,contacts)(full backup), pc(scanned documents, downloaded documents,etc)(incremental), NAS(password db, configuration files, file list)(full backup) gets copied the NAS during the day and to a seprate hdd(1TB) within the nas at a fixed cut off time(7PM).
Step 2: The data from step 1, is then copied over to a low powered secondary nas at a fixed cutoff time(8PM). Once the data is copied to secondary nas the then main copy is deleted from the primary nas's temp disk.
Step 3: Once the data reaches the step 2, it is checked if the data can fit a dvd(3gb & above only) or bd disk(20gb & above only).
Step 3.1: Case 1: Data is >3GB and <4.5GB, the data is writtern to the dvd and the dvd ejected, I write the backup number, date on the disk and place a new disk in the drive and close it.
Step 3.1: Case 2: Data is >20GB and <25GB, the data is writtern to the bd disk and the bd ejected, I write the backup number, date on the disk and place a new disk in the drive and close it.
Step 3.1: Case 3: Data is >4.5GB and <20GB | <3GB, then the data is copied to the attached 32GB pendrive and pendrive is unmounted and ejected and a local mail is sent to the user.
Step 3.1: Case 3.1: Other than the regular backup a seprate copy of the back is stored in a 1TB drive till the combined size of the data in the pendrives reach either case 1|2
Step 3.1: Case 4: If the comulative data size in the pendrive reach either case 1|2, then the data is writtern to either dvd | blueray. The data is accessed from the temp drive(1TB ones) and purged after write.
Step 3.2: Case 1: Once the data is either writtern to dvd/bd a file csv file is created with the files names and date of backup. This is then pushed(manually) to the database server for file location search.
Step 3.2: Case 2: If the data is writtern to pendrive, then a seprate file is created numbering from 1 to 10 which lists the files in that perticular pendrive.
Step 4: Once the writtern data is verified the process moves to step 1.

To prevent ransomware from encrypting the pendrive I have 11 of them with number from 1 to 10 and Number 11 as main backup. Once the data is copied I remove that pendrive and insert the next one. Once I have used all 10 of then, I insert the 11th and copy data from the 1 to 10 from the temp storage(1TB disk).

Each time an automated backup(number 1 to 10) is performed to the pendrive, I have a script that will
1. Format the pendrive
2. Mount the pendrive
3. Copy data
4. Create the file list with backup date and file names.
5. Unmount the pendrive
6. Eject the pendrive
7. Send an internal mail.

For main backup(Number 11)
I do this manually, I copy all the data from the 1TB drive to the pendrive, clear out the hdd, delete the file list(1-10-11) and create a new file list(number 11) that has all the file names and backup date and remove the pendrive.

Hope this helps.