Large Digital Storage Device Help - Off network back-up for home computers, device etc.

[Carlos Thin]

After having many computers fail on me and the constant changing of devices, I was hoping to build my own storage device for backing up the essential / sentimental digital media I have in my household. Would prefer to keep it offline, and definitely not deal with a cloud service.

I know hard drives drop dead worse than soldiers in a trench, but do you guys know of an amateur project that isn't just add a bunch of hard drives to your computer? My own sleuthing has led to devices which cost $10,000+ which is obviously out of my range.

I have good knowledge of personal computer building /programming, and rusty but workable knowledge of soldiering and electrical engineering

 

[herefortheactualthreads]

i am interested in this as well, i wanna store all possible roms in every single game console possible, but all i get suggested for 100 TB of storage is one big ass machine with 5 harddrives attached to it which costs 3000 bucks, if theres any ideas for retarded end-users, i am all ears.

 

[Seymour Duncan]

You pretty much have three options.

- HDD based stuff. You know what this costs and how this goes, a NAS that you regularly back up to a separate offline USB device is reasonably robust. SSDs lose charge and corrupt data if you keep them unpowered for years, so warm storage only
- M-DISC. Explicitly designed for long term cold archival, but since it's basically fancy Blu-Rays, it's also slow and gets very expensive if you need to hoard lots of data. Consider as an extra secure third backup for stuff you really want to keep?
- tape. Hideous cost of entry unless you go used, but very fast, scales to effectively arbitrary amounts of data and unbelievably cheap media, shelf life in the couple decades

Whatever you choose, VERIFY YOUR BACKUPS REGULARLY. There's a way too large pool of reasons why backups can go bad and you want to make sure they're accessible and working before you actually need them, there's nothing like the feeling of going to your backups to "restore" oops nope you're fucked from both ends.

 

[Sneed's Feed n Seed]

Aite so my advice is:

1)Proxmox or baremetal for TrueNAS Core/Scale
2) "Shuck" some external USB drives. Literally just pull the drive out of the plastic bullshit and stick them in a proper SATA. 4 8TB drives in RAIDZ1 gives you 24TB useful with single drive failure tolerance. Or go RAIDZ2.
3) External LTO if you want to get into the real autism. $2-400ish for the drive, add another 50 for the card, SAS cable, and a tape. Tapes are $25ish.

shucks.top tracks the shuckable drives
r/DataHoarder has interactive autism

Make sure your server has SSDs or disks and RAM with space for swap so you don't stop the tape from streaming and making it stop, rewind, reposition, etc, blah blah blah. Doesn't need to be the fastest, just big enough.

Just remember random access on tape is literal shit from a butt while sequential is surprising.

(Apparently 3.3v pin issues on white-label drives is a thing so check if that's a problem with a newer WD white-label drive i you happen to get one. Molex-to-SATA adapters or some kapton tape will work to your autistic taste)

 

[SCV]

i am interested in this as well, i wanna store all possible roms in every single game console possible, but all i get suggested for 100 TB of storage is one big ass machine with 5 harddrives attached to it which costs 3000 bucks, if theres any ideas for retarded end-users, i am all ears.

View attachment 8455240

Don't get one of these all in one appliances. They are over priced, eventually the vendor stops supporting it so you need to buy another one, migration between them is somewhere between annoying and impossible, they often (thinking of synology) have custom filesystems (ie. vendor lock in / if something goes wrong you are SOL), and on and on and on. One of the vendors even tried to disallow hard drives that you didn't buy through them. DO NOT REDEEM.

I know hard drives drop dead worse than soldiers in a trench, but do you guys know of an amateur project that isn't just add a bunch of hard drives to your computer? My own sleuthing has led to devices which cost $10,000+ which is obviously out of my range.

I have a nasty little secret for you: All of these specialized appliances are literally just under powered x86 (or sometimes <shudder> arm) computers in a box with an ancient linux kernel, webui for retards and some harddrive bays sold at a massive markup. There is no special sauce only vendor lock in. For a laugh you can compare the specs of these to a comparable computer. The only positive is retards might be able to set it up (though connecting all their cloud accounts, naturally).

I have good knowledge of personal computer building /programming

Then you definitely do not want a fixed function appliance.

Now that I've hopefully convinced you of that we're past step 1. Step 2 is what you are actually going to buy. This is surprisingly simple: Literally just a PC with storage in it. The specs almost don't matter and if you have an old PC lying around that is perfect. The only real questions will be does your case support enough HDDs/SSDs and if you have a lot you might need a bigger PSU. Sidenote: tape drives are for when you're much farther down the rabbit hole and unless your data is tiny even M-DISC/blu-ray will be annoying. Ignore those for now.

The final and actually hard/complicated step is what software you're going use. Personally I just have linux installed and send btrfs snapshots. That's probably not a good fit for you unless all your devices are on btrfs. There are a ton of options. You could do a network share on normal linux, I've heard good thing about trueNAS, self-host a cloud thing (own cloud, next cloud), syncthing, etc. My only recommendation right now would be that the backups need to be automated so you don't get lazy. Beyond that I need more details about your goals and devices you have.

i wanna store all possible roms in every single game console possible

Since this is more specific I can give you a bit more advice. That will be a lot of space so you're going to need multiple hard drives i.e. a raid array. Given the effort involved you'll also want to have at least some partiy/redudancy. You'll want to buy/scavenge a cheap desktop with a case that can ideally support 8+ HDDs, put linux on it, then make either a ZFS or BTRFS raid array with some used drives. Probably setup the array as a network share for conveinience.

Get your cheap HDDs here, try to make them 8tb or more each. Used to be ~5$/TB was a good deal but with muh AI that's no longer the case. Also don't be afraid of SAS drives they just need a 30$ HBA. https://pricepergig.com/en/ebay-us

Any questions please ask.