Most active commenters
  • sandreas(4)
  • Dylan16807(4)

←back to thread

431 points ingve | 12 comments | | HN request time: 1.95s | source | bottom
Show context
sandreas ◴[] No.44466616[source]
While it may be tempting to go "mini" and NVMe, for a normal use case I think this is hardly cost effective.

You give up so much by using an all in mini device...

No Upgrades, no ECC, harder cooling, less I/O.

I have had a Proxmox Server with a used Fujitsu D3417 and 64gb ecc for roughly 5 years now, paid 350 bucks for the whole thing and upgraded the storage once from 1tb to 2tb. It draws 12-14W in normal day use and has 10 docker containers and 1 windows VM running.

So I would prefer a mATX board with ECC, IPMI 4xNVMe and 2.5GB over these toy boxes...

However, Jeff's content is awesome like always

replies(8): >>44466782 #>>44466835 #>>44467230 #>>44467786 #>>44467994 #>>44468973 #>>44470088 #>>44475321 #
fnord77 ◴[] No.44467230[source]
these little boxes are perfect for my home

My use case is a backup server for my macs and cold storage for movies.

6x2Tb drives will give me a 9Tb raid-5 for $809 ($100 each for the drives, $209 for the nas).

Very quiet so I can have it in my living room plugged into my TV. < 10W power.

I have no room for a big noisy server.

replies(2): >>44467725 #>>44468532 #
1. sandreas ◴[] No.44467725[source]
While I get your point about size, I'd not use RAID-5 for my personal homelab. I'd also say that 6x2TB drives are not the optimal solution for low power consumption. You're also missing out server quality BIOS, Design/Stability/x64 and remote management. However, not bad.

While my Server is quite big compared to a "mini" device, it's silent. No CPU Fan only 120mm case fans spinning around 500rpm, maybe 900rpm on load - hardly noticable. I've also a completely passive backup solution with a Streacom FC5, but I don't really trust it for the chipsets, so I also installed a low rpm 120mm fan.

How did you fit 6 drives in a "mini" case? Using Asus Flashstor or beelink?

replies(3): >>44468123 #>>44469640 #>>44469909 #
2. epistasis ◴[] No.44468123[source]
I'm interested in learning more about your setup. What sort of system did you put together for $350? Is it a normal ATX case? I really like the idea of running proxmox but I don't know how to get something cheap!
replies(1): >>44470274 #
3. Dylan16807 ◴[] No.44469640[source]
> I'd not use RAID-5 for my personal homelab.

What would you use instead?

ZFS is better than raw RAID, but 1 parity per 5 data disks is a pretty good match for the reliability you can expect out of any one machine.

Much more important than better parity is having backups. Maybe more important than having any parity, though if you have no parity please use JBOD and not RAID-0.

replies(2): >>44470252 #>>44470281 #
4. j45 ◴[] No.44469909[source]
I agreed with this generally until learning the long way why RAID 5 minimum is the only way to have some peace of mind and always a nas with at least 1-2 extra bays than you need.

Storage is easier as an appliance that just runs.

5. sandreas ◴[] No.44470252[source]
I'd almost always use RAID-1 or if I had > 4 disks, maybe RAID-6. RAID-5 seems very cost effective at first, but if you loose a drive the probability of losing another one in the restoring process is pretty high (I don't have the numbers, but I researched that years ago). The disk-replacement process produces very high load on the non defective disks and the more you have the riskier the process. Another aspect is that 5 drives draw way more power than 2 and you cannot (easily) upgrade the capacity, although ZFS offers a feature for RAID5-expansion.

Since RAID is not meant for backup, but for reliability, losing a drive while restoring will kill your storage pool and having to restore the whole data from a backup (e.g. from a cloud drive)is probably not what you want, since it takes time where the device is offline. If you rely on RAID5 without having a backup you're done.

So I have a RAID1, which is simple, reliable and easy to maintain. Replacing 2 drives with higher capacity ones and increasing the storage is easy.

6. sandreas ◴[] No.44470274[source]
My current config:

  Fujitsu D3417-B12
  Intel Xeon 1225
  64GB ecc
  WD SN850x 2TB
  mATX case
  Pico PSU 150
For backup I use a 2TB enterprise HDD and ZFS send

For snapshotting i use zfs-auto-snapshot

So really nothing recommendable for buying today. You could go for this

https://www.aliexpress.com/item/1005006369887180.html

Or an old Fujitsu Celsius W580 Workstation with a Bojiadafast ATX Power Supply Adapter, if you need harddisks.

Unfortunately there is no silver bullet these days. The old stuff is... well too old or no longer available and the new stuff is either to pricey, lacks features (ECC and 2.5G mainly) or to power hungry.

A year ago there were bargains for Gigabyte MC12-LE0 board available for < 50bucks, but nowadays these cost about 250 again. These boards also had the problem of drawing too much power for an ultra low power homelab.

If I HAD to buy one today, I'd probably go for a Ryzen Pro 5700 with a gaming board (like ASUS ROG Strix B550-F Gaming) with ECC RAM, which is supported on some boards.

7. timc3 ◴[] No.44470281[source]
I would run 2 or more parity disks always. I have had disks fail and rebuilding with only one parity drive is scary (have seen rebuilds go bad because a second drive failed whilst rebuilding).

But agree about backups.

replies(1): >>44470465 #
8. Dylan16807 ◴[] No.44470465{3}[source]
Were those arrays doing regular scrubs, so that they experience rebuild-equivalent load every month or two and it's not a sudden shock to them?

If your odds of disk failure in a rebuild are "only" 10x normal failure rate, and it takes a week, 5 disks will all survive that week 98% of the time. That's plenty for a NAS.

replies(1): >>44471796 #
9. dwedge ◴[] No.44471796{4}[source]
If the drives are the same age and large parts of the drive haven't been read from for a long time until the rebuild you might find it already failed. Anecdotally around 12 years ago the chances of a second disk failing during a raid 5 rebuild (in our setup) was probably more like 10-20%
replies(1): >>44471914 #
10. Dylan16807 ◴[] No.44471914{5}[source]
> and large parts of the drive haven't been read from for a long time

Hence the first sentence of my three sentence post.

replies(1): >>44472596 #
11. dwedge ◴[] No.44472596{6}[source]
If I wanted to deal with snark I'd reply to people on Reddit.
replies(1): >>44475109 #
12. Dylan16807 ◴[] No.44475109{7}[source]
My goal isn't to be rude, but when you skip over a critical part of what I'm saying it causes a communication issue. Are you correcting my numbers, or intentionally giving numbers for a completely different scenario, or something in between? Is it none of those and you weren't taking my comment seriously enough to read 50 words? The way you replied made it hard to tell.

So I made a simple comment to point out the conflict, a little bit rude but not intended to escalate the level of rudeness, and easier for both of us than writing out a whole big thing.