My customers have a rather large hunger for storage-capacity, but not the budgets to buy expensive SAN networks or NAS-heads. This post will describe a low-cost, high capacity storage platform that should be able to provide decent performance at a cost of about 110 euro per TB usable capacity (given raid-6, with 6+2 disks per diskgroup).
The basis system consists of:
- Norco RPC4224 4U 24-Bay hot-swap server cabinet (est price 440,-)
- 6x SFF Cables (96,-)
- 850Watt Seasonic PSU (128,-)
- Rail kit (32,-)
- Supermicro X8SIA-F Mainboard (200,-)
- Supermicro AOC-SASLP-MV8 Sata controller (3x, total price 300,-)
- 32 GB SSD, 2x 50,- (just to boot from) (100,-)
- Xeon X3440 Boxed CPU (200,-)
- 16GB ECC/Registered DDR3 Ram KVR1333D3D4R9SK2/16GB (130,-)
This brings the base-system price to about 1600,-
To this base system we can then add disks as needed, in sets of 8. Currently I would recommend the following disks:
- Hitachi Deskstar 7k3000 (3tb, ~200 euro)
- Hitachi Deskstar 7k4000 (4tb, ~270 euro)
- Seagate Constellation ES2 (3tb, ~270 euro)
When adding 24 4TB disks, the total system price would end up at ~8100 euro, and have a total capacity of about 72 TB in 3 diskgroups of 24TB each (with 8 disks, of which any 2 may fail). To spread out the costs over a longer period, and to be able to grow, you could consider buying the disks in sets of 8. I would however recommend getting at least 1 or 2 disks as cold-spare to be able to quickly swap out broken disks.
On the software side, install your linux distribution of choice, use dm-raid and LVM, and put XFS or Ext4 filesystems on top. In the future BTRFS might be a better solution, but for now it’s still a bit too immature. A solaris or BSD install with ZFS would also be possible, but is not something I’ve tested on this setup yet.