OmniOS: The Hardware

Choice of the components and BOM

This article is part of a series focused on the building and setup of a home-NAS based on ZFS and OmniOS. The index is located here: An OmniOS ZFS Fileserver.

Requirements and Options

The server I was going to build was meant to last five years and to be used both as simple data storage but also to transcode on the fly music and videos (e.g. for mobile devices based on iOS or Android). The data storage and streaming does not require a lot of processing power and a weak cpu would suffice, but transcoding is more computationally intensive and it will get even more so, given the coming of H.265 (the successor of H.264) and the coming of higher resolution devices and sources.
The server was meant to perform also some side tasks such as file download and mail server, but none of them poses particular problems.

Given the current cpu requirements for transcoding I needed at least a Core2 Duo @ 2 GHz, but given the expected future ones I preferred having at least twice that power: a cpu with the same frequency from the newest Intel Core series (i3, i5, i7) with 4 cores was preferred, or something equivalent from AMD. This puts small boxes like the HP Microserver N40L (very common among home-made boxes) out: the cpu is too weak.

Another requirement is ECC RAM, able to detect many memory errors and to correct them on the fly, or to shut down/reboot the machine in case of uncorrectable ones. The probability may not be very high, but the severity increases the longer the computer stays on and a server is always-on. An interesting discussion with differet opinions is available on Reddit. Obviously, ECC RAM is not only useful against errors caused by cosmic radiation: it is also useful to detect bad memory modules or modules once working well but then going bad. On the topic, check this scenario.

Since I wanted to keep the energy bill low (every watt always-on increases the yearly bill by about 2 Euro in Germany), I had to choose an Intel processor, because they are usually more energy efficient. Furthermore, newer AMD processors do not always support ECC memory.

List of Components and Prices

I tried to sum the price of separate components, but it was relatively high: a motherboard with ECC support and a Xeon processor alone could reach easily 400 Euro. As result, I decided to buy an HP ProLiant ML110 G7 on Amazon. The processor, a Xeon E3-1220 (v1, not v2), is much more than what I needed and it will surely draw more power from the socket than a more modest alternative, but the initial price was so low that I was happy anyway.

The standard 2 GB ECC DDR3 were not enough for optimal performances with ZFS, so I decided initially to add two 8 GiB sticks, then I decided to simply max it: right now I have 4x 8 GiB sticks (the max officially supported is 4x 4 GiB but 8 GiB sticks work too).

The hard drives chosen are WD Red, because they tolerate vibrations better than normal drives: this is a problem when multiple disks are stacked close together and written data have to be reliable. Since 3 TB were about what I needed at the time but I wanted some room for growth and since I am using mirrored disks, I bought 4x 3 TB disks. \\ Important: I bought two WD Reds in December and two in March/April from different shops and I kept the first two running (and used) during the months in between. At the end I coupled them to have one old and one new in each mirror pair. This is useful to avoid both disks in each pair to fail at the same time: if two disks are built at the same time, used always in the same way and for the same amount of time, very probably they will also break at almost the same time, before I have time to replace the first broken one, thus causing complete loss of data.

The case of the ProLiant has space for 4 disks and two (not included) 5.25" optical units, but I won't ever need the optical units, so I bought a LianLi cage to convert the two 5.25" bays into three 3.5" bays for additional hard disks. The case can hold now seven 3.5" hard disks.

The motherboard has a single SAS/SATA connector for the main bay holding 4 HDDs, plus anoter two SATA connectors for the optical bays. Given the better cooling of the main cage, I put the 4 data disks on the original bay and the (original) 250 GB boot disk on the new cage, but I encoutered an issue: when the SATA drives are operated in AHCI mode (the best one), booting from the two additional SATA connectors is not possible anymore. I therefore bought a 4x SATA II PCIe card for that use. Obviously, no RAID features have been enabled on the card, everything is performed using ZFS. I chose a SATA card containing the Sil3114 controller, fully supported by the illumos kernel.

The ProLiant is shipped with a pretty useless Seagate 7200.12 250 GB hard disk. The capacity would be enough, but I quickly noticed that configuring and customizing the server requires a LOT of time and I absolutely don't want to do it again after the failure of the 250 GB hard disk (it's not a matter of "if", it's a matter of "when"). I therefore bought the two cheapest solid state drives I found (Kingston SSDNow V300) and I set them up as mirrored boot drive. These SSDs have the lowest idle power consumption: 0.5 W compared to 8-9 W of the original HDD. The savings in the electrical bill will repay completely one of them over the next years! Furthermore, they simply do not heat, so I didn't even use a proper bay and I fixed them somewhere in the case, so I still have 7 bays for data hard drives.

To fully use the Gigabit connection offered by the server and by the desktop computers, I added a Netgear GS108E-100PES 8-Port GigaBit Switch: the Gigabit switches of modem/routers often can't saturate even a single connection.

To summarize,
1x HP ProLiant ML110 G7: € 374.90
1x InLine 4x SATA II, Pcie: € 62.43
4x Transcend 8GB DDR3 1333 MHz unbuffered ECC with thermal sensor: € 220.50
4x Western Digital WD30EFRX Red 3TB: € 579.61
1x Lian Li EX-23NB: € 23.11
2x Kingston SSDNow V300 60 GB: € 119.30
1x Netgear GS108E-100PES: € 40.67
SATA cables: about € 20

Total price: € 1440.52

The product is clearly in the "expensive" price range, but price was not an issue and I preferred a good system rather than a cheap one. Basically, I bought everything needed to have a safe system, except for the hard disks: the WD Red are the best for home use in servers, but I could have chosen enterprise hard disks. These tolerate even better vibrations and ensure an higher integrity of the data. The price however doubles and that would have been too much for a little gain: my server runs 24/7 (and therefore normal desktop hard drives should not be used), but the effective load is low, therefore enterprise class drives are not worth the increased price.

Can the server be made cheaper? of course.
The main unit is already cheaper then the components it includes and, as long as ECC ram and enough cpu power are desired, no cheaper pre-built options were available at the time. If cpu power is not required, a cheaper processor from AMD would be enough. If even ECC ram is not desired, an Atom processor would cut the price of the main unit in half. If 4 hard drives are enough and the server is not going to be customized, a very good HP Microserver N40L is the best option: Booting is performed from a USB thumb drive and the cage holds up to 4 disks.
Given my requirements, I could have saved about € 170 by limiting the RAM to 10 GB (2 GB included plus one 8 GiB stick).
If I were willing to sacrifice some hard disk cooling and expansion options, I could have mounted the two boot drives in the main cage anche put two data drives in the LianLi cage, thus saving the PCIe SATA card.
By using only the original 250 GB boot drive and by making constant backups of the boot environment to the internal SD card (there is a slot soldered on the motherboard!), I could have saved the two SSD disks.
I could have skipped the Netgear switch by accepting slower network performances (but I will be working on 250 MB files...).
In total, about 400 Euro less: 1040 Euro for 12 TB (+120 GB boot) installed (6 TB available, due to mirroring).

As written in the earlier article, I already had 2x WD Green 3 TB drives. These have been put in external USB3/FW800/eSATA cases to act as backup: RAID is not a backup itself.

Measured Power Consumption

I chose this kind of server because it allows me to have custom applications and features, a lot of expansion options (I can easily add a PCIe eSATA card and mount external eSATA cages to go up to 30 TB total capacity) and a lot of cpu power, but I was still interested to compare the power usage to other two alternatives: the HP Microserver N40L (4 slots for hard disks, limited cpu power, limited power consumption) and to an Atom-based high-end Synology solution (good expansion options and web interface, limited customization possible, no ZFS).

I measured the power consumption of the server with cpu almost idling and power saving enabled and with hard disks spinning: 55 W. The four hard disks draw a total of about 20 W, therefore 35 W for the server itself.
An HP Microserver N40L with 2 disks running and the processor idling in power save mode consumes about 21 W, meaning about 12 W for the server itself.
The Synology DS1512+ (five hard disks slots, plus connectors for 10 additional external disks) consumes in idle without hard disks spinning about 22 W.
Given the additional cpu power I needed, 15-20 W more are acceptable. Still, the HP Microserver N40L proves to be a very good product for all the people interested in maximum data safety (it has ECC ram) and in the features provided by NAS4free for a competitive price: € 230. The Synology DS1512+ offers a very good and integrated product with many more features than the HP Microserver, but it costs about € 700.

Useful Links

Author: Olaf Marzocchi

First revision: 2013-05-11.
Last revision: 2015-02-02.