In early November of 2012, I started piecing together a rackmount computer to act as my backup and media server. The intent here was to have a machine with at least 8 hot-swap bays in a rackmount chassis that would accomodate at least an ATX motherboard. It will run the latest version of FreeBSD-stable, configured with one or more ZFS pools.

I wound up with a MicroATX motherboard, and am using an i5 2405S CPU to help keep the power and heat down. This is still mostly overkill CPU-wise, but I didn't want to run into scenarios where I didn't have the CPU to do things on the machine. I also wanted to be able to use 32 gigabytes of RAM, since ZFS is somewhat RAM-hungry. The desire to use at least 32 gigabytes of RAM ruled out low-power CPUs like the Intel Atom processors I use in ria and www.

Below is a list of hardware I've received, with prices.

depot hardware

• Part	Description	P/N	Qty.	Unit Price	Total
  CPU	Core i5-2405S Sandy Bridge 2.5GHz LGA 1155 Quad-Core processor	BX80623I52405S	1	$219.99	$219.99
  case	Rosewill RSV-L4411	RSV-L4411	1	$179.99	$179.99
  RAM	G.SKILL Ares Series 32GB (4x8GB) 240-Pin DDR3 1333 (PC3 10666)	F3-1333C9Q-32GAO	1	$129.99	$129.99
  motherboard	ASRock H77 Pro4-M	H77 Pro4-M	1	$89.99	$89.99
  8-port SATA card	IBM M1015	M1015	1	$80.00	$80.00
  test drive	Western Digital Velociraptor 600GB 10,000 rpm SATA 6.0Gb/s 3.5" hard drive	WD6000HLHX	1	$69.99	$69.99
  power supply	Seasonic S12II 430B 430W 80-Plus Bronze	S12II 430B	1	$59.99	$59.99
  SFF-8087 to SATA cable	3Ware CBL-SFF8087OCF-10M 1-meter SFF-8087 to SATA cable	CBL-SFF8087OCF-10M	2	$14.69	$29.38
  Total					$859.32

I still need to purchase more hard drives.

I currently have a Samsung 840 Pro 512G as my boot drive. A pair of HGST Deskstar 4TB drives are configured as a mirror and acting as backup for other hosts on my LAN. In terms of disk location, it looks like this in the chassis:

The ZFS pool:

% zpool status
  pool: zfs1
 state: ONLINE
  scan: none requested
config:

NAME              STATE     READ WRITE CKSUM
zfs1              ONLINE       0     0     0
  mirror          ONLINE       0     0     0
    gpt/gpzfs1_0  ONLINE       0     0     0
    gpt/gpzfs1_1  ONLINE       0     0     0

errors: No known data errors

Given my current intended usage, it's unlikely that I'll need an SSD for L2ARC nor a SLOG (dedicated ZIL). Right now, my usage is just backups. That's writes only; reads will only occur when I need to do a restore, and cache is of no benefit in that case.

The current disk geometry:

IBM M1015 as an 8-port SATA 6.0Gb/s Host Bus Adapter

I have an IBM M1015 installed so I can utilize all of the hot-swap drive bays. I have it in the PCI xPress x16 slot of the ASRock H77 Pro4-M motherboard, since I have no need for discrete graphics in a rackmounted server that lives in my basement. The card is x8 electrically, and the ASRock H77 Pro4-M has no x8 slots. I could have placed it in an x4 slot, but with the x16 slot unoccupied, it made sense to use it.

More information on using the IBM M1015 in IT (Initiator Target) mode can be found here. Most of the information was gleaned from various sources on the web, but I wrote up my own version so I don't have to hunt it down or encounter dead links if/when I need the information again.

depot wishlist

• Part	Description	P/N	Qty.	Unit Price	Total
  raidz2 pool drives	Western Digital Red WD30EFRX 3.0TB IntelliPower 64M cache SATA 6.0Gb/s	WD30EFRX	6	$170.00	$1020.00
  120mm fans	Noctua NF-F12 PWM	NF-F12 PWM	3	$21.00	$63.00
  80mm fans	Noctua NF-R8	NF-R8	2	$15.00	$30.00
  SSD for ZIL			1	$0.00	$0.00
  SSD for L2ARC			1	$0.00	$0.00
  serial cable	USB to serial cable to connect to Powerware 5125 UPS		1	$0.00	$0.00
  Total					$1113.00

Diary

• May 29, 2024
  
  • depot is now running FreeBSD 13.3-STABLE.

• May 13, 2023
  
  • depot is now running FreeBSD 12.4-STABLE.

• October 20, 2019
  
  • depot has been updated to FreeBSD 11.3-STABLE.

• July 21, 2018
  
  • depot is now running FreeBSD 11.2-STABLE.

• December 19, 2017
  
  • depot (www) is now running FreeBSD 11.1-STABLE.

• June 14, 2017
  
  • depot (www) is now running FreeBSD 11.1-BETA1.

• January 8, 2017
  
  • depot is now running FreeBSD 10.3-STABLE.

• August 2, 2015
  
  • Depot's backup disk space is now a ZFS mirrored pool of two HGST 4TB drives (one Deskstar, one Deskstar NAS).

• July 26, 2015
  
  • depot is now my web server, and has been renamed.

• July 25, 2015
  
  • The backup of /backups finished overnight. I recreated the partition on ada0 using gpart and gave it a label of backup1fs. I started the restore over the network from ria. It finished in less than 2 hours.

• July 24, 2015
  
  • I migrated the operating system of depot to a Samsung 840 Pro SSD I had previously used in my hackintosh. I used GPT partitioning and labelled all of the partitions. It's up and running on the SSD, and I'm now in the process of backing up the 4TB HGST drive (which only hosts backups) over the network to ria so I can repartition it and give it a gpart label. This will let me move it around in the hot-swap bays without complications. With an IPMI-enabled machine this isn't an issue since I can get at the console remotely, but depot doesn't have IPMI.

    Thanks to Warren Block for his concise write-up on partitioning with GPT at http://www.wonkity.com/~wblock/docs/html/disksetup.html

• May 9, 2015
  
  • I installed an HGST 7K4000 4TB drive in depot to temporarily host TimeMachine backups of my hackintosh. Eventually I'll add more 4TB drives and use ZFS, but I needed something to replace the backups that were going to ria but haven't worked for 4 months due to insufficient space. The disk that was previously hosting hackintosh backups on ria is now hosting backups of depot's operating system (/, /var and /usr).

• December 7, 2014
  
  • I am updating depot to FreeBSD 10.1-STABLE as of today. It was previously running 10.1-PRERELEASE.

• November 9, 2014
  
  • I updated depot to FreeBSD 10-STABLE. I need to finally start putting it into service since I need to upgrade my firewall and web server. Both need more memory, and unfortunately that means new motherboards and processors. :-(

    I'm still working on installing the ports I need on depot.

• December 8, 2012
  
  • I decided I wanted to run my sitetrafficd daemon on depot to track traffic to and from it. In the process, I remembered that sitetrafficd assumed that the configured pcap filter always specified TCP-only traffic. So I fixed that issue, it will now happily record other IP traffic. So now it assumes that the pcap filter only allows IP packets, and that's fine; I don't really care about non-IP traffic.

    In the process, I discovered that GetLocalInterfaces() in my libDwm library did not work correctly on FreeBSD 9.1. So I had to fix it. It now works correctly and sitetrafficd is now running on depot, recording IP traffic totals and TCP handshake round-trip times.

• December 7, 2012
  
  • I decided I wanted to run my sitehealthd daemon on depot, to record historical disk usage, CPU utilization and CPU temperatures. It's running now. I'll worry about the front end later.

• December 5, 2012
  
  • I configured and started upsmon (as a client for now).

• December 2, 2012
  
  • I inserted an old WD 74GB VelociRaptor drive in the first hot-swap bay that's connected to the IBM M1015. The system recognized it correctly:

            da0 at mps0 bus 0 scbus0 target 0 lun 0
            da0:  Fixed Direct Access SCSI-5 device
            da0: 150.000MB/s transfers
            da0: Command Queueing enabled
            da0: 70911MB (145226112 512 byte sectors: 255H 63S/T 9039C)
            

    I then issued camcontrol stop da0 to stop the drive, and removed the drive. This also went fine:

            mps0: mpssas_alloc_tm freezing simq
            mps0: mpssas_remove_complete on handle 0x0009, IOCStatus= 0x0
            mps0: mpssas_free_tm releasing simq
            (da0:mps0:0:0:0): lost device - 0 outstanding, 1 refs
            (pass1:mps0:0:0:0): passdevgonecb: devfs entry is gone
            (da0:mps0:0:0:0): removing device entry
            

    Hence I believe I'm all set for adding drives to be used with ZFS.

    I installed smartmontools from ports, and enabled smartd in /etc/rc.conf. I configured the disks I want monitored in /etc/periodic.conf. I then started smartd.

• November 30, 2012
  
  • I finished modifying the handles on the Rosewill RSV-L4411 case to allow the case to be fully inserted into the rack. Since all I had for the job was my handheld jigsaw and Dremel, It's not pretty. But it's functional and this part of the handles is not visible when the case is in the rack.

    The SFF-8087 to SATA forward breakout cables arrived. I installed them and ziptied them neatly around the perimeter of the case so they don't have an adverse effect on the air flow.

    I put depot's case top on, installed the modified handles and put depot back in the rack. I believe I'm ready for pool drives.

• November 29, 2012
  
  • The IBM M1015 card arrived today. My first attempt at installing it failed, I think because FreeBSD attached the mfi driver to it. I wound up with a lot of 'mfi0: TIMEOUT...' messages and the kernel loops forever emitting them.

    I wound up flashing the IT sofware to the IBM M1015 by temporarily installing it in the motherboard of my hackintosh. I used FreeDOS and the sas2flsh utility, using 2108it.bin and mpt2sas2.rom as input files for sas2flsh. I had to do this because depot's motherboard has UEFI BIOS (and hence sas2flsh would not work on it). I could not get the EFI shell to work, and hence could not use sas2flsh.efi. Once flashed, I put it back in depot, and as near as I can tell, it's all OK. From dmesg:

            mps0:  port 0xe000-0xe0ff mem 0xf7dc0000-0xf7dc3fff,
            0xf7d80000-0xf7dbffff irq 16 at device 0.0 on pci1
            mps0: Firmware: 09.00.00.00, Driver: 14.00.00.01-fbsd
            mps0: IOCCapabilities: 1285c
            

    And camcontrol sees it:

            dwm@depot:/home/dwm% sudo camcontrol devlist -v
            scbus0 on mps0 bus 0:
            ...
            

    I have a plan to modify the handles on the RSV-L4411 case so I can get it fully into the rack on the rails without conflict. I modified one of them and it works.

• November 28, 2012
  
  • The Rosewill RSV-L4411 case arrived for depot. I installed the inner part of the rails on it, and adjusted the outer section of the rails in my rack to allow for the 1.125" that the case handles consume behind the front plate. I'll test the fitment soon.

    I removed the feet from the Rosewill RSV-L4411. I'm a bit surprised feet were included on a rackmount case, though I guess they'd be handy if I were using a shelf instead of sliding rails. I'll put them in the Middle Atlantic drawer that's installed in my rack in case I ever need them.

    I installed the power supply, motherboard and temporary boot drive in the Rosewill RSV-L4411. It appears that the IBM M1015 card will arrive tomorrow, at which point I'll install it in the motherboard and try to get it working. Amazon has not yet shipped my 1-meter SFF-8087 to SATA forward breakout cables yet, so I don't know if I'll receive them this week, despite it being a Prime order.

    Well, the Rosewill rack rails suck. The way the brackets are made, combined with the design of the handles on the RSV-L4411, means the case can't go all the way in to be flush with the rack and the rack screws will not reach. I'll have to modify things to make it work. Makes me wish I had just saved up for a Supermicro 3U case.

• November 27, 2012
  
  • I ordered an IBM M1015 card from eBay for $79.99 plus tax and shipping. This card is an SAS2008 card, with a pair of SFF-8087 connectors for up to 8 SATA drives. I'm hoping it has IT firmware loaded, but if not I will reflash it.

    I need cables for the IBM M1015 card. I don't think 18" cables will suffice for a super-clean installation in the Rosewill RSV-L4411 case, since the case is 25" deep. So I ordered a pair of 1-meter 3Ware CBL-SFF8087OCF-10M cables from Amazon. Once the IBM M1015 card and the cables arrive, I can finish the internal work on depot and hence be ready to install drives into the hot-swap bays. I may pull some of the 1TB drives from other machines and create a small zpool to test things. But my first production pool will be 6 drives in raidz2. I will later probably add a second 6-drive raidz2 pool, and use a Samsung 840 Pro Series SSD as my boot drive.

    It's worth noting that the critical I/O bottleneck for this machine will be the gigabit ethernet. It won't be doing anything itself other than acting as a network storage server, mostly for backups of all of my other machines and maybe for audio and video storage.

• November 26, 2012
  
  • The case for depot should arrive on Wednesday.

• November 25, 2012
  
  • I installed the rack rails for depot in the Startech 25U rack. Given that the RSV-L4411 case's holes for rack rails are centered vertically, I wish the rails had it's uppermost and lowermost holes set to line up when the rail is mounted to straddle two rack spaces. No matter, the two center holes on each face work in this manner, and should work fine. I am disappointed, however, that the recessed washers were not included. Others have had the same complaint. In the end, this means that the server will not sit flush.

• November 21, 2012
  
  • NewEgg processed my RMA for the Rosewill RSV-L4411 case. They haven't shipped the replacement yet, it's in the 'packaging' state (whatever that means). Hopefully they get it right this time. I'm not sure, but it appears they may have given me a $30 discount for my trouble. That would be welcome news. I'll know once it's all posted to my credit card.

    I installed nut for UPS monitoring. Of course I haven't bought new batteries for either of my Powerware 5115 UPS units yet. So for now I'll just be using it in slave mode; depot will be plugged in to the Powerware 5125, whose master is www.

• November 19, 2012
  
  • In tne interest of being prepared for the arrival of depot's case, I put the stuck CPU cooler on the ASRock H77 Pro4-M motherboard. I then connected the power supply and the 600G VelociRaptor drive and am about to try installing FreeBSD 9.1-RC3 via the USB memstick image. I'm waiting for the dd to my USB thumb drive to finish... I wish I had bought a speedier USB thumb drive for this kind of work.

    The motherboard has UEFI BIOS, and it looks like it's very speedy. That'll be nice for the occasions where I need to reboot. It will let me boot from a USB thumb drive (I checked), and it sees all of the RAM modules so at least I know they're all good as far as being recognized. The stock CPU cooler runs very quiet when the CPU is idle, which is good news. The power supply is very quiet at idle too. Not that it matters... I know the real noise from this machine will emanate from the hard drives and case fans.

    The dd of the memstick image completed, and I put the USB thumb drive in one of depot's rear ports. It booted fine and I performed the installation of FreeBSD 9.1-RC3. Everything went smoothly. Note that I consider this whole activity temporary... the 600G VelociRaptor drive is much larger than I need for a boot drive. But this is a good test of running a boot drive from the ASMedia SATA ports so that I can use the H77 SATA ports for raidz2 drives.

    I created a new kernel configuration (/sys/amd64/conf/depot) which just adds device coretemp to the GENERIC kernel. I want to be able to monitor CPU core temperatures.

    So, I'm all set to do some testing once the new case arrives. Of course I don't have array drives yet. I'll be buying those piecemeal over the next few months.

• November 15, 2012
  
  • I dropped off the Rosewill RSV-L4500 4U case at the UPS Store for return to NewEgg. Tracking number is 1Z02319W9099161979. Hopefully it'll get there next week (estimated arrival at NewEgg is Wednesday 11/21/2012), and hopefully the correct product will arrive the following week.

    The ASRock H77 Pro4-M motherboard, 32G of DDR3 1333 memory, the Seasonic SS12II 430B power supply (430 watts), and a Western Digital WD6000HLHX 600G VelociRaptor hard drive arrived. All of these parts except the hard drive are for depot.

    I installed the CPU and the memory in the ASRock H77 Pro4-M motherboard. I have not yet installed the CPU cooler, only because I'm not sure I'll use the stock one and the motherboard won't fit back in its box with the CPU cooler on it. No sense in installing the cooler until I have a case to put it in.

    The StarTech 25U rack arrived via freight shipping today, and it appears to be intact. I'll know when I take it all out of the box and try to put it together.

    I quickly put together the StarTech 25U rack. It's all good. The only indication that it's refurbished: a few paint scratches and chips on some of the aluminum braces. It's a non-issue for me since this isn't a piece of furniture, it's a tool. It's not going to be placed in living space, but it looks very good. More importantly, it's much more functional for my purposes than my existing full-height racks. The one drawback: I can't see moving it out of the home on the casters when it's loaded... they're not strong enough. It's also worth noting that the first rack space is not at the floor of the rack, it's about 1.5" above it. That means I probably can't use my Powerware 5125 on the floor of the rack with no rails. I'm calling this a non-issue since I have other options.

    .

    I've decided that I want the CSB HRL634WF2 batteries from atbatt.com for my Powerware 5115 UPS units. Free shipping for orders over $75, and these particular batteries are 9Ah and designed for long-life and 260 full cycles.

• November 14, 2012
  
  • The CPU for depot arrived today. So did a Rosewill case, but it's the wrong one. :-( Newegg sent a Rosewill RSV-L4500, but I ordered an RSV-L4411. They are very different products, with very different price tags. The RSV-L4500 is about half the price and has no hot-swap drive bays. Hence I requested an RMA and a pre-paid shipping label since the fault was solely on their side. Sad this kind of mistake is made, especially when the model number of the case is written clearly in several locations on the outside of outer box (clearly visible). In fact I knew it was the wrong product without opening anything, but I opened it anyway to see if maybe they had repackaged. Unfortunately this is not the first time I've had a Newegg screw-up. I should probably start taking my business elsewhere as frequently as possible, just to avoid the headaches. Their prices are generally competitive, but historically they've made more mistakes than Amazon, CDW, TigerDirect, directron.com and others. What's annoying about this kind of issue: it sets me back several weeks, which pushes things into the holiday season. My time is very valuable, I don't like having to spend it talking to newegg to make them pay for return shipping (their on-line process wanted me to spend $60 of my own money for return shipping), waiting for them to send a shipping label via email, printing and attaching shipping labels and RMA forms and then hauling a very large box to the UPS Store just because newegg doesn't seem to be able to manage shipping the correct product. If I had been smart, I would have ordered a SuperMicro case from somewhere else (probably eBay). In the end I might wind up with SuperMicro drive cages anyway, depending on the quality of the Rosewill cages.

• November 12, 2012
  
  • I'm expecting the CPU for depot to arrive on Wednesday. Other parts should arrive on Thursday. On Thursday night I'll see if the case fits in 4U with the rack rails installed. I read a review that complained that the case doesn't really fit in 4U with the rails installed, since the rails are centered on the case. That may have been the particular user's issue with a particular rack. I'm hoping I don't have that problem, since I'd like to have more than 3U of space remaining in the new rack after installing depot. Of course, there's no particular reason to keep the tower cases I'm using now for ria and www; those could easily be 2U or 3U rackmount cases. There are a couple of iStarUSA 3U cases that I like that are not terribly expensive and have good cooling setups. Getting good cooling in a 2U case winds up being fairly expensive, and I'm well aware of the noise created as a result since I previously had a pair of 2U cases in my rack with dual Xeon motherboards.

• November 11, 2012
  
  • I downloaded memstick images of FreeBSD 9.0 and FreeBSD 9.1-RC2. FreeBSD 9.1-RELEASE is behind schedule, it was supposed to be released a couple of days ago. I suspect we won't see it until December.

    I'm leaning toward an IBM M1015 card for ZFS. It will give me 8 SATA ports, and is readily available on eBay for cheap. To reflash the card to IT-mode, I can use the instructions here: http://lime-technology.com/forum/index.php?topic=12767.msg124393#msg124393

• November 10, 2012
  
  • I ordered enough parts to start the build of depot. A Rosewill RSV-L4411 case (12 hot-swap bays), the Rosewill RSV-R26LX rack rail kit, an ASRock H77 Pro4-M motherboard, an Intel i5-2405S CPU, 32G of DDR3 1333 memory, a Seasonic SS12II 430B power supply (430 watts), and a Western Digital WD6000HLHX 600G VelociRaptor hard drive. The hard drive is overkill, and will probably be used in my hackintosh instead, but it was cheap since it was part of a bundle deal with the Rosewill RSV-L4411 case. I chose the i5-2405S for its 65W TDP, to avoid too much heat in the case while still having 4 cores and integrated graphics. The ASRock motherboard isn't my ideal choice, but it has VGA output and a PS/2 port for keyboard, both of which I need for my existing KVM. Sadly, it doens't have any PCI xPress 2.0 x8 slots, but since I'm not using a graphics card, I'm hoping I can put a SATA controller card in the PCI 3.0 x16 slot. I have yet to decide which card I want. I don't need or want RAID, since I'll be using ZFS with raidz2. I've also not yet decided which drives I'll be using. WD Red makes sense, except that their warranty is only 3 years. There's also the question of size; I need 2TB drives at a minimum, but can I use 3TB drives? This is really a question of space (will the 3TB drives fit in the hot-swap trays), and will it mean waiting too long for resilvering and scrubbing?

    Once the new rack and server parts arrive, I'll basically be set for the next several years with respect to home computing, with the exception of an HTPC. I'll probably buy a Mac mini for my HTPC.