Niall’s virtual diary archives – Sunday 26th August 2018

Sunday 26th August 2018: 12.44pm. I've been very slowly, bit by bit, doing a little bit of work when home each weekend to replace the main house server, whose install of Debian EOLed last June.

Right now my ZFS pool is kept within a VM running FreeNAS, as back when I set this up I didn't trust ZFS on Linux (for good reason). Now I do, so I've set up a mirrored ZFS pair of those cheap shingled SMR 8Tb Seagate drives, and I set the old pool rsyncing to the new pool last night.

Those shingled Seagate drives have a 20Gb PMR acceleration cache, and writes of up to that much go quickly, indeed for random 4Kb i/o these drives have amazing benchmarks for spinning rust, because the drive ignores the randomness and just writes your random i/o into sequential blocks in the PMR cache. So, perversely, for small bursts of random writes, these drives outperform any spinning rust drive out there by orders of magnitude.

However, once the PMR cache fills, then writes must block on SMR, which is dog slow. I knew from conveyancing tests for each of these drives that we are really talking slow here, it takes nearly a week to run badblocks on them. But I had no idea until now how slow ZFS would be on these, or more specifically, if one of these 8Tb drives fails, how long does it take to resilver a replacement?

I can tell you now that ZFS is writing to each of these SMR drives at about 11.3Mb/sec! So it'll take approximately 7.52 days to resilver one of these 8Tb drives. That's probably an overestimate, my new server is running inside a VM on the current server, and rsync is running over two virtualised network interfaces, so there's going to be overhead there. Still though, that's an order of magnitude slower than a PMR drive. If the single remaining drive failed during resilver, game over for all your data.

Are SMR 8Tb drives therefore worth it? If you buy the Seagate SMR drive in its Backup Hub Pro USB case, it costs €170 or so (just extract the drive from its case, it's far cheaper than buying these drives loose). The next cheapest 8Tb PMR drive I could find is a Western Digital Purple which is currently selling for €240, which is 40% more expensive.

I am half tempted to triple mirror these SMR drives, that reduces chance of pool loss to nothing. But I am also minded that Linux lets me stitch together drives easily, so I can create a faked 8Tb drive from 3x 3Tb drives which I have spare, and thus guard against failure during a SMR drive resilver. In other words, I can buy that week through more work by me at the time. So, for now, onwards and upwards!