As anyone who has been running HAMMER1 or HAMMER2 has noticed, snapshots and copy on write and infinite history can eat a lot of disk space, even if the actual file volume isn’t changing much. There’s now an ‘emergency mode‘ for HAMMER2, where disk operations can happen even if there isn’t space for the normal history activity. It’s dangerous, in that the normal protections against data loss if power is cut go away, and snapshots created while in this mode will be mangled. So definitely don’t leave it on!
If you have a whole lot of I/O on a HAMMER2 system, this change will help. This is I assume an outgrowth of dsynth testing, cause that causes many, many threads to be reading and writing.
HAMMER2 is Copy on Write, meaning changes are made to copies of existing data. This means operations are generally atomic and can survive a power outage, etc. (You should read up on it!) However, there’s now a fsck command, useful if you want a report of data validity rather than any manual repair process.
This slipped in just before the 5.6 release, and I thought I had already noted it: DragonFly now defaults to HAMMER2 for disks during install, instead of HAMMER1.
Matthew Dillon’s committed some performance work for HAMMER2, dealing with write-clustering. I don’t have statistics to note, so here’s the commit message.
It’s always helpful to see other people’s questions and answers; in this case it’s a conversation about HAMMER2 snapshots and how to manage them. (Follow the thread)
It’s possible to have data corrupted on a HAMMER2 volume during a specific combination of a bulkfree operation and a lot of writing to disk. Matthew Dillon has a potential fix already. As he announced, it’s scheduled to go into 5.4 this weekend. It’s a rare bug, but if you want to check for it, look for CHECK FAIL entries in /var/log/messages.
And because every cloud has a silver lining: some not-yet-quantified performance improvements.
This week’s BSD Now covers assembly on OpenBSD, games on FreeBSD, and disk space on DragonFly.
For future edification: If you have HAMMER2 installed, the bulkfree operation will create console/dmesg activity even when nothing is wrong, to show operations are happening.
Matthew Dillon recently fixed a TRIM bug, where a TRIM command was being issued unconditionally, regardless of the mount flag, and duplicating the action if it was set normally. It’s fixed now. This would only have any significant slowdown on UFS, which means it would only affect installworld – the rest of your mounted volumes are HAMMER, right?
As part of a larger conversation about HAMMER, Matthew Dillon noted that he is planning to work on master-to-multiple-slave for HAMMER2, which would function similar to HAMMER1 mirror-stream.
Remember the upgrade for dragonflybsd.org machines? It completed, and it’s interesting to see that SSDs have become so easily available that “spinning rust” hard disk drives are only still useful for bulk storage, and even then probably not for much longer.
Another neat side effect: disk usage on developer system leaf.dragonflybsd.org was cut in half, thanks to HAMMER2 dedup/compression. It’s a ‘free’ half-terabyte.
These would be ‘In Other BSD’ links, but this isn’t Other BSD – It’s DragonFly:
- Towards a HAMMER1 master/slave encrypted setup with LUKS.
- Intro, Installation, and Fun with Hammer2.
I’m pulling a quote off of IRC to show some of the testing on HAMMER2, specifically as the background for this commit:
14:22 <@dillon_> ^^^ hammer2 bug, could reproduce it around once a day doing a continuous rm -rf on hardlinked snapshots. reproduced about once every 500 million directory entries or so
I am somewhat tickled by the notion that you might have a problem after deleting half a billion directory entries.
The regular maintenance scripts for HAMMER1 assume that it’s mounted at the time of cleanup. If you have them unmounted, they won’t go through that regular maintenance, but it’s easy enough to fix.