I have a huge backlog of things to post, so this is originating from the 17th: Matthew Dillon has been working for some time on hardlinks and Hammer 2. Hardlinks are the same file, presented in multiple places. This can be a problem when your filesystem keeps infinite, writable snapshots. The solution he just commited is called ‘xlink’ and the commit message has details.
The disk scheduler apparatus in DragonFly has been removed. This may not affect you much, since alternate scheduling setups were never utilized much with it. It may fix some rare Hammer cleanup issues, though, and you may need to adjust your custom kernel config, if you have one.
HAMMER2 recently gained the ability to be used as the root mount for your DragonFly system. Live deduplication of data is also now possible, which means fast copy operations, less space used, and no need to wait for an overnight batch process to do it. If you want to try it, you need a bleeding edge DragonFly system and the WANT_HAMMER2 option. It’s still not ready for production use, so don’t try it with any data you want to keep.
I don’t note it enough, but Tomohiro Kusumi has been making constant updates to HAMMER, the version we have now. Often they are the sort of update that makes the code more readable, or fixes possible problems, and so on. Very essential, but hard to post about it. In any case, I’m using his recent improvements to hammer volume-del to note his contributions, of which there are much more than the day’s worth I link here.
Hammer 2 now uses LZ4 compression by default. It also uses a new CRC algorithm that performs much better, and there’s numbers to prove it. It helps iSCSI too. When I say new, it appears to be from the 1980s? I may be looking at the wrong place.
Matthew Dillon has been doing a lot of Hammer 2 work lately. Well, he’s been doing it for quite some time, but the recent commits contain the sort of things that are easier to link to, like deletion speedups, freemap changes, and stats tracking/compression results.
Hammer will perform daily housekeeping tasks each night. If you’re up late enough, it may kick off while you are working. If you want to stop the process after it’s already started (since it’s disk-intensive), John Marino has added the ‘abort-cleanup‘ command.
DragonFly committer Joris Giovannangeli has a Google Summer of Code project. He’s bringing Hammer2 to OpenBSD, in single-node form. It’s a very difficult project, but Joris is a very talented worker.
Tomohiro Kusumi has been quietly making a lot of commits to Hammer. I haven’t been linking them because they don’t necessarily equate to new features, but here’s an recent exception: the -A argument will make your Hammer command run on every PFS. It only affects reblocking/rebalancing – for now.
You can now export Hammer slave volumes as NFS mounts – but since slave volumes are updated from master, you’re mounting a snapshot of that point in time. That may actually be an advantage.
I haven’t been drawing enough attention to it, but there’s been a bunch of HAMMER filesystem activity lately: First, Tomohiro Kusumi has been working on HAMMER – these posts are a small subset of his commits. Second, Matthew Dillon has been working full steam ahead on HAMMER2. The HAMMER2 design document has been updated (read this!), and he’s already accomplished master->slave disk syncing.
It’s not ready for production, of course, which you may already realize, so don’t install it unless you want to work on the code.
This week’s BSDNow has the usual roundup of news, including some… suspicious items, plus an interview of Kamila Soucková about conferences and Google Summer of Code. They note this Hammer2 proposal.
Matthew Dillon answered some mailing list questions on how clustering and data copies will work in HAMMER2 – no due date, of course, because this is very complex. If you’re really into it, there’s always watching the recent commits.
If you’re monitoring your DragonFly systems with Nagios, here’s a way to check the health of your Hammer mirror-streams. Thanks, Mike!
Here’s a number of DragonFly links to clear out my backlog:
- Peter Avalos has updated OpenSSH in DragonFly to version 6.7.
- He updated file, too.
- Bill Yuan’s ipfw branch has been updated.
- Matthew Dillon’s been making more Hammer2 commits; check the TODO for status.
- Michael Neumann has Rust working on DragonFly.
- Filesystem encryption on Hammer.
There’s a FreeBSD Forums thread about ZFS and Hammer, as several people have pointed out to me. It’s interesting to see, but there isn’t a lot of quantitative discussion. (It’s a forum post, not a white paper, though.)
John Marino has created something very useful: a graphical tool for Hammer file history. It’s called ‘Slider’, and it uses curses to work in a terminal. It shows historic versions of files and can restore those old versions as needed. This was already possible in Hammer, of course, but it required a sequence of commands that were not straight-forward. I’ve been slow enough posting it that version 2.0 is already out, offering a way to see files that no longer exist, but are still in history. (i.e. deleted some time ago) ‘Time Machine’ sounds like the best name, but that seems to be taken.
One way to keep file history on an very active Hammer disk from eating up all the space: more snapshots. This may seem counterproductive, but disk pruning eliminates historical data between snapshots, so you can keep older data at the cost of some temporal accuracy.
Not the first time that pun has been deployed, of course, but: BSDNow 053 has a Hammer tutorial, along with an interview of Reyk Floeter about OpenBSD’s httpd, along with the usual recent news and some links to some things I haven’t seen, like pfSense tricks.
.
A frequent question people ask when trying Hammer is “How can I do software RAID to cover a disk failure?” Hammer provides for streaming one volume to another, so you can duplicate drives, but there isn’t an automatic failover mechanism as there is with a RAID setup. The first answer is usually “get hardware RAID“; my preferred solution. The remaining software solutions are vinum, ccd, and lvm for DragonFly.