A note for everyone: use Hammer default on a very busy filesystem, and you will eat a lot of disk space since all file changes are recorded. (I’ve done this to myself a few times.) Francois Tigeot has a list of tips on how to keep that from happening.
Are you running a Hammer filesystem on a low-memory system? You may get some warnings. It’s possible to tweak some settings to accommodate it, or just deal.
The May BSD Magazine is out, and Siju George has written an article about using Hammer on DragonFly. It’s a free download to read.
(link fixed)
The March issue of BSD Magazine is out, and this month has an article written by Siju George about how his company is using DragonFly and Hammer for backups.
Matthew Dillon committed the start of a Hammer 2 cluster API. I noticed, while looking at the commit, that there’s a design document, a freemap design document, a changes list, and – most important for anyone interested – a TODO list.
Hammer’s ability to stream to remote disks is great, but what if you have storage that uses some other file system? Antonio Huete Jimenez put together a shell script that will dump out the contents of a Hammer PFS, for upload to whatever. Read the README for the details.
If you have a Hammer volume that is offline, meaning that you don’t have the pseudo-file-systems null-mounted anywhere, it won’t get cleaned up in overnight processing. You just have to manually specify it.
This is a little old, but Matthew Dillon noted the status of his Hammer2 work a little while ago. Some highlights: he’s intending Hammer2 to be usable on a single host by the time of the next DragonFly release (summer 2014), the Summer of Code project for compression has already been integrated, and he listed different parts of the work that may be interesting for anyone wanting to chip in.
Slightly related: Matt posted some Hammer2 comments on the DragonFly 3.6 release story on Slashdot that may be interesting. Don’t bother reading the other comments; they’ll make your eyeballs bleed.
If you want to boot from a Hammer 2 /boot volume, you now can. Hammer 1 never worked well as /boot, though it was technically possible. Hammer 2 will be just fine.
Note that you can’t turn on recently-added disk compression since the bootloader doesn’t understand it, and Hammer 2 is not ready for anything but being worked on. Don’t try it unless you’re ready to be submitting code changes to fix Hammer2.
Remember my recent disk issues? As a side effect of protecting myself, I have a good example of deduplication results.
I have a second disk in my server, with slave Hammer PFSs to match what’s on my main disk. I hadn’t put them in fstab, so they weren’t getting mounted and updated. I got them re-created, but they were nearly full. Here’s an abbreviated df, from which you should be able to tell which drives I have :
Size Used Avail Capacity 929G 729G 200G 78% /slave/slavehome 929G 729G 200G 78% /slave/slavevar 929G 729G 200G 78% /slave/slaveusr 929G 729G 200G 78% /slave/slaveslash
That 78% is how full the Hammer volume was. I turned on Hammer deduplication, since it’s off by default. The very next day:
Size Used Avail Capacity 929G 612G 318G 66% /slave/slavehome 929G 612G 318G 66% /slave/slavevar 929G 612G 318G 66% /slave/slaveusr 929G 612G 318G 66% /slave/slaveslash
It’s a 1 terabyte disk, and I gained more than 10% back – That’s 100g of disk space that I gained overnight. There might be more tomorrow, given that it was all of 5 minutes of dedup work.
This won’t surprise you if you’ve seen previous deduplication links here, like my previous results or some real-world tests. It’s still great. I’d suggest turning it on if you haven’t – hammer viconfig the appropriate PFS and uncomment the dedup line.
Something new and odd: A port of the Hammer (1) filesystem into Go, for go-fuse. As the author has said, it’s more for the practice of learning Go and Hammer than for producing anything useful. Still, an interesting way to learn.
Encryption seems to be the accidental theme tonight. A question about Hammer 2 and encryption prompted this list of possible solutions from Matthew Dillon. Hammer 2 is still months out, so these features both require time and someone interesting in doing them – though they sound quite possible.
Still not sure if I should be writing Hammer or HAMMER.
If you were wanting to encrypt your /home directory, Pierre Abbat has written up the explicit steps he took to do that very thing.
Earlier this week, Daniel Flores posted the first-week report on his Google Summer of Code project, file compression in Hammer. He mentioned that the LZ4 algorithm he is using seems to have the best performance with repeating text data, as in logfiles. I asked for numbers, and he provided them. The important data in the results is the total compression column. It shows how many 64k blocks were able to be compressed, with that type of data.
John Marino managed to update GCC from 4.7.2 to 4.7.3 (4.7 changelog), zlib from 1.2.7 to 1.2.8 (changelog), and awk from 20110810 to 20121220 (can’t find a changelog).
In other update news, Matt Dillon has been working on HAMMER2’s flush sequencing.
Update: tcsh too.
If you’ve ever wondered about how you can resize/move a HAMMER filesystem, follow this thread for a variety of answers.
If you’ve been feeling the need for reading about filesystems, Daniel Phillips has posted more notes about his Tux3 filesystem design, which can be contrasted with HAMMER. (thanks, Venkatesh Srinivas)
Matthew Dillon has written up another update on his progress with HAMMER2. (I need to be consistent in how I write that.) He has disks being exported and mounted on other systems, and adds an explanation of some of the issues around creating reliable multi-master setups. Before you get too excited, no, multi-master isn’t working yet, and this is not production ready.
Matthew Dillon’s put more of his Hammer work into DragonFly, with notable parts being the creation of a ‘dmsg’ setup for advertising available block devices to share between machines using Hammer. To anticipate your next question: No, it’s not something you can run right now as a test; this is the underlying framework.