As I mentioned on kernel@, I’m going to roll a point release of DragonFly soon. Push in your changes if you want to get them in!
Antonio Huete put together a list of goals for the next release on the DragonFly bugtracker. Some of them are pretty ambitious, some of them are relatively easy, but they are all very useful.
Probably because of the C-state changes, Sepherosa Ziehau wants people to use a new set of sysctls instead of the hw.cpu_mwait* ones – at least on x86_64. This won’t affect you if you aren’t already familiar with them, probably.
Recent updates to tzcode apparently fixed a long-standing time zone bug in DragonFly. POSIX says the America/New_York timezone is picked as default if nothing else has been selected. That didn’t happen in DragonFly – until recently. If your timezone seemed to suddenly jump to U.S. Eastern time, that’s because you never picked before.
If you want to test out the latest (20131218) update to ACPICA, Sepherosa Ziehau’s got a patch for you.
This will be good for anyone who wants to use less electricity. (updated to reflect this doesn’t enable deeper C-states as I thought it did.)
This post from Konrad Neuwirth asking how to do a minimal installation of DragonFly led to this list of all the ‘knobs’ you can set to make your installation smaller, from John Marino. (And your buildworld faster, if that’s appealing to you.) I also pointed at rconfig and PFI, which are criminally underdocumented.
pkg 1.2 is coming out. This brings a number of new features, but as John Marino posted, you may want to delete your old pkg.conf to keep the new version from complaining about an old config file. This upgrade is a step on the way to signed packages, which is a Good Idea.
John Marino posted a possible ‘roadmap’ for DragonFly, now that we’re past the 3.6 release. The thread went on for some ways as it was discussed, including my crazy ideas. Notably, several suggested items have already been tackled – an iwn(4) upgrade has already happened, and an update to bmake, based on John’s vendor branch update instructions.
This is a little old, but Matthew Dillon noted the status of his Hammer2 work a little while ago. Some highlights: he’s intending Hammer2 to be usable on a single host by the time of the next DragonFly release (summer 2014), the Summer of Code project for compression has already been integrated, and he listed different parts of the work that may be interesting for anyone wanting to chip in.
Slightly related: Matt posted some Hammer2 comments on the DragonFly 3.6 release story on Slashdot that may be interesting. Don’t bother reading the other comments; they’ll make your eyeballs bleed.
Eitan Adler is the newest DragonFly committer; you may recognize his name from some previous commits added by others, where he synced up various work between the BSDs.
For those updating from 3.4 to 3.6: there’s an ABI change, so you will have to upgrade all your packages. If you’re using pkgsrc and ready to switch to dports, now’s the time. If you already switched to dports on your 3.4 system, binary packages for 3.6 have already been built and you can use pkg to upgrade.
Also for upgrades from 3.4: You can pull the 3.6 source normally:
git fetch origin
git branch DragonFly_RELEASE_3_6 origin/DragonFly_RELEASE_3_6
git checkout DragonFly_RELEASE_3_6
But there’s a slight change needed for the 3.4 to 3.6 transition: an extra reboot in the build process:
# make buildworld && make buildkernel && make installkernel && make installworld && reboot
# make upgrade
This is all noted in /usr/src/UPDATING and in the release notes, but I’m taking no chances.
As noted on the kernel@ list, it’s tagged but not yet in image form.
Matthew Dillon did some more performance tuning for DragonFly. I’ll just pull a paragraph from the commit message, since that will have more impact than anything I say:
Improves fork/exec concurrency on monster of static binaries from 14200/sec to 55000/sec+. For dynamic binaries improve from around 2500/sec to 9000/sec or so (48 cores fork/exec’ing different dynamic binaries). For the same dynamic binary it’s more around 5000/sec or so.
“monster” is a 48-core machine used for testing.
The venerable (from 1979!) program, lpr, has been superseded by CUPS in many installations. Francois Tigeot suggested removing it, but it’s still directly usable in specific situations and easier to just shift out of the way. It’s staying, but it’s interesting to see how it still gets used.
Update: Predrag Punosevac has descriptions of the various tools involved.
I’m planning to branch DragonFly 3.6 this weekend. The actual release will come 2 weeks later. (Ignore what I wrote about a dports installer/image.)
There is a search plugin for Mozilla that searches DragonFly man pages. (Thanks Samuel Greear)
I got some PC-BSD items this week, too.
- Open Source Snapshot: GhostBSD.
- (Free)BSD and Dropbox.
- FreeBSD finally dumped rcs.
- FreeBSD’s igb(4) driver is updated to 2.4.0.
- FreeBSD’s binutils now has “support for assembling and disassembling Intel Random Number Generator extensions“.
- You can now use ‘athsurvey’ on AR5212 chipset ath(4) devices in FreeBSD.
- FreeBSD branched version 11.
- FreeBSD has changes contributed by… Microsoft?
- PC-BSD has added a GUI version of their Life Preserver application.
- PC-BSD has a new ‘pc-zmanager’ program for managing ZFS and disks.
- PC-BSD has branched version 10, I think.
- NetBSD runs on the iMX233/OLinuXino.
- OpenBSD replaced rc4 with ChaCha20. No, I’m not sure what that means. (via)
- OpenBSD now has the vmwpvs(4) driver, for VMWare paravirtualized SCSI.
- OpenBSD has imported Mesa 9.2.1 and Freetype 22.214.171.124.
- OpenBSD supports the AM335x EDMA3 controller.
- OpenBSD supports the RTL8106E and RTL8168G/8111G networking chipsets.
- Diffe-Hellman key size increased in OpenBSD. It’s from NIST Special Publication 800-57, which is unavailable as of this typing because of the stupid U.S. government shutdown.
The pkgsrc repository in git for DragonFly is currently frozen. This is because many people have switched over to dports, and also because it’s a lot of work to keep it functional. If you do want to pull newer pkgsrc material, use cvs and grab it from a NetBSD server.
As the message notes, don’t go switching to DragonFly-current right now, cause there’s a lot of new material in there and it may not be quite safe. (There’s an ABI change that will require all new builds of your ports, for instance.)
I put together a list of what I’m thinking could be in the next DragonFly release. Going by our regular schedule, that’s a bit more than a month off. Of note: Summer of Code material and defaulting to dports. Follow the thread for more.
ZFS was originally created at Sun and open sourced. Sun was absorbed by Oracle and stopped being open (or even really existing), so ZFS was taken up by several separate groups – FreeBSD and Illumos being two examples. OpenZFS has been announced, in part to provide common reference for other platforms that might implement it and probably to avoid capability fragmentation. It’s certainly a good idea.
(If I have my history wrong, please correct me.)
DragonFly has two included compilers – GCC 4.4, and GCC 4.7. Traditionally, we switch from one compiler to the other as default, and then replace the old one with a newer release, and so on.
Until recently, dports built almost exclusively using GCC 4.4. John Marino’s switching to GCC 4.7, for a variety of reasons he lists in a recent post to users@. An interesting point that he raises: GCC 4.4 won’t necessarily be replaced with a newer GCC, but perhaps clang?
Sepherosa Ziehau has made a number of improvements to TCP in DragonFly – specifically, nonblocking and blocking connect(2) performance. See each of his commits for statistics on how much this has reduced processor use under high load. He has also written up an extensive description of how all this TCP stuff works in DragonFly.
In similar news, he has a nginx patch that delivers a significant performance increase. It may go into nginx itself.
avalon.dragonflybsd.org, also known as mirror-master, is the final dragonflybsd.org system to be moved into the new colocated blade server. Your downloads of binary packages or DragonFly images should be speedier.
I’ll be working on the 3.4.3 release of DragonFly within the next 24 hours, and it should be available this week. I’ll have a list of the bugfixes it contains…
The mailing list archives for DragonFly (lists.dragonflybsd.org) have been moved to new hardware. (Yay!) The patch that actually shows date in the listings needs to reapplied, cause Mailman is somewhat stale. (Boo!) I applied the patch and I’m regenerating all the archives now. (Yay!) There’s some garbled messages in the archives that cause a bunch of “no subject” partial messages to be dumped at the end. (Boo!) I’ll manually fix them if I can, someday. (Yay?)
Definitely Saturdays for this summary. In other BSDs this week:
- It’s FreeBSD, but it can apply to any BSD where a personal attribution license is used: Julian Elischer’s name comes with every iPhone.
- cxgbe(4) cards can now display their temperature under FreeBSD.
- ciss(4) supports additional HP RAID controllers under FreeBSD.
- Bind has been updated to 9.8.5-P2 in FreeBSD.
- FreeBSD has initial support for the Cubieboard 2.
- FreeBSD now has a USB test program.
- NetBSD supports some additional ZTE modem devices.
- NetBSD has cgram, a substitution-cipher solver. For amusement purposes?
- NetBSD supports the Nuvoton W83795G monitoring device.
- OpenBSD now supports wireless devices using the Ralink RT3060.
Wired has an article up about Jordan Hubbard and his move from Apple to iXsystems. It’s not a bad article, though it doesn’t delve into the why of BSD very much. In any case, iXsystems has been really bulking up lately to be more than a generic hardware provider.
Speaking of which, that blade system going in now for dragonflybsd.org was sold by iXsystems.
Joris Giovannangeli, one of the Summer of Code students for DragonFly, posted his thoughts on credential descriptors – have a read. He is working on capsicum and DragonFly, so this is a natural thought process.
Every year, people ask “Why can’t writing documentation be part of Summer of Code?” (Not necessarily for DragonFly, but in general) Google has a “Doc Camp”, where a whole lot of documentation gets produced in sprints, and anyone can participate – not just Summer of Code students.
If this sounds interesting to you, your application has to be in by August
7th 9th. (URL and date updated)
Please welcome our newest DragonFly committer: Johannes Hofmann. He earned this by coming up with a significant chunk of DragonFly’s upcoming KMS/915 support, and it’s now easier to just have him work directly than to be constantly committing for him.
Since there’s a newer set of dports binary packages uploaded, I thought I’d spend my weekend upgrading, to catch up.
And that was it. Well, not really. I had to dump and restore my Postgres databases, cause of the switch from 9.0 to 9.2 as default. I had to build php5 from source to get the Apache module. Those two things together took longer than the entire download and upgrade of the rest of my system – some ~200 packages?
Sepherosa Ziehau added SO_REUSEPORT to DragonFly. I don’t know how the mechanism works, because he didn’t include a description, but he did include a explanation of just how much it reduces CPU usage during as-high-as-physically-possible network load. He even wrote tools to test it more heavily.
Some of the links this week go pretty in-depth. Enjoy!
- This short story from 1954 might serve as a reason to avoid single system image computing… (via Sascha Wildner)
- Vim and Ctags tips and tricks. (via)
- Psygnosis game box designs. Nostalgia for some, neat art for anyone else. (via)
- 50 years of ASCII, and here’s the table it comes from. Some other neat links there, too. (via)
- Richard Feynman and The Connection Machine. If you like stories about Feynman, who was a very interesting person, you may want to read Feynman, the comic book. I met the writer, Jim Ottaviani, years ago, and he was very energetic about both science and comics. Look up his other work if that sounds interesting – which it should. Here’s a sample from the Feynman book. (via)
Back to the Future: Preserving the History of Video Games. This is right around the corner from me. The game museum is as neat as it sounds (yes, they have games out to play), but the article doesn’t mention that it’s attached to a fantastic and huge kid’s museum.
- Building a Cray at home. Similar to this previously-linked idea. (via)
Your unrelated link(s) of the week: Candy Box and A Dark Room. Both are text-only games, but they use HTML5 for animation. They start minimal, and build up – be patient; there’s a lot of gameplay in there. These minimal games fascinate me. It’s like reading a book, where it goes from just static text to an entire world being built. (somewhat via)
Your bonus unrelated comics link of the week: Jack Kirby double-page spreads. It’s not an exaggeration to say this artwork crackles. (via I forget)
Whoops, I missed this when it happened, but: the freeze for pkgsrc-2013Q2 has started. That new quarterly release is anticipated for the end of the month.
Phoronix has another set of benchmarks that include DragonFly and PC-BSD, along with several Linux distributions. It’s interesting to see, though don’t take them as performance measurements. 7-Zip as a benchmark doesn’t describe much other than the program itself, and the Himeno benchmark results are because of the compiler in use rather than any underlying performance aspect of the operating system – for instance. The DragonFly benchmarks disappear after page 3.
I’ve tagged DragonFly 3.4.2. The major reasons for this point release were fixes for DragonFly under Xen with more than 2 CPUs specified, and for booting x86_64 DragonFly in KVM. The 3.4.2 tagged commit has every detail.
If you’ve already got a working 3.4.1 installation, you don’t need to rush to upgrade; this is mostly for the people affected by the issues listed above. I’m working on 3.4.2 install images; give that some time to complete and upload if you need one.
I’ve put the 3.4 release images up on terasaur, a Bittorrent seeding site. Please try pulling them and let me know how it goes. I haven’t torrented many things, so I am unsure how to even verbify “torrent’. Hopefully that sentence and those links work out.
‘william opensource4you’ posted a summary of the steps he took for setting up a DragonFly system with XFCE4, using dports. It’s pretty straightforward, and thanks to dport’s binary nature, should be exactly reproducible.
As I described in a post to the kernel@ mailing list, the DragonFly 3.4 images are getting uploaded for mirroring and downloaded for testing. Assuming no surprises happen, we will be able to release very soon.
John Marino has a concise explanation of why dports mostly uses gcc 4.4 still to compile, even if you’re building DragonFly itself with the default 4.7. It’s a reason to not use NO_GCC44 – yet.
Eric Radman sent along a plug for a utility he is working on called entr(1). The desciption is “Run arbitrary commands when files change.” The site for it has several nifty examples – run make when *.c files change, or convert Markdown files to HTML as soon as they are modified. The really nice thing about it is that it’s perfectly BSD-friendly, and uses kqueue, but will also work on Linux. This beats the “This runs on the one flavor of Linux I use, in one particular shell!” approach I’ve seen from some other developers. See the reddit discussion of it for comparisons to inotify.
No, it’s not in pkgsrc/ports yet.
NASA’s International Space Apps Challenge is this weekend, 4/20/2013. Fancy as it sounds, it’s really a single-day hackathon around open software and hardware, with the problems to fix coming from NASA and therefore probably very unique. It’s happening in a bunch of places around the world, but there’s one right here in my town.
Peter Hansteen has an extensive writeup of how he has managed the bsdly.net spam blacklists. Normally I’d stick this article in the Lazy Reading links, but the article is good enough to call out separately. It’s excellent not just for the mechanical aspects of how the blacklists were maintained, but for his strict description on how the process is simple, verifiable, and transparent. That last item, transparency, is how many anti-spam groups fall down.
Here’s a status report on the 3.4 release, pulled right from my mailing list post:
- We have the ability to use pkgsrc or dports (building from source in either case) now
- Several people have committed the remaining last-minute fixes
- I’m not going to have pkgsrc binaries built for the release.
- dports binaries – John Marino and Francois Tigeot are uploading now.
I’d like to have the release available with binary packages for dports immediately, because I anticipate a number of people wanting to try it out. So, the release will be delayed a few days while the packages upload.
The DragonFly page on the Summer of Code site is set up. If you are a potential mentor that I’ve talked to before, I’ve already sent you an email with details. If you are a potential mentor I haven’t talked to, you can email me or send a request via the DragonFly page. (Google has a new ‘connections’ method for signup this year.)
If you’re an interested student, take a look at the DragonFly Projects Page. Keep in mind that your proposal does not have to be one of those ideas – new projects are always welcome, and often have the advantage of being unique instead of being one of several similar proposals. (hint, hint)
Constantine Aleksandrovich Murenin has put together a new site, bxr.su. His announcement to users@ goes into a lot of detail, but here’s a preview: it’s an OpenGrok site that has a forked version of OpenGrok that’s both speedy and takes BSD into account, along with other nice features.
Here’s the catch: it’s currently IPv6 only. IPv4 will be on as a test just today, and on for good shortly after. Read that announcement I mentioned for details.
If you have a DragonFly 3.2 system and you want to try the 3.4 release candidate, you can delete your local source, edit the Makefile to pull down 3.4 instead of 3.2, and run it.
rm -rf src
(in vi) :%s/DragonFly_RELEASE_3_2/DragonFly_RELEASE_3_4/g
(save, quit vi)
… then proceed to make buildworld and so on, as normal.
The caveats: I haven’t tested this yet, and this assumes you don’t have any local changes in /usr/src that you want to save. The usual warnings about lighting your computer on fire, etc., apply.
The DragonFly Git repository of pkgsrc now has the 2013Q1 branch. You can switch to it by editing your /usr/Makefile (look for existing references to either pkgsrc master or pkgsrc-2012Q3) and using the normal commands.
DragonFly 3.4 is branched – as a release candidate, with the current target for 3.4.0 release as the weekend of April 13-14. See the tagging commit note for a list of all the commit messages.
Note that in previous releases, we tagged “x.y.0″ on branch, and “x.y.1″ on release. I’m now tagging “x.y.0rc” for the release candidate at branch time, and we’ll tag with a more normal (to my ears) “x.y.0″ for the release.
If you build a 3.4.0rc image right now, you’ll get an older quarterly release of pkgsrc. That’ll be changed tomorrow as the DragonFly pkgsrc git source is updated and I change where 3.4’s /usr/Makefile points.
The 2013Q1 branch of pkgsrc has been announced. Along with the normal quarterly material, there’s several notes: preliminary Cygwin support is present, ruby 1.8 will be retired in favor of 1.9 after this release, and the pkgsrc.org web page now has a very nice new look and logo.
I plan to branch DragonFly 3.4 very soon, and that version will have 2013Q1 as default.
Update: The 2013Q1 branch should be available by tomorrow on DragonFly’s git; the repository needs to update and convert from NetBSD’s CVS and that takes a little time. I’ll post when it’s ready.
If you were thinking, “Hey, I’d like to try an early version of DragonFly 3.4 before it’s released”, I’ll just point you at the recent daily snapshots of 3.3. These are close enough to a release candidate, I think.
The next release of DragonFly will be 3.4, and it’s probably going to be mid-April.