Just so nobody’s surprised: DragonFly process IDs now go an order of magnitude higher.
All the dragonflybsd.org sites (www, bugs, gitweb, lists, leaf) should be available via https now, thanks to a wildcard certificate from InterNetX. Also, all the machines have an up-to-date version (1.0.1g) of OpenSSL installed to prevent the Heartbleed issue.
I wrote up some thoughts for the next release of DragonFly. There’s some project work in there for anyone interested. The next release should be near the end of May.
One of the requirements to get NSS/LDAP working on (most) any unixlike system is to have dynamic binaries; meaning they are dependent on various libraries to run. Since you’re talking about programs for login when you’re talking about NSS/LDAP, that means if the libraries aren’t available, you can’t log in. DragonFly has static binaries just to avoid that problem.
Francois Tigeot proposed switching to dynamic binaries and building a /rescue directory with static backups, as is the case with I think FreeBSD and NetBSD. If you follow the thread, it looks like the best path is to use initrd instead. Initrd stands for INITial Ram Disk, and is the first volume the computer sets up to boot from BIOS. Since initrd gives the computer enough space to load all the needed modules (like Hammer2…), it works without making the computer dependent on various libraries or having a bloated /rescue directory.
(Someone correct me if I have the details wrong.) As long as we’re talking about things that would help DragonFly in a larger environment, can someone work on a VM balloon memory driver, too?
If you noticed the lack of a GUI DVD image for the 3.6 release of DragonFly, I posted a followup note on the users@ list that talks about the steps to get X installed. It’s not much work, with pkg set up.
Normally I’d save this for Lazy Reading, but I’m indirectly involved: the Rochester Institute of Technology now has a minor in Open Source and Free Culture. Here’s the press release. I taught one of the precursor classes, Humanitarian Free/Open Source Development (essentially open source development methods) last spring. Steve Jacobs was my advisor years ago and Remy Decausemaker was my (best) student from the HFOSS class. In any case, the courses are definitely worth it. (via)
I’ve tagged version 3.6.1 of DragonFly, and built ISO/img files of it. They should be available by now on mirrors if you need them, or you can just upgrade as normal. See the linked tag commit message for what’s changed.
As I mentioned on kernel@, I’m going to roll a point release of DragonFly soon. Push in your changes if you want to get them in!
Antonio Huete put together a list of goals for the next release on the DragonFly bugtracker. Some of them are pretty ambitious, some of them are relatively easy, but they are all very useful.
Probably because of the C-state changes, Sepherosa Ziehau wants people to use a new set of sysctls instead of the hw.cpu_mwait* ones – at least on x86_64. This won’t affect you if you aren’t already familiar with them, probably.
Recent updates to tzcode apparently fixed a long-standing time zone bug in DragonFly. POSIX says the America/New_York timezone is picked as default if nothing else has been selected. That didn’t happen in DragonFly – until recently. If your timezone seemed to suddenly jump to U.S. Eastern time, that’s because you never picked before.
If you want to test out the latest (20131218) update to ACPICA, Sepherosa Ziehau’s got a patch for you.
This will be good for anyone who wants to use less electricity. (updated to reflect this doesn’t enable deeper C-states as I thought it did.)
This post from Konrad Neuwirth asking how to do a minimal installation of DragonFly led to this list of all the ‘knobs’ you can set to make your installation smaller, from John Marino. (And your buildworld faster, if that’s appealing to you.) I also pointed at rconfig and PFI, which are criminally underdocumented.
pkg 1.2 is coming out. This brings a number of new features, but as John Marino posted, you may want to delete your old pkg.conf to keep the new version from complaining about an old config file. This upgrade is a step on the way to signed packages, which is a Good Idea.
John Marino posted a possible ‘roadmap’ for DragonFly, now that we’re past the 3.6 release. The thread went on for some ways as it was discussed, including my crazy ideas. Notably, several suggested items have already been tackled – an iwn(4) upgrade has already happened, and an update to bmake, based on John’s vendor branch update instructions.
This is a little old, but Matthew Dillon noted the status of his Hammer2 work a little while ago. Some highlights: he’s intending Hammer2 to be usable on a single host by the time of the next DragonFly release (summer 2014), the Summer of Code project for compression has already been integrated, and he listed different parts of the work that may be interesting for anyone wanting to chip in.
Slightly related: Matt posted some Hammer2 comments on the DragonFly 3.6 release story on Slashdot that may be interesting. Don’t bother reading the other comments; they’ll make your eyeballs bleed.
Eitan Adler is the newest DragonFly committer; you may recognize his name from some previous commits added by others, where he synced up various work between the BSDs.
For those updating from 3.4 to 3.6: there’s an ABI change, so you will have to upgrade all your packages. If you’re using pkgsrc and ready to switch to dports, now’s the time. If you already switched to dports on your 3.4 system, binary packages for 3.6 have already been built and you can use pkg to upgrade.
Also for upgrades from 3.4: You can pull the 3.6 source normally:
git fetch origin
git branch DragonFly_RELEASE_3_6 origin/DragonFly_RELEASE_3_6
git checkout DragonFly_RELEASE_3_6
But there’s a slight change needed for the 3.4 to 3.6 transition: an extra reboot in the build process:
# make buildworld && make buildkernel && make installkernel && make installworld && reboot
# make upgrade
This is all noted in /usr/src/UPDATING and in the release notes, but I’m taking no chances.
As noted on the kernel@ list, it’s tagged but not yet in image form.
Matthew Dillon did some more performance tuning for DragonFly. I’ll just pull a paragraph from the commit message, since that will have more impact than anything I say:
Improves fork/exec concurrency on monster of static binaries from 14200/sec to 55000/sec+. For dynamic binaries improve from around 2500/sec to 9000/sec or so (48 cores fork/exec’ing different dynamic binaries). For the same dynamic binary it’s more around 5000/sec or so.
“monster” is a 48-core machine used for testing.
The venerable (from 1979!) program, lpr, has been superseded by CUPS in many installations. Francois Tigeot suggested removing it, but it’s still directly usable in specific situations and easier to just shift out of the way. It’s staying, but it’s interesting to see how it still gets used.
Update: Predrag Punosevac has descriptions of the various tools involved.
I’m planning to branch DragonFly 3.6 this weekend. The actual release will come 2 weeks later. (Ignore what I wrote about a dports installer/image.)
There is a search plugin for Mozilla that searches DragonFly man pages. (Thanks Samuel Greear)
I got some PC-BSD items this week, too.
- Open Source Snapshot: GhostBSD.
- (Free)BSD and Dropbox.
- FreeBSD finally dumped rcs.
- FreeBSD’s igb(4) driver is updated to 2.4.0.
- FreeBSD’s binutils now has “support for assembling and disassembling Intel Random Number Generator extensions“.
- You can now use ‘athsurvey’ on AR5212 chipset ath(4) devices in FreeBSD.
- FreeBSD branched version 11.
- FreeBSD has changes contributed by… Microsoft?
- PC-BSD has added a GUI version of their Life Preserver application.
- PC-BSD has a new ‘pc-zmanager’ program for managing ZFS and disks.
- PC-BSD has branched version 10, I think.
- NetBSD runs on the iMX233/OLinuXino.
- OpenBSD replaced rc4 with ChaCha20. No, I’m not sure what that means. (via)
- OpenBSD now has the vmwpvs(4) driver, for VMWare paravirtualized SCSI.
- OpenBSD has imported Mesa 9.2.1 and Freetype 184.108.40.206.
- OpenBSD supports the AM335x EDMA3 controller.
- OpenBSD supports the RTL8106E and RTL8168G/8111G networking chipsets.
- Diffe-Hellman key size increased in OpenBSD. It’s from NIST Special Publication 800-57, which is unavailable as of this typing because of the stupid U.S. government shutdown.
The pkgsrc repository in git for DragonFly is currently frozen. This is because many people have switched over to dports, and also because it’s a lot of work to keep it functional. If you do want to pull newer pkgsrc material, use cvs and grab it from a NetBSD server.
As the message notes, don’t go switching to DragonFly-current right now, cause there’s a lot of new material in there and it may not be quite safe. (There’s an ABI change that will require all new builds of your ports, for instance.)
I put together a list of what I’m thinking could be in the next DragonFly release. Going by our regular schedule, that’s a bit more than a month off. Of note: Summer of Code material and defaulting to dports. Follow the thread for more.
ZFS was originally created at Sun and open sourced. Sun was absorbed by Oracle and stopped being open (or even really existing), so ZFS was taken up by several separate groups – FreeBSD and Illumos being two examples. OpenZFS has been announced, in part to provide common reference for other platforms that might implement it and probably to avoid capability fragmentation. It’s certainly a good idea.
(If I have my history wrong, please correct me.)
DragonFly has two included compilers – GCC 4.4, and GCC 4.7. Traditionally, we switch from one compiler to the other as default, and then replace the old one with a newer release, and so on.
Until recently, dports built almost exclusively using GCC 4.4. John Marino’s switching to GCC 4.7, for a variety of reasons he lists in a recent post to users@. An interesting point that he raises: GCC 4.4 won’t necessarily be replaced with a newer GCC, but perhaps clang?
Sepherosa Ziehau has made a number of improvements to TCP in DragonFly – specifically, nonblocking and blocking connect(2) performance. See each of his commits for statistics on how much this has reduced processor use under high load. He has also written up an extensive description of how all this TCP stuff works in DragonFly.
In similar news, he has a nginx patch that delivers a significant performance increase. It may go into nginx itself.
avalon.dragonflybsd.org, also known as mirror-master, is the final dragonflybsd.org system to be moved into the new colocated blade server. Your downloads of binary packages or DragonFly images should be speedier.
I’ll be working on the 3.4.3 release of DragonFly within the next 24 hours, and it should be available this week. I’ll have a list of the bugfixes it contains…
The mailing list archives for DragonFly (lists.dragonflybsd.org) have been moved to new hardware. (Yay!) The patch that actually shows date in the listings needs to reapplied, cause Mailman is somewhat stale. (Boo!) I applied the patch and I’m regenerating all the archives now. (Yay!) There’s some garbled messages in the archives that cause a bunch of “no subject” partial messages to be dumped at the end. (Boo!) I’ll manually fix them if I can, someday. (Yay?)
Definitely Saturdays for this summary. In other BSDs this week:
- It’s FreeBSD, but it can apply to any BSD where a personal attribution license is used: Julian Elischer’s name comes with every iPhone.
- cxgbe(4) cards can now display their temperature under FreeBSD.
- ciss(4) supports additional HP RAID controllers under FreeBSD.
- Bind has been updated to 9.8.5-P2 in FreeBSD.
- FreeBSD has initial support for the Cubieboard 2.
- FreeBSD now has a USB test program.
- NetBSD supports some additional ZTE modem devices.
- NetBSD has cgram, a substitution-cipher solver. For amusement purposes?
- NetBSD supports the Nuvoton W83795G monitoring device.
- OpenBSD now supports wireless devices using the Ralink RT3060.
Wired has an article up about Jordan Hubbard and his move from Apple to iXsystems. It’s not a bad article, though it doesn’t delve into the why of BSD very much. In any case, iXsystems has been really bulking up lately to be more than a generic hardware provider.
Speaking of which, that blade system going in now for dragonflybsd.org was sold by iXsystems.
Joris Giovannangeli, one of the Summer of Code students for DragonFly, posted his thoughts on credential descriptors – have a read. He is working on capsicum and DragonFly, so this is a natural thought process.
Every year, people ask “Why can’t writing documentation be part of Summer of Code?” (Not necessarily for DragonFly, but in general) Google has a “Doc Camp”, where a whole lot of documentation gets produced in sprints, and anyone can participate – not just Summer of Code students.
If this sounds interesting to you, your application has to be in by August
7th 9th. (URL and date updated)
Please welcome our newest DragonFly committer: Johannes Hofmann. He earned this by coming up with a significant chunk of DragonFly’s upcoming KMS/915 support, and it’s now easier to just have him work directly than to be constantly committing for him.
Since there’s a newer set of dports binary packages uploaded, I thought I’d spend my weekend upgrading, to catch up.
And that was it. Well, not really. I had to dump and restore my Postgres databases, cause of the switch from 9.0 to 9.2 as default. I had to build php5 from source to get the Apache module. Those two things together took longer than the entire download and upgrade of the rest of my system – some ~200 packages?
Sepherosa Ziehau added SO_REUSEPORT to DragonFly. I don’t know how the mechanism works, because he didn’t include a description, but he did include a explanation of just how much it reduces CPU usage during as-high-as-physically-possible network load. He even wrote tools to test it more heavily.
Some of the links this week go pretty in-depth. Enjoy!
- This short story from 1954 might serve as a reason to avoid single system image computing… (via Sascha Wildner)
- Vim and Ctags tips and tricks. (via)
- Psygnosis game box designs. Nostalgia for some, neat art for anyone else. (via)
- 50 years of ASCII, and here’s the table it comes from. Some other neat links there, too. (via)
- Richard Feynman and The Connection Machine. If you like stories about Feynman, who was a very interesting person, you may want to read Feynman, the comic book. I met the writer, Jim Ottaviani, years ago, and he was very energetic about both science and comics. Look up his other work if that sounds interesting – which it should. Here’s a sample from the Feynman book. (via)
Back to the Future: Preserving the History of Video Games. This is right around the corner from me. The game museum is as neat as it sounds (yes, they have games out to play), but the article doesn’t mention that it’s attached to a fantastic and huge kid’s museum.
- Building a Cray at home. Similar to this previously-linked idea. (via)
Your unrelated link(s) of the week: Candy Box and A Dark Room. Both are text-only games, but they use HTML5 for animation. They start minimal, and build up – be patient; there’s a lot of gameplay in there. These minimal games fascinate me. It’s like reading a book, where it goes from just static text to an entire world being built. (somewhat via)
Your bonus unrelated comics link of the week: Jack Kirby double-page spreads. It’s not an exaggeration to say this artwork crackles. (via I forget)
Whoops, I missed this when it happened, but: the freeze for pkgsrc-2013Q2 has started. That new quarterly release is anticipated for the end of the month.
Phoronix has another set of benchmarks that include DragonFly and PC-BSD, along with several Linux distributions. It’s interesting to see, though don’t take them as performance measurements. 7-Zip as a benchmark doesn’t describe much other than the program itself, and the Himeno benchmark results are because of the compiler in use rather than any underlying performance aspect of the operating system – for instance. The DragonFly benchmarks disappear after page 3.
I’ve tagged DragonFly 3.4.2. The major reasons for this point release were fixes for DragonFly under Xen with more than 2 CPUs specified, and for booting x86_64 DragonFly in KVM. The 3.4.2 tagged commit has every detail.
If you’ve already got a working 3.4.1 installation, you don’t need to rush to upgrade; this is mostly for the people affected by the issues listed above. I’m working on 3.4.2 install images; give that some time to complete and upload if you need one.
I’ve put the 3.4 release images up on terasaur, a Bittorrent seeding site. Please try pulling them and let me know how it goes. I haven’t torrented many things, so I am unsure how to even verbify “torrent’. Hopefully that sentence and those links work out.
‘william opensource4you’ posted a summary of the steps he took for setting up a DragonFly system with XFCE4, using dports. It’s pretty straightforward, and thanks to dport’s binary nature, should be exactly reproducible.
As I described in a post to the kernel@ mailing list, the DragonFly 3.4 images are getting uploaded for mirroring and downloaded for testing. Assuming no surprises happen, we will be able to release very soon.
John Marino has a concise explanation of why dports mostly uses gcc 4.4 still to compile, even if you’re building DragonFly itself with the default 4.7. It’s a reason to not use NO_GCC44 – yet.
Eric Radman sent along a plug for a utility he is working on called entr(1). The desciption is “Run arbitrary commands when files change.” The site for it has several nifty examples – run make when *.c files change, or convert Markdown files to HTML as soon as they are modified. The really nice thing about it is that it’s perfectly BSD-friendly, and uses kqueue, but will also work on Linux. This beats the “This runs on the one flavor of Linux I use, in one particular shell!” approach I’ve seen from some other developers. See the reddit discussion of it for comparisons to inotify.
No, it’s not in pkgsrc/ports yet.
NASA’s International Space Apps Challenge is this weekend, 4/20/2013. Fancy as it sounds, it’s really a single-day hackathon around open software and hardware, with the problems to fix coming from NASA and therefore probably very unique. It’s happening in a bunch of places around the world, but there’s one right here in my town.
Peter Hansteen has an extensive writeup of how he has managed the bsdly.net spam blacklists. Normally I’d stick this article in the Lazy Reading links, but the article is good enough to call out separately. It’s excellent not just for the mechanical aspects of how the blacklists were maintained, but for his strict description on how the process is simple, verifiable, and transparent. That last item, transparency, is how many anti-spam groups fall down.