Remember I posted that LibreSSL is in base DragonFly, but not default? Well, it’s default now. You can have a system without OpenSSL at all, by rebuilding DragonFly-current and using up-to-date dports.
Update: see John’s comments for clarification: LibreSSL is default; the change is that OpenSSL isn’t even built any more. The result is still the same good news: you can have an OpenSSL-free DragonFly system now.
I don’t have it uploaded yet, but DragonFly 4.6.1 is tagged. Anyone with an existing 4.6.0 or earlier system can upgrade now. Use the 4.6 release instructions if you are unsure on how to upgrade. The 4.6.1 tag commit message has all the changes.
OpenSSL in DragonFly 4.6 has been updated to 1.0.1u. It’s time for a DragonFly 4.6.1, to catch up on this and other updates since the 4.6.0 release. I plan to work on it this weekend.
It’s now possible to start up a vkernel(7) using a COW disk, meaning copy-on-write. One image can be used and reused for multiple vkernels without changing, and all disk activity goes to memory instead.
If you had trouble getting your laptop’s touchpad to work under DragonFly, try again. (If you are running DragonFly-current)
It’s now possible to build dports using LibreSSL instead of OpenSSL. Set SSL_DEFAULT in make.conf to the appropriate port name, and start building. Use synth for fastest results, of course.
LibreSSL will eventually become the default library. This is in addition to the previously-mentioned, already-completed in DragonFly 4.7, base system switch to LibreSSL.
Matthew Dillon has added powerd, a utility that will automatically step down processor speed based on reported temperature. The range is configurable, and there’s some other nice-to-have features. This will save your CPU from melting, and probably also your thighs from being burned.
Recent changes for virtual machine support and the new powerd utility have been rolled into the release branch for DragonFly. They’ll probably be in the next point release, or you can rebuild a release machine now for immediate access.
Also mentioned in the update from Matthew Dillon, DragonFly-master users should upgrade carefully as DragonFly migrates to using LibreSSL in base, and dports-based LibreSSL in dports.
DragonFly now has version 2.4.2 of LibreSSL and uses it in base. Ports may still link to OpenSSL, though – it’s still built by default, though make.conf can be configured to prevent that.
DragonFly-master (i.e. 4.7) now disables DSA keys by default. If you are using a DSA key for SSH/SFTP/whatever, you should change it anyway. Otherwise, it won’t work without workarounds after your next 4.7 upgrade, or by the time of the next DragonFly release.
GCC has been updated by John Marino from 5.3 to 5.4 in DragonFly – the 5.4 closed bug list on the GCC site is a good way to find out the benefits.
There’s been a number of commits lately around higher optimization levels for your DragonFly kernel. It looks you can even set it systemwide. Boot code remains at -O; any higher level will make it explode. Is this safe? I have no idea!
If you are on DragonFly-current, AKA DragonFly 4.7, make sure to perform a full buildworld on your next upgrade. Tomohiro Kusumi changed a Hammer ioctl, and the buildworld is needed to keep everything in sync.
The last bits of Linux emulation have been removed from DragonFly. It’s 32-bit, so it’s been unsupported since DragonFly went to 64-bit only with the 4.0 release. Also, some other 32-bit only items are gone, including the cs, ep, ex, fe, and vx network drivers. It’s almost impossible that anyone was using it, but it’s notable because that’s some… 15-20k lines of code gone? Removal of unused code is also positive.
Because this always happens just after I create a DragonFly release, there’s a new version of OpenSSL. However, this is for version 1.0.2. 1.0.1 is what’s in the release, and it’s supported through the end of the year.
OpenSSH has a major version bump in DragonFly, to 7.3p1. This means some features – specifically patches for High Performance Networking – are no longer there, and you’ll get an error if your config file requires them. Either remove the options from your config, or install OpenSSH from dports.
Did you know that ACPICA has its own internal ‘coding language’, called AML? I did not, but it’s in DragonFly now in any case. Every program eventually grows big enough to read email, and every specification eventually includes its own programming segment.
If you’re one of the people who can easily read ‘systat -vm’ output, the data presented there has been modified. If you’re not one of those people, it’s a good way to monitor system health.
After some testing of different ways to pre-zero out memory pages, Matthew Dillon came to the conclusion: page zeroing doesn’t matter any more. The idea dates all the way back to CSRG, and he’s removed it from DragonFly.
I’m a bit late on this, but: If you are using DragonFly-current, you will need to rebuild world. If you are on 4.4, this won’t matter until you go to 4.6, and you’d be rebuilding world and kernel for that anyway.
(4.6 will probably be tagged this weekend.)
the i915 support in DragonFly now matches the Linux 4.4 kernel, which is good news if you have a Broxton, Skylake, or Cherryview processor, plus it adds a variety of fixes.
It’s exactly what the title is: ipfw3 now does NAT in-kernel, without locking. I have no benchmarks to point at, unfortunately. The commit has usage examples.
Hammer2 now has inode indexing, which Matthew Dillon was avoiding while trying to create more efficient hardlink support. The result is now with that problem solved, more updates can come in: NFS support, mtime updates, output changes, code removal, and lots of other changes, not all of which I’m even linking.
If you have a NVMe chipset under DragonFly, you now can use a special utility to retrieve status information: nvmectl. Right now, only ‘info’ is implemented.
If you are running DragonFly 4.5 (i.e. bleeding edge), Sepherosa Ziehau made an ifnet change that will require a full buildkernel/world if you want things like netstat to keep working.
Sepherosa Ziehau needs to run DragonFly under Hyper-V at work, so he’s making improvements .
As part of his NVMe work, Matthew Dillon found I/O speed so fast that CRC checking actually got in the way of disk activity. He’s brought in a new CRC algorithm called xxHash. He also brought in Mark Adler’s hardware iscsi_crc32 implementation, but did not add it to Hammer2. There’s some work on read-ahead operations too, to deal with the NVMe throughput.
Remember how DragonFly now has autofs? That obsoletes amd, amq, and so on, in the am-utils suite. Now, am-utils has been removed. This may affect nobody, as am-utils wasn’t working well.
The South East Linux Fest is starting tomorrow, and there will be a BSD presence (booth and talks) there – PC-BSD. Stop by if you are the Charlotte, NC area.
(I’d normally save this for In Other BSDs but the event would be half-done by then.)
Remember Sepherosa Ziehau’s nginx tests on DragonFly? He’s using the same configuration to test performance of the accept(2) and close(2) calls. The result? Over 8000 concurrent connections, for 580,000 connections per second. That’s on one DragonFly machine.
Tomohiro Kusumi has finished his port of autofs to DragonFly; you can now have a filesystem automatically mount when accessed, rather than requiring it at boot.
If you are running DragonFly in a virtual environment, there’s been some improvements to virtio(4). Update and try if you’ve had problems in the past.
Sepherosa Ziehau has been working on network performance, including making more network calls asynchronous. His test case using nginx shows that a single DragonFly machine can now take enough traffic to max out 2 10Gb links. That’s with 16Kb requests, and 30,000 of them at the same time.
I usually link to new features, additions, and so on. That’s fine, but there’s often necessary work that goes on which doesn’t correlate to a new function – just better code. Rimvydas Jasinskas just did one of those cleanups, and I’m mentioning it to give credit where’s it’s due.
If you happen to be testing kernel modules, DragonFly can now load them from a modules.local directory. This keeps modules that aren’t part of the base system, separate. This is probably of most use to developers. It’s controlled by local_modules being set in /boot/loader.conf, and defaults to on.
(Updated for correct file location – thanks, swildner)
If you’re using qemu and DragonFly, the latest update of ACPICA to version 20160422 may fix some issues introduced in a previous update. (I don’t have a specific bug report to point you at; sorry!)
Tomohiro Kusumi has been creating a near-constant stream of bugfixes and cleanups to Hammer for quite some time. I don’t often link to it, because they are incremental improvements and hard to linkblog, so to speak. In an effort to make up for this deficit, I do want to draw attention to his two recent commits: “Make hammer commands print root volume path“, and “Print volume list after volume-add|del“. Small changes, but this is what makes complex systems usable.
I keep posting about Sepherosa Ziehau’s work on sustaining extremely high traffic loads in DragonFly. Now I’m posting about a tool to create that load: kq_sendrecv. It creates tens of thousands of TCP connections, without creating a process for each, and uses kqueue, as you might guess from the name. This may be useful if you really want to tax another system.
unzip has been added to DragonFly, making it present in every BSD but I think OpenBSD.
Imre Vadasz has added the ability to create a UEFI bootloader in DragonFly. Can you use it? I don’t know; I haven’t tried it yet and I can’t tell from the commit.
John Marino has added the starting framework to use clang as the alternate base compiler in DragonFly. Note that it’s not hooked into the build yet. This is the first non-GCC compiler added into DragonFly, so there’s some work yet before you can have an all-clang system. This should replace GCC 4.7, which is the current alternate compiler. GCC 5.0 is the default, if you didn’t know.
Note that clang is present in dports, so it’s already been available for general use, for some time. This framework is for building DragonFly itself.
If you are running bleeding-edge DragonFly, Sepherosa Ziehau has made some networking changes that both reduce CPU usage in high-traffic situations and change some underlying network structures. This means a full buildworld is needed on your next update.
If you’re using DragonFly 4.4.x or older, you are unaffected.
Sepherosa Ziehau has continued his quest of making large-scale data transmission on DragonFly effortless; his latest change has cut the kqueue contention rate by two-thirds when dealing with a connection rate of nearly 400,000 connections per second. Note that’s number of connections, without even tracking the bandwidth used by each.
Bill Yuan has added ‘ipfwsync’ to ipfw3 in DragonFly. As you may expect from the name, it’s a way to sync ipfw3 configurations across multiple devices.
Welcome the newest DragonFly committer: Bill Yuan. His ipfw3 work has been going on for a while.
If you have a Broadwell system, the drm.i915.enable_execlists tunable added by Imre Vadász may keep your system stable. (thanks, zach on EFNet #dragonflybsd)
That’s a pretty cryptic headline, isn’t it? John Marino has ‘privatized’ several libraries in DragonFly, so that they can’t get included involuntarily as part of a port build. That may mean you will need to perform a full rebuild of your system if you are tracking DragonFly-current.
(This is the way to fix ‘system’ languages like Perl was in FreeBSD 4.x – keep them clearly separate from the port version. It’s about a decade too late for that idea to work out, though.)
This has no effect on the actual operation of DragonFly, but it makes me feel better that it’s done: Rimvydas Jasinskas has gone through DragonFly source and removed the unnecessary 3rd BSD license clause, which is no longer needed.
Please welcome DragonFly’s newest committer: Rimvydas Jasinskas. He’s already done some adding and removing, and he’s been making a ton of dports changes for some time.
A number of people have reported problems with qemu and DragonFly, both running locally and on a host. It turns out to be a problem with the getcontext(), setcontext(), and swapcontext() functions, but Matthew Dillon fixed it in a way that doesn’t affect performance very much.
That apparently wasn’t good enough, so he added _quick versions of those same functions, so it became not just a fix, but an improvement.
In related qemu news: qemu-devel can use vknetd similar to a vkernel, now.
Hammer now defaults to ‘noatime’, meaning the date and time of last access are not updated on every file action. Note that creation and modification date and time are still recorded. This will help with speed and disk activity.
This may cause a problem with any software expecting this to change – mutt, possibly? We will find out. This change was done after the 4.4 branch, so it’s not in the current release of DragonFly.
I am taking this moment away from my significant backlog of things to post to note that there have been a lot of games fixes in DPorts lately. Thanks to Rimvydas, many small bugs that kept games from compiling on DragonFly are now fixed. The easiest way to see is to look at the commits from December 8th and back, but the best way is to pick one and play.
If you are running DragonFly-master (i.e. 4.5), and you have a system between these two updates (roughly between November 27th and now), please rebuild your kernel to avoid a TCP bug.
I have a huge backlog of things to post, so this is originating from the 17th: Matthew Dillon has been working for some time on hardlinks and Hammer 2. Hardlinks are the same file, presented in multiple places. This can be a problem when your filesystem keeps infinite, writable snapshots. The solution he just commited is called ‘xlink’ and the commit message has details.
The default linker in DragonFly has been switched to gold, the newer version of ld. (get it, go-ld?) It’s faster, cleaner, going by the commit message. It’s possible to switch back to the old one if needed. This predates the recent branch for 4.4, so it will be default in the release, too.
Imre Vadász fixed top so that hitting ‘c’ filters displayed processes by command name. I am mentioning this not because it’s a huge change, but because I forget about all the interactive elements that are possible with top.
Does that count as alliteration? Anyway, Matthew Dillon has increased the size of the starting window in TCP. If you are on a higher-latency link and/or fetching lots of small files, you should notice better performance.
The disk scheduler apparatus in DragonFly has been removed. This may not affect you much, since alternate scheduling setups were never utilized much with it. It may fix some rare Hammer cleanup issues, though, and you may need to adjust your custom kernel config, if you have one.
If you are using bleeding-edge DragonFly (4.3) on a machine with Intel video, the i915 module has been renamed. This means you will probably need to rebuild xf86-video-intel from source to have it match. There should be a matching binary package soon.
If you are on DragonFly 4.2, this does not affect you.
John Marino’s made a number of updates to contributed software in DragonFly recently, and here’s the list:
libelf (not contrib as John pointed out), libexecinfo, xz, libedit, binutils, grep, tcsh, libdialog, and (tn)ftp.
Remember what I was saying about Sepherosa Ziehau and improving performance? Well, here he goes again, three times.
I think at this point, Sepherosa Ziehau is able to improve the DragonFly network stack by just standing near his computer and concentrating for a few minutes. For example, he’s unearthed another improvement to connect rate/reduction of CPU usage.
Charles Musser updated rtadvd and added rtadvctl for DragonFly, based on what’s in FreeBSD (which is based on KAME? I’m not sure). This is most useful if you are using IPv6.
John Marino is working on versioning libc, and as part of that process, libc is no longer loaded into executable memory. Here is I think an explanation of lib versioning that may apply, and of course moving things that aren’t supposed to execute, out of executable memory areas, is good for security. There’s more on that topic, too – W^X may be a similar example.
This is a complicated topic that I’m not part of, so suggest better descriptions in the comments, please.
I don’t note it enough, but Tomohiro Kusumi has been making constant updates to HAMMER, the version we have now. Often they are the sort of update that makes the code more readable, or fixes possible problems, and so on. Very essential, but hard to post about it. In any case, I’m using his recent improvements to hammer volume-del to note his contributions, of which there are much more than the day’s worth I link here.
Did you know that AT&T maintains a
regex library and test suite? I did not, but now DragonFly has both, in part for better multibyte character support.
(corrected to note that the regex library is not from AT&T – thanks, anonymous commenter)
The vi in any BSD is not the original Berkeley vi – instead it’s usually nvi. However, thanks to John Marino, DragonFly has the up-to-date, multibyte-supporting nvi2. (I know I’ve made reference to the nv/nvi difference before.)
If you are running DragonFly-master, there have been fixes for a wrong uname (my fault) and initrd image booting with encrypted drives. Update if you are running on the bleeding edge, if you haven’t already.
John Marino has been updating locale support in DragonFly. There’s no single explanatory commit to point at, so I’ll instead link to the many, many commits and changes he’s been making to show the size of the work. If you are anywhere other than UTF-8 (or maybe even then), this will help you.
I’m globbing these DragonFly updates together in a single post because I’m running behind:
ACPICA was updated to Intel’s newest version: 20150717.
GCC in DragonFly was updated to the 5.2 release.
DragonFly DRM (that’s Direct Rendering) now supports ValleyView chipsets.
hostapd, for creating a wireless access point, has been included in DragonFly along with wpa_supplicant, for a long time. Like wpa_supplicant, there’s a version in dports that is the latest version and is easier to update (e.g. no system update required to get a newer version.) Unlike wpa_supplicant, there’s no chicken-and-egg installation problem if it’s not in the base system – so out it goes.
There’s yet another security problem with OpenSSL, and it’s been updated in DragonFly. I’ll probably roll 4.2.2 this weekend so that it’s in the release image.
The more eagle-eyed may have noticed a branching for DragonFly 4.2, and for DragonFly 4.0.6. The 4.2 branch is currently only a release candidate, so don’t necessarily change over yet – it’s for testing, not release.
Note that packages for 4.2 are not yet built, so you’ll have to manually specify a package path to install with pkg on 4.2 – for now.. That won’t be the case for the actual release, of course. DragonFly 4.3 users will have to specify PKG_PATH manually to use 4.2 images until new ones are built. 4.2 release candidate users will be fine. (see comments for correction.)
The 4.0.6 release is mostly to get the recent OpenSSL update into a 4.0.x build.
I am working on image building for both.
Even sysctl accesses can be made to handle multiprocessor environments. This can actually make a difference when you’ve got a lot of processors building a lot of software, as in all of dports.
From IRC today:
“19:43 <@dillon> I’m bored at the moment. lemme try to speed up module builds“
Buildkernel, depending on your processor count, now may have tripled in speed.
Those changes I mentioned yesterday for text console support? They’re in DragonFly-master now, along with a loader tunable to turn it on and off.
If you’re using nginx on DragonFly, version 1.9.1 has specific DragonFly speedup options built in.
Hammer 2 now uses LZ4 compression by default. It also uses a new CRC algorithm that performs much better, and there’s numbers to prove it. It helps iSCSI too. When I say new, it appears to be from the 1980s? I may be looking at the wrong place.
Matthew Dillon has been doing a lot of Hammer 2 work lately. Well, he’s been doing it for quite some time, but the recent commits contain the sort of things that are easier to link to, like deletion speedups, freemap changes, and stats tracking/compression results.
Hammer will perform daily housekeeping tasks each night. If you’re up late enough, it may kick off while you are working. If you want to stop the process after it’s already started (since it’s disk-intensive), John Marino has added the ‘abort-cleanup‘ command.