Sepherosa Ziehau has been working on network performance, including making more network calls asynchronous. His test case using nginx shows that a single DragonFly machine can now take enough traffic to max out 2 10Gb links. That’s with 16Kb requests, and 30,000 of them at the same time.
Matthew Dillon and Adrian Chadd have updated the wifi setup in DragonFly, incorporating Adrian’s FreeBSD changes (and merging back some of Matt’s from DragonFly). This affects the ath, rum, iwm, iwn, run, bwn, urtwn, wi, ral, iwi, ndis, and wpi drivers. The ‘an’ driver has been removed, too. I’m not going to even try to link to all the commits.
If you’re on DragonFly master and are using one of these devices, now is the time to update and try. Note that this removes the separate network interface that’s specific to the device and creates only a wlanX device.
Update: Matt reminded me that at least half the work came from Imre Vadasz; I missed it because I was only looking at the commit email names – mea culpa.
karu.pruun managed to get xwayland working on DragonFly, and also took notes while doing it. That means you can try it out, too.
If you get “libGL error: failed to open drm device: Permission denied” when using direct rendering, make sure to add your user id to the ‘video’ group.
Some DragonFly links are sneaking in here just to get them cleared out.
- May 17th: Ike Levy speaks at SemiBUG. Go if you are anywhere near; Ike’s a good speaker and passionate about BSD.
- Speaking of scheduling: BSDCan 2016 is less than a month away.
- Why OpenBSD Is Important to Me. (via)
- BSD Unix: Power to the people, from the code. (via)
- FreeBSD PowerPC 32bit pkg repository (unofficial). ~19,500 packages, more to come. (via)
- As a Linux user, where should I start with experimenting with BSD?
- DragonFly i915 driver updated to Linux 4.3. (via)
- DiscoverBSD for 2016/05/09.
- Cons of staying on an old -RELEASE version ?
- More p2k16: ajacoutot@ on Gnome, rc and rcctl improvements, krw@ on pdisk, softraid and more.
- SROP mitigation committed. (OpenBSD)
- The 50th Quarterly pkgsrc Release, pkgsrc-2016Q1. Also, stats.
- Thomas Levine’s notes from the recent NYCBUG presentation on Urchin.
- NetBSD on the Sega Dreamcast, presented on a Dreamcast.
- How BSD was built, and how it lost the lead to Linux.
- Running Tor in a NetBSD rump unikernel. (via)
- Running FreeBSD / OpenBSD / NetBSD as a virtualised guest on Online.net.
- Meet Joe Maloney – Lead System Architect for PC-BSD. I like the transition from volunteer to employee.
- LinuxFest Northwest 2016: The Devil in the Details: Switching to BSD from Linux. Apparently one of the most popular videos.
If you are on the Skylake series of processors, and also running xorg on DragonFly, pick ‘uxa’ video acceleration. Andrew Slaughter found this made a significant different in visual quality.
Sepherosa Ziehau posted an extended description of his work with nginx on DragonFly, and the kind of performance he was able to wring out of it. Of special note: he posts all his sysctl changes, which might be useful to anyone else in high-traffic environments, and notes that he was able to saturate a 10Gb link with one DragonFly machine.
Also: a followup comparing interrupt vs. polling performance.
The drm/i915 driver has been updated by Francois Tigeot to match what’s in Linux kernel 4.3. His commit post has the general detail; you will especially want this if on DragonFly-current and running on Skylake architecture.
Tomohiro Kusumi has been working on a port of autofs to DragonFly. If you aren’t familiar with it, autofs is an automatic file system mounter, so when you access a network file system at its local mount point, autofs kicks in and makes sure the remote file system is automatically mounted. He has an initial report on his progress, and expects it to be in DragonFly master in the next month.
If you’ve ever wondered how having multiple swap devices can work, here’s your DragonFly-specific answer.
If you happen to be testing kernel modules, DragonFly can now load them from a modules.local directory. This keeps modules that aren’t part of the base system, separate. This is probably of most use to developers. It’s controlled by local_modules being set in /boot/loader.conf, and defaults to on.
(Updated for correct file location – thanks, swildner)
If you’re on DragonFly, or maybe even if you aren’t, and you are using NFS, here’s some tips on how to wring the best performance out of it.
Not older people that use DragonFly, but people of any age using an older release of DragonFly: Bezitopo is Pierre Abbat’s topographical program, and he needs testers on versions 4.4 of DragonFly or before. Please give his open-source program a run if you are on the appropriate versions. Trying other BSDs, even though not requested, can’t hurt.
If you’re using qemu and DragonFly, the latest update of ACPICA to version 20160422 may fix some issues introduced in a previous update. (I don’t have a specific bug report to point you at; sorry!)
The DragonFly 4.4.3 point release is out. There’s a commit page listing the changes between 4.4.2 and 4.4.3. Nobody will be surprised that there’s an OpenSSL update in there.
cd /usr; make src-update (or src-create-shallow if you don't already have source) make buildworld && make buildkernel make installkernel && make installworld make upgrade reboot
This is one of those weeks where a bunch of release all tumble together by chance.
- UbuntuBSD Is Looking To Become An Official Ubuntu Flavor. (still confusing)
- PC-BSD 10.3 out; PC-BSD 11 out next. 10.3 was out last week; I missed this link before.
- pfSense 2.3-RELEASE Now Available! (also seen here and here)
- PostgreSQL – Add BSD authentication method. (via)
- BSD and Toshiba Chromebook 2.
- FreeBSD 10.3-Release on AWS. As Colin Percival points out, the last half-dozen releases have been on AWS too.
- Undeadly and HTTPS. (via)
- Penguicon 2016 Lucas Track Schedule. For being called “Penguicon”, there’s a lot of BSD events there.
- DiscoverBSD for 2016/04/11.
- OPNsense 16.1.9 released.
- OPNsense 16.1.10 released.
- Unix’s file durability problem, which leads in comment to disks from the perspective of a file system (McKusick), which I thought I had linked before but maybe not. (via)
- FreeNAS, TrueNAS, and BadLock.
- UbuntuBSD Should Heed Kubuntu’s Cautionary Tale.
Tomohiro Kusumi would like to port Hammer (1) to FreeBSD, as noted in this bug discussion. It’s not even begun to happen, but if you can contribute, please do.
Tomohiro Kusumi has been creating a near-constant stream of bugfixes and cleanups to Hammer for quite some time. I don’t often link to it, because they are incremental improvements and hard to linkblog, so to speak. In an effort to make up for this deficit, I do want to draw attention to his two recent commits: “Make hammer commands print root volume path“, and “Print volume list after volume-add|del“. Small changes, but this is what makes complex systems usable.
If you remember this Baytrail problem, Daniel Bilik has gone and found a fix, as this appears to be a cross-platform bug, and he has patches for DragonFly. If it’s affecting you, you don’t have to wait for the patches to be added in; he’s made them available directly.
Update: it’s committed to DragonFly now.
I keep posting about Sepherosa Ziehau’s work on sustaining extremely high traffic loads in DragonFly. Now I’m posting about a tool to create that load: kq_sendrecv. It creates tens of thousands of TCP connections, without creating a process for each, and uses kqueue, as you might guess from the name. This may be useful if you really want to tax another system.
By the time you read this, I will have already been at my second job for 5 hours.
- Integrating FreeBSD w/ FreeIPA/SSSD.
- UbuntuBSD, mentioned here, here, here, here. Best reaction here.
- RocketGraph FreeBSD commits on Github for 2015.
- Why OpenBSD? (via)
- KnoxBUG: A new BSD User Group in Knoxville area. (via)
- Install OPNSense on the Monowall Appliance box. (via)
- OPNSense 16.1.8 released.
- FreeBSD – a lesson in poor defaults. Some axegrinding going on. (via and via)
- MidnightBSD with Lucas Holt.
- DiscoverBSD for 2016/03/21.
- Using Firefox to watch Netflix on FreeBSD/PC-BSD. (via)
- Larry the BSD Guy’s BSD summary for the week.
- New routing table code (ART) enabled in [OpenBSD]-current.
- Can’t control screen brightness in Broadwell.
- fractal cells – FreeBSD-based All-In-One solution for software development startups. (via)
unzip has been added to DragonFly, making it present in every BSD but I think OpenBSD.
Imre Vadasz has added the ability to create a UEFI bootloader in DragonFly. Can you use it? I don’t know; I haven’t tried it yet and I can’t tell from the commit.
John Marino has added the starting framework to use clang as the alternate base compiler in DragonFly. Note that it’s not hooked into the build yet. This is the first non-GCC compiler added into DragonFly, so there’s some work yet before you can have an all-clang system. This should replace GCC 4.7, which is the current alternate compiler. GCC 5.0 is the default, if you didn’t know.
Note that clang is present in dports, so it’s already been available for general use, for some time. This framework is for building DragonFly itself.
If you somehow have a device with multiple SD/MMC card slots, you can now access all of them under DragonFly. (Apparently done to make a tablet run DragonFly better, going by IRC conversation)
If you are running bleeding-edge DragonFly, Sepherosa Ziehau has made some networking changes that both reduce CPU usage in high-traffic situations and change some underlying network structures. This means a full buildworld is needed on your next update.
If you’re using DragonFly 4.4.x or older, you are unaffected.
If you find yourself using gpt and disklabel64 for a new disk, and aren’t quite sure what order to type everything in to create a disk slice, why not crib from Tim Darby’s notes? (note that the archive has added some line breaks to it.)
The ‘hammer show’ command can be used to dump the B-Tree structure of a Hammer volume, and CRC errors can be spotted. It’s rare that anyone would need it, but if you do, this dumped information will include file hierarchy information.
If that makes you a bit nervous to repost any of that information when talking about it in public, Tomohiro Kusumi has added an ‘obfuscate’ option to ‘hammer show’ that does just that – it hides path information from the debug output.
Sepherosa Ziehau has continued his quest of making large-scale data transmission on DragonFly effortless; his latest change has cut the kqueue contention rate by two-thirds when dealing with a connection rate of nearly 400,000 connections per second. Note that’s number of connections, without even tracking the bandwidth used by each.
John Marino rearranged how GCC5 handles CPUTYPE settings. If you are specifically setting the target CPU when compiling, his commit will give you an exact list of what to target.
Note that I am not saying another architecture – this is all x86_64. I also don’t recommend doing this unless you have a specific use for it – compiler overoptimizations often create more problems than they fix.
Daniel Bilik has found there’s an issue with i915 acceleration, Baytrail CPUs, and some AUTODEEP low-power states. This will only affect you if you are using that specific hardware combo and setting certain low power modes. Interestingly, it affects other platforms, too, as it appears to be a symptom of how the video is addressed, not a DragonFly-specific bug.
Bill Yuan has added ‘ipfwsync’ to ipfw3 in DragonFly. As you may expect from the name, it’s a way to sync ipfw3 configurations across multiple devices.
I see this bite people irregularly over the years: if your default shell on login can’t run, what do you do? I’ve seen it happen because of a missing /usr/lib, and it can happen with out-of-date library references, too. There’s several different ways to deal with it:
- Run a shell that can’t have this problem, like /bin/tcsh (the root default).
- Or, rebuild in single-user mode from the console.
- Or, perform the bullet-proof upgrade.
That last one may be useful if your dports setup gets mangled, somehow – though ‘pkg upgrade’ has always worked for me.
Francois Tigeot has again updated Intel i915 video support in DragonFly, bringing it even with what’s in Linux 4.2. This will be very useful for Broadwell and Skylake users, and even Broxton, apparently the newest Atom platform.
DragonFly 4.4.2, a bugfix release to 4.4.1, is out. This was mostly prompted by the recent OpenSSL update, but other little fixes have made it in, too. It’s available for download and is probably available at your nearest mirror by now, if you want an image. The release page is updated, and there’s always the Git tag summary for 4.4.2 for the most exact details.
I have DragonFly shirts, helpfully printed up by Sepherosa Ziehau in China. I have a list of people that are interested in shirts, most of whom remembered to give a shirt size. I don’t have anyone’s email address or mailing address on that list.
If you are on that list, send me your mailing address.
The shirts are marked L/XL/XXL/XXXL, but they run smaller than U.S. versions of those sizes. I usually find a U.S. XL shirt baggy, but “XXXL” is the one that fit me, for instance. I’ll do my best to place the appropriate one. This is just an advance apology, since it’s too late to change anything if it turns out tight.
I’ll mail these out as I have the spare cash and time on hand. (I hope most of you live in the continental U.S.)
If you have a Broadwell system, the drm.i915.enable_execlists tunable added by Imre Vadász may keep your system stable. (thanks, zach on EFNet #dragonflybsd)
Rimvydas Jasinskas has consolidated/restored/updated a large number of papers into share/doc on DragonFly. I’m not going to link to his large number of commits, but instead point you at the directory where they all went. There’s a number of design documents in there that date back to 4.4BSD and beyond (and some much newer), which may interest or educate you. Of special note: The Guide to the Dungeons of Doom, for rogue, or the KAME IPv6 implementation notes.
These probably apply cross-BSD, but in this case, it’s DragonFly tips for printing with CUPS.
If you have a Core2 processor in a DragonFly system, it may not work with accelerated video. If that happens to you with this (admittedly old) processor, switch to VESA for now.
For those of you running DragonFly-current, the already-mentioned library privatization going on means that ports have to be rebuilt. You will want to do it yourself, or wait a little bit before upgrading if you want to install binaries.
BSDNow 126 has an interview with Ken Moore and Kris Moore of PC-BSD, along with the usual news roundup. There’s a DragonFly mention in the “open source work helps your career” news item that I did not know about but am happy to see.
That’s a pretty cryptic headline, isn’t it? John Marino has ‘privatized’ several libraries in DragonFly, so that they can’t get included involuntarily as part of a port build. That may mean you will need to perform a full rebuild of your system if you are tracking DragonFly-current.
(This is the way to fix ‘system’ languages like Perl was in FreeBSD 4.x – keep them clearly separate from the port version. It’s about a decade too late for that idea to work out, though.)
For those of you with i915 video on your DragonFly system, there’s another update bringing DragonFly support to match what’s in the Linux 4.1 kernel. ValleyView and Skylake processor owners will benefit, along with a slew of other bugfixes and improvements.
Are you using a i915 video chipset? Are you using the DisplayPort? Imre Vadasz has added a tunable that may make it work better.
Note: keep in mind this is a client bug – it’s an information leak when you as a client connect out to somewhere else. A server, as an endpoint, is not affected.
John Marino has opened up his new utility for testing: Synth. It’s made for building custom package repositories, similar to poudriere, but much less setup work. If you’ve ever said “I like binary installs, but I want my own build options”, this is for you. The README includes screenshots to show all the things it can do.
Francois Tigeot has updated DragonFly to match the video support found in the Linux 4.0 kernel. This will benefit you most if you are running Skylake, Cherryview, or Valleyview chipsets. Don’t ask me how to tell; the improvement has been so rapid I’ve lost track of which model codename is which.
There’s some DragonFly links I snuck in here because why not?
- OpenBSD Innovation List. (via)
- How to block traffic based off country – pFSense (via)
- pfSense 2.2.6 is released.
- Orchestrating multiple FreeBSDs?
- Hacking the PS4, part 3: FreeBSD Kernel exploitation. (via)
- PIC32-RetroBSD Open Source Hardware Board running Unix like RetroBSD OS. (via)
- Is there a way to cite the FreeBSD handbook and other documentation in APA format?
- Newbie testing out new OS’s
- OPNsense 15.7.23 Released
- [PSA] 1920×1080 on DragonFlyBSD 4.4 under QEMU/KVM.
- The DragonFly 4.4 release article on linuxfr.org – always in-depth.
- Faces of FreeBSD 2015: Erin Clark.
- n2k15: bluhm@ on MP networking (out from under biglock)
- n2k15: vgross@ on deep surgery in TCP/IP stack code
- n2k15: krw@ on fdisk, installboot, dhclient, GPT fixes
- n2k15: reyk@ on hosting a hackathon, vmd, and the switch
- n2k15: mpi@ on MP networking progress
- n2k15: stsp@ on 11n mode wifi, testing
- OpenBSD’s sndiod: now with privsep
- Problems with Systemd and Why I Like BSD Init. (via)
- DiscoverBSD for 2015/12/21.
- AsiaBSDCon 2016 is happening March 10-13, 2016, in Tokyo. The call for papers is out and due by January 8th. Tutorial proposals are due at the end of the month.
A number of people have reported problems with qemu and DragonFly, both running locally and on a host. It turns out to be a problem with the getcontext(), setcontext(), and swapcontext() functions, but Matthew Dillon fixed it in a way that doesn’t affect performance very much.
That apparently wasn’t good enough, so he added _quick versions of those same functions, so it became not just a fix, but an improvement.
In related qemu news: qemu-devel can use vknetd similar to a vkernel, now.
For those of you that are very bandwidth-constrained, or just impatient, there are xz-compressed images of DragonFly 4.4 available. (see ‘download live image’ area) The mirrors should have them too.
The DragonFly installer has been modified to produce disk arrangements that will generally match between UFS and Hammer installs, plus directories where you usually don’t want Hammer history or backups (like /tmp or /usr/obj) are now under /build and null-mounted to where you’d expect, since null-mounting works transparently well on DragonFly. Matthew Dillon has a note explaining the whole thing.
For those of you looking to rent a place to run DragonFly, Nuno Antunes has very helpfully written out his procedure for installing DragonFly on a Digital Ocean ‘droplet’.
As mentioned previously, Sepherosa Ziehau is printing up some DragonFly T-shirts for WeChat users. He’s going to have a few left over, so he is sending them to me to hand to non-China people. If you want one, leave a note saying so in the comments. Here’s the front and back.
You need to provide some way for me to contact you – preferably email, and the size you’d want. (Use the Land’s End Men’s Shirts chart for sizing, because why not.) I’ll only have a few, so no guarantees.
Update: I have more responses than probable shirts at this point – sorry! I’ll get in contact with each of you once the shirts come in and arrange delivery.
Hammer now defaults to ‘noatime’, meaning the date and time of last access are not updated on every file action. Note that creation and modification date and time are still recorded. This will help with speed and disk activity.
This may cause a problem with any software expecting this to change – mutt, possibly? We will find out. This change was done after the 4.4 branch, so it’s not in the current release of DragonFly.
If you are a WeChat user and want a DragonFly BSD shirt, send your Chinese address and mobile number to firstname.lastname@example.org, or scan this image to join the WeChat DragonFly BSD group.
This is exclusive to China right now, as it’s being done by DragonFly developer Sepherosa Ziehau – who, as you might guess by now, is based in China.