GCC version 4.7 is already available now in DragonFly 3.2, but it’s not the default compiler. John Marino intends to make it default for the next release. What’s that mean for us? Nothing other than a new compiler, since he’s already fixing related issues.
Markus Pfeiffer reports success using Xen HVM to run DragonFly, which may be useful for any of you Xen users. He reports not being able to use more than 2 virtual CPUs, though Scott Tincman reports successfully using 4 (with qemu), so your mileage may vary.
Updated: noting qemu usage as Markus pointed out in comments.
Sepherosa Ziehau makes commits almost daily to DragonFly’s network infrastructure, but I have a hard time quantifying it into Digest posts in part because it’s often very technical. His most recent commits come with an explanation, however. He has done plenty of work to improve overall transmission speeds in DragonFly, and now he’s working on ‘fairness’. Fair, in this case, means ensuring that packet transmitting and receiving happen without either one monopolizing the connection. In real world terms, this translates to much more constant speeds. His recent commit details what he’s doing and some numbers to prove it.
Remember I said he’s improved speeds? Note that in his example, he’s reaching stable peaks of 981 Mbps. This is on a line that I assume theoretically maxes out at 1000.
Based on this bug report on the recently updated m4, you may need to perform some extra steps to update m4 as part of a normal upgrade:
# cd /usr/src/usr.bin/m4 # make # make install clean
If you have a DragonFly 3.3 system with DPorts, can you install xorg, then ssh -Y from another machine to there, and see if you can remotely run an X program like xterm with local display? I’ve done this twice on two different machines with DPorts and it won’t work. xorg won’t write the security info to ~/.Xauthority, with ssh or xhost or whatever. It’s driving me crazy.
(Yeah, slow news day.)
Peter Avalos has updated m4 for DragonFly. This will bring us a little more in sync with the other BSDs. Also, John Marino has updated flex, which is apparently 17 years old? Meaning it hasn’t been updated in DragonFly ever, and then not in FreeBSD before that, for a long time. Looking at the timeline on the flex web page appears to match.
Dave Hayes asked for some “best practices” ideas for setting up a HAMMER (1) system. I replied, and the conversation turned to RAID, as these often do. If you’re thinking of purchasing disk hardware in the near future, this will be useful to you.
Sepherosa Ziehau has added a generic form of support for multiple transmit queues in DragonFly. This means less contention when transmitting. It’s not done; he has drivers to set up and as he said, it’s “step 1 of many“.
If you are a brave soul and have an IPv6-only DragonFly installation, there’s now a git mirror of DragonFly that is available on IPv6.
I meant to post this a while ago; it’s a few days old but still useful. John Marino gave some stats on DPorts progress, plus he and Francois Tigeot also had some tips on xorg setup. The successful build count is higher by now, and I think KDE3 is done, though I haven’t tried it.
If you are on DragonFly 3.3, and you are running a kernel built after January 1st, there’s a bug in the way FP context is handled when the kernel supports AVX. (January 1st is when AVX support was committed.) Matthew Dillon has committed a fix and issued a note to update for everyone.
John Marino’s DPorts project, mentioned here briefly before, is interesting. I had two separate people ask me how it works, so a better explanation is in order. I’ve tried it out on a test machine over the past few weeks.
Background:
Dports is an effort to use FreeBSD’s ports system as a base for DragonFly, and the pkg tool as a way to manage binary packages built from DPorts. This is complicated, so I’ll explain each part in order.
- FreeBSD ports are a FreeBSD-specific collection of software installation files that automate building 3rd-party software on FreeBSD. You’ve probably already heard of them. (Note there’s no mention of DragonFly.)
- DPorts is a collection of files that map to existing FreeBSD ports, and contain any changes necessary to make that port also build on DragonFly. Many of those programs build without changes on DragonFly. DPorts builds from source.
- pkg is used for package management, and is usable on FreeBSD and on DragonFly. The binary packages produced from building with DPorts can be installed from remote locations and managed separately using pkg, so that software upgrades and installation can be performed with binaries only. (It’s much faster that way.)
Every port seen in DPorts is known to build on DragonFly. John Marino adds a port only after it builds successfully, using poudriere as a bulk software tool. Ports are only updated to a newer version when that newer version builds, too, so once something arrives in DPorts, it should never break from being updated at some point in the future.
Installing:
To use DPorts, you need two things:
- DragonFly 3.3 or later, though 3.3 is the most recent right now.
- You need to rename /usr/pkg so that your existing pkgsrc binary programs don’t get accidentally used while working with DPorts, causing confusion. If anything goes wrong with DPorts when you are installing it and you want to go back, remove all the DPorts packages and rename /usr/pkg back to normal.
(Don’t confuse pkg, the management tool, with /usr/pkg, the normal installation directory for pkgsrc. ) For the installation of the base port files:
cd /usr make dports-create-shallow
If you’ve already renamed your /usr/pkg directory, git won’t be in your path any more. You can instead download a tarball and unpack it, which also happens to be possible automatically via that same Makefile.
cd /usr make dports-download
Downloading via git is fastest, so if you do need to use the tarball via make dports-download, build devel/git, delete /usr/dports, and then pull it again with make dports-create-shallow. This all comes from John Marino’s Github site for DPorts.
Managing DPorts
DPorts doesn’t use pkg_info, pkg_add, and the other tools traditionally seen on DragonFly for pkgsrc. Instead, package management is done with pkg. Use pkg info, pkg install, pkg remove, and pkg update to list, install, delete, and upgrade various packages on your system. Packages built from source or downloaded as prebuilt binaries are managed the same way, using these tools.
See some of the other writing about pkg for FreeBSD for details on how it works.
Since DPorts doesn’t update a package until it gets a successful build, and installations are of successfully built binary packages, upgrades with prebuilt packages should always succeed. Since they’re binary, they should be fast. There’s a lot of ‘shoulds’ in this sentence, but these are reasonable suppositions.
What about pkgsrc?
Pkgsrc and DPorts shouldn’t be used at the same time, since one system’s packages may be at different versions but still get picked up during building for the other system. That’s about it for restrictions.
I intend to try building an experimental release of DragonFly with DPorts, to see if all the right packages can be added, but no guarantees. DPorts is brand new and does not yet have a repository for downloading packages, so the normal caveats apply; don’t install it on a mission-critical machine, and be ready to deal with any surprises from using it if you do try it out.
What packages are available?
Browsing the Github repo will show you all listed packages. More complex packages like xorg, openjdk7, and libreoffice install, as does xfce. Parts of KDE 3 and KDE 4 are in there. (I haven’t tried either.) I’m not sure about Gnome, but I don’t think anyone ever is. There’s no vim, but there is emacs.
That’s just what I see at this exact minute. It changes daily as more packages are built. Changes from DragonFly builds are sometimes relevant to the original FreeBSD port, so there’s benefits for everyone here.
What next?
Try it now if it has all the packages you need, or wait for a binary repository to be created to speed things up. Remember, this is a new project, so a willingness to deal with problems and contribute to fixes is necessary.
Stéphane Russell, on the users@ mailing list, pointed out an in-depth article about DragonFly’s 3.2 release, on linuxfr.org. It’s in French, which means I’m just going to have to trust his word about the contents.
Venkatesh Srinivas and Tim Bisson have been working for some time on a port of FreeBSD’s virtio and virtio-block drivers. (see man page commit) They’ve now been committed. This should make your virtual disk perform better, if nothing else.
If you recall, Phoronix recently ran a bunch of benchmarks on DragonFly. One spot that didn’t look good was the “Himeno Poisson Pressure Solver”. I’m no closer to knowing what capability it actually tests other than itself, but Alex Hornung, Matt Dillon, and Venkatesh Srinivas figured out that cache coloring was the missing ingredient. DragonFly now scores the same as Linux.
Tangentially related, this cache coloring is happening in nmalloc, which is now used on 64-bit DragonFly systems. The previous one, dmalloc, had problems in long-running programs.
John Marino has been working for some time on a project he calls, ‘DPorts’. You may have noticed his recent commits for it. He wrote up a summary on users@ to explain what he’s doing. It’s translating FreeBSD ports to DragonFly in a way that appears to be (relatively) low-maintenance. It only works on DragonFly 3.3 and up and you can’t use it at the same time as pkgsrc.
Most interesting to me, it gets rid of the quarterly release chase that happens with pkgsrc releases. Since it’s primarily a binary install system, packages are only upgraded when the results are known to work.
Ishan Thilina asked for some project ideas, and Samuel Greear gave a list of links that may be useful for anyone looking for a project of their own. I offered strategy. It didn’t work out, but this information’s still useful.
Adam Sakareassen submitted a patch for AVX support for 64-bit DragonFly, and Alex Hornung has committed it. If you’re like me and have only the vaguest idea what AVX is, it’s a set of processor instructions added by Intel to Sandy Bridge and later CPUs.
There’s a short thread running on the DragonFly users@ list about disk encryption; there’s some descriptions of encryption work there for the curious.
I’m not sure what IFQ stands for, but Sepherosa Ziehau’s added it. It appears to be based on an idea from Luigi Rizzo called ‘netmap‘. In this case, network packets are grouped together before being placed onto the network interface’s hardware queue. That means better packet per second performance without a corresponding increase in CPU usage, as Sepherosa Ziehau’s report lists, along with needed sysctls.