How long does it take to build all 24,000 packages in the DragonFly ports collection? Apparently about 22 hours on a dual Xeon machine (with I think 36 cores) or 48-core Opteron. This is with synth. I used to measure pkgsrc builds in weeks.
I don’t know how I ended up with 3 pfSense items to lead with – it just happened.
- pfsense 2.3.x passive ftp.
- PFsense DMZ on ESXi.
- Assistance with routing issue with pfSense VM.
- FreeNAS: Open Source Storage Operating System. (via)
- User manages to get OpenBSD and FreeBSD working with Libreboot. (via)
- HardenedBSD switches to LibreSSL in base as the default crypto lib. (via)
- BSD Question.
- Hardened Operating Systems.
- Performance Improvements for FreeBSD Kernel Debugging. (via)
- SNI support added to libtls, httpd in -current.
- Cover reveal for “PAM Mastery”.
- DiscoverBSD for 2016/08/22.
- Synth – A simple, fast drop-in alternative to 3Ps: Portmaster, Portupgrade, and Poudriere (for FreeBSD and DragonFly). Surely you knew of this already? (via)
There’s been multiple reports of pulseaudio causing problems for DragonFly users. It would get pulled in as a dependency, and audio would suddenly stop working. Uninstall, and audio is fine. John Marino has removed it from dports, to prevent that exact problem.
Because this always happens just after I create a DragonFly release, there’s a new version of OpenSSL. However, this is for version 1.0.2. 1.0.1 is what’s in the release, and it’s supported through the end of the year.
OpenSSH has a major version bump in DragonFly, to 7.3p1. This means some features – specifically patches for High Performance Networking – are no longer there, and you’ll get an error if your config file requires them. Either remove the options from your config, or install OpenSSH from dports.
If you happen to be testing kernel modules, DragonFly can now load them from a modules.local directory. This keeps modules that aren’t part of the base system, separate. This is probably of most use to developers. It’s controlled by local_modules being set in /boot/loader.conf, and defaults to on.
(Updated for correct file location – thanks, swildner)
I see this bite people irregularly over the years: if your default shell on login can’t run, what do you do? I’ve seen it happen because of a missing /usr/lib, and it can happen with out-of-date library references, too. There’s several different ways to deal with it:
- Run a shell that can’t have this problem, like /bin/tcsh (the root default).
- Or, rebuild in single-user mode from the console.
- Or, perform the bullet-proof upgrade.
That last one may be useful if your dports setup gets mangled, somehow – though ‘pkg upgrade’ has always worked for me.
For those of you running DragonFly-current, the already-mentioned library privatization going on means that ports have to be rebuilt. You will want to do it yourself, or wait a little bit before upgrading if you want to install binaries.
That’s a pretty cryptic headline, isn’t it? John Marino has ‘privatized’ several libraries in DragonFly, so that they can’t get included involuntarily as part of a port build. That may mean you will need to perform a full rebuild of your system if you are tracking DragonFly-current.
(This is the way to fix ‘system’ languages like Perl was in FreeBSD 4.x – keep them clearly separate from the port version. It’s about a decade too late for that idea to work out, though.)
John Marino has opened up his new utility for testing: Synth. It’s made for building custom package repositories, similar to poudriere, but much less setup work. If you’ve ever said “I like binary installs, but I want my own build options”, this is for you. The README includes screenshots to show all the things it can do.
John Marino has created two custom make variables – .MAKE.DF.OSREL and .MAKE.DF.VERSION. (They return the current DragonFly versioning, if you can’t tell from the name.) Apparently, if you build all 22,000 or so ports together, about 15% of the total time is just awk looking up the system version, and this removes that repeated task.
I am taking this moment away from my significant backlog of things to post to note that there have been a lot of games fixes in DPorts lately. Thanks to Rimvydas, many small bugs that kept games from compiling on DragonFly are now fixed. The easiest way to see is to look at the commits from December 8th and back, but the best way is to pick one and play.
Since DragonFly 4.4 has been branched, bleeding-edge DragonFly is now at version 4.5. As John Marino detailed in his post, that means pkg on 4.5 systems will look in a new place for downloads. (“dragonfly:4.6:x86:64”, since it always uses even numbers)
To cover for this, set ABI to point at DragonFly 4.4 packages in pkg.conf for now. They’re freshly built and functionally the same, anyway. Once there’s a 4.6 download path, that ABI setting can be removed. Packages for DragonFly-current are available now and probably at the mirrors by the time this posts.
Update: as John Marino pointed out to me, anyone on DragonFly-master who upgrades now will be at version 4.5. This means pkg will get the new (4.5) packages on the next pkg upgrade. That means a mix of old and new packages unless you either reinstall anything (pkg update -f) or hardcode the 4.4 download path until you are ready to switch everything.
So: DragonFly-current users should either hardcode the 4.4 path for now or force an pkg upgrade for everything. DragonFly 4.2-release users are unaffected.
If you happen to still be running DragonFly 4.0 – that’s two releases ago and not supported – you may be noticing less ports are building. There’s been enough significant changes in DragonFly since that release that it’s reducing the number of buildable ports.
DragonFly 4.0 to 4.2 is not a difficult jump, so jump when you can. The converse of this, of course, is that there’s even more building on 4.2 and DragonFly-current.
Matthew Dillon posted an extended description of how to run Firefox in a way that completely locks it away from your user account. As a side effect of this, the current crop of dports binaries has been updated.
Some time ago, I acquired a Chromebook with the help of all you kind readers. Here’s a mini-report on how DragonFly works as a desktop.
The hardware: what I have is an Acer c720 Chromebook. The C720p is the touchscreen model, and is equally well-supported by DragonFly. A larger-capacity M.2 SSD (which is relatively easy to install) is the only real need, as the installed one is only 16G. It’s easy enough to see what the laptops look like; it’s nothing fancy but it’s suitably light.
The software: There’s a wide-ranging and complete install/tweak guide for the c720 and c720p on the DragonFly site. Note that it goes down to the point of even changing the keymap for the special keys on the keyboard.
Things I don’t like:
- The mousepad needs a physical click, not a tap, which decreases accuracy.
- There’s only 2G of RAM, and not expandable. You will notice this if you tend to open a lot of tabs when web browsing.
- I’ve had mousepad trouble, but I’m the only one reporting it, so I think it’s just bad hardware luck on my part.
Things I do like:
- pkg is a godsend, making installation and upgrades almost effortless. I’ve gone binary-only so far.
- Many things Just Work – for example, the xfce4 battery plugin.
- xscreensaver works great; even the 3D modules. I don’t know why it entertains me so.
- I haven’t run the battery out to make sure, but it looks like it would last a few hours. Suspend/hibernate are not supported, but low power modes are.
- There’s a lot of multi-touch shortcuts built into the touchpad.
It’s an excellent BSD laptop, for light use, at low cost. The next step up would be into Thinkpad territory, which raises the cost or increases the age – and may not be as consistently supported.
The other day, I updated some packages using pkg. The default version of PHP went from 5.4 to 5.6. I ended up doing what /usr/dports/UPGRADING says and making a list of all PHP packages on my system, before removing PHP and its dependencies. I then reinstalled the packages that used PHP, bringing the needed packages back in at the right version. pkg 1.4 didn’t handle the transition cleanly, unfortunately. I also had to specify mod_php56 because pkg was trying to get the 5.4 version despite it not being default.
None of these are insurmountable problems, but it never hurts to be forewarned. pkg 1.5 is on the horizon and may have an easier time with sorting these types of dependency/version changes. This may apply to FreeBSD in addition to DragonFly.
Well, might rather than will , but I had to make a music reference. There’s a bug in versions of pkg from 1.4.6(ish) to 1.4.11 that can make it accidentally delete itself while updating packages. If this happens to you, there’s an easy fix, as posted to users@:
# cd /usr && make pkg-bootstrap
Once you’re on version 1.4.12+, you’re fine.
Do you remember the BSDNow story a while ago about a Tanzanian community effort using FreeBSD to build a library? They’re looking at DragonFly, too, because of the low resource requirements. From that discussion: a hardware reason for an ‘indefinite wait buffer’ error, and a note on how to most efficiently download packages for multiple machines.
BSDNow’s episode this week focuses on the just-released Bitrig 1.0, and has an interview with Patrick Wildt of that project. There’s also coverage of other topics, including the new poudriere release – that’s the tool that bulk builds packages for DragonFly and FreeBSD, though I don’t know if it’s unified across both operating systems yet.
This week’s BSDNow episode, 8,000,000 Mogofoo-ops, includes an interview with Brendan Gregg of Netflix, along with more recent convention video links. It also mentions GNOME3 working on FreeBSD – it’s working on DragonFly too.
Despite my complete lack of good planning, John Marino and Francois Tigeot have packages available for the DragonFly 4.0 release candidate that I assembled. Point at this directory to use them.
There’s been so much work in DragonFly recently that makes a desktop easier (i915 support, dports, and so on), that I decided to resurrect an older Dell machine and use it as my desktop.
The Dell that I’m using is a leftover from someone else’s workplace; it’s 7 years old, and has “only” 4G of RAM and a Core 2 DuoE6600 CPU in it. It works, however.
Setting up DragonFly and installing xorg and so on is pretty straightforward. Using dports makes it crazy quick to add all the packages. I went for XFCE4 because I could. Starting X gave me some trouble at first; the default config couldn’t find the mouse and would eventually crash.
Running ‘X -configure’ created a xorg.conf file I could edit, and these lines in /etc/rc.conf gave me a working mouse:
moused_enable="YES" moused_type="auto" moused_port="/dev/ums0"
The crashing problem with my radeon-driven video card was fixed by turning off the acceleration – uncommenting this line in xorg.conf did it:
Video performance isn’t as nice as I would like it with acceleration, but this is an older machine anyway.
I couldn’t get sound working. Francois Tigeot has a branch of DragonFly that contains newer sound drivers brought over from FreeBSD, here:
git://leaf.dragonflybsd.org/~ftigeot/dragonfly.git (pcm_2014_september branch.)
It doesn’t support device cloning, so I can run Youtube videos and XMMS, but not audio from both at the same time. (for instance; not that you’d want to do this other than by accident)
I installed x11/webfonts, and web pages look a bit better after changing my default font preferences.
And… that’s about it. It’s a working desktop. Digging up a half-height video card that has working acceleration is a next step, but I can’t imagine that’ll be expensive. I wish I had done this a long time ago.
Why is it so warm out? I want autumn to start.
- BSD compared to Linux, an explanation.
- A description of what rcctl does. (via several places)
- NetBSD runs on the OpenRISK 1000.
- tmux-resurrect, making tmux survive machine restarts.
- OpenBSD version numbering explained.
- PC-BSD has its own subreddit.
- Lumina is now available as a port – will it work on DragonFly? Someone try!
- DiscoverBSD news for 2014/09/01.
- NetBSD 5 systems now use modular xorg.
- The 2014Q3 pkgsrc freeze is coming up.
- PXE installs of OpenBSD with Serva.
- Are you a “connoisseur of old time stamps“?
- FreeBSD has some support for the Altera SOCFPGA.
- Your cross-pollination moment of the week.
- Yes, it runs (Retro)BSD.
It seems pkg 1.3.6 was slightly scrambled. If you happen to have built and installed it, John Marino has special instructions on how to update to 1.3.7. If you are on DragonFly 3.8, you can follow those instructions now, and if you are on 3.9, that repo should be ready for an update in the next few days.
DragonFly’s using pkg 1.3, at least on master, and I’ve seen a few people report an error message when performing ‘pkg upgrade’. The error message usually includes something like:
pkg: need to re-create repo Avalon to upgrade schema vers
If you get this, do ‘pkg update -f’ and it will complete.
Here’s a nice advantage for dports and DragonFly: since it’s an overlay on FreeBSD ports, it’s possible to move to newer or different versions of software without waiting for it to happen in FreeBSD. For example: there’s a newer version of the xorg intel driver now in dports – newer than what’s in ports.
The dports binary packages built for DragonFly 3.4 are removed. If you have a 3.4 system, you can build from source, or preferably just upgrade. Note that the 3.4 release images are still out there if needed.
If you’re building ports, it will treat OpenSSL as a dependency and bring in whatever version is available. If perhaps you want to use the version of OpenSSL installed as part of your base system, Robin Hahling has the answer for how. (This probably works on FreeBSD too.)
Remember: If you have a particular port that’s not building in DragonFly, there may be a patch in pkgsrc that could be brought over, as John Marino points out.
If you are upgrading packages on your DragonFly 3.6 system, and you have docbook installed, there’s an extra step needed because of the moving around of several docbook packages. If you don’t have docbook installed – nothing to see here.
We’ve got Go builders running for DragonFly, but nobody actively maintaining Go itself on DragonFly. The dports version builds, but there’s a Go release coming up and having native support would be much better than relying on chance FreeBSD build compatibility.
The current error as I type this is a TLS problem that sounds like a simple fix, if only I knew where it was.
Brad Fitzpatrick showed up on the users@ list and mentioned that for DragonFly to be supported in Go, it needed to show up in the Go Dashboard with building reports. I now have the Go builder running on pkgbox32/pkgbox64.dragonflybsd.org. Check the builder page to see status.
Note: Installing the port of Go from Dports works just fine; this is the mechanism for testing Go on a per-commit basis for the people who work on Go – so a ‘fail’ notice on the builder page doesn’t necessarily mean anything, unless you are developing Go itself. This may already be clear to you.
Here’s how my upgrade from DragonFly 3.4 to 3.6 for this server went.
The system install went normally. I rebooted before performing ‘make upgrade’, as noted in UPGRADING and elsewhere.
I already have dports installed, so a binary upgrade should be possible. I had heard of people with older version of pkg, having trouble getting it to notice upgrades. I rebuilt pkg, and ran ‘pkg upgrade’. A number of the updates coredumped. Here’s one example:
[156/160] Upgrading gtk2 from 2.24.19 to 2.24.19_2...Segmentation fault (core dumped)
After the upgrade, I had two problems: PHP wasn’t working for the website, and some programs would segfault.
The random segfault was fixable by forcing a binary upgrade of all packages. Since there were some programs on the system that were still new enough that the version number was the same as on the remote repository, pkg didn’t upgrade them. Those packages were linked against old versions of system libraries that predated the locale changes in DragonFly 3.6, so they’d crash. Forcing the update for all packages fixed the issue.
The other problem, PHP on the web server, is not new to me. The binary package for PHP does not include the module for Apache. The solution is to build from source with that option selected. I understand that pkg is destined to support (some?) port options in the future. There’s also an immediate workaround for locking it.
However, the port would not build because of a security issue. The binary package installed without any warning. This, I am told, will change to pkg giving you the option to install if you are aware of the security problem, and whether it really affects you. (which is just what I want, yay!)
Anyway, other than the system changes biting me because I didn’t realize some packages weren’t updated, it went very quickly. That is the reason for binary updates through pkg, or at least a major one.
Odds and ends for the quieter holidays.
- Hubert Feyrer spotted this video interview of Amitai ‘schmonz’ Schlair about NetBSD.
- OpenBSD has tmpfs.
- PC-BSD has made it through a pkg upgrade.
- pkgsrc is frozen until at least the end of the month, for pkgsrc-2013Q4.
- OpenBSD wants to shift electrical costs. (via)
- The DiscoverBSD weekly roundup.
- Managing custom ports. (can apply to dports too)
- Building tcsh on 4.3BSD-Quasijarus. This led me to…
- 4.5BSD. An ambitious project.
- A pfSense video review.
- Steryana Shopova is this past week’s Faces of FreeBSD.
- OpenBSD had a head start on not trusting RNGs.
- OpenBSD has a new vioscsi(4) driver.
- Michael W. Lucas’s books are available through OpenBSD.
- FreeBSD Kitten. (via NYCBUG)
I had a sometimes-great, sometimes-difficult trip to New York City over the past few days, and while I was there, I met the ball of energy that is George Rosamond of NYCBUG (which is having a huge party right now.) He and I talked for a bit about various aspects of the BSD ecosystem, and one thing he noted was that people aren’t generally aware of all the licenses in use for the different software packages on the system, or even the individual licenses in the system files.
There is an ACCEPTABLE_LICENSES setting in pkgsrc, where software licensed under terms not in that list won’t install. That’s useful, but frustrating, because it keeps people from getting what they asked for – a software install. Something that would be useful – and it could be cross-BSD very easily – would be a license audit summary.
There’s meta-data on every package in FreeBSD’s ports and DragonFly’s dports and pkgsrc and OpenBSD’s port system. Why not say ‘pkg licenses’ in the same way you can say ‘pkg info’, and get a summary of the licenses you have installed in the system? (or pkg_licenses, etc. You get the idea) This wouldn’t prevent people from installing software, but it would give a very quick view of what you were using.
> pkg licenses
Software package License
foo-2.2.26 Apache license
It could be extended to the base system, but I’d like to see this in all the packaging systems as a common idea, in the same way that ‘info’ in a packaging command always shows what’s installed.
Rett Kent has volunteered for maintaining i386 support under dports. Good luck! 3rd-party software management is difficult.
pkg 1.2 is coming out. This brings a number of new features, but as John Marino posted, you may want to delete your old pkg.conf to keep the new version from complaining about an old config file. This upgrade is a step on the way to signed packages, which is a Good Idea.
If you’re upgrading dports (and you probably are if you are going from DragonFly 3.4 to 3.6), there’s a minor issue in dports, inherited from FreeBSD ports: you need to manually remove perl before upgrading. It’s all of one command, so it’s not a huge burden. Joris Giovanngeli spotted it first.
John Marino isn’t interested in supporting the i386 architeecture for DragonFly and dports, so he’s not going to actively work on it. (Packages for DragonFly 3.6 are already built, so that’s not a problem for release.) If you feel like taking on a significant but interesting workload, check his message about the work involved.
Matthew Dillon wrote a roundup post summarizing all the changes he’s made to DragonFly to improve SMP performance in the last few weeks. He’s removed almost all contention from DragonFly. This means better performance, scaling upward depending on the number of processors.
‘monster’, the system that builds all 20,000 items in dports, can complete the run in 15 hours. Compare this to the 2 weeks it used to take me to build the 12,000 packages in pkgsrc. This is admittedly on different hardware and different packaging systems, but it gives a sense of the scale of the improvement.
As a followup to news that the git feed of pkgsrc through dragonflybsd.org is not being updated, Max Herrgard wrote out how to fetch pkgsrc via CVS, or tarball, or another git feed. CVS is still the ‘official’ way.
DragonFly has generally shifted over to dports for 3rd-party software management, away from pkgsrc. Because of that, I haven’t been building binary packages of the quarterly pkgsrc releases. Pierre Abbat asked why on users@, and here’s my explanation of the change.
Since there’s a newer set of dports binary packages uploaded, I thought I’d spend my weekend upgrading, to catch up.
And that was it. Well, not really. I had to dump and restore my Postgres databases, cause of the switch from 9.0 to 9.2 as default. I had to build php5 from source to get the Apache module. Those two things together took longer than the entire download and upgrade of the rest of my system – some ~200 packages?
Michael W. Lucas wrote a blog post about pkgng and Ansible on FreeBSD. Will it work on DragonFly? We already have pkgng on DragonFly in the form of dports, and Ansible… might work? Please, someone try.
In part of a long thread about dports packages on the users@ list, Matthew Dillon notes that a new set of packages for i386 and x86_64, for 3.4 and for “3.6” (meaning bleeding-edge DragonFly, even though that’s numbered 3.5) is mostly uploaded. He also notes that a Haswell-processor-based blade server for DragonFly is in the works, so much of the dragonflybsd.org infrastructure is going to move from his house to a datacenter, with the benefits that provides. It’ll also help automate binary package building.
It looks like OpenJDK7 works in pkgsrc for DragonFly, thanks to Ryo ONODERA, and I think it’s working in dports too.
FreeBSDNews.net has a nice summary up of video from all (?) the presentations at BSDCan 2013. Of particular interest to DragonFly users: a video about pkg, the tool used for package maintenance in dports. In this presentation, it’s talking about use on FreeBSD, but the future stuff applies to DragonFly too.
Since dports uses FreeBSD ports as a base, adding something to FreeBSD ports means it will show in dports, too. However, it doesn’t have to go that way. It’s possible to have dports packages that exist only in dports. If you have changes to a port that make it compile on DragonFly, that can be added too. For all of that, go to the dports issues page on GitHub.
I pointed out in my converting-to-dports post from yesterday that I had to download dports and build pkg by hand in order to install binary packages. This was because my DragonFly system was upgraded from 3.2 to 3.4 and therefore didn’t have pkg installed.
John Marino has added a ‘pkg-bootstrap’ option to /usr/Makefile, for fixing exactly that problem. It downloads a static version of pkg, which then lets you upgrade to the full pkg and install binaries as you’d expect.
I changed shiningsilence.com over from pkgsrc to dports over the last 48 hours or so. Here’s how it went, in a series of bullet points:
- I had to download dports source and build the pkg tool by hand; since this system was upgraded from DragonFly 3.2 to DragonFly 3.4, pkg wasn’t automatically present as it would be for a new installation.
- I took the output of ‘pkg_info’ and culled it down to the applications I knew I used, and that formed my ‘to-install’ list for dports. That worked in a very straightforward way.
- It took so long mostly because of two things: I was also dealing with an email problem at my workplace, which usually took precedence. Also, I had several applications that I had previously installed by hand and needed to reconfigure to work as a dports item.
- Installing from binaries is really fast! Really, the dports part of this was possibly the most brief.
- The only thing I needed to compile from source was php, in order to get the Apache plugin. I’m sort of surprised the option isn’t on by default.
- Using ‘pkg search packagename’ is a good idea, because ‘pkg install’ can pick up multiple versions of a package. e.g. ‘pkg install mysql-server’ selects mysql-server51, mysql-server55, and mysql-server56. You probably don’t want to install all three. Or even one, depending on your opinions.
- Overall, it went more easily than I had expected, given it only had half of my attention.
I’m switching this server from pkgsrc to dports. No post while I fight with old, stale configs and etc.
Are you using it and unable to upgrade to KDE4 for a specific reason other than aesthetic preference? You should check this thread about support for 3.5, at least in dports.
There’s more download statistics on dports and pkgsrc packages, from Francois Tigeot. There’s a heck of a lot of dports activity, though there’s probably much more pkgsrc building from source than this would report on. So, not necessarily representative of actual numbers, but an interesting ratio none the less.
New builds of dports have been uploaded and updated, for x86_64 and i386. (x86_64 was already done; I linked the note about i386) This means you can change PACKAGESITE in /usr/local/etc/pkg.conf to point at LATEST instead of RELEASE and get newer packages. ‘pkg upgrade’ is all it takes, with dports.
‘william opensource4you’ posted a summary of the steps he took for setting up a DragonFly system with XFCE4, using dports. It’s pretty straightforward, and thanks to dport’s binary nature, should be exactly reproducible.