entr(1); Run arbitrary commands when files change.

Eric Radman sent along a plug for a utility he is working on called entr(1).  The desciption is “Run arbitrary commands when files change.”  The site for it has several nifty examples – run make when *.c files change, or convert Markdown files to HTML as soon as they are modified.  The really nice thing about it is that it’s perfectly BSD-friendly, and uses kqueue, but will also work on Linux.  This beats the “This runs on the one flavor of Linux I use, in one particular shell!” approach I’ve seen from some other developers.  See the reddit discussion of it for comparisons to inotify.  No, it’s not in pkgsrc/ports yet.

Update: And thanks to Thomas Klausner, it’s in pkgsrc as sysutils/entr, and in ports as sysutils/entr thanks to Eitan Adler.  You have no reason not to try it now.

Lazy Reading for 2013/03/31

I hope you like reading; there’s some very meaty links this week.  Go get a cup of tea and settle in.  You drink tea, don’t you?  You ought to.

  • Reading about KDE’s repository near-meltdown makes me think we need more checks for DragonFly.  We have the advantage of Hammer, of course, which would help in the same way that the linked article names ZFS as a ‘fix’.  (via multiple places)
  • We know that Apple will reject apps it disagrees with.  Google also will do so.  Has there ever been a program rejected from pkgsrc or (FreeBSD/OpenBSD) ports on content grounds?  Not that I know of – anyone remember differently?  I’d argue that’s a favorable point for the BSD packaging systems, though it may just be that no application has tested those boundaries yet.
  • Portscanning all IPv4 addresses on the planet.  Possibly the largest distributed effort ever?  The detail in the maps and returned services is especially interesting.  (via)
  • Scale Fail, a Youtube video of a 2011 talk about screwing up your services.  Mostly about the humor, but the underlying points are valid.   (via #dragonflybsd IRC)
  • There’s still improvement possible to fsck, apparently based on this.  That’s UFS2 fsck.
  • What is your most productive shortcut with Vim?  A very thorough explanation of verbs, marks, and registers.  Holy cow, I wish I had known about ‘: … v’ before.  It’s long, but worth it.  (via)
  • Matthew Garret’s description of Secure Boot vs. Restricted Boot with UEFI, (via a coworker who went to Libreplanet 2013).  I’m still not sure what DragonFly will need to do about this.
  • I missed mentioning this earlier: 20 years of NetBSD.  We’re coming up on 10 soon.
  • Dragonfly drones.  Unrelated except for name.
  • That guy who starts to froth madly every time BSD is mentioned on Phoronix is still there (see comments).
  • Mainframe computer supercut.  (via)

Your unrelated comics link of the week: Tom Spurgeon of the Comics Reporter asked people for their lists of webcomics that could go in a ‘Hall of Fame’.  The resulting list is a lot of really, really good material.  Go use up a few hours reading.

Lazy Reading for 2013/03/03

I am all over the place with links this week – some of them pretty far off the path.  There’s a lot, too, so enjoy!

Your unrelated link of the week: I’ve already been offbeat enough in this Lazy Reading; I don’t have anything else.

3 very different commits

Here’s 3 recent and different commits to DragonFly that I’m commenting on all at once:

  1. Peter Avalos upgraded libarchive in DragonFly to 3.1.2, with a note of the changes.  An ordinary and appreciated update.
  2. Sascha Wildner updated the ISO639 file to include the newest update: “Standard Moroccan Tamazight”.  There’s no particular utility to that; I just like saying “Standard Moroccan Tamazight” out loud.
  3. Work on poudriere, the utility for bulk-building DPorts packages, has caused some nice speedups for DragonFly in extremely stressful situations.  See one of Matthew Dillon’s recent commits.

I really wish the other BSD projects would include commit lines in the mail message subjects, so it was easier to catch things like these.

An early DPorts education

John Marino’s DPorts project, mentioned here briefly before, is interesting.  I had two separate people ask me how it works, so a better explanation is in order.  I’ve tried it out on a test machine over the past few weeks.

Background:

Dports is an effort to use FreeBSD’s ports system as a base for DragonFly, and the pkg tool as a way to manage binary packages built from DPorts.  This is complicated, so I’ll explain each part in order.

  • FreeBSD ports are a FreeBSD-specific collection of software installation files that automate building 3rd-party software on FreeBSD.  You’ve probably already heard of them.  (Note there’s no mention of DragonFly.)
  • DPorts is a collection of files that map to existing FreeBSD ports, and contain any changes necessary to make that port also build on DragonFly.  Many of those programs build without changes on DragonFly.  DPorts builds from source.
  • pkg is used for package management, and is usable on FreeBSD and on DragonFly.  The binary packages produced from building with DPorts can be installed from remote locations and managed separately using pkg, so that software upgrades and installation can be performed with binaries only.  (It’s much faster that way.)

Every port seen in DPorts is known to build on DragonFly.  John Marino adds a port only after it builds successfully, using poudriere as a bulk software tool.   Ports are only updated to a newer version when that newer version builds, too, so once something arrives in DPorts, it should never break from being updated at some point in the future.

Installing:

To use DPorts, you need two things:

  1. DragonFly 3.3 or later, though 3.3 is the most recent right now.
  2. You need to rename /usr/pkg so that your existing pkgsrc binary programs don’t get accidentally used while working with DPorts, causing confusion.  If anything goes wrong with DPorts when you are installing it and you want to go back, remove all the DPorts packages and rename /usr/pkg back to normal.

(Don’t confuse pkg, the management tool, with /usr/pkg, the normal installation directory for pkgsrc. ) For the installation of the base port files:

cd /usr
make dports-create-shallow

If you’ve already renamed your /usr/pkg directory, git won’t be in your path any more.  You can instead download a tarball and unpack it, which also happens to be possible automatically via that same Makefile.

cd /usr
make dports-download

Downloading via git is fastest, so if you do need to use the tarball via make dports-download, build devel/git, delete /usr/dports, and then pull it again with make dports-create-shallow.  This all comes from John Marino’s Github site for DPorts.

Managing DPorts

DPorts doesn’t use pkg_info, pkg_add, and the other tools traditionally seen on DragonFly for pkgsrc.  Instead, package management is done with pkg.   Use pkg info, pkg install, pkg remove, and pkg update to list, install, delete, and upgrade various packages on your system.  Packages built from source or downloaded as prebuilt binaries are managed the same way, using these tools.

See some of the other writing about pkg for FreeBSD for details on how it works.

Since DPorts doesn’t update a package until it gets a successful build, and installations are of successfully built binary packages, upgrades with prebuilt packages should always succeed.  Since they’re binary, they should be fast.  There’s a lot of ‘shoulds’  in this sentence, but these are reasonable suppositions.

What about pkgsrc?

Pkgsrc and DPorts shouldn’t be used at the same time, since one system’s packages may be at different versions but still get picked up during building for the other system.  That’s about it for restrictions.

I intend to try building an experimental release of DragonFly with DPorts, to see if all the right packages can be added, but no guarantees.  DPorts is brand new and does not yet have a repository for downloading packages, so the normal caveats apply; don’t install it on a mission-critical machine, and be ready to deal with any surprises from using it if you do try it out.

What packages are available?

Browsing the Github repo will show you all listed packages.  More complex packages like xorg, openjdk7, and libreoffice install, as does xfce.  Parts of KDE 3 and KDE 4 are in there.  (I haven’t tried either.)  I’m not sure about Gnome, but I don’t think anyone ever is.  There’s no vim, but there is emacs.

That’s just what I see at this exact minute.  It changes daily as more packages are built.  Changes from DragonFly builds are sometimes relevant to the original FreeBSD port, so there’s benefits for everyone here.

What next?

Try it now if it has all the packages you need, or wait for a binary repository to be created to speed things up.  Remember, this is a new project, so a willingness to deal with problems and contribute to fixes is necessary.