Posted on
How much swapcache can help (with graphs)

I love graphs.  Jan Lentfer made some! Both of these show recent speed improvements in DragonFly – especially some spectacular results from swapcache(8) and the recent NCQ tagging improvements.  (Note that only the third graph represents the NCQ improvements; the first two graphs were done before.)

The first one is a comparison of pgbench running on the same hardware twice – once with the 2.8 release of DragonFly, and once with a recent 2.9 version.  2.9 is definitely looking to be faster than 2.8.

Next up is a 2.9 system run with and without swapcache, showing an astounding difference between the two.  It’s pretty clear just how much performance improvement you can get from swapcache…  (see Jan’s notes on the setup after the graphic.)

 

Jan’s notes, from EFNet #dragonflybsd on IRC:

15:08 < lentferj> these are SELECT-Only tests
15:09 < lentferj> JustinS: it’S important to note, that the database is 2,5x
bigger than RAM on the swapchache test
15:11 < lentferj> JustinS: I did a Select-Only ramp-up of 30 minutes to get
caches and swapcache filled
15:12 < lentferj> JustinS: and then I ran
15:13 < lentferj> for i in 1 2 3 4 6 8 12 16 24 32; do
/usr/lib/postgresql/8.4/bin/pgbench -U pgsql -h atom -s 400
-S -c “$i” -T 600 pgbench; done
15:13 < lentferj> so, select only pgbench for 10 minutes each
15:13 < lentferj> with increasing numbers of client
15:14 < lentferj> pgbench on another box, 100MBit switched network
15:15 < lentferj> JustinS: the first graph (2.8.2 vs current) is the same w/ a
database that fits in RAM entirely
15:15 < lentferj> so measuring concurrency performance (w/o I/O)
15:17 < lentferj> the swapcache comparison was on a 2GB box with a 5GB database
and 16GB swapcache (INTEL) attached to a sili card
15:17 < lentferj> on a atom 330 :)

Now, here’s testing with the recent NCQ tagging update for AHCI:

These results are astonishing.  Please, someone compare with other operating systems!

Here’s the stats for this last test:

 

  • 5.6GB database, system w/ 2GB RAM –> io benchmark
  • pgbench with increasing no of client 1->32, SELECT-Only Mode
  • sili controller Dawicontrol DC-3410 SATA PCI controller which is using a Silicon Image 3124-2 chip
  • 2 Seagate Barracuda ES.2 250GB SATA II disks
  • lvm stripe over those disks
  • postgresql.conf is default, except shared_buffers set to 512MB and effective_cache_size to 1024MB
  • atom330 on a Foxconn mobo
  • SSD is SATA INTEL SSDSA2M040 2CV1

 

Posted on
Lazy Reading for 2011/04/13

Get out your wallet!  I encourage purchasing here.

  • You should buy a SSD.  Not necessarily news to you, but that article does a good job of summarizing why.
  • On the other hand, SSD prices are already on their way up/availability is way down.  Japan’s disasters are having a ripple effect through the high-tech supply chain.  Either buy immediately or get ready to wait for a while…
  • Introduction to Architecting Systems for Scale – you either don’t care, or find scaling questions immediately engaging.  I am one of the latter, so here’s the link.
  • I’ve been watching pkgsrc-changes@netbsd.org for a little while.  One thing I’ve discovered: there’s a lot of updates going on!  Another thing that’s nice to see: DragonFlyupdates, including ones that help with our move to gcc 4.4.
  • Aw, no more Kermit.  (via)  Not that I have a use for it at this point, but still: aww.  I bet in about 10 years I’ll say the same thing about… gopher?  Remember that?  It’s not even supported in Firefox 4 now, which kinda makes me feel sad.  And old.
  • Server plans: Facebook vs. Google.  (warning: Facebook article is somewhat giddy.)
  • The infinite hard drive.  (via I lost it, sorry)

Here’s an extra little thing: next time you’re dealing with dusty computer equipment, remember this picture:

That is what happens to an exposed RJ45 port after a few years in a salt mine (my employer).  This was inside an enclosed, mostly-sealed  structure, too.

Posted on
Swapcache updates, long-term status

Matthew Dillon made some changes to swapcache(8).  Swapcache is now able to cache a lot more data, and the result is that general disk performance for _all_ disks is accelerated by an included SSD using swapcache.  Performance previously restricted to all-SSD systems or serious RAID setups is now possible with much less investment.

In addition to that, the long-term wear on the SSD appears to be less of a problem than expected.

Posted on
pkgsrc-2011Q1 details

I already noted that the quarterly release is out, but the pkgsrc-2011Q1 release announcement is available now.  There’s good reasons to link to it – the list of updated packages, new packages, and credits for the work people have been doing.  Here’s the part I really want to pick out:

We’re aiming to make this the last branch to support non-DESTDIR packages. We have almost finished the transition to DESTDIR installation, where a staging directory is used to make a binary package, which is then managed by the pkg_install tools.

The reason I’m highlighting this is: it’s good news!  One of the long-term complaints with pkgsrc is that the upgrade process is painful.  If you try to build an upgrade and the build processfails after uninstalling the existing package, not only are you not getting the upgrade, but you’ve lost the existing package.  Binary packages for download helps with this (and generally is faster), but only so many packages can be built separately and made available for download.

Building a package separately and then installing from there removes these issues.  No binary redistribution issues, actual downtime is minimal, and the package is known to work when an upgrade happens.  This removes most of the problems I’ve heard raised about pkgsrc over the years.