New mechanism: kcollect

There’s a new facility in DragonFly: kcollect(8).  It holds automatically-collected kernel data for about the last day, and can output to gnuplot.  Note the automatic collection part; your system will always be able to tell you about weirdness – assuming that weirdness extends to one of the features kcollect tracks.  Here’s some of the commits.

14 Replies to “New mechanism: kcollect”

  1. Anonymous says:

    I wonder why Matt thought it was necessary to add this functionality.

  2. Pedro says:

    Sounds misguided … ignorance of the DTrace way.

  3. Anonymous says:

    Pedro. Can you explain more. I’m not following.

  4. Does DTrace do the same thing?

  5. Pedro says:

    DTrace has specific probes and is more fine-grained: it lets you know exactly what the kernel is doing. More importantly, it carries its own language and the interface is already popular in Mac, Solaris, FreeBSD, and to some extent linux and NetBSD.

    Brendan Gregg is famous for his approach measuring performance and his flamegraphs.

  6. SolarFlame says:

    @Pedro

    those probes only fire when you’re tracing for them, kcollect runs all the time.

    I think it’s better to compare it with Solaris’ kstats. OTOH DFly is already collecting stats via sysctl’s, so why another mechanism like kcollect?

  7. mer says:

    sysctl’s may be already collecting data, but it sounds like kcollect is a “periodic snapshot” of some sysctls, aggregated and presented via a plottable interface. Basically, saving you the trouble of writing a userland that periodically polls a bunch of sysctls and doing the same thing. A quick glance at commit messages at least one mentioned “lockless”, which not all the sysctl interfaces are.
    DTrace is good, but again, requires you to do something in userland to collect and present the data.

  8. Pedro says:

    @SolarFlame

    Dtrace is also lockless. Yes, you have to collect the data but you can do the same with much better detail and no performance impact.

  9. Jenkins says:

    Serious question. Do people find DTrace useful and a good tool?

  10. For what it’s worth, I think DTrace and kcollect are solving two different problems. kcollect is very light and was written in an afternoon, as far as I can tell, to assist with problem diagnosis in as automatic a way as possible – autocollection, graph generation, etc. DTrace is very capable but it’s a porting effort and a different approach.

    I saw this without directly interacting with either right now, so I could be wrong.

  11. Kacy says:

    On the topic of system load, a Netflix engineer has a great blog post today on what system load even means and how to interrupt kernel system data.

    http://www.brendangregg.com/blog/2017-08-08/linux-load-averages.html

  12. SolarFlame says:

    @Jenkins

    Yes people do…

    http://dtrace.org/blogs/brendan/2012/08/09/10-performance-wins/

    https://milek.blogspot.de/2012/10/running-openafs-on-solaris-11-x86-zfs.html

    and the all-time favourite question do I need to kldload aesni to make openssl AES go faster?!

    https://crc32c.blogspot.com/2017/05/freebsd-aesni4-and-openssl.html

    This list could go on and on…

Comments are closed.