14 Replies to “PCI MSI-X support in virtio”

  1. Please don’t take this as trolling but instead as a genuine good natured question.

    So I’m the CTO & co-founder of an early stage startup. A portion of my job is understanding when to use the best tools and what those tools are.

    Since my company is young, I have the benefit of changing the OS we use (which is currently FreeBSD).

    I keep hearing DragonFly is excellent but haven’t heard exactly *why*.

    Given that I’m already invested in the mindset of BSD, a single distribution that contains with the kernel and userland … what merits does DragonFly have that should cause me to switch from FreeBSD to DragonFly?

    Thanks in advance.

  2. In other words we should do a portion of your job for you. Everyone has whys, if you don’t have at least one why, then you are like a newborn and anything will do. Why don’t you use FreeBSD? From there you can try DragonFlyBSD, OpenBSD, Bitrig, NetBSD.. Midnight, Ghost, TrueOS. What exactly do you want? Start from somewhere. GCC or Clang? LibreSSL or OpenSSL? What is the speed of your network? How many cpus? And why are you considering a kernel+userland monolith? There is a series of distributions based on the illumos kernel, for example.

    For me Dragonfly beats FreeBSD because of Dillon+Marino. DragonflyBSD has some amazing devs but these two, especially, are nonFreeBSD commiters, simply because they are banned from FreeBSD.

    Dragonfly is more K.I.S.S than FreeBSD, Dragonfly is more about performance at the end and simplicity from the start, while other BSDs, especially FreeBSD, are more about politics.

  3. @bsd

    With all due respect, I try to base my decisions on Facts and Data.

    The trouble I’m having with DragonFly in general, and your post above is a great example of this, is that people say it’s better “just because”. But you don’t give any concrete examples HOW it’s actually better.

    Yes, Dillon & Marino are awesome.

    But back to Facts & Data, what specifically makes DragonFly better

    – if it’s more performant, where can I read up on those benchmarks (and I don’t mean the benchmarks on the website from 4 years ago)

    – does it have functionality that the other BSDs don’t? If so what?

    – does it support more hardware than the other BSDs? Etc

    The trouble I have is that it’s unclear what goal DragonFly is trying to address.

    OpenBSD focuses on security. NetBSD focuses on cross platform support. FreeBSD feature completeness and performance.

    What is DragonFly?

    Please don’t read my comment as being negative.

    I’m just expressing candid feedback in that I’ve read quite a bit about DragonFly and people say it’s “better” but don’t give concrete data support How it’s exactly better … other than simply saying the developers are superior.

  4. Said another way … if DragonFly existence it solely due to Dillon and now recently Marino being banned to commit to FreeBSD , and no reason more – then please just say that.

  6. Timothy – why are you picking a BSD at all? You noted the combined userland and kernel, but I can’t think of how that’s a competitive advantage for your startup.

    I don’t know what your company does or your goals are or anything about your business other than it’s possibly tech-related. It is very hard to recommend anything to you based on that.

    If you are indeed fact-based and as the CTO of a new company responsible for the technology direction of the company, you should have some test cases in mind.

  7. We’re a FinTech company that is predominantly network constrained, not CPU. As well as concurrency.

    The charateristics of our platforms is

    – averaging ~18M concurrent users.

    – processing ~6.3B network requests. With an average payload of 21kb

    ====

    – FreeBSD & erlang.

  9. Ah, that helps. Network performance on DragonFly has been very good. Sepherosa Ziehau, one of the developers, has been specifically tuning DragonFly to handle very large numbers of connections without overwhelming the CPU. Until recently, he worked for an online video-streaming platform so he had a good test scenario. My suspicion is that you may get better network handling with DragonFly compared to FreeBSD – but that will take actual testing.

    The other question is Erlang. I don’t use it, but in theory whatever is in FreeBSD ports should be in DragonFly dports. That should be easy to evaluate based on looking at whatever libraries and versions of code you have in place.

    Last – and this is the advantage of DragonFly for me – is that it’s a small group of developers who are generally friendly. You are likely to be able to talk to someone about problems if you have them, but there’s no guarantee (with this or any other open source project) that there’s someone who can help you with a specific problem; we’re all volunteers. I’m sure this is not a surprise to you; I’m mentioning it cause it’s important.

  11. Justin – given the amount of work Netflix and WhatsApp has contributed to FreeBSD network stack … do you still think DragonFly would outperform FreeBSD?

  12. As for networking stuffs.

    For short lived connections, e.g. 1 request/connection HTTP/1.1 workload w/ 1KB web object, 2x2620v2 w/ dual 82599, Dfly can do 207Krequests/s, average latency is 50ms, latency stdev is ~80ms, @30K concurrent connections. Plain TCP connection/accept rate is ~480Kconnections/s on the hw configuration I mentioned above; it is improved a bit recently w/ Matt’s VM work.

    Last time I measured, Dfly need to use 16KB web object to saturate 20Gbps network w/ 1 requests/connection and @30K concurrent connections; 8KB web object fails to saturate the 20Gbps network under the same condition.

    IPv4 forwarding rate is ~5Mpps (only counts RXed packets on both ends), w/ 400+ routes in routing table w/o fastforwarding.

    I will do some measurement on FreeBSD-current and Linux (ubuntu 16.10 probably), for my talk on AsiaBSDCon.

  13. Sepherosa Ziehau

    Fantastic post. Can’t say thanks enough for all you do.

    I’m really looking forward to reading about your benchmarking vs other BSD and the result.

    So much has been improved in 4.x yet I haven’t come across many general benchmarks of Dragonfly vs other BSD … except for the outdated 3.2 benchmarks

    https://www.dragonflybsd.org/performance/

  14. Advait – I have not been tracking what Netflix/WhatsApp have contributed, so that’s unanswerable. Following your other question – benchmarking is a lot of work and difficult to do well. If you have a specific test you want to try, it’s worth doing yourself and publishing the results; I’d be interested to know the results. That’s if you have to have time and hardware to do it, and I don’t know if you do.

    Timothy – was that… sarcastic? I can’t tell if my answer was useful to you or if I’m missing the mark.

Comments are closed.