As part of his NVMe work, Matthew Dillon found I/O speed so fast that CRC checking actually got in the way of disk activity. He’s brought in a new CRC algorithm called xxHash. He also brought in Mark Adler’s hardware iscsi_crc32 implementation, but did not add it to Hammer2. There’s some work on read-ahead operations too, to deal with the NVMe throughput.
One Reply to “Hammer 2 and CRC”
Comments are closed.
FYI, I found Adler’s pipelined, hardware-supported crc32c quite a bit faster than xxHash64 on my Haswell hardware:
$ ./xxhsum -b
./xxhsum 0.6.0 (64-bits little endian), by Yann Collet (Jun 9 2016)
Sample of 100 KB…
CRC32C-Adler : 102400 -> 28119.6 MB/s
XXH64 : 102400 -> 14309.4 MB/s
XXH32 : 102400 -> 6883.9 MB/s