What is the competition?
For testing I used other algorithms available for .NET. They needed to have a .NET implementation or a .NET wrapper. There are three categories they can compete in:
- native: The algorithm uses native code, so it's not a real .NET assembly (includes P/Invoke and Mixed Mode)
- unsafe: The algorithm has been implemented in .NET but uses unsafe code (C++/CLI technically should be in this category but I ignored it)
- safe: The algorithm has been implemented in .NET and uses safe code only.
Testing
I used the same method as in
Performance Testing. This I didn't spend that much time and I ran the test for 3 hours only.
Please note: I used
Silesia Corpus so results are averages for compressing different kind of data (binary, database dumps, text data, XML, images, incompressible) in some ratios. If you want to check how algorithms perform on
your data - use
your data. The results might be different.
Pareto frontier
For 'real' definition please refer to
Pareto frontier.
Simply speaking Pareto frontier for compression algorithms is a set of algorithms which cannot be beaten by other algorithms.
Take a look at this picture:
- Algoritm B beats algorithm D because D offers the same compression ratio (56%) but is much slower (250MB/s vs 100MB/s).
- Algorithms B also beats algorithm E because E work as fast as B (both 250MB/s) but offers worse compression ratio (57% vs 56%).
- Obviously B also beats algorithm F because F is not only slower but also offers worse compression ratio.
- Algorithm B does not beat algorithm A. Although A is slower it offers better compression ratio.
- Algorithm B does not beat algorithm C. Although C offers worse compression ration it is faster.
- Algorithms A, B and C are on Pareto frontier - meaning they are most effective in their category.
Native
The competitors are:
- LZ4 - Mixed mode, this one
- Snappy - P/Invoke wrapper, can be found here
- LZO - actually multiple versions of LZO: LZO1X, LZO1X11, LZO1X12, LZO1X15; P/Invoke wrapper, can be found here; Note: this wrapper does not have 64-bit assembly which means it can be used only in 32-bit applications (it's becomes a little bit 'no-no')
- QuickLZ - P/Invoke wrapper; can be found here; Note: there two implementations on this page - one is safe implementation and the other one is a wrapper
Compression

Pareto frontier:
QuickLZ,
LZ4
Decompression

Pareto frontier:
QuickLZ,
LZO1X,
LZ4Note: LZO1X15 is in fact on Pareto frontier but as you can see it's speed advantage over LZO1X and compression ratio advantage over LZ4 is negligible.
Safe
The competitors are:
- LZ4 - Safe, this one
- QuickLZ - Safe, can be found here
- LZF - Safe, can be be found here
- Deflate - standard .NET defalte algorithm; not really 'fast compression algorithm' and I'm not sure if it's implementation is safe, but because it is available from safe code it belong here
Compression

Pareto frontier:
Deflate (because of compression ratio, by no means because it's speed),
QuickLZ (absolutely great performance here)
Note: LZ4 has been overshadowed by QuickLZ, maybe not by a mile but still. No Pareto frontier award. I'll have to take a look at safe compression implementation again, maybe something can be done
Decompression

Pareto frontier:
Deflate (again, compression ratio only),
QuickLZ (good compression ratio, decent speed),
LZ4 (not so much worse compression ratio and great speed)
Unsafe
I couldn't find any unsafe compression implementations. They all are native (P/Invoke) or Safe.
So the only competitor is different implementation of LZ4.
- LZ4 - unsafe, this one
- LZ4Sharp - unsafe, can be found here
Compression
Note: My implementation of LZ4 is a little bit better. I used a little bit more mature C version as a template and I spent some time on tuning it. When you a take a closer look you'll notice that differences are quite small, though.
Decompression
Note: My implementation of LZ4 is a little bit better. I used a little bit more mature C version as a template and I spent some time on tuning it. When you a take a closer look you'll notice that differences are quite small, though.