If you’re a fan of HBO’s Silicon Valley, then you know about Pied Piper, the fictional company featured on the show that has created a lossless compression algorithm. Turns out, Facebook has actually made a Pied Piper of its own in real life, and today it open-sourced it so any developer can use it.
Called Zstandard, the algorithm brings 5-10% more efficiency to general data compression, and provides CPU savings. The end result is data processing that happens 2-3x faster than other similar algorithms.
And just like Pied Piper, Zstandard is lossless. Facebook has already applied the standard to some of its storage and communication systems, such as databases, and the tech has allowed things that weren’t compressed before, like Messages, to be trimmed down, making everything run much smoother.