Zstd compression is coming to Arch, but what is it?

Arch-Linux has been implementing a lot of changes recently, to keep the system lean, customizable and versatile also in the future. Pacman 5.1, which was released in October 2019, heralded the big change in the Arch eco-system by adding support for Zstd, but what is it?

Zstd stands for Zstandard and is a lossless data compression algorithm written by Yann Collet. Its first version was released on 31 August 2016.

making the program or file portable and storable with a smaller footprint, without losing any of the designed features.

“Great, but what is lossless data compression?” you might ask me behind your screen.

When a program or file is written, it can contain quite a lot of lines of codes and by compressing the file using a few bits less in size, is very essential for storing and transmitting.

In the classic sense of compressing some “unnecessary” code lines were simply thrown out to make the file smaller and lightweight to transmit and store. The most well-known data compression technique that makes use of this by a large audience are MP3 and MP4.

With lossless data compression, no line of code is discarded and without being too technical, it just reduces or eliminates “wasted space” of the file to make its footprint smaller. You can compare it with the way companies like IKEA package their pillows and mattresses, by sucking out the air in the product itself and rolling or folding it in a vacuum space, you are able to transport a mattress in a regular-sized car and when you’re at home and unpack the product, your pillow or mattress will unfold to its full size with all the comfort features it was originally designed with.

The air in the mattress or pillow isn’t necessary when being transported or stored but plays a vital role when it’s in use. This is what lossless data compression exactly does, making the program or file portable and storable with a smaller footprint, without losing any of the designed features.

Zstd or Zstandard

With traditional compressing techniques, the focus was primarily on making the file or program significantly smaller and lightweight, the unpacking of the file, decompressing, takes quite some time.

These techniques are still based on internet and computer techniques from when bandwidth and fast CPUs were only available for power users.

When Yann Collet released its first version, Facebook saw the potential of it and hired him to develop the code further. Now don’t be alarmed by this fact, Zstd is still an open-source project and found official support in Linux kernel 4.14 released in November 2017.

You can imagine with a tech giant behind this open-source project, development increased and the stable version found its way on 5 November 2019, just three years after version one.

The major advantage of Zstd, besides creating a small footprint, is its speed in decompressing. The benefits range from 30% and in some cases 50% better ratio to 3x improvements in speed, without compromising in quality.

Zstd has more advantages and is clearly developed not only for current techniques but for future developments also. If you want to have a more in-depth article, I advise you to read the Facebook blog written by the original creator of Zstd.

With Arch-Linux implementing this faster technique by default, it ensures its position in using leading technologies for the future it is known for. Since EndeavourOS is close to Arch, we are ready for the transition.

Follow us:

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: