XCTB – X Compression Tool Benchmarker

I deal with a lot of big files at work. While storage capacity is not infinite indeed. So it’s in my interest to keep the file sizes as low as possible.

One way to achieve that is by using compression. Especially when dealing with log files, or database archive, you can save a ton of space with the right compression tool.

But space saving is not the only consideration.

You also need to weighs in other factors. Such as :

  • File type : different tool will compress different type of file differently
  • CPU multi-core capabilities
  • Compression speed
  • Compression size
  • Decompression time

But there are so many great compression tools available in Unix / Linux. It can be really confusing to choose which one to use even for a seasoned expert.

So I created X Compression Tool Benchmarker to help with this.

Features :

  • Test any kind of file : just put the file’s name as the parameter when calling the script. Then it will be tested against all the specified compression tools.
  • Add more compression tool easily : just edit the compressor_list & ext_file variable, and that’s it
  • Fire and forget : just run the script, and forget it. It will run without needing any intervention
  • CSV output : ready to be opened with Libre Office / Excel, and made into graphs in seconds.

Here’s a sample result for a Database archive file (type MySQL dump) :

The bar chart on top of this article is based from this result.

As you can see, currently this script will benchmark the following compression tools automatically : pigz – gzip – bzip2 – pbzip2 – lrzip – rzip – zstd – pixz – plzip – xz

The result, for each different file types, may surprise you 🙂

For example ; I was surprised to see rzip beat lrzip – because lrzip is supposed to be the enhancement of rzip.

Then I was even more surprised to find out that :

  • I was testing Debian Buster’s version of rzip, which turned out to be pretty old – it does not even have multi-thread/core capability
  • But when I tested the latest version of rzip, which can use all the 16 cores in my server – it turned out to be slower than the old rzip from Debian Buster !
  • No, disk speed is not an issue – I made sure that all the benchmark was run from NVME SSD

So I was grinning at how Debian Buster packaged a very old version of rzip instead of the new one – turned out the joke’s on me : the old rzip perform better than the new one. Even without the multi-core capability.

Also it was amazing to see how really REALLY fast zstd is, while still giving decent compression size. When you absolutely need compression speed, this not so well known compression tool turned out to be the clear winner.

And so on, etc

Yes, indeed I had fun 🙂

I hope you will too. Enjoy !


UPDATE : My friend , Eko Juniarto, published his results here and have permitted me to publish it here as well – thanks. Very interesting, indeed.

Franz Kafka — Montblanc

Franz Kafka (3 July 1883 – 3 June 1924) was a Bohemian author, widely regarded as one of the major figures of 20th-century literature. His work fuses elements of realism and the fantastic. It typically features isolated protagonists facing bizarre or surrealistic predicaments and incomprehensible socio-bureaucratic powers. It has been interpreted as exploring themes of alienation, existential anxiety, guilt, and absurdity. His best known works include Die Verwandlung (The Metamorphosis), Der Process (The Trial), and Das Schloss (The Castle).

In 2004, Montblanc has launched a special edition for Franz Kafka. The design of the fountain pen of this series is not only reminiscent of the simple and yet artful style of its eponym, but also pays tribute to Kafka’s The Metamorphosis. In the fascinating change from square to round in the writing instrument’s form, the metamorphosis described in Kafka’s story has found its equivalent.

Top, clip and rings made of 925 sterling silver offer a thrilling contrast to the dark red translucent aura of the barrel. A further reference to The Metamorphosis is on the rhodium-plated 18-karat gold nib: a finely engraved cockroach.

Die ungeheure Welt, die ich im Kopfe habe. Aber wie mich befreien und sie befreien, ohne zu zerreißen. Und tausendmal lieber zerreißen, als in mir sie zurückhalten oder begraben. Dazu bin ich ja hier, das ist mir ganz klar

Franz Kafka 21-06-1913

Franz Kafka — Montblanc

Franz Kafka (3 July 1883 – 3 June 1924) was a Bohemian author, widely regarded as one of the major figures of 20th-century literature. His work fuses elements of realism and the fantastic. It typically features isolated protagonists facing bizarre or surrealistic predicaments and incomprehensible socio-bureaucratic powers. It has been interpreted as exploring themes of alienation, existential anxiety, guilt, and absurdity. His best known works include Die Verwandlung (The Metamorphosis), Der Process (The Trial), and Das Schloss (The Castle).

In 2004, Montblanc has launched a special edition for Franz Kafka. The design of the fountain pen of this series is not only reminiscent of the simple and yet artful style of its eponym, but also pays tribute to Kafka’s The Metamorphosis. In the fascinating change from square to round in the writing instrument’s form, the metamorphosis described in Kafka’s story has found its equivalent.

Top, clip and rings made of 925 sterling silver offer a thrilling contrast to the dark red translucent aura of the barrel. A further reference to The Metamorphosis is on the rhodium-plated 18-karat gold nib: a finely engraved cockroach.

Die ungeheure Welt, die ich im Kopfe habe. Aber wie mich befreien und sie befreien, ohne zu zerreißen. Und tausendmal lieber zerreißen, als in mir sie zurückhalten oder begraben. Dazu bin ich ja hier, das ist mir ganz klar

Franz Kafka 21-06-1913

BCA – daftar bank korespondensi di Amerika

Suatu hari saya ditanyakan hal ini (bank korespondensi BCA di Amerika) setelah selesai seminar di Hawaii, untuk mentransfer honorarium saya.

Ternyata info ini tidak ketemu dimana-mana.

Tanya via Call center BCA di 1500888, mereka juga tidak tahu.

Akhirnya ketika istri saya kebetulan ada perlu ke BCA, dia tanyakan sekalian. Dijawab bahwa musti saya sendiri yang datang menanyakan.

Istri saya marah besar 😀 hahahaha

Apa logikanya cuma menanya “informasi bank korespondensi BCA” dengan saya musti datang sendiri ke BCA 😀 ha ha ha

Kalau karena musti nasabah BCA – istri saya juga nasabah BCA, dia juga punya rekening di BCA.

Akhirnya customer service BCA menyerah, dan memberitahu informasi tsb, hahaha. Ada-ada saja.

Saya lampirkan informasi tsb disini. Maka moga yang membutuhkannya tidak perlu mengalami kekonyolan serupa & terbuang-buang waktunya juga.

NAMA BANK : Bank of New York
ABA ROUTING NUMBER : IRVTUS3N

NAMA BANK : Bank of America
ABA ROUTING NUMBER : BOFAUS6S

NAMA BANK : Wells Fargo Bank
ABA ROUTING NUMBER : PNBPUS3NNYC

NAMA BANK : JP Morgan Chase Bank
ABA ROUTING NUMBER : CHASUS33

NAMA BANK : Citibank
ABA ROUTING NUMBER : CITIUS33

NAMA BANK : Standard Chartered Bank
ABA ROUTING NUMBER : SCBLUS33