this post was submitted on 13 Sep 2023
684 points (97.6% liked)

Programmer Humor

34180 readers
27 users here now

Post funny things about programming here! (Or just rant about your favourite programming language.)

Rules:

founded 5 years ago
MODERATORS
AgreeableLandscape@lemmy.ml
cat_programmer@lemmy.ml
684
New File Format (lemmy.ml)
submitted 2 years ago by yogthos@lemmy.ml to c/programmerhumor@lemmy.ml
83 comments fedilink hide all child comments
 
you are viewing a single comment's thread
view the rest of the comments
[–] gamma@programming.dev 13 points 2 years ago (3 children)

zstd or leave

[–] dan@upvote.au 5 points 2 years ago (1 children)

They both have their use cases. Zstandard is for compression of a stream of data (or a single file), while 7-Zip is actually two parts: A directory structure (like tar) plus a compression algorithm (like LZMA which it uses by default) in a single app.

7-Zip is actually adding zstd support: https://sourceforge.net/p/sevenzip/feature-requests/1580/

[–] TechieDamien@lemmy.ml 1 points 2 years ago* (last edited 2 years ago) (1 children)

Well when using zstd, you tar first, something like tar -I zstd -cf my_tar.tar.zst my_files/*. You almost never call zstd directly and always use some kind of wrapper.

[–] dan@upvote.au 7 points 2 years ago (1 children)

Sure, you can tar first. That has various issues though, for example if you just want to extract one file in the middle of the archive, it still needs to decompress everything up to that point. Something like 7-Zip is more sophisticated in terms of how it indexes files in the archive, so I'm looking forward to them adding zstd support.

FWIW most of my uses of zstd don't involve tar, but it's in things like Borgbackup, database systems, etc.

[–] TechieDamien@lemmy.ml 1 points 2 years ago

Yes, definitely. My biggest use is transparent filesystem compression, so I completely agree!

[–] derpgon@programming.dev 4 points 2 years ago

I'll gunzip you to oblivion!

[–] AVincentInSpace@pawb.social 3 points 2 years ago (2 children)

zstd may be newer and faster but lzma still compresses more

[–] gamma@programming.dev 11 points 2 years ago* (last edited 2 years ago) (2 children)

Thought I'd check on the Linux source tree tar. zstd -19 vs lzma -9:

❯ ls -lh
total 1,6G
-rw-r--r-- 1 pmo pmo 1,4G Sep 13 22:16 linux-6.6-rc1.tar
-rw-r--r-- 1 pmo pmo 128M Sep 13 22:16 linux-6.6-rc1.tar.lzma
-rw-r--r-- 1 pmo pmo 138M Sep 13 22:16 linux-6.6-rc1.tar.zst

About +8% compared to lzma. Decompression time though:

zstd -d -k -T0 *.zst  0,68s user 0,46s system 162% cpu 0,700 total
lzma -d -k -T0 *.lzma  4,75s user 0,51s system 99% cpu 5,274 total

Yeah, I'm going with zstd all the way.

[–] jodanlime@midwest.social 6 points 2 years ago

Nice data. Thanks for reminding me why I prefer zstd

[–] AVincentInSpace@pawb.social 1 points 2 years ago (1 children)

damn I did not know zstd was that good. Never thought I'd hear myself say this unironically but thanks Facebook

[–] gamma@programming.dev 5 points 2 years ago (1 children)

*Thank you engineers who happen to be working at Facebook

[–] AVincentInSpace@pawb.social 1 points 2 years ago

Very true, good point

[–] gamer@lemm.ee 2 points 2 years ago

As always, you gotta know both so that you can pick the right tool for the job.