+- Makestation (https://makestation.net)
+-- Forum: Technical Arts (https://makestation.net/forumdisplay.php?fid=45)
+--- Forum: Software (https://makestation.net/forumdisplay.php?fid=104)
+--- Thread: ZPAQ - Insane File Compression (/showthread.php?tid=3155)
ZPAQ - Insane File Compression - Darth-Apple - September 27th, 2020
I have around ~70GB of old music projects (Logic Pro X) sitting on my hard drive. Every single track is a 48khz/24 bit wav/aiff, meaning that some tracks are over 50% bigger than normal CD-Quality WAV files. These projects easily reach several gigabytes, especially as they near completion, and WAV files are very difficult to compress with traditional compression algorithms. I am in the process of saving them externally to free up space, so I began my search.
Surprisingly, 7-zip did a pretty good job of shrinking the projects by 20-30%, but that didn't do a lot of good for situations when I had a lot of duplicate files (bounces and alternative versions) that added bloat. LZMA can detect duplicate files, but it can only do so if they are very close to each other (within 16-64MB of each other, depending on the dictionary size/configuation).
I found a tool called ZPAQ for that. It's a command-line only tool and it's a bit obscure, but it relies on implementing several compression algorithms and determining which algorithm is best for each specific type of file. This allows it to do much better on raw audio files that are generally difficult to compress with standard zip/7z-style compression.
The results: On about 9.5GB of data, I shrunk it down to a mere 1.3GB. This was a whopping reduction. (7-zip got it down to about 5.95 GB, for comparison). On files without duplicates, ZPAQ generally saw about a 35% reduction, and a 30% reduction on standard 7z/LZMA (compared to 10-20% on standard ZIP). The differences were far less extreme, but still very impressive considering the difficult-to-compress nature of lossless audio.
That being said, I will likely continue to use the 7z format because it's far better supported and doesn't require command-line interfaces to extract/compress (storage is cheap anyway
). However, with such extreme reductions seen by ZPAQ, I may look into it a little more for the future.
RE: ZPAQ - Insane File Compression - Guardian - September 30th, 2020
WOW.... that's impressive. I'm going to have to look into this one...
RE: ZPAQ - Insane File Compression - Kyng - September 30th, 2020
That is mightily impressive
. I can't see myself using it on a routine basis, but it's good to know it exists for if I ever need to compress large files!