Awesome utility of the day: pigz

Pronounced “pig-zee”, the description off their site is pretty self-explanatory:

A parallel implementation of gzip for modern multi-processor, multi-core machines.

Since most servers now have multi processors, compressing large data can be accelerated using pigz (download here) as opposed to common built-in single-threaded tools like gzip/bzip2.

A quick comparison of gzip vs pigz on a server running CentOS 5 with Dual Xeon 5530 Quad Core cpus shows a noticeable difference in elapsed time. The target file access.log is 5G consisting of typical apache access log entries.

$ time gzip -1 access.log
real    1m53.697s
user    1m45.261s
sys     0m7.221s
$ du -sk access.log.gz
1105524 access.log.gz

$ time pigz -1 access.log
real    0m13.565s
user    3m12.221s
sys     0m9.694s
$ du -sk access.log.gz
1105396 access.log.gz

Thanks pigz! Now to update some db backup scripts :)

141 Notes/ Hide

  1. homedecorlighting reblogged this from andrew
  2. sportsmemorabiliablog reblogged this from andrew
  3. injurylawyerblog reblogged this from andrew
  4. auto-insurance-quotes-today reblogged this from andrew
  5. estantres reblogged this from andrew
  6. istoselidon1j reblogged this from andrew
  7. trampoline3f reblogged this from andrew
  8. istoselidon9t reblogged this from andrew
  9. limitlessprofitsx reblogged this from andrew
  10. free-bets reblogged this from andrew
  11. andrew posted this