mailcow-dockerized/data/Dockerfiles
Claas Flint 1b833be760
Replace pigz with zstd for backup compression (#6897)
* Replace pigz with zstd for backup compression

This change replaces pigz (parallel gzip) with zstd (Zstandard) as the
compression algorithm for mailcow backups while maintaining full backward
compatibility with existing .tar.gz backups.

Benefits:
- Better compression ratios (12-37% improvement in tests)
- Improved compression speed with modern algorithm
- Maintains rsyncable functionality for incremental backups
- Full backward compatibility for restoring old .tar.gz backups
- Wide industry adoption and active development

Changes:
- Backup compression: pigz --rsyncable -p → zstd --rsyncable -T
- Backup decompression: pigz -d -p → zstd -d -T
- File extensions: .tar.gz → .tar.zst
- Added get_archive_info() function for intelligent format detection
- Updated backup Dockerfile to install zstd alongside pigz
- Restore function now auto-detects and handles both formats
- Updated FILE_SELECTION regex to recognize both .tar.zst and .tar.gz
- Updated comments to reflect new file extension

Backward Compatibility:
- Restore automatically detects .tar.zst (preferred) or .tar.gz (legacy)
- Existing .tar.gz backups can still be restored without issues
- pigz remains installed in backup image for legacy support
- Graceful fallback if backup file format not found

Testing:
- Added comprehensive test suite (test_backup_and_restore.sh)
- 12 automated tests covering all scenarios:
  * Backup creation (both formats)
  * Restore (both formats)
  * Format detection and priority
  * Error handling (missing files, empty dirs)
  * Content integrity verification
  * Multi-threading configuration
  * Large file compression (8.59 MB realistic data)

Test Results:
✓ zstd compression working
✓ pigz compression working (legacy)
✓ zstd decompression working
✓ pigz decompression working (backward compatible)
✓ Archive detection working
✓ Content integrity verified
✓ Format priority correct (.tar.zst preferred)
✓ Error handling for missing files
✓ Error handling for empty directories
✓ Multi-threading configuration verified
✓ Large file compression: 37.05% improvement
✓ Small file compression: 12.18% improvement

* move testing script into development folder

---------

Co-authored-by: DerLinkman <niklas.meyer@servercow.de>
2025-11-12 10:06:36 +01:00
..
acme pf/php: add mta-sts support (outbound) (#6686) 2025-08-26 09:57:05 +02:00
backup Replace pigz with zstd for backup compression (#6897) 2025-11-12 10:06:36 +01:00
clamd Changed clamavs tmp folder structure 2025-08-29 18:27:34 +02:00
dockerapi fix(ui): Swap translations for oversized dropdown 2025-03-25 21:24:22 +01:00
dovecot fix: imapsync gets correct timeouts 2025-08-21 18:36:01 +00:00
netfilter netfilter: improve logging and mark iptables-legacy as deprecated 2025-10-09 16:37:05 +02:00
nginx [Nginx] do not invert ENABLE_IPV6 2025-09-22 14:16:42 +02:00
olefy Allow disabling Olefy 2025-03-25 20:24:33 +01:00
phpfpm Merge pull request #6696 from mailcow/renovate/krakjoe-apcu-5.x 2025-10-10 14:07:24 +02:00
postfix Postfix: Split TLSPol companion app into separate container (#6688) 2025-08-28 10:18:18 +02:00
postfix-tlspol [postfix-tlspol] build with NOOPT=1 for wider CPU compatibility 2025-09-10 10:14:37 +02:00
rspamd [Rspamd] Update to 3.13.2 2025-10-10 13:04:01 +02:00
sogo sogo: removed URLDecrpytion by default, make it configurable in sogo.conf 2025-11-12 09:50:41 +01:00
unbound os: updated alpine containers to 3.21 2025-03-10 11:56:12 +01:00
watchdog watchdog: added postfix-tlspol check (#6691) 2025-09-03 08:18:04 +02:00