* Replace pigz with zstd for backup compression
This change replaces pigz (parallel gzip) with zstd (Zstandard) as the
compression algorithm for mailcow backups while maintaining full backward
compatibility with existing .tar.gz backups.
Benefits:
- Better compression ratios (12-37% improvement in tests)
- Improved compression speed with modern algorithm
- Maintains rsyncable functionality for incremental backups
- Full backward compatibility for restoring old .tar.gz backups
- Wide industry adoption and active development
Changes:
- Backup compression: pigz --rsyncable -p → zstd --rsyncable -T
- Backup decompression: pigz -d -p → zstd -d -T
- File extensions: .tar.gz → .tar.zst
- Added get_archive_info() function for intelligent format detection
- Updated backup Dockerfile to install zstd alongside pigz
- Restore function now auto-detects and handles both formats
- Updated FILE_SELECTION regex to recognize both .tar.zst and .tar.gz
- Updated comments to reflect new file extension
Backward Compatibility:
- Restore automatically detects .tar.zst (preferred) or .tar.gz (legacy)
- Existing .tar.gz backups can still be restored without issues
- pigz remains installed in backup image for legacy support
- Graceful fallback if backup file format not found
Testing:
- Added comprehensive test suite (test_backup_and_restore.sh)
- 12 automated tests covering all scenarios:
* Backup creation (both formats)
* Restore (both formats)
* Format detection and priority
* Error handling (missing files, empty dirs)
* Content integrity verification
* Multi-threading configuration
* Large file compression (8.59 MB realistic data)
Test Results:
✓ zstd compression working
✓ pigz compression working (legacy)
✓ zstd decompression working
✓ pigz decompression working (backward compatible)
✓ Archive detection working
✓ Content integrity verified
✓ Format priority correct (.tar.zst preferred)
✓ Error handling for missing files
✓ Error handling for empty directories
✓ Multi-threading configuration verified
✓ Large file compression: 37.05% improvement
✓ Small file compression: 12.18% improvement
* move testing script into development folder
---------
Co-authored-by: DerLinkman <niklas.meyer@servercow.de>
This commit fixes two bugs in the cold-standby script:
1. Support digits in COMPOSE_PROJECT_NAME
The script was stripping digits from COMPOSE_PROJECT_NAME, while
backup_and_restore.sh (fixed in a71d991c) correctly supports them.
Added '0-9' to the tr character set to align behavior.
2. Support docker-compose.override.yml on remote
Lines 172 and 287 explicitly used '-f docker-compose.yml' which
causes Docker Compose to ignore docker-compose.override.yml even
when present. Changed to 'cd && compose' pattern (matching line 296)
to auto-discover override files.
Impact: Users with custom volumes/services in override file would
experience silent failures - volumes not created, images not pulled,
data syncing to wrong locations.
Both fixes ensure cold-standby works correctly with standard Docker
Compose conventions and user customizations.
Co-authored-by: Christophe Neuerburg <c.neuerburg@sdsys.ch>
