mirror of
https://github.com/checkpoint-restore/criu.git
synced 2026-07-17 16:47:50 +00:00
2d1335b755
Serving engines have different memory layouts and restore behavior, so SGLang results do not establish the cost or correctness of compression for vLLM. Add a vLLM frontend to the shared Podman benchmark. Support CPU and GPU images and the supported server entrypoint forms, compare uncompressed, per-page LZ4, and region LZ4 images, and verify deterministic inference after restore. Report archive, checkpoint, restore, and post-restore request costs, and add a bounded CPU test runner. Reuse the shared runc.conf transaction, CRIU configuration isolation, inventory validation, secret redaction, failure diagnostics, and cleanup semantics so both serving benchmarks follow the same methodology. Assisted-by: Codex:GPT-5 Assisted-by: Claude:claude-fable-5 Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
30 lines
733 B
Bash
Executable file
30 lines
733 B
Bash
Executable file
#!/bin/sh
|
|
|
|
set -eu
|
|
|
|
root=$(cd -- "$(dirname -- "$0")/../../../.." && pwd)
|
|
|
|
: "${VLLM_CPU_IMAGE:=vllm/vllm-openai-cpu:latest-x86_64}"
|
|
: "${VLLM_CPU_MODEL:=Qwen/Qwen2.5-0.5B-Instruct}"
|
|
|
|
if [ "$(id -u)" -ne 0 ]; then
|
|
echo "compression/vllm: must run as root" >&2
|
|
exit 1
|
|
fi
|
|
|
|
timeout --foreground --kill-after=60s 3600s python3 \
|
|
"$root/contrib/compression-benchmark/podman-vllm.py" \
|
|
--accelerator cpu \
|
|
--image "$VLLM_CPU_IMAGE" \
|
|
--model "$VLLM_CPU_MODEL" \
|
|
--iterations 1 \
|
|
--modes uncompressed lz4-page lz4-region \
|
|
--region-sizes 65536 \
|
|
--decompress-threads 4 \
|
|
--archive-compression none \
|
|
--cpu-kvcache-space 1 \
|
|
--max-model-len 512 \
|
|
--max-tokens 4 \
|
|
--shm-size 4g \
|
|
--wait-seconds 1200 \
|
|
--request-timeout 300
|