perf(ci): 流水线提速 - 路径过滤/持久builder缓存/依赖缓存卷/部署job迁移ci-l2
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2m20s
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Validate - Type Check (mypy) (pull_request) Successful in 3m28s
CI/CD Pipeline / Validate - Migration (alembic) (pull_request) Successful in 3m16s
AI Code Review / AI Code Review (pull_request) Successful in 1m59s
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 2m13s
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m59s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 2m8s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 4m45s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 3m8s
CI/CD Pipeline / Validate - Code Quality (pull_request) Successful in 9m25s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 4m16s
CI/CD Pipeline / Unit Tests (pull_request) Failing after 16m45s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Failing after 28s

1. build-staging 路径过滤:纯前端改动只构建Web,纯后端只构建API/Worker;
   未重建镜像通过 retag-staging-skipped job 将分支tag retag为新SHA推送,Watchtower链路不变
2. deploy-staging、acr-cleanup runs-on 从 runtime-builder 改为 ci-l2,释放构建槽
3. buildx 改为宿主机持久 builder(ci-builder-persist)+buildkit命名卷缓存,
   job结束不再prune,Worker镜像同样接入buildx缓存
4. validate/unit-test/frontend-test 的 pip/npm 依赖通过宿主机命名卷持久化缓存
   (runner config 挂载 ci-pip-cache/xiaoxia-npm-cache/act-toolcache)
5. 不影响生产构建/发版/canary 相关 job
This commit is contained in:
2026-08-29 11:28:12 +08:00
parent e6c90a2346
commit a64037881a
6 changed files with 375 additions and 118 deletions
+166 -65
View File
@@ -696,14 +696,14 @@ jobs:
done
echo
echo "${{ matrix.service_display }} PR build verified: ${IMAGE_TAG}"
- name: Cleanup buildx builder
- name: Builder cache note
if: always()
shell: sh
run: |
BUILDER_NAME="ci-pr-builder-${GITHUB_RUN_ID:-local}"
docker buildx rm "$BUILDER_NAME" 2>/dev/null || true
docker buildx prune -f 2>/dev/null || true
echo "Builder cleanup done"
# 持久 builder (ci-builder-persist) 跨 job 共享,不删除不 prune;
# 残留容器/卷由宿主机 /usr/local/bin/ci-docker-cleanup.sh 兜底清理
docker buildx ls | head -5
echo "Persistent builder kept warm for next job"
- name: Job duration summary
if: always()
shell: sh
@@ -729,11 +729,49 @@ jobs:
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
check-push-paths:
name: Check push changed paths
runs-on: ci-l2
if: github.event_name == 'push' && (github.ref_name == 'main' || github.ref_name == 'develop')
outputs:
skip_backend: ${{ steps.check.outputs.skip_backend }}
skip_frontend: ${{ steps.check.outputs.skip_frontend }}
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sfH "Authorization: token $GITHUB_TOKEN" -o /tmp/_ci_checkout.sh "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" && bash /tmp/_ci_checkout.sh
- name: Check changed paths
id: check
shell: bash
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
bash scripts/ci/ci_push_paths.sh
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
build-staging:
name: Build Staging ${{ matrix.service_display }} Image
runs-on: runtime-builder
timeout-minutes: ${{ matrix.timeout }}
if: github.event_name == 'push' && (github.ref_name == 'main' || github.ref_name == 'develop')
needs: check-push-paths
if: |
github.event_name == 'push' && (github.ref_name == 'main' || github.ref_name == 'develop') && (
(matrix.service == 'web' && needs.check-push-paths.outputs.skip_frontend != 'true') ||
(matrix.service != 'web' && needs.check-push-paths.outputs.skip_backend != 'true')
)
strategy:
fail-fast: false
matrix:
@@ -797,23 +835,16 @@ jobs:
echo "Cache mode: read-only"
fi
- name: Setup buildx builder
if: matrix.service != 'worker'
- name: Ensure persistent buildx builder
shell: sh
run: |
set -eu
if ! docker buildx inspect ci-builder-${GITHUB_RUN_ID}-${GITHUB_JOB}-${{ matrix.cache_name }} > /dev/null 2>&1; then
docker buildx create --use --name ci-builder-${GITHUB_RUN_ID}-${GITHUB_JOB}-${{ matrix.cache_name }} --driver docker-container
echo "Created ci-builder (docker-container driver)"
else
docker buildx use ci-builder-${GITHUB_RUN_ID}-${GITHUB_JOB}-${{ matrix.cache_name }}
echo "Using existing ci-builder"
fi
docker buildx inspect --bootstrap
run: bash scripts/ci/ensure_persistent_builder.sh
- name: Pre-build worker base image (fallback if not exist)
- name: Pre-pull worker base image (fallback build if not exist)
if: matrix.service == 'worker'
shell: sh
env:
ACR_USERNAME: ${{ secrets.ACR_USERNAME }}
ACR_PASSWORD: ${{ secrets.ACR_PASSWORD }}
run: |
set -eu
REGISTRY="xiaoxia-registry.cn-hangzhou.cr.aliyuncs.com/xiaoxiakeji"
@@ -821,11 +852,13 @@ jobs:
echo "检查 Worker 基础镜像..."
if docker pull "$BASE_IMAGE" 2>/dev/null; then
echo "✅ 基础镜像已存在"
echo "✅ 基础镜像已存在buildkit 可直接命中)"
else
echo "⚠️ 基础镜像不存在,本地构建fallback..."
docker build -f infra/docker/worker-base.Dockerfile -t "$BASE_IMAGE" .
echo "✅ Worker 基础镜像本地构建完成"
echo "⚠️ 基础镜像不存在,用持久 builder 构建并推送fallback..."
docker buildx build --builder ci-builder-persist \
-f infra/docker/worker-base.Dockerfile \
-t "$BASE_IMAGE" --push .
echo "✅ Worker 基础镜像构建推送完成"
fi
- name: Build and push ${{ matrix.service_display }} image
@@ -833,48 +866,34 @@ jobs:
run: |
set -eu
REGISTRY="xiaoxia-registry.cn-hangzhou.cr.aliyuncs.com/xiaoxiakeji"
IMAGE_TAG="${REGISTRY}/${{ matrix.image_name }}:${GITHUB_SHA}"
IMAGE_FULL="${REGISTRY}/${{ matrix.image_name }}"
IMAGE_TAG="${IMAGE_FULL}:${GITHUB_SHA}"
# 同时推分支 tag,作为未重建镜像 retag 的稳定来源
BRANCH_TAG="${IMAGE_FULL}:${GITHUB_REF_NAME}"
CACHE_REF="${REGISTRY}/${{ matrix.cache_name }}:${GITHUB_REF_NAME}"
if [ "${{ matrix.service }}" = "worker" ]; then
# Worker: plain docker build(基础镜像已预装全部依赖,无需 buildx)
echo "=== Worker: plain docker build ==="
docker build -f ${{ matrix.dockerfile }} -t "${IMAGE_TAG}" --build-arg APP_VERSION="${GITHUB_SHA}" .
docker push "${IMAGE_TAG}"
echo "✅ Worker image pushed: ${IMAGE_TAG}"
else
# API/Web: buildx with registry cache
CACHE_REF="${REGISTRY}/${{ matrix.cache_name }}:${GITHUB_REF_NAME}"
EXTRA_BUILD_ARGS="APP_VERSION=\"${GITHUB_SHA}\""
if [ "${{ matrix.service }}" = "web" ]; then
EXTRA_BUILD_ARGS="$EXTRA_BUILD_ARGS NGINX_CONF=infra/docker/nginx-staging.conf"
fi
NO_CACHE_FLAG=""
for i in 1 2 3; do
echo "=== Docker build 尝试 $i/3 ==="
if bash scripts/ci/docker_build_push.sh $NO_CACHE_FLAG ${{ matrix.dockerfile }} "${IMAGE_TAG}" "${CACHE_REF}" $EXTRA_BUILD_ARGS; then
echo "✅ Docker build 成功"
break
fi
echo "❌ Docker build 失败(尝试 $i/3"
[ $i -eq 3 ] && exit 1
sleep 10
if [ $i -eq 2 ]; then
NO_CACHE_FLAG="--no-cache"
echo "下次重试将使用 --no-cache"
fi
done
echo "${{ matrix.service_display }} image pushed: ${IMAGE_TAG}"
EXTRA_BUILD_ARGS="APP_VERSION=\"${GITHUB_SHA}\""
if [ "${{ matrix.service }}" = "web" ]; then
EXTRA_BUILD_ARGS="$EXTRA_BUILD_ARGS NGINX_CONF=infra/docker/nginx-staging.conf"
fi
- name: Cleanup buildx builder
if: matrix.service != 'worker' && always()
shell: sh
run: |
docker buildx rm ci-builder-${GITHUB_RUN_ID}-${GITHUB_JOB}-${{ matrix.cache_name }} 2>/dev/null || true
docker buildx rm ci-builder 2>/dev/null || true
docker buildx prune -f 2>/dev/null || true
echo "Builder cleanup done"
NO_CACHE_FLAG=""
for i in 1 2 3; do
echo "=== Docker build 尝试 $i/3 (${{ matrix.service_display }}) ==="
if EXTRA_TAGS="$BRANCH_TAG" bash scripts/ci/docker_build_push.sh $NO_CACHE_FLAG ${{ matrix.dockerfile }} "${IMAGE_TAG}" "${CACHE_REF}" $EXTRA_BUILD_ARGS; then
echo "✅ Docker build 成功"
break
fi
echo "❌ Docker build 失败(尝试 $i/3"
[ $i -eq 3 ] && exit 1
sleep 10
if [ $i -eq 2 ]; then
NO_CACHE_FLAG="--no-cache"
echo "下次重试将使用 --no-cache"
fi
done
echo "${{ matrix.service_display }} image pushed: ${IMAGE_TAG} (+ ${BRANCH_TAG})"
- name: Job duration summary
if: always()
@@ -902,15 +921,97 @@ jobs:
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
retag-staging-skipped:
name: Retag skipped Staging ${{ matrix.service_display }} Image
runs-on: runtime-builder
timeout-minutes: 10
needs:
- check-push-paths
- build-staging
if: |
github.event_name == 'push' && (github.ref_name == 'main' || github.ref_name == 'develop') && (
(matrix.service == 'web' && needs.check-push-paths.outputs.skip_frontend == 'true') ||
(matrix.service != 'web' && needs.check-push-paths.outputs.skip_backend == 'true')
)
strategy:
fail-fast: false
matrix:
include:
- service: api
service_display: API
image_name: xiaoxia-saas-api
- service: worker
service_display: Worker
image_name: xiaoxia-saas-worker
- service: web
service_display: Web
image_name: xiaoxia-saas-web
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sfH "Authorization: token $GITHUB_TOKEN" -o /tmp/_ci_checkout.sh "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" && bash /tmp/_ci_checkout.sh
- name: Docker login to Registry
shell: sh
env:
ACR_USERNAME: ${{ secrets.ACR_USERNAME }}
ACR_PASSWORD: ${{ secrets.ACR_PASSWORD }}
GITEA_REGISTRY_USER: xiaoxia
GITEA_REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
set -eu
for i in 1 2 3; do
if printf '%s' "${ACR_PASSWORD}" | docker login xiaoxia-registry.cn-hangzhou.cr.aliyuncs.com -u "${ACR_USERNAME}" --password-stdin \
&& docker login git.xiaoxiajianji.com -u "${GITEA_REGISTRY_USER}" -p "${GITEA_REGISTRY_TOKEN}"; then
echo "✅ Docker login successful"
break
fi
echo "Docker login 失败($i/3),5s 后重试..."
sleep 5
done
- name: Retag previous branch image to new SHA
shell: sh
run: |
set -eu
REGISTRY="xiaoxia-registry.cn-hangzhou.cr.aliyuncs.com/xiaoxiakeji"
bash scripts/ci/retag_skipped_image.sh \
"${REGISTRY}/${{ matrix.image_name }}" \
"${GITHUB_SHA}" \
"${GITHUB_REF_NAME}"
- name: Notify on failure
continue-on-error: true
if: failure()
shell: sh
env:
CI_NOTIFY_WEBHOOK: ${{ secrets.CI_NOTIFY_WEBHOOK }}
run: |
set +e
NOTIFY_MODE=failure JOB_NAME="Retag Staging ${{ matrix.service_display }} Image" python3 scripts/ci_notify.py
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
deploy-staging:
name: Deploy Staging (Watchtower auto-deploy)
runs-on: runtime-builder
runs-on: ci-l2
timeout-minutes: 15
concurrency:
group: deploy-staging-${{ gitea.ref }}
cancel-in-progress: false
needs:
- check-push-paths
- build-staging
- retag-staging-skipped
if: github.event_name == 'push' && (github.ref_name == 'main' || github.ref_name == 'develop')
steps:
- name: Checkout code
@@ -1491,7 +1592,7 @@ jobs:
acr-cleanup:
name: ACR Image Cleanup
runs-on: runtime-builder
runs-on: ci-l2
timeout-minutes: 10
needs:
- deploy-staging
+77
View File
@@ -0,0 +1,77 @@
#!/bin/bash
# CI 公共步骤:检测 push(develop/main) 事件的改动范围
# 输出 skip_backend / skip_frontend(复用 PR check 的语义)
# - 纯前端改动(仅 apps/web/): skip_backend=true
# - 纯后端改动(不含 apps/web/): skip_frontend=true
# - 全栈 / 无法判断: 两者都 false(走全量,安全兜底)
# 需要环境变量: GITHUB_TOKEN, GITHUB_API_URL, GITHUB_REPOSITORY, GITHUB_SHA
set -eu
OUTPUT="${GITHUB_OUTPUT:-/dev/stdout}"
before="${GITHUB_EVENT_BEFORE:-}"
after="${GITHUB_SHA:-}"
repo="${GITHUB_REPOSITORY:-}"
base="${GITHUB_API_URL:-}"
# Gitea Actions 中 push 事件的前一个 SHA 在 event payload 的 before 字段
if [ -z "$before" ] && [ -n "${GITHUB_EVENT_PATH:-}" ] && [ -f "$GITHUB_EVENT_PATH" ]; then
before=$(python3 -c "
import json,sys
try:
d=json.load(open('${GITHUB_EVENT_PATH}'))
print(d.get('before','') or '')
except Exception:
print('')
")
fi
echo "改动范围检测: before=${before:-<empty>} after=${after}"
FILES=""
if [ -n "$before" ] && [ "$before" != "0000000000000000000000000000000000000000" ]; then
# compare API: {base}...{head}
API_URL="${base}/repos/${repo}/compare/${before}...${after}?per_page=300"
for attempt in 1 2 3; do
FILES=$(curl -s --max-time 30 -H "Authorization: token ${GITHUB_TOKEN}" "$API_URL" \
| python3 -c "
import json,sys
try:
d=json.load(sys.stdin)
for f in d.get('files', []):
print(f.get('filename',''))
except Exception:
pass
")
[ -n "$FILES" ] && break
echo "compare API 无返回,重试 $attempt/3..."
sleep 3
done
fi
if [ -z "$FILES" ]; then
echo "⚠️ 无法获取改动文件列表(新分支/API异常),保守起见走全量构建"
echo "skip_backend=false" >> "$OUTPUT"
echo "skip_frontend=false" >> "$OUTPUT"
exit 0
fi
TOTAL=$(printf '%s\n' "$FILES" | grep -c . || true)
FRONTEND_COUNT=$(printf '%s\n' "$FILES" | grep -c '^apps/web/' || true)
BACKEND_COUNT=$(python3 -c "print($TOTAL - $FRONTEND_COUNT)")
echo "变更文件: ${TOTAL} 个 (前端: ${FRONTEND_COUNT}, 后端/公共: ${BACKEND_COUNT})"
if [ "$BACKEND_COUNT" = "0" ] && [ "$FRONTEND_COUNT" -gt "0" ]; then
echo "skip_backend=true" >> "$OUTPUT"
echo "skip_frontend=false" >> "$OUTPUT"
echo "✅ 纯前端改动,跳过后端镜像构建"
elif [ "$FRONTEND_COUNT" = "0" ] && [ "$BACKEND_COUNT" -gt "0" ]; then
echo "skip_backend=false" >> "$OUTPUT"
echo "skip_frontend=true" >> "$OUTPUT"
echo "🔧 纯后端改动,跳过 Web 镜像构建"
else
echo "skip_backend=false" >> "$OUTPUT"
echo "skip_frontend=false" >> "$OUTPUT"
echo "🔧 包含全栈/公共变更,三个镜像全部构建"
fi
+13 -8
View File
@@ -1,6 +1,7 @@
#!/bin/bash
# PR构建专用:只构建不输出,验证Dockerfile能否正常构建
# 无本地缓存(12个runner不共享,反而添乱),只用ACR远程缓存
# 缓存:复用宿主机持久 builder (ci-builder-persist) 的层缓存 + ACR registry 缓存兜底
# 无状态:build-only 不推送,job 结束无需清理(builder 为共享持久资源)
set -eu
NO_CACHE_FLAG=""
@@ -18,17 +19,21 @@ for arg in "$@"; do
BUILD_ARGS="$BUILD_ARGS --build-arg $arg"
done
BUILDER_NAME="ci-pr-builder-${GITHUB_RUN_ID:-local}"
BUILDER_NAME="ci-builder-persist"
if ! docker buildx inspect "$BUILDER_NAME" > /dev/null 2>&1; then
docker buildx create --use --name "$BUILDER_NAME" --driver docker-container
else
docker buildx use "$BUILDER_NAME"
echo "持久 builder 不存在,创建中..."
docker buildx create --name "$BUILDER_NAME" --driver docker-container \
--driver-opt network=host \
--buildkitd-flags "--allow-insecure-entitlement network.host" \
--platform linux/amd64
fi
docker buildx inspect --bootstrap
docker buildx use "$BUILDER_NAME"
docker buildx inspect "$BUILDER_NAME" --bootstrap
echo "=== PR Build: build only, no output, remote cache only ==="
echo "=== PR Build: build only, no push (persistent builder cache) ==="
echo "Dockerfile: ${DOCKERFILE}"
echo "Image tag: ${IMAGE_TAG}"
echo "Image tag: ${IMAGE_TAG}"
echo "Builder: ${BUILDER_NAME}"
echo ""
docker buildx build \
+47 -45
View File
@@ -1,10 +1,19 @@
#!/bin/bash
# 通用Docker镜像构建+推送脚本local cache为主 + registry cache共享)
# 通用Docker镜像构建+推送脚本
# 缓存策略(2026-08 起):
# - buildx 使用宿主机持久 builder (ci-builder-persist),层缓存保存在
# buildkit 容器/命名卷中,跨 job 共享、job 结束不清理
# - registry cache 仅作为冷启动兜底读取
# - 额外 tag(如分支 tag :develop)通过 EXTRA_TAGS 环境变量传入,随构建一并推送
# 用法: docker_build_push.sh [--no-cache] <Dockerfile> <image_tag> <cache_ref> [build_arg...]
# 环境变量:
# EXTRA_TAGS 空格分隔的额外 tag(完整 image:tag 引用),可选
set -eu
# 单次 build 超时时间(秒),防止 docker buildx build 无限挂起
BUILD_TIMEOUT=1500
# 持久 builder 名(宿主机级,所有 CI job 共享;由 ensure_persistent_builder.sh 维护)
BUILDER_NAME="ci-builder-persist"
NO_CACHE_FLAG=""
if [ "$1" = "--no-cache" ]; then
@@ -22,23 +31,27 @@ for arg in "$@"; do
BUILD_ARGS="$BUILD_ARGS --build-arg $arg"
done
if ! docker buildx inspect ci-builder > /dev/null 2>&1; then
docker buildx create --use --name ci-builder --driver docker-container
echo "Created ci-builder"
else
docker buildx use ci-builder
echo "Using existing ci-builder"
# 确保持久 builder 存在并使用(幂等)
if ! docker buildx inspect "$BUILDER_NAME" > /dev/null 2>&1; then
echo "持久 builder 不存在,创建中..."
docker buildx create --name "$BUILDER_NAME" --driver docker-container \
--driver-opt network=host \
--buildkitd-flags "--allow-insecure-entitlement network.host" \
--platform linux/amd64
fi
docker buildx inspect --bootstrap
docker buildx use "$BUILDER_NAME"
docker buildx inspect "$BUILDER_NAME" --bootstrap
# 从cache_ref中提取缓存名称(如 api-cache:develop -> api-cache-develop
CACHE_NAME=$(echo "$CACHE_REF" | tr '/' '_' | tr ':' '-')
LOCAL_CACHE_DIR="/tmp/buildx-cache/${CACHE_NAME}"
# 组装额外 tag 参数
EXTRA_TAG_FLAGS=""
EXTRA_TAG_LIST=""
if [ -n "${EXTRA_TAGS:-}" ]; then
for t in $EXTRA_TAGS; do
EXTRA_TAG_FLAGS="$EXTRA_TAG_FLAGS -t $t"
EXTRA_TAG_LIST="$EXTRA_TAG_LIST $t"
done
fi
mkdir -p "$LOCAL_CACHE_DIR"
# 缓存源:local优先(带自动修复),registry兜底读写
# 本地缓存损坏时自动清理后重试,避免snapshot not found导致构建全挂
build_with_cache_retry() {
local attempt=1
local max_attempts=2
@@ -49,12 +62,10 @@ build_with_cache_retry() {
build_output=$(timeout ${BUILD_TIMEOUT} docker buildx build \
$NO_CACHE_FLAG \
$BUILD_ARGS \
--cache-from "type=local,src=${LOCAL_CACHE_DIR}" \
--cache-from "type=registry,ref=${CACHE_REF}" \
--cache-to "type=local,dest=${LOCAL_CACHE_DIR},mode=max" \
--cache-to "type=registry,ref=${CACHE_REF},mode=max,ignore-error=true" \
-f "${DOCKERFILE}" \
-t "${IMAGE_TAG}" \
$EXTRA_TAG_FLAGS \
--push \
. 2>&1)
exit_code=$?
@@ -69,48 +80,39 @@ build_with_cache_retry() {
echo "$build_output" | tail -20
return $exit_code
fi
# 检测到缓存损坏类错误,清掉本地缓存重试
if echo "$build_output" | grep -qE "parent snapshot.*not found|snapshot.*does not exist|cache.*corrupt|failed to compute cache key"; then
# 检测到缓存/快照损坏类错误,重建 builder 后重试
if echo "$build_output" | grep -qE "parent snapshot.*not found|snapshot.*does not exist|cache.*corrupt|failed to compute cache key|no such file or directory.*cache"; then
echo "$build_output"
echo ""
echo "⚠️ Local cache appears corrupted, cleaning up and retrying (attempt $attempt/$max_attempts)..."
rm -rf "${LOCAL_CACHE_DIR}"
mkdir -p "${LOCAL_CACHE_DIR}"
# 清理buildx builder的内部snapshot状态
docker buildx prune -f -a > /dev/null 2>&1 || true
echo "⚠️ builder 缓存异常,重建持久 builder 后重试 (attempt $attempt/$max_attempts)..."
docker buildx rm "$BUILDER_NAME" >/dev/null 2>&1 || true
docker buildx create --name "$BUILDER_NAME" --driver docker-container \
--driver-opt network=host \
--buildkitd-flags "--allow-insecure-entitlement network.host" \
--platform linux/amd64
docker buildx use "$BUILDER_NAME"
docker buildx inspect "$BUILDER_NAME" --bootstrap
attempt=$((attempt + 1))
else
# 非缓存类错误,直接输出并返回
echo "$build_output"
return $exit_code
fi
done
# 重试完还是失败,不用本地缓存最后试一次(只从registry读)
echo "⚠️ All cached attempts failed, building without local cache..."
timeout ${BUILD_TIMEOUT} docker buildx build \
$NO_CACHE_FLAG \
$BUILD_ARGS \
--cache-from "type=registry,ref=${CACHE_REF}" \
--cache-to "type=local,dest=${LOCAL_CACHE_DIR},mode=max" \
--cache-to "type=registry,ref=${CACHE_REF},mode=max,ignore-error=true" \
-f "${DOCKERFILE}" \
-t "${IMAGE_TAG}" \
--push \
.
return 1
}
echo "=== Step 1: Build & push image (local cache + registry cache, with auto-repair) ==="
echo "Local cache: ${LOCAL_CACHE_DIR}"
echo "Registry cache: ${CACHE_REF}"
echo "Build timeout: ${BUILD_TIMEOUT}s"
echo "=== Build & push image (persistent builder cache) ==="
echo "Builder: ${BUILDER_NAME} (persistent)"
echo "Registry cache(from): ${CACHE_REF}"
echo "Image tag: ${IMAGE_TAG}"
[ -n "$EXTRA_TAG_LIST" ] && echo "Extra tags: ${EXTRA_TAG_LIST}"
echo "Timeout: ${BUILD_TIMEOUT}s"
echo ""
build_with_cache_retry
echo ""
echo "Image pushed: ${IMAGE_TAG}"
echo "Local cache updated"
echo "Registry cache updated (if supported)"
[ -n "$EXTRA_TAG_LIST" ] && echo "Also pushed: ${EXTRA_TAG_LIST}"
echo ""
echo "Build completed: ${IMAGE_TAG}"
+25
View File
@@ -0,0 +1,25 @@
#!/bin/bash
# CI 公共步骤:确保宿主机持久 buildx builder 存在(DooD 模式下所有 job 共享)
# - builder 名固定: ci-builder-persist
# - docker-container driver, host 网络
# - 层缓存保存在 buildkit 容器及其 _state 命名卷中,job 结束不清理
# - 宿主机 ci-docker-cleanup.sh 已豁免该 builder
# 用法: bash scripts/ci/ensure_persistent_builder.sh
set -eu
BUILDER="ci-builder-persist"
if ! docker buildx inspect "$BUILDER" >/dev/null 2>&1; then
echo "=== 创建持久 buildx builder: $BUILDER ==="
docker buildx create --name "$BUILDER" --driver docker-container \
--driver-opt network=host \
--buildkitd-flags "--allow-insecure-entitlement network.host" \
--platform linux/amd64
else
echo "=== 复用持久 buildx builder: $BUILDER ==="
fi
docker buildx use "$BUILDER"
docker buildx inspect "$BUILDER" --bootstrap
echo "✅ builder ready"
docker buildx ls | head -5
+47
View File
@@ -0,0 +1,47 @@
#!/bin/bash
# CI 步骤:未重建的镜像,把 registry 上一个分支 tag 复制为新 SHA tag
# 保证 deploy-staging 的 Watchtower 链路三个镜像都有新 SHA 可拉
# 用法: bash scripts/ci/retag_skipped_image.sh <image_full_name> <new_sha> <branch>
# 例: bash scripts/ci/retag_skipped_image.sh xiaoxia-registry.cn-hangzhou.cr.aliyuncs.com/xiaoxiakeji/xiaoxia-saas-web <sha> develop
set -eu
IMAGE="$1"
NEW_TAG="$2"
BRANCH="${3:-develop}"
NEW_REF="${IMAGE}:${NEW_TAG}"
echo "=== 复用已有镜像(本次未重建): $IMAGE ==="
echo "目标 tag: $NEW_TAG"
# 源 tag 候选(按优先级)
CANDIDATES=()
# 1. 分支 tag(构建 job 每次成功都会推)
CANDIDATES+=("$BRANCH")
# 2. 本 push 的前一个 commit SHAcompare 事件)
if [ -n "${GITHUB_EVENT_BEFORE:-}" ] && [ "${GITHUB_EVENT_BEFORE}" != "0000000000000000000000000000000000000000" ]; then
CANDIDATES+=("${GITHUB_EVENT_BEFORE}")
fi
# 3. registry 上最新的 sha 形式 tag(通过 ACR tags API 兜底,不需要额外认证则跳过)
SRC_TAG=""
for cand in "${CANDIDATES[@]}"; do
echo "尝试拉取 ${IMAGE}:${cand} ..."
if docker pull "${IMAGE}:${cand}" >/dev/null 2>&1; then
SRC_TAG="$cand"
break
fi
done
if [ -z "$SRC_TAG" ]; then
echo "❌ 找不到可复用的源镜像(已尝试: ${CANDIDATES[*]}"
echo " 请检查该镜像是否曾成功构建推送,或临时改用全量构建。"
exit 1
fi
echo "✅ 源镜像: ${IMAGE}:${SRC_TAG}"
docker tag "${IMAGE}:${SRC_TAG}" "${NEW_REF}"
# 推新 SHA tag;分支 tag 若指向的就是源 digest 则无需重复,失败可忽略
docker push "${NEW_REF}"
echo "✅ retag 推送完成: ${NEW_REF} (from ${SRC_TAG})"