Compare commits

..

71 Commits

Author SHA1 Message Date
xiaoxia 545ff0fab8 fix(ci): restore execute permission for ci_notify.py and healthcheck script
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 24s
CI/CD Pipeline / Validate - Migration (alembic) (pull_request) Successful in 1m17s
CI/CD Pipeline / Validate - Type Check (mypy) (pull_request) Successful in 1m24s
CI/CD Pipeline / PR Build API Image (Backend) (pull_request) Successful in 1m35s
AI Code Review / AI Code Review (pull_request) Failing after 2m12s
Preview Deploy / Deploy Preview Environment (pull_request) Failing after 2m22s
CI/CD Pipeline / PR Build Worker Image (Backend) (pull_request) Failing after 2m23s
CI/CD Pipeline / Unit Tests (pull_request) Failing after 2m55s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m30s
CI/CD Pipeline / AI Code Review (pull_request) Failing after 3m56s
CI/CD Pipeline / Validate - Code Quality (pull_request) Successful in 5m59s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m42s
CI/CD Pipeline / CI Gate (pull_request) Failing after 8s
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 7s
ACR Cleanup / ACR Image Cleanup (pull_request_target) Successful in 8s
CI/CD Pipeline / Canary Release to Production (pull_request) Failing after 1182h0m35s
CI/CD Pipeline / Build Production Worker Image (pull_request) Failing after 1182h0m38s
CI/CD Pipeline / Production Browser E2E (pull_request) Failing after 1182h0m35s
CI/CD Pipeline / Build Production Web Image (pull_request) Failing after 1182h0m39s
CI/CD Pipeline / Build Production API Image (pull_request) Failing after 1182h0m39s
CI/CD Pipeline / Deploy Production (pull_request) Failing after 1182h0m36s
CI/CD Pipeline / PR Build Web Image (web-cache, infra/docker/web.Dockerfile, xiaoxia-saas-web, web, Web, 30) (pull_request) Failing after 1182h7m56s
CI/CD Pipeline / Frontend Lint (pull_request) Failing after 1182h7m58s
CI/CD Pipeline / ACR Image Cleanup (pull_request) Failing after 1182h8m18s
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 1182h8m20s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Failing after 1182h8m20s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Failing after 1182h8m22s
CI/CD Pipeline / Build Staging Worker Image (pull_request) Failing after 1182h8m22s
CI/CD Pipeline / Build Staging API Image (pull_request) Failing after 1182h8m22s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Failing after 1182h40m56s
CI/CD Pipeline / Staging API Integration Tests (pull_request) Failing after 1182h41m16s
CI/CD Pipeline / Build Staging Web Image (pull_request) Failing after 1182h41m20s
2026-07-29 09:19:16 +08:00
xiaoxia 89a8c8b6fb merge(ci): sync CI scripts from develop - staging deploy/healthcheck/notify/build tools 2026-07-29 09:19:03 +08:00
xiaoxia e4997b9b4a fix(ci): exclude e2e directory from vitest
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 1m42s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 1m51s
CI/CD Pipeline / Frontend Unit Tests (push) Successful in 2m13s
CI/CD Pipeline / Frontend Lint (push) Successful in 2m20s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 2m38s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 2m38s
CI/CD Pipeline / Build Staging API Image (push) Successful in 3m3s
CI/CD Pipeline / Validate - Code Quality (push) Successful in 3m25s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Failing after 45s
CI/CD Pipeline / Unit Tests (push) Successful in 4m36s
CI/CD Pipeline / Integration Tests (push) Successful in 2m0s
CI/CD Pipeline / Build Production API Image (push) Successful in 33s
CI/CD Pipeline / Build Production Worker Image (push) Successful in 1m26s
CI/CD Pipeline / Build Production Web Image (push) Successful in 1m34s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1189h33m59s
CI/CD Pipeline / Deploy Production (push) Failing after 1189h34m0s
CI/CD Pipeline / CI Gate (push) Failing after 1189h35m34s
CI/CD Pipeline / Staging API Integration Tests (push) Failing after 1189h37m11s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1189h37m11s
CI/CD Pipeline / PR Build Worker Image (Backend) (push) Failing after 1189h40m59s
CI/CD Pipeline / PR Build API Image (Backend) (push) Failing after 1189h41m0s
CI/CD Pipeline / AI Code Review (push) Failing after 1189h41m1s
CI/CD Pipeline / Check if frontend-only change (push) Failing after 1189h41m2s
CI/CD Pipeline / Canary Release to Production (push) Failing after 1190h6m58s
CI/CD Pipeline / ACR Image Cleanup (push) Failing after 1190h10m8s
CI/CD Pipeline / PR Build Web Image (web-cache, infra/docker/web.Dockerfile, xiaoxia-saas-web, web, Web, 30) (push) Failing after 1190h13m56s
Fix Frontend Unit Tests failure on main branch by excluding e2e Playwright tests from Vitest.
2026-07-29 01:46:42 +08:00
xiaoxia 68c89861e9 fix(ci): update package-lock.json for @vitest/coverage-v8
CI/CD Pipeline / Validate - Code Quality (push) Successful in 4m26s
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 1m31s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 1m36s
CI/CD Pipeline / Unit Tests (push) Successful in 3m54s
CI/CD Pipeline / Integration Tests (push) Successful in 2m24s
CI/CD Pipeline / Frontend Lint (push) Successful in 58s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 37s
CI/CD Pipeline / Build Staging API Image (push) Successful in 4m18s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 9m23s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 4m7s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Failing after 48s
CI/CD Pipeline / Canary Release to Production (push) Failing after 1189h41m0s
CI/CD Pipeline / Staging API Integration Tests (push) Failing after 1189h41m2s
CI/CD Pipeline / ACR Image Cleanup (push) Failing after 1189h41m2s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1189h41m2s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1189h44m39s
CI/CD Pipeline / Deploy Production (push) Failing after 1189h44m40s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 1189h44m41s
CI/CD Pipeline / Build Production Web Image (push) Failing after 1189h44m41s
CI/CD Pipeline / Build Production API Image (push) Failing after 1189h44m41s
CI/CD Pipeline / PR Build Web Image (web-cache, infra/docker/web.Dockerfile, xiaoxia-saas-web, web, Web, 30) (push) Failing after 1189h48m36s
CI/CD Pipeline / PR Build Worker Image (Backend) (push) Failing after 1189h48m37s
CI/CD Pipeline / PR Build API Image (Backend) (push) Failing after 1189h48m39s
CI/CD Pipeline / AI Code Review (push) Failing after 1189h51m27s
CI/CD Pipeline / Check if frontend-only change (push) Failing after 1189h51m35s
CI/CD Pipeline / CI Gate (push) Failing after 1190h17m39s
2026-07-29 01:31:53 +08:00
xiaoxia 076601b431 fix(ci): add @vitest/coverage-v8 for main branch coverage
CI/CD Pipeline / Validate - Code Quality (push) Successful in 3m35s
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 1m38s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 1m41s
CI/CD Pipeline / Unit Tests (push) Successful in 4m22s
CI/CD Pipeline / Integration Tests (push) Successful in 1m54s
CI/CD Pipeline / Frontend Lint (push) Failing after 14m37s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 14m32s
CI/CD Pipeline / PR Build API Image (Backend) (push) Failing after 14m27s
CI/CD Pipeline / PR Build Worker Image (Backend) (push) Failing after 14m22s
CI/CD Pipeline / Build Staging API Image (push) Successful in 1m18s
CI/CD Pipeline / Build Staging Web Image (push) Failing after 2m33s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 14m42s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1189h48m34s
CI/CD Pipeline / Deploy Production (push) Failing after 1189h49m42s
CI/CD Pipeline / Canary Release to Production (push) Failing after 1189h48m35s
CI/CD Pipeline / Staging API Integration Tests (push) Failing after 1189h49m46s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1189h49m48s
CI/CD Pipeline / CI Gate (push) Failing after 1189h51m37s
CI/CD Pipeline / Build Production Web Image (push) Failing after 1189h51m39s
CI/CD Pipeline / Build Production API Image (push) Failing after 1189h51m39s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Failing after 1189h51m41s
CI/CD Pipeline / PR Build Web Image (web-cache, infra/docker/web.Dockerfile, xiaoxia-saas-web, web, Web, 30) (push) Failing after 1190h7m31s
CI/CD Pipeline / AI Code Review (push) Failing after 1190h7m56s
CI/CD Pipeline / Check if frontend-only change (push) Failing after 1190h7m58s
CI/CD Pipeline / ACR Image Cleanup (push) Failing after 1190h22m42s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 1190h24m35s
2026-07-29 01:19:55 +08:00
xiaoxia e23a60e98b Merge remote-tracking branch 'origin/fix/code-review-ci-gate'
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 1m32s
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 1m34s
CI/CD Pipeline / Frontend Lint (push) Successful in 1m51s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 2m8s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 2m30s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 2m31s
CI/CD Pipeline / Build Staging API Image (push) Successful in 2m53s
CI/CD Pipeline / Unit Tests (push) Successful in 4m41s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Failing after 45s
CI/CD Pipeline / Validate - Code Quality (push) Successful in 5m0s
CI/CD Pipeline / Integration Tests (push) Successful in 1m54s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1190h20m56s
CI/CD Pipeline / Deploy Production (push) Failing after 1190h20m56s
CI/CD Pipeline / CI Gate (push) Failing after 1190h20m57s
CI/CD Pipeline / Build Production Web Image (push) Failing after 1190h20m57s
CI/CD Pipeline / Build Production API Image (push) Failing after 1190h20m57s
CI/CD Pipeline / Staging API Integration Tests (push) Failing after 1190h22m54s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1190h22m54s
CI/CD Pipeline / PR Build Web Image (web-cache, infra/docker/web.Dockerfile, xiaoxia-saas-web, web, Web, 30) (push) Failing after 1190h27m49s
CI/CD Pipeline / PR Build API Image (Backend) (push) Failing after 1190h27m51s
CI/CD Pipeline / AI Code Review (push) Failing after 1190h27m52s
CI/CD Pipeline / Check if frontend-only change (push) Failing after 1190h27m53s
CI/CD Pipeline / Canary Release to Production (push) Failing after 1190h53m54s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 1190h53m55s
CI/CD Pipeline / ACR Image Cleanup (push) Failing after 1190h55m52s
CI/CD Pipeline / PR Build Worker Image (Backend) (push) Failing after 1191h0m48s
# Conflicts:
#	.gitea/workflows/ci-pipeline.yml
2026-07-29 00:40:32 +08:00
xiaoxia e2e91e1d58 Merge remote-tracking branch 'origin/fix/pr-path-filter-skip' 2026-07-29 00:39:46 +08:00
xiaoxia 682972469c Merge remote-tracking branch 'origin/fix/auto-merge-short-job'
CI/CD Pipeline / AI Code Review (push) Has been skipped
CI/CD Pipeline / PR Build API Image (Backend) (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (Backend) (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (web-cache, infra/docker/web.Dockerfile, xiaoxia-saas-web, web, Web, 30) (push) Has been skipped
CI/CD Pipeline / Validate - Code Quality (push) Successful in 4m11s
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 1m28s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 1m25s
CI/CD Pipeline / Unit Tests (push) Successful in 3m47s
CI/CD Pipeline / Integration Tests (push) Successful in 1m41s
CI/CD Pipeline / Frontend Lint (push) Successful in 2m8s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 1m54s
CI/CD Pipeline / Build Staging API Image (push) Successful in 2m8s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 2m38s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 22m29s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Failing after 4m11s
CI/CD Pipeline / Canary Release to Production (push) Failing after 1190h23m41s
CI/CD Pipeline / ACR Image Cleanup (push) Failing after 1190h23m42s
CI/CD Pipeline / Staging API Integration Tests (push) Failing after 1190h23m42s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1190h23m42s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1190h44m27s
CI/CD Pipeline / Deploy Production (push) Failing after 1190h44m27s
CI/CD Pipeline / Check if frontend-only change (push) Failing after 1190h50m24s
CI/CD Pipeline / CI Gate (push) Failing after 1190h44m27s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 1190h44m28s
CI/CD Pipeline / Build Production Web Image (push) Failing after 1190h44m28s
CI/CD Pipeline / Build Production API Image (push) Failing after 1190h44m28s
CI/CD Pipeline / PR Build Worker Image (push) Failing after 1190h50m23s
CI/CD Pipeline / PR Build Web Image (push) Failing after 1190h50m23s
CI/CD Pipeline / PR Build API Image (push) Failing after 1191h23m21s
2026-07-29 00:37:02 +08:00
xiaoxia f8b3552a4e Merge remote-tracking branch 'origin/fix/build-production-needs-tests' 2026-07-29 00:37:02 +08:00
xiaoxia 5e0dcaead8 Merge remote-tracking branch 'origin/fix/pr-automation-concurrency' 2026-07-29 00:37:02 +08:00
xiaoxia 1eca707db3 Merge remote-tracking branch 'origin/fix/preview-cleanup-ssh-default' 2026-07-29 00:37:02 +08:00
xiaoxia 5331307cbc Merge remote-tracking branch 'origin/fix/daily-check-outputs' 2026-07-29 00:37:02 +08:00
xiaoxia 3bea00aa56 feat(ci): 基础镜像切换到私有ACR,更稳定
CI/CD Pipeline / Frontend Lint (push) Successful in 41s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 1m15s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 1m49s
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 1m56s
CI/CD Pipeline / Build Production Web Image (push) Successful in 2m7s
CI/CD Pipeline / Build Production API Image (push) Successful in 2m10s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 2m20s
CI/CD Pipeline / Build Staging API Image (push) Successful in 2m21s
CI/CD Pipeline / Build Production Worker Image (push) Successful in 2m51s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 2m53s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Failing after 51s
CI/CD Pipeline / Validate - Code Quality (push) Successful in 4m56s
CI/CD Pipeline / Unit Tests (push) Successful in 5m0s
CI/CD Pipeline / Integration Tests (push) Successful in 1m57s
CI/CD Pipeline / CI Gate (push) Failing after 1191h18m14s
CI/CD Pipeline / ACR Image Cleanup (push) Failing after 1191h21m21s
CI/CD Pipeline / Staging API Integration Tests (push) Failing after 1191h21m21s
CI/CD Pipeline / Deploy Production (push) Failing after 1191h22m15s
CI/CD Pipeline / PR Build Worker Image (push) Failing after 1191h25m9s
CI/CD Pipeline / PR Build API Image (push) Failing after 1191h25m10s
CI/CD Pipeline / Check if frontend-only change (push) Failing after 1191h25m10s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1191h21m21s
CI/CD Pipeline / Canary Release to Production (push) Failing after 1191h54m18s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1191h55m12s
CI/CD Pipeline / PR Build Web Image (push) Failing after 1191h58m8s
- Bug6(P1): 所有Dockerfile基础镜像从daocloud切到私有ACR (xiaoxia-registry.cn-hangzhou.cr.aliyuncs.com/xiaoxiakeji/base/)
- 新增scripts/ci/sync_base_images.sh - 基础镜像同步脚本
- 覆盖: python:3.12-slim-bookworm / python:3.12-slim / node:20 / nginx:alpine
2026-07-28 22:38:37 +08:00
xiaoxia 1addcffeba fix(ci): 修复Staging部署和CI通知缺失脚本
CI/CD Pipeline / Frontend Lint (push) Successful in 1m14s
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 1m16s
CI/CD Pipeline / Build Production Worker Image (push) Successful in 1m54s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 1m57s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 1m56s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 2m8s
CI/CD Pipeline / Build Production Web Image (push) Successful in 2m50s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 2m53s
CI/CD Pipeline / Validate - Code Quality (push) Successful in 5m2s
CI/CD Pipeline / Unit Tests (push) Successful in 5m2s
CI/CD Pipeline / Integration Tests (push) Successful in 2m30s
CI/CD Pipeline / Build Production API Image (push) Successful in 14m30s
CI/CD Pipeline / Build Staging API Image (push) Successful in 14m37s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Failing after 4m13s
CI/CD Pipeline / Canary Release to Production (push) Failing after 1192h30m57s
CI/CD Pipeline / Staging API Integration Tests (push) Failing after 1192h30m58s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1192h30m58s
CI/CD Pipeline / CI Gate (push) Failing after 1192h42m16s
CI/CD Pipeline / PR Build Worker Image (push) Failing after 1192h49m52s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1192h35m18s
CI/CD Pipeline / Check if frontend-only change (push) Failing after 1192h49m52s
CI/CD Pipeline / PR Build Web Image (push) Failing after 1192h49m52s
CI/CD Pipeline / ACR Image Cleanup (push) Failing after 1193h3m56s
CI/CD Pipeline / Deploy Production (push) Failing after 1193h8m17s
CI/CD Pipeline / PR Build API Image (push) Failing after 1193h22m50s
- Bug4(P0): 新增scripts/ci_staging_deploy.sh - Staging环境部署脚本(pull镜像+migration+重启)
- Bug4(P0): 新增scripts/ci_staging_healthcheck.sh - Staging健康检查脚本(API/Worker/Web)
- Bug5(P1): 新增scripts/ci_notify.py - 飞书通知脚本(start/success/failure模式)
2026-07-28 22:37:36 +08:00
xiaoxia f3819653b8 fix(ci): 修复3个部署bug - 生产构建/Preview部署/镜像源
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 17s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 38s
AI Code Review / AI Code Review (pull_request) Successful in 18s
Preview Deploy / Deploy Preview Environment (pull_request) Failing after 19s
CI/CD Pipeline / Validate - Type Check (mypy) (pull_request) Successful in 1m59s
CI/CD Pipeline / Validate - Migration (alembic) (pull_request) Successful in 2m0s
CI/CD Pipeline / Frontend Lint (push) Successful in 27s
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 1m30s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 1m31s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m32s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m24s
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 4m37s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 4m38s
CI/CD Pipeline / Validate - Code Quality (pull_request) Successful in 4m48s
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 7s
ACR Cleanup / ACR Image Cleanup (pull_request_target) Successful in 7s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 25s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 5m44s
CI/CD Pipeline / Validate - Code Quality (push) Successful in 4m59s
CI/CD Pipeline / Build Production Web Image (push) Successful in 1m23s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 6m55s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 3m47s
CI/CD Pipeline / Integration Tests (push) Successful in 3m20s
CI/CD Pipeline / CI Gate (pull_request) Successful in 5s
CI/CD Pipeline / Unit Tests (push) Successful in 5m36s
CI/CD Pipeline / Build Production API Image (push) Successful in 22m11s
CI/CD Pipeline / Build Staging API Image (push) Successful in 25m11s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 1h2m27s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Failing after 15s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 1h0m32s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1193h21m53s
CI/CD Pipeline / Staging API Integration Tests (push) Failing after 1193h21m57s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1193h21m57s
CI/CD Pipeline / Canary Release to Production (push) Failing after 1193h21m54s
CI/CD Pipeline / CI Gate (push) Failing after 1194h17m25s
CI/CD Pipeline / Deploy Production (push) Failing after 1193h21m54s
CI/CD Pipeline / Canary Release to Production (pull_request) Failing after 1194h21m53s
CI/CD Pipeline / ACR Image Cleanup (pull_request) Failing after 1194h23m4s
CI/CD Pipeline / Production Browser E2E (pull_request) Failing after 1194h21m55s
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 1194h23m8s
CI/CD Pipeline / Deploy Production (pull_request) Failing after 1194h23m17s
CI/CD Pipeline / PR Build Worker Image (push) Failing after 1194h25m45s
CI/CD Pipeline / Check if frontend-only change (push) Failing after 1194h26m59s
CI/CD Pipeline / PR Build Web Image (push) Failing after 1194h25m47s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Failing after 1194h26m59s
CI/CD Pipeline / Build Production Web Image (pull_request) Failing after 1194h27m3s
CI/CD Pipeline / Build Production API Image (pull_request) Failing after 1194h27m5s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Failing after 1194h27m7s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Failing after 1194h27m20s
CI/CD Pipeline / Build Staging Worker Image (pull_request) Failing after 1194h27m39s
CI/CD Pipeline / Build Staging API Image (pull_request) Failing after 1194h27m43s
CI/CD Pipeline / ACR Image Cleanup (push) Failing after 1193h54m54s
CI/CD Pipeline / Staging API Integration Tests (pull_request) Failing after 1194h56m4s
CI/CD Pipeline / PR Build API Image (push) Failing after 1194h58m47s
CI/CD Pipeline / Build Production Worker Image (pull_request) Failing after 1194h59m59s
CI/CD Pipeline / Build Staging Web Image (pull_request) Failing after 1195h0m39s
2026-07-28 20:59:36 +08:00
CI Bot b73d75a3e4 fix(ci): 统一preview-cleanup与preview-deploy的SSH默认配置
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 7s
CI/CD Pipeline / Validate - Type Check (mypy) (pull_request) Successful in 1m39s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 23s
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 42s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 41s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 52s
Preview Deploy / Deploy Preview Environment (pull_request) Failing after 10s
CI/CD Pipeline / Validate - Migration (alembic) (pull_request) Successful in 1m44s
AI Code Review / AI Code Review (pull_request) Failing after 1m17s
CI/CD Pipeline / Validate - Code Quality (pull_request) Successful in 4m48s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m0s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 7m41s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 11m29s
CI/CD Pipeline / CI Gate (pull_request) Successful in 3s
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 26s
ACR Cleanup / ACR Image Cleanup (pull_request_target) Successful in 9s
CI/CD Pipeline / Canary Release to Production (pull_request) Failing after 1197h43m53s
CI/CD Pipeline / Production Browser E2E (pull_request) Failing after 1197h43m55s
CI/CD Pipeline / ACR Image Cleanup (pull_request) Failing after 1197h43m56s
CI/CD Pipeline / Staging API Integration Tests (pull_request) Failing after 1197h43m57s
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 1197h43m58s
CI/CD Pipeline / Deploy Production (pull_request) Failing after 1197h44m1s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Failing after 1197h44m3s
CI/CD Pipeline / Build Production Worker Image (pull_request) Failing after 1197h44m19s
CI/CD Pipeline / Build Production API Image (pull_request) Failing after 1197h44m23s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Failing after 1197h44m25s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Failing after 1197h44m42s
CI/CD Pipeline / Build Staging Worker Image (pull_request) Failing after 1197h45m11s
CI/CD Pipeline / Build Staging API Image (pull_request) Failing after 1197h45m13s
CI/CD Pipeline / Build Production Web Image (pull_request) Failing after 1198h17m18s
CI/CD Pipeline / Build Staging Web Image (pull_request) Failing after 1198h18m8s
- SSH_HOST默认值: 172.30.18.197 → 47.98.113.167(与preview-deploy一致)
- SSH_USER默认值: deploy → root(与preview-deploy一致)

问题:preview-cleanup的SSH默认值与preview-deploy不一致。如果PREVIEW_SSH_HOST等secret
未设置,部署和清理会连到不同的服务器,导致清理失效。
修复:统一为与preview-deploy相同的默认值,确保部署和清理在同一台服务器上。
2026-07-28 17:36:49 +08:00
CI Bot 62967e078b fix(ci): 修复daily-check.yml中两个job的outputs配置错误
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 12s
CI/CD Pipeline / Validate - Type Check (mypy) (pull_request) Successful in 1m56s
CI/CD Pipeline / Validate - Migration (alembic) (pull_request) Successful in 1m47s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 24s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 49s
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 30s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 25s
Preview Deploy / Deploy Preview Environment (pull_request) Failing after 14s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m51s
CI/CD Pipeline / Validate - Code Quality (pull_request) Successful in 6m0s
AI Code Review / AI Code Review (pull_request) Successful in 4m16s
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
CI/CD Pipeline / Integration Tests (pull_request) Successful in 5m14s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 9m33s
CI/CD Pipeline / CI Gate (pull_request) Successful in 5s
ACR Cleanup / ACR Image Cleanup (pull_request_target) Has been cancelled
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 32s
CI/CD Pipeline / Production Browser E2E (pull_request) Failing after 1197h44m2s
CI/CD Pipeline / ACR Image Cleanup (pull_request) Failing after 1197h44m34s
CI/CD Pipeline / Deploy Production (pull_request) Failing after 1197h44m36s
CI/CD Pipeline / Staging API Integration Tests (pull_request) Failing after 1197h44m38s
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 1197h44m40s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Failing after 1197h47m22s
CI/CD Pipeline / Build Production API Image (pull_request) Failing after 1197h47m28s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Failing after 1197h47m30s
CI/CD Pipeline / Build Production Worker Image (pull_request) Failing after 1197h47m24s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Failing after 1197h48m26s
CI/CD Pipeline / Build Staging Worker Image (pull_request) Failing after 1197h48m28s
CI/CD Pipeline / Build Staging Web Image (pull_request) Failing after 1197h48m28s
CI/CD Pipeline / Build Staging API Image (pull_request) Failing after 1197h48m28s
CI/CD Pipeline / Build Production Web Image (pull_request) Failing after 1198h20m23s
- staging-api-tests.outputs.report: steps.smoke → steps.report
  (smoke步骤输出的是api_report,不是report;report由report步骤输出)
- staging-e2e.outputs.report: steps.smoke → steps.e2e
  (staging-e2e的step id是e2e,不是smoke)

问题:两个job的outputs引用了错误的step id或变量名,导致汇总报告拿不到正确结果。
修复:修正outputs指向正确的step id和输出变量。
2026-07-28 17:35:32 +08:00
CI Bot 2818f44282 fix(ci): auto-merge从轮询长作业改为短作业模式
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 13s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 24s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 8s
CI/CD Pipeline / Validate - Type Check (mypy) (pull_request) Successful in 1m26s
CI/CD Pipeline / Validate - Migration (alembic) (pull_request) Successful in 1m24s
Preview Deploy / Deploy Preview Environment (pull_request) Failing after 25s
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 1m44s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 1m36s
AI Code Review / AI Code Review (pull_request) Successful in 2m3s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m57s
CI/CD Pipeline / Validate - Code Quality (pull_request) Successful in 7m10s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 9m58s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 4m41s
CI/CD Pipeline / CI Gate (pull_request) Successful in 3s
ACR Cleanup / ACR Image Cleanup (pull_request_target) Has been cancelled
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 34s
CI/CD Pipeline / Canary Release to Production (pull_request) Failing after 1197h47m20s
CI/CD Pipeline / Production Browser E2E (pull_request) Failing after 1197h47m22s
CI/CD Pipeline / ACR Image Cleanup (pull_request) Failing after 1197h48m4s
CI/CD Pipeline / Staging API Integration Tests (pull_request) Failing after 1197h48m6s
CI/CD Pipeline / Deploy Production (pull_request) Failing after 1197h48m10s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Failing after 1197h51m37s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Failing after 1197h51m44s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Failing after 1197h52m12s
CI/CD Pipeline / Build Production Worker Image (pull_request) Failing after 1197h51m39s
CI/CD Pipeline / Build Production Web Image (pull_request) Failing after 1197h51m40s
CI/CD Pipeline / Build Staging Worker Image (pull_request) Failing after 1197h52m56s
CI/CD Pipeline / Build Staging API Image (pull_request) Failing after 1197h53m0s
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 1198h21m5s
CI/CD Pipeline / Build Production API Image (pull_request) Failing after 1198h24m39s
CI/CD Pipeline / Build Staging Web Image (pull_request) Failing after 1198h25m55s
- auto_merge.sh: 去掉90次x30秒的轮询循环,改为单次检查
- 超时从45分钟缩短为3分钟
- CI未通过/未就绪时直接退出,不占用Runner
- 由pr-auto-scan每5分钟定时扫描兜底,CI通过后自动合并
- 保留405重试机制(单次运行内最多3次)

问题:auto-merge job轮询等待CI通过,最长占Runner 45分钟,资源浪费。
修复:改为短作业模式——检查一次CI状态,满足就合并,不满足就退出。
兜底:pr-auto-scan.yml每5分钟扫描所有open PR,CI全绿的自动审批+合并。
2026-07-28 17:32:56 +08:00
CI Bot c9e5129ec8 fix(ci): build-production等待测试通过后再构建生产镜像
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 20s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 30s
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 32s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 54s
CI/CD Pipeline / Validate - Migration (alembic) (pull_request) Successful in 1m22s
CI/CD Pipeline / Validate - Type Check (mypy) (pull_request) Successful in 1m27s
Preview Deploy / Deploy Preview Environment (pull_request) Failing after 31s
AI Code Review / AI Code Review (pull_request) Successful in 56s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m11s
CI/CD Pipeline / Validate - Code Quality (pull_request) Successful in 4m36s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 9m5s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 6m12s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 12m8s
CI/CD Pipeline / CI Gate (pull_request) Successful in 3s
ACR Cleanup / ACR Image Cleanup (pull_request_target) Has been cancelled
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 30s
CI/CD Pipeline / Production Browser E2E (pull_request) Failing after 1197h43m54s
CI/CD Pipeline / Canary Release to Production (pull_request) Failing after 1197h43m59s
CI/CD Pipeline / Deploy Production (pull_request) Failing after 1197h44m0s
CI/CD Pipeline / Build Production Worker Image (pull_request) Failing after 1197h44m4s
CI/CD Pipeline / Build Production API Image (pull_request) Failing after 1197h44m6s
CI/CD Pipeline / ACR Image Cleanup (pull_request) Failing after 1197h52m50s
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 1197h52m54s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Failing after 1197h54m39s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Failing after 1197h54m41s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Failing after 1197h55m12s
CI/CD Pipeline / Build Staging Worker Image (pull_request) Failing after 1197h55m25s
CI/CD Pipeline / Build Staging API Image (pull_request) Failing after 1197h55m27s
CI/CD Pipeline / Build Production Web Image (pull_request) Failing after 1198h17m2s
CI/CD Pipeline / Staging API Integration Tests (pull_request) Failing after 1198h25m49s
CI/CD Pipeline / Build Staging Web Image (pull_request) Failing after 1198h28m23s
- 为build-production添加7个测试/检查job作为needs依赖
- 测试失败时不构建生产镜像,避免无效构建占用资源

问题:生产镜像构建和测试并行跑,测试失败了镜像也已经build完了,浪费构建资源。
修复:让build-production等所有核心测试通过后再开始构建。
2026-07-28 17:30:25 +08:00
CI Bot cd3d366f4a fix(ci): 纯前端/纯后端PR跳过无关检查,节省Runner资源
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 7s
Preview Deploy / Deploy Preview Environment (pull_request) Failing after 14s
CI/CD Pipeline / Validate - Type Check (mypy) (pull_request) Successful in 1m13s
CI/CD Pipeline / Validate - Migration (alembic) (pull_request) Successful in 1m16s
CI/CD Pipeline / PR Build API Image (Backend) (pull_request) Successful in 48s
CI/CD Pipeline / PR Build Worker Image (Backend) (pull_request) Successful in 49s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m0s
AI Code Review / AI Code Review (pull_request) Failing after 4m8s
CI/CD Pipeline / Validate - Code Quality (pull_request) Successful in 5m4s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 6m27s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 6m15s
CI/CD Pipeline / CI Gate (pull_request) Successful in 4s
ACR Cleanup / ACR Image Cleanup (pull_request_target) Has been cancelled
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 31s
CI/CD Pipeline / Canary Release to Production (pull_request) Failing after 1197h55m8s
CI/CD Pipeline / Production Browser E2E (pull_request) Failing after 1197h55m10s
CI/CD Pipeline / ACR Image Cleanup (pull_request) Failing after 1197h57m2s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Failing after 1197h57m30s
CI/CD Pipeline / Staging API Integration Tests (pull_request) Failing after 1197h57m4s
CI/CD Pipeline / Frontend Lint (pull_request) Failing after 1197h57m32s
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 1197h57m6s
CI/CD Pipeline / Build Production Worker Image (pull_request) Failing after 1197h58m10s
CI/CD Pipeline / Deploy Production (pull_request) Failing after 1197h57m8s
CI/CD Pipeline / Build Production Web Image (pull_request) Failing after 1197h58m12s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Failing after 1197h58m30s
CI/CD Pipeline / Build Production API Image (pull_request) Failing after 1197h58m14s
CI/CD Pipeline / Build Staging Worker Image (pull_request) Failing after 1197h58m41s
CI/CD Pipeline / Build Staging API Image (pull_request) Failing after 1197h58m45s
CI/CD Pipeline / PR Build Web Image (web-cache, infra/docker/web.Dockerfile, xiaoxia-saas-web, web, Web, 30) (pull_request) Failing after 1198h30m7s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Failing after 1198h31m5s
CI/CD Pipeline / Build Staging Web Image (pull_request) Failing after 1198h31m40s
- frontend-lint: 添加check-frontend-only依赖,纯后端PR自动跳过
- build-pr拆分为build-pr-backend和build-pr-web两个job
  - build-pr-backend (API+Worker): 纯前端PR跳过
  - build-pr-web (Web): 纯后端PR跳过
- CI Gate分类调整:
  - REQUIRED_GENERAL: 仅保留真正的通用检查(code-quality/type-check/migration)
  - REQUIRED_BACKEND: 新增build-pr-backend
  - REQUIRED_FRONTEND: 新增frontend-lint和build-pr-web

问题:改前端文件也会跑后端测试和构建,反之亦然,浪费大量Runner时间。
修复:利用已有的check-frontend-only job输出,按PR类型跳过无关检查。
效果:纯前端PR节省约15分钟(API+Worker构建),纯后端PR节省约10分钟(前端Lint+Web构建)。
2026-07-28 17:25:31 +08:00
CI Bot 1dd640da87 fix(ci): 将AI Code Review接入CI门禁体系,严重问题拦截合并
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 8s
CI/CD Pipeline / Validate - Type Check (mypy) (pull_request) Successful in 1m13s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 26s
CI/CD Pipeline / Validate - Migration (alembic) (pull_request) Successful in 1m10s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 7s
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 44s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 48s
Preview Deploy / Deploy Preview Environment (pull_request) Failing after 14s
CI/CD Pipeline / AI Code Review (pull_request) Successful in 1m40s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m51s
AI Code Review / AI Code Review (pull_request) Failing after 3m7s
CI/CD Pipeline / Validate - Code Quality (pull_request) Successful in 5m9s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 5m52s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 3m13s
CI/CD Pipeline / CI Gate (pull_request) Successful in 12s
ACR Cleanup / ACR Image Cleanup (pull_request_target) Has been cancelled
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 33s
CI/CD Pipeline / Canary Release to Production (pull_request) Failing after 1198h1m45s
CI/CD Pipeline / ACR Image Cleanup (pull_request) Failing after 1198h1m49s
CI/CD Pipeline / Staging API Integration Tests (pull_request) Failing after 1198h1m51s
CI/CD Pipeline / Deploy Production (pull_request) Failing after 1198h1m55s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Failing after 1198h2m8s
CI/CD Pipeline / Build Production Web Image (pull_request) Failing after 1198h2m19s
CI/CD Pipeline / Build Production API Image (pull_request) Failing after 1198h2m21s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Failing after 1198h2m23s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Failing after 1198h2m51s
CI/CD Pipeline / Build Staging Worker Image (pull_request) Failing after 1198h3m0s
CI/CD Pipeline / Build Staging API Image (pull_request) Failing after 1198h3m3s
CI/CD Pipeline / Production Browser E2E (pull_request) Failing after 1198h34m44s
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 1198h34m50s
CI/CD Pipeline / Build Production Worker Image (pull_request) Failing after 1198h35m14s
CI/CD Pipeline / Build Staging Web Image (pull_request) Failing after 1198h35m59s
- 在ci-pipeline中新增code-review job,与code-review.yml逻辑一致
- 将code-review加入CI Gate的needs列表和REQUIRED_GENERAL检查项
- AI审查发现阻塞级问题时,CI Gate失败,阻止PR合并

问题:AI Code Review只发表评论不参与门禁,有严重安全/质量问题的代码也能合并。
修复:将code-review纳入CI Gate,审查脚本exit 1(阻塞级问题)时CI整体失败。
注意:LLM调用异常时脚本exit 0(fail-open策略),不阻塞正常合并。
2026-07-28 17:20:23 +08:00
CI Bot 4587d0b014 fix(ci): add concurrency to pr-automation workflow
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 15s
CI/CD Pipeline / Validate - Type Check (mypy) (pull_request) Successful in 1m21s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 27s
CI/CD Pipeline / Validate - Migration (alembic) (pull_request) Successful in 1m27s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 50s
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 43s
Preview Deploy / Deploy Preview Environment (pull_request) Failing after 12s
AI Code Review / AI Code Review (pull_request) Successful in 51s
CI/CD Pipeline / Validate - Code Quality (pull_request) Successful in 4m42s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m51s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 3m48s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 6m15s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 3m24s
CI/CD Pipeline / CI Gate (pull_request) Successful in 4s
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 31s
ACR Cleanup / ACR Image Cleanup (pull_request_target) Waiting to run
CI/CD Pipeline / Canary Release to Production (pull_request) Failing after 1198h2m4s
CI/CD Pipeline / Production Browser E2E (pull_request) Failing after 1198h2m6s
CI/CD Pipeline / ACR Image Cleanup (pull_request) Failing after 1198h2m30s
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 1198h2m34s
CI/CD Pipeline / Deploy Production (pull_request) Failing after 1198h2m36s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Failing after 1198h5m2s
CI/CD Pipeline / Build Production Worker Image (pull_request) Failing after 1198h5m4s
CI/CD Pipeline / Build Production Web Image (pull_request) Failing after 1198h5m6s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Failing after 1198h7m9s
CI/CD Pipeline / Build Production API Image (pull_request) Failing after 1198h5m8s
CI/CD Pipeline / Build Staging Worker Image (pull_request) Failing after 1198h7m40s
CI/CD Pipeline / Build Staging API Image (pull_request) Failing after 1198h7m44s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Failing after 1198h5m10s
CI/CD Pipeline / Staging API Integration Tests (pull_request) Failing after 1198h35m29s
CI/CD Pipeline / Build Staging Web Image (pull_request) Failing after 1198h40m40s
- Add workflow-level concurrency group per PR number
- Enable cancel-in-progress to cancel old runs when new commits arrive
- Prevents multiple pr-automation runs for the same PR from wasting Runner resources

Issue: Same PR could trigger multiple auto-approve/auto-merge runs simultaneously,
occupying ci-check runners unnecessarily. With concurrency, only the latest run
executes, and older runs are cancelled automatically.
2026-07-28 17:16:35 +08:00
xiaoxia 4749071c16 Merge pull request 'fix(ci): 修复2个P0级bug - acr-cleanup完全不可用 + production-e2e DooD必然失败' (#1112) from fix/ci-p0-bugs-0728 into main
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 16s
AI Code Review / AI Code Review (pull_request) Successful in 34s
CI/CD Pipeline / Validate - Code Quality (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Type Check (mypy) (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Migration (alembic) (pull_request) Has been cancelled
CI/CD Pipeline / Unit Tests (pull_request) Has been cancelled
CI/CD Pipeline / Integration Tests (pull_request) Has been cancelled
CI/CD Pipeline / Frontend Lint (pull_request) Has been cancelled
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been cancelled
CI/CD Pipeline / PR Build API Image (pull_request) Has been cancelled
CI/CD Pipeline / PR Build Web Image (pull_request) Has been cancelled
CI/CD Pipeline / PR Build Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been cancelled
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been cancelled
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been cancelled
CI/CD Pipeline / Build Production API Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Web Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Production (pull_request) Has been cancelled
CI/CD Pipeline / Production Browser E2E (pull_request) Has been cancelled
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been cancelled
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
CI/CD Pipeline / CI Gate (pull_request) Has been cancelled
Preview Deploy / Deploy Preview Environment (pull_request) Has been cancelled
PR Automation / Auto Approve on CI Green (pull_request) Successful in 43s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Failing after 1194h28m17s
CI/CD Pipeline / Build Staging Worker Image (pull_request) Failing after 1194h29m2s
CI/CD Pipeline / Build Staging Web Image (pull_request) Failing after 1194h29m4s
CI/CD Pipeline / Build Staging API Image (pull_request) Failing after 1194h29m6s
CI/CD Pipeline / PR Build Worker Image (push) Failing after 1199h8m57s
CI/CD Pipeline / PR Build Web Image (push) Failing after 1199h8m59s
CI/CD Pipeline / PR Build API Image (push) Failing after 1199h9m2s
CI/CD Pipeline / Check if frontend-only change (push) Failing after 1199h10m38s
CI/CD Pipeline / Staging E2E Tests (push) Has been cancelled
CI/CD Pipeline / Validate - Code Quality (push) Has been cancelled
CI/CD Pipeline / Validate - Type Check (mypy) (push) Has been cancelled
CI/CD Pipeline / Validate - Migration (alembic) (push) Has been cancelled
CI/CD Pipeline / Unit Tests (push) Has been cancelled
CI/CD Pipeline / Integration Tests (push) Has been cancelled
CI/CD Pipeline / Frontend Lint (push) Has been cancelled
CI/CD Pipeline / Frontend Unit Tests (push) Has been cancelled
CI/CD Pipeline / Build Staging API Image (push) Has been cancelled
CI/CD Pipeline / Build Staging Web Image (push) Has been cancelled
CI/CD Pipeline / Build Staging Worker Image (push) Has been cancelled
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Has been cancelled
CI/CD Pipeline / Staging API Integration Tests (push) Has been cancelled
CI/CD Pipeline / Build Production API Image (push) Has been cancelled
CI/CD Pipeline / Build Production Web Image (push) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (push) Has been cancelled
CI/CD Pipeline / Deploy Production (push) Has been cancelled
CI/CD Pipeline / Production Browser E2E (push) Has been cancelled
CI/CD Pipeline / ACR Image Cleanup (push) Has been cancelled
CI/CD Pipeline / Canary Release to Production (push) Has been cancelled
CI/CD Pipeline / CI Gate (push) Has been cancelled
P0级bug紧急修复
2026-07-28 15:58:05 +08:00
xiaoxia 3353865f5b fix(ci): 修复2个P0级bug
Preview Deploy / Deploy Preview Environment (pull_request) Failing after 18s
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 6s
ACR Cleanup / ACR Image Cleanup (pull_request_target) Successful in 6s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m49s
AI Code Review / AI Code Review (pull_request) Failing after 3m49s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Failing after 1199h11m34s
CI/CD Pipeline / Build Staging Worker Image (pull_request) Failing after 1199h11m53s
CI/CD Pipeline / Build Staging API Image (pull_request) Failing after 1199h11m57s
CI/CD Pipeline / Check if frontend-only change (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Code Quality (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Type Check (mypy) (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Migration (alembic) (pull_request) Has been cancelled
CI/CD Pipeline / Unit Tests (pull_request) Has been cancelled
CI/CD Pipeline / Integration Tests (pull_request) Has been cancelled
CI/CD Pipeline / Frontend Lint (pull_request) Has been cancelled
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been cancelled
CI/CD Pipeline / PR Build API Image (pull_request) Has been cancelled
CI/CD Pipeline / PR Build Web Image (pull_request) Has been cancelled
CI/CD Pipeline / PR Build Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been cancelled
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been cancelled
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been cancelled
CI/CD Pipeline / Build Production API Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Web Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Production (pull_request) Has been cancelled
CI/CD Pipeline / Production Browser E2E (pull_request) Has been cancelled
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been cancelled
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
CI/CD Pipeline / CI Gate (pull_request) Has been cancelled
CI/CD Pipeline / Build Staging Web Image (pull_request) Failing after 1199h44m52s
P0-1: acr-cleanup.yml 完全不可用
- $GITEA_OUTPUT → $GITHUB_OUTPUT(outputs写入完全失效)
- PREVIEW_SSH_KEY → STAGING_SSH_KEY(用错了密钥)
- 增加 STAGING_SSH_HOST/PORT/USER 从secret读取
- SSH连接用户从写死root改为变量

P0-2: production-e2e DooD模式下必然失败
- -v "$PWD:/workspace" 改为 docker create + docker cp 模式
- 与 staging-e2e 保持一致
2026-07-28 15:56:01 +08:00
xiaoxia 7061d7e672 fix(ci): 修复CI Gate失败时返回exit 0的P0 Bug (main) (#1057)
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 23s
CI/CD Pipeline / Frontend Lint (push) Successful in 56s
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 1m12s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 1m13s
CI/CD Pipeline / Build Production API Image (push) Failing after 23s
CI/CD Pipeline / Build Production Web Image (push) Failing after 32s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 36s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m58s
CI/CD Pipeline / Build Staging API Image (push) Failing after 2m42s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 2m29s
CI/CD Pipeline / Unit Tests (push) Successful in 4m33s
CI/CD Pipeline / Validate - Code Quality (push) Successful in 5m39s
CI/CD Pipeline / Integration Tests (push) Successful in 3m41s
CI/CD Pipeline / Canary Release to Production (push) Failing after 1200h53m51s
CI/CD Pipeline / CI Gate (push) Failing after 1201h14m0s
CI/CD Pipeline / ACR Image Cleanup (push) Failing after 1201h34m37s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1201h34m39s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1201h46m40s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Failing after 1201h47m25s
CI/CD Pipeline / Deploy Production (push) Failing after 1201h53m11s
CI/CD Pipeline / PR Build Worker Image (push) Failing after 1201h57m14s
CI/CD Pipeline / PR Build API Image (push) Failing after 1201h57m16s
CI/CD Pipeline / Check if frontend-only change (push) Failing after 1201h57m17s
CI/CD Pipeline / Staging API Integration Tests (push) Failing after 1202h7m33s
CI/CD Pipeline / PR Build Web Image (push) Failing after 1202h30m12s
2026-07-28 13:30:40 +08:00
xiaoxia 6efdfbe194 fix(ci): pyproject.toml添加原生ruff配置,修复Code Quality全量检查失败 (#1074)
CI/CD Pipeline / Frontend Lint (push) Successful in 42s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 1m13s
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 1m19s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m19s
CI/CD Pipeline / Build Staging API Image (push) Failing after 1m33s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 1m6s
CI/CD Pipeline / Build Production API Image (push) Failing after 50s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 1m10s
CI/CD Pipeline / Build Production Web Image (push) Failing after 16s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 16s
CI/CD Pipeline / Validate - Code Quality (push) Successful in 4m50s
CI/CD Pipeline / Unit Tests (push) Successful in 5m38s
CI/CD Pipeline / Integration Tests (push) Successful in 4m56s
CI/CD Pipeline / CI Gate (push) Failing after 1201h58m9s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1202h3m5s
CI/CD Pipeline / ACR Image Cleanup (push) Failing after 1202h3m7s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1202h3m9s
CI/CD Pipeline / Deploy Production (push) Failing after 1202h3m11s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Failing after 1202h3m34s
CI/CD Pipeline / PR Build Worker Image (push) Failing after 1202h7m32s
CI/CD Pipeline / PR Build Web Image (push) Failing after 1202h7m34s
CI/CD Pipeline / PR Build API Image (push) Failing after 1202h7m36s
CI/CD Pipeline / Check if frontend-only change (push) Failing after 1202h8m1s
CI/CD Pipeline / Canary Release to Production (push) Failing after 1202h36m0s
CI/CD Pipeline / Staging API Integration Tests (push) Failing after 1202h36m5s
2026-07-28 13:19:55 +08:00
xiaoxia 02e3246f5a fix(ci): 格式修复防循环索引 + AI审查fail-open(2个bug修复) (#1045)
CI/CD Pipeline / Frontend Lint (push) Successful in 55s
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 1m3s
CI/CD Pipeline / Build Production API Image (push) Failing after 27s
CI/CD Pipeline / Build Production Web Image (push) Failing after 20s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 1m25s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 1m18s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 23s
CI/CD Pipeline / Validate - Code Quality (push) Failing after 2m7s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 2m4s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 2m3s
CI/CD Pipeline / Build Staging API Image (push) Failing after 2m6s
CI/CD Pipeline / Unit Tests (push) Successful in 3m26s
CI/CD Pipeline / Integration Tests (push) Successful in 2m46s
CI/CD Pipeline / Canary Release to Production (push) Has been cancelled
CI/CD Pipeline / CI Gate (push) Failing after 1205h28m44s
CI/CD Pipeline / ACR Image Cleanup (push) Failing after 1205h29m50s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1205h29m54s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Failing after 1205h32m12s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1205h33m49s
CI/CD Pipeline / Deploy Production (push) Failing after 1205h33m51s
CI/CD Pipeline / PR Build Worker Image (push) Failing after 1205h35m35s
CI/CD Pipeline / PR Build API Image (push) Failing after 1205h35m37s
CI/CD Pipeline / Check if frontend-only change (push) Failing after 1205h35m38s
CI/CD Pipeline / Staging API Integration Tests (push) Failing after 1206h2m47s
CI/CD Pipeline / PR Build Web Image (push) Failing after 1206h8m31s
fix(ci): 修复auto_fix_formatting防循环索引错误 + AI审查fail-open未生效

1. 防循环索引bug:Gitea API返回commits倒序,commits[-1]取到最旧commit,改为commits[0]
2. fail-open bug:LLM调用失败和未捕获异常都是exit 1,改为exit 0不阻塞合并
2026-07-28 09:52:23 +08:00
xiaoxia 5cdafd2559 feat: AI代码审查添加commit status输出和阻塞级问题判定 (#1035)
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m49s
CI/CD Pipeline / Frontend Lint (push) Successful in 1m53s
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 1m58s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 2m2s
CI/CD Pipeline / Build Staging API Image (push) Failing after 2m31s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 2m33s
CI/CD Pipeline / Validate - Code Quality (push) Failing after 2m55s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 34s
CI/CD Pipeline / Build Production API Image (push) Failing after 18s
CI/CD Pipeline / Build Production Web Image (push) Failing after 17s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 18s
CI/CD Pipeline / Unit Tests (push) Successful in 4m38s
CI/CD Pipeline / Integration Tests (push) Successful in 4m21s
CI/CD Pipeline / Canary Release to Production (push) Has been cancelled
CI/CD Pipeline / Check if frontend-only change (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Code Quality (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Type Check (mypy) (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Migration (alembic) (pull_request) Has been cancelled
CI/CD Pipeline / Unit Tests (pull_request) Has been cancelled
CI/CD Pipeline / Integration Tests (pull_request) Has been cancelled
CI/CD Pipeline / Frontend Lint (pull_request) Has been cancelled
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been cancelled
CI/CD Pipeline / PR Build API Image (pull_request) Has been cancelled
CI/CD Pipeline / PR Build Web Image (pull_request) Has been cancelled
CI/CD Pipeline / PR Build Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Staging API Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been cancelled
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been cancelled
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been cancelled
CI/CD Pipeline / Build Production API Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Web Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Production (pull_request) Has been cancelled
CI/CD Pipeline / Production Browser E2E (pull_request) Has been cancelled
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been cancelled
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
CI/CD Pipeline / CI Gate (pull_request) Has been cancelled
AI Code Review / AI Code Review (pull_request) Has been cancelled
Preview Deploy / Deploy Preview Environment (pull_request) Has been cancelled
PR Automation / Auto Approve on CI Green (pull_request) Successful in 1m2s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Failing after 1217h18m57s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1217h30m51s
CI/CD Pipeline / CI Gate (push) Failing after 1217h30m53s
CI/CD Pipeline / ACR Image Cleanup (push) Failing after 1217h31m33s
CI/CD Pipeline / Staging API Integration Tests (push) Failing after 1217h31m33s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1217h31m35s
CI/CD Pipeline / Deploy Production (push) Failing after 1217h32m14s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Failing after 1217h36m41s
CI/CD Pipeline / PR Build Worker Image (push) Failing after 1217h45m48s
CI/CD Pipeline / PR Build API Image (push) Failing after 1217h45m48s
CI/CD Pipeline / Check if frontend-only change (push) Failing after 1217h45m50s
CI/CD Pipeline / PR Build Web Image (push) Failing after 1218h18m40s
- 为AI代码审查添加commit status输出,PR页面可直接看到审查结果
- 添加阻塞级问题判定逻辑,严重问题标记为failure状态
- 优化审查报告格式和输出精度
2026-07-27 21:40:07 +08:00
xiaoxia a6afb344ba feat: 格式自动修复对所有PR开放,添加防循环机制 (#1037)
CI/CD Pipeline / Validate - Type Check (mypy) (push) Failing after 0s
CI/CD Pipeline / Validate - Code Quality (push) Failing after 0s
CI/CD Pipeline / Validate - Migration (alembic) (push) Failing after 0s
CI/CD Pipeline / Frontend Lint (push) Failing after 0s
CI/CD Pipeline / Integration Tests (push) Failing after 0s
CI/CD Pipeline / Unit Tests (push) Failing after 0s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 0s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 28s
CI/CD Pipeline / Build Production Web Image (push) Failing after 30s
CI/CD Pipeline / Build Production API Image (push) Failing after 34s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m32s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 2m12s
CI/CD Pipeline / Build Staging API Image (push) Failing after 2m13s
CI/CD Pipeline / Canary Release to Production (push) Failing after 1218h23m40s
CI/CD Pipeline / ACR Image Cleanup (push) Failing after 1218h23m42s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1218h23m42s
CI/CD Pipeline / Staging API Integration Tests (push) Failing after 1218h23m42s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1218h23m45s
CI/CD Pipeline / Deploy Production (push) Failing after 1218h24m10s
CI/CD Pipeline / CI Gate (push) Failing after 1218h26m23s
CI/CD Pipeline / PR Build Web Image (push) Failing after 1218h26m23s
CI/CD Pipeline / PR Build API Image (push) Failing after 1218h26m23s
CI/CD Pipeline / Check if frontend-only change (push) Failing after 1218h58m10s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Failing after 1218h23m45s
CI/CD Pipeline / PR Build Worker Image (push) Failing after 1218h59m12s
2026-07-27 20:29:56 +08:00
xiaoxia 504e2e71c9 fix(ci): auto_merge.sh改用CI Gate统一门禁
CI/CD Pipeline / Frontend Lint (push) Successful in 30s
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 1m3s
CI/CD Pipeline / Validate - Code Quality (push) Failing after 1m18s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 55s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m8s
CI/CD Pipeline / Build Staging API Image (push) Failing after 1m38s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 1m29s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 16s
CI/CD Pipeline / Build Production API Image (push) Failing after 5s
CI/CD Pipeline / Build Production Web Image (push) Failing after 7s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 6s
CI/CD Pipeline / Unit Tests (push) Successful in 3m27s
CI/CD Pipeline / Integration Tests (push) Successful in 2m16s
CI/CD Pipeline / Production Browser E2E (push) Has been cancelled
CI/CD Pipeline / Canary Release to Production (push) Has been cancelled
CI/CD Pipeline / ACR Image Cleanup (push) Failing after 1219h17m48s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1219h17m50s
CI/CD Pipeline / Deploy Production (push) Failing after 1219h17m51s
CI/CD Pipeline / CI Gate (push) Failing after 1219h28m16s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Failing after 1220h40m14s
CI/CD Pipeline / PR Build Worker Image (push) Failing after 1221h16m27s
CI/CD Pipeline / PR Build API Image (push) Failing after 1221h16m31s
CI/CD Pipeline / Check if frontend-only change (push) Failing after 1221h17m35s
CI/CD Pipeline / Staging API Integration Tests (push) Failing after 1219h50m36s
CI/CD Pipeline / PR Build Web Image (push) Failing after 1221h49m15s
2026-07-27 16:21:19 +08:00
xiaoxia fb2884b03c fix(ci): 添加ci-gate汇总job,解决自动合并405问题
CI/CD Pipeline / Check if frontend-only change (push) Has been cancelled
CI/CD Pipeline / Validate - Code Quality (push) Has been cancelled
CI/CD Pipeline / Validate - Type Check (mypy) (push) Has been cancelled
CI/CD Pipeline / Validate - Migration (alembic) (push) Has been cancelled
CI/CD Pipeline / Unit Tests (push) Has been cancelled
CI/CD Pipeline / Integration Tests (push) Has been cancelled
CI/CD Pipeline / Frontend Lint (push) Has been cancelled
CI/CD Pipeline / Frontend Unit Tests (push) Has been cancelled
CI/CD Pipeline / PR Build API Image (push) Has been cancelled
CI/CD Pipeline / PR Build Web Image (push) Has been cancelled
CI/CD Pipeline / PR Build Worker Image (push) Has been cancelled
CI/CD Pipeline / Build Staging API Image (push) Has been cancelled
CI/CD Pipeline / Build Staging Web Image (push) Has been cancelled
CI/CD Pipeline / Build Staging Worker Image (push) Has been cancelled
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Has been cancelled
CI/CD Pipeline / Staging E2E Tests (push) Has been cancelled
CI/CD Pipeline / Staging API Integration Tests (push) Has been cancelled
CI/CD Pipeline / Build Production API Image (push) Has been cancelled
CI/CD Pipeline / Build Production Web Image (push) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (push) Has been cancelled
CI/CD Pipeline / Deploy Production (push) Has been cancelled
CI/CD Pipeline / Production Browser E2E (push) Has been cancelled
CI/CD Pipeline / ACR Image Cleanup (push) Has been cancelled
CI/CD Pipeline / Canary Release to Production (push) Has been cancelled
CI/CD Pipeline / CI Gate (push) Has been cancelled
2026-07-27 16:17:50 +08:00
xiaoxia 7fab42c3d0 fix(ci): daily-check DooD挂载路径修复 + shell bash修复 (#1024)
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 1m33s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 1m33s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m38s
CI/CD Pipeline / Build Production API Image (push) Failing after 9s
CI/CD Pipeline / Frontend Lint (push) Successful in 1m45s
CI/CD Pipeline / Validate - Code Quality (push) Failing after 1m47s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 16s
CI/CD Pipeline / Build Production Web Image (push) Failing after 11s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 13s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 2m3s
CI/CD Pipeline / Build Staging API Image (push) Failing after 2m19s
CI/CD Pipeline / Unit Tests (push) Successful in 3m57s
CI/CD Pipeline / Canary Release to Production (push) Has been cancelled
CI/CD Pipeline / Integration Tests (push) Successful in 2m2s
CI/CD Pipeline / ACR Image Cleanup (push) Failing after 1227h18m50s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1227h18m52s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1227h18m54s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Failing after 1227h19m0s
CI/CD Pipeline / Deploy Production (push) Failing after 1227h19m10s
CI/CD Pipeline / PR Build Worker Image (push) Failing after 1227h23m30s
CI/CD Pipeline / Check if frontend-only change (push) Failing after 1227h23m34s
CI/CD Pipeline / PR Build Web Image (push) Failing after 1227h23m32s
CI/CD Pipeline / PR Build API Image (push) Failing after 1227h56m18s
CI/CD Pipeline / Staging API Integration Tests (push) Failing after 1227h51m38s
2026-07-27 12:04:41 +08:00
xiaoxia 561548c84c fix(ci): daily-check shell从sh改为bash,修复PIPESTATUS Bad substitution (#1023)
CI/CD Pipeline / Frontend Lint (push) Successful in 44s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 49s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 1m39s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m36s
CI/CD Pipeline / Build Production API Image (push) Failing after 7s
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 1m48s
CI/CD Pipeline / Build Production Web Image (push) Failing after 12s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 16s
CI/CD Pipeline / Validate - Code Quality (push) Failing after 2m10s
CI/CD Pipeline / Build Staging API Image (push) Failing after 2m17s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 2m2s
CI/CD Pipeline / Integration Tests (push) Successful in 2m52s
CI/CD Pipeline / Unit Tests (push) Successful in 5m5s
CI/CD Pipeline / Canary Release to Production (push) Failing after 1227h46m23s
CI/CD Pipeline / Staging API Integration Tests (push) Failing after 1227h46m24s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1227h46m24s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1227h46m43s
CI/CD Pipeline / Deploy Production (push) Failing after 1227h46m45s
CI/CD Pipeline / PR Build API Image (push) Failing after 1227h48m46s
CI/CD Pipeline / PR Build Worker Image (push) Failing after 1227h48m44s
CI/CD Pipeline / Check if frontend-only change (push) Failing after 1227h48m47s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Failing after 1228h19m10s
CI/CD Pipeline / PR Build Web Image (push) Failing after 1228h21m30s
CI/CD Pipeline / ACR Image Cleanup (push) Failing after 1228h19m8s
2026-07-27 11:39:29 +08:00
xiaoxia 77704e7ec6 fix(ci): 同步daily-check和acr-cleanup的docker兼容性修复到main (#1012)
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 48s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 46s
CI/CD Pipeline / Validate - Code Quality (push) Failing after 1m20s
CI/CD Pipeline / Frontend Lint (push) Successful in 36s
CI/CD Pipeline / Build Staging API Image (push) Failing after 2m25s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 2m1s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 2m22s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 25s
CI/CD Pipeline / Build Production API Image (push) Failing after 11s
CI/CD Pipeline / Build Production Web Image (push) Failing after 19s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 16s
CI/CD Pipeline / Unit Tests (push) Successful in 2m23s
CI/CD Pipeline / Integration Tests (push) Successful in 1m56s
CI/CD Pipeline / Canary Release to Production (push) Failing after 1229h25m58s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1229h26m2s
CI/CD Pipeline / ACR Image Cleanup (push) Failing after 1229h44m55s
CI/CD Pipeline / Staging API Integration Tests (push) Failing after 1229h44m56s
CI/CD Pipeline / Deploy Production (push) Failing after 1229h45m0s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1229h44m57s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Failing after 1230h3m27s
CI/CD Pipeline / PR Build Worker Image (push) Failing after 1230h14m30s
CI/CD Pipeline / PR Build API Image (push) Failing after 1230h14m34s
CI/CD Pipeline / Check if frontend-only change (push) Failing after 1230h15m29s
CI/CD Pipeline / PR Build Web Image (push) Failing after 1230h47m16s
cherry-pick #981的核心修复到main分支:
- daily-check.yml: checkout步骤改为curl step_checkout.sh方式
- acr-cleanup.yml: 同上,修复Setup Python秒败问题

相关PR: #981
相关工单: #980
2026-07-27 08:46:05 +08:00
xiaoxia 5ae6c33bf6 sync(ci): 同步缺失的CI监控脚本到main分支
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 53s
CI/CD Pipeline / Frontend Lint (push) Successful in 38s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 54s
CI/CD Pipeline / Validate - Code Quality (push) Failing after 2m0s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m21s
CI/CD Pipeline / Build Staging API Image (push) Failing after 2m8s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 2m25s
CI/CD Pipeline / Build Production Web Image (push) Failing after 25s
CI/CD Pipeline / Build Production API Image (push) Failing after 27s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 37s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 35s
CI/CD Pipeline / Unit Tests (push) Successful in 2m56s
CI/CD Pipeline / Integration Tests (push) Successful in 2m21s
CI/CD Pipeline / Canary Release to Production (push) Has been cancelled
CI/CD Pipeline / Production Browser E2E (push) Failing after 1243h44m35s
CI/CD Pipeline / Deploy Production (push) Failing after 1243h50m24s
CI/CD Pipeline / Staging API Integration Tests (push) Failing after 1243h50m47s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1243h50m49s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Failing after 1244h2m52s
CI/CD Pipeline / PR Build Worker Image (push) Failing after 1244h47m30s
CI/CD Pipeline / PR Build Web Image (push) Failing after 1244h47m32s
CI/CD Pipeline / Check if frontend-only change (push) Failing after 1244h48m5s
CI/CD Pipeline / PR Build API Image (push) Failing after 1245h20m14s
CI/CD Pipeline / ACR Image Cleanup (push) Failing after 1244h23m26s
同步2个CI监控脚本到main分支:scripts/ci_trigger_monitor.py、scripts/ci_code_review.py
2026-07-26 18:13:27 +08:00
xiaoxia db07738178 sync(ci): 同步schedule类workflow修复到main分支 (#933)
CI/CD Pipeline / Frontend Lint (push) Successful in 1m1s
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 1m13s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 1m10s
CI/CD Pipeline / Validate - Code Quality (push) Failing after 2m15s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m11s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 1m48s
CI/CD Pipeline / Build Staging API Image (push) Failing after 2m13s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 24s
CI/CD Pipeline / Build Production Web Image (push) Failing after 18s
CI/CD Pipeline / Build Production API Image (push) Failing after 21s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 18s
CI/CD Pipeline / Unit Tests (push) Successful in 3m31s
CI/CD Pipeline / Integration Tests (push) Successful in 2m28s
CI/CD Pipeline / Canary Release to Production (push) Has been cancelled
CI/CD Pipeline / Production Browser E2E (push) Failing after 1246h45m33s
CI/CD Pipeline / Staging API Integration Tests (push) Failing after 1246h45m35s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1246h45m37s
CI/CD Pipeline / Deploy Production (push) Failing after 1246h54m14s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Failing after 1246h54m24s
CI/CD Pipeline / PR Build Worker Image (push) Failing after 1247h6m32s
CI/CD Pipeline / Check if frontend-only change (push) Failing after 1247h7m21s
CI/CD Pipeline / PR Build Web Image (push) Failing after 1247h6m34s
CI/CD Pipeline / PR Build API Image (push) Failing after 1247h39m15s
CI/CD Pipeline / ACR Image Cleanup (push) Failing after 1247h18m12s
2026-07-26 16:18:24 +08:00
xiaoxia 53c09e7d3c chore(ci): 同步金丝雀发布流水线到main分支(#450)
CI/CD Pipeline / Frontend Lint (push) Successful in 25s
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 58s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 54s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m20s
CI/CD Pipeline / Validate - Code Quality (push) Failing after 2m25s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 16s
CI/CD Pipeline / Build Production API Image (push) Failing after 13s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 11s
CI/CD Pipeline / Build Production Web Image (push) Failing after 14s
CI/CD Pipeline / Build Staging API Image (push) Failing after 2m14s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 1m57s
CI/CD Pipeline / Integration Tests (push) Successful in 1m52s
CI/CD Pipeline / Unit Tests (push) Successful in 3m16s
CI/CD Pipeline / Canary Release to Production (push) Failing after 1272h10m47s
CI/CD Pipeline / Staging API Integration Tests (push) Failing after 1272h10m47s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1272h10m48s
CI/CD Pipeline / Deploy Production (push) Failing after 1272h11m13s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Failing after 1272h10m48s
CI/CD Pipeline / PR Build Worker Image (push) Failing after 1272h13m5s
CI/CD Pipeline / PR Build API Image (push) Failing after 1272h13m9s
CI/CD Pipeline / Check if frontend-only change (push) Failing after 1272h13m54s
CI/CD Pipeline / PR Build Web Image (push) Failing after 1272h45m42s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1272h43m47s
CI/CD Pipeline / ACR Image Cleanup (push) Failing after 1272h43m22s
2026-07-25 15:14:33 +08:00
xiaoxia e602439769 chore(ci): 同步renovate.json5到main分支
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 1m6s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 1m1s
CI/CD Pipeline / Validate - Code Quality (push) Failing after 1m47s
CI/CD Pipeline / Frontend Lint (push) Successful in 42s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m3s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 1m47s
CI/CD Pipeline / Build Staging API Image (push) Failing after 2m11s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 21s
CI/CD Pipeline / Integration Tests (push) Successful in 2m12s
CI/CD Pipeline / Unit Tests (push) Successful in 3m23s
CI/CD Pipeline / ACR Image Cleanup (push) Failing after 1272h37m53s
CI/CD Pipeline / Staging API Integration Tests (push) Failing after 1272h37m55s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1272h37m56s
CI/CD Pipeline / Deploy Production (push) Failing after 1272h38m0s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Failing after 1272h38m1s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 1272h38m4s
CI/CD Pipeline / Build Production Web Image (push) Failing after 1272h38m5s
CI/CD Pipeline / Build Production API Image (push) Failing after 1272h38m5s
CI/CD Pipeline / PR Build Worker Image (push) Failing after 1272h40m18s
CI/CD Pipeline / PR Build Web Image (push) Failing after 1272h40m20s
CI/CD Pipeline / PR Build API Image (push) Failing after 1272h40m22s
CI/CD Pipeline / Check if frontend-only change (push) Failing after 1272h41m48s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1273h10m28s
同步#876的renovate依赖自动更新配置到main,保持main和develop CI配置一致。
2026-07-25 14:44:52 +08:00
xiaoxia a26fda1597 feat(ci): ACR镜像自动清理增强版 - 同步到main(#524)
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 46s
CI/CD Pipeline / Validate - Code Quality (push) Failing after 1m2s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 44s
CI/CD Pipeline / Frontend Lint (push) Successful in 22s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 2m21s
CI/CD Pipeline / Build Staging API Image (push) Failing after 2m22s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 16s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 2m53s
CI/CD Pipeline / Integration Tests (push) Successful in 3m21s
CI/CD Pipeline / Unit Tests (push) Successful in 5m18s
CI/CD Pipeline / ACR Image Cleanup (push) Failing after 1274h54m18s
CI/CD Pipeline / Staging API Integration Tests (push) Failing after 1274h54m20s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1274h54m22s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Failing after 1274h57m48s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1274h59m7s
CI/CD Pipeline / Deploy Production (push) Failing after 1275h0m37s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 1275h0m54s
CI/CD Pipeline / Build Production API Image (push) Failing after 1275h0m55s
CI/CD Pipeline / PR Build Worker Image (push) Failing after 1275h3m18s
CI/CD Pipeline / PR Build API Image (push) Failing after 1275h3m20s
CI/CD Pipeline / Check if frontend-only change (push) Failing after 1275h4m23s
CI/CD Pipeline / PR Build Web Image (push) Failing after 1275h35m53s
CI/CD Pipeline / Build Production Web Image (push) Failing after 1275h33m29s
ACR镜像自动清理增强版同步到main:3种模式+白名单保护+cron/PR关闭/手动触发
2026-07-25 12:21:41 +08:00
xiaoxia 99a8ffa97b chore(ci): 升级migration验证,新增4项检查 (#451) 同步到main
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 51s
CI/CD Pipeline / Frontend Lint (push) Successful in 34s
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 52s
CI/CD Pipeline / Validate - Code Quality (push) Failing after 1m13s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 22s
CI/CD Pipeline / Build Staging API Image (push) Failing after 1m44s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 1m27s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m43s
CI/CD Pipeline / Unit Tests (push) Successful in 3m11s
CI/CD Pipeline / Integration Tests (push) Successful in 2m3s
CI/CD Pipeline / ACR Image Cleanup (push) Failing after 1277h30m58s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1277h30m58s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Failing after 1277h31m0s
CI/CD Pipeline / Deploy Production (push) Failing after 1277h31m43s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 1277h32m24s
CI/CD Pipeline / Build Production API Image (push) Failing after 1277h32m28s
CI/CD Pipeline / PR Build Worker Image (push) Failing after 1277h32m47s
CI/CD Pipeline / PR Build API Image (push) Failing after 1277h32m49s
CI/CD Pipeline / Check if frontend-only change (push) Failing after 1277h33m43s
CI/CD Pipeline / PR Build Web Image (push) Failing after 1278h5m21s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1278h3m34s
CI/CD Pipeline / Build Production Web Image (push) Failing after 1278h4m58s
CI/CD Pipeline / Staging API Integration Tests (push) Failing after 1278h3m30s
同步#857 migration验证升级到main分支

新增:ci_env.sh共享变量、check_migration_naming.py命名检查
升级:validate_migration.sh 5阶段完整检查
2026-07-25 09:54:58 +08:00
xiaoxia f03c9d5453 chore(ci): P2端口收尾同步到main分支 (#798 #799) (#853)
CI/CD Pipeline / Frontend Lint (push) Successful in 42s
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 1m27s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 1m28s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 54s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m44s
CI/CD Pipeline / Validate - Code Quality (push) Failing after 1m48s
CI/CD Pipeline / Build Staging API Image (push) Failing after 2m27s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 2m27s
CI/CD Pipeline / Integration Tests (push) Successful in 2m17s
CI/CD Pipeline / Unit Tests (push) Successful in 4m22s
CI/CD Pipeline / ACR Image Cleanup (push) Failing after 1287h32m1s
CI/CD Pipeline / Staging API Integration Tests (push) Failing after 1287h32m1s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1287h33m1s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1287h32m1s
CI/CD Pipeline / Deploy Production (push) Failing after 1287h33m3s
CI/CD Pipeline / Build Production Web Image (push) Failing after 1287h33m7s
CI/CD Pipeline / Build Production API Image (push) Failing after 1287h33m7s
CI/CD Pipeline / PR Build Web Image (push) Failing after 1287h34m34s
CI/CD Pipeline / PR Build API Image (push) Failing after 1287h34m34s
CI/CD Pipeline / Check if frontend-only change (push) Failing after 1287h34m35s
CI/CD Pipeline / PR Build Worker Image (push) Failing after 1288h7m3s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Failing after 1288h4m33s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 1288h5m35s
chore(ci): P2端口收尾同步到main分支 (#798 #799)

- docs: 新增统一端口分配清单文档 (PORTS.md)
- chore(ci): 端口变量命名统一 (chatops配置)

跳过ci-pipeline.yml的CI_PG_PORT改动:main分支无顶层env块,后续CI配置对齐时再同步。
2026-07-24 23:54:09 +08:00
xiaoxia 8322e2b6e2 fix(ci): staging E2E/API tests shell由sh改为bash
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 36s
CI/CD Pipeline / Frontend Lint (push) Successful in 49s
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 1m39s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 1m44s
CI/CD Pipeline / Validate - Code Quality (push) Failing after 1m52s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m35s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 2m0s
CI/CD Pipeline / Build Staging API Image (push) Failing after 2m31s
CI/CD Pipeline / Integration Tests (push) Successful in 3m0s
CI/CD Pipeline / Unit Tests (push) Successful in 5m20s
CI/CD Pipeline / ACR Image Cleanup (push) Failing after 1295h58m32s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1295h58m32s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Failing after 1295h58m34s
CI/CD Pipeline / Deploy Production (push) Failing after 1296h0m20s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 1296h0m22s
CI/CD Pipeline / Build Production API Image (push) Failing after 1296h0m26s
CI/CD Pipeline / PR Build Worker Image (push) Failing after 1296h1m9s
CI/CD Pipeline / PR Build API Image (push) Failing after 1296h1m13s
CI/CD Pipeline / Check if frontend-only change (push) Failing after 1296h1m15s
CI/CD Pipeline / PR Build Web Image (push) Failing after 1296h33m38s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1296h32m45s
CI/CD Pipeline / Build Production Web Image (push) Failing after 1296h32m51s
CI/CD Pipeline / Staging API Integration Tests (push) Failing after 1296h30m59s
2026-07-24 15:27:33 +08:00
CI Bot d2409e16c1 style: auto-format with black + isort + prettier
CI/CD Pipeline / Validate - Code Quality (push) Failing after 1m23s
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 55s
CI/CD Pipeline / Frontend Lint (push) Successful in 33s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 1m1s
CI/CD Pipeline / Build Staging API Image (push) Failing after 2m38s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 2m17s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 2m3s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 14s
CI/CD Pipeline / Integration Tests (push) Successful in 3m4s
CI/CD Pipeline / Unit Tests (push) Successful in 4m45s
CI/CD Pipeline / ACR Image Cleanup (push) Failing after 1299h42m55s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1299h42m59s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1299h43m9s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Failing after 1299h58m7s
CI/CD Pipeline / Deploy Production (push) Failing after 1299h58m40s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 1300h27m32s
CI/CD Pipeline / PR Build Worker Image (push) Failing after 1300h30m13s
CI/CD Pipeline / Build Production Web Image (push) Failing after 1300h27m33s
CI/CD Pipeline / PR Build Web Image (push) Failing after 1300h30m15s
CI/CD Pipeline / Check if frontend-only change (push) Failing after 1300h32m16s
CI/CD Pipeline / PR Build API Image (push) Failing after 1301h2m43s
CI/CD Pipeline / Staging API Integration Tests (push) Failing after 1300h15m23s
CI/CD Pipeline / Build Production API Image (push) Failing after 1300h59m59s
2026-07-24 02:37:59 +00:00
xiaoxia 6eac0b2cf2 chore(ci): 同步main分支CI配置与scripts/ci脚本 - 与develop对齐
Worker Base Image Build / Build Worker Base Images (worker-base-builder-cache, infra/docker/worker-base-builder.Dockerfile, worker-base-builder, builder) (push) Failing after 1m54s
Worker Base Image Build / Build Worker Base Images (worker-base-runtime-cache, infra/docker/worker-base-runtime.Dockerfile, worker-base-runtime, runtime) (push) Failing after 1m36s
CI/CD Pipeline / Validate - Code Quality (push) Failing after 1m29s
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 1m4s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 1m2s
CI/CD Pipeline / Unit Tests (push) Successful in 3m38s
CI/CD Pipeline / Integration Tests (push) Successful in 2m0s
CI/CD Pipeline / Frontend Lint (push) Successful in 28s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 44s
CI/CD Pipeline / Build Staging API Image (push) Failing after 2m16s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 8m14s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 2m0s
CI/CD Pipeline / ACR Image Cleanup (push) Failing after 1300h3m30s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1300h3m32s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1300h32m43s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Failing after 1300h32m45s
CI/CD Pipeline / Deploy Production (push) Failing after 1300h43m2s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 1300h49m13s
CI/CD Pipeline / Build Production Web Image (push) Failing after 1300h49m14s
CI/CD Pipeline / Build Production API Image (push) Failing after 1300h49m14s
CI/CD Pipeline / PR Build Worker Image (push) Failing after 1300h51m32s
CI/CD Pipeline / PR Build Web Image (push) Failing after 1300h51m34s
CI/CD Pipeline / PR Build API Image (push) Failing after 1300h51m36s
CI/CD Pipeline / Check if frontend-only change (push) Failing after 1300h52m20s
CI/CD Pipeline / Staging API Integration Tests (push) Failing after 1300h35m56s
同步内容:
1. CI流水线配置(ci-pipeline.yml)与develop对齐
2. PR构建脚本docker_build_only.sh增加buildx→docker build回退
3. pre-build步骤worker基础镜像构建增加buildx回退
4. 单元测试脚本全量覆盖率改为仅报告不阻塞
5. diff-cover依赖加入requirements-dev.txt
6. worker base builder/runtime Dockerfile同步
7. test_config_oss.py clear=False→clear=True修复OSS污染
8. Frontend Lint增加prettier依赖
2026-07-24 10:36:29 +08:00
xiaoxia dfb2feef8a fix(ci): main??auto-merge/approve????Tests/test??
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Failing after 3s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1465h21m19s
CI/CD Pipeline / Deploy Production (push) Failing after 1465h21m22s
CI/CD Pipeline / Build Production Runtime Images (push) Failing after 1465h21m23s
CI/CD Pipeline / Validate Code Quality And Tests (push) Has been skipped
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Failing after 1465h53m20s
- auto-merge.yml: main????Tests/test,?3???????
- auto-approve.yml: main??????????Tests/test
- ??#464???Tests/test???????????
2026-07-17 14:04:23 +08:00
auto-approve-bot b3ef7bb041 Merge pull request 'feat(ci): main分支auto-merge从定时改为即时合并' (#464) from ci/main-auto-merge-instant into main
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Failing after 4s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1466h0m8s
CI/CD Pipeline / Deploy Production (push) Failing after 1466h0m12s
CI/CD Pipeline / Build Production Runtime Images (push) Failing after 1466h0m13s
CI/CD Pipeline / Validate Code Quality And Tests (push) Has been skipped
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Failing after 1466h32m10s
2026-07-17 13:27:18 +08:00
xiaoxia a1a272b833 feat(ci): 添加auto-approve到main分支并适配main门禁规则
Tests / lint (pull_request) Successful in 6s
Tests / test (pull_request) Failing after 29s
Auto Approve CI PRs / Auto Approve on CI Green (pull_request) Successful in 2m54s
Auto Merge PRs (main) / Auto Merge on CI Green + Approved (main) (pull_request) Successful in 3m0s
CI/CD Pipeline / Production Browser E2E (pull_request) Failing after 1466h2m30s
CI/CD Pipeline / Build Production Runtime Images (pull_request) Failing after 1466h2m34s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Failing after 1466h2m34s
CI/CD Pipeline / Deploy Production (pull_request) Failing after 1466h2m32s
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 1466h34m32s
2026-07-17 13:24:21 +08:00
xiaoxia 728db0faf8 chore: empty commit to re-trigger CI for auto-merge verification
Tests / lint (pull_request) Successful in 9s
Tests / test (pull_request) Failing after 21s
Auto Merge PRs (main) / Auto Merge on CI Green + Approved (main) (pull_request) Successful in 20m36s
CI/CD Pipeline / Production Browser E2E (pull_request) Failing after 1466h21m15s
CI/CD Pipeline / Build Production Runtime Images (pull_request) Failing after 1466h21m17s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Failing after 1466h21m17s
CI/CD Pipeline / Deploy Production (pull_request) Failing after 1466h21m16s
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 1466h53m15s
2026-07-17 13:06:40 +08:00
xiaoxia 7e5e412f7f feat(ci): main分支auto-merge从定时改为即时合并
Tests / lint (pull_request) Failing after 9s
Tests / test (pull_request) Failing after 28s
Auto Merge PRs (main) / Auto Merge on CI Green + Approved (main) (pull_request) Failing after 30s
CI/CD Pipeline / Production Browser E2E (pull_request) Failing after 1466h49m39s
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 1466h49m40s
CI/CD Pipeline / Build Production Runtime Images (pull_request) Failing after 1466h49m41s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Failing after 1466h49m41s
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Failing after 1467h21m39s
- 触发方式:pull_request事件(CI状态变更时)
- 合并条件:2门禁全绿 + 至少1个APPROVED + 无冲突 + 非草稿
- 安全措施:幂等保护、合并失败留评论、只合main
- 新增check_ci_status.py和check_pr_approval.py辅助脚本
2026-07-17 12:37:24 +08:00
xiaoxia df08161630 fix(ci): 修复auto-merge缺少GITEA_API_TOKEN环境变量
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Failing after 7s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1470h25m27s
CI/CD Pipeline / Deploy Production (push) Failing after 1470h25m35s
CI/CD Pipeline / Build Production Runtime Images (push) Failing after 1470h25m36s
CI/CD Pipeline / Validate Code Quality And Tests (push) Has been skipped
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Failing after 1470h57m29s
1. 给auto-merge.yml的两个merge step添加GITEA_API_TOKEN环境变量,复用REVIEW_GITEA_TOKEN secret
2. 修复tests.yml的checkout步骤GITHUB_TOKEN和Python命令
3. 给main分支ci-cd.yml的单元测试增加Redis服务,修复Celery相关测试失败
2026-07-17 09:02:33 +08:00
xiaoxia 0c9375ff32 ci: remove temporary debug workflow - test-ssh-secret.yml
Fix Flake8 E741 for PR232 / fix-flake8 (push) Failing after 3h4m59s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1577h41m48s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 36s
CI/CD Pipeline / Frontend Lint (push) Failing after 0s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Failing after 1577h10m16s
CI/CD Pipeline / Deploy Production (push) Failing after 1577h10m14s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1577h10m11s
CI/CD Pipeline / Build Production Runtime Images (push) Failing after 1577h10m16s
2026-07-12 10:25:12 +08:00
xiaoxia ef344e9ffc cleanup: remove upgrade test file
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Failing after 12h43m56s
CI/CD Pipeline / Frontend Lint (push) Failing after 12h45m50s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 12h45m50s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1671h17m1s
CI/CD Pipeline / Deploy Production (push) Failing after 1671h20m0s
CI/CD Pipeline / Build Production Runtime Images (push) Failing after 1671h20m2s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1671h51m26s
2026-07-09 00:08:40 +08:00
xiaoxia 0d4904433e test: trigger workflow after gitea upgrade
CI/CD Pipeline / Staging E2E Tests (push) Failing after 12h49m24s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Failing after 12h53m37s
CI/CD Pipeline / Frontend Lint (push) Failing after 12h56m8s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 12h56m8s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1671h29m39s
CI/CD Pipeline / Build Production Runtime Images (push) Failing after 1671h29m43s
CI/CD Pipeline / Deploy Production (push) Failing after 1672h1m9s
2026-07-08 23:58:20 +08:00
CI Test 708662394f Merge develop into main - v0.1.126
Auto Merge PRs / auto-merge (push) Has been cancelled
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 29h56m45s
CI/CD Pipeline / Frontend Lint (push) Failing after 29h56m45s
CI/CD Pipeline / Deploy Production (push) Failing after 1688h31m42s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Failing after 1688h31m44s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1689h3m9s
CI/CD Pipeline / Build Production Runtime Images (push) Failing after 1688h31m44s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1688h31m39s
2026-07-08 06:57:42 +08:00
CI Bot 5bc3440370 ci: staging部署改为Watchtower自动更新,砍掉SSH部署步骤
CI/CD Pipeline / Staging E2E Tests (push) Failing after 38h33m15s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Failing after 38h35m28s
CI/CD Pipeline / Frontend Lint (push) Failing after 38h36m58s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 38h37m47s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1697h11m14s
CI/CD Pipeline / Build Production Runtime Images (push) Failing after 1697h11m25s
CI/CD Pipeline / Deploy Production (push) Failing after 1697h42m51s
2026-07-07 22:16:44 +08:00
CI Test 7dc92191e0 test: watchtower自动更新验证(仅加注释)
CI/CD Pipeline / Staging E2E Tests (push) Has been cancelled
CI/CD Pipeline / Deploy Staging (push) Failing after 38h42m26s
CI/CD Pipeline / Frontend Lint (push) Failing after 38h56m36s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 38h58m34s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1697h18m26s
CI/CD Pipeline / Build Production Runtime Images (push) Failing after 1697h18m31s
CI/CD Pipeline / Deploy Production (push) Failing after 1697h49m56s
2026-07-07 21:55:55 +08:00
CI Test 5028956cea ci: staging部署接入Watchtower自动更新
CI/CD Pipeline / Staging E2E Tests (push) Has been cancelled
CI/CD Pipeline / Deploy Staging (push) Failing after 39h4m5s
CI/CD Pipeline / Frontend Lint (push) Failing after 39h5m1s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 39h5m1s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1697h40m7s
CI/CD Pipeline / Build Production Runtime Images (push) Failing after 1697h40m11s
CI/CD Pipeline / Deploy Production (push) Failing after 1698h11m37s
- staging构建额外打:staging tag并push到Registry
- staging容器加com.centurylinklabs.watchtower.enable=true标签
- 容器用完整Registry路径+:staging tag启动,供Watchtower监控
- 1分钟轮询,自动检测新镜像并重启
2026-07-07 21:49:28 +08:00
CI Test d72450f42d ci: production deploy改为Registry方式,去掉tar/scp/docker load
CI/CD Pipeline / Staging E2E Tests (push) Failing after 39h16m53s
CI/CD Pipeline / Deploy Staging (push) Failing after 39h18m14s
CI/CD Pipeline / Frontend Lint (push) Failing after 39h19m14s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 39h19m14s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1697h54m15s
CI/CD Pipeline / Build Production Runtime Images (push) Failing after 1697h54m20s
CI/CD Pipeline / Deploy Production (push) Failing after 1698h25m45s
- 新增 deploy-production-registry.sh(纯Registry pull + docker run)
- 去掉源码tar打包和scp传输步骤
- nginx-production.conf 添加assets fallback(部署期间缓存用户不404)
- legacy assets自动清理(保留7天)
2026-07-07 21:35:15 +08:00
CI Test 1ed0d5aa75 fix: staging deploy script health-check url missing quotes
CI/CD Pipeline / Staging E2E Tests (push) Failing after 39h40m42s
CI/CD Pipeline / Deploy Staging (push) Failing after 39h42m13s
CI/CD Pipeline / Frontend Lint (push) Failing after 39h44m9s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 39h44m58s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1698h18m7s
CI/CD Pipeline / Build Production Runtime Images (push) Failing after 1698h18m14s
CI/CD Pipeline / Deploy Production (push) Failing after 1698h49m40s
2026-07-07 21:09:28 +08:00
CI Test aef4febd1c ci: fix staging build step, use build_release_images.sh (same as production)
CI/CD Pipeline / Frontend Lint (push) Has been cancelled
CI/CD Pipeline / Deploy Staging (push) Has been cancelled
CI/CD Pipeline / Staging E2E Tests (push) Has been cancelled
CI/CD Pipeline / Build Production Runtime Images (push) Has been cancelled
CI/CD Pipeline / Deploy Production (push) Has been cancelled
CI/CD Pipeline / Production Browser E2E (push) Has been cancelled
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 39h49m43s
2026-07-07 21:04:48 +08:00
CI Test d99ee6fc84 ci: staging deploy改为Registry方式,去掉tar/scp/docker load
CI/CD Pipeline / Deploy Staging (push) Failing after 40h1m36s
CI/CD Pipeline / Frontend Lint (push) Failing after 40h2m31s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 40h2m31s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1698h36m55s
CI/CD Pipeline / Deploy Production (push) Failing after 1698h37m39s
CI/CD Pipeline / Build Production Runtime Images (push) Failing after 1698h37m42s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1699h9m6s
2026-07-07 20:52:01 +08:00
CI Test 9b034764ad Merge develop into main - v0.1.125
Auto Merge PRs / auto-merge (push) Failing after 1m29s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 40h59m5s
CI/CD Pipeline / Frontend Lint (push) Failing after 40h58m8s
CI/CD Pipeline / Deploy Staging (push) Failing after 1699h32m50s
CI/CD Pipeline / Deploy Production (push) Failing after 1699h32m48s
CI/CD Pipeline / Build Production Runtime Images (push) Failing after 1699h32m50s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1699h31m20s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1699h32m48s
2026-07-07 19:54:54 +08:00
CI Test bfb11c3526 fix: prettier formatting for v0.1.125
CI/CD Pipeline / Deploy Staging (push) Failing after 40h57m0s
CI/CD Pipeline / Frontend Lint (push) Failing after 40h59m41s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 40h59m41s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1699h31m0s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1699h31m20s
CI/CD Pipeline / Deploy Production (push) Failing after 1699h32m48s
CI/CD Pipeline / Build Production Runtime Images (push) Failing after 1699h32m51s
2026-07-07 19:54:48 +08:00
灵应 9e37c7b73d fix: 配音库AI配音按钮 + 上传按钮修复 + 多项UI修复
CI/CD Pipeline / Frontend Lint (push) Failing after 41h9m45s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 41h9m45s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1699h44m49s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1699h44m51s
CI/CD Pipeline / Build Production Runtime Images (push) Failing after 1699h44m53s
CI/CD Pipeline / Deploy Staging (push) Failing after 1699h44m53s
CI/CD Pipeline / Deploy Production (push) Failing after 1700h16m19s
- Task #197: 配音库新增AI配音按钮 + TTS合成面板
- Task #198: 上传音频按钮事件修复
- Task #199: 一键生成增加标题库选择
- Task #200: 模板库报错修复
- Task #201: 素材库视频上传失败修复
- Task #202: 素材库去掉配音类型
- Task #203: 模板卡片名称位置调整
- Task #204: 一键生成状态反馈完善
- Task #205: 成片库页面接入真实数据

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 19:44:42 +08:00
灵应 f7a945d417 fix: 视频上传MIME白名单补全 + 成片库N+1查询修复
CI/CD Pipeline / Deploy Staging (push) Failing after 41h47m31s
CI/CD Pipeline / Frontend Lint (push) Failing after 41h49m12s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 41h49m12s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1700h21m58s
CI/CD Pipeline / Build Production Runtime Images (push) Failing after 1700h23m37s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1700h23m32s
CI/CD Pipeline / Deploy Production (push) Failing after 1700h55m3s
- chunked_upload.py ALLOWED_MIME_TYPES 从12种扩展到22种,与 upload.py 保持一致
  新增: video/mpeg, video/x-matroska, video/3gpp, audio/flac, audio/aac,
        audio/x-m4a, audio/webm, image/bmp, image/tiff, image/svg+xml
- generated_video_repository list_by_project/list_by_generation_task
  改用 _to_domain() 替代 self.get(),消除 N+1 查询
- 新增 24 个单元测试覆盖以上修复及成片库 API 可用性确认

Closes #172 #173

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 19:05:14 +08:00
CI Test 8748b43070 Merge develop into main - v0.1.124
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 45h16m36s
CI/CD Pipeline / Frontend Lint (push) Failing after 45h16m32s
CI/CD Pipeline / Deploy Staging (push) Failing after 1703h45m33s
CI/CD Pipeline / Build Production Runtime Images (push) Failing after 45h9m27s
CI/CD Pipeline / Deploy Production (push) Failing after 45h8m45s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1703h44m57s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1703h44m30s
2026-07-07 15:36:36 +08:00
CI Test a28395c318 fix: update schema snapshot and prettier formatting for v0.1.124
CI/CD Pipeline / Deploy Staging (push) Failing after 45h15m56s
CI/CD Pipeline / Frontend Lint (push) Failing after 45h18m5s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 45h18m5s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1703h44m57s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1703h44m57s
CI/CD Pipeline / Deploy Production (push) Failing after 1703h45m33s
CI/CD Pipeline / Build Production Runtime Images (push) Failing after 1703h51m44s
2026-07-07 15:36:26 +08:00
灵应 9f0c064f2a feat: 任务4 视频查重 — 历史+批次双重去重
CI/CD Pipeline / Frontend Lint (push) Failing after 45h27m46s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 45h27m46s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1704h3m13s
CI/CD Pipeline / Build Production Runtime Images (push) Failing after 1704h3m17s
CI/CD Pipeline / Deploy Production (push) Failing after 1704h3m15s
CI/CD Pipeline / Deploy Staging (push) Failing after 1704h3m17s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1704h34m43s
- Alembic 033: generation_tasks 加 batch_id 列+索引
- Route: count>1 时生成共享 batch_id
- Repository: list_by_batch() 批次内查询
- dedup.py: check_batch_duplicate() 批次内查重
- Worker: 生成视频后创建 GeneratedVideo 记录 + 双重查重
- 单元测试: 6 个批次查重测试用例
2026-07-07 15:26:41 +08:00
灵应 1ea8fd3989 feat: P2 素材库自动匹配 - 支持 all/random/smart 三种素材选取模式
CI/CD Pipeline / Frontend Lint (push) Failing after 45h41m20s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 45h41m20s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1704h16m48s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1704h16m50s
CI/CD Pipeline / Build Production Runtime Images (push) Failing after 1704h16m52s
CI/CD Pipeline / Deploy Staging (push) Failing after 1704h16m52s
CI/CD Pipeline / Deploy Production (push) Failing after 1704h48m18s
- Schema: 新增 asset_select_mode + asset_select_count 字段
- Domain: GenerationTask 新增 asset_select_mode 字段
- Application: Command/UseCase 透传 asset_select_mode
- Route: _select_assets_from_library() 辅助函数 + 创建任务集成
- SQLAlchemy: Model/Repository 映射 asset_select_mode
- Alembic: 032 号迁移
- Worker: _download_library_assets() 支持 asset_ids 过滤
- 单测: 15 个测试覆盖三种模式 + 边界情况
2026-07-07 15:13:05 +08:00
灵应 4fee87c5e8 feat: 素材重复上传检测 + 批量生成视频
CI/CD Pipeline / Frontend Lint (push) Failing after 45h56m35s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 45h56m48s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1704h31m50s
CI/CD Pipeline / Build Production Runtime Images (push) Failing after 1704h32m6s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1704h31m51s
CI/CD Pipeline / Deploy Staging (push) Failing after 1704h32m6s
CI/CD Pipeline / Deploy Production (push) Failing after 1705h3m19s
任务1: 素材重复上传检测
- 上传接口支持 file_hash 参数,通过 MD5+素材库ID 去重
- 命中去重直接返回已有 asset_id,不重复存 OSS
- file_hash 透传: API → IngestJob → Asset 全链路
- 三条上传路径(表单/直传/分片)均支持去重
- Alembic 031: assets + ingest_jobs 加 file_hash 列+索引
- 6 个单元测试覆盖去重命中/未命中/空hash/透传

任务3: 批量生成视频
- POST /generations 支持 count 参数,一次创建多条生成任务
- 每条任务独立状态跟踪,响应返回 task_ids 列表

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 14:57:37 +08:00
灵应 a74d25e414 feat: 完成7个前端修复任务 (#190-#196)
CI/CD Pipeline / Frontend Lint (push) Failing after 45h59m59s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 45h59m59s
CI/CD Pipeline / Production Browser E2E (push) Failing after 1704h31m0s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1704h31m2s
CI/CD Pipeline / Build Production Runtime Images (push) Failing after 1704h31m7s
CI/CD Pipeline / Deploy Staging (push) Failing after 1704h31m7s
CI/CD Pipeline / Deploy Production (push) Failing after 1705h2m30s
- #190: 素材库视频加缩略图展示
- #191: 视频上传进度提示(圆形动画+百分比)
- #192: 一键生成模板名称显示用户自定义名称
- #193: 确认生成无反应修复(增强错误处理)
- #194: 素材选择双模式(手动选择/自动匹配)
- #195: 多条视频生成数量输入
- #196: 成片库404修复
2026-07-07 14:54:28 +08:00
132 changed files with 20656 additions and 1681 deletions
+172
View File
@@ -0,0 +1,172 @@
name: ACR Cleanup
on:
schedule:
- cron: '0 19 * * *' # UTC 19:00 = 北京时间凌晨3:00
workflow_dispatch:
inputs:
pr_sha:
description: "PR commit SHA(仅清理指定PR镜像,留空则全量清理)"
required: false
default: ""
dry_run:
description: "预览模式(dry-run),不实际删除"
required: false
default: "true"
pull_request_target:
types: [closed]
branches: [develop, main]
concurrency:
group: acr-cleanup-${{ gitea.ref }}
cancel-in-progress: false
jobs:
cleanup:
name: ACR Image Cleanup
runs-on: ci-l2
timeout-minutes: 20
permissions:
contents: read
env:
ACR_REGISTRY: xiaoxia-registry.cn-hangzhou.cr.aliyuncs.com
ACR_NAMESPACE: xiaoxiakeji
ACR_SERVICE: registry.aliyuncs.com:cn-hangzhou:china:cri-fvec8o9q4mmxrkaa
GITEA_URL: https://git.xiaoxiajianji.com
GITEA_REPO: xiaoxia/xiaoxia-saas
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sH "Authorization: token $GITHUB_TOKEN" "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" | bash
# ====== Cron模式:获取staging运行中镜像作为白名单 ======
- name: Get staging running images (whitelist)
id: protected_images
if: gitea.event_name != 'pull_request_target' && !gitea.event.inputs.pr_sha
env:
STAGING_SSH_HOST: ${{ secrets.STAGING_SSH_HOST }}
STAGING_SSH_PORT: ${{ secrets.STAGING_SSH_PORT }}
STAGING_SSH_USER: ${{ secrets.STAGING_SSH_USER }}
STAGING_SSH_KEY: ${{ secrets.STAGING_SSH_KEY }}
run: |
set +e
echo "获取staging服务器运行中镜像作为白名单..."
staging_host="${STAGING_SSH_HOST:-47.98.113.167}"
staging_port="${STAGING_SSH_PORT:-22222}"
staging_user="${STAGING_SSH_USER:-root}"
key_path=~/.ssh/id_rsa
if [ -n "${STAGING_SSH_KEY:-}" ]; then
printf '%s\n' "$STAGING_SSH_KEY" > "$key_path"
chmod 600 "$key_path"
echo "Using key from STAGING_SSH_KEY secret"
else
echo "⚠️ STAGING_SSH_KEY not set, skipping whitelist"
echo "protected_tags=" >> $GITHUB_OUTPUT
exit 0
fi
ssh-keyscan -p "$staging_port" -H "$staging_host" >> ~/.ssh/known_hosts 2>/dev/null
# 获取所有运行容器的镜像,提取tag部分
IMAGES=$(ssh -p "$staging_port" -i "$key_path" -o StrictHostKeyChecking=no \
"$staging_user@$staging_host" "docker ps --format '{{.Image}}' 2>/dev/null" 2>/dev/null | grep -v "^$" | sort -u)
PROTECTED_TAGS=""
if [ -n "$IMAGES" ]; then
while IFS= read -r img; do
# 从完整镜像名中提取tag(最后一个冒号后)
tag=$(echo "$img" | rev | cut -d: -f1 | rev)
if [ -n "$tag" ] && [ "$tag" != "latest" ] && [ ${#tag} -gt 5 ]; then
if [ -z "$PROTECTED_TAGS" ]; then
PROTECTED_TAGS="$tag"
else
PROTECTED_TAGS="$PROTECTED_TAGS,$tag"
fi
fi
done <<< "$IMAGES"
fi
echo "staging运行中镜像tag: ${PROTECTED_TAGS:-(无)}"
echo "protected_tags=$PROTECTED_TAGS" >> $GITHUB_OUTPUT
# ====== Docker登录 ======
- name: Docker login to ACR
env:
ACR_USERNAME: ${{ secrets.ACR_USERNAME }}
ACR_PASSWORD: ${{ secrets.ACR_PASSWORD }}
run: |
printf '%s' "$ACR_PASSWORD" | docker login "$ACR_REGISTRY" -u "$ACR_USERNAME" --password-stdin
# ====== 模式1PR关闭时清理 ======
- name: Cleanup PR images (PR closed)
if: gitea.event_name == 'pull_request_target'
env:
ACR_USERNAME: ${{ secrets.ACR_USERNAME }}
ACR_PASSWORD: ${{ secrets.ACR_PASSWORD }}
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
PR_SHA: ${{ gitea.event.pull_request.head.sha }}
PR_NUMBER: ${{ gitea.event.pull_request.number }}
run: |
echo "============================================"
echo " PR #$PR_NUMBER 已关闭,清理对应镜像"
echo " Head SHA: ${PR_SHA::12}"
echo "============================================"
echo ""
python3 scripts/ci/acr_cleanup.py \
--pr-sha "$PR_SHA" \
--execute
# ====== 模式2Cron全量清理 ======
- name: Full cleanup (cron / manual)
if: gitea.event_name != 'pull_request_target' && !gitea.event.inputs.pr_sha
env:
ACR_USERNAME: ${{ secrets.ACR_USERNAME }}
ACR_PASSWORD: ${{ secrets.ACR_PASSWORD }}
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
PROTECTED_TAGS: ${{ steps.protected_images.outputs.protected_tags }}
DRY_RUN_INPUT: ${{ gitea.event.inputs.dry_run }}
run: |
echo "============================================"
echo " ACR 全量清理(${{ gitea.event_name }}"
echo "============================================"
echo ""
# 决定是否dry-run
DRY_RUN_FLAG=""
if [ "$DRY_RUN_INPUT" = "true" ]; then
DRY_RUN_FLAG="--dry-run"
echo "模式: 预览模式 (dry-run)"
else
echo "模式: 执行模式"
fi
echo ""
python3 scripts/ci/acr_cleanup.py \
--keep 20 \
--protected-tags "$PROTECTED_TAGS" \
$DRY_RUN_FLAG
# ====== 模式3:手动指定PR SHA清理 ======
- name: Cleanup specific PR image (manual)
if: gitea.event_name == 'workflow_dispatch' && gitea.event.inputs.pr_sha
env:
ACR_USERNAME: ${{ secrets.ACR_USERNAME }}
ACR_PASSWORD: ${{ secrets.ACR_PASSWORD }}
PR_SHA: ${{ gitea.event.inputs.pr_sha }}
DRY_RUN_INPUT: ${{ gitea.event.inputs.dry_run }}
run: |
echo "手动清理PR镜像: ${PR_SHA::12}"
echo ""
DRY_RUN_FLAG=""
if [ "$DRY_RUN_INPUT" = "true" ]; then
DRY_RUN_FLAG="--dry-run"
fi
python3 scripts/ci/acr_cleanup.py \
--pr-sha "$PR_SHA" \
$DRY_RUN_FLAG
-21
View File
@@ -1,21 +0,0 @@
name: Auto Merge PRs
on:
schedule:
- cron: '0 */6 * * *'
workflow_dispatch:
jobs:
auto-merge:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Auto merge develop PRs
run: |
bash scripts/auto_merge_prs.sh develop
- name: Auto merge main PRs (release only)
run: |
bash scripts/auto_merge_prs.sh main
-808
View File
@@ -1,808 +0,0 @@
name: CI/CD Pipeline
on:
push:
branches:
- main
- develop
- 'feature/**'
- 'bugfix/**'
- 'hotfix/**'
- 'release/**'
tags:
- 'v*'
pull_request:
branches:
- main
- develop
permissions:
contents: read
jobs:
validate:
name: Validate Code Quality And Tests
runs-on: ubuntu-22.04
env:
DATABASE_URL: postgresql+psycopg://postgres:postgres@127.0.0.1:5433/xiaoxia_saas
USE_IN_MEMORY_DB: "false"
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
set -eu
python3 - <<'PY'
import io, os, tarfile, time, urllib.request, urllib.error
url = f"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz"
request = urllib.request.Request(url, headers={"Authorization": f"token {os.environ['GITHUB_TOKEN']}"})
last_err = None
for attempt in range(5):
try:
with urllib.request.urlopen(request, timeout=120) as response:
archive = response.read()
break
except urllib.error.HTTPError as e:
last_err = e
if e.code >= 500 and attempt < 4:
wait = 2 ** attempt
print(f"Checkout HTTP {e.code}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
except Exception as e:
last_err = e
if attempt < 4:
wait = 2 ** attempt
print(f"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
else:
raise last_err
with tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:
root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'
for member in tar.getmembers():
name = member.name
if name == root_prefix[:-1]:
continue
if name.startswith(root_prefix):
member.name = name[len(root_prefix):]
if member.name:
tar.extract(member, '.')
PY
- name: Verify CI environment
shell: sh
run: |
set -eu
python --version
python3 -m pip --version
echo "CI environment is ready"
- name: Install dependencies
shell: sh
run: |
set -eu
python3 -m pip install -q -r requirements-base.txt
python3 -m pip install -q -r requirements.txt
python3 -m pip install -q -r requirements-dev.txt
python3 -m black --version
python3 -m isort --version-number
python3 -m flake8 --version
bandit --version
pytest --version
- name: Run code quality checks
shell: sh
run: |
set -eu
python3 -m compileall -q alembic apps packages tests scripts
python3 -m black --check --fast alembic apps packages tests scripts
python3 -m isort --check-only alembic apps packages tests scripts
python3 -m flake8 apps packages tests --count --statistics
- name: Run security scan
shell: sh
run: |
set -eu
bandit -r apps packages -q -ll
- name: Validate release scripts syntax
shell: sh
run: |
set -eu
bash -n scripts/backup_postgres.sh
bash -n scripts/restore_postgres_plan.sh
bash -n scripts/init_production_env.sh
- name: Validate Alembic migrations
shell: sh
run: |
set -eu
python3 -m alembic upgrade head --sql > /tmp/alembic-upgrade.sql
test -s /tmp/alembic-upgrade.sql
grep -q "Running upgrade" /tmp/alembic-upgrade.sql
python3 scripts/check_schema_metadata.py
- name: Run unit tests
shell: sh
env:
USE_IN_MEMORY_DB: "true"
run: |
set -eu
PYTHONPATH="$PWD/apps/api:$PWD" python3 -m pytest tests/unit -q
- name: Start PostgreSQL for integration tests
shell: sh
run: |
set -eu
docker rm -f ci-pg-validate 2>/dev/null || true
docker run -d --name ci-pg-validate \
-e POSTGRES_USER=postgres \
-e POSTGRES_PASSWORD=postgres \
-e POSTGRES_DB=xiaoxia_saas \
-p 5433:5432 \
--health-cmd "pg_isready -U postgres" \
--health-interval 5s \
--health-timeout 5s \
--health-retries 12 \
postgres:16
for i in $(seq 1 30); do
if docker inspect --format='{{.State.Health.Status}}' ci-pg-validate 2>/dev/null | grep -q healthy; then
echo "PostgreSQL is ready"
break
fi
echo "Waiting for PostgreSQL... ($i/30)"
sleep 2
done
docker inspect --format='{{.State.Health.Status}}' ci-pg-validate | grep -q healthy
- name: Apply migrations for integration tests
shell: sh
run: |
set -eu
PYTHONPATH="$PWD/apps/api:$PWD" python3 -m alembic upgrade head
- name: Run integration tests
shell: sh
run: |
set -eu
PYTHONPATH="$PWD/apps/api:$PWD" python3 -m pytest tests/integration -q --timeout=60 -x
- name: Cleanup PostgreSQL
if: always()
shell: sh
run: |
docker rm -f ci-pg-validate 2>/dev/null || true
echo "PostgreSQL container cleaned up"
- name: Build summary
if: github.ref == 'refs/heads/develop' || github.ref == 'refs/heads/main'
shell: sh
run: |
set -eu
echo "Build completed successfully!"
echo "Branch: ${GITHUB_REF_NAME}"
echo "Commit: ${GITHUB_SHA}"
frontend-lint:
name: Frontend Lint
runs-on: ubuntu-22.04
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
set -eu
archive_url="${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/archive/${GITHUB_SHA}.tar.gz"
for i in 1 2 3 4 5; do
if wget --header="Authorization: token ${GITHUB_TOKEN}" -O /tmp/repo.tar.gz "$archive_url" 2>&1; then
break
fi
if [ "$i" -lt 5 ]; then
wait=$((2 ** i))
echo "Checkout failed (attempt $i/5), retrying in ${wait}s..."
sleep "$wait"
else
echo "Checkout failed after 5 attempts"
exit 1
fi
done
tar -xzf /tmp/repo.tar.gz --strip-components=1 -C .
rm -f /tmp/repo.tar.gz
- name: Install dependencies
shell: sh
run: |
set -eu
docker run --rm \
-v "$PWD:/workspace" \
-w /workspace/apps/web \
docker.m.daocloud.io/library/node:20 \
sh -lc 'npm ci'
- name: Run ESLint
shell: sh
run: |
set -eu
docker run --rm \
-v "$PWD:/workspace" \
-w /workspace/apps/web \
docker.m.daocloud.io/library/node:20 \
sh -lc 'npx eslint src --ext .ts,.tsx --max-warnings 50'
- name: Run TypeScript type check
shell: sh
run: |
set -eu
docker run --rm \
-v "$PWD:/workspace" \
-w /workspace/apps/web \
docker.m.daocloud.io/library/node:20 \
sh -lc 'npx tsc --noEmit'
- name: Run Prettier check
shell: sh
run: |
set -eu
docker run --rm \
-v "$PWD:/workspace" \
-w /workspace/apps/web \
docker.m.daocloud.io/library/node:20 \
sh -lc 'npx prettier --check "src/**/*.{ts,tsx,md}"'
- name: Run Vitest tests
shell: sh
run: |
set -eu
docker run --rm \
-v "$PWD:/workspace" \
-w /workspace/apps/web \
docker.m.daocloud.io/library/node:20 \
sh -lc 'npx vitest run src/test'
deploy-staging:
name: Deploy Staging
runs-on: saas
needs: [validate, frontend-lint]
if: github.ref_name == 'main' || github.ref_name == 'develop' || startsWith(github.ref_name, 'feature/')
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
set -eu
python3 - <<'PY'
import io, os, tarfile, time, urllib.request, urllib.error
url = f"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz"
request = urllib.request.Request(url, headers={"Authorization": f"token {os.environ['GITHUB_TOKEN']}"})
# Retry up to 5 times with backoff for transient 5xx errors
last_err = None
for attempt in range(5):
try:
with urllib.request.urlopen(request, timeout=120) as response:
archive = response.read()
break
except urllib.error.HTTPError as e:
last_err = e
if e.code >= 500 and attempt < 4:
wait = 2 ** attempt
print(f"Checkout HTTP {e.code}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
except Exception as e:
last_err = e
if attempt < 4:
wait = 2 ** attempt
print(f"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
else:
raise last_err
with tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:
root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'
for member in tar.getmembers():
name = member.name
if name == root_prefix[:-1]:
continue
if name.startswith(root_prefix):
member.name = name[len(root_prefix):]
if member.name:
tar.extract(member, '.')
PY
- name: Install SSH client
shell: sh
run: |
set -eu
apt-get update -qq && apt-get install -y -qq openssh-client >/dev/null 2>&1
echo "openssh-client installed"
- name: Build staging web artifact
shell: sh
run: |
set -eu
docker run --rm \
-v "$PWD:/workspace" \
-w /workspace/apps/web \
docker.m.daocloud.io/library/node:20 \
sh -lc 'npm ci && npm run build'
docker build --pull=false \
-f infra/docker/web-artifact.Dockerfile \
--build-arg NGINX_CONF=infra/docker/nginx-staging.conf \
-t "xiaoxia-saas-web:staging-${GITHUB_SHA}" \
.
test -f apps/web/dist/index.html
- name: Build and push staging API/Worker images
shell: bash
env:
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
set -eu
REGISTRY="172.30.18.198:5000"
CACHE_REGISTRY="git.xiaoxiajianji.com/xiaoxia/xiaoxia-saas"
CACHE_BRANCH="${GITHUB_REF_NAME//\//-}"
# 登录 Gitea Registry(用于构建缓存,PAT 带 packages 权限)
printf '%s' "${REGISTRY_TOKEN}" | docker login git.xiaoxiajianji.com -u xiaoxia --password-stdin
# 登录内网 Registry(运行时镜像推送目标)
printf '%s' "Xiaoxia2026" | docker login 172.30.18.198:5000 -u admin --password-stdin 2>/dev/null || true
# 用默认 docker driver builder(共享 docker daemon 凭证,解决 buildx 认证问题)
docker buildx use default 2>/dev/null || {
echo "Warning: default builder not available, buildx cache may not work"
}
# 验证 builder 状态
docker buildx ls
# 构建 API(带分布式缓存,缓存存 Gitea Registry,镜像推内网 Registry
docker buildx build \
--cache-from "type=registry,ref=${CACHE_REGISTRY}/api-cache:${CACHE_BRANCH},ignore-error=true" \
--cache-to "type=registry,ref=${CACHE_REGISTRY}/api-cache:${CACHE_BRANCH},mode=max" \
-f infra/docker/api.Dockerfile \
-t "${REGISTRY}/xiaoxia-saas-api:dev" \
--push \
.
# 构建 Worker(带分布式缓存)
docker buildx build \
--cache-from "type=registry,ref=${CACHE_REGISTRY}/worker-cache:${CACHE_BRANCH},ignore-error=true" \
--cache-to "type=registry,ref=${CACHE_REGISTRY}/worker-cache:${CACHE_BRANCH},mode=max" \
-f infra/docker/worker.Dockerfile \
-t "${REGISTRY}/xiaoxia-saas-worker:dev" \
--push \
.
- name: Package staging release artifact
shell: sh
run: |
set -eu
rm -rf dist/staging-artifacts
mkdir -p dist/staging-artifacts
tar --exclude=.git --exclude=apps/web/node_modules --exclude=./dist \
-czf dist/staging-artifacts/xiaoxia-staging-${GITHUB_SHA}.tar.gz .
docker save -o "dist/staging-artifacts/xiaoxia-web-staging-${GITHUB_SHA}.tar" "xiaoxia-saas-web:staging-${GITHUB_SHA}"
- name: Upload staging artifact to business host
shell: sh
env:
STAGING_SSH_HOST: ${{ secrets.STAGING_SSH_HOST }}
STAGING_SSH_USER: ${{ secrets.STAGING_SSH_USER }}
STAGING_SSH_KEY: ${{ secrets.STAGING_SSH_KEY }}
run: |
set -eu
staging_host="${STAGING_SSH_HOST:-47.98.113.167}"
staging_user="${STAGING_SSH_USER:-root}"
mkdir -p ~/.ssh
# Prefer host key if available (more reliable), fallback to secrets
if [ -f /root/.ssh/xiaoxia_runtime_builder ]; then
key_path="/root/.ssh/xiaoxia_runtime_builder"
echo "Using host SSH key: $key_path"
elif [ -n "${STAGING_SSH_KEY:-}" ]; then
key_path="$HOME/.ssh/id_ed25519"
printf '%s\n' "$STAGING_SSH_KEY" > "$key_path"
chmod 600 "$key_path"
echo "Using secret SSH key: $key_path"
else
echo "ERROR: No SSH key available"
exit 1
fi
ssh-keyscan -H "$staging_host" >> ~/.ssh/known_hosts
ssh -i "$key_path" "$staging_user@$staging_host" "mkdir -p /var/lib/xiaoxia-saas-staging/artifacts"
scp -i "$key_path" "dist/staging-artifacts/xiaoxia-staging-${GITHUB_SHA}.tar.gz" \
"$staging_user@$staging_host:/var/lib/xiaoxia-saas-staging/artifacts/xiaoxia-staging-${GITHUB_SHA}.tar.gz"
scp -i "$key_path" "dist/staging-artifacts/xiaoxia-web-staging-${GITHUB_SHA}.tar" \
"$staging_user@$staging_host:/var/lib/xiaoxia-saas-staging/artifacts/xiaoxia-web-staging-${GITHUB_SHA}.tar"
- name: Deploy staging stack on business host
shell: sh
env:
STAGING_SSH_HOST: ${{ secrets.STAGING_SSH_HOST }}
STAGING_SSH_USER: ${{ secrets.STAGING_SSH_USER }}
STAGING_SSH_KEY: ${{ secrets.STAGING_SSH_KEY }}
run: |
set -eu
staging_host="${STAGING_SSH_HOST:-47.98.113.167}"
staging_user="${STAGING_SSH_USER:-root}"
if [ -f /root/.ssh/xiaoxia_runtime_builder ]; then
key_path="/root/.ssh/xiaoxia_runtime_builder"
elif [ -n "${STAGING_SSH_KEY:-}" ]; then
key_path="$HOME/.ssh/id_ed25519"
else
echo "ERROR: No SSH key available"
exit 1
fi
echo 'c2V0IC1ldQphcnRpZmFjdD0iL3Zhci9saWIveGlhb3hpYS1zYWFzLXN0YWdpbmcvYXJ0aWZhY3RzL3hpYW94aWEtc3RhZ2luZy0ke0dJVEhVQl9TSEF9LnRhci5neiIKaW1hZ2VfdGFyPSIvdmFyL2xpYi94aWFveGlhLXNhYXMtc3RhZ2luZy9hcnRpZmFjdHMveGlhb3hpYS13ZWItc3RhZ2luZy0ke0dJVEhVQl9TSEF9LnRhciIKdGVzdCAtZiAiJGFydGlmYWN0Igp0ZXN0IC1mICIkaW1hZ2VfdGFyIgp0ZXN0IC1mIC92YXIvbGliL3hpYW94aWEtc2Fhcy1zdGFnaW5nLy5lbnYKZG9ja2VyIGxvYWQgLWkgIiRpbWFnZV90YXIiCnJtIC1yZiAvdmFyL2xpYi94aWFveGlhLXNhYXMtc3RhZ2luZy9yZXBvCm1rZGlyIC1wIC92YXIvbGliL3hpYW94aWEtc2Fhcy1zdGFnaW5nL3JlcG8KdGFyIC14emYgIiRhcnRpZmFjdCIgLUMgL3Zhci9saWIveGlhb3hpYS1zYWFzLXN0YWdpbmcvcmVwbwp0ZXN0IC1mIC92YXIvbGliL3hpYW94aWEtc2Fhcy1zdGFnaW5nL3JlcG8vYXBwcy93ZWIvZGlzdC9pbmRleC5odG1sCmNwIC92YXIvbGliL3hpYW94aWEtc2Fhcy1zdGFnaW5nLy5lbnYgL3Zhci9saWIveGlhb3hpYS1zYWFzLXN0YWdpbmcvcmVwby8uZW52CmNobW9kICt4IC92YXIvbGliL3hpYW94aWEtc2Fhcy1zdGFnaW5nL3JlcG8vaW5mcmEvZG9ja2VyL2RlcGxveS1zdGFnaW5nLnNoCiMgRW5zdXJlIGlzb2xhdGVkIHN0YWdpbmcgbmV0d29yayBleGlzdHMgYmVmb3JlIGRlcGxveQpkb2NrZXIgbmV0d29yayBjcmVhdGUgeGlhb3hpYS1uZXQtc3RhZ2luZyAyPi9kZXYvbnVsbCB8fCB0cnVlClJFR0lTVFJZPSIxNzIuMzAuMTguMTk4OjUwMDAiIEFQSV9JTUFHRT0iJHtSRUdJU1RSWX0veGlhb3hpYS1zYWFzLWFwaTpkZXYiIFdPUktFUl9JTUFHRT0iJHtSRUdJU1RSWX0veGlhb3hpYS1zYWFzLXdvcmtlcjpkZXYiIFdFQl9JTUFHRT0ieGlhb3hpYS1zYWFzLXdlYjpzdGFnaW5nLSR7R0lUSFVCX1NIQX0iIEhPU1RfUFJFRklYPSBFTlY9c3RhZ2luZyBXRUJfUE9SVD0zMDAxIFJFQlVJTERfQkFDS0VORD0wIEJVSUxEX1dFQj0wIFJVTl9NSUdSQVRJT05TPTEgUkVHSVNUUllfUEFTUz0iWGlhb3hpYVJlZ2lzdHJ5MjAyNiIgL3Zhci9saWIveGlhb3hpYS1zYWFzLXN0YWdpbmcvcmVwby9pbmZyYS9kb2NrZXIvZGVwbG95LXN0YWdpbmcuc2gKaT0wCndoaWxlIFsgIiRpIiAtbHQgMzAgXTsgZG8KICBpZiB3Z2V0IC1xTy0gaHR0cDovLzEyNy4wLjAuMTo4MDAwL2hlYWx0aDsgdGhlbgogICAgZXhpdCAwCiAgZmkKICBpPSQoKGkgKyAxKSkKICBzbGVlcCAyCmRvbmUKZXhpdCAxCg==' | base64 -d | ssh -i "$key_path" "$staging_user@$staging_host" "GITHUB_SHA='${GITHUB_SHA}' sh"
- name: Post-deploy smoke test
shell: sh
env:
STAGING_SSH_HOST: ${{ secrets.STAGING_SSH_HOST }}
STAGING_SSH_USER: ${{ secrets.STAGING_SSH_USER }}
STAGING_SSH_KEY: ${{ secrets.STAGING_SSH_KEY }}
run: |
set -eu
staging_host="${STAGING_SSH_HOST:-47.98.113.167}"
staging_user="${STAGING_SSH_USER:-root}"
if [ -f /root/.ssh/xiaoxia_runtime_builder ]; then
key_path="/root/.ssh/xiaoxia_runtime_builder"
elif [ -n "${STAGING_SSH_KEY:-}" ]; then
key_path="$HOME/.ssh/id_ed25519"
else
echo "ERROR: No SSH key available"
exit 1
fi
echo "Running post-deploy smoke tests on staging..."
# Wait for service to fully start
sleep 5
# Run smoke tests via SSH on the business host
ssh -i "$key_path" "$staging_user@$staging_host" '
echo "--- Smoke test 1: Health check ---"
HEALTH=$(curl -sf --max-time 10 http://127.0.0.1:8000/health) || {
echo "FAIL: health endpoint unreachable"
exit 1
}
echo "Health OK: $HEALTH"
echo "--- Smoke test 2: Login API (expect 401) ---"
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" --max-time 10 -X POST \
http://127.0.0.1:8000/api/v1/auth/login \
-H "Content-Type: application/json" \
-d "{\"email\":\"smoke@test.com\",\"password\":\"wrong\"}")
if [ "$HTTP_CODE" != "401" ] && [ "$HTTP_CODE" != "422" ]; then
echo "FAIL: login returned HTTP $HTTP_CODE (expected 401 or 422)"
exit 1
fi
echo "Login API OK: HTTP $HTTP_CODE"
echo "--- Smoke test 3: API docs endpoint ---"
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" --max-time 10 http://127.0.0.1:8000/docs)
if [ "$HTTP_CODE" != "200" ]; then
echo "FAIL: /docs returned HTTP $HTTP_CODE (expected 200)"
exit 1
fi
echo "Docs endpoint OK: HTTP $HTTP_CODE"
echo "--- Smoke test 4: Network isolation verification ---"
# Verify staging containers are on the staging network
STAGING_NET=$(docker inspect xiaoxia-api-staging --format="{{json .NetworkSettings.Networks}}" 2>/dev/null)
if [ -z "$STAGING_NET" ]; then
echo "WARN: Could not inspect staging container networks (container may not exist yet)"
else
echo "Staging API container networks: $STAGING_NET"
if echo "$STAGING_NET" | grep -q "xiaoxia-net-staging"; then
echo "Network isolation OK: staging containers on xiaoxia-net-staging"
else
echo "WARN: staging containers not on expected xiaoxia-net-staging network"
echo " Current networks: $STAGING_NET"
fi
fi
# Verify cross-environment DNS isolation
# staging API should resolve to staging container, not production
STAGING_API_IP=$(docker exec xiaoxia-web-staging getent hosts xiaoxia-api-staging 2>/dev/null | awk "{print \$1}" || true)
PRODUCTION_API_IP=$(docker exec xiaoxia-web-staging getent hosts xiaoxia-api-production 2>/dev/null | awk "{print \$1}" || true)
if [ -n "$STAGING_API_IP" ]; then
echo "Staging API resolves to: $STAGING_API_IP (from web container)"
fi
if [ -n "$PRODUCTION_API_IP" ]; then
echo "FAIL: staging web container can resolve production API address ($PRODUCTION_API_IP) - network isolation broken!"
exit 1
else
echo "Network isolation OK: staging web cannot resolve xiaoxia-api-production"
fi
echo ""
echo "=== All smoke tests passed! ==="
'
staging-e2e:
name: Staging E2E Tests
runs-on: saas
if: github.ref_name == 'develop' || github.ref_name == 'main'
needs: deploy-staging
steps:
- name: Install SSH client
shell: sh
run: |
set -eu
apt-get update -qq && apt-get install -y -qq openssh-client >/dev/null 2>&1
echo "openssh-client installed"
- name: Run Playwright E2E on staging server
shell: sh
env:
STAGING_SSH_HOST: ${{ secrets.STAGING_SSH_HOST }}
STAGING_SSH_USER: ${{ secrets.STAGING_SSH_USER }}
STAGING_SSH_KEY: ${{ secrets.STAGING_SSH_KEY }}
run: |
set -eu
staging_host="${STAGING_SSH_HOST:-47.98.113.167}"
staging_user="${STAGING_SSH_USER:-root}"
if [ -f /root/.ssh/xiaoxia_runtime_builder ]; then
key_path="/root/.ssh/xiaoxia_runtime_builder"
elif [ -n "${STAGING_SSH_KEY:-}" ]; then
key_path="$HOME/.ssh/id_ed25519"
printf '%s\n' "$STAGING_SSH_KEY" > "$key_path"
chmod 600 "$key_path"
else
echo "ERROR: No SSH key available"
exit 1
fi
ssh-keyscan -H "$staging_host" >> ~/.ssh/known_hosts
# 在业务服务器上跑 Playwright E2E(用 host 网络访问 staging 3001/8000 端口)
ssh -i "$key_path" "$staging_user@$staging_host" '
cd /var/lib/xiaoxia-saas-staging/repo
docker run --rm \
-e E2E_BASE_URL=http://127.0.0.1:3001 \
-e E2E_API_BASE=http://127.0.0.1:8000/api/v1 \
-e E2E_BROWSER_CHANNEL=chromium \
-v "$PWD:/workspace" \
-w /workspace/apps/web \
--network host \
mcr.microsoft.com/playwright:v1.45.0-jammy \
sh -lc "npm ci && npx playwright test --reporter=line --project=chromium"
'
build-production-runtime-images:
name: Build Production Runtime Images
runs-on: saas
needs: [validate, frontend-lint]
if: startsWith(github.ref, 'refs/tags/v')
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
set -eu
python3 - <<'PY'
import io, os, tarfile, time, urllib.request, urllib.error
url = f"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz"
request = urllib.request.Request(url, headers={"Authorization": f"token {os.environ['GITHUB_TOKEN']}"})
last_err = None
for attempt in range(5):
try:
with urllib.request.urlopen(request, timeout=120) as response:
archive = response.read()
break
except urllib.error.HTTPError as e:
last_err = e
if e.code >= 500 and attempt < 4:
wait = 2 ** attempt
print(f"Checkout HTTP {e.code}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
except Exception as e:
last_err = e
if attempt < 4:
wait = 2 ** attempt
print(f"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
else:
raise last_err
with tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:
root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'
for member in tar.getmembers():
name = member.name
if name == root_prefix[:-1]:
continue
if name.startswith(root_prefix):
member.name = name[len(root_prefix):]
if member.name:
tar.extract(member, '.')
PY
- name: Build and push all images (api + worker + web, with buildx cache)
shell: sh
env:
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
set -eu
chmod +x scripts/build_release_images.sh
REGISTRY_TOKEN="${REGISTRY_TOKEN}" scripts/build_release_images.sh "${GITHUB_REF_NAME}"
- name: Package release source artifact
shell: sh
run: |
set -eu
mkdir -p dist/release-artifacts
tar --exclude=.git --exclude=apps/web/node_modules --exclude=./dist \
-czf "dist/release-artifacts/xiaoxia-release-${GITHUB_REF_NAME}.tar.gz" .
- name: Upload source artifact to production
shell: sh
env:
PRODUCTION_SSH_HOST: ${{ secrets.PRODUCTION_SSH_HOST }}
PRODUCTION_SSH_USER: ${{ secrets.PRODUCTION_SSH_USER }}
PRODUCTION_SSH_KEY: ${{ secrets.PRODUCTION_SSH_KEY }}
run: |
set -eu
production_host="${PRODUCTION_SSH_HOST:-47.98.113.167}"
production_user="${PRODUCTION_SSH_USER:-root}"
mkdir -p ~/.ssh
if [ -f /root/.ssh/xiaoxia_runtime_builder ]; then
key_path="/root/.ssh/xiaoxia_runtime_builder"
echo "Using host SSH key: $key_path"
elif [ -n "${PRODUCTION_SSH_KEY:-}" ]; then
key_path="$HOME/.ssh/id_ed25519"
printf '%s\n' "$PRODUCTION_SSH_KEY" > "$key_path"
chmod 600 "$key_path"
echo "Using secret SSH key: $key_path"
else
echo "ERROR: No SSH key available"
exit 1
fi
ssh-keyscan -H "$production_host" >> ~/.ssh/known_hosts
scp -i "$key_path" "dist/release-artifacts/xiaoxia-release-${GITHUB_REF_NAME}.tar.gz" \
"$production_user@$production_host:/var/lib/xiaoxia-saas-production/release-${GITHUB_REF_NAME}.tar.gz"
- name: Cleanup old Docker images
if: always()
shell: sh
run: |
set -eu
if [ -f scripts/cleanup_old_images.sh ]; then
chmod +x scripts/cleanup_old_images.sh
scripts/cleanup_old_images.sh
else
echo "Cleanup script not found, doing basic prune..."
docker image prune -f 2>/dev/null || true
fi
echo "Disk usage after cleanup:"
df -h / | tail -1
deploy-production:
name: Deploy Production
runs-on: saas
if: startsWith(github.ref, 'refs/tags/v')
needs: build-production-runtime-images
steps:
- name: Install SSH client
shell: sh
run: |
set -eu
apt-get update -qq && apt-get install -y -qq openssh-client >/dev/null 2>&1
echo "openssh-client installed"
- name: Deploy production over SSH (pull images from registry)
shell: sh
env:
PRODUCTION_SSH_HOST: ${{ secrets.PRODUCTION_SSH_HOST }}
PRODUCTION_SSH_USER: ${{ secrets.PRODUCTION_SSH_USER }}
PRODUCTION_SSH_KEY: ${{ secrets.PRODUCTION_SSH_KEY }}
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
set -eu
production_host="${PRODUCTION_SSH_HOST:-47.98.113.167}"
production_user="${PRODUCTION_SSH_USER:-root}"
mkdir -p ~/.ssh
if [ -f /root/.ssh/xiaoxia_runtime_builder ]; then
key_path="/root/.ssh/xiaoxia_runtime_builder"
elif [ -n "${PRODUCTION_SSH_KEY:-}" ]; then
key_path="$HOME/.ssh/id_ed25519"
printf '%s\n' "$PRODUCTION_SSH_KEY" > "$key_path"
chmod 600 "$key_path"
else
echo "ERROR: No SSH key available"
exit 1
fi
ssh-keyscan -H "$production_host" >> ~/.ssh/known_hosts
# 部署脚本(base64 编码避免转义问题)
DEPLOY_B64="c2V0IC1ldQpyZWxlYXNlX3Rhcj0iL3Zhci9saWIveGlhb3hpYS1zYWFzLXByb2R1Y3Rpb24vcmVsZWFzZS0ke1JFTEVBU0VfVkVSU0lPTn0udGFyLmd6Igp0ZXN0IC1mICIkcmVsZWFzZV90YXIiCm1rZGlyIC1wIC92YXIvbGliL3hpYW94aWEtc2Fhcy1wcm9kdWN0aW9uCm9sZF9hc3NldHNfZGlyPSIvdG1wL3hpYW94aWEtcHJldmlvdXMtd2ViLWFzc2V0cy0ke1JFTEVBU0VfVkVSU0lPTn0iCnJtIC1yZiAiJG9sZF9hc3NldHNfZGlyIgpta2RpciAtcCAiJG9sZF9hc3NldHNfZGlyIgppZiBkb2NrZXIgaW5zcGVjdCB4aWFveGlhLXdlYi1wcm9kdWN0aW9uID4vZGV2L251bGwgMj4mMTsgdGhlbgogIGRvY2tlciBjcCB4aWFveGlhLXdlYi1wcm9kdWN0aW9uOi91c3Ivc2hhcmUvbmdpbngvaHRtbC9hc3NldHMvLiAiJG9sZF9hc3NldHNfZGlyLyIgMj4vZGV2L251bGwgfHwgdHJ1ZQpmaQppZiBbIC1kIC92YXIvbGliL3hpYW94aWEtc2Fhcy1wcm9kdWN0aW9uL3JlcG8vYXBwcy93ZWIvZGlzdC9hc3NldHMgXTsgdGhlbgogIGNwIC1hIC92YXIvbGliL3hpYW94aWEtc2Fhcy1wcm9kdWN0aW9uL3JlcG8vYXBwcy93ZWIvZGlzdC9hc3NldHMvLiAiJG9sZF9hc3NldHNfZGlyLyIKZmkKcm0gLXJmIC92YXIvbGliL3hpYW94aWEtc2Fhcy1wcm9kdWN0aW9uL3JlcG8KbWtkaXIgLXAgL3Zhci9saWIveGlhb3hpYS1zYWFzLXByb2R1Y3Rpb24vcmVwbwp0YXIgLXh6ZiAiJHJlbGVhc2VfdGFyIiAtQyAvdmFyL2xpYi94aWFveGlhLXNhYXMtcHJvZHVjdGlvbi9yZXBvCnRlc3QgLWYgL3Zhci9saWIveGlhb3hpYS1zYWFzLXByb2R1Y3Rpb24vcmVwby9hcHBzL3dlYi9kaXN0L2luZGV4Lmh0bWwKaWYgWyAtZCAiJG9sZF9hc3NldHNfZGlyIiBdOyB0aGVuCiAgbWtkaXIgLXAgL3Zhci9saWIveGlhb3hpYS1zYWFzLXByb2R1Y3Rpb24vcmVwby9hcHBzL3dlYi9kaXN0L2Fzc2V0cwogIGZvciBhc3NldCBpbiAiJG9sZF9hc3NldHNfZGlyIi8qOyBkbwogICAgWyAtZSAiJGFzc2V0IiBdIHx8IGNvbnRpbnVlCiAgICBuYW1lPSIkKGJhc2VuYW1lICIkYXNzZXQiKSIKICAgIGlmIFsgISAtZSAiL3Zhci9saWIveGlhb3hpYS1zYWFzLXByb2R1Y3Rpb24vcmVwby9hcHBzL3dlYi9kaXN0L2Fzc2V0cy8kbmFtZSIgXTsgdGhlbgogICAgICBjcCAtYSAiJGFzc2V0IiAiL3Zhci9saWIveGlhb3hpYS1zYWFzLXByb2R1Y3Rpb24vcmVwby9hcHBzL3dlYi9kaXN0L2Fzc2V0cy8kbmFtZSIKICAgIGZpCiAgZG9uZQogIHJtIC1yZiAiJG9sZF9hc3NldHNfZGlyIgpmaQp0ZXN0IC1mIC92YXIvbGliL3hpYW94aWEtc2Fhcy1wcm9kdWN0aW9uLy5lbnYKY3AgL3Zhci9saWIveGlhb3hpYS1zYWFzLXByb2R1Y3Rpb24vLmVudiAvdmFyL2xpYi94aWFveGlhLXNhYXMtcHJvZHVjdGlvbi9yZXBvLy5lbnYKZG9ja2VyIG5ldHdvcmsgY3JlYXRlIHhpYW94aWEtbmV0LXByb2R1Y3Rpb24gMj4vZGV2L251bGwgfHwgdHJ1ZQpSRUdJU1RSWV9UT0tFTj0iJHtSRUdJU1RSWV9UT0tFTn0iIFJFTEVBU0VfVkVSU0lPTj0iJHtSRUxFQVNFX1ZFUlNJT059IiBIT1NUX1BSRUZJWD0gRU5WPXByb2R1Y3Rpb24gc2ggL3Zhci9saWIveGlhb3hpYS1zYWFzLXByb2R1Y3Rpb24vcmVwby9pbmZyYS9kb2NrZXIvZGVwbG95LXByb2R1Y3Rpb24uc2gKaT0wCndoaWxlIFsgIiRpIiAtbHQgMzAgXTsgZG8KICBpZiB3Z2V0IC1xTy0gaHR0cDovLzEyNy4wLjAuMTo4MDAxL2hlYWx0aDsgdGhlbgogICAgZXhpdCAwCiAgZmkKICBpPSQoKGkgKyAxKSkKICBzbGVlcCAyCmRvbmUKZXhpdCAxCg=="
echo "$DEPLOY_B64" | base64 -d | ssh -i "$key_path" "$production_user@$production_host" "RELEASE_VERSION='${GITHUB_REF_NAME}' REGISTRY_TOKEN='${REGISTRY_TOKEN}' sh"
production-e2e:
name: Production Browser E2E
runs-on: saas
if: startsWith(github.ref, 'refs/tags/v')
needs: deploy-production
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
set -eu
python3 - <<'PY'
import io, os, tarfile, time, urllib.request, urllib.error
url = f"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz"
request = urllib.request.Request(url, headers={"Authorization": f"token {os.environ['GITHUB_TOKEN']}"})
# Retry up to 5 times with backoff for transient 5xx errors
last_err = None
for attempt in range(5):
try:
with urllib.request.urlopen(request, timeout=120) as response:
archive = response.read()
break
except urllib.error.HTTPError as e:
last_err = e
if e.code >= 500 and attempt < 4:
wait = 2 ** attempt
print(f"Checkout HTTP {e.code}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
except Exception as e:
last_err = e
if attempt < 4:
wait = 2 ** attempt
print(f"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
else:
raise last_err
with tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:
root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'
for member in tar.getmembers():
name = member.name
if name == root_prefix[:-1]:
continue
if name.startswith(root_prefix):
member.name = name[len(root_prefix):]
if member.name:
tar.extract(member, '.')
PY
- name: Run production browser E2E
shell: sh
run: |
set -eu
docker run --rm \
-e E2E_BASE_URL=https://saas.xiaoxiajianji.com \
-e E2E_API_BASE=https://api.xiaoxiajianji.com/api/v1 \
-e E2E_BROWSER_CHANNEL=chromium \
-v "$PWD:/workspace" \
-w /workspace/apps/web \
mcr.microsoft.com/playwright:v1.45.0-jammy \
sh -lc 'npm ci && npx playwright test --reporter=line --project=chromium e2e/core-upload.spec.ts e2e/core-generation.spec.ts e2e/core-titles.spec.ts'
+78
View File
@@ -0,0 +1,78 @@
name: CI Failure Monitor
on:
schedule:
- cron: '0 */6 * * *' # 每6小时检查一次
workflow_dispatch:
inputs:
days:
description: '统计最近N天的失败'
required: false
default: '7'
fail_threshold:
description: '失败次数阈值'
required: false
default: '3'
fail_rate_threshold:
description: '失败率阈值(%)'
required: false
default: '30'
permissions:
contents: read
jobs:
monitor:
name: CI重复失败检测
runs-on: ci-l2
timeout-minutes: 10
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sH "Authorization: token $GITHUB_TOKEN" \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" \
| bash
- name: Record job start time
shell: sh
run: bash scripts/ci/step_timer_start.sh
- name: Run failure detection
shell: sh
env:
GITEA_API_TOKEN: ${{ secrets.REVIEW_GITEA_TOKEN }}
GITEA_URL: https://git.xiaoxiajianji.com
GITEA_REPO: xiaoxia/xiaoxia-saas
CI_NOTIFY_WEBHOOK: ${{ secrets.CI_NOTIFY_WEBHOOK }}
FAIL_CHECK_DAYS: ${{ inputs.days || 7 }}
FAIL_THRESHOLD: ${{ inputs.fail_threshold || 3 }}
FAIL_RATE_THRESHOLD: ${{ inputs.fail_rate_threshold || 30 }}
run: |
set +e
python3 scripts/ci/ci_repeated_failure_detector.py
EXIT_CODE=$?
echo "检测完成,退出码: $EXIT_CODE"
# 0=无异常, 1=有警告, 2=有严重问题
# 监控脚本永远不fail,避免告警风暴
exit 0
- name: Job duration summary
if: always()
shell: sh
run: bash scripts/ci/step_timer_end.sh
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
+103
View File
@@ -0,0 +1,103 @@
name: CI Health Daily Report
on:
schedule:
- cron: '0 1 * * *' # UTC 01:00 = 北京时间 09:00
workflow_dispatch:
permissions:
contents: read
jobs:
ci-health-report:
name: CI健康度每日巡检
runs-on: ci-l2
timeout-minutes: 15
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
set -eu
python3 - <<'PY'
import io, os, tarfile, time, urllib.request, urllib.error
url = f"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz"
request = urllib.request.Request(url, headers={"Authorization": f"token {os.environ['GITHUB_TOKEN']}"})
last_err = None
for attempt in range(5):
try:
with urllib.request.urlopen(request, timeout=120) as response:
archive = response.read()
break
except urllib.error.HTTPError as e:
last_err = e
if e.code >= 500 and attempt < 4:
wait = 2 ** attempt
print(f"Checkout HTTP {e.code}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
except Exception as e:
last_err = e
if attempt < 4:
wait = 2 ** attempt
print(f"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
else:
raise last_err
with tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:
root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'
for member in tar.getmembers():
name = member.name
if name == root_prefix[:-1]:
continue
if name.startswith(root_prefix):
member.name = name[len(root_prefix):]
if member.name:
tar.extract(member, '.')
PY
- name: Generate CI Dashboard HTML
shell: sh
env:
GITEA_TOKEN: ${{ github.token }}
run: |
set +e
echo "=== 生成 CI 健康度 HTML 看板 ==="
echo "时间: $(date '+%Y-%m-%d %H:%M:%S')"
echo ""
python3 scripts/ci/ci_dashboard.py --days 7 --html --html-output ci_dashboard.html
EXIT_CODE=$?
if [ $EXIT_CODE -eq 0 ] && [ -f ci_dashboard.html ]; then
HTML_SIZE=$(wc -c < ci_dashboard.html)
echo ""
echo "✅ HTML 看板生成成功 (${HTML_SIZE} bytes)"
echo "路径: $(pwd)/ci_dashboard.html"
# 输出文件内容前几行,方便在 Actions 日志中确认
echo ""
echo "--- 看板预览 (前 5 行) ---"
head -5 ci_dashboard.html
echo "...(完整内容见产物文件)"
else
echo "❌ HTML 看板生成失败 (exit code: $EXIT_CODE)"
fi
echo ""
# 永远成功,看板生成失败不影响主流程
exit 0
- name: Run CI health check and report
shell: sh
env:
GITEA_TOKEN: ${{ github.token }}
CI_NOTIFY_WEBHOOK: ${{ secrets.CI_NOTIFY_WEBHOOK }}
run: |
set +e
echo "=== CI健康度每日巡检 ==="
echo "时间: $(date '+%Y-%m-%d %H:%M:%S')"
echo ""
python3 scripts/ci/ci_health_report.py --limit 30
EXIT_CODE=$?
echo ""
echo "巡检完成 (exit code: $EXIT_CODE)"
# 永远成功,不影响CI状态(通知失败不应该标红)
exit 0
+2083
View File
File diff suppressed because it is too large Load Diff
+52
View File
@@ -0,0 +1,52 @@
name: CI Trigger Monitor
on:
schedule:
- cron: '*/5 * * * *' # 每5分钟检查一次
workflow_dispatch:
inputs:
stale_threshold:
description: 'CI未触发告警阈值(分钟)'
required: false
default: '5'
permissions:
contents: read
jobs:
monitor:
name: Monitor CI Trigger Reliability
runs-on: ci-l2
timeout-minutes: 5
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sH "Authorization: token $GITHUB_TOKEN" "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" | bash
- name: Check CI trigger status for all open PRs
env:
GITEA_API_TOKEN: ${{ secrets.REVIEW_GITEA_TOKEN }}
GITEA_URL: https://git.xiaoxiajianji.com
GITEA_REPO: xiaoxia/xiaoxia-saas
CI_NOTIFY_WEBHOOK: ${{ secrets.CI_NOTIFY_WEBHOOK }}
STALE_THRESHOLD_MIN: ${{ inputs.stale_threshold || 5 }}
run: |
set +e
python3 scripts/ci_trigger_monitor.py
# 监控脚本永远不fail,避免告警风暴
exit 0
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
+79
View File
@@ -0,0 +1,79 @@
name: AI Code Review
on:
pull_request:
types:
- opened
- synchronize
- reopened
# 同一个 PR 只跑一个 review,新的取消旧的
concurrency:
group: code-review-${{ gitea.repository }}-${{ gitea.event.pull_request.number }}
cancel-in-progress: true
jobs:
code-review:
name: AI Code Review
runs-on: ci-l2
# 跳过草稿 PR
if: ${{ !gitea.event.pull_request.draft }}
steps:
# actions/checkout 由 runner 在宿主机层面处理,不受容器网络影响
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sH "Authorization: token $GITHUB_TOKEN" "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" | bash
- name: Install dependencies
run: |
# 确保 python3-pip 可用(兼容不同基础镜像)
if ! python3 -m pip --version >/dev/null 2>&1; then
apt-get update -qq && apt-get install -y -qq python3-pip python3-venv >/dev/null 2>&1
fi
# 部分镜像 ensurepip 方式兜底
if ! python3 -m pip --version >/dev/null 2>&1; then
python3 -m ensurepip --upgrade 2>/dev/null || curl -sS https://bootstrap.pypa.io/get-pip.py | python3
fi
python3 -m pip install --upgrade pip
python3 -m pip install requests
- name: Run AI Code Review
env:
# Gitea 配置(自动从运行环境获取)
GITEA_API_URL: ${{ gitea.server_url }}
GITEA_TOKEN: ${{ secrets.REVIEW_GITEA_TOKEN }}
REPO_NAME: ${{ gitea.repository }}
PR_NUMBER: ${{ gitea.event.pull_request.number }}
PR_HEAD_SHA: ${{ gitea.event.pull_request.head.sha }}
# LLM 提供商: coze (扣子原生Bot) / openai (OpenAI兼容)
LLM_PROVIDER: "coze"
# 扣子模式配置(默认国内站 api.coze.cn
LLM_BASE_URL: ${{ secrets.LLM_BASE_URL }}
LLM_API_KEY: ${{ secrets.LLM_API_KEY }}
COZE_BOT_ID: ${{ secrets.COZE_BOT_ID }}
LLM_MODEL: ${{ secrets.LLM_MODEL }}
# 可选参数
MAX_DIFF_CHARS: "30000"
LLM_TIMEOUT: "120"
run: |
python3 scripts/ci_code_review.py
# 注意:脚本退出码决定job状态
# - 有阻塞级问题 → exit 1 → job失败 → 门禁拦截
# - 无阻塞级问题/LLM异常 → exit 0 → 通过(fail-open
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
+624
View File
@@ -0,0 +1,624 @@
name: Daily Health Check
# 注意:使用 curl step_checkout.sh 方式以兼容 docker runner
on:
schedule:
- cron: '0 19 * * *' # UTC 19:00 = 北京时间凌晨 3:00
workflow_dispatch:
permissions:
contents: read
jobs:
# ── 1. 生产环境冒烟测试 ─────────────────────────────────────────────
production-smoke:
name: Production Smoke Test
runs-on: ci-l2
timeout-minutes: 8
outputs:
report: ${{ steps.smoke.outputs.report }}
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sH "Authorization: token $GITHUB_TOKEN" \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" \
| bash
- name: Production health check & smoke test
id: smoke
shell: bash
env:
SMOKE_ENV: production
EXISTING_TOKEN: ${{ secrets.PROD_E2E_TOKEN }}
MODULES: health,assets,generation,subscription,nginx
run: |
set +e
START_TIME=$(date +%s)
chmod +x tests/e2e/api_smoke_test.sh
BASE_URL="https://api.xiaoxiajianji.com" \
WEB_URL="https://saas.xiaoxiajianji.com" \
SMOKE_ENV="${SMOKE_ENV}" \
EXISTING_TOKEN="${EXISTING_TOKEN}" \
MODULES="${MODULES}" \
CLEANUP_ENABLED=0 \
PERF_CHECK_ENABLED=1 \
PERF_WARN_THRESHOLD_MS=500 \
PERF_FAIL_THRESHOLD_MS=5000 \
bash tests/e2e/api_smoke_test.sh 2>&1 | tee /tmp/prod-smoke.log
SMOKE_EXIT=${PIPESTATUS[0]}
END_TIME=$(date +%s)
ELAPSED=$((END_TIME - START_TIME))
echo ""
echo "========== 生产冒烟测试报告 =========="
echo "环境: https://api.xiaoxiajianji.com"
echo "耗时: ${ELAPSED}s"
# 提取通过/失败数
grep "测试完成:" /tmp/prod-smoke.log || true
if [ "$SMOKE_EXIT" -eq 0 ]; then
echo "结果: PASS"
echo "report=PASS" >> "${GITHUB_OUTPUT}"
else
echo "结果: FAIL"
grep "失败用例:" /tmp/prod-smoke.log || true
echo "report=FAIL" >> "${GITHUB_OUTPUT}"
fi
echo "======================================"
exit $SMOKE_EXIT
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
# ── 2. Staging API 集成测试 ─────────────────────────────────────────
staging-api-tests:
name: Staging API Integration Tests
runs-on: ci-l2
timeout-minutes: 10
outputs:
report: ${{ steps.report.outputs.report }}
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sH "Authorization: token $GITHUB_TOKEN" \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" \
| bash
- name: Run API smoke test on staging
id: smoke
shell: bash
env:
STAGING_TEST_USER: ${{ secrets.STAGING_TEST_USER }}
STAGING_TEST_PASSWORD: ${{ secrets.STAGING_TEST_PASSWORD }}
run: |
set +e
START_TIME=$(date +%s)
chmod +x tests/e2e/api_smoke_test.sh
CONTAINER_NAME="ci-test-$$"
docker create --name "$CONTAINER_NAME" \
-e BASE_URL=https://staging-api.xiaoxiajianji.com \
-e WEB_URL=https://staging.xiaoxiajianji.com \
-e TEST_USER="$STAGING_TEST_USER" \
-e TEST_PASSWORD="$STAGING_TEST_PASSWORD" \
-e CLEANUP_ENABLED=1 \
-e PERF_CHECK_ENABLED=1 \
-e PERF_WARN_THRESHOLD_MS=500 \
-e PERF_FAIL_THRESHOLD_MS=3000 \
-w /workspace \
git.xiaoxiajianji.com/xiaoxia/base/playwright:v1.45.0-jammy \
bash tests/e2e/api_smoke_test.sh 2>&1
docker cp . "$CONTAINER_NAME:/workspace"
docker start -a "$CONTAINER_NAME" 2>&1 | tee /tmp/staging-api-smoke.log
SMOKE_EXIT=${PIPESTATUS[0]}
docker rm "$CONTAINER_NAME" > /dev/null 2>&1 || true
END_TIME=$(date +%s)
ELAPSED=$((END_TIME - START_TIME))
echo ""
echo "========== Staging API 冒烟测试报告 =========="
echo "环境: https://staging-api.xiaoxiajianji.com"
echo "耗时: ${ELAPSED}s"
grep "测试完成:" /tmp/staging-api-smoke.log || true
if [ "$SMOKE_EXIT" -eq 0 ]; then
echo "结果: PASS"
echo "api_report=PASS" >> "${GITHUB_OUTPUT}"
else
echo "结果: FAIL"
grep "失败用例:" /tmp/staging-api-smoke.log || true
echo "api_report=FAIL" >> "${GITHUB_OUTPUT}"
fi
echo "=============================================="
exit $SMOKE_EXIT
- name: Run Staging API Integration Tests (Playwright)
id: e2e_api
shell: bash
run: |
set +e
START_TIME=$(date +%s)
CONTAINER_NAME="ci-test-$$"
docker create --name "$CONTAINER_NAME" \
-e E2E_BASE_URL=https://staging.xiaoxiajianji.com \
-e E2E_API_BASE=https://staging-api.xiaoxiajianji.com/api/v1 \
-w /workspace/apps/web \
git.xiaoxiajianji.com/xiaoxia/base/playwright:v1.45.0-jammy \
sh -lc "npm ci && npx playwright test --reporter=line e2e/test_auth.spec.ts e2e/test_asset.spec.ts e2e/test_project.spec.ts" 2>&1
docker cp . "$CONTAINER_NAME:/workspace"
docker start -a "$CONTAINER_NAME" 2>&1 | tee /tmp/staging-api-e2e.log
EXIT_CODE=${PIPESTATUS[0]}
docker rm "$CONTAINER_NAME" > /dev/null 2>&1 || true
END_TIME=$(date +%s)
ELAPSED=$((END_TIME - START_TIME))
echo ""
echo "========== Staging API 集成测试报告 =========="
echo "环境: https://staging-api.xiaoxiajianji.com"
echo "耗时: ${ELAPSED}s"
grep -E "passed|failed|timed out" /tmp/staging-api-e2e.log || true
if [ "$EXIT_CODE" -eq 0 ]; then
echo "结果: PASS"
echo "int_report=PASS" >> "${GITHUB_OUTPUT}"
else
echo "结果: FAIL"
echo "int_report=FAIL" >> "${GITHUB_OUTPUT}"
fi
echo "=============================================="
exit $EXIT_CODE
- name: Set report output
id: report
shell: sh
run: |
if [ "${{ steps.smoke.outputs.api_report }}" = "PASS" ] && [ "${{ steps.e2e_api.outputs.int_report }}" = "PASS" ]; then
echo "report=PASS" >> "${GITHUB_OUTPUT}"
else
echo "report=FAIL" >> "${GITHUB_OUTPUT}"
fi
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
# ── 3. Staging 浏览器 E2E ──────────────────────────────────────────
staging-e2e:
name: Staging Browser E2E
runs-on: ci-l2
timeout-minutes: 15
outputs:
report: ${{ steps.e2e.outputs.report }}
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sH "Authorization: token $GITHUB_TOKEN" \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" \
| bash
- name: Run Playwright E2E on staging
id: e2e
shell: bash
run: |
set +e
START_TIME=$(date +%s)
CONTAINER_NAME="ci-test-$$"
docker create --name "$CONTAINER_NAME" --ipc=host \
-e E2E_BASE_URL=https://staging.xiaoxiajianji.com \
-e E2E_API_BASE=https://staging-api.xiaoxiajianji.com/api/v1 \
-e E2E_BROWSER_CHANNEL=chromium \
-e PLAYWRIGHT_HEADLESS=1 \
-w /workspace/apps/web \
git.xiaoxiajianji.com/xiaoxia/base/playwright:v1.45.0-jammy \
sh -lc 'npm ci && npx playwright test --reporter=line --project=chromium e2e/auth.spec.ts e2e/auth-guard.spec.ts e2e/core-upload.spec.ts e2e/core-generation.spec.ts e2e/core-titles.spec.ts' 2>&1
docker cp . "$CONTAINER_NAME:/workspace"
docker start -a "$CONTAINER_NAME" 2>&1 | tee /tmp/staging-e2e.log
EXIT_CODE=${PIPESTATUS[0]}
docker rm "$CONTAINER_NAME" > /dev/null 2>&1 || true
END_TIME=$(date +%s)
ELAPSED=$((END_TIME - START_TIME))
echo ""
echo "========== Staging E2E 测试报告 =========="
echo "环境: https://staging.xiaoxiajianji.com"
echo "耗时: ${ELAPSED}s"
grep -E "passed|failed|timed out" /tmp/staging-e2e.log || true
if [ "$EXIT_CODE" -eq 0 ]; then
echo "结果: PASS"
echo "report=PASS" >> "${GITHUB_OUTPUT}"
else
echo "结果: FAIL"
echo "report=FAIL" >> "${GITHUB_OUTPUT}"
fi
echo "=========================================="
exit $EXIT_CODE
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
# ── 4. 性能基线巡检 ────────────────────────────────────────────────
performance-check:
name: Performance Baseline Check
runs-on: ci-l2
timeout-minutes: 8
outputs:
report: ${{ steps.report.outputs.report }}
steps:
- name: Run performance baseline checks
id: perf
shell: sh
env:
STAGING_TEST_USER: ${{ secrets.STAGING_TEST_USER }}
STAGING_TEST_PASSWORD: ${{ secrets.STAGING_TEST_PASSWORD }}
run: |
set +e
START_TIME=$(date +%s)
echo "=========================================="
echo " 性能基线巡检 - Staging API"
echo " 目标: https://staging-api.xiaoxiajianji.com"
echo "=========================================="
echo ""
TOTAL=0
PASS=0
FAIL=0
WARN=0
WARN_LIST=""
FAIL_LIST=""
# 核心接口配置: 名称|路径|方法|阈值(ms)|失败阈值(ms)
# 核心接口(core): 500ms
# 普通接口(normal): 1000ms
# 重操作接口(heavy): 3000ms
ENDPOINTS="
登录|/api/v1/auth/login|POST|500|3000
获取当前用户|/api/v1/auth/me|GET|500|3000
项目列表|/api/v1/projects|GET|500|3000
素材列表|/api/v1/assets|GET|500|3000
模板列表|/api/v1/templates|GET|500|3000
剪辑计划列表|/api/v1/edit-plans|GET|500|3000
生成任务列表|/api/v1/generation/tasks|GET|500|3000
订阅信息|/api/v1/subscription/current|GET|500|3000
音色列表|/api/v1/voices|GET|1000|5000
健康检查|/health|GET|200|1000
"
# 先登录获取 token
echo "--- 准备: 获取测试 Token ---"
LOGIN_BODY="{\"email\":\"$STAGING_TEST_USER\",\"password\":\"$STAGING_TEST_PASSWORD\"}"
AUTH_RESP=$(curl -s -w "\n%{http_code}" -X POST \
-H "Content-Type: application/json" \
-d "$LOGIN_BODY" \
"https://staging-api.xiaoxiajianji.com/api/v1/auth/login" \
--max-time 10 2>&1)
AUTH_CODE=$(echo "$AUTH_RESP" | tail -1)
AUTH_BODY=$(echo "$AUTH_RESP" | sed '$d')
if [ "$AUTH_CODE" = "200" ]; then
TOKEN=$(echo "$AUTH_BODY" | python3 -c "import sys,json; d=json.load(sys.stdin); print(d.get('access_token',''))" 2>/dev/null)
if [ -n "$TOKEN" ]; then
echo "Token 获取成功"
else
echo "Token 解析失败,部分接口可能无法测试"
TOKEN=""
fi
else
echo "登录失败 (HTTP $AUTH_CODE),部分接口将跳过鉴权测试"
TOKEN=""
fi
echo ""
echo "--- 开始性能测试 ---"
echo ""
echo "$ENDPOINTS" | while IFS='|' read -r name path method warn_ms fail_ms; do
[ -z "$name" ] && continue
TOTAL=$((TOTAL + 1))
# 构建 curl 命令
CURL_ARGS="-s -o /dev/null -w '%{http_code} %{time_total}' --max-time 30"
if [ "$method" = "POST" ]; then
CURL_ARGS="$CURL_ARGS -X POST -H 'Content-Type: application/json' -d \"$LOGIN_BODY\""
fi
if [ -n "$TOKEN" ] && [ "$name" != "健康检查" ]; then
CURL_ARGS="$CURL_ARGS -H 'Authorization: Bearer $TOKEN'"
fi
# 执行请求
RESP=$(eval curl $CURL_ARGS "https://staging-api.xiaoxiajianji.com${path}" 2>&1)
HTTP_CODE=$(echo "$RESP" | awk '{print $1}')
TIME_TOTAL=$(echo "$RESP" | awk '{print $2}')
ELAPSED_MS=$(python3 -c "print(int(float('${TIME_TOTAL:-0}') * 1000))" 2>/dev/null || echo "0")
if [ "$HTTP_CODE" -ge 500 ] 2>/dev/null; then
FAIL=$((FAIL + 1))
FAIL_LIST="$FAIL_LIST\n ❌ $name - HTTP $HTTP_CODE (${ELAPSED_MS}ms)"
echo "❌ $name - HTTP $HTTP_CODE - ${ELAPSED_MS}ms (FAIL)"
elif [ "$ELAPSED_MS" -ge "$fail_ms" ] 2>/dev/null; then
FAIL=$((FAIL + 1))
FAIL_LIST="$FAIL_LIST\n ❌ $name - ${ELAPSED_MS}ms > ${fail_ms}ms"
echo "❌ $name - ${ELAPSED_MS}ms > ${fail_ms}ms (FAIL)"
elif [ "$ELAPSED_MS" -ge "$warn_ms" ] 2>/dev/null; then
WARN=$((WARN + 1))
WARN_LIST="$WARN_LIST\n ⚠️ $name - ${ELAPSED_MS}ms > ${warn_ms}ms"
echo "⚠️ $name - ${ELAPSED_MS}ms (WARN, threshold: ${warn_ms}ms)"
PASS=$((PASS + 1))
else
PASS=$((PASS + 1))
echo "✅ $name - ${ELAPSED_MS}ms (OK, threshold: ${warn_ms}ms)"
fi
done
# 由于 while 在子 shell 中执行,用文件传递结果
# 重新跑一次用文件计数方式
echo ""
echo "--- 汇总性能数据 ---"
END_TIME=$(date +%s)
ELAPSED=$((END_TIME - START_TIME))
echo ""
echo "========== 性能基线巡检报告 =========="
echo "环境: https://staging-api.xiaoxiajianji.com"
echo "耗时: ${ELAPSED}s"
echo "======================================"
- name: Generate performance report
id: report
shell: sh
env:
STAGING_TEST_USER: ${{ secrets.STAGING_TEST_USER }}
STAGING_TEST_PASSWORD: ${{ secrets.STAGING_TEST_PASSWORD }}
run: |
set +e
echo ""
echo "=========================================="
echo " 性能基线巡检 - 详细报告"
echo "=========================================="
TOTAL=0
PASS=0
FAIL=0
WARN=0
RESULTS=""
START_TIME=$(date +%s)
LOGIN_BODY="{\"email\":\"$STAGING_TEST_USER\",\"password\":\"$STAGING_TEST_PASSWORD\"}"
# 先登录获取 token
AUTH_RESP=$(curl -s -w "\n%{http_code}" -X POST \
-H "Content-Type: application/json" \
-d "$LOGIN_BODY" \
"https://staging-api.xiaoxiajianji.com/api/v1/auth/login" \
--max-time 10 2>&1)
AUTH_CODE=$(echo "$AUTH_RESP" | tail -1)
AUTH_BODY=$(echo "$AUTH_RESP" | sed '$d')
TOKEN=""
if [ "$AUTH_CODE" = "200" ]; then
TOKEN=$(echo "$AUTH_BODY" | python3 -c "import sys,json; d=json.load(sys.stdin); print(d.get('access_token',''))" 2>/dev/null || echo "")
fi
run_perf_test() {
local name="$1" path="$2" method="$3" warn_ms="$4" fail_ms="$5"
TOTAL=$((TOTAL + 1))
local CURL_ARGS="-s -o /dev/null -w '%{http_code} %{time_total}' --max-time 30"
if [ "$method" = "POST" ]; then
CURL_ARGS="$CURL_ARGS -X POST -H 'Content-Type: application/json' -d \"$LOGIN_BODY\""
fi
if [ -n "$TOKEN" ] && [ "$name" != "健康检查" ]; then
CURL_ARGS="$CURL_ARGS -H 'Authorization: Bearer $TOKEN'"
fi
local RESP=$(eval curl $CURL_ARGS "https://staging-api.xiaoxiajianji.com${path}" 2>&1)
local HTTP_CODE=$(echo "$RESP" | awk '{print $1}')
local TIME_TOTAL=$(echo "$RESP" | awk '{print $2}')
local ELAPSED_MS=$(python3 -c "print(int(float('${TIME_TOTAL:-0}') * 1000))" 2>/dev/null || echo "0")
if echo "$HTTP_CODE" | grep -q "^[5]"; then
FAIL=$((FAIL + 1))
RESULTS="$RESULTS\n ❌ $name - HTTP $HTTP_CODE (${ELAPSED_MS}ms)"
echo "❌ $name - HTTP $HTTP_CODE - ${ELAPSED_MS}ms [FAIL]"
return 1
elif [ "$ELAPSED_MS" -ge "$fail_ms" ] 2>/dev/null; then
FAIL=$((FAIL + 1))
RESULTS="$RESULTS\n ❌ $name - ${ELAPSED_MS}ms > ${fail_ms}ms [FAIL]"
echo "❌ $name - ${ELAPSED_MS}ms > ${fail_ms}ms [FAIL]"
return 1
elif [ "$ELAPSED_MS" -ge "$warn_ms" ] 2>/dev/null; then
WARN=$((WARN + 1))
PASS=$((PASS + 1))
RESULTS="$RESULTS\n ⚠️ $name - ${ELAPSED_MS}ms (阈值: ${warn_ms}ms) [WARN]"
echo "⚠️ $name - ${ELAPSED_MS}ms > 阈值 ${warn_ms}ms [WARN]"
return 0
else
PASS=$((PASS + 1))
RESULTS="$RESULTS\n ✅ $name - ${ELAPSED_MS}ms (阈值: ${warn_ms}ms) [OK]"
echo "✅ $name - ${ELAPSED_MS}ms (阈值: ${warn_ms}ms) [OK]"
return 0
fi
}
echo ""
echo "=== 核心接口 (阈值: 500ms / 3000ms) ==="
run_perf_test "登录" "/api/v1/auth/login" "POST" 500 3000 || true
run_perf_test "获取当前用户" "/api/v1/auth/me" "GET" 500 3000 || true
run_perf_test "项目列表" "/api/v1/projects" "GET" 500 3000 || true
run_perf_test "素材列表" "/api/v1/assets" "GET" 500 3000 || true
run_perf_test "模板列表" "/api/v1/templates" "GET" 500 3000 || true
run_perf_test "剪辑计划列表" "/api/v1/edit-plans" "GET" 500 3000 || true
run_perf_test "生成任务列表" "/api/v1/generation/tasks" "GET" 500 3000 || true
run_perf_test "订阅信息" "/api/v1/subscription/current" "GET" 500 3000 || true
echo ""
echo "=== 普通接口 (阈值: 1000ms / 5000ms) ==="
run_perf_test "音色列表" "/api/v1/voices" "GET" 1000 5000 || true
echo ""
echo "=== 基础接口 (阈值: 200ms / 1000ms) ==="
run_perf_test "健康检查" "/health" "GET" 200 1000 || true
END_TIME=$(date +%s)
ELAPSED=$((END_TIME - START_TIME))
echo ""
echo "========== 性能基线巡检报告 =========="
echo "环境: https://staging-api.xiaoxiajianji.com"
echo "总接口: ${TOTAL}"
echo "通过: ${PASS}"
echo "失败: ${FAIL}"
echo "警告: ${WARN}"
echo "耗时: ${ELAPSED}s"
echo "======================================"
# 写入结果文件供 report job 使用
echo "${TOTAL}" > /tmp/perf_total
echo "${PASS}" > /tmp/perf_pass
echo "${FAIL}" > /tmp/perf_fail
echo "${WARN}" > /tmp/perf_warn
echo "${ELAPSED}" > /tmp/perf_elapsed
if [ "$FAIL" -gt 0 ]; then
echo "report=FAIL" >> "${GITHUB_OUTPUT}"
echo "perf_detail=fail:${FAIL}:warn:${WARN}" >> "${GITHUB_OUTPUT}"
exit 1
else
echo "report=PASS" >> "${GITHUB_OUTPUT}"
if [ "$WARN" -gt 0 ]; then
echo "perf_detail=pass:warn:${WARN}" >> "${GITHUB_OUTPUT}"
else
echo "perf_detail=pass" >> "${GITHUB_OUTPUT}"
fi
exit 0
fi
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
# ── 5. 每日巡检汇总报告 ────────────────────────────────────────────
daily-report:
name: Daily Check Report
runs-on: ci-l2
timeout-minutes: 2
if: always()
needs:
- production-smoke
- staging-api-tests
- staging-e2e
- performance-check
steps:
- name: Print summary report
shell: sh
run: |
echo ""
echo "╔══════════════════════════════════════════════════════╗"
echo "║ 每日巡检报告 ║"
echo "╠══════════════════════════════════════════════════════╣"
# 获取各 job 状态
PROD_STATUS="${{ needs.production-smoke.result }}"
STAGING_API_STATUS="${{ needs.staging-api-tests.result }}"
STAGING_E2E_STATUS="${{ needs.staging-e2e.result }}"
PERF_STATUS="${{ needs.performance-check.result }}"
format_result() {
if [ "$1" = "success" ]; then
echo "✅ PASS"
elif [ "$1" = "failure" ]; then
echo "❌ FAIL"
elif [ "$1" = "skipped" ]; then
echo "⏭️ SKIP"
else
echo "❓ UNKNOWN ($1)"
fi
}
echo "║"
echo "║ 生产冒烟测试: $(format_result "$PROD_STATUS")"
echo "║ Staging API: $(format_result "$STAGING_API_STATUS")"
echo "║ Staging E2E: $(format_result "$STAGING_E2E_STATUS")"
echo "║ 性能基线巡检: $(format_result "$PERF_STATUS")"
echo "║"
echo "║ 巡检时间: $(date '+%Y-%m-%d %H:%M:%S UTC')"
echo "║"
# 判断整体状态
ALL_PASS=true
FAILED_ITEMS=""
for status_name in "$PROD_STATUS:生产冒烟" "$STAGING_API_STATUS:Staging API" "$STAGING_E2E_STATUS:Staging E2E" "$PERF_STATUS:性能基线"; do
STATUS=$(echo "$status_name" | cut -d: -f1)
NAME=$(echo "$status_name" | cut -d: -f2)
if [ "$STATUS" != "success" ] && [ "$STATUS" != "skipped" ]; then
ALL_PASS=false
FAILED_ITEMS="$FAILED_ITEMS $NAME"
fi
done
echo "╠══════════════════════════════════════════════════════╣"
if [ "$ALL_PASS" = "true" ]; then
echo "║ 整体状态: ✅ 全部通过 ║"
else
echo "║ 整体状态: ❌ 存在失败 ║"
echo "║ 失败项: ${FAILED_ITEMS} ║"
fi
echo "╚══════════════════════════════════════════════════════╝"
echo ""
# 如果有失败项,以非零退出码结束(方便 Gitea 标记流水线失败)
if [ "$ALL_PASS" = "false" ]; then
echo "⚠️ 部分巡检项失败,请检查上方日志获取详细信息。"
# 不 exit 1,因为我们用了 always(),保持 report job 成功,
# 但其他失败的 job 已经让整体流水线标记为失败
fi
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
+56
View File
@@ -0,0 +1,56 @@
name: PR Auto Scan
# 定时扫描所有open PR,对CI全绿的触发审批/合并
# 作为短作业模式的兜底,防止事件驱动遗漏
on:
schedule:
- cron: "*/5 * * * *" # 每5分钟扫描一次
workflow_dispatch:
permissions:
contents: read
jobs:
auto-scan:
name: Auto Scan Open PRs
runs-on: ci-check
timeout-minutes: 5
if: github.repository == 'xiaoxia/xiaoxia-saas'
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sH "Authorization: token $GITHUB_TOKEN" "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/pr_auto_scan.py?ref=develop" -o /tmp/pr_auto_scan.py
python3 /tmp/pr_auto_scan.py --help > /dev/null 2>&1 || {
# fallback: checkout
echo "使用checkout方式"
curl -sH "Authorization: token $GITHUB_TOKEN" "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=develop" | bash
}
- name: Scan and auto process PRs
shell: bash
env:
GITHUB_TOKEN: ${{ github.token }}
REVIEW_TOKEN: ${{ secrets.REVIEW_GITEA_TOKEN }}
MERGE_TOKEN: ${{ secrets.REVIEW_GITEA_TOKEN }}
run: |
set -eu
echo "=== 扫描所有open PR并自动处理 ==="
echo "时间: $(date)"
echo
python3 /tmp/pr_auto_scan.py --token "$REVIEW_TOKEN" --repo "$GITHUB_REPOSITORY" --base develop --approve --merge --dry-run false
echo ""
echo "✅ 扫描完成"
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "" || true
+117
View File
@@ -0,0 +1,117 @@
name: PR Automation
on:
pull_request:
types: [synchronize, opened, ready_for_review, review_requested]
workflow_dispatch:
permissions:
contents: read
concurrency:
group: pr-automation-${{ gitea.event.pull_request.number }}
cancel-in-progress: true
jobs:
auto-approve:
name: Auto Approve on CI Green
runs-on: ci-check
if: github.event_name == 'pull_request' && !github.event.pull_request.draft
timeout-minutes: 3 # 长等待模式:等CI全绿后自动合并,不遗漏任何PR
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sH "Authorization: token $GITHUB_TOKEN" "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" | bash
- name: "🔍 脚本语法自检"
shell: bash
run: |
ERROR=0
for f in scripts/ci/*.sh; do [ -f "$f" ] && bash -n "$f" 2>&1 || ERROR=$((ERROR+1)); done
for f in scripts/ci/*.py; do [ -f "$f" ] && python3 -m py_compile "$f" 2>&1 || ERROR=$((ERROR+1)); done
if [ "$ERROR" -ne 0 ]; then echo "❌ 语法自检失败 ($ERROR个)"; exit 1; fi
echo "✅ 脚本语法自检通过"
- name: Auto approve when CI passes
shell: bash
env:
GITHUB_TOKEN: ${{ github.token }}
REVIEW_TOKEN: ${{ secrets.REVIEW_GITEA_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
bash scripts/ci/auto_approve.sh
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
auto-merge:
name: Auto Merge on CI Green + Approved
runs-on: ci-check
if: github.event_name == 'pull_request' && !github.event.pull_request.draft && github.event.pull_request.base.ref == 'develop'
timeout-minutes: 3 # 短作业模式:检查一次,不满足就退出,由pr-auto-scan每5分钟定时兜底
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sH "Authorization: token $GITHUB_TOKEN" "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" | bash
- name: "🔍 脚本语法自检(防止脚本bug导致所有PR挂掉)"
shell: bash
run: |
echo "=== CI脚本语法自检 ==="
ERROR=0
for f in scripts/ci/*.sh; do
[ -f "$f" ] || continue
if ! bash -n "$f" 2>&1; then
echo "FAIL: $f"
ERROR=1
fi
done
for f in scripts/ci/*.py; do
[ -f "$f" ] || continue
if ! python3 -m py_compile "$f" 2>&1; then
echo "FAIL: $f"
ERROR=1
fi
done
if [ "$ERROR" -ne 0 ]; then
echo "❌ 脚本语法自检失败"
exit 1
fi
echo "✅ 所有CI脚本语法自检通过"
- name: Auto merge when CI passes and approved
shell: bash
env:
GITHUB_TOKEN: ${{ github.token }}
MERGE_TOKEN: ${{ secrets.REVIEW_GITEA_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
BASE_REF: ${{ github.event.pull_request.base.ref }}
run: |
bash scripts/ci/auto_merge.sh
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
+207
View File
@@ -0,0 +1,207 @@
name: Preview Cleanup
on:
pull_request:
types:
- closed
branches:
- main
- develop
permissions:
contents: read
pull-requests: write
jobs:
cleanup-preview:
name: Cleanup Preview Environment
runs-on: runtime-builder
timeout-minutes: 10
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
set -eu
python3 - <<'PY'
import io, os, tarfile, time, urllib.request, urllib.error
url = f"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz"
request = urllib.request.Request(url, headers={"Authorization": f"token {os.environ['GITHUB_TOKEN']}"})
last_err = None
for attempt in range(5):
try:
with urllib.request.urlopen(request, timeout=120) as response:
archive = response.read()
break
except urllib.error.HTTPError as e:
last_err = e
if e.code >= 500 and attempt < 4:
wait = 2 ** attempt
print(f"Checkout HTTP {e.code}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
except Exception as e:
last_err = e
if attempt < 4:
wait = 2 ** attempt
print(f"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
else:
raise last_err
with tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:
root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'
for member in tar.getmembers():
name = member.name
if name == root_prefix[:-1]:
continue
if name.startswith(root_prefix):
member.name = name[len(root_prefix):]
if member.name:
tar.extract(member, '.')
PY
- name: Extract PR number
shell: sh
run: |
set -eu
# 优先从event payload中读取(兼容所有PR事件类型)
if [ -n "${GITHUB_EVENT_PATH:-}" ] && [ -f "$GITHUB_EVENT_PATH" ]; then
PR_NUMBER=$(python3 -c "import json,sys; print(json.load(sys.stdin).get('number',''))" < "$GITHUB_EVENT_PATH")
fi
# fallback: 从GITHUB_REF中提取
if [ -z "${PR_NUMBER:-}" ]; then
PR_NUMBER=$(echo "$GITHUB_REF" | sed -n 's|refs/pull/\([0-9]*\)/.*|\1|p')
fi
# 再fallback: 兼容纯数字ref
if [ -z "${PR_NUMBER:-}" ] || ! echo "$PR_NUMBER" | grep -qE '^[0-9]+$'; then
echo "WARNING: Could not extract PR number cleanly, using raw ref suffix"
PR_NUMBER=$(echo "$GITHUB_REF" | sed 's|refs/pull/||; s|/.*||')
fi
echo "PR_NUMBER=$PR_NUMBER" >> $GITHUB_ENV
echo "PR number: $PR_NUMBER"
echo "Preview dir: /var/www/preview/pr-${PR_NUMBER}"
- name: Install SSH client
shell: sh
run: |
set -eu
# 先检查是否已存在ssh
if command -v ssh >/dev/null 2>&1 && command -v ssh-keyscan >/dev/null 2>&1; then
echo "SSH client already available: $(ssh -V 2>&1)"
exit 0
fi
# 尝试多种包管理器安装
if command -v apk >/dev/null 2>&1; then
apk add --no-cache openssh-client >/dev/null 2>&1
echo "openssh-client installed via apk"
elif command -v apt-get >/dev/null 2>&1; then
apt-get update -qq && apt-get install -y -qq openssh-client >/dev/null 2>&1
echo "openssh-client installed via apt-get"
elif command -v yum >/dev/null 2>&1; then
yum install -y openssh-clients >/dev/null 2>&1
echo "openssh-client installed via yum"
elif command -v dnf >/dev/null 2>&1; then
dnf install -y openssh-clients >/dev/null 2>&1
echo "openssh-client installed via dnf"
else
echo "ERROR: No package manager found and ssh not pre-installed"
which ssh 2>/dev/null || echo " ssh: not found"
which ssh-keyscan 2>/dev/null || echo " ssh-keyscan: not found"
exit 1
fi
- name: Remove preview directory from server
shell: sh
env:
PREVIEW_SSH_HOST: ${{ secrets.PREVIEW_SSH_HOST }}
PREVIEW_SSH_USER: ${{ secrets.PREVIEW_SSH_USER }}
PREVIEW_SSH_PORT: ${{ secrets.PREVIEW_SSH_PORT }}
PREVIEW_SSH_KEY: ${{ secrets.PREVIEW_SSH_KEY }}
run: |
set -eux
preview_host="${PREVIEW_SSH_HOST:-47.98.113.167}"
preview_user="${PREVIEW_SSH_USER:-root}"
preview_port="${PREVIEW_SSH_PORT:-22222}"
preview_dir="/var/www/preview/pr-${PR_NUMBER}"
mkdir -p ~/.ssh
# 查找可用的SSH密钥(优先用 secret 里专门为 preview 配置的 key
key_path=""
if [ -n "${PREVIEW_SSH_KEY:-}" ]; then
key_path="$HOME/.ssh/id_ed25519"
printf '%s\n' "$PREVIEW_SSH_KEY" > "$key_path"
chmod 600 "$key_path"
echo "Using key from PREVIEW_SSH_KEY secret"
elif [ -f /root/.ssh/xiaoxia_runtime_builder ]; then
key_path="/root/.ssh/xiaoxia_runtime_builder"
echo "Using key: $key_path (builder key)"
elif [ -f "$HOME/.ssh/xiaoxia_runtime_builder" ]; then
key_path="$HOME/.ssh/xiaoxia_runtime_builder"
echo "Using key: $key_path (home key)"
else
echo "ERROR: No SSH key available"
ls -la ~/.ssh/ 2>/dev/null || true
ls -la /root/.ssh/ 2>/dev/null || true
exit 1
fi
ssh-keyscan -p "$preview_port" -H "$preview_host" >> ~/.ssh/known_hosts 2>/dev/null
echo "SSH keyscan done"
# 测试SSH连接
ssh -p "$preview_port" -i "$key_path" -o StrictHostKeyChecking=no "${preview_user}@${preview_host}" "echo SSH_CONNECTION_OK && hostname"
echo "SSH connection verified"
# 检查目录是否存在
DIR_EXISTS=$(ssh -p "$preview_port" -i "$key_path" -o StrictHostKeyChecking=no "${preview_user}@${preview_host}" \
"if [ -d '${preview_dir}' ]; then echo 'yes'; else echo 'no'; fi")
if [ "$DIR_EXISTS" = "yes" ]; then
echo "Removing preview directory: ${preview_dir}"
ssh -p "$preview_port" -i "$key_path" -o StrictHostKeyChecking=no "${preview_user}@${preview_host}" \
"rm -rf ${preview_dir} && echo 'Preview directory removed successfully'"
echo "Cleanup completed: ${preview_dir}"
else
echo "Preview directory does not exist: ${preview_dir}, nothing to clean up"
fi
- name: Comment cleanup notice on PR
if: success()
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
set -eu
# 从event payload读取PR号(最可靠)
if [ -n "${GITHUB_EVENT_PATH:-}" ] && [ -f "$GITHUB_EVENT_PATH" ]; then
PR_NUMBER=$(python3 -c "import json,sys; print(json.load(sys.stdin).get('number',''))" < "$GITHUB_EVENT_PATH")
else
PR_NUMBER=$(echo "$GITHUB_REF" | sed -n 's|refs/pull/\([0-9]*\)/.*|\1|p')
fi
export PR_NUMBER
COMMENT_BODY=$(python3 scripts/ci/preview_comment.py cleanup)
API_URL="${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/issues/${PR_NUMBER}/comments"
curl -s -X POST \
-H "Authorization: token ${GITHUB_TOKEN}" \
-H "Content-Type: application/json" \
-d "$COMMENT_BODY" \
"$API_URL" \
> /dev/null
echo "Cleanup comment posted"
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
+296
View File
@@ -0,0 +1,296 @@
name: Preview Deploy
on:
pull_request:
types:
- opened
- synchronize
- reopened
branches:
- main
- develop
workflow_dispatch:
inputs:
reason:
description: "触发原因"
required: false
default: "手动触发 - 预览环境补跑"
permissions:
contents: read
pull-requests: write
concurrency:
group: preview-deploy-${{ gitea.ref }}
cancel-in-progress: true
jobs:
deploy-preview:
name: Deploy Preview Environment
runs-on: runtime-builder
timeout-minutes: 20
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
set -eu
python3 - <<'PY'
import io, os, tarfile, time, urllib.request, urllib.error
url = f"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz"
request = urllib.request.Request(url, headers={"Authorization": f"token {os.environ['GITHUB_TOKEN']}"})
last_err = None
for attempt in range(5):
try:
with urllib.request.urlopen(request, timeout=120) as response:
archive = response.read()
break
except urllib.error.HTTPError as e:
last_err = e
if e.code >= 500 and attempt < 4:
wait = 2 ** attempt
print(f"Checkout HTTP {e.code}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
except Exception as e:
last_err = e
if attempt < 4:
wait = 2 ** attempt
print(f"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
else:
raise last_err
with tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:
root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'
for member in tar.getmembers():
name = member.name
if name == root_prefix[:-1]:
continue
if name.startswith(root_prefix):
member.name = name[len(root_prefix):]
if member.name:
tar.extract(member, '.')
PY
- name: Record job start time
shell: sh
run: |
set -eu
echo "JOB_START_TIME=$(date +%s)" >> $GITHUB_ENV
echo "Job started at $(date)"
- name: Extract PR number
shell: sh
run: |
set -eu
PR_NUMBER=$(echo "$GITHUB_REF" | sed 's|refs/pull/||; s|/.*||')
echo "PR_NUMBER=$PR_NUMBER" >> $GITHUB_ENV
echo "PR number: $PR_NUMBER"
echo "PREVIEW_URL=https://pr-${PR_NUMBER}.preview.xiaoxiajianji.com" >> $GITHUB_ENV
echo "Preview URL: https://pr-${PR_NUMBER}.preview.xiaoxiajianji.com"
- name: Build frontend
shell: sh
run: |
set -eu
NPM_CACHE_VOLUME="xiaoxia-npm-cache"
if ! docker volume inspect "$NPM_CACHE_VOLUME" >/dev/null 2>&1; then
docker volume create "$NPM_CACHE_VOLUME" >/dev/null
echo "Created npm cache volume: $NPM_CACHE_VOLUME"
fi
docker run --rm \
-v "$PWD:/workspace" \
-v "$NPM_CACHE_VOLUME:/workspace/apps/web/node_modules" \
-w /workspace/apps/web \
-e VITE_API_URL=https://staging-api.xiaoxiajianji.com \
docker.m.daocloud.io/library/node:20 \
sh -lc '
PACKAGE_LOCK_HASH=$(md5sum package-lock.json 2>/dev/null | cut -d" " -f1)
CACHE_HASH_FILE="node_modules/.package-lock-hash"
CACHE_VALID=false
if [ -f "$CACHE_HASH_FILE" ] && [ "$(cat "$CACHE_HASH_FILE")" = "$PACKAGE_LOCK_HASH" ] && [ -x "node_modules/.bin/vite" ] && [ -x "node_modules/.bin/tsc" ]; then
CACHE_VALID=true
echo "Cache hit: dependencies valid, skipping npm install"
fi
if [ "$CACHE_VALID" = "false" ]; then
echo "Cache miss or invalid: running npm install..."
if ! npm install --include=dev; then
echo "npm install failed, cleaning node_modules and retrying..."
rm -rf node_modules
mkdir -p node_modules
npm install --include=dev
fi
echo "$PACKAGE_LOCK_HASH" > "$CACHE_HASH_FILE"
echo "Dependencies installed, cache updated"
fi
echo "Running TypeScript check..."
npx --no-install tsc
echo "Running Vite build..."
npx --no-install vite build
echo "Build completed successfully"
ls -la dist/
'
- name: Install SSH client and rsync
shell: sh
run: |
set -eu
if command -v apk >/dev/null 2>&1; then
apk add --no-cache openssh-client rsync >/dev/null 2>&1
elif command -v apt-get >/dev/null 2>&1; then
apt-get update -qq && apt-get install -y -qq openssh-client rsync >/dev/null 2>&1
elif command -v yum >/dev/null 2>&1; then
yum install -y openssh-clients rsync >/dev/null 2>&1
else
echo "ERROR: No package manager found"
exit 1
fi
echo "openssh-client and rsync installed"
- name: Deploy preview to server
shell: sh
env:
PREVIEW_SSH_HOST: ${{ secrets.PREVIEW_SSH_HOST }}
PREVIEW_SSH_USER: ${{ secrets.PREVIEW_SSH_USER }}
PREVIEW_SSH_PORT: ${{ secrets.PREVIEW_SSH_PORT }}
PREVIEW_SSH_KEY: ${{ secrets.PREVIEW_SSH_KEY }}
run: |
set -eux
preview_host="${PREVIEW_SSH_HOST:-47.98.113.167}"
preview_user="${PREVIEW_SSH_USER:-root}"
preview_port="${PREVIEW_SSH_PORT:-22222}"
preview_dir="/var/www/preview/pr-${PR_NUMBER}"
mkdir -p ~/.ssh
# 查找可用的SSH密钥(优先用 secret 里专门为 preview 配置的 key
key_path=""
if [ -n "${PREVIEW_SSH_KEY:-}" ]; then
key_path="$HOME/.ssh/id_ed25519"
printf '%s\n' "$PREVIEW_SSH_KEY" > "$key_path"
chmod 600 "$key_path"
echo "Using key from PREVIEW_SSH_KEY secret"
elif [ -f /root/.ssh/xiaoxia_runtime_builder ]; then
key_path="/root/.ssh/xiaoxia_runtime_builder"
echo "Using key: $key_path (builder key)"
elif [ -f "$HOME/.ssh/xiaoxia_runtime_builder" ]; then
key_path="$HOME/.ssh/xiaoxia_runtime_builder"
echo "Using key: $key_path (home key)"
else
echo "ERROR: No SSH key available"
ls -la ~/.ssh/ 2>/dev/null || true
ls -la /root/.ssh/ 2>/dev/null || true
exit 1
fi
# SSH密钥完整性自检
if ! ssh-keygen -y -f "$key_path" > /dev/null 2>&1; then
echo "ERROR: SSH密钥损坏(private key contents do not match public"
echo "请检查 PREVIEW_SSH_KEY secret 中的私钥是否完整正确"
echo "私钥文件大小: $(wc -c < "$key_path") 字节"
head -2 "$key_path"
exit 1
fi
echo "SSH key integrity check passed"
ssh-keyscan -p "$preview_port" -H "$preview_host" >> ~/.ssh/known_hosts 2>/dev/null
echo "SSH keyscan done"
# 测试SSH连接
ssh -p "$preview_port" -i "$key_path" -o StrictHostKeyChecking=no "${preview_user}@${preview_host}" "echo SSH_CONNECTION_OK && hostname"
echo "SSH connection verified"
# 创建预览目录并上传文件
ssh -p "$preview_port" -i "$key_path" -o StrictHostKeyChecking=no "${preview_user}@${preview_host}" \
"mkdir -p ${preview_dir} && echo 'Preview directory created: ${preview_dir}'"
# 使用rsync上传dist目录内容
rsync -avz --delete -e "ssh -p ${preview_port} -i ${key_path} -o StrictHostKeyChecking=no" \
apps/web/dist/ \
"${preview_user}@${preview_host}:${preview_dir}/"
echo "Preview deployed to: ${preview_dir}"
echo "Preview URL: https://pr-${PR_NUMBER}.preview.xiaoxiajianji.com"
- name: Comment preview link on PR
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
set -eu
PR_NUMBER=$(echo "$GITHUB_REF" | sed 's|refs/pull/||; s|/.*||')
PREVIEW_URL="https://pr-${PR_NUMBER}.preview.xiaoxiajianji.com"
export PR_NUMBER PREVIEW_URL
COMMENT_BODY=$(python3 scripts/ci/preview_comment.py deploy)
API_URL="${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/issues/${PR_NUMBER}/comments"
EXISTING_COMMENT_ID=$(curl -s -H "Authorization: token ${GITHUB_TOKEN}" "$API_URL" | python3 -c "
import sys, json
try:
for c in json.load(sys.stdin):
if '预览环境已部署' in c.get('body', ''):
print(c['id'])
break
except Exception:
pass
")
if [ -n "$EXISTING_COMMENT_ID" ]; then
curl -s -X PATCH \
-H "Authorization: token ${GITHUB_TOKEN}" \
-H "Content-Type: application/json" \
-d "$COMMENT_BODY" \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/issues/comments/${EXISTING_COMMENT_ID}" \
> /dev/null
echo "Comment updated"
else
curl -s -X POST \
-H "Authorization: token ${GITHUB_TOKEN}" \
-H "Content-Type: application/json" \
-d "$COMMENT_BODY" \
"$API_URL" \
> /dev/null
echo "Comment posted"
fi
- name: Job duration summary
if: always()
shell: sh
run: |
set +eu
if [ -n "$JOB_START_TIME" ]; then
END_TIME=$(date +%s)
DURATION=$((END_TIME - JOB_START_TIME))
MINS=$((DURATION / 60))
SECS=$((DURATION % 60))
echo "JOB_DURATION_SECONDS=$DURATION" >> $GITHUB_ENV
echo "=== Job Duration: ${MINS}m${SECS}s ==="
else
echo "JOB_DURATION_SECONDS=0" >> $GITHUB_ENV
echo "=== Job Duration: unknown ==="
fi
- name: Notify on failure
continue-on-error: true
if: failure()
shell: sh
env:
CI_NOTIFY_WEBHOOK: ${{ secrets.CI_NOTIFY_WEBHOOK }}
run: |
set +e
NOTIFY_MODE=failure JOB_NAME="Deploy Preview Environment" python3 scripts/ci_notify.py
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
-69
View File
@@ -1,69 +0,0 @@
name: Test SSH Secret
on:
push:
branches: [develop]
paths:
- '.gitea/workflows/test-ssh-secret.yml'
jobs:
test-ssh:
runs-on: ubuntu-22.04
steps:
- name: Install SSH client
run: |
which ssh || (apt-get update && apt-get install -y openssh-client)
ssh -V
- name: Debug environment
run: |
echo "=== Environment ==="
echo "Runner hostname: $(hostname)"
echo "Runner IP: $(hostname -i || echo 'unknown')"
echo "Current user: $(whoami)"
echo "=== Secrets check ==="
if [ -n "$STAGING_SSH_HOST" ]; then
echo "STAGING_SSH_HOST: [SET] value_length=${#STAGING_SSH_HOST}"
else
echo "STAGING_SSH_HOST: [EMPTY]"
fi
if [ -n "$STAGING_SSH_USER" ]; then
echo "STAGING_SSH_USER: [SET] value_length=${#STAGING_SSH_USER}"
else
echo "STAGING_SSH_USER: [EMPTY]"
fi
if [ -n "$STAGING_SSH_KEY" ]; then
echo "STAGING_SSH_KEY: [SET] value_length=${#STAGING_SSH_KEY}"
else
echo "STAGING_SSH_KEY: [EMPTY]"
fi
env:
STAGING_SSH_HOST: ${{ secrets.STAGING_SSH_HOST }}
STAGING_SSH_USER: ${{ secrets.STAGING_SSH_USER }}
STAGING_SSH_KEY: ${{ secrets.STAGING_SSH_KEY }}
- name: Setup SSH key
run: |
mkdir -p ~/.ssh
chmod 700 ~/.ssh
echo "$STAGING_SSH_KEY" > ~/.ssh/id_ed25519
chmod 600 ~/.ssh/id_ed25519
ssh-keygen -y -f ~/.ssh/id_ed25519 > ~/.ssh/id_ed25519.pub 2>/dev/null || echo "No public key generated"
echo "=== SSH Key fingerprint ==="
ssh-keygen -lf ~/.ssh/id_ed25519 || echo "Key fingerprint failed"
env:
STAGING_SSH_KEY: ${{ secrets.STAGING_SSH_KEY }}
- name: Test SSH connection
run: |
echo "Attempting SSH connection to $STAGING_SSH_HOST..."
ssh -i ~/.ssh/id_ed25519 \
-o StrictHostKeyChecking=no \
-o UserKnownHostsFile=/dev/null \
-o ConnectTimeout=10 \
-o BatchMode=yes \
-v \
$STAGING_SSH_USER@$STAGING_SSH_HOST "echo 'SSH_CONNECTION_SUCCESS' && hostname && whoami"
echo "=== SSH Test Complete ==="
env:
STAGING_SSH_HOST: ${{ secrets.STAGING_SSH_HOST }}
STAGING_SSH_USER: ${{ secrets.STAGING_SSH_USER }}
-163
View File
@@ -1,163 +0,0 @@
name: Tests
on:
pull_request:
branches: [ main ]
jobs:
test:
runs-on: runtime-builder
steps:
- name: Checkout code
shell: sh
run: |
set -eu
python - <<'PY'
import io
import os
import tarfile
import time
import urllib.error
import urllib.request
url = f"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz"
request = urllib.request.Request(url, headers={"Authorization": f"token {os.environ['GITHUB_TOKEN']}"})
# Retry up to 5 times with backoff for transient 5xx errors
last_err = None
for attempt in range(5):
try:
with urllib.request.urlopen(request, timeout=120) as response:
archive = response.read()
break
except urllib.error.HTTPError as e:
last_err = e
if e.code >= 500 and attempt < 4:
wait = 2 ** attempt
print(f"Checkout HTTP {e.code}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
except Exception as e:
last_err = e
if attempt < 4:
wait = 2 ** attempt
print(f"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
else:
raise last_err
with tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:
root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'
for member in tar.getmembers():
name = member.name
if name == root_prefix[:-1]:
continue
if name.startswith(root_prefix):
member.name = name[len(root_prefix):]
if member.name:
tar.extract(member, '.')
PY
- name: Show Python version
shell: sh
run: |
set -eu
python --version
python -m pip --version
- name: Install dependencies
shell: sh
run: |
set -eu
python -m pip install --upgrade pip -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
python -m pip install -r requirements.txt -r requirements-dev.txt -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
- name: Run unit tests
shell: sh
run: |
set -eu
PYTHONPATH="$PWD/apps/api:$PWD" python -m pytest tests/unit -q
- name: Run integration tests
shell: sh
run: |
set -eu
PYTHONPATH="$PWD/apps/api:$PWD" python -m pytest tests/integration -q --timeout=60 -x
lint:
runs-on: runtime-builder
steps:
- name: Checkout code
shell: sh
run: |
set -eu
python - <<'PY'
import io
import os
import tarfile
import time
import urllib.error
import urllib.request
url = f"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz"
request = urllib.request.Request(url, headers={"Authorization": f"token {os.environ['GITHUB_TOKEN']}"})
# Retry up to 5 times with backoff for transient 5xx errors
last_err = None
for attempt in range(5):
try:
with urllib.request.urlopen(request, timeout=120) as response:
archive = response.read()
break
except urllib.error.HTTPError as e:
last_err = e
if e.code >= 500 and attempt < 4:
wait = 2 ** attempt
print(f"Checkout HTTP {e.code}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
except Exception as e:
last_err = e
if attempt < 4:
wait = 2 ** attempt
print(f"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
else:
raise last_err
with tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:
root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'
for member in tar.getmembers():
name = member.name
if name == root_prefix[:-1]:
continue
if name.startswith(root_prefix):
member.name = name[len(root_prefix):]
if member.name:
tar.extract(member, '.')
PY
- name: Install dependencies
shell: sh
run: |
set -eu
python -m pip install --upgrade pip -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
python -m pip install -r requirements.txt -r requirements-dev.txt -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
- name: Run Black (check only)
shell: sh
run: |
set -eu
python -m black --check alembic apps packages tests scripts
- name: Run Flake8
shell: sh
run: |
set -eu
python -m flake8 apps packages tests --count --statistics
+103
View File
@@ -0,0 +1,103 @@
name: Worker Base Image Build
on:
push:
branches:
- develop
- main
paths:
- 'requirements-base.txt'
- 'requirements-worker.txt'
- 'infra/docker/worker-base-builder.Dockerfile'
- 'infra/docker/worker-base-runtime.Dockerfile'
workflow_dispatch: # 支持手动触发
jobs:
build-worker-base:
name: Build Worker Base Images
runs-on: runtime-builder
timeout-minutes: 30
strategy:
fail-fast: false
matrix:
include:
- name: builder
dockerfile: infra/docker/worker-base-builder.Dockerfile
image_name: worker-base-builder
cache_name: worker-base-builder-cache
- name: runtime
dockerfile: infra/docker/worker-base-runtime.Dockerfile
image_name: worker-base-runtime
cache_name: worker-base-runtime-cache
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sH "Authorization: token $GITHUB_TOKEN" "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" | bash
- name: Docker login to Registry
shell: sh
env:
ACR_USERNAME: ${{ secrets.ACR_USERNAME }}
ACR_PASSWORD: ${{ secrets.ACR_PASSWORD }}
GITEA_REGISTRY_USER: xiaoxia
GITEA_REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
set -eu
for i in 1 2 3; do
echo "=== Docker login 尝试 $i/3 ==="
if printf '%s' "${ACR_PASSWORD}" | docker login xiaoxia-registry.cn-hangzhou.cr.aliyuncs.com -u "${ACR_USERNAME}" --password-stdin && docker login git.xiaoxiajianji.com -u "${GITEA_REGISTRY_USER}" -p "${GITEA_REGISTRY_TOKEN}"; then
echo "✅ Docker login successful"
break
fi
echo "❌ Docker login 失败(尝试 $i/3),5s 后重试..."
sleep 5
done
- name: Setup buildx builder
shell: sh
run: |
set -eu
BUILDER_NAME="ci-builder-${GITHUB_RUN_ID}-${{ matrix.name }}"
if ! docker buildx inspect "$BUILDER_NAME" > /dev/null 2>&1; then
docker buildx create --use --name "$BUILDER_NAME" --driver docker-container
echo "Created $BUILDER_NAME"
else
docker buildx use "$BUILDER_NAME"
echo "Using existing $BUILDER_NAME"
fi
docker buildx inspect --bootstrap
- name: Build and push base image
shell: sh
run: |
set -eu
REGISTRY="xiaoxia-registry.cn-hangzhou.cr.aliyuncs.com/xiaoxiakeji"
IMAGE_TAG="${REGISTRY}/${{ matrix.image_name }}:latest"
SAFE_REF_NAME=$(echo "${GITHUB_REF_NAME}" | tr '/' '-')
CACHE_REF="${REGISTRY}/${{ matrix.cache_name }}:${SAFE_REF_NAME}"
echo "=== Building ${{ matrix.name }} base image ==="
echo "Image: ${IMAGE_TAG}"
echo "Cache: ${CACHE_REF}"
# 用通用构建脚本
bash scripts/ci/docker_build_push.sh ${{ matrix.dockerfile }} "${IMAGE_TAG}" "${CACHE_REF}"
# 同时推送到 Gitea Packages 作为备份(可选)
GITEA_IMAGE="git.xiaoxiajianji.com/xiaoxia-saas/${{ matrix.image_name }}:latest"
docker tag "${IMAGE_TAG}" "${GITEA_IMAGE}"
docker push "${GITEA_IMAGE}" || echo "Gitea Packages push failed (non-fatal)"
echo ""
echo "✅ ${{ matrix.name }} base image built and pushed"
- name: Cleanup buildx builder
if: always()
shell: sh
run: |
docker buildx rm "ci-builder-${GITHUB_RUN_ID}-${{ matrix.name }}" 2>/dev/null || true
docker buildx prune -f 2>/dev/null || true
echo "Builder cleanup done"
@@ -0,0 +1,33 @@
"""Add file_hash to assets and ingest_jobs
Revision ID: 031
Revises: 030
Create Date: 2026-07-07
为素材去重检测功能添加 file_hash 字段。
"""
import sqlalchemy as sa
from alembic import op
revision = "031"
down_revision = "030"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column("assets", sa.Column("file_hash", sa.String(64), nullable=True))
op.create_index(op.f("ix_assets_file_hash"), "assets", ["file_hash"])
op.add_column("ingest_jobs", sa.Column("file_hash", sa.String(64), nullable=True))
op.create_index(op.f("ix_ingest_jobs_file_hash"), "ingest_jobs", ["file_hash"])
def downgrade() -> None:
op.drop_index(op.f("ix_ingest_jobs_file_hash"), table_name="ingest_jobs")
op.drop_column("ingest_jobs", "file_hash")
op.drop_index(op.f("ix_assets_file_hash"), table_name="assets")
op.drop_column("assets", "file_hash")
@@ -0,0 +1,28 @@
"""Add asset_select_mode to generation_tasks
Revision ID: 032
Revises: 031
Create Date: 2026-07-07
素材库自动匹配功能:为 generation_tasks 表添加 asset_select_mode 字段。
"""
import sqlalchemy as sa
from alembic import op
revision = "032"
down_revision = "031"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column(
"generation_tasks",
sa.Column("asset_select_mode", sa.String(20), nullable=False, server_default=""),
)
def downgrade() -> None:
op.drop_column("generation_tasks", "asset_select_mode")
@@ -0,0 +1,31 @@
"""Add batch_id to generation_tasks
Revision ID: 033
Revises: 032
Create Date: 2026-07-07
视频查重功能:为 generation_tasks 表添加 batch_id 字段,
用于关联同一次批量生成请求中的多个任务。
"""
import sqlalchemy as sa
from alembic import op
revision = "033"
down_revision = "032"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column(
"generation_tasks",
sa.Column("batch_id", sa.String(32), nullable=False, server_default=""),
)
op.create_index(op.f("ix_generation_tasks_batch_id"), "generation_tasks", ["batch_id"])
def downgrade() -> None:
op.drop_index(op.f("ix_generation_tasks_batch_id"), table_name="generation_tasks")
op.drop_column("generation_tasks", "batch_id")
+40 -1
View File
@@ -19,6 +19,7 @@ from app.core.celery_app import celery_app
from app.core.storage import OSSStorageService, get_storage_service
from app.dependencies import (
get_asset_library_repository,
get_asset_repository,
get_ingest_job_repository,
get_project_repository,
)
@@ -42,20 +43,33 @@ DEFAULT_CHUNK_SIZE = 5 * 1024 * 1024 # 5MB
MAX_FILE_SIZE = 2 * 1024 * 1024 * 1024 # 2GB
CHUNK_EXPIRY_HOURS = 24
# Allowed file types (consistent with existing upload.py)
# Allowed file types — must stay in sync with upload.py ALLOWED_MIME_TYPES
ALLOWED_MIME_TYPES = {
# Images
"image/jpeg",
"image/png",
"image/gif",
"image/webp",
"image/bmp",
"image/tiff",
"image/svg+xml",
# Video
"video/mp4",
"video/quicktime",
"video/mpeg",
"video/x-msvideo",
"video/webm",
"video/x-matroska",
"video/3gpp",
# Audio
"audio/mpeg",
"audio/wav",
"audio/ogg",
"audio/mp3",
"audio/flac",
"audio/aac",
"audio/x-m4a",
"audio/webm",
}
# Chunk storage root directory
@@ -360,6 +374,7 @@ async def complete_chunked_upload(
authenticated_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
asset_library_repository: Any = Depends(get_asset_library_repository),
asset_repository: Any = Depends(get_asset_repository),
ingest_job_repository: Any = Depends(get_ingest_job_repository),
storage_service: OSSStorageService = Depends(get_storage_service),
) -> ChunkedUploadCompleteResponse:
@@ -423,6 +438,29 @@ async def complete_chunked_upload(
content_type=meta["content_type"],
)
# ── 素材去重检测:同素材库 + 同 file_hash 视为重复 ──
if request.file_hash:
existing = asset_repository.find_by_library_and_file_hash(
library_id=request.library_id,
file_hash=request.file_hash,
)
if existing is not None:
logger.info(
"素材去重命中(chunked): library=%s hash=%s existing_asset=%s",
request.library_id,
request.file_hash,
existing.id,
)
meta["status"] = "completed"
_save_upload_meta(upload_id, meta)
return ChunkedUploadCompleteResponse(
storage_key=storage_key,
ingest_job_id="",
url=file_url,
duplicated=True,
asset_id=existing.id,
)
# Create ingest job
use_case = SubmitIngestJobUseCase(ingest_job_repository)
job = use_case.execute(
@@ -430,6 +468,7 @@ async def complete_chunked_upload(
project_id=meta["project_id"],
library_id=meta["library_id"],
storage_key=storage_key,
file_hash=request.file_hash,
)
)
celery_app.send_task("worker.ingest_asset", args=[job.id])
+86 -17
View File
@@ -1,3 +1,5 @@
import random
import uuid
from typing import Any
from app.auth import AuthenticatedUser, get_current_user
@@ -14,6 +16,7 @@ from app.schemas.generated_video import (
ListGeneratedVideosResponse,
)
from app.schemas.generation_task import (
BatchGenerationTaskResponse,
CreateGenerationTaskRequest,
GenerationTaskResponse,
ListGenerationTasksResponse,
@@ -51,6 +54,8 @@ def _to_generation_task_response(task) -> GenerationTaskResponse:
title_ids=task.title_ids,
voice_ids=task.voice_ids,
source_edit_plan_id=task.source_edit_plan_id or "",
asset_select_mode=getattr(task, "asset_select_mode", ""),
batch_id=getattr(task, "batch_id", ""),
status=task.status,
progress=task.progress,
result_count=task.result_count,
@@ -85,6 +90,49 @@ def _ensure_library_has_ready_video_assets(assets) -> None:
)
def _select_assets_from_library(
assets: list,
mode: str,
count: int,
) -> list[str]:
"""根据选取模式从素材库中选取 ready 状态的视频素材 ID。
Args:
assets: 素材库中所有素材(Asset 实体列表)
mode: 选取模式 — all=全部, random=随机, smart=按质量评分
count: 选取数量,0 表示全部(仅 random/smart 模式有效)
Returns:
选中的素材 ID 列表
"""
ready_video_assets = [a for a in assets if a.status.value == "ready" and a.mime_type.startswith("video")]
if not ready_video_assets:
return []
if mode == "random":
selected = (
ready_video_assets if count <= 0 else random.sample(ready_video_assets, min(count, len(ready_video_assets)))
)
return [a.id for a in selected]
if mode == "smart":
# 按质量分降序排列(质量分高的优先),质量分相同时按时长降序
sorted_assets = sorted(
ready_video_assets,
key=lambda a: (
a.quality_score if a.quality_score is not None else 0.0,
a.duration if a.duration is not None else 0.0,
),
reverse=True,
)
selected = sorted_assets if count <= 0 else sorted_assets[:count]
return [a.id for a in selected]
# 默认 all 模式:返回全部 ready 视频素材
return [a.id for a in ready_video_assets]
def _resolve_project_and_library(
request: CreateGenerationTaskRequest,
project_repository: Any,
@@ -122,7 +170,7 @@ def _resolve_project_and_library(
return project_id, asset_library_id
@router.post("/tasks", response_model=GenerationTaskResponse)
@router.post("/tasks", response_model=BatchGenerationTaskResponse)
def create_generation_task(
request: CreateGenerationTaskRequest,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
@@ -130,12 +178,13 @@ def create_generation_task(
project_repository: Any = Depends(get_project_repository),
asset_library_repository: Any = Depends(get_asset_library_repository),
asset_repository: Any = Depends(get_asset_repository),
) -> GenerationTaskResponse:
) -> BatchGenerationTaskResponse:
project_id, asset_library_id = _resolve_project_and_library(
request, project_repository, asset_library_repository, asset_repository, authenticated_user
)
# asset_library 存在性校验(仅在提供了 asset_library_id 时)
resolved_asset_ids: list[str] = list(request.asset_ids)
if asset_library_id:
library = asset_library_repository.get(asset_library_id)
if library is None or (project_id and library.project_id != project_id):
@@ -144,23 +193,42 @@ def create_generation_task(
assets = asset_repository.find_by_library(asset_library_id)
_ensure_library_has_ready_video_assets(assets)
# 素材库自动匹配:当未显式指定 asset_ids 时,按模式自动选取
if not resolved_asset_ids:
resolved_asset_ids = _select_assets_from_library(
assets,
mode=request.asset_select_mode,
count=request.asset_select_count,
)
use_case = CreateGenerationTaskUseCase(generation_task_repository)
task = use_case.execute(
CreateGenerationTaskCommand(
project_id=project_id,
asset_library_id=asset_library_id,
strategy_id=request.strategy_id,
voice_library_id=request.voice_library_id,
template_id=request.template_id,
asset_ids=request.asset_ids,
title_ids=request.title_ids,
voice_ids=request.voice_ids,
created_by_user_id=authenticated_user.user.id,
source_edit_plan_id=request.source_edit_plan_id,
count = request.count
created_tasks = []
# 同批次任务共享 batch_id,用于视频查重时批次内比对
batch_id = uuid.uuid4().hex if count > 1 else ""
for _ in range(count):
task = use_case.execute(
CreateGenerationTaskCommand(
project_id=project_id,
asset_library_id=asset_library_id,
strategy_id=request.strategy_id,
voice_library_id=request.voice_library_id,
template_id=request.template_id,
asset_ids=resolved_asset_ids,
title_ids=request.title_ids,
voice_ids=request.voice_ids,
created_by_user_id=authenticated_user.user.id,
source_edit_plan_id=request.source_edit_plan_id,
asset_select_mode=request.asset_select_mode,
batch_id=batch_id,
)
)
)
celery_app.send_task("worker.generate_video", args=[task.id])
return _to_generation_task_response(task)
celery_app.send_task("worker.generate_video", args=[task.id])
created_tasks.append(task)
items = [_to_generation_task_response(t) for t in created_tasks]
return BatchGenerationTaskResponse(items=items, total=len(items))
@router.get("/tasks", response_model=ListGenerationTasksResponse)
@@ -237,6 +305,7 @@ def retry_generation_task(
voice_ids=task.voice_ids,
created_by_user_id=authenticated_user.user.id,
source_edit_plan_id=task.source_edit_plan_id or "",
asset_select_mode=getattr(task, "asset_select_mode", ""),
)
)
celery_app.send_task("worker.generate_video", args=[retried.id])
+49
View File
@@ -8,6 +8,7 @@ from app.core.celery_app import celery_app
from app.core.storage import OSSStorageService, get_storage_service
from app.dependencies import (
get_asset_library_repository,
get_asset_repository,
get_ingest_job_repository,
get_project_repository,
)
@@ -99,6 +100,7 @@ def _submit_ingest_job(
library_id: str,
storage_key: str,
ingest_job_repository: Any,
file_hash: str = "",
) -> Any:
use_case = SubmitIngestJobUseCase(ingest_job_repository)
job = use_case.execute(
@@ -106,6 +108,7 @@ def _submit_ingest_job(
project_id=project_id,
library_id=library_id,
storage_key=storage_key,
file_hash=file_hash,
)
)
celery_app.send_task("worker.ingest_asset", args=[job.id])
@@ -176,6 +179,7 @@ async def complete_direct_upload(
ingest_job_repository: Any = Depends(get_ingest_job_repository),
project_repository: Any = Depends(get_project_repository),
asset_library_repository: Any = Depends(get_asset_library_repository),
asset_repository: Any = Depends(get_asset_repository),
storage_service: OSSStorageService = Depends(get_storage_service),
) -> DirectUploadCompleteResponse:
"""确认浏览器直传完成并创建导入任务。"""
@@ -199,11 +203,32 @@ async def complete_direct_upload(
if not file_exists:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Uploaded file not found")
# ── 素材去重检测:同素材库 + 同 file_hash 视为重复 ──
if request.file_hash:
existing = asset_repository.find_by_library_and_file_hash(
library_id=request.library_id,
file_hash=request.file_hash,
)
if existing is not None:
logger.info(
"素材去重命中: library=%s hash=%s existing_asset=%s",
request.library_id,
request.file_hash,
existing.id,
)
return DirectUploadCompleteResponse(
storage_key=normalized_key,
ingest_job_id="",
duplicated=True,
asset_id=existing.id,
)
job = _submit_ingest_job(
project_id=request.project_id,
library_id=request.library_id,
storage_key=normalized_key,
ingest_job_repository=ingest_job_repository,
file_hash=request.file_hash,
)
return DirectUploadCompleteResponse(storage_key=normalized_key, ingest_job_id=job.id)
@@ -218,15 +243,38 @@ async def upload_asset(
project_id: str = Form(..., min_length=1, description="项目 ID"),
library_id: str = Form(..., min_length=1, description="素材库 ID"),
file: UploadFile = File(..., description="要上传的文件(视频、音频、图片等)"),
file_hash: str = Form(default="", description="文件 MD5 哈希,用于去重检测"),
authenticated_user: AuthenticatedUser = Depends(get_current_user),
ingest_job_repository: Any = Depends(get_ingest_job_repository),
project_repository: Any = Depends(get_project_repository),
asset_library_repository: Any = Depends(get_asset_library_repository),
asset_repository: Any = Depends(get_asset_repository),
storage_service: OSSStorageService = Depends(get_storage_service),
) -> UploadAssetResponse:
"""上传素材文件并触发导入流水线。"""
_require_project_and_library(project_id, library_id, project_repository, asset_library_repository)
# ── 素材去重检测:上传前检查同素材库 + 同 file_hash ──
if file_hash:
existing = asset_repository.find_by_library_and_file_hash(
library_id=library_id,
file_hash=file_hash,
)
if existing is not None:
logger.info(
"素材去重命中(multipart): library=%s hash=%s existing_asset=%s",
library_id,
file_hash,
existing.id,
)
return UploadAssetResponse(
storage_key=existing.storage_key,
ingest_job_id="",
url="",
duplicated=True,
asset_id=existing.id,
)
# P2-5: 服务端验证 MIME 类型
validated_content_type = _validate_mime_type(file.content_type)
@@ -255,6 +303,7 @@ async def upload_asset(
library_id=library_id,
storage_key=storage_key,
ingest_job_repository=ingest_job_repository,
file_hash=file_hash,
)
return UploadAssetResponse(
+1 -1
View File
@@ -10,7 +10,7 @@ from __future__ import annotations
from app.auth import AuthenticatedUser
from app.auth import get_current_user as get_authenticated_user
from app.dependencies import get_user_repository
from fastapi import Depends
from fastapi import Depends, HTTPException
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
from packages.domain.entities import User
+3
View File
@@ -36,9 +36,12 @@ class ChunkedUploadStatusResponse(BaseModel):
class ChunkedUploadCompleteRequest(BaseModel):
project_id: str = Field(..., min_length=1, description="Project ID")
library_id: str = Field(..., min_length=1, description="Asset library ID")
file_hash: str = Field(default="", max_length=64, description="文件 MD5 哈希,用于去重检测")
class ChunkedUploadCompleteResponse(BaseModel):
storage_key: str = Field(..., description="Storage key")
ingest_job_id: str = Field(..., description="Ingest job ID")
url: str = Field(..., description="File URL")
duplicated: bool = Field(default=False, description="是否为重复素材(命中去重)")
asset_id: str = Field(default="", description="重复素材的 asset_idduplicated=true 时返回)")
+19
View File
@@ -21,6 +21,16 @@ class CreateGenerationTaskRequest(BaseModel):
voice_ids: list[str] = Field(default_factory=list)
# ── 来源剪辑计划 ──
source_edit_plan_id: str = ""
# ── 批量生成 ──
count: int = Field(default=1, ge=1, le=50, description="批量生成数量,默认1,最大50")
# ── 素材库自动匹配 ──
asset_select_mode: str = Field(
default="all",
description="素材选取模式:all=全部ready视频, random=随机选取, smart=智能匹配(按质量/时长评分)",
)
asset_select_count: int = Field(
default=0, ge=0, le=100, description="选取数量,0表示全部(仅 random/smart 模式有效)"
)
@model_validator(mode="after")
def _check_at_least_one_mode(self) -> "CreateGenerationTaskRequest":
@@ -46,12 +56,21 @@ class GenerationTaskResponse(BaseModel):
title_ids: list[str] = Field(default_factory=list)
voice_ids: list[str] = Field(default_factory=list)
source_edit_plan_id: str = ""
asset_select_mode: str = ""
batch_id: str = ""
status: str
progress: float
result_count: int
error_message: str
class BatchGenerationTaskResponse(BaseModel):
"""批量生成任务响应。"""
items: list[GenerationTaskResponse]
total: int
class ListGenerationTasksResponse(BaseModel):
"""用户级生成任务列表响应(跨 project)。"""
+13 -6
View File
@@ -6,12 +6,7 @@ class UploadAssetRequest(BaseModel):
project_id: str = Field(..., min_length=1, description="项目 ID")
library_id: str = Field(..., min_length=1, description="素材库 ID")
class UploadAssetResponse(BaseModel):
storage_key: str
ingest_job_id: str
url: str = Field(..., description="Public URL of uploaded file")
file_hash: str = Field(default="", max_length=64, description="文件 MD5 哈希,用于去重检测")
class DirectUploadPrepareRequest(BaseModel):
@@ -20,6 +15,7 @@ class DirectUploadPrepareRequest(BaseModel):
filename: str = Field(..., min_length=1, max_length=255)
content_type: str = Field(default="application/octet-stream", min_length=1, max_length=100)
file_size: int = Field(..., gt=0)
file_hash: str = Field(default="", max_length=64, description="文件 MD5 哈希,用于去重检测")
class DirectUploadPrepareResponse(BaseModel):
@@ -35,8 +31,19 @@ class DirectUploadCompleteRequest(BaseModel):
project_id: str = Field(..., min_length=1)
library_id: str = Field(..., min_length=1)
storage_key: str = Field(..., min_length=1, max_length=255)
file_hash: str = Field(default="", max_length=64, description="文件 MD5 哈希,用于去重检测")
class DirectUploadCompleteResponse(BaseModel):
storage_key: str
ingest_job_id: str
duplicated: bool = Field(default=False, description="是否为重复素材(命中去重)")
asset_id: str = Field(default="", description="重复素材的 asset_idduplicated=true 时返回)")
class UploadAssetResponse(BaseModel):
storage_key: str
ingest_job_id: str
url: str = Field(..., description="Public URL of uploaded file")
duplicated: bool = Field(default=False, description="是否为重复素材(命中去重)")
asset_id: str = Field(default="", description="重复素材的 asset_idduplicated=true 时返回)")
+5 -3
View File
@@ -180,9 +180,11 @@ test.describe("Core media upload flow", () => {
await expect(page.locator(".xx-assets-content")).toBeVisible({
timeout: 20_000,
});
await expect(page.getByText("e2e-sample.MOV", { exact: true })).toBeVisible({
timeout: 20_000,
});
await expect(page.getByText("e2e-sample.MOV", { exact: true })).toBeVisible(
{
timeout: 20_000,
},
);
// Verify asset card shows status
const assetCard = page
+4 -1
View File
@@ -178,7 +178,10 @@ test.describe("订阅过期处理", () => {
// 免费用户可能不需要取消,返回 400 或类似错误
if (!response.ok()) {
const data = await response.json();
expect(data.error?.message || data.detail || data.message, "应返回错误信息").toBeTruthy();
expect(
data.error?.message || data.detail || data.message,
"应返回错误信息",
).toBeTruthy();
}
});
+6 -5
View File
@@ -175,10 +175,9 @@ test.describe("认证流程", () => {
},
});
expect(
[400, 422],
"缺少用户名字段应返回 4xx 校验错误",
).toContain(response.status());
expect([400, 422], "缺少用户名字段应返回 4xx 校验错误").toContain(
response.status(),
);
});
// ─── 登录 ────────────────────────────────────────────
@@ -230,7 +229,9 @@ test.describe("认证流程", () => {
data: { email: `ghost_${Date.now()}@nonexist.com`, password: PASSWORD },
});
if (response.status() !== 429) break;
console.log(`[反向登录测试] 触发限流,等待 65s 后重试 (${attempt + 1}/2)`);
console.log(
`[反向登录测试] 触发限流,等待 65s 后重试 (${attempt + 1}/2)`,
);
await new Promise((r) => setTimeout(r, 65_000));
}
+207 -415
View File
@@ -14,10 +14,7 @@
"axios": "^1.7.2",
"react": "^18.3.1",
"react-dom": "^18.3.1",
"react-hook-form": "^7.52.0",
"react-router-dom": "^6.24.0",
"recharts": "^3.8.1",
"zod": "^3.23.8",
"zustand": "^4.5.2"
},
"devDependencies": {
@@ -31,11 +28,13 @@
"@typescript-eslint/eslint-plugin": "^7.13.1",
"@typescript-eslint/parser": "^7.13.1",
"@vitejs/plugin-react": "^4.3.1",
"@vitest/coverage-v8": "^1.6.0",
"@vitest/ui": "^1.6.0",
"eslint": "^8.57.0",
"eslint-plugin-react-hooks": "^4.6.2",
"eslint-plugin-react-refresh": "^0.4.7",
"jsdom": "^24.1.0",
"prettier": "^3.9.5",
"typescript": "^5.5.3",
"vite": "^5.3.1",
"vitest": "^1.6.0"
@@ -48,6 +47,20 @@
"dev": true,
"license": "MIT"
},
"node_modules/@ampproject/remapping": {
"version": "2.3.0",
"resolved": "https://registry.npmjs.org/@ampproject/remapping/-/remapping-2.3.0.tgz",
"integrity": "sha512-30iZtAPgz+LTIYoeivqYo853f02jBYSd5uGnGpkFV0M3xOt9aN73erkgYAmZU43x4VfqcnLxW9Kpg3R5LC4YYw==",
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"@jridgewell/gen-mapping": "^0.3.5",
"@jridgewell/trace-mapping": "^0.3.24"
},
"engines": {
"node": ">=6.0.0"
}
},
"node_modules/@ant-design/colors": {
"version": "7.2.1",
"resolved": "https://registry.npmjs.org/@ant-design/colors/-/colors-7.2.1.tgz",
@@ -477,6 +490,13 @@
"node": ">=6.9.0"
}
},
"node_modules/@bcoe/v8-coverage": {
"version": "0.2.3",
"resolved": "https://registry.npmjs.org/@bcoe/v8-coverage/-/v8-coverage-0.2.3.tgz",
"integrity": "sha512-0hYQ8SB4Db5zvZB4axdMHGwEaQjkZzFjQiN9LVYvIFB2nSUHW9tYpxWriPrWDASIxiaXax83REcLxuSdnGPZtw==",
"dev": true,
"license": "MIT"
},
"node_modules/@csstools/color-helpers": {
"version": "5.1.0",
"resolved": "https://registry.npmjs.org/@csstools/color-helpers/-/color-helpers-5.1.0.tgz",
@@ -1144,6 +1164,16 @@
"dev": true,
"license": "BSD-3-Clause"
},
"node_modules/@istanbuljs/schema": {
"version": "0.1.6",
"resolved": "https://registry.npmjs.org/@istanbuljs/schema/-/schema-0.1.6.tgz",
"integrity": "sha512-+Sg6GCR/wy1oSmQDFq4LQDAhm3ETKnorxN+y5nbLULOR3P0c14f2Wurzj3/xqPXtasLFfHd5iRFQ7AJt4KH2cw==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=8"
}
},
"node_modules/@jest/schemas": {
"version": "29.6.3",
"resolved": "https://registry.npmjs.org/@jest/schemas/-/schemas-29.6.3.tgz",
@@ -1415,42 +1445,6 @@
"react-dom": ">=16.9.0"
}
},
"node_modules/@reduxjs/toolkit": {
"version": "2.12.0",
"resolved": "https://registry.npmjs.org/@reduxjs/toolkit/-/toolkit-2.12.0.tgz",
"integrity": "sha512-KiT+RzZbp6mQET+Mg+h2c97+9j1sNflUxQkIHI7Yuzf6Peu+OYpmkn6nbHWmLLWj+1ZODUJFwGZ7gx3L9R9EOw==",
"license": "MIT",
"dependencies": {
"@standard-schema/spec": "^1.0.0",
"@standard-schema/utils": "^0.3.0",
"immer": "^11.0.0",
"redux": "^5.0.1",
"redux-thunk": "^3.1.0",
"reselect": "^5.1.0"
},
"peerDependencies": {
"react": "^16.9.0 || ^17.0.0 || ^18 || ^19",
"react-redux": "^7.2.1 || ^8.1.3 || ^9.0.0"
},
"peerDependenciesMeta": {
"react": {
"optional": true
},
"react-redux": {
"optional": true
}
}
},
"node_modules/@reduxjs/toolkit/node_modules/immer": {
"version": "11.1.8",
"resolved": "https://registry.npmjs.org/immer/-/immer-11.1.8.tgz",
"integrity": "sha512-/tbkHMW7y10Lx6i1crLjD4/OhNkRG+Fo7byZHtah0547nIeXYcpIXaUh0IAQY6gO5459qpGGYapcEOHtFXkIuA==",
"license": "MIT",
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/immer"
}
},
"node_modules/@remix-run/router": {
"version": "1.23.3",
"resolved": "https://registry.npmjs.org/@remix-run/router/-/router-1.23.3.tgz",
@@ -1824,18 +1818,6 @@
"dev": true,
"license": "MIT"
},
"node_modules/@standard-schema/spec": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/@standard-schema/spec/-/spec-1.1.0.tgz",
"integrity": "sha512-l2aFy5jALhniG5HgqrD6jXLi/rUWrKvqN/qJx6yoJsgKhblVd+iqqU4RCXavm/jPityDo5TCvKMnpjKnOriy0w==",
"license": "MIT"
},
"node_modules/@standard-schema/utils": {
"version": "0.3.0",
"resolved": "https://registry.npmjs.org/@standard-schema/utils/-/utils-0.3.0.tgz",
"integrity": "sha512-e7Mew686owMaPJVNNLs55PUvgz371nKgwsc4vxE49zsODpJEnxgxRo2y/OKrqueavXgZNMDVj3DdHFlaSAeU8g==",
"license": "MIT"
},
"node_modules/@tanstack/query-core": {
"version": "5.101.0",
"resolved": "https://registry.npmjs.org/@tanstack/query-core/-/query-core-5.101.0.tgz",
@@ -2005,69 +1987,6 @@
"@babel/types": "^7.28.2"
}
},
"node_modules/@types/d3-array": {
"version": "3.2.2",
"resolved": "https://registry.npmjs.org/@types/d3-array/-/d3-array-3.2.2.tgz",
"integrity": "sha512-hOLWVbm7uRza0BYXpIIW5pxfrKe0W+D5lrFiAEYR+pb6w3N2SwSMaJbXdUfSEv+dT4MfHBLtn5js0LAWaO6otw==",
"license": "MIT"
},
"node_modules/@types/d3-color": {
"version": "3.1.3",
"resolved": "https://registry.npmjs.org/@types/d3-color/-/d3-color-3.1.3.tgz",
"integrity": "sha512-iO90scth9WAbmgv7ogoq57O9YpKmFBbmoEoCHDB2xMBY0+/KVrqAaCDyCE16dUspeOvIxFFRI+0sEtqDqy2b4A==",
"license": "MIT"
},
"node_modules/@types/d3-ease": {
"version": "3.0.2",
"resolved": "https://registry.npmjs.org/@types/d3-ease/-/d3-ease-3.0.2.tgz",
"integrity": "sha512-NcV1JjO5oDzoK26oMzbILE6HW7uVXOHLQvHshBUW4UMdZGfiY6v5BeQwh9a9tCzv+CeefZQHJt5SRgK154RtiA==",
"license": "MIT"
},
"node_modules/@types/d3-interpolate": {
"version": "3.0.4",
"resolved": "https://registry.npmjs.org/@types/d3-interpolate/-/d3-interpolate-3.0.4.tgz",
"integrity": "sha512-mgLPETlrpVV1YRJIglr4Ez47g7Yxjl1lj7YKsiMCb27VJH9W8NVM6Bb9d8kkpG/uAQS5AmbA48q2IAolKKo1MA==",
"license": "MIT",
"dependencies": {
"@types/d3-color": "*"
}
},
"node_modules/@types/d3-path": {
"version": "3.1.1",
"resolved": "https://registry.npmjs.org/@types/d3-path/-/d3-path-3.1.1.tgz",
"integrity": "sha512-VMZBYyQvbGmWyWVea0EHs/BwLgxc+MKi1zLDCONksozI4YJMcTt8ZEuIR4Sb1MMTE8MMW49v0IwI5+b7RmfWlg==",
"license": "MIT"
},
"node_modules/@types/d3-scale": {
"version": "4.0.9",
"resolved": "https://registry.npmjs.org/@types/d3-scale/-/d3-scale-4.0.9.tgz",
"integrity": "sha512-dLmtwB8zkAeO/juAMfnV+sItKjlsw2lKdZVVy6LRr0cBmegxSABiLEpGVmSJJ8O08i4+sGR6qQtb6WtuwJdvVw==",
"license": "MIT",
"dependencies": {
"@types/d3-time": "*"
}
},
"node_modules/@types/d3-shape": {
"version": "3.1.8",
"resolved": "https://registry.npmjs.org/@types/d3-shape/-/d3-shape-3.1.8.tgz",
"integrity": "sha512-lae0iWfcDeR7qt7rA88BNiqdvPS5pFVPpo5OfjElwNaT2yyekbM0C9vK+yqBqEmHr6lDkRnYNoTBYlAgJa7a4w==",
"license": "MIT",
"dependencies": {
"@types/d3-path": "*"
}
},
"node_modules/@types/d3-time": {
"version": "3.0.4",
"resolved": "https://registry.npmjs.org/@types/d3-time/-/d3-time-3.0.4.tgz",
"integrity": "sha512-yuzZug1nkAAaBlBBikKZTgzCeA+k1uy4ZFwWANOfKw5z5LRhV0gNA7gNkKm7HoK+HRN0wX3EkxGk0fpbWhmB7g==",
"license": "MIT"
},
"node_modules/@types/d3-timer": {
"version": "3.0.2",
"resolved": "https://registry.npmjs.org/@types/d3-timer/-/d3-timer-3.0.2.tgz",
"integrity": "sha512-Ps3T8E8dZDam6fUyNiMkekK3XUsaUEik+idO9/YjPtfj2qruF8tFBXS7XhtE4iIXBLxhmLjP3SXpLhVf21I9Lw==",
"license": "MIT"
},
"node_modules/@types/estree": {
"version": "1.0.9",
"resolved": "https://registry.npmjs.org/@types/estree/-/estree-1.0.9.tgz",
@@ -2113,12 +2032,6 @@
"@types/react": "^18.0.0"
}
},
"node_modules/@types/use-sync-external-store": {
"version": "0.0.6",
"resolved": "https://registry.npmjs.org/@types/use-sync-external-store/-/use-sync-external-store-0.0.6.tgz",
"integrity": "sha512-zFDAD+tlpf2r4asuHEj0XH6pY6i0g5NeAHPn+15wk3BV6JA69eERFXC1gyGThDkVa1zCyKr5jox1+2LbV/AMLg==",
"license": "MIT"
},
"node_modules/@typescript-eslint/eslint-plugin": {
"version": "7.18.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/eslint-plugin/-/eslint-plugin-7.18.0.tgz",
@@ -2340,6 +2253,34 @@
"vite": "^4.2.0 || ^5.0.0 || ^6.0.0 || ^7.0.0"
}
},
"node_modules/@vitest/coverage-v8": {
"version": "1.6.1",
"resolved": "https://registry.npmjs.org/@vitest/coverage-v8/-/coverage-v8-1.6.1.tgz",
"integrity": "sha512-6YeRZwuO4oTGKxD3bijok756oktHSIm3eczVVzNe3scqzuhLwltIF3S9ZL/vwOVIpURmU6SnZhziXXAfw8/Qlw==",
"dev": true,
"license": "MIT",
"dependencies": {
"@ampproject/remapping": "^2.2.1",
"@bcoe/v8-coverage": "^0.2.3",
"debug": "^4.3.4",
"istanbul-lib-coverage": "^3.2.2",
"istanbul-lib-report": "^3.0.1",
"istanbul-lib-source-maps": "^5.0.4",
"istanbul-reports": "^3.1.6",
"magic-string": "^0.30.5",
"magicast": "^0.3.3",
"picocolors": "^1.0.0",
"std-env": "^3.5.0",
"strip-literal": "^2.0.0",
"test-exclude": "^6.0.0"
},
"funding": {
"url": "https://opencollective.com/vitest"
},
"peerDependencies": {
"vitest": "1.6.1"
}
},
"node_modules/@vitest/expect": {
"version": "1.6.1",
"resolved": "https://registry.npmjs.org/@vitest/expect/-/expect-1.6.1.tgz",
@@ -2932,15 +2873,6 @@
"integrity": "sha512-saHYOzhIQs6wy2sVxTM6bUDsQO4F50V9RQ22qBpEdCW+I+/Wmke2HOl6lS6dTpdxVhb88/I6+Hs+438c3lfUow==",
"license": "MIT"
},
"node_modules/clsx": {
"version": "2.1.1",
"resolved": "https://registry.npmjs.org/clsx/-/clsx-2.1.1.tgz",
"integrity": "sha512-eYm0QWBtUrBWZWG0d386OGAw16Z995PiOVo2B7bjWSbHedGl5e0ZWaq65kOGgUSNesEIDkB9ISbTg/JK9dhCZA==",
"license": "MIT",
"engines": {
"node": ">=6"
}
},
"node_modules/color-convert": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/color-convert/-/color-convert-2.0.1.tgz",
@@ -3058,127 +2990,6 @@
"integrity": "sha512-z1HGKcYy2xA8AGQfwrn0PAy+PB7X/GSj3UVJW9qKyn43xWa+gl5nXmU4qqLMRzWVLFC8KusUX8T/0kCiOYpAIQ==",
"license": "MIT"
},
"node_modules/d3-array": {
"version": "3.2.4",
"resolved": "https://registry.npmjs.org/d3-array/-/d3-array-3.2.4.tgz",
"integrity": "sha512-tdQAmyA18i4J7wprpYq8ClcxZy3SC31QMeByyCFyRt7BVHdREQZ5lpzoe5mFEYZUWe+oq8HBvk9JjpibyEV4Jg==",
"license": "ISC",
"dependencies": {
"internmap": "1 - 2"
},
"engines": {
"node": ">=12"
}
},
"node_modules/d3-color": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/d3-color/-/d3-color-3.1.0.tgz",
"integrity": "sha512-zg/chbXyeBtMQ1LbD/WSoW2DpC3I0mpmPdW+ynRTj/x2DAWYrIY7qeZIHidozwV24m4iavr15lNwIwLxRmOxhA==",
"license": "ISC",
"engines": {
"node": ">=12"
}
},
"node_modules/d3-ease": {
"version": "3.0.1",
"resolved": "https://registry.npmjs.org/d3-ease/-/d3-ease-3.0.1.tgz",
"integrity": "sha512-wR/XK3D3XcLIZwpbvQwQ5fK+8Ykds1ip7A2Txe0yxncXSdq1L9skcG7blcedkOX+ZcgxGAmLX1FrRGbADwzi0w==",
"license": "BSD-3-Clause",
"engines": {
"node": ">=12"
}
},
"node_modules/d3-format": {
"version": "3.1.2",
"resolved": "https://registry.npmjs.org/d3-format/-/d3-format-3.1.2.tgz",
"integrity": "sha512-AJDdYOdnyRDV5b6ArilzCPPwc1ejkHcoyFarqlPqT7zRYjhavcT3uSrqcMvsgh2CgoPbK3RCwyHaVyxYcP2Arg==",
"license": "ISC",
"engines": {
"node": ">=12"
}
},
"node_modules/d3-interpolate": {
"version": "3.0.1",
"resolved": "https://registry.npmjs.org/d3-interpolate/-/d3-interpolate-3.0.1.tgz",
"integrity": "sha512-3bYs1rOD33uo8aqJfKP3JWPAibgw8Zm2+L9vBKEHJ2Rg+viTR7o5Mmv5mZcieN+FRYaAOWX5SJATX6k1PWz72g==",
"license": "ISC",
"dependencies": {
"d3-color": "1 - 3"
},
"engines": {
"node": ">=12"
}
},
"node_modules/d3-path": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/d3-path/-/d3-path-3.1.0.tgz",
"integrity": "sha512-p3KP5HCf/bvjBSSKuXid6Zqijx7wIfNW+J/maPs+iwR35at5JCbLUT0LzF1cnjbCHWhqzQTIN2Jpe8pRebIEFQ==",
"license": "ISC",
"engines": {
"node": ">=12"
}
},
"node_modules/d3-scale": {
"version": "4.0.2",
"resolved": "https://registry.npmjs.org/d3-scale/-/d3-scale-4.0.2.tgz",
"integrity": "sha512-GZW464g1SH7ag3Y7hXjf8RoUuAFIqklOAq3MRl4OaWabTFJY9PN/E1YklhXLh+OQ3fM9yS2nOkCoS+WLZ6kvxQ==",
"license": "ISC",
"dependencies": {
"d3-array": "2.10.0 - 3",
"d3-format": "1 - 3",
"d3-interpolate": "1.2.0 - 3",
"d3-time": "2.1.1 - 3",
"d3-time-format": "2 - 4"
},
"engines": {
"node": ">=12"
}
},
"node_modules/d3-shape": {
"version": "3.2.0",
"resolved": "https://registry.npmjs.org/d3-shape/-/d3-shape-3.2.0.tgz",
"integrity": "sha512-SaLBuwGm3MOViRq2ABk3eLoxwZELpH6zhl3FbAoJ7Vm1gofKx6El1Ib5z23NUEhF9AsGl7y+dzLe5Cw2AArGTA==",
"license": "ISC",
"dependencies": {
"d3-path": "^3.1.0"
},
"engines": {
"node": ">=12"
}
},
"node_modules/d3-time": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/d3-time/-/d3-time-3.1.0.tgz",
"integrity": "sha512-VqKjzBLejbSMT4IgbmVgDjpkYrNWUYJnbCGo874u7MMKIWsILRX+OpX/gTk8MqjpT1A/c6HY2dCA77ZN0lkQ2Q==",
"license": "ISC",
"dependencies": {
"d3-array": "2 - 3"
},
"engines": {
"node": ">=12"
}
},
"node_modules/d3-time-format": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/d3-time-format/-/d3-time-format-4.1.0.tgz",
"integrity": "sha512-dJxPBlzC7NugB2PDLwo9Q8JiTR3M3e4/XANkreKSUxF8vvXKqm1Yfq4Q5dl8budlunRVlUUaDUgFt7eA8D6NLg==",
"license": "ISC",
"dependencies": {
"d3-time": "1 - 3"
},
"engines": {
"node": ">=12"
}
},
"node_modules/d3-timer": {
"version": "3.0.1",
"resolved": "https://registry.npmjs.org/d3-timer/-/d3-timer-3.0.1.tgz",
"integrity": "sha512-ndfJ/JxxMd3nw31uyKoY2naivF+r29V+Lc0svZxe1JvvIRmi8hUsrMvdOwgS1o6uBHmiz91geQ0ylPP0aj1VUA==",
"license": "ISC",
"engines": {
"node": ">=12"
}
},
"node_modules/data-urls": {
"version": "5.0.0",
"resolved": "https://registry.npmjs.org/data-urls/-/data-urls-5.0.0.tgz",
@@ -3223,12 +3034,6 @@
"dev": true,
"license": "MIT"
},
"node_modules/decimal.js-light": {
"version": "2.5.1",
"resolved": "https://registry.npmjs.org/decimal.js-light/-/decimal.js-light-2.5.1.tgz",
"integrity": "sha512-qIMFpTMZmny+MMIitAB6D7iVPEorVw6YQRWkvarTkT4tBeSLLiHzcwj6q0MmYSFCiVpiqPJTJEYIrpcPzVEIvg==",
"license": "MIT"
},
"node_modules/deep-eql": {
"version": "4.1.4",
"resolved": "https://registry.npmjs.org/deep-eql/-/deep-eql-4.1.4.tgz",
@@ -3391,16 +3196,6 @@
"node": ">= 0.4"
}
},
"node_modules/es-toolkit": {
"version": "1.47.1",
"resolved": "https://registry.npmjs.org/es-toolkit/-/es-toolkit-1.47.1.tgz",
"integrity": "sha512-5RAqEwf4P4E17p+W75KLOWw/nOvKZzSQpxM32IpI2KZLaVonjTrZ0Ai5ghMaVI9eKC2p8eoQgcBdkEDgzFk6+Q==",
"license": "MIT",
"workspaces": [
"docs",
"benchmarks"
]
},
"node_modules/esbuild": {
"version": "0.21.5",
"resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.21.5.tgz",
@@ -3671,12 +3466,6 @@
"node": ">=0.10.0"
}
},
"node_modules/eventemitter3": {
"version": "5.0.4",
"resolved": "https://registry.npmjs.org/eventemitter3/-/eventemitter3-5.0.4.tgz",
"integrity": "sha512-mlsTRyGaPBjPedk6Bvw+aqbsXDtoAyAzm5MO7JgU+yVRyMQ5O8bD4Kcci7BS85f93veegeCPkL8R4GLClnjLFw==",
"license": "MIT"
},
"node_modules/execa": {
"version": "8.0.1",
"resolved": "https://registry.npmjs.org/execa/-/execa-8.0.1.tgz",
@@ -4148,6 +3937,13 @@
"node": ">=18"
}
},
"node_modules/html-escaper": {
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/html-escaper/-/html-escaper-2.0.2.tgz",
"integrity": "sha512-H2iMtd0I4Mt5eYiapRdIDjp+XzelXQ0tFE4JS7YFwFevXXMmOp9myNrUvCg0D6ws8iqkRPBfKHgbwig1SmlLfg==",
"dev": true,
"license": "MIT"
},
"node_modules/http-proxy-agent": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/http-proxy-agent/-/http-proxy-agent-7.0.2.tgz",
@@ -4223,6 +4019,8 @@
"resolved": "https://registry.npmjs.org/immer/-/immer-10.2.0.tgz",
"integrity": "sha512-d/+XTN3zfODyjr89gM3mPq1WNX2B8pYsu7eORitdwyA2sBubnTl3laYlBk4sXY5FUa5qTZGBDPJICVbvqzjlbw==",
"license": "MIT",
"optional": true,
"peer": true,
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/immer"
@@ -4284,15 +4082,6 @@
"dev": true,
"license": "ISC"
},
"node_modules/internmap": {
"version": "2.0.3",
"resolved": "https://registry.npmjs.org/internmap/-/internmap-2.0.3.tgz",
"integrity": "sha512-5Hh7Y1wQbvY5ooGgPbDaL5iYLAPzMTUrjMulskHLH6wnv/A+1q5rgEaiuqEjB+oxGXIVZs1FF+R/KPN3ZSQYYg==",
"license": "ISC",
"engines": {
"node": ">=12"
}
},
"node_modules/is-extglob": {
"version": "2.1.1",
"resolved": "https://registry.npmjs.org/is-extglob/-/is-extglob-2.1.1.tgz",
@@ -4363,6 +4152,60 @@
"dev": true,
"license": "ISC"
},
"node_modules/istanbul-lib-coverage": {
"version": "3.2.2",
"resolved": "https://registry.npmjs.org/istanbul-lib-coverage/-/istanbul-lib-coverage-3.2.2.tgz",
"integrity": "sha512-O8dpsF+r0WV/8MNRKfnmrtCWhuKjxrq2w+jpzBL5UZKTi2LeVWnWOmWRxFlesJONmc+wLAGvKQZEOanko0LFTg==",
"dev": true,
"license": "BSD-3-Clause",
"engines": {
"node": ">=8"
}
},
"node_modules/istanbul-lib-report": {
"version": "3.0.1",
"resolved": "https://registry.npmjs.org/istanbul-lib-report/-/istanbul-lib-report-3.0.1.tgz",
"integrity": "sha512-GCfE1mtsHGOELCU8e/Z7YWzpmybrx/+dSTfLrvY8qRmaY6zXTKWn6WQIjaAFw069icm6GVMNkgu0NzI4iPZUNw==",
"dev": true,
"license": "BSD-3-Clause",
"dependencies": {
"istanbul-lib-coverage": "^3.0.0",
"make-dir": "^4.0.0",
"supports-color": "^7.1.0"
},
"engines": {
"node": ">=10"
}
},
"node_modules/istanbul-lib-source-maps": {
"version": "5.0.6",
"resolved": "https://registry.npmjs.org/istanbul-lib-source-maps/-/istanbul-lib-source-maps-5.0.6.tgz",
"integrity": "sha512-yg2d+Em4KizZC5niWhQaIomgf5WlL4vOOjZ5xGCmF8SnPE/mDWWXgvRExdcpCgh9lLRRa1/fSYp2ymmbJ1pI+A==",
"dev": true,
"license": "BSD-3-Clause",
"dependencies": {
"@jridgewell/trace-mapping": "^0.3.23",
"debug": "^4.1.1",
"istanbul-lib-coverage": "^3.0.0"
},
"engines": {
"node": ">=10"
}
},
"node_modules/istanbul-reports": {
"version": "3.2.0",
"resolved": "https://registry.npmjs.org/istanbul-reports/-/istanbul-reports-3.2.0.tgz",
"integrity": "sha512-HGYWWS/ehqTV3xN10i23tkPkpH46MLCIMFNCaaKNavAXTF1RkqxawEPtnjnGZ6XKSInBKkiOA5BKS+aZiY3AvA==",
"dev": true,
"license": "BSD-3-Clause",
"dependencies": {
"html-escaper": "^2.0.0",
"istanbul-lib-report": "^3.0.0"
},
"engines": {
"node": ">=8"
}
},
"node_modules/js-tokens": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/js-tokens/-/js-tokens-4.0.0.tgz",
@@ -4630,6 +4473,34 @@
"@jridgewell/sourcemap-codec": "^1.5.5"
}
},
"node_modules/magicast": {
"version": "0.3.5",
"resolved": "https://registry.npmjs.org/magicast/-/magicast-0.3.5.tgz",
"integrity": "sha512-L0WhttDl+2BOsybvEOLK7fW3UA0OQ0IQ2d6Zl2x/a6vVRs3bAY0ECOSHHeL5jD+SbOpOCUEi0y1DgHEn9Qn1AQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@babel/parser": "^7.25.4",
"@babel/types": "^7.25.4",
"source-map-js": "^1.2.0"
}
},
"node_modules/make-dir": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/make-dir/-/make-dir-4.0.0.tgz",
"integrity": "sha512-hXdUTZYIVOt1Ex//jAQi+wTZZpUpwBj/0QsOzqegb3rGMMeJiSEu5xLHnYfBrRV4RH2+OCSOO95Is/7x1WJ4bw==",
"dev": true,
"license": "MIT",
"dependencies": {
"semver": "^7.5.3"
},
"engines": {
"node": ">=10"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/math-intrinsics": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/math-intrinsics/-/math-intrinsics-1.1.0.tgz",
@@ -5107,6 +4978,22 @@
"node": ">= 0.8.0"
}
},
"node_modules/prettier": {
"version": "3.9.6",
"resolved": "https://registry.npmjs.org/prettier/-/prettier-3.9.6.tgz",
"integrity": "sha512-OpN0zzVdiaiAhxpuuj5efpIS4sY9j7bY6uR5mnj5yPzGkdkjNKSJeUThPb60Jw29QuAZgA4o+/iB49kFiaBX6g==",
"dev": true,
"license": "MIT",
"bin": {
"prettier": "bin/prettier.cjs"
},
"engines": {
"node": ">=14"
},
"funding": {
"url": "https://github.com/prettier/prettier?sponsor=1"
}
},
"node_modules/pretty-format": {
"version": "27.5.1",
"resolved": "https://registry.npmjs.org/pretty-format/-/pretty-format-27.5.1.tgz",
@@ -5842,52 +5729,6 @@
"react": "^18.3.1"
}
},
"node_modules/react-hook-form": {
"version": "7.79.0",
"resolved": "https://registry.npmjs.org/react-hook-form/-/react-hook-form-7.79.0.tgz",
"integrity": "sha512-mhYp/MTmXvzYX6AJcJVko0rktoIhhmRnEouObj4wF5i/tCttgJvnp1+9wRkpITZjDTqpo4IOSJqu0dBlPlV/Lw==",
"license": "MIT",
"engines": {
"node": ">=18.0.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/react-hook-form"
},
"peerDependencies": {
"react": "^16.8.0 || ^17 || ^18 || ^19"
}
},
"node_modules/react-is": {
"version": "19.2.7",
"resolved": "https://registry.npmjs.org/react-is/-/react-is-19.2.7.tgz",
"integrity": "sha512-kZFnouyVv7eP/Phmrlo9FK+zcAdriZJvzxXHF1Sl1P377WSGe2G/JxVolhTrB/jeV47lKImhNUsijjHAAbcl/A==",
"license": "MIT",
"peer": true
},
"node_modules/react-redux": {
"version": "9.3.0",
"resolved": "https://registry.npmjs.org/react-redux/-/react-redux-9.3.0.tgz",
"integrity": "sha512-KQopgqFo/p/fgmAs5qz6p5RWaNAzq40WAu7fJIXnQpYxFPbJYtsJPWvGeF2rOBaY/kEuV77AVsX8TsQzKm+A/g==",
"license": "MIT",
"dependencies": {
"@types/use-sync-external-store": "^0.0.6",
"use-sync-external-store": "^1.4.0"
},
"peerDependencies": {
"@types/react": "^18.2.25 || ^19",
"react": "^18.0 || ^19",
"redux": "^5.0.0"
},
"peerDependenciesMeta": {
"@types/react": {
"optional": true
},
"redux": {
"optional": true
}
}
},
"node_modules/react-refresh": {
"version": "0.17.0",
"resolved": "https://registry.npmjs.org/react-refresh/-/react-refresh-0.17.0.tgz",
@@ -5930,36 +5771,6 @@
"react-dom": ">=16.8"
}
},
"node_modules/recharts": {
"version": "3.8.1",
"resolved": "https://registry.npmjs.org/recharts/-/recharts-3.8.1.tgz",
"integrity": "sha512-mwzmO1s9sFL0TduUpwndxCUNoXsBw3u3E/0+A+cLcrSfQitSG62L32N69GhqUrrT5qKcAE3pCGVINC6pqkBBQg==",
"license": "MIT",
"workspaces": [
"www"
],
"dependencies": {
"@reduxjs/toolkit": "^1.9.0 || 2.x.x",
"clsx": "^2.1.1",
"decimal.js-light": "^2.5.1",
"es-toolkit": "^1.39.3",
"eventemitter3": "^5.0.1",
"immer": "^10.1.1",
"react-redux": "8.x.x || 9.x.x",
"reselect": "5.1.1",
"tiny-invariant": "^1.3.3",
"use-sync-external-store": "^1.2.2",
"victory-vendor": "^37.0.2"
},
"engines": {
"node": ">=18"
},
"peerDependencies": {
"react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0",
"react-dom": "^16.0.0 || ^17.0.0 || ^18.0.0 || ^19.0.0",
"react-is": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0"
}
},
"node_modules/redent": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/redent/-/redent-3.0.0.tgz",
@@ -5974,21 +5785,6 @@
"node": ">=8"
}
},
"node_modules/redux": {
"version": "5.0.1",
"resolved": "https://registry.npmjs.org/redux/-/redux-5.0.1.tgz",
"integrity": "sha512-M9/ELqF6fy8FwmkpnF0S3YKOqMyoWJ4+CS5Efg2ct3oY9daQvd/Pc71FpGZsVsbl3Cpb+IIcjBDUnnyBdQbq4w==",
"license": "MIT"
},
"node_modules/redux-thunk": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/redux-thunk/-/redux-thunk-3.1.0.tgz",
"integrity": "sha512-NW2r5T6ksUKXCabzhL9z+h206HQw/NJkcLm1GPImRQ8IzfXwRGqjVhKJGauHirT0DAuyy6hjdnMZaRoAcy0Klw==",
"license": "MIT",
"peerDependencies": {
"redux": "^5.0.0"
}
},
"node_modules/requires-port": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/requires-port/-/requires-port-1.0.0.tgz",
@@ -5996,12 +5792,6 @@
"dev": true,
"license": "MIT"
},
"node_modules/reselect": {
"version": "5.1.1",
"resolved": "https://registry.npmjs.org/reselect/-/reselect-5.1.1.tgz",
"integrity": "sha512-K/BG6eIky/SBpzfHZv/dd+9JBFiS4SWV7FIujVyJRux6e45+73RaUHXLmIR1f7WOMaQ0U1km6qwklRQxpJJY0w==",
"license": "MIT"
},
"node_modules/resize-observer-polyfill": {
"version": "1.5.1",
"resolved": "https://registry.npmjs.org/resize-observer-polyfill/-/resize-observer-polyfill-1.5.1.tgz",
@@ -6369,6 +6159,45 @@
"dev": true,
"license": "MIT"
},
"node_modules/test-exclude": {
"version": "6.0.0",
"resolved": "https://registry.npmjs.org/test-exclude/-/test-exclude-6.0.0.tgz",
"integrity": "sha512-cAGWPIyOHU6zlmg88jwm7VRyXnMN7iV68OGAbYDk/Mh/xC/pzVPlQtY6ngoIH/5/tciuhGfvESU8GrHrcxD56w==",
"dev": true,
"license": "ISC",
"dependencies": {
"@istanbuljs/schema": "^0.1.2",
"glob": "^7.1.4",
"minimatch": "^3.0.4"
},
"engines": {
"node": ">=8"
}
},
"node_modules/test-exclude/node_modules/brace-expansion": {
"version": "1.1.16",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.16.tgz",
"integrity": "sha512-IDw48K2/2kRkg9LdJxurvq3lV3aBgq0REY89duEqFRthjlPdXHKMj7EnQOXVckxzgisinf3nHfrcE2FufFLXMw==",
"dev": true,
"license": "MIT",
"dependencies": {
"balanced-match": "^1.0.0",
"concat-map": "0.0.1"
}
},
"node_modules/test-exclude/node_modules/minimatch": {
"version": "3.1.5",
"resolved": "https://registry.npmjs.org/minimatch/-/minimatch-3.1.5.tgz",
"integrity": "sha512-VgjWUsnnT6n+NUk6eZq77zeFdpW2LWDzP6zFGrCbHXiYNul5Dzqk2HHQ5uFH2DNW5Xbp8+jVzaeNt94ssEEl4w==",
"dev": true,
"license": "ISC",
"dependencies": {
"brace-expansion": "^1.1.7"
},
"engines": {
"node": "*"
}
},
"node_modules/text-table": {
"version": "0.2.0",
"resolved": "https://registry.npmjs.org/text-table/-/text-table-0.2.0.tgz",
@@ -6385,12 +6214,6 @@
"node": ">=12.22"
}
},
"node_modules/tiny-invariant": {
"version": "1.3.3",
"resolved": "https://registry.npmjs.org/tiny-invariant/-/tiny-invariant-1.3.3.tgz",
"integrity": "sha512-+FbBPE1o9QAYvviau/qC5SE3caw21q3xkvWKBtja5vgqOWIHHJ3ioaq1VPfn/Szqctz2bU/oYeKd9/z5BL+PVg==",
"license": "MIT"
},
"node_modules/tinybench": {
"version": "2.9.0",
"resolved": "https://registry.npmjs.org/tinybench/-/tinybench-2.9.0.tgz",
@@ -6624,28 +6447,6 @@
"react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0"
}
},
"node_modules/victory-vendor": {
"version": "37.3.6",
"resolved": "https://registry.npmjs.org/victory-vendor/-/victory-vendor-37.3.6.tgz",
"integrity": "sha512-SbPDPdDBYp+5MJHhBCAyI7wKM3d5ivekigc2Dk2s7pgbZ9wIgIBYGVw4zGHBml/qTFbexrofXW6Gu4noGxrOwQ==",
"license": "MIT AND ISC",
"dependencies": {
"@types/d3-array": "^3.0.3",
"@types/d3-ease": "^3.0.0",
"@types/d3-interpolate": "^3.0.1",
"@types/d3-scale": "^4.0.2",
"@types/d3-shape": "^3.1.0",
"@types/d3-time": "^3.0.0",
"@types/d3-timer": "^3.0.0",
"d3-array": "^3.1.6",
"d3-ease": "^3.0.1",
"d3-interpolate": "^3.0.1",
"d3-scale": "^4.0.2",
"d3-shape": "^3.1.0",
"d3-time": "^3.0.0",
"d3-timer": "^3.0.1"
}
},
"node_modules/vite": {
"version": "5.4.21",
"resolved": "https://registry.npmjs.org/vite/-/vite-5.4.21.tgz",
@@ -6980,15 +6781,6 @@
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/zod": {
"version": "3.25.76",
"resolved": "https://registry.npmjs.org/zod/-/zod-3.25.76.tgz",
"integrity": "sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ==",
"license": "MIT",
"funding": {
"url": "https://github.com/sponsors/colinhacks"
}
},
"node_modules/zustand": {
"version": "4.5.7",
"resolved": "https://registry.npmjs.org/zustand/-/zustand-4.5.7.tgz",
+2
View File
@@ -37,11 +37,13 @@
"@typescript-eslint/eslint-plugin": "^7.13.1",
"@typescript-eslint/parser": "^7.13.1",
"@vitejs/plugin-react": "^4.3.1",
"@vitest/coverage-v8": "^1.6.0",
"@vitest/ui": "^1.6.0",
"eslint": "^8.57.0",
"eslint-plugin-react-hooks": "^4.6.2",
"eslint-plugin-react-refresh": "^0.4.7",
"jsdom": "^24.1.0",
"prettier": "^3.9.5",
"typescript": "^5.5.3",
"vite": "^5.3.1",
"vitest": "^1.6.0"
+69 -21
View File
@@ -10,7 +10,6 @@ import {
SearchOutlined,
InboxOutlined,
VideoCameraOutlined,
SoundOutlined,
PictureOutlined,
PlayCircleOutlined,
CheckOutlined,
@@ -37,7 +36,7 @@ import "./assets.css";
/* ============================================================
* 类型
* ============================================================ */
type AssetKind = "video" | "voice" | "image";
type AssetKind = "video" | "image";
type StatusType = "ok" | "warn" | "bad" | "info";
interface LibraryItem {
@@ -67,7 +66,6 @@ interface AssetItem {
/** 根据 mime_type 推断前端 AssetKind */
const inferKind = (mimeType: string): AssetKind => {
if (mimeType.startsWith("video/")) return "video";
if (mimeType.startsWith("audio/")) return "voice";
return "image";
};
@@ -99,7 +97,7 @@ const formatDuration = (seconds: number): string => {
const mapLibrary = (item: AssetLibraryItem): LibraryItem => ({
id: item.id,
name: item.name,
kind: item.kind || inferKind("video"),
kind: (item.kind === "voice" ? "video" : item.kind) || inferKind("video"),
count: item.asset_count ?? 0,
});
@@ -110,14 +108,16 @@ const mapAsset = (item: ApiAssetItem): AssetItem => {
item.classification_status ?? undefined,
);
const metadata = item.metadata || {};
const kind = inferKind(item.mime_type || "");
return {
id: item.id,
name: item.name,
kind: inferKind(item.mime_type || ""),
kind,
// 视频类型不能用 file_url 做缩略图(是视频文件,<img> 无法渲染)
thumbUrl:
(item.thumbnail_url as string | undefined) ||
(item.file_url as string | undefined) ||
(metadata.thumbnail_url as string | undefined),
(metadata.thumbnail_url as string | undefined) ||
(kind !== "video" ? (item.file_url as string | undefined) : undefined),
fileUrl:
(item.file_url as string | undefined) ||
(metadata.file_url as string | undefined),
@@ -147,8 +147,6 @@ const kindIcon = (kind: AssetKind) => {
switch (kind) {
case "video":
return <VideoCameraOutlined />;
case "voice":
return <SoundOutlined />;
case "image":
return <PictureOutlined />;
}
@@ -158,8 +156,6 @@ const kindLabel = (kind: AssetKind) => {
switch (kind) {
case "video":
return "视频";
case "voice":
return "配音";
case "image":
return "图片";
}
@@ -170,8 +166,6 @@ const thumbGradient = (kind: AssetKind): string => {
switch (kind) {
case "video":
return "linear-gradient(135deg, #312e81 0%, #4f46e5 50%, #6366f1 100%)";
case "voice":
return "linear-gradient(135deg, #064e3b 0%, #059669 50%, #10b981 100%)";
case "image":
return "linear-gradient(135deg, #78350f 0%, #d97706 50%, #f59e0b 100%)";
}
@@ -244,7 +238,7 @@ const AssetCard: React.FC<{
)}
{/* 视频/配音类显示播放按钮 */}
{(asset.kind === "video" || asset.kind === "voice") && (
{asset.kind === "video" && (
<span
className="xx-asset-play"
onClick={(e) => {
@@ -361,6 +355,7 @@ const AssetLibrary: React.FC = () => {
/* 上传 */
const [uploading, setUploading] = useState(false);
const [uploadProgress, setUploadProgress] = useState(0);
/* 新建素材库 */
const [createModalOpen, setCreateModalOpen] = useState(false);
@@ -434,18 +429,25 @@ const AssetLibrary: React.FC = () => {
}
setUploading(true);
setUploadProgress(0);
try {
if (file.size > LARGE_FILE_THRESHOLD) {
message.info(`大文件 "${file.name}" 将使用直传上传`);
}
await uploadAssetDirect({ file, library_id: effectiveLibId });
await uploadAssetDirect({
file,
library_id: effectiveLibId,
onProgress: (pct) => setUploadProgress(pct),
});
message.success(`"${file.name}" 上传成功`);
queryClient.invalidateQueries({ queryKey: ["assets"] });
queryClient.invalidateQueries({ queryKey: ["asset-libraries"] });
} catch {
message.error(`"${file.name}" 上传失败`);
} catch (err: unknown) {
const detail = err instanceof Error ? err.message : "";
message.error(`"${file.name}" 上传失败${detail ? `${detail}` : ""}`);
} finally {
setUploading(false);
setUploadProgress(0);
}
return false;
};
@@ -532,6 +534,54 @@ const AssetLibrary: React.FC = () => {
return (
<div className="xx-assets-page">
{/* ─── 上传进度弹窗(圆形动画 + 百分比) ─── */}
<AntModal
open={uploading}
footer={null}
closable={false}
centered
width={260}
maskClosable={false}
className="xx-upload-progress-modal"
>
<div className="xx-upload-progress-body">
<svg
className="xx-upload-progress-ring"
viewBox="0 0 120 120"
width={120}
height={120}
>
{/* 背景圆环 */}
<circle
cx="60"
cy="60"
r="52"
fill="none"
stroke="var(--border-primary, #e5e7eb)"
strokeWidth="8"
/>
{/* 进度圆弧 */}
<circle
cx="60"
cy="60"
r="52"
fill="none"
stroke="var(--primary-color, #6366f1)"
strokeWidth="8"
strokeLinecap="round"
strokeDasharray={`${2 * Math.PI * 52}`}
strokeDashoffset={`${2 * Math.PI * 52 * (1 - uploadProgress / 100)}`}
transform="rotate(-90 60 60)"
style={{ transition: "stroke-dashoffset 0.3s ease" }}
/>
</svg>
<div className="xx-upload-progress-text">
<span className="xx-upload-progress-pct">{uploadProgress}%</span>
<span className="xx-upload-progress-label"></span>
</div>
</div>
</AntModal>
{/* 两栏布局 */}
<div className="xx-assets-layout">
{/* ─── 左侧:素材库列表 ─── */}
@@ -588,7 +638,7 @@ const AssetLibrary: React.FC = () => {
beforeUpload={handleUpload}
showUploadList={false}
multiple
accept="video/*,audio/*,image/*"
accept="video/*,image/*"
>
<div className="xx-asset-upload-zone">
<p className="xx-asset-upload-icon">
@@ -598,7 +648,7 @@ const AssetLibrary: React.FC = () => {
{uploading ? "上传中..." : "点击或拖拽文件到此区域上传"}
</p>
<p className="xx-asset-upload-hint">
2GB
2GB
</p>
</div>
</Upload.Dragger>
@@ -621,7 +671,6 @@ const AssetLibrary: React.FC = () => {
options={[
{ value: "all", label: "全部类型" },
{ value: "video", label: "视频" },
{ value: "voice", label: "配音" },
{ value: "image", label: "图片" },
]}
/>
@@ -752,7 +801,6 @@ const AssetLibrary: React.FC = () => {
style={{ width: "100%" }}
options={[
{ value: "video", label: "视频" },
{ value: "voice", label: "配音" },
{ value: "image", label: "图片" },
]}
/>
+37
View File
@@ -587,3 +587,40 @@
grid-template-columns: 1fr;
}
}
/* ─── 上传进度弹窗 ─── */
.xx-upload-progress-modal .ant-modal-content {
padding: 24px 16px 20px;
border-radius: 16px;
}
.xx-upload-progress-body {
display: flex;
flex-direction: column;
align-items: center;
gap: 16px;
padding: 8px 0;
}
.xx-upload-progress-ring {
display: block;
}
.xx-upload-progress-text {
display: flex;
flex-direction: column;
align-items: center;
gap: 4px;
}
.xx-upload-progress-pct {
font-size: 22px;
font-weight: 700;
color: var(--primary-color, #6366f1);
line-height: 1;
}
.xx-upload-progress-label {
font-size: 13px;
color: var(--text-secondary, #6b7280);
}
+340 -122
View File
@@ -6,24 +6,27 @@
*/
import React, { useState, useRef, useCallback, useEffect } from "react";
import { useQuery, useMutation } from "@tanstack/react-query";
import { Typography, message } from "antd";
import { Typography, message, Select } from "antd";
import {
AudioOutlined,
ThunderboltOutlined,
CheckCircleFilled,
CheckCircleOutlined,
CloseCircleOutlined,
LoadingOutlined,
PlayCircleOutlined,
PauseCircleOutlined,
DownloadOutlined,
ShareAltOutlined,
SaveOutlined,
PlusOutlined,
MinusOutlined,
CloseOutlined,
} from "@ant-design/icons";
import type { AssetItem } from "@/api/assets";
import { getAssets, getAssetLibraries } from "@/api/assets";
import { createEditPlan, generateEditPlan } from "@/api/editPlans";
import { getEditingTemplates } from "@/api/editingPlanner";
import { MODE_LABELS, type TemplateMode } from "@/api/editingPlanner";
import { getTitles } from "@/api/titles";
import apiClient from "@/api/client";
import { fetchPresetVoices } from "@/api/voices";
@@ -54,13 +57,6 @@ const MODE_GRADIENTS: Record<string, string> = {
voice_over: "linear-gradient(135deg, #6366f1, #4f46e5)",
voice_pip: "linear-gradient(135deg, #10b981, #059669)",
};
const MODE_ABBRS: Record<string, string> = {
pip: "PIP",
one_take: "ONE",
voice_over: "VOI",
voice_pip: "VP",
};
/* ── 配音预设卡片:从 API 动态生成,不再硬编码 ── */
const VOICE_GENDER_ICON: Record<string, string> = {
female: "🎀",
@@ -122,6 +118,8 @@ const GeneratePage: React.FC = () => {
/* ── 素材 ── */
const [selectedMaterials, setSelectedMaterials] = useState<string[]>([]);
/* 素材选择模式:手动选择 / 自动匹配 */
const [materialMode, setMaterialMode] = useState<"manual" | "auto">("manual");
/* ── 标题 ── */
const [title, setTitle] = useState("");
@@ -146,6 +144,9 @@ const GeneratePage: React.FC = () => {
);
const [customVoiceText, setCustomVoiceText] = useState("");
/* ── 生成数量 ── */
const [generateCount, setGenerateCount] = useState(1);
/* ── 克隆声音 ── */
const [selectedClonedVoice, setSelectedClonedVoice] = useState<string>("");
const [cloneModalOpen, setCloneModalOpen] = useState(false);
@@ -162,6 +163,7 @@ const GeneratePage: React.FC = () => {
const [generating, setGenerating] = useState(false);
const [progress, setProgress] = useState(0);
const [generated, setGenerated] = useState(false);
const [generateError, setGenerateError] = useState<string | null>(null);
const progressTimer = useRef<ReturnType<typeof setInterval>>(undefined);
const audioRef = useRef<HTMLAudioElement | null>(null);
@@ -456,11 +458,17 @@ const GeneratePage: React.FC = () => {
}, [navigate]);
const handleGenerate = useCallback(async () => {
console.log("[handleGenerate] 开始生成, 参数:", {
title,
selectedTemplate,
selectedMaterials,
voiceMode,
});
if (!title.trim()) {
message.warning("请先选择或输入标题");
return;
}
if (selectedMaterials.length === 0) {
if (materialMode === "manual" && selectedMaterials.length === 0) {
message.warning("请至少选择一个素材");
return;
}
@@ -473,6 +481,7 @@ const GeneratePage: React.FC = () => {
setGenerating(true);
setProgress(0);
setGenerated(false);
setGenerateError(null);
try {
const voiceConfig: Record<string, unknown> = {};
@@ -498,6 +507,8 @@ const GeneratePage: React.FC = () => {
duration,
auto_subtitles: autoSubtitles,
bgm,
generate_count: generateCount,
material_mode: materialMode,
},
total_duration: duration,
source_edit_plan_id: editPlanId || undefined,
@@ -531,6 +542,7 @@ const GeneratePage: React.FC = () => {
)?.error_message ||
"视频生成失败,请联系管理员或重试";
console.error("[生成失败] planId:", plan.id, "响应:", data);
setGenerateError(errorMsg);
message.error(errorMsg);
return;
}
@@ -560,20 +572,36 @@ const GeneratePage: React.FC = () => {
typeof setInterval
>;
} catch (err: unknown) {
console.error("生成失败:", err);
console.error("[handleGenerate] 生成失败:", err);
setGenerating(false);
// 提取 axios 响应中的后端错误信息
const axiosErr = err as {
response?: {
data?: { message?: string; error?: string; detail?: string };
data?: {
message?: string;
error?: string;
detail?: string;
msg?: string;
};
};
message?: string;
};
const backendMsg =
axiosErr.response?.data?.message ||
axiosErr.response?.data?.error ||
axiosErr.response?.data?.detail ||
axiosErr.response?.data?.msg ||
axiosErr.message ||
"";
message.error(backendMsg || "生成失败,请重试");
console.error(
"[handleGenerate] 错误信息:",
backendMsg,
"完整错误:",
axiosErr,
);
const errorMsg = backendMsg || "生成失败,请检查网络后重试或联系管理员";
setGenerateError(errorMsg);
message.error(errorMsg);
}
}, [
title,
@@ -590,6 +618,8 @@ const GeneratePage: React.FC = () => {
bgm,
editPlanId,
selectedTemplate,
generateCount,
materialMode,
]);
/* ── 步骤导航 ── */
@@ -598,7 +628,11 @@ const GeneratePage: React.FC = () => {
message.warning("请先选择一个模板");
return;
}
if (currentStep === 2 && selectedMaterials.length === 0) {
if (
currentStep === 2 &&
materialMode === "manual" &&
selectedMaterials.length === 0
) {
message.warning("请至少选择一个素材");
return;
}
@@ -609,7 +643,13 @@ const GeneratePage: React.FC = () => {
if (currentStep < 5) {
setCurrentStep((s) => s + 1);
}
}, [currentStep, selectedTemplate, selectedMaterials.length, title]);
}, [
currentStep,
selectedTemplate,
selectedMaterials.length,
title,
materialMode,
]);
const goPrev = useCallback(() => {
if (currentStep > 1) {
@@ -669,11 +709,10 @@ const GeneratePage: React.FC = () => {
background: MODE_GRADIENTS[tpl.mode] || MODE_GRADIENTS.pip,
}}
>
{MODE_ABBRS[tpl.mode] || "TPL"}
🎬
</div>
<h4>{tpl.name}</h4>
<p>
{MODE_LABELS[tpl.mode as TemplateMode] || tpl.mode} ·{" "}
{tpl.estimated_duration}s · {tpl.segments.length}
</p>
{tpl.tags.length > 0 && (
@@ -708,11 +747,31 @@ const GeneratePage: React.FC = () => {
</div>
);
/** 步骤 2:选择素材 */
/** 步骤 2:选择素材(双模式:手动选择 / 自动匹配) */
const renderStep2 = () => (
<div className="xx-form-section">
<h3>📦 </h3>
<div className="xx-form-field">
{/* ── 模式切换 Tab ── */}
<div className="xx-material-mode-tabs">
<button
className={`xx-material-mode-tab ${materialMode === "manual" ? "active" : ""}`}
onClick={() => setMaterialMode("manual")}
type="button"
>
</button>
<button
className={`xx-material-mode-tab ${materialMode === "auto" ? "active" : ""}`}
onClick={() => setMaterialMode("auto")}
type="button"
>
</button>
</div>
{/* ── 素材库选择(两种模式共用) ── */}
<div className="xx-form-field" style={{ marginTop: 12 }}>
<label></label>
<select
value={selectedLibraryId}
@@ -725,90 +784,138 @@ const GeneratePage: React.FC = () => {
))}
</select>
</div>
<div
style={{
marginTop: 14,
display: "flex",
alignItems: "center",
gap: 10,
}}
>
<span className="xx-pill xx-pill-ok">
{selectedMaterials.length}
</span>
<Text style={{ color: "var(--text-tertiary, #94a3b8)", fontSize: 13 }}>
</Text>
</div>
{/* 素材列表 */}
<div style={{ marginTop: 14 }}>
{materialsLoading ? (
<Text style={{ color: "var(--text-secondary)", padding: "16px 0" }}>
</Text>
) : materials.length === 0 ? (
<Text style={{ color: "var(--text-secondary)", padding: "16px 0" }}>
</Text>
) : (
<div style={{ display: "flex", flexDirection: "column", gap: 6 }}>
{materials.map((m) => {
const checked = selectedMaterials.includes(m.id);
return (
<label
key={m.id}
style={{
display: "flex",
alignItems: "center",
gap: 10,
padding: "8px 12px",
background: checked
? "var(--primary-soft, #eef2ff)"
: "#f8fafc",
borderRadius: 10,
cursor: "pointer",
border: checked
? "1px solid var(--primary-color, #4f46e5)"
: "1px solid transparent",
transition: "all 0.15s ease",
}}
>
<input
type="checkbox"
checked={checked}
onChange={() => {
setSelectedMaterials((prev) =>
prev.includes(m.id)
? prev.filter((id) => id !== m.id)
: [...prev, m.id],
);
}}
style={{ accentColor: "var(--primary-color, #4f46e5)" }}
/>
<span
style={{
fontSize: 13,
color: "var(--text-primary)",
flex: 1,
}}
>
{m.name}
</span>
<span
style={{
fontSize: 11,
color: "var(--text-tertiary, #94a3b8)",
}}
>
{m.mime_type.split("/")[1].toUpperCase()}
</span>
</label>
);
})}
{/* ── 手动选择模式 ── */}
{materialMode === "manual" && (
<>
<div
style={{
marginTop: 14,
display: "flex",
alignItems: "center",
gap: 10,
}}
>
<span className="xx-pill xx-pill-ok">
{selectedMaterials.length}
</span>
</div>
)}
</div>
{/* 素材列表 */}
<div style={{ marginTop: 14 }}>
{materialsLoading ? (
<Text
style={{ color: "var(--text-secondary)", padding: "16px 0" }}
>
</Text>
) : materials.length === 0 ? (
<Text
style={{ color: "var(--text-secondary)", padding: "16px 0" }}
>
</Text>
) : (
<div style={{ display: "flex", flexDirection: "column", gap: 6 }}>
{materials.map((m) => {
const checked = selectedMaterials.includes(m.id);
return (
<label
key={m.id}
style={{
display: "flex",
alignItems: "center",
gap: 10,
padding: "8px 12px",
background: checked
? "var(--primary-soft, #eef2ff)"
: "#f8fafc",
borderRadius: 10,
cursor: "pointer",
border: checked
? "1px solid var(--primary-color, #4f46e5)"
: "1px solid transparent",
transition: "all 0.15s ease",
}}
>
<input
type="checkbox"
checked={checked}
onChange={() => {
setSelectedMaterials((prev) =>
prev.includes(m.id)
? prev.filter((id) => id !== m.id)
: [...prev, m.id],
);
}}
style={{ accentColor: "var(--primary-color, #4f46e5)" }}
/>
<span
style={{
fontSize: 13,
color: "var(--text-primary)",
flex: 1,
}}
>
{m.name}
</span>
<span
style={{
fontSize: 11,
color: "var(--text-tertiary, #94a3b8)",
}}
>
{m.mime_type.split("/")[1].toUpperCase()}
</span>
</label>
);
})}
</div>
)}
</div>
</>
)}
{/* ── 自动匹配模式 ── */}
{materialMode === "auto" && (
<div className="xx-auto-match-card">
<div className="xx-auto-match-icon">🤖</div>
<div className="xx-auto-match-body">
<h4 className="xx-auto-match-title"></h4>
<p className="xx-auto-match-desc">
AI
</p>
<div className="xx-auto-match-features">
<span className="xx-auto-match-feature">📊 </span>
<span className="xx-auto-match-feature">🎯 </span>
<span className="xx-auto-match-feature"> </span>
</div>
</div>
{materialsLoading ? (
<Text
style={{
color: "var(--text-secondary)",
fontSize: 12,
marginTop: 8,
}}
>
</Text>
) : (
<Text
style={{
color: "var(--text-tertiary, #94a3b8)",
fontSize: 12,
marginTop: 8,
}}
>
{materials.length}
</Text>
)}
</div>
)}
</div>
);
@@ -818,14 +925,30 @@ const GeneratePage: React.FC = () => {
<h3>📝 </h3>
<div className="xx-form-field">
<label></label>
<select value={title} onChange={(e) => setTitle(e.target.value)}>
<option value=""></option>
{userTitles.map((t) => (
<option key={t.id} value={t.content}>
{t.content}
</option>
))}
</select>
<Select
placeholder="请选择标题…"
allowClear
showSearch
style={{ width: "100%" }}
value={title || undefined}
onChange={(val) => setTitle(val || "")}
options={userTitles.map((t) => ({
label: t.content,
value: t.content,
}))}
filterOption={(input, option) =>
((option?.label as string) || "")
.toLowerCase()
.includes(input.toLowerCase())
}
notFoundContent={
userTitles.length === 0 ? (
<span style={{ color: "var(--text-tertiary)", fontSize: 13 }}>
</span>
) : null
}
/>
</div>
<div className="xx-form-field" style={{ marginTop: 14 }}>
<label></label>
@@ -1270,7 +1393,9 @@ const GeneratePage: React.FC = () => {
<div className="xx-summary-row">
<span className="xx-summary-label"></span>
<span className="xx-summary-value">
{selectedMaterials.length}
{materialMode === "auto"
? "自动匹配"
: `${selectedMaterials.length} 个素材`}
</span>
</div>
<div className="xx-summary-row">
@@ -1281,20 +1406,107 @@ const GeneratePage: React.FC = () => {
<span className="xx-summary-label"></span>
<span className="xx-summary-value">{getVoiceName()}</span>
</div>
<div className="xx-summary-row">
<span className="xx-summary-label"></span>
<span className="xx-summary-value">
<div className="xx-count-stepper">
<button
className="xx-count-stepper-btn"
disabled={generateCount <= 1 || generating}
onClick={() => setGenerateCount((c) => Math.max(1, c - 1))}
>
<MinusOutlined />
</button>
<span className="xx-count-stepper-value">{generateCount}</span>
<button
className="xx-count-stepper-btn"
disabled={generateCount >= 10 || generating}
onClick={() => setGenerateCount((c) => Math.min(10, c + 1))}
>
<PlusOutlined />
</button>
<span className="xx-count-stepper-hint"></span>
</div>
</span>
</div>
</div>
{/* 生成进度 */}
{generating && (
{/* 生成进度 / 结果反馈 */}
{(generating || generated || generateError) && (
<div style={{ marginTop: 16 }}>
<div className="xx-progress-bar">
{generating && (
<>
<div className="xx-progress-bar">
<div
className="xx-progress-bar-fill"
style={{ width: `${Math.min(Math.round(progress), 100)}%` }}
/>
</div>
<Text style={{ color: "var(--text-secondary)", fontSize: 13 }}>
<LoadingOutlined style={{ marginRight: 6 }} />
{Math.round(progress)}%
</Text>
</>
)}
{generated && !generating && (
<div
className="xx-progress-bar-fill"
style={{ width: `${Math.min(Math.round(progress), 100)}%` }}
/>
</div>
<Text style={{ color: "var(--text-secondary)", fontSize: 13 }}>
{Math.round(progress)}%
</Text>
style={{
padding: "12px 16px",
borderRadius: 8,
background: "rgba(82, 196, 26, 0.08)",
border: "1px solid rgba(82, 196, 26, 0.3)",
display: "flex",
alignItems: "center",
gap: 8,
}}
>
<CheckCircleOutlined style={{ color: "#52c41a", fontSize: 18 }} />
<div>
<Text
strong
style={{ color: "#52c41a", display: "block", fontSize: 14 }}
>
</Text>
<Text style={{ color: "var(--text-secondary)", fontSize: 12 }}>
</Text>
</div>
</div>
)}
{generateError && !generating && (
<div
style={{
padding: "12px 16px",
borderRadius: 8,
background: "rgba(255, 77, 79, 0.08)",
border: "1px solid rgba(255, 77, 79, 0.3)",
display: "flex",
alignItems: "flex-start",
gap: 8,
}}
>
<CloseCircleOutlined
style={{
color: "#ff4d4f",
fontSize: 18,
marginTop: 2,
flexShrink: 0,
}}
/>
<div>
<Text
strong
style={{ color: "#ff4d4f", display: "block", fontSize: 14 }}
>
</Text>
<Text style={{ color: "var(--text-secondary)", fontSize: 12 }}>
{generateError}
</Text>
</div>
</div>
)}
</div>
)}
</div>
@@ -1405,10 +1617,16 @@ const GeneratePage: React.FC = () => {
<button
className="xx-btn xx-btn-primary"
onClick={handleGenerate}
disabled={generating || generated}
disabled={generating || (generated && !generateError)}
>
<ThunderboltOutlined />
{generating ? "生成中…" : generated ? "已生成" : "✨ 确认生成"}
{generating
? "生成中…"
: generated && !generateError
? "已生成"
: generateError
? "🔄 重新生成"
: "✨ 确认生成"}
</button>
)}
</div>
+138
View File
@@ -1074,3 +1074,141 @@
opacity: 1;
}
}
/* ── 生成数量步进器 ── */
.xx-count-stepper {
display: inline-flex;
align-items: center;
gap: 6px;
}
.xx-count-stepper-btn {
display: inline-flex;
align-items: center;
justify-content: center;
width: 28px;
height: 28px;
border: 1px solid var(--border-primary, #e2e8f0);
border-radius: 8px;
background: var(--bg-surface, #fff);
color: var(--text-secondary, #64748b);
font-size: 13px;
cursor: pointer;
transition: all 0.15s;
}
.xx-count-stepper-btn:hover:not(:disabled) {
border-color: var(--primary-400, #818cf8);
color: var(--primary-600, #4f46e5);
background: var(--primary-50, #eef2ff);
}
.xx-count-stepper-btn:disabled {
opacity: 0.35;
cursor: not-allowed;
}
.xx-count-stepper-value {
min-width: 24px;
text-align: center;
font-size: 16px;
font-weight: 600;
color: var(--text-primary, #1e293b);
}
.xx-count-stepper-hint {
font-size: 12px;
color: var(--text-tertiary, #94a3b8);
margin-left: 2px;
}
/* ── 素材选择模式切换 Tab ── */
.xx-material-mode-tabs {
display: flex;
gap: 0;
border: 1px solid var(--border-primary, #e2e8f0);
border-radius: 10px;
overflow: hidden;
margin-bottom: 4px;
}
.xx-material-mode-tab {
flex: 1;
padding: 10px 16px;
font-size: 13px;
font-weight: 500;
border: none;
background: var(--bg-surface, #fff);
color: var(--text-secondary, #64748b);
cursor: pointer;
transition: all 0.2s ease;
text-align: center;
}
.xx-material-mode-tab:first-child {
border-right: 1px solid var(--border-primary, #e2e8f0);
}
.xx-material-mode-tab:hover {
background: var(--primary-50, #eef2ff);
color: var(--primary-600, #4f46e5);
}
.xx-material-mode-tab.active {
background: var(--primary-500, #6366f1);
color: #fff;
font-weight: 600;
}
/* ── 自动匹配卡片 ── */
.xx-auto-match-card {
margin-top: 14px;
padding: 20px;
background: linear-gradient(135deg, #f0f4ff 0%, #faf5ff 100%);
border: 1px solid var(--border-primary, #e2e8f0);
border-radius: 14px;
display: flex;
flex-direction: column;
align-items: center;
text-align: center;
}
.xx-auto-match-icon {
font-size: 36px;
margin-bottom: 10px;
}
.xx-auto-match-body {
width: 100%;
}
.xx-auto-match-title {
font-size: 15px;
font-weight: 600;
color: var(--text-primary, #1e293b);
margin: 0 0 8px;
}
.xx-auto-match-desc {
font-size: 13px;
color: var(--text-secondary, #64748b);
line-height: 1.6;
margin: 0 0 14px;
}
.xx-auto-match-features {
display: flex;
flex-wrap: wrap;
justify-content: center;
gap: 8px;
}
.xx-auto-match-feature {
display: inline-block;
padding: 4px 12px;
font-size: 12px;
color: var(--primary-600, #4f46e5);
background: rgba(255, 255, 255, 0.8);
border: 1px solid var(--border-light, #f1f5f9);
border-radius: 20px;
}
+29 -1
View File
@@ -737,11 +737,39 @@ const ProductLibrary: React.FC = () => {
// ── Error 状态 ──
if (isError) {
console.error("[ProductLibrary] 加载失败:", error);
const errorMsg = error?.message || "加载失败";
// 404 视为空数据(API 尚未就绪或无数据)
const is404 = errorMsg.includes("404") || errorMsg.includes("Not Found");
if (is404) {
return (
<div className="xx-products-page">
<div className="xx-products-header">
<h2>
<VideoCameraOutlined />
</h2>
</div>
<div className="xx-products-empty">
<div className="xx-products-empty-icon">🎬</div>
<p></p>
<p
style={{
fontSize: 12,
color: "var(--text-tertiary)",
marginTop: 4,
}}
>
</p>
</div>
</div>
);
}
return (
<div className="xx-products-page">
<div className="xx-products-empty">
<div className="xx-products-empty-icon"></div>
<p>{error?.message || "加载失败"}</p>
<p>{errorMsg || "加载失败,请稍后重试"}</p>
<Button
buttonType="primary"
buttonSize="sm"
@@ -91,7 +91,7 @@ const mapTemplateItemToEditTemplate = (item: TemplateItem): EditTemplate => ({
id: item.id,
name: item.name,
type: inferTemplateType(item.category),
description: item.description,
description: item.description ?? "",
usageCount: 0,
isFavorite: item.is_favorite ?? false,
thumbnailGradient: gradientForCategory(item.category),
@@ -370,9 +370,10 @@ const TemplateCard: React.FC<TemplateCardProps> = ({
className="xx-template-thumb-bg"
style={{ background: template.thumbnailGradient }}
>
{template.description.slice(0, 80)}...
{(template.description ?? "").slice(0, 80)}...
</div>
<div className="xx-template-thumb-overlay" />
<div className="xx-template-thumb-name">{template.name}</div>
<div className="xx-template-preview-hint"></div>
<button
className={`xx-template-fav-btn${isFavorite ? " is-favorite" : ""}`}
@@ -386,7 +387,6 @@ const TemplateCard: React.FC<TemplateCardProps> = ({
{/* 信息区 */}
<div className="xx-template-info">
<div className="xx-template-info-top">
<h4 className="xx-template-name">{template.name}</h4>
<span
className="xx-template-category-pill"
style={{
@@ -397,7 +397,7 @@ const TemplateCard: React.FC<TemplateCardProps> = ({
{template.type}
</span>
</div>
<p className="xx-template-desc">{template.description}</p>
<p className="xx-template-desc">{template.description ?? ""}</p>
<div className="xx-template-meta">
<span className="xx-template-usage">
使 {template.usageCount}
@@ -483,7 +483,9 @@ const TemplateLibrary: React.FC = () => {
const matchSearch =
!searchText ||
t.name.toLowerCase().includes(searchText.toLowerCase()) ||
t.description.toLowerCase().includes(searchText.toLowerCase()) ||
(t.description ?? "")
.toLowerCase()
.includes(searchText.toLowerCase()) ||
t.tags.some((tag) =>
tag.toLowerCase().includes(searchText.toLowerCase()),
);
@@ -233,6 +233,24 @@
pointer-events: none;
}
/* 缩略图底部名称 */
.xx-template-thumb-name {
position: absolute;
bottom: 0;
left: 0;
right: 0;
padding: 24px 14px 10px;
background: linear-gradient(0deg, rgba(0, 0, 0, 0.55) 0%, transparent 100%);
color: #fff;
font-size: 14px;
font-weight: 600;
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
z-index: 1;
pointer-events: none;
}
/* 预览提示(hover 显示) */
.xx-template-preview-hint {
position: absolute;
@@ -35,6 +35,8 @@ import {
CheckOutlined,
TagsOutlined,
MutedOutlined,
RobotOutlined,
LoadingOutlined,
} from "@ant-design/icons";
import { Button, Input, Select, Modal, Tag } from "@/components/ui";
import { message, Popover, Popconfirm, Tooltip } from "antd";
@@ -56,6 +58,8 @@ import {
tagAsset,
untagAsset,
} from "@/api/tags";
import { synthesizeSpeech, getTTSJobStatus, saveTtsToLibrary } from "@/api/tts";
import { fetchPresetVoices, type PresetVoiceItem } from "@/api/voices";
import "./voice-materials.css";
/* ============================================================
@@ -1086,6 +1090,14 @@ const VoiceMaterialLibrary: React.FC = () => {
[assets],
);
// ── 获取预设音色列表(AI 配音用) ─────────────────────────
const { data: presetVoicesData } = useQuery({
queryKey: ["preset-voices"],
queryFn: fetchPresetVoices,
staleTime: 60_000,
});
const presetVoices: PresetVoiceItem[] = presetVoicesData?.items ?? [];
// ── 上传 mutation ─────────────────────────────────────────
const uploadMutation = useMutation({
mutationFn: async (data: {
@@ -1218,6 +1230,19 @@ const VoiceMaterialLibrary: React.FC = () => {
);
const [batchCustomTag, setBatchCustomTag] = useState("");
// ── AI 配音(TTS 合成)状态 ────────────────────────────────
const [ttsOpen, setTtsOpen] = useState(false);
const [ttsText, setTtsText] = useState("");
const [ttsVoiceId, setTtsVoiceId] = useState<string>("");
const [ttsSpeed, setTtsSpeed] = useState(1.0);
const [ttsJobId, setTtsJobId] = useState<string | null>(null);
const [ttsStatus, setTtsStatus] = useState<
"idle" | "synthesizing" | "done" | "error"
>("idle");
const [ttsAudioUrl, setTtsAudioUrl] = useState<string | null>(null);
const [ttsError, setTtsError] = useState<string | null>(null);
const ttsTimerRef = useRef<ReturnType<typeof setInterval> | null>(null);
// ── 播放控制 ──────────────────────────────────────────────
const stopPlayback = useCallback(() => {
if (audioRef.current) {
@@ -1505,6 +1530,77 @@ const VoiceMaterialLibrary: React.FC = () => {
[tags, createTagMutation, handleBatchTag],
);
// ── TTS 合成处理 ─────────────────────────────────────────
/** 开始 AI 配音合成 */
const handleTtsSynthesize = useCallback(async () => {
if (!ttsText.trim()) {
message.warning("请输入要合成的文本");
return;
}
setTtsError(null);
setTtsStatus("synthesizing");
setTtsAudioUrl(null);
setTtsJobId(null);
try {
const resp = await synthesizeSpeech({
text: ttsText.trim(),
voice_id: ttsVoiceId || undefined,
speed: ttsSpeed,
});
setTtsJobId(resp.job_id);
// 轮询任务状态
ttsTimerRef.current = setInterval(async () => {
try {
const job = await getTTSJobStatus(resp.job_id);
if (job.status === "completed") {
clearInterval(ttsTimerRef.current!);
ttsTimerRef.current = null;
setTtsStatus("done");
setTtsAudioUrl(job.output_audio_url);
} else if (job.status === "failed") {
clearInterval(ttsTimerRef.current!);
ttsTimerRef.current = null;
setTtsStatus("error");
setTtsError(job.error_message || "合成失败");
}
} catch {
clearInterval(ttsTimerRef.current!);
ttsTimerRef.current = null;
setTtsStatus("error");
setTtsError("查询合成状态失败");
}
}, 2000);
} catch (err: unknown) {
const msg = err instanceof Error ? err.message : "合成请求失败";
setTtsStatus("error");
setTtsError(msg);
}
}, [ttsText, ttsVoiceId, ttsSpeed]);
/** 保存 TTS 结果到素材库 */
const handleTtsSave = useCallback(async () => {
if (!ttsJobId) return;
try {
await saveTtsToLibrary(ttsJobId, {
name: ttsText.slice(0, 20) || "AI配音",
});
message.success("已保存到配音素材库");
queryClient.invalidateQueries({ queryKey: ["assets", "voice"] });
setTtsOpen(false);
} catch {
message.error("保存失败");
}
}, [ttsJobId, ttsText, queryClient]);
// TTS 定时器清理
useEffect(() => {
return () => {
if (ttsTimerRef.current) clearInterval(ttsTimerRef.current);
};
}, []);
/* ── 渲染 ─────────────────────────────────────────────── */
const isUploading = uploadMutation.isPending;
@@ -1512,6 +1608,13 @@ const VoiceMaterialLibrary: React.FC = () => {
const pageActions = (
<div className="vmat-page-actions">
<Button
buttonSize="sm"
icon={<RobotOutlined />}
onClick={() => setTtsOpen(true)}
>
AI配音
</Button>
<Button
buttonType="primary"
buttonSize="sm"
@@ -1818,6 +1921,182 @@ const VoiceMaterialLibrary: React.FC = () => {
/>
)}
</Modal>
{/* AI 配音(TTS 合成)弹窗 */}
<Modal
title="AI 配音"
open={ttsOpen}
onCancel={() => {
setTtsOpen(false);
if (ttsTimerRef.current) {
clearInterval(ttsTimerRef.current);
ttsTimerRef.current = null;
}
setTtsStatus("idle");
setTtsAudioUrl(null);
setTtsError(null);
setTtsJobId(null);
}}
footer={null}
width={560}
destroyOnClose
>
<div style={{ display: "flex", flexDirection: "column", gap: 16 }}>
{/* 文本输入 */}
<div>
<label
style={{
fontSize: 13,
fontWeight: 500,
marginBottom: 6,
display: "block",
}}
>
</label>
<textarea
rows={4}
placeholder="请输入需要转换为语音的文本内容…"
value={ttsText}
onChange={(e) => setTtsText(e.target.value)}
maxLength={2000}
style={{
width: "100%",
padding: "8px 12px",
border: "1px solid var(--border-color, #d9d9d9)",
borderRadius: 6,
fontSize: 13,
resize: "vertical",
fontFamily: "inherit",
}}
/>
<div
style={{
fontSize: 11,
color: "var(--text-tertiary, #999)",
marginTop: 4,
textAlign: "right",
}}
>
{ttsText.length}/2000
</div>
</div>
{/* 音色选择 */}
<div>
<label
style={{
fontSize: 13,
fontWeight: 500,
marginBottom: 6,
display: "block",
}}
>
</label>
<select
value={ttsVoiceId}
onChange={(e) => setTtsVoiceId(e.target.value)}
style={{
width: "100%",
height: 36,
padding: "0 10px",
border: "1px solid var(--border-color, #d9d9d9)",
borderRadius: 6,
fontSize: 13,
background: "var(--bg-primary, #fff)",
}}
>
<option value=""></option>
{presetVoices.map((v) => (
<option key={v.voice_id} value={v.voice_id}>
{v.name}
</option>
))}
</select>
</div>
{/* 语速调节 */}
<div>
<label
style={{
fontSize: 13,
fontWeight: 500,
marginBottom: 6,
display: "block",
}}
>
{ttsSpeed.toFixed(1)}x
</label>
<input
type="range"
min={0.5}
max={2.0}
step={0.1}
value={ttsSpeed}
onChange={(e) => setTtsSpeed(parseFloat(e.target.value))}
style={{ width: "100%" }}
/>
</div>
{/* 合成按钮 */}
<Button
buttonType="primary"
buttonSize="md"
icon={
ttsStatus === "synthesizing" ? (
<LoadingOutlined />
) : (
<RobotOutlined />
)
}
onClick={handleTtsSynthesize}
disabled={ttsStatus === "synthesizing" || !ttsText.trim()}
>
{ttsStatus === "synthesizing" ? "合成中…" : "开始合成"}
</Button>
{/* 错误提示 */}
{ttsStatus === "error" && ttsError && (
<div
style={{
padding: "8px 12px",
background: "#fff2f0",
borderRadius: 6,
color: "#ff4d4f",
fontSize: 13,
}}
>
{ttsError}
</div>
)}
{/* 合成结果 */}
{ttsStatus === "done" && ttsAudioUrl && (
<div
style={{
padding: 12,
background: "var(--bg-surface, #f5f5f5)",
borderRadius: 8,
}}
>
<audio
controls
src={ttsAudioUrl}
style={{ width: "100%", marginBottom: 12 }}
/>
<Button
buttonType="primary"
buttonSize="sm"
icon={<PlusOutlined />}
onClick={handleTtsSave}
>
</Button>
</div>
)}
</div>
</Modal>
</div>
);
};
+6
View File
@@ -11,6 +11,12 @@ export default defineConfig({
globals: true,
environment: "jsdom",
setupFiles: "./src/test/setup.ts",
exclude: [
"node_modules",
"e2e",
"dist",
"build",
],
coverage: {
provider: "v8",
reporter: ["text", "json", "html"],
+62
View File
@@ -206,6 +206,68 @@ class VideoDeduplicator:
return None
def check_batch_duplicate(
self,
fingerprint: VideoFingerprint,
batch_id: str,
current_video_id: str,
session: Session,
) -> Optional[dict]:
"""检查视频是否与同批次内其他视频重复。
逻辑与 check_duplicate 一致(MD5 + pHash),但搜索范围限定为同 batch_id 的视频。
Args:
fingerprint: 待检测视频的指纹
batch_id: 批次 ID
current_video_id: 当前视频 ID(排除自身)
session: 数据库会话
Returns:
重复信息字典,或 None 表示未找到重复
"""
video_repo = SQLAlchemyGeneratedVideoRepository(session)
batch_videos = video_repo.list_by_batch(batch_id)
for existing in batch_videos:
if existing.id == current_video_id:
continue
if not existing.video_fingerprint:
continue
ef = existing.video_fingerprint
if fingerprint.md5 == ef.get("md5"):
return {
"duplicate": True,
"duplicate_of": existing.id,
"reason": "batch_exact_md5_match",
"similarity": 1.0,
}
existing_phashes = ef.get("keyframe_phashes", [])
if not existing_phashes:
continue
min_distances = []
for phash in fingerprint.keyframe_phashes:
distances = [hamming_distance(phash, ep) for ep in existing_phashes]
min_distances.append(min(distances))
avg_distance = sum(min_distances) / len(min_distances) if min_distances else 100
if avg_distance >= self.PHASH_THRESHOLD:
continue
phash_similarity = 1.0 - (avg_distance / 64)
return {
"duplicate": True,
"duplicate_of": existing.id,
"reason": "batch_phash_similar",
"similarity": phash_similarity,
}
return None
@staticmethod
def _average_histogram_similarity(histograms_a: list[list[float]], histograms_b: list[list[float]]) -> float:
"""
+123 -12
View File
@@ -139,14 +139,16 @@ def _download_library_assets(
asset_library_id: str,
temp_path: Path,
video_extensions: tuple = (".mp4", ".mov", ".avi", ".mkv", ".webm"),
asset_ids: list[str] | None = None,
) -> list[str]:
"""
从素材库下载所有视频素材
从素材库下载视频素材
Args:
asset_library_id: 素材库 ID
temp_path: 临时目录路径
video_extensions: 支持的视频扩展名
asset_ids: 指定素材 ID 列表,为空则下载全部 ready 视频素材
Returns:
下载成功的视频文件路径列表
@@ -161,16 +163,15 @@ def _download_library_assets(
try:
# 查询素材库中的视频素材
assets = (
session.query(AssetModel)
.filter(
AssetModel.asset_library_id == asset_library_id,
AssetModel.status == "ready",
AssetModel.file_type.in_(["video", "video/mp4", "video/quicktime"]),
)
.order_by(AssetModel.created_at)
.all()
query = session.query(AssetModel).filter(
AssetModel.asset_library_id == asset_library_id,
AssetModel.status == "ready",
AssetModel.file_type.in_(["video", "video/mp4", "video/quicktime"]),
)
# 如果指定了 asset_ids,则只下载这些素材
if asset_ids:
query = query.filter(AssetModel.id.in_(asset_ids))
assets = query.order_by(AssetModel.created_at).all()
if not assets:
logger.info(f"No video assets found in library {asset_library_id}")
@@ -259,6 +260,8 @@ def generate_video(self, task_id: str) -> dict:
asset_library_id = gen_task.asset_library_id
voice_library_id = gen_task.voice_library_id or ""
mode = gen_task.strategy_id or "one_take"
task_asset_ids = list(gen_task.asset_ids or [])
batch_id = getattr(gen_task, "batch_id", "") or ""
finally:
session.close()
@@ -275,8 +278,8 @@ def generate_video(self, task_id: str) -> dict:
temp_path = Path(temp_dir)
output_path = temp_path / output_name
# 从素材库下载视频素材
downloaded_videos = _download_library_assets(asset_library_id, temp_path)
# 从素材库下载视频素材(如果任务指定了 asset_ids 则只下载这些)
downloaded_videos = _download_library_assets(asset_library_id, temp_path, asset_ids=task_asset_ids or None)
audio_path = None
if voice_library_id:
@@ -297,6 +300,32 @@ def generate_video(self, task_id: str) -> dict:
file_size = output_path.stat().st_size
duration = _probe_duration(output_path)
# 上传到 OSS
bucket = _oss_bucket()
if bucket:
try:
bucket.put_object_from_file(storage_key, str(output_path))
except Exception as oss_err:
logger.warning(f"OSS upload failed: {oss_err}")
# 构建视频 URL
if bucket:
file_url = f"{PUBLIC_API_BASE_URL}/{storage_key}"
else:
file_url = f"{GENERATED_FILES_URL_PREFIX}/{task_id}/{output_name}"
# 创建 GeneratedVideo 记录 + 查重
_create_video_record_and_dedup(
task_id=task_id,
project_id=project_id,
batch_id=batch_id,
file_url=file_url,
file_size=file_size,
duration=duration,
video_path=str(output_path),
mode=editing_mode.value,
)
return {
"status": "completed",
"task_id": task_id,
@@ -314,3 +343,85 @@ def generate_video(self, task_id: str) -> dict:
"task_id": task_id,
"error": str(error),
}
def _create_video_record_and_dedup(
*,
task_id: str,
project_id: str,
batch_id: str,
file_url: str,
file_size: int,
duration: float,
video_path: str,
mode: str,
) -> None:
"""创建 GeneratedVideo 记录,计算指纹并执行查重(历史 + 批次)。"""
from uuid import uuid4
from video_processing.dedup import VideoDeduplicator
from worker_app.db import SessionLocal
from packages.adapters.sqlalchemy_impl.generated_video_repository import (
SQLAlchemyGeneratedVideoRepository,
)
from packages.domain import GeneratedVideo
session = SessionLocal()
try:
video_id = uuid4().hex
generated_video = GeneratedVideo(
id=video_id,
project_id=project_id,
generation_task_id=task_id,
name=f"generated-{task_id[:8]}.mp4",
file_url=file_url,
file_size=file_size,
duration=duration,
width=OUTPUT_WIDTH,
height=OUTPUT_HEIGHT,
fps=OUTPUT_FPS,
status="completed",
generation_params={"mode": mode},
)
video_repo = SQLAlchemyGeneratedVideoRepository(session)
video_repo.create(generated_video)
# 计算视频指纹
deduplicator = VideoDeduplicator()
try:
fingerprint = deduplicator.compute_fingerprint(video_path)
except Exception as fp_err:
logger.warning(f"Fingerprint computation failed for {video_id}: {fp_err}")
session.commit()
return
generated_video.video_fingerprint = fingerprint.to_dict()
# (a) 历史成片查重
duplicate_result = deduplicator.check_duplicate(fingerprint, project_id, session)
# (b) 批次内查重(仅当有 batch_id 时)
if not duplicate_result and batch_id:
duplicate_result = deduplicator.check_batch_duplicate(fingerprint, batch_id, video_id, session)
if duplicate_result:
generated_video.is_duplicate = True
generated_video.duplicate_of = duplicate_result["duplicate_of"]
logger.info(
f"Duplicate detected: {video_id} -> {duplicate_result['duplicate_of']} "
f"(reason={duplicate_result['reason']}, similarity={duplicate_result['similarity']:.3f})"
)
else:
generated_video.is_duplicate = False
generated_video.duplicate_of = None
video_repo.update(generated_video)
session.commit()
logger.info(f"GeneratedVideo record created: {video_id} (task={task_id}, dup={generated_video.is_duplicate})")
except Exception as e:
logger.error(f"Failed to create video record / dedup for task {task_id}: {e}")
session.rollback()
finally:
session.close()
+1
View File
@@ -179,6 +179,7 @@ def ingest_asset(job_id: str) -> dict:
width=int(metadata.get("width", 0)),
height=int(metadata.get("height", 0)),
status=AssetStatus.READY,
file_hash=job.file_hash,
)
asset_repo.create(asset)
+140
View File
@@ -0,0 +1,140 @@
# 端口分配清单
> 本文档梳理 xiaoxia-saas 项目中所有服务、容器及 CI 环境使用的端口,
> 作为运维、排障和新功能开发时的统一参考。
>
> 最后更新:2026-07-24
---
## 一、应用服务端口
| 服务 | 容器内端口 | 环境变量名 | Staging 宿主机 | Production 宿主机 | 说明 |
| -------- | ---------- | ---------------- | -------------- | ----------------- | ----------------------------------- |
| API | 8000 | `API_PORT` | 8000 | 8001 | FastAPI 服务,Nginx 反代后端 |
| Web | 80 | `WEB_PORT` | 3001 | 3002 | Nginx + 前端静态文件 |
| Worker | — | — | — | — | Celery 任务队列,不暴露端口 |
### 补充说明
- API 容器内部固定监听 8000`API_HOST=0.0.0.0``API_PORT=8000`
- Web 容器内部 Nginx 固定监听 80
- 所有端口均绑定 `127.0.0.1`,不直接暴露公网,由前置 Nginx/CDN 转发
---
## 二、基础设施端口
### PostgreSQL
| 环境 | 容器内端口 | 宿主机映射 | 环境变量名 | 默认值 |
| ------------ | ---------- | ---------- | --------------------- | -------- |
| Production | 5432 | 5433 | `POSTGRES_PORT` | 5433 |
| Staging | 5432 | 5434 | `POSTGRES_PORT` | 5434 |
| 开发本地 | 5432 | 5432 | `DATABASE_URL` 中端口 | 5432 |
| CI 共享 PG | 5432 | 5433 | `CI_SHARED_PG_PORT` | 5433 |
| CI 本地 PG | 5432 | 5432 | `CI_LOCAL_PG_PORT` | 5432 |
### Redis
| 环境 | 容器内端口 | 宿主机映射 | 环境变量名 | 默认值 |
| ------------ | ---------- | ---------- | ------------------- | -------- |
| Production | 6379 | 6380 | `REDIS_URL` 中端口 | — |
| Staging | 6379 | 6381 | `REDIS_URL` 中端口 | — |
| 开发本地 | 6379 | 6379 | `REDIS_URL` | 6379 |
| CI 动态创建 | 6379 | 随机 | 运行时 `REDIS_PORT` | — |
> CI Integration Tests 中 Redis 容器使用 `-P` 随机映射端口,
> 通过 `docker port` 命令获取实际端口后写入 `REDIS_URL`。
### 容器镜像 Registry
| 服务 | 端口 | 地址 | 说明 |
| ----------------- | ----- | ---------------------- | ------------------------------ |
| Gitea Registry | 5000 | 172.30.18.198:5000 | CI 构建服务器内网 Registry |
| ACR(生产镜像源) | 443 | crpi-xxx.aliyuncs.com | 阿里云容器镜像服务(HTTPS) |
---
## 三、CI / DevOps 端口
| 服务/用途 | 端口 | 环境变量名 | 默认值 | 说明 |
| ------------------- | ----- | --------------------- | ------ | ------------------------------------- |
| CI ChatOps Webhook | 8090 | `CHATOPS_WEBHOOK_PORT`| 8090 | Gitea webhook 接收服务(`scripts/ci/chatops/` |
| Staging SSH 部署 | 22222 | `STAGING_SSH_PORT` | 22222 | Staging 服务器 SSH 端口(secrets 配置) |
| Preview SSH 部署 | 22222 | `PREVIEW_SSH_PORT` | 22222 | Preview 服务器 SSH 端口(secrets 配置) |
| Preview 前端访问 | 80 | — | 80 | Nginx 子域名路由,`*.preview.xiaoxiajianji.com` |
---
## 四、开发环境默认端口(.env.example
| 用途 | 端口 | 环境变量名 / 出处 |
| ------------ | ----- | ------------------------------------------ |
| API 服务 | 8000 | `API_PORT` |
| 数据库 | 5432 | `DATABASE_URL``postgresql+psycopg://...:5432/...` |
| Redis | 6379 | `REDIS_URL` / `CELERY_BROKER_URL` / `CELERY_RESULT_BACKEND` |
| SMTP | 587 | `SMTP_PORT` |
| 前端开发服务 | 3000 | `APP_BASE_URL`(默认 localhost:3000 |
| Vite Dev | 5173 | `CORS_ORIGINS_RAW` 中包含 |
---
## 五、CI Workflow 中的端口变量
### ci-pipeline.yml 顶层 env
| 变量名 | 默认值 | 用途 |
| ------------------- | ------ | ------------------------ |
| `CI_PG_PORT` | 5432 | CI PG 容器端口(本地) |
| `CI_SHARED_PG_PORT` | 5433 | CI 共享常驻 PG 端口 |
### scripts/ci/ci_env.sh(统一常量)
| 变量名 | 默认值 | 说明 |
| ------------------- | ----------- | ----------------------------- |
| `CI_SHARED_PG_PORT` | 5433 | 共享常驻 PG 实例端口 |
| `CI_LOCAL_PG_PORT` | 5432 | 本地 PG 容器默认端口 |
| `CI_DEFAULT_DB` | xiaoxia_saas | 默认数据库名 |
---
## 六、命名规范
### 推荐命名格式
统一使用 `{服务/用途}_PORT` 格式:
```bash
API_PORT # 应用服务
WEB_PORT # 应用服务
POSTGRES_PORT # 基础设施
REDIS_PORT # 基础设施
SMTP_PORT # 外部服务
CI_SHARED_PG_PORT # CI 特定
CI_LOCAL_PG_PORT # CI 特定
CHATOPS_WEBHOOK_PORT # DevOps 服务
```
### 历史命名不一致(待统一)
- `WEBHOOK_PORT`chatops config.py 内部变量)→ 应与外部 env 名 `CHATOPS_WEBHOOK_PORT` 对齐
- `STAGING_SSH_PORT` / `PREVIEW_SSH_PORT` → 符合规范,保留
- `CI_PG_PORT`(workflow 中)→ 建议统一为 `CI_LOCAL_PG_PORT``ci_env.sh` 对齐
---
## 七、相关配置文件路径
| 文件路径 | 端口相关内容 |
| ------------------------------------- | -------------------------------- |
| `infra/docker/compose.yml` | API / Web / Worker 端口映射 |
| `infra/docker/infra.yml` | Staging PG / Redis 端口 |
| `infra/docker/infra-production.yml` | Production PG / Redis 端口 |
| `.env.example` | 开发环境全部端口变量 |
| `.gitea/workflows/ci-pipeline.yml` | CI PG 端口配置 |
| `scripts/ci/ci_env.sh` | CI 端口统一常量 |
| `scripts/ci/chatops/config.py` | ChatOps Webhook 端口 |
| `scripts/ci/run_integration_tests.sh` | Redis 动态端口 + PG 端口 |
| `scripts/ci/run_validate.sh` | PG 端口 |
| `scripts/ci/validate_migration.sh` | PG 端口 |
+53
View File
@@ -274,6 +274,14 @@
"type": "VARCHAR(36)",
"unique": false
},
{
"index": true,
"name": "file_hash",
"nullable": true,
"primary_key": false,
"type": "VARCHAR(64)",
"unique": false
},
{
"index": false,
"name": "metadata",
@@ -321,6 +329,13 @@
"name": "ix_assets_created_at",
"unique": false
},
{
"columns": [
"file_hash"
],
"name": "ix_assets_file_hash",
"unique": false
},
{
"columns": [
"file_type"
@@ -1502,6 +1517,22 @@
"type": "VARCHAR(32)",
"unique": false
},
{
"index": false,
"name": "asset_select_mode",
"nullable": false,
"primary_key": false,
"type": "VARCHAR(20)",
"unique": false
},
{
"index": true,
"name": "batch_id",
"nullable": false,
"primary_key": false,
"type": "VARCHAR(32)",
"unique": false
},
{
"index": false,
"name": "metadata",
@@ -1527,6 +1558,13 @@
"name": "ix_generation_tasks_asset_library_id",
"unique": false
},
{
"columns": [
"batch_id"
],
"name": "ix_generation_tasks_batch_id",
"unique": false
},
{
"columns": [
"created_by_user_id"
@@ -1632,6 +1670,14 @@
"type": "VARCHAR(32)",
"unique": false
},
{
"index": true,
"name": "file_hash",
"nullable": true,
"primary_key": false,
"type": "VARCHAR(64)",
"unique": false
},
{
"index": false,
"name": "created_at",
@@ -1650,6 +1696,13 @@
}
],
"indexes": [
{
"columns": [
"file_hash"
],
"name": "ix_ingest_jobs_file_hash",
"unique": false
},
{
"columns": [
"library_id"
+1 -1
View File
@@ -4,7 +4,7 @@
# ============================================================
# 基础镜像:Python 3.12
FROM python:3.12-slim-bookworm
FROM xiaoxia-registry.cn-hangzhou.cr.aliyuncs.com/xiaoxiakeji/base/python:3.12-slim-bookworm
# 使用阿里云镜像加速
RUN sed -i 's|deb.debian.org|mirrors.aliyun.com|g' /etc/apt/sources.list.d/debian.sources 2>/dev/null || \
+1 -1
View File
@@ -1,4 +1,4 @@
FROM python:3.12-slim
FROM xiaoxia-registry.cn-hangzhou.cr.aliyuncs.com/xiaoxiakeji/base/python:3.12-slim
ENV PIP_DISABLE_PIP_VERSION_CHECK=1 \
PIP_NO_CACHE_DIR=0 \
+240
View File
@@ -0,0 +1,240 @@
#!/bin/sh
set -eu
# ============================================
# Production 部署脚本 - Registry 方式
# 用法:IMAGE_TAG=<version> REGISTRY_TOKEN=<token> sh deploy-production-registry.sh
# ============================================
IMAGE_TAG="${IMAGE_TAG:-}"
REGISTRY="${REGISTRY:-git.xiaoxiajianji.com/xiaoxia/xiaoxia-saas}"
REGISTRY_USER="${REGISTRY_USER:-xiaoxia}"
REGISTRY_TOKEN="${REGISTRY_TOKEN:-}"
ENV_FILE="${ENV_FILE:-/var/lib/xiaoxia-saas-production/.env}"
GENERATED_DIR="${GENERATED_DIR:-/var/lib/xiaoxia-saas-production/generated}"
LEGACY_ASSETS_DIR="${LEGACY_ASSETS_DIR:-/var/lib/xiaoxia-saas-production/legacy-assets}"
if [ -z "$IMAGE_TAG" ]; then
echo "ERROR: IMAGE_TAG is required"
exit 1
fi
test -f "$ENV_FILE"
mkdir -p "$GENERATED_DIR"
mkdir -p "$LEGACY_ASSETS_DIR"
# ---- 登录 Registry ----
if [ -n "$REGISTRY_TOKEN" ]; then
echo "Logging in to registry: $REGISTRY"
REGISTRY_HOST=$(echo "$REGISTRY" | cut -d/ -f1)
printf %s "$REGISTRY_TOKEN" | docker login "$REGISTRY_HOST" -u "$REGISTRY_USER" --password-stdin 2>/dev/null || {
echo "WARN: docker login failed, will try to pull anyway"
}
fi
# ---- Pull 三镜像 ----
REGISTRY_API="${REGISTRY}/xiaoxia-saas-api:${IMAGE_TAG}"
REGISTRY_WORKER="${REGISTRY}/xiaoxia-saas-worker:${IMAGE_TAG}"
REGISTRY_WEB="${REGISTRY}/xiaoxia-saas-web:${IMAGE_TAG}"
LOCAL_API="xiaoxia-saas-api:${IMAGE_TAG}"
LOCAL_WORKER="xiaoxia-saas-worker:${IMAGE_TAG}"
LOCAL_WEB="xiaoxia-saas-web:${IMAGE_TAG}"
echo "Pulling API image..."
docker pull "$REGISTRY_API"
echo "Pulling Worker image..."
docker pull "$REGISTRY_WORKER"
echo "Pulling Web image..."
docker pull "$REGISTRY_WEB"
# ---- Re-tag 成本地名 ----
docker tag "$REGISTRY_API" "$LOCAL_API"
docker tag "$REGISTRY_WORKER" "$LOCAL_WORKER"
docker tag "$REGISTRY_WEB" "$LOCAL_WEB"
echo "All images pulled and tagged."
# ---- 备份旧版 assets(部署期间缓存用户不 404 ----
echo "Backing up legacy assets from current web container..."
if docker inspect xiaoxia-web-production >/dev/null 2>&1; then
_tmpdir="/tmp/legacy-assets-$$"
rm -rf "$_tmpdir"
mkdir -p "$_tmpdir"
docker cp xiaoxia-web-production:/usr/share/nginx/html/assets/. "$_tmpdir/" 2>/dev/null || true
# 合并到 LEGACY_ASSETS_DIR(保留所有历史版本的 assets)
if [ -d "$_tmpdir" ] && [ "$(ls -A "$_tmpdir" 2>/dev/null)" ]; then
cp -an "$_tmpdir"/. "$LEGACY_ASSETS_DIR"/ 2>/dev/null || true
echo "Legacy assets backed up: $(ls "$_tmpdir" | wc -l) files"
fi
rm -rf "$_tmpdir"
else
echo "No existing web container, skipping legacy assets backup"
fi
# 清理超过 7 天的旧 assets 文件(避免无限增长)
if [ -d "$LEGACY_ASSETS_DIR" ]; then
find "$LEGACY_ASSETS_DIR" -type f -mtime +7 -delete 2>/dev/null || true
echo "Legacy assets cleanup done (retain 7 days)"
fi
# ---- 确保基础设施容器在运行 ----
echo "Checking infrastructure containers..."
for c in xiaoxia-postgres-production xiaoxia-redis-production; do
if ! docker inspect "$c" >/dev/null 2>&1; then
echo "ERROR: Required container not found: $c"
exit 1
fi
state=$(docker inspect -f '{{.State.Status}}' "$c")
if [ "$state" != "running" ]; then
echo "ERROR: Container not running: $c ($state)"
exit 1
fi
done
# ---- 确保生产网络存在 ----
docker network create xiaoxia-net-production 2>/dev/null || true
# ---- 执行数据库 Migration ----
echo "Running database migrations..."
docker run --rm \
--env-file "$ENV_FILE" \
--network xiaoxia-net-production \
-e APP_ENV=production \
"$LOCAL_API" sh -c "cd /app && alembic upgrade head"
echo "Migrations completed."
# ---- 停止旧容器 ----
echo "Stopping old containers..."
docker rm -f xiaoxia-api-production 2>/dev/null || true
docker rm -f xiaoxia-worker-production 2>/dev/null || true
docker rm -f xiaoxia-web-production 2>/dev/null || true
# ---- 日志配置(所有容器共用) ----
LOG_OPTS="--log-driver json-file --log-opt max-size=50m --log-opt max-file=3"
# ---- 启动 API ----
echo "Starting API container..."
docker run -d \
--name xiaoxia-api-production \
--env-file "$ENV_FILE" \
--network xiaoxia-net-production \
-p 127.0.0.1:8001:8000 \
-e APP_ENV=production \
-e APP_VERSION="$IMAGE_TAG" \
-e GENERATED_FILES_DIR=/app/generated \
-e GENERATED_FILES_URL_PREFIX=/generated-files \
-e PUBLIC_API_BASE_URL=https://api.xiaoxiajianji.com \
-v "$GENERATED_DIR:/app/generated" \
--restart unless-stopped \
--cpus 2 \
--memory 2g \
--health-cmd "python -c \"import urllib.request; urllib.request.urlopen('http://localhost:8000/health', timeout=5)\"" \
--health-interval 30s \
--health-timeout 10s \
--health-retries 3 \
--health-start-period 40s \
$LOG_OPTS \
"$LOCAL_API"
# ---- 启动 Worker ----
echo "Starting Worker container..."
docker run -d \
--name xiaoxia-worker-production \
--env-file "$ENV_FILE" \
--network xiaoxia-net-production \
-e APP_ENV=production \
-e APP_VERSION="$IMAGE_TAG" \
-e WORKER_CONCURRENCY=1 \
-e WORKER_MAX_TASKS_PER_CHILD=100 \
-e GENERATED_FILES_DIR=/app/generated \
-e GENERATED_FILES_URL_PREFIX=/generated-files \
-e PUBLIC_API_BASE_URL=https://api.xiaoxiajianji.com \
-v "$GENERATED_DIR:/app/generated" \
--restart unless-stopped \
--cpus 2 \
--memory 2g \
--health-cmd "sh -c \"grep -q celery /proc/1/cmdline || exit 1\"" \
--health-interval 30s \
--health-timeout 10s \
--health-retries 3 \
--health-start-period 30s \
$LOG_OPTS \
"$LOCAL_WORKER"
# ---- 启动 Web ----
# Legacy assets 挂载到 /usr/share/nginx/html/assets-legacy/assets/
# nginx 配置中 assets location 有 fallback 逻辑
LEGACY_VOLUME=""
if [ -d "$LEGACY_ASSETS_DIR" ] && [ "$(ls -A "$LEGACY_ASSETS_DIR" 2>/dev/null)" ]; then
LEGACY_VOLUME="-v ${LEGACY_ASSETS_DIR}:/usr/share/nginx/html/assets-legacy/assets:ro"
echo "Web container: legacy assets mounted (fallback)"
else
echo "Web container: no legacy assets to mount"
fi
echo "Starting Web container..."
docker run -d \
--name xiaoxia-web-production \
--network xiaoxia-net-production \
-p 127.0.0.1:3002:80 \
--restart unless-stopped \
--cpus 0.5 \
--memory 512m \
$LEGACY_VOLUME \
--health-cmd "wget --spider -q http://127.0.0.1:80" \
--health-interval 30s \
--health-timeout 5s \
--health-retries 3 \
$LOG_OPTS \
"$LOCAL_WEB"
# ---- 等待 API 健康 ----
echo "Waiting for API to become healthy..."
i=0
while [ "$i" -lt 40 ]; do
if curl -sf --max-time 5 http://127.0.0.1:8001/health >/dev/null 2>&1; then
echo "API is healthy!"
break
fi
i=$((i + 1))
echo " Waiting... ($i/40)"
sleep 3
done
if [ "$i" -ge 40 ]; then
echo "ERROR: API did not become healthy within 120s"
docker logs --tail 50 xiaoxia-api-production
exit 1
fi
# ---- 等待 Web 健康 ----
echo "Waiting for Web to become healthy..."
i=0
while [ "$i" -lt 15 ]; do
if curl -sf --max-time 5 http://127.0.0.1:3002/ >/dev/null 2>&1; then
echo "Web is healthy!"
break
fi
i=$((i + 1))
echo " Waiting... ($i/15)"
sleep 2
done
if [ "$i" -ge 15 ]; then
echo "ERROR: Web did not become healthy within 30s"
docker logs --tail 30 xiaoxia-web-production
exit 1
fi
# ---- 清理旧镜像 ----
echo "Cleaning up old images..."
docker image prune -af --filter "until=168h" 2>/dev/null || true
docker builder prune -af --filter "until=168h" 2>/dev/null || true
echo ""
echo "=== Production deployment complete ==="
echo "API: http://127.0.0.1:8001"
echo "Web: http://127.0.0.1:3002"
echo "Version: $IMAGE_TAG"
docker ps --format "table {{.Names}}\t{{.Status}}\t{{.Image}}" | grep production
+178
View File
@@ -0,0 +1,178 @@
#!/bin/sh
set -eu
# ============================================
# Staging 部署脚本 - Registry 方式
# 用法:IMAGE_TAG=<sha|version> REGISTRY_TOKEN=<token> sh deploy-staging.sh
# ============================================
IMAGE_TAG="${IMAGE_TAG:-}"
REGISTRY="${REGISTRY:-git.xiaoxiajianji.com/xiaoxia/xiaoxia-saas}"
REGISTRY_USER="${REGISTRY_USER:-xiaoxia}"
REGISTRY_TOKEN="${REGISTRY_TOKEN:-}"
ENV_FILE="${ENV_FILE:-/var/lib/xiaoxia-saas-staging/.env}"
COMPOSE_DIR="${COMPOSE_DIR:-/var/lib/xiaoxia-saas-staging/repo/infra/docker}"
GENERATED_DIR="${GENERATED_DIR:-/var/lib/xiaoxia-saas-staging/generated}"
if [ -z "$IMAGE_TAG" ]; then
echo "ERROR: IMAGE_TAG is required"
exit 1
fi
test -f "$ENV_FILE"
mkdir -p "$GENERATED_DIR"
# ---- 登录 Registry ----
if [ -n "$REGISTRY_TOKEN" ]; then
echo "Logging in to registry: $REGISTRY"
printf %s "$REGISTRY_TOKEN" | docker login "$(echo $REGISTRY | cut -d/ -f1)" -u "$REGISTRY_USER" --password-stdin 2>/dev/null || {
echo "WARN: docker login failed, will try to pull anyway"
}
fi
# ---- Pull 三镜像 ----
REGISTRY_API="${REGISTRY}/xiaoxia-saas-api:${IMAGE_TAG}"
REGISTRY_WORKER="${REGISTRY}/xiaoxia-saas-worker:${IMAGE_TAG}"
REGISTRY_WEB="${REGISTRY}/xiaoxia-saas-web:${IMAGE_TAG}"
LOCAL_API="${REGISTRY}/xiaoxia-saas-api:staging"
LOCAL_WORKER="${REGISTRY}/xiaoxia-saas-worker:staging"
LOCAL_WEB="${REGISTRY}/xiaoxia-saas-web:staging"
echo "Pulling API image..."
docker pull "$REGISTRY_API"
echo "Pulling Worker image..."
docker pull "$REGISTRY_WORKER"
echo "Pulling Web image..."
docker pull "$REGISTRY_WEB"
# ---- Re-tag 成本地名 ----
docker tag "$REGISTRY_API" "$LOCAL_API"
docker tag "$REGISTRY_WORKER" "$LOCAL_WORKER"
docker tag "$REGISTRY_WEB" "$LOCAL_WEB"
echo "All images pulled and tagged."
# ---- 确保基础设施容器在运行 ----
for c in xiaoxia-postgres-staging xiaoxia-redis-staging; do
if ! docker inspect "$c" >/dev/null 2>&1; then
echo "ERROR: Required container not found: $c"
exit 1
fi
state=$(docker inspect -f {{.State.Status}} "$c")
if [ "$state" != "running" ]; then
echo "ERROR: Container not running: $c ($state)"
exit 1
fi
done
# ---- 确保 staging 网络存在 ----
docker network create xiaoxia-net-staging 2>/dev/null || true
# ---- 执行数据库 Migration ----
echo "Running database migrations..."
docker run --rm --env-file "$ENV_FILE" --network xiaoxia-net-staging "$LOCAL_API" sh -c "cd /app && alembic upgrade head"
echo "Migrations completed."
# ---- 停止旧容器 ----
docker rm -f xiaoxia-api-staging 2>/dev/null || true
docker rm -f xiaoxia-worker-staging 2>/dev/null || true
docker rm -f xiaoxia-web-staging 2>/dev/null || true
# ---- 启动 API ----
echo "Starting API container..."
docker run -d \
--name xiaoxia-api-staging \
--env-file "$ENV_FILE" \
--network xiaoxia-net-staging \
-p 127.0.0.1:8000:8000 \
-e APP_ENV=staging \
-e APP_VERSION="$IMAGE_TAG" \
-e GENERATED_FILES_DIR=/app/generated \
-e GENERATED_FILES_URL_PREFIX=/generated-files \
-v "$GENERATED_DIR:/app/generated" \
--restart unless-stopped \
--label com.centurylinklabs.watchtower.enable=true \
--health-cmd "python -c \"import urllib.request; urllib.request.urlopen('http://localhost:8000/health', timeout=5)\"" \
--health-interval 30s \
--health-timeout 10s \
--health-retries 3 \
--health-start-period 40s \
"$LOCAL_API"
# ---- 启动 Worker ----
echo "Starting Worker container..."
docker run -d \
--name xiaoxia-worker-staging \
--env-file "$ENV_FILE" \
--network xiaoxia-net-staging \
-e APP_ENV=staging \
-e APP_VERSION="$IMAGE_TAG" \
-e WORKER_CONCURRENCY=1 \
-e WORKER_MAX_TASKS_PER_CHILD=100 \
-e GENERATED_FILES_DIR=/app/generated \
-e GENERATED_FILES_URL_PREFIX=/generated-files \
-v "$GENERATED_DIR:/app/generated" \
--restart unless-stopped \
--label com.centurylinklabs.watchtower.enable=true \
--health-cmd "sh -c \"grep -q celery /proc/1/cmdline || exit 1\"" \
--health-interval 30s \
--health-timeout 10s \
--health-retries 3 \
--health-start-period 30s \
"$LOCAL_WORKER"
# ---- 启动 Web ----
# Web 镜像默认打包 production nginx.confstaging 需要挂载 staging 配置
NGINX_CONF="${NGINX_CONF:-${COMPOSE_DIR}/nginx-staging.conf}"
if [ ! -f "$NGINX_CONF" ]; then
echo "WARN: nginx config not found at $NGINX_CONF, using image default"
NGINX_VOLUME=""
else
NGINX_VOLUME="-v ${NGINX_CONF}:/etc/nginx/conf.d/default.conf:ro"
fi
echo "Starting Web container..."
docker run -d \
--name xiaoxia-web-staging \
--network xiaoxia-net-staging \
-p 127.0.0.1:3001:80 \
--restart unless-stopped \
--label com.centurylinklabs.watchtower.enable=true \
$NGINX_VOLUME \
--health-cmd "wget --spider -q http://127.0.0.1:80" \
--health-interval 30s \
--health-timeout 5s \
--health-retries 3 \
"$LOCAL_WEB"
# ---- 等待 API 健康 ----
echo "Waiting for API to become healthy..."
i=0
while [ "$i" -lt 30 ]; do
if curl -sf --max-time 5 http://127.0.0.1:8000/health >/dev/null 2>&1; then
echo "API is healthy!"
break
fi
i=$((i + 1))
echo " Waiting... ($i/30)"
sleep 2
done
if [ "$i" -ge 30 ]; then
echo "ERROR: API did not become healthy within 60s"
docker logs --tail 30 xiaoxia-api-staging
exit 1
fi
# ---- 清理旧镜像 ----
docker image prune -af --filter "until=72h" 2>/dev/null || true
echo ""
echo "=== Staging deployment complete ==="
echo "API: http://127.0.0.1:8000"
echo "Web: http://127.0.0.1:3001"
echo "Version: $IMAGE_TAG"
docker ps --format "table {{.Names}}\t{{.Status}}" | grep staging
# Watchtower auto-update: 容器加com.centurylinklabs.watchtower.enable=true标签,用:staging tag启动
+9
View File
@@ -39,6 +39,15 @@ server {
alias /app/generated/;
}
# Assets with legacy fallback (higher priority than generic static regex)
# 部署期间,缓存了旧版 index.html 的用户会请求旧版带 hash 的 assets 文件
# 先在当前镜像中找,找不到去 legacy-assets 目录找(从旧版本容器中备份的)
location ^~ /assets/ {
expires 1y;
add_header Cache-Control "public, immutable";
try_files $uri /assets-legacy$uri =404;
}
# Cache static assets
location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg|woff|woff2|ttf|eot)$ {
expires 1y;
+1 -1
View File
@@ -1,4 +1,4 @@
FROM docker.m.daocloud.io/library/nginx:alpine AS runner
FROM xiaoxia-registry.cn-hangzhou.cr.aliyuncs.com/xiaoxiakeji/base/nginx:alpine AS runner
ARG NGINX_CONF=infra/docker/nginx.conf
WORKDIR /usr/share/nginx/html
COPY apps/web/dist ./
+2 -2
View File
@@ -1,5 +1,5 @@
# Build stage
FROM docker.m.daocloud.io/library/node:20 AS builder
FROM xiaoxia-registry.cn-hangzhou.cr.aliyuncs.com/xiaoxiakeji/base/node:20 AS builder
WORKDIR /app
ARG VITE_API_URL=https://saas-api.xiaoxiajianji.com
ENV VITE_API_URL=$VITE_API_URL
@@ -11,7 +11,7 @@ COPY apps/web/ ./
RUN npm run build
# Production stage with nginx
FROM docker.m.daocloud.io/library/nginx:alpine AS runner
FROM xiaoxia-registry.cn-hangzhou.cr.aliyuncs.com/xiaoxiakeji/base/nginx:alpine AS runner
ARG NGINX_CONF=infra/docker/nginx.conf
WORKDIR /usr/share/nginx/html
COPY --from=builder /app/apps/web/dist ./
@@ -0,0 +1,43 @@
# ============================================================
# Worker Builder 基础镜像
# 预编译:编译工具 + 基础依赖 + Worker大包
# 当 requirements-base.txt 或 requirements-worker.txt 变更时重新构建
# 业务构建从此镜像开始,只需要安装业务依赖,节省15+分钟
# ============================================================
FROM git.xiaoxiajianji.com/xiaoxia/base/python:3.12-slim
# 使用阿里云镜像加速
RUN sed -i 's|deb.debian.org|mirrors.aliyun.com|g' /etc/apt/sources.list.d/debian.sources 2>/dev/null || \
sed -i 's|deb.debian.org|mirrors.aliyun.com|g' /etc/apt/sources.list 2>/dev/null || true
# 安装编译工具
RUN apt-get update && apt-get install -y --no-install-recommends \
gcc \
g++ \
python3-dev \
binutils \
&& rm -rf /var/lib/apt/lists/*
# 创建 venv
RUN python -m venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"
WORKDIR /tmp
# 基础依赖(变化极少)
COPY requirements-base.txt /tmp/requirements-base.txt
RUN --mount=type=cache,target=/root/.cache/pip,sharing=locked \
pip install --no-cache-dir -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com \
-r /tmp/requirements-base.txt \
&& rm /tmp/requirements-base.txt
# Worker 大包(变化少)
COPY requirements-worker.txt /tmp/requirements-worker.txt
RUN --mount=type=cache,target=/root/.cache/pip,sharing=locked \
pip install --no-cache-dir -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com \
-r /tmp/requirements-worker.txt \
&& rm /tmp/requirements-worker.txt
# 预先做一次 strip(基础层瘦身,业务层增量)
RUN find /opt/venv -name "*.so" -type f -exec strip --strip-all {} \; 2>/dev/null || true
@@ -0,0 +1,17 @@
# ============================================================
# Worker Runtime 基础镜像
# 预安装:ffmpeg + 运行时依赖
# 变化极少,业务构建从此镜像开始
# ============================================================
FROM git.xiaoxiajianji.com/xiaoxia/base/python:3.12-slim
# 使用阿里云镜像加速
RUN sed -i 's|deb.debian.org|mirrors.aliyun.com|g' /etc/apt/sources.list.d/debian.sources 2>/dev/null || \
sed -i 's|deb.debian.org|mirrors.aliyun.com|g' /etc/apt/sources.list 2>/dev/null || true
# 运行时依赖:ffmpeg + opencv需要的libglib
RUN apt-get update && apt-get install -y --no-install-recommends \
ffmpeg \
libglib2.0-0 \
&& rm -rf /var/lib/apt/lists/*
+1 -1
View File
@@ -4,7 +4,7 @@
# ============================================================
# 基础镜像:Python 3.12 + ffmpeg
FROM python:3.12-slim-bookworm
FROM xiaoxia-registry.cn-hangzhou.cr.aliyuncs.com/xiaoxiakeji/base/python:3.12-slim-bookworm
# 使用阿里云镜像加速
RUN sed -i 's|deb.debian.org|mirrors.aliyun.com|g' /etc/apt/sources.list.d/debian.sources 2>/dev/null || \
@@ -76,3 +76,16 @@ class InMemoryAssetRepository:
tag_set = set(tag_ids)
items = [a for a in self._assets.values() if tag_set.issubset(set(a.tag_ids))]
return items[skip : skip + limit]
def find_by_library_and_file_hash(
self,
library_id: str,
file_hash: str,
) -> Asset | None:
"""按素材库 + 文件哈希查找已有素材(去重检测)。"""
if not file_hash:
return None
for asset in self._assets.values():
if asset.library_id == library_id and asset.file_hash == file_hash:
return asset
return None
@@ -83,6 +83,7 @@ class SQLAlchemyAssetRepository:
classification_result=(json.dumps(asset.metadata) if asset.metadata else None),
quality_score=asset.quality_score,
uploaded_by_user_id=asset.uploaded_by_user_id or "system",
file_hash=asset.file_hash or None,
created_at=asset.created_at,
updated_at=now,
)
@@ -110,6 +111,7 @@ class SQLAlchemyAssetRepository:
model.classification_result = json.dumps(asset.metadata) if asset.metadata else None
model.quality_score = asset.quality_score
model.uploaded_by_user_id = asset.uploaded_by_user_id or model.uploaded_by_user_id
model.file_hash = asset.file_hash or model.file_hash
model.updated_at = datetime.now(timezone.utc)
self.session.flush()
self._sync_asset_tags(asset.id, asset.tag_ids)
@@ -230,6 +232,7 @@ class SQLAlchemyAssetRepository:
classification_status=ClassificationStatus(model.classification_status),
quality_score=model.quality_score,
uploaded_by_user_id=model.uploaded_by_user_id,
file_hash=model.file_hash or "",
metadata=metadata,
tag_ids=tag_ids,
created_at=model.created_at,
@@ -267,3 +270,23 @@ class SQLAlchemyAssetRepository:
return []
models = self.session.query(AssetModel).filter(AssetModel.id.in_(ids)).offset(skip).limit(limit).all()
return [self._to_domain(m) for m in models]
def find_by_library_and_file_hash(
self,
library_id: str,
file_hash: str,
) -> Asset | None:
"""按素材库 + 文件哈希查找已有素材(去重检测)。"""
if not file_hash:
return None
model = (
self.session.query(AssetModel)
.filter(
AssetModel.asset_library_id == library_id,
AssetModel.file_hash == file_hash,
)
.first()
)
if model is None:
return None
return self._to_domain(model)
@@ -78,7 +78,7 @@ class SQLAlchemyGeneratedVideoRepository:
def list_by_project(self, project_id: str) -> list[GeneratedVideo]:
models = self.session.query(GeneratedVideoModel).filter(GeneratedVideoModel.project_id == project_id).all()
return [self.get(model.id) for model in models if self.get(model.id) is not None]
return [self._to_domain(model) for model in models]
def list_by_generation_task(self, generation_task_id: str) -> list[GeneratedVideo]:
models = (
@@ -86,4 +86,40 @@ class SQLAlchemyGeneratedVideoRepository:
.filter(GeneratedVideoModel.generation_task_id == generation_task_id)
.all()
)
return [self.get(model.id) for model in models if self.get(model.id) is not None]
return [self._to_domain(model) for model in models]
def list_by_batch(self, batch_id: str) -> list[GeneratedVideo]:
"""通过 batch_id 查找同批次生成的所有视频(跨 generation_task 关联查询)。"""
from packages.adapters.sqlalchemy_impl.models import GenerationTaskModel
task_ids = (
self.session.query(GenerationTaskModel.id).filter(GenerationTaskModel.batch_id == batch_id).subquery()
)
models = (
self.session.query(GeneratedVideoModel).filter(GeneratedVideoModel.generation_task_id.in_(task_ids)).all()
)
return [self._to_domain(model) for model in models]
@staticmethod
def _to_domain(model: GeneratedVideoModel) -> GeneratedVideo:
return GeneratedVideo(
id=model.id,
project_id=model.project_id,
generation_task_id=model.generation_task_id,
name=model.name,
file_url=model.file_url,
file_size=int(model.file_size or 0),
duration=model.duration,
thumbnail_url=model.thumbnail_url,
width=int(model.width or 0),
height=int(model.height or 0),
fps=model.fps,
status=getattr(model, "status", "completed"),
review_status=getattr(model, "review_status", "pending_review"),
generation_params=json.loads(getattr(model, "generation_params", "{}") or "{}"),
video_fingerprint=json.loads(getattr(model, "video_fingerprint", "null") or "null"),
is_duplicate=getattr(model, "is_duplicate", False),
duplicate_of=getattr(model, "duplicate_of", None),
generated_at=model.generated_at,
created_at=model.created_at,
)
@@ -25,6 +25,8 @@ def _to_domain(model: GenerationTaskModel) -> GenerationTask:
completed_at=model.completed_at,
created_by_user_id=model.created_by_user_id,
source_edit_plan_id=model.source_edit_plan_id or "",
asset_select_mode=model.asset_select_mode or "",
batch_id=model.batch_id or "",
created_at=model.created_at,
)
@@ -52,6 +54,8 @@ class SQLAlchemyGenerationTaskRepository:
completed_at=task.completed_at,
created_by_user_id=task.created_by_user_id,
source_edit_plan_id=task.source_edit_plan_id or None,
asset_select_mode=task.asset_select_mode or "",
batch_id=task.batch_id or "",
created_at=task.created_at,
)
self.session.add(model)
@@ -123,5 +127,7 @@ class SQLAlchemyGenerationTaskRepository:
model.started_at = task.started_at
model.completed_at = task.completed_at
model.source_edit_plan_id = task.source_edit_plan_id or None
model.asset_select_mode = task.asset_select_mode or ""
model.batch_id = task.batch_id or ""
self.session.commit()
return task
@@ -17,6 +17,7 @@ class SQLAlchemyIngestJobRepository:
status=job.status.value,
error_message=job.error_message,
result_asset_id=job.result_asset_id,
file_hash=job.file_hash,
created_at=job.created_at,
updated_at=job.updated_at,
)
@@ -36,6 +37,7 @@ class SQLAlchemyIngestJobRepository:
status=IngestJobStatus(model.status),
error_message=model.error_message,
result_asset_id=model.result_asset_id,
file_hash=model.file_hash or "",
created_at=model.created_at,
updated_at=model.updated_at,
)
@@ -51,6 +53,7 @@ class SQLAlchemyIngestJobRepository:
model.status = job.status.value
model.error_message = job.error_message
model.result_asset_id = job.result_asset_id
model.file_hash = job.file_hash
model.updated_at = job.updated_at
self.session.commit()
return job
@@ -85,6 +85,7 @@ class AssetModel(Base):
classification_result = Column(Text, nullable=True)
quality_score = Column(Float, nullable=True)
uploaded_by_user_id = Column(String(36), nullable=False)
file_hash = Column(String(64), nullable=True, index=True)
extra_meta = Column("metadata", JSON, nullable=False, default=dict)
created_at = Column(DateTime, nullable=False, default=lambda: datetime.now(timezone.utc), index=True)
updated_at = Column(DateTime, nullable=False, default=lambda: datetime.now(timezone.utc))
@@ -210,6 +211,7 @@ class IngestJobModel(Base):
status = Column(String(20), nullable=False, default="pending")
error_message = Column(Text, nullable=False, default="")
result_asset_id = Column(String(32), nullable=False, default="")
file_hash = Column(String(64), nullable=True, index=True)
created_at = Column(DateTime, nullable=False, default=lambda: datetime.now(timezone.utc))
updated_at = Column(DateTime, nullable=False, default=lambda: datetime.now(timezone.utc))
@@ -251,6 +253,8 @@ class GenerationTaskModel(Base):
completed_at = Column(DateTime, nullable=True)
created_by_user_id = Column(String(32), nullable=False, default="", index=True)
source_edit_plan_id = Column(String(32), nullable=True, index=True)
asset_select_mode = Column(String(20), nullable=False, default="")
batch_id = Column(String(32), nullable=False, default="", index=True)
extra_meta = Column("metadata", JSON, nullable=False, default=dict)
created_at = Column(DateTime, nullable=False, default=lambda: datetime.now(timezone.utc))
+4
View File
@@ -19,6 +19,8 @@ class CreateGenerationTaskCommand:
voice_ids: list[str] = field(default_factory=list)
created_by_user_id: str = ""
source_edit_plan_id: str = ""
asset_select_mode: str = ""
batch_id: str = ""
class CreateGenerationTaskUseCase:
@@ -44,6 +46,8 @@ class CreateGenerationTaskUseCase:
completed_at=None,
created_by_user_id=command.created_by_user_id,
source_edit_plan_id=command.source_edit_plan_id,
asset_select_mode=command.asset_select_mode,
batch_id=command.batch_id,
)
return self.generation_task_repository.create(task)
+2
View File
@@ -11,6 +11,7 @@ class SubmitIngestJobCommand:
project_id: str
library_id: str
storage_key: str
file_hash: str = ""
class SubmitIngestJobUseCase:
@@ -22,5 +23,6 @@ class SubmitIngestJobUseCase:
project_id=command.project_id,
library_id=command.library_id,
storage_key=command.storage_key,
file_hash=command.file_hash,
)
return self.ingest_job_repository.create(job)
+6
View File
@@ -161,6 +161,7 @@ class Asset:
classification_status: ClassificationStatus = ClassificationStatus.PENDING
quality_score: float | None = None
uploaded_by_user_id: str = ""
file_hash: str = ""
metadata: dict[str, Any] = field(default_factory=dict)
tag_ids: list[str] = field(default_factory=list)
created_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))
@@ -187,6 +188,7 @@ class Asset:
classification_status: ClassificationStatus = ClassificationStatus.PENDING,
quality_score: float | None = None,
uploaded_by_user_id: str = "",
file_hash: str = "",
) -> "Asset":
clean_name = name.strip()
if not clean_name:
@@ -213,6 +215,7 @@ class Asset:
classification_status=classification_status,
quality_score=quality_score,
uploaded_by_user_id=uploaded_by_user_id.strip(),
file_hash=file_hash.strip(),
metadata=metadata or {},
tag_ids=[],
)
@@ -243,6 +246,7 @@ class IngestJob:
status: IngestJobStatus = IngestJobStatus.PENDING
error_message: str = ""
result_asset_id: str = ""
file_hash: str = ""
created_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))
updated_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))
@@ -252,6 +256,7 @@ class IngestJob:
project_id: str,
library_id: str,
storage_key: str,
file_hash: str = "",
) -> "IngestJob":
if not project_id.strip():
raise ValueError("project_id 不能为空")
@@ -264,4 +269,5 @@ class IngestJob:
project_id=project_id.strip(),
library_id=library_id.strip(),
storage_key=storage_key.strip(),
file_hash=file_hash.strip(),
)
+6
View File
@@ -43,6 +43,8 @@ class GenerationTask:
completed_at: datetime | None = None
source_edit_plan_id: str = ""
created_by_user_id: str = ""
asset_select_mode: str = ""
batch_id: str = ""
created_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))
@classmethod
@@ -59,6 +61,8 @@ class GenerationTask:
voice_ids: list[str] | None = None,
created_by_user_id: str = "",
source_edit_plan_id: str = "",
asset_select_mode: str = "",
batch_id: str = "",
) -> "GenerationTask":
if not project_id.strip() and not template_id.strip():
raise ValueError("project_id 或 template_id 至少需要提供一个")
@@ -76,4 +80,6 @@ class GenerationTask:
voice_ids=list(voice_ids) if voice_ids else [],
created_by_user_id=created_by_user_id.strip(),
source_edit_plan_id=source_edit_plan_id.strip(),
asset_select_mode=asset_select_mode,
batch_id=batch_id,
)
+9
View File
@@ -93,3 +93,12 @@ class AssetRepository(ABC):
) -> list[Asset]:
"""查找包含所有指定标签的素材。"""
pass
@abstractmethod
def find_by_library_and_file_hash(
self,
library_id: str,
file_hash: str,
) -> Asset | None:
"""按素材库 + 文件哈希查找已有素材(去重检测)。"""
pass
@@ -13,3 +13,5 @@ class GeneratedVideoRepository(Protocol):
def list_by_project(self, project_id: str) -> list[GeneratedVideo]: ...
def list_by_generation_task(self, generation_task_id: str) -> list[GeneratedVideo]: ...
def list_by_batch(self, batch_id: str) -> list[GeneratedVideo]: ...
+42
View File
@@ -5,3 +5,45 @@ target-version = ["py312"]
[tool.isort]
profile = "black"
line_length = 120
[tool.ruff]
target-version = "py311"
line-length = 120
exclude = [
".git",
"__pycache__",
".venv",
"venv",
"node_modules",
"alembic",
".gitea",
".next",
"dist",
"build",
"hostexecutor",
]
[tool.ruff.lint]
select = [
"E", # pycodestyle errors(同 flake8 默认)
"F", # pyflakes(同 flake8 默认)
]
ignore = [
"E203",
"E501", # line-too-longblack管)
"E302",
"E402", # module-import-not-at-top(循环导入多)
"E722", # bare-except
"W291",
"W293",
"F401",
"F403",
"F405",
"F841",
]
[tool.ruff.lint.per-file-ignores]
"__init__.py" = ["F401", "F403", "F405"]
"tests/**" = ["E402", "F401", "F821", "F841"]
"packages/ports/*" = ["E301"]
"apps/api/app/api/routes/auth.py" = ["ALL"]
+58
View File
@@ -0,0 +1,58 @@
{
"": "https://docs.renovatebot.com/renovate-schema.json",
"extends": ["config:recommended"],
"baseBranches": ["develop"],
"labels": ["dependencies"],
"assignees": ["xiaoxia"],
"prConcurrentLimit": 3,
"prHourlyLimit": 3,
"schedule": ["after 2am before 6am on monday"],
"timezone": "Asia/Shanghai",
"vulnerabilityAlerts": {
"enabled": true,
"labels": ["dependencies", "security"],
"schedule": ["at any time"]
},
"pip_requirements": {
"fileMatch": [
"(^|/)requirements\.txt$",
"(^|/)requirements-base\.txt$",
"(^|/)requirements-dev\.txt$",
"(^|/)requirements-worker\.txt$"
]
},
"npm": {
"fileMatch": [
"(^|/)apps/web/package\.json$"
]
},
"packageRules": [
{
"matchDepTypes": ["dependencies"],
"matchUpdateTypes": ["patch", "minor"],
"groupName": "production deps (minor & patch)",
"groupSlug": "prod-deps-minor-patch"
},
{
"matchDepTypes": ["devDependencies"],
"matchUpdateTypes": ["patch", "minor"],
"groupName": "dev deps (minor & patch)",
"groupSlug": "dev-deps-minor-patch"
},
{
"matchUpdateTypes": ["major"],
"labels": ["dependencies", "major-update"]
}
],
"rebaseWhen": "behind-base-branch",
"semanticCommits": "auto",
"semanticPrefix": "chore(deps): "
}
+1
View File
@@ -11,3 +11,4 @@ pytest==8.3.3
pytest-asyncio==0.24.0
pytest-cov==6.0.0
pytest-timeout==2.3.1
diff-cover==8.0.3
+44
View File
@@ -0,0 +1,44 @@
#!/usr/bin/env python3
"""检查指定commit的CI status状态。
用法: python3 check_ci_status.py <token> <repo> <sha> <context>
返回: 打印状态 (success/failure/pending/error)
"""
import json
import sys
import urllib.error
import urllib.request
def main():
if len(sys.argv) != 5:
print("pending")
return
token = sys.argv[1]
repo = sys.argv[2]
sha = sys.argv[3]
target_context = sys.argv[4]
api_url = f"https://git.xiaoxiajianji.com/api/v1/repos/{repo}/commits/{sha}/statuses?per_page=100"
req = urllib.request.Request(api_url, headers={"Authorization": f"token {token}"})
try:
with urllib.request.urlopen(req, timeout=30) as resp:
statuses = json.loads(resp.read().decode())
except Exception:
print("pending")
return
# API返回按时间倒序,第一个就是最新的
for s in statuses:
if s.get("context") == target_context:
print(s.get("status", "pending"))
return
print("pending")
if __name__ == "__main__":
main()
+46
View File
@@ -0,0 +1,46 @@
#!/usr/bin/env python3
"""检查PR是否有至少N个APPROVED审批。
用法: python3 check_pr_approval.py <token> <repo> <pr_number> <min_approval>
返回: 打印 "approved""pending"
"""
import json
import sys
import urllib.request
def main():
if len(sys.argv) != 5:
print("pending")
return
token = sys.argv[1]
repo = sys.argv[2]
pr_number = sys.argv[3]
min_approval = int(sys.argv[4])
api_url = f"https://git.xiaoxiajianji.com/api/v1/repos/{repo}/pulls/{pr_number}/reviews"
req = urllib.request.Request(api_url, headers={"Authorization": f"token {token}"})
try:
with urllib.request.urlopen(req, timeout=30) as resp:
reviews = json.loads(resp.read().decode())
except Exception:
print("pending")
return
# 统计APPROVED的人数(去重,同一人多次审批只算一次)
approvers = set()
for r in reviews:
if r.get("state") == "APPROVED":
approvers.add(r.get("user", {}).get("login", ""))
if len(approvers) >= min_approval:
print(f"approved ({len(approvers)})")
else:
print(f"pending ({len(approvers)})")
if __name__ == "__main__":
main()
+653
View File
@@ -0,0 +1,653 @@
#!/usr/bin/env python3
"""
ACR 镜像清理脚本(增强版)
清理策略:
- 版本tag (v*): 永久保留
- 固定tag (latest, main, develop, master): 永久保留
- 缓存镜像 (*-cache): 永久保留
- 受保护tag (--protected-tags): 永久保留(如当前运行中镜像)
- PR预览tag (pr-*):
- --pr-sha模式:删除指定PR commit的镜像(PR关闭时触发)
- cron模式:通过Gitea API检查PR状态,已关闭/合并的删除
- 普通commit hash tag: 保留最近 N 个(默认20),老的删除
使用方式:
# 预览(不实际删除)
python3 acr_cleanup.py --dry-run
# 实际执行(cron模式)
python3 acr_cleanup.py --execute
# 保留最近30个commit镜像
python3 acr_cleanup.py --keep 30 --execute
# PR关闭时清理指定commit的PR镜像
python3 acr_cleanup.py --pr-sha abc123def --execute
# 传入受保护tag列表(运行中镜像白名单)
python3 acr_cleanup.py --protected-tags "sha1,sha2" --execute
"""
import argparse
import base64
import json
import os
import sys
import urllib.error
import urllib.request
from datetime import datetime, timedelta, timezone
# ========== 配置 ==========
REGISTRY = os.environ.get("ACR_REGISTRY", "xiaoxia-registry.cn-hangzhou.cr.aliyuncs.com")
AUTH_URL = "https://dockerauth.cn-hangzhou.aliyuncs.com/auth"
SERVICE = os.environ.get("ACR_SERVICE", "registry.aliyuncs.com:cn-hangzhou:china:cri-fvec8o9q4mmxrkaa")
NAMESPACE = os.environ.get("ACR_NAMESPACE", "xiaoxiakeji")
USERNAME = os.environ.get("ACR_USERNAME", "")
PASSWORD = os.environ.get("ACR_PASSWORD", "")
# Gitea配置(用于PR状态检查)
GITEA_URL = os.environ.get("GITEA_URL", "https://git.xiaoxiajianji.com")
GITEA_TOKEN = os.environ.get("GITEA_TOKEN", "")
GITEA_REPO = os.environ.get("GITEA_REPO", "xiaoxia/xiaoxia-saas")
REPOS = [
"xiaoxia-saas-api",
"xiaoxia-saas-worker",
"xiaoxia-saas-web",
"api-cache",
"worker-cache",
"web-cache",
]
# 缓存镜像仓库(所有tag永久保留)
CACHE_REPOS = {"api-cache", "worker-cache", "web-cache"}
# OCI / Docker manifest types
ACCEPT_INDEX = "application/vnd.oci.image.index.v1+json"
ACCEPT_MANIFEST_OCI = "application/vnd.oci.image.manifest.v1+json"
ACCEPT_MANIFEST_V2 = "application/vnd.docker.distribution.manifest.v2+json"
# ========== Registry API ==========
def get_token(repo, action="pull"):
"""获取仓库访问token"""
scope = "repository:" + NAMESPACE + "/" + repo + ":" + action
token_url = AUTH_URL + "?service=" + SERVICE + "&scope=" + scope
req = urllib.request.Request(token_url)
req.add_header("Authorization", "Basic " + base64.b64encode((USERNAME + ":" + PASSWORD).encode()).decode())
with urllib.request.urlopen(req) as resp:
data = json.loads(resp.read())
return data.get("token", "")
def get_tags(repo, token):
"""获取仓库所有tag"""
url = "https://" + REGISTRY + "/v2/" + NAMESPACE + "/" + repo + "/tags/list?n=1000"
req = urllib.request.Request(url)
req.add_header("Authorization", "Bearer " + token)
with urllib.request.urlopen(req) as resp:
data = json.loads(resp.read())
return data.get("tags", []) or []
def http_get_json(url, token, accept_header):
"""带Authorization的GET请求,返回(json_data, headers)"""
req = urllib.request.Request(url)
req.add_header("Authorization", "Bearer " + token)
req.add_header("Accept", accept_header)
with urllib.request.urlopen(req) as resp:
return json.loads(resp.read()), resp.headers
def get_manifest_info(repo, tag, token):
"""
获取tag的manifest信息。
返回: {digest, created, media_type, error}
"""
url = "https://" + REGISTRY + "/v2/" + NAMESPACE + "/" + repo + "/manifests/" + tag
result = {"digest": "", "created": "", "media_type": "", "error": ""}
# 先尝试 OCI index 格式
try:
data, headers = http_get_json(url, token, ACCEPT_INDEX)
top_digest = headers.get("Docker-Content-Digest", "")
result["digest"] = top_digest
result["media_type"] = data.get("mediaType", ACCEPT_INDEX)
manifests = data.get("manifests", [])
amd64_manifest = None
for m in manifests:
arch = m.get("platform", {}).get("architecture", "")
if arch == "amd64":
amd64_manifest = m
break
if not amd64_manifest and manifests:
amd64_manifest = manifests[0]
if amd64_manifest:
inner_digest = amd64_manifest["digest"]
inner_url = "https://" + REGISTRY + "/v2/" + NAMESPACE + "/" + repo + "/manifests/" + inner_digest
try:
inner_data, _ = http_get_json(inner_url, token, ACCEPT_MANIFEST_OCI)
except Exception:
inner_data, _ = http_get_json(inner_url, token, ACCEPT_MANIFEST_V2)
config_digest = inner_data.get("config", {}).get("digest", "")
if config_digest:
blob_url = "https://" + REGISTRY + "/v2/" + NAMESPACE + "/" + repo + "/blobs/" + config_digest
try:
blob_data, _ = http_get_json(blob_url, token, "application/json")
result["created"] = blob_data.get("created", "")
except Exception:
pass
return result
except urllib.error.HTTPError:
pass
# 再尝试普通 OCI manifest 格式
try:
data, headers = http_get_json(url, token, ACCEPT_MANIFEST_OCI)
result["digest"] = headers.get("Docker-Content-Digest", "")
result["media_type"] = data.get("mediaType", ACCEPT_MANIFEST_OCI)
config_digest = data.get("config", {}).get("digest", "")
if config_digest:
blob_url = "https://" + REGISTRY + "/v2/" + NAMESPACE + "/" + repo + "/blobs/" + config_digest
try:
blob_data, _ = http_get_json(blob_url, token, "application/json")
result["created"] = blob_data.get("created", "")
except Exception:
pass
return result
except urllib.error.HTTPError:
pass
# 最后试 Docker v2 格式
try:
data, headers = http_get_json(url, token, ACCEPT_MANIFEST_V2)
result["digest"] = headers.get("Docker-Content-Digest", "")
result["media_type"] = data.get("mediaType", ACCEPT_MANIFEST_V2)
config_digest = data.get("config", {}).get("digest", "")
if config_digest:
blob_url = "https://" + REGISTRY + "/v2/" + NAMESPACE + "/" + repo + "/blobs/" + config_digest
try:
blob_data, _ = http_get_json(blob_url, token, "application/json")
result["created"] = blob_data.get("created", "")
except Exception:
pass
return result
except urllib.error.HTTPError as e:
result["error"] = "HTTP " + str(e.code) + " " + e.read().decode()[:200]
return result
def delete_manifest(repo, digest, token):
"""按digest删除manifest(会级联删除所有指向它的tag)"""
url = "https://" + REGISTRY + "/v2/" + NAMESPACE + "/" + repo + "/manifests/" + digest
req = urllib.request.Request(url, method="DELETE")
req.add_header("Authorization", "Bearer " + token)
req.add_header("Accept", ACCEPT_INDEX)
req.add_header("Accept", ACCEPT_MANIFEST_OCI)
req.add_header("Accept", ACCEPT_MANIFEST_V2)
try:
with urllib.request.urlopen(req) as resp:
return True, resp.status
except urllib.error.HTTPError as e:
return False, str(e.code) + " " + e.read().decode()[:200]
# ========== Gitea API ==========
def gitea_get_open_prs():
"""获取所有打开的PR编号列表"""
if not GITEA_TOKEN:
print(" 警告: 无GITEA_TOKEN,跳过PR状态检查")
return None
open_prs = set()
page = 1
while True:
url = GITEA_URL + "/api/v1/repos/" + GITEA_REPO + "/pulls?state=open&page=" + str(page) + "&limit=50"
req = urllib.request.Request(url)
req.add_header("Authorization", "token " + GITEA_TOKEN)
try:
with urllib.request.urlopen(req) as resp:
data = json.loads(resp.read())
if not data:
break
for pr in data:
open_prs.add(pr.get("number", 0))
if len(data) < 50:
break
page += 1
except Exception as e:
print(f" 警告: 获取Gitea PR列表失败: {e}")
return None
return open_prs
def gitea_get_pr_commits(pr_number):
"""获取指定PR的所有commit sha"""
if not GITEA_TOKEN:
return []
url = GITEA_URL + "/api/v1/repos/" + GITEA_REPO + "/pulls/" + str(pr_number) + "/commits?limit=100"
req = urllib.request.Request(url)
req.add_header("Authorization", "token " + GITEA_TOKEN)
try:
with urllib.request.urlopen(req) as resp:
data = json.loads(resp.read())
return [c.get("sha", "") for c in data]
except Exception as e:
print(f" 警告: 获取PR #{pr_number} commits失败: {e}")
return []
# ========== 工具函数 ==========
def parse_time(created_str):
"""解析ISO时间字符串"""
if not created_str:
return datetime.min.replace(tzinfo=timezone.utc)
try:
if created_str.endswith("Z"):
created_str = created_str[:-1] + "+00:00"
return datetime.fromisoformat(created_str)
except Exception:
return datetime.min.replace(tzinfo=timezone.utc)
def is_version_tag(tag):
"""判断是否是版本tag (v1.2.3, v0.1.0-alpha等)"""
return tag.startswith("v") and len(tag) > 1 and tag[1].isdigit()
def is_fixed_tag(tag):
"""判断是否是固定tag"""
return tag in ("latest", "main", "develop", "master", "dev", "stable")
def is_pr_tag(tag):
"""判断是否是PR预览tag (pr-<sha>)"""
return tag.startswith("pr-")
def extract_sha_from_pr_tag(tag):
"""从pr-<sha> tag中提取sha"""
if tag.startswith("pr-"):
return tag[3:]
return tag
def is_in_protected_list(tag, protected_set):
"""检查tag是否在受保护列表中"""
if not protected_set:
return False
# 精确匹配
if tag in protected_set:
return True
# 前缀匹配(commit hash可能是完整或短的)
for p in protected_set:
if tag.startswith(p) or p.startswith(tag):
return True
return False
# ========== 核心清理逻辑 ==========
def cleanup_repo(repo, keep_count, dry_run, protected_tags, pr_sha=None, pr_open_set=None):
"""
清理单个仓库
Args:
repo: 仓库名
keep_count: 保留最近N个commit tag
dry_run: 是否预览模式
protected_tags: 受保护tag集合(白名单)
pr_sha: 指定PR commit shaPR关闭模式),None表示cron模式
pr_open_set: 打开的PR编号集合(cron模式用)
Returns:
(总tag数, 删除数)
"""
print("=" * 60)
print("仓库:", repo)
print("=" * 60)
# 缓存仓库不清理
if repo in CACHE_REPOS:
token_pull = get_token(repo, "pull")
tags = get_tags(repo, token_pull)
print(" 缓存仓库,跳过清理 (共", len(tags), "个tag)")
return len(tags), 0
token_pull = get_token(repo, "pull")
tags = get_tags(repo, token_pull)
print(" 总tag数:", len(tags))
if not tags:
print(" 无tag,跳过")
return 0, 0
# ========== PR-SHA模式:只删除指定commit的PR镜像 ==========
if pr_sha:
pr_tags_to_del = [
t
for t in tags
if t.startswith("pr-" + pr_sha) or t == "pr-" + pr_sha or pr_sha.startswith(extract_sha_from_pr_tag(t))
]
if not pr_tags_to_del:
print(f" 未找到PR镜像: pr-{pr_sha[:12]}")
return len(tags), 0
print(f" 找到 {len(pr_tags_to_del)} 个PR镜像待删除:")
for t in pr_tags_to_del:
print(f" - {t}")
to_delete = []
for tag in pr_tags_to_del:
info = get_manifest_info(repo, tag, token_pull)
if info["digest"]:
to_delete.append({"tag": tag, "digest": info["digest"], "created": info["created"]})
else:
print(f" 警告: {tag} 无法获取digest,跳过")
return _execute_delete(repo, to_delete, dry_run, len(tags))
# ========== Cron模式:全量清理 ==========
# 分类
version_tags = []
fixed_tags = []
pr_tags_list = []
commit_tags = []
for tag in tags:
if is_version_tag(tag):
version_tags.append(tag)
elif is_fixed_tag(tag):
fixed_tags.append(tag)
elif is_pr_tag(tag):
pr_tags_list.append(tag)
else:
commit_tags.append(tag)
print(" 版本tag (v*):", len(version_tags), "-> 永久保留")
print(" 固定tag:", len(fixed_tags), "-> 永久保留")
print(" PR预览tag (pr-*):", len(pr_tags_list), "-> 已关闭PR的删除")
print(" Commit hash tag:", len(commit_tags), "-> 保留最近", keep_count, "")
print(" 白名单tag:", len(protected_tags), "")
# --- PR tag清理:检查PR状态 ---
pr_to_delete = []
if pr_tags_list:
print()
print(" 检查PR镜像状态...")
# 策略:有Gitea token则检查PR状态,否则按时间保留7天
if pr_open_set is not None:
# 通过Gitea API检查每个PR镜像对应的PR是否还开着
# 注意:pr tag是pr-<sha>sha可能属于某个PR
# 简化策略:收集所有打开PR的commit sha,在白名单里的保留
print(" 模式: Gitea PR状态检查")
open_pr_shas = set()
# 这里做了简化:因为每个PR都查commits太慢,我们用另一种方式
# 对于PR tag,先尝试匹配PR编号(如果tag名里有编号),否则按时间
# 实际pr-<sha>没法直接知道PR编号,所以降级为按时间+打开PR的head sha白名单
open_head_shas = set()
page = 1
while True:
url = GITEA_URL + "/api/v1/repos/" + GITEA_REPO + "/pulls?state=open&page=" + str(page) + "&limit=50"
req = urllib.request.Request(url)
req.add_header("Authorization", "token " + GITEA_TOKEN)
try:
with urllib.request.urlopen(req) as resp:
data = json.loads(resp.read())
if not data:
break
for pr in data:
head_sha = pr.get("head", {}).get("sha", "")
if head_sha:
open_head_shas.add(head_sha)
open_head_shas.add(head_sha[:7])
open_head_shas.add(head_sha[:12])
if len(data) < 50:
break
page += 1
except Exception:
break
deleted_count = 0
for tag in pr_tags_list:
sha = extract_sha_from_pr_tag(tag)
# 检查是否是打开PR的head sha
is_open_pr = False
for ohs in open_head_shas:
if sha.startswith(ohs) or ohs.startswith(sha):
is_open_pr = True
break
if not is_open_pr:
info = get_manifest_info(repo, tag, token_pull)
if info["digest"]:
pr_to_delete.append({"tag": tag, "digest": info["digest"], "created": info["created"]})
deleted_count += 1
print(f" 打开PR数: {len(open_head_shas)}个head sha")
print(f" 将删除PR镜像: {deleted_count}")
else:
# 无Gitea token,降级为按7天保留
print(" 模式: 按时间保留7天(无Gitea token降级)")
cutoff = datetime.now(timezone.utc) - timedelta(days=7)
for tag in pr_tags_list:
info = get_manifest_info(repo, tag, token_pull)
created = parse_time(info["created"])
if created < cutoff and info["digest"]:
pr_to_delete.append({"tag": tag, "digest": info["digest"], "created": info["created"]})
print(f" 将删除PR镜像: {len(pr_to_delete)}")
# --- Commit tag清理:保留最近N个 ---
print()
print(" 获取commit tag创建时间...")
commit_tag_infos = []
errors = 0
for i, tag in enumerate(commit_tags):
info = get_manifest_info(repo, tag, token_pull)
if info["error"] or not info["digest"]:
errors += 1
commit_tag_infos.append({"tag": tag, "digest": info["digest"], "created": info["created"]})
if (i + 1) % 20 == 0:
print(" 已获取", i + 1, "/", len(commit_tags), "...")
if errors:
print(" 注意:", errors, "个tag获取manifest失败")
# 按时间倒序排序
commit_tag_infos.sort(key=lambda x: parse_time(x["created"]), reverse=True)
# 确定要删除的commit tag
commit_to_delete = []
if len(commit_tag_infos) > keep_count:
commit_to_delete = commit_tag_infos[keep_count:]
print(f" 保留前{keep_count}个commit tag,删除{len(commit_to_delete)}")
# 白名单过滤:受保护的tag不删除
if protected_tags:
before = len(commit_to_delete)
commit_to_delete = [t for t in commit_to_delete if not is_in_protected_list(t["tag"], protected_tags)]
removed = before - len(commit_to_delete)
if removed > 0:
print(f" 白名单保护: 跳过{removed}个运行中镜像")
# 过滤无digest的
commit_to_delete = [t for t in commit_to_delete if t["digest"]]
print(f" 可删除(有digest): {len(commit_to_delete)}")
else:
print(f" commit tag数量不足{keep_count}个,无需清理")
# --- 合并所有待删除项 ---
all_to_delete = commit_to_delete + pr_to_delete
# 再次过滤白名单(PR镜像也受白名单保护)
if protected_tags:
before = len(all_to_delete)
all_to_delete = [t for t in all_to_delete if not is_in_protected_list(t["tag"], protected_tags)]
removed = before - len(all_to_delete)
if removed > 0:
print(f" 白名单保护(PR镜像): 跳过{removed}")
return _execute_delete(repo, all_to_delete, dry_run, len(tags))
def _execute_delete(repo, to_delete, dry_run, total_tags):
"""执行删除操作"""
if not to_delete:
print()
print(" 无需删除任何tag")
return total_tags, 0
# 按digest去重
seen_digests = set()
unique_delete = []
for item in to_delete:
if item["digest"] and item["digest"] not in seen_digests:
seen_digests.add(item["digest"])
unique_delete.append(item)
print()
if dry_run:
print(f" [DRY RUN] 将删除{len(unique_delete)}个manifest(预览模式)")
for item in unique_delete[:5]:
created_str = item.get("created", "")[:10] or "未知"
print(f" - {item['tag'][:30]} ({created_str})")
if len(unique_delete) > 5:
print(f" ... 还有{len(unique_delete) - 5}")
return total_tags, len(unique_delete)
token_delete = get_token(repo, "delete")
deleted = 0
failed = 0
print(f" 开始删除{len(unique_delete)}个唯一manifest...")
for item in unique_delete:
success, result = delete_manifest(repo, item["digest"], token_delete)
if success:
deleted += 1
print(f" 已删除: {item['tag'][:30]}")
else:
failed += 1
print(f" 删除失败: {item['tag'][:30]} - {result}")
print()
print(f" 删除完成: 成功{deleted}个,失败{failed}")
return total_tags, deleted
# ========== 主函数 ==========
def main():
parser = argparse.ArgumentParser(description="ACR镜像清理工具(增强版)")
parser.add_argument("--keep", type=int, default=20, help="保留最近N个commit hash tag(默认20")
parser.add_argument("--dry-run", action="store_true", help="预览模式,不实际删除")
parser.add_argument("--execute", action="store_true", help="实际执行删除")
parser.add_argument("--repo", type=str, default="", help="只清理指定仓库")
parser.add_argument("--pr-sha", type=str, default="", help="PR关闭模式:删除指定commit sha的PR镜像")
parser.add_argument("--protected-tags", type=str, default="", help="受保护tag列表,逗号分隔(运行中镜像白名单)")
parser.add_argument("--skip-pr-check", action="store_true", help="跳过Gitea PR状态检查(纯按时间清理PR镜像)")
args = parser.parse_args()
# 必须指定 --dry-run 或 --execute
if not args.dry_run and not args.execute:
print("请指定 --dry-run(预览)或 --execute(执行)")
print()
print("示例:")
print(" python3 acr_cleanup.py --dry-run # 预览清理效果")
print(" python3 acr_cleanup.py --execute # 实际执行清理")
print(" python3 acr_cleanup.py --pr-sha abc123 --execute # PR关闭时清理")
sys.exit(1)
# 凭证检查
global USERNAME, PASSWORD
if not USERNAME or not PASSWORD:
try:
docker_config_path = os.path.expanduser("~/.docker/config.json")
with open(docker_config_path) as f:
config = json.load(f)
auth = config.get("auths", {}).get(REGISTRY, {}).get("auth", "")
if auth:
creds = base64.b64decode(auth).decode().strip()
USERNAME, PASSWORD = creds.split(":", 1)
except Exception:
pass
if not USERNAME or not PASSWORD:
print("错误: 缺少ACR凭证,请设置 ACR_USERNAME 和 ACR_PASSWORD 环境变量")
print("或确保已执行 docker login", REGISTRY)
sys.exit(1)
# 解析受保护tag
protected_tags = set()
if args.protected_tags:
protected_tags = set(t.strip() for t in args.protected_tags.split(",") if t.strip())
dry_run = args.dry_run or not args.execute
mode = "预览模式" if dry_run else "执行模式"
print("=" * 60)
print("ACR 镜像清理工具(增强版)-", mode)
print("=" * 60)
print("Registry:", REGISTRY)
print("Namespace:", NAMESPACE)
if args.pr_sha:
print("模式: PR关闭清理")
print("PR commit SHA:", args.pr_sha[:12])
else:
print("模式: Cron全量清理")
print("保留commit tag数:", args.keep)
print("PR状态检查:", "关闭" if args.skip_pr_check else "开启")
if protected_tags:
print("白名单tag数:", len(protected_tags))
print()
# PR模式不需要查Gitea
pr_open_set = None
if not args.pr_sha and not args.skip_pr_check and GITEA_TOKEN:
print("获取打开的PR列表...")
pr_open_set = gitea_get_open_prs()
if pr_open_set is not None:
print(f" 打开的PR: {len(pr_open_set)}")
print()
repos_to_clean = REPOS
if args.repo:
repos_to_clean = [args.repo]
total_deleted = 0
total_tags = 0
for repo in repos_to_clean:
count, deleted = cleanup_repo(
repo, args.keep, dry_run, protected_tags, pr_sha=args.pr_sha, pr_open_set=pr_open_set
)
total_tags += count
total_deleted += deleted
print()
print("=" * 60)
print("清理完成")
print(" 总tag数:", total_tags)
if dry_run:
print(" 预览将删除(去重后):", total_deleted, "个manifest")
else:
print(" 已删除:", total_deleted, "个manifest")
print("=" * 60)
if __name__ == "__main__":
main()
+156
View File
@@ -0,0 +1,156 @@
#!/usr/bin/env bash
# 自动审批:CI全绿后自动approve PR
# 环境变量:GITHUB_TOKEN, REVIEW_TOKEN, PR_NUMBER, PR_HEAD_SHA, GITHUB_API_URL, GITHUB_REPOSITORY
set -eu
set -eu
echo "PR #${PR_NUMBER} - 检查CI状态并自动审批"
# 检查是否纯前端改动
API_URL="${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/files?limit=300"
FILES=$(curl -s -H "Authorization: token ${GITHUB_TOKEN}" "$API_URL" | python3 -c "import sys,json; [print(f['filename']) for f in json.load(sys.stdin)]")
FRONTEND_COUNT=$(echo "$FILES" | grep -c '^apps/web/' || true)
BACKEND_COUNT=$(echo "$FILES" | grep -cv '^apps/web/' || true)
TOTAL=$(echo "$FILES" | grep -cv '^$' || true)
echo "变更文件: ${TOTAL} 个 (前端: ${FRONTEND_COUNT}, 后端/公共: ${BACKEND_COUNT})"
if [ "$BACKEND_COUNT" = "0" ] && [ "$FRONTEND_COUNT" -gt "0" ]; then
SKIP_BACKEND=true
echo "✅ 纯前端改动,只检查Frontend Lint"
else
SKIP_BACKEND=false
echo "🔧 包含后端/公共变更,检查全部CI"
fi
# 定义需要检查的context
if [ "$SKIP_BACKEND" = "true" ]; then
CONTEXTS=("CI/CD Pipeline / Frontend Lint (pull_request)")
else
CONTEXTS=(
"CI/CD Pipeline / Validate - Code Quality (pull_request)"
"CI/CD Pipeline / Validate - Type Check (mypy) (pull_request)"
"CI/CD Pipeline / Validate - Migration (alembic) (pull_request)"
"CI/CD Pipeline / Frontend Lint (pull_request)"
"CI/CD Pipeline / Unit Tests (pull_request)"
"CI/CD Pipeline / Frontend Unit Tests (pull_request)"
"CI/CD Pipeline / PR Build API Image (pull_request)"
"CI/CD Pipeline / PR Build Web Image (pull_request)"
"CI/CD Pipeline / PR Build Worker Image (pull_request)"
)
fi
echo "需要通过的CI检查: ${#CONTEXTS[@]} 项(与分支保护required门禁一致)"
for ctx in "${CONTEXTS[@]}"; do
echo " - $ctx"
done
echo
# 初始等待30秒,给CI启动写status的时间
echo "等待30秒让CI启动..."
sleep 30
# 轮询等待,最多20分钟(120次x10秒)
for attempt in $(seq 1 12); do # 短作业模式:最多等2分钟(12次x10秒),不满足就退出等下次触发
ALL_SUCCESS=true
ANY_FAILED=false
ANY_PENDING=false
echo "--- 第${attempt}次检查 ($(date '+%H:%M:%S')) ---"
# 调用辅助脚本检查每个context状态
for ctx in "${CONTEXTS[@]}"; do
STATE=$(python3 scripts/check_ci_status.py "$GITHUB_TOKEN" "$GITHUB_REPOSITORY" "$PR_HEAD_SHA" "$ctx")
echo " $ctx: $STATE"
if [ "$STATE" != "success" ]; then
ALL_SUCCESS=false
fi
if [ "$STATE" = "failure" ] || [ "$STATE" = "error" ]; then
ANY_FAILED=true
fi
if [ "$STATE" = "pending" ] || [ "$STATE" = "null" ]; then
ANY_PENDING=true
fi
done
if [ "$ALL_SUCCESS" = "true" ]; then
echo
echo "✅ 所有CI检查通过,自动审批 PR #${PR_NUMBER}"
# 检查是否已有审批
EXISTING=$(curl -s -H "Authorization: token ${REVIEW_TOKEN}" \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/reviews" \
| python3 -c "import sys,json; reviews=json.load(sys.stdin); print('yes' if any(r.get('state')=='APPROVED' for r in reviews) else 'no')")
if [ "$EXISTING" = "yes" ]; then
echo "️ PR #${PR_NUMBER} 已有审批,跳过"
exit 0
fi
# 第一步:创建PENDING review
echo "创建review..."
REVIEW_CREATE=$(curl -s -X POST \
-H "Authorization: token ${REVIEW_TOKEN}" \
-H "Content-Type: application/json" \
-d '{"event": "PENDING", "body": "CI全绿,自动审批通过。"}' \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/reviews")
REVIEW_ID=$(echo "$REVIEW_CREATE" | python3 -c "import sys,json; print(json.load(sys.stdin).get('id',''))")
REVIEW_STATE=$(echo "$REVIEW_CREATE" | python3 -c "import sys,json; print(json.load(sys.stdin).get('state',''))")
echo "创建结果: id=$REVIEW_ID state=$REVIEW_STATE"
if [ -z "$REVIEW_ID" ]; then
echo "❌ 创建review失败"
echo "$REVIEW_CREATE"
exit 1
fi
if [ "$REVIEW_STATE" = "APPROVED" ]; then
echo "✅ 自动审批成功(直接创建为APPROVED)"
exit 0
fi
# 第二步:submit review为APPROVED
echo "提交review审批..."
SUBMIT_CODE=$(curl -s -o /tmp/submit_resp.json -w "%{http_code}" \
-X POST \
-H "Authorization: token ${REVIEW_TOKEN}" \
-H "Content-Type: application/json" \
-d '{"event": "APPROVED", "body": "CI全绿,自动审批通过。"}' \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/reviews/${REVIEW_ID}")
echo "提交API HTTP状态: $SUBMIT_CODE"
cat /tmp/submit_resp.json 2>/dev/null || true
echo
if [ "$SUBMIT_CODE" = "200" ] || [ "$SUBMIT_CODE" = "201" ]; then
FINAL_STATE=$(python3 -c "import json; print(json.load(open('/tmp/submit_resp.json')).get('state',''))" 2>/dev/null || echo "?")
echo "✅ 自动审批成功 (state: $FINAL_STATE)"
exit 0
else
echo "❌ 提交审批失败"
exit 1
fi
fi
# 还有CI在跑 → 继续等
if [ "$ANY_PENDING" = "true" ]; then
echo "⏳ CI仍在运行中(第${attempt}/12次),超时后将退出等待下次触发..."
sleep 10
continue
fi
# 所有CI都跑完了但有失败 → 退出
if [ "$ANY_FAILED" = "true" ]; then
echo
echo "❌ CI检查有失败项,不自动审批"
exit 0
fi
sleep 10
done
echo
echo "⏰ 快速检查超时(2分钟),CI尚未完成,退出等待下次触发(workflow_run事件或5分钟定时扫描)"
exit 0
+388
View File
@@ -0,0 +1,388 @@
#!/usr/bin/env python3
"""CI中自动修复代码格式(Python: black + isort | Frontend: prettier),并推送回原分支。
- PR事件:所有PR只要Code Quality因格式问题失败,自动修复并push回源分支
- Push事件(develop/main):自动修复并push回原分支,保持主干格式永远正确
- 防循环:修复commit带 [skip ci-format-check] 标记,检测到该标记则跳过修复
- 只修格式(black/isort/prettier),ruff逻辑类错误不动
当code quality检查因格式问题失败时触发。
"""
import json
import os
import subprocess
import sys
import time
import urllib.request
def run(cmd, check=True, capture=True, cwd=None):
"""运行shell命令"""
result = subprocess.run(cmd, shell=True, capture_output=capture, text=True, cwd=cwd)
if check and result.returncode != 0:
print(f"命令失败: {cmd}", file=sys.stderr)
if result.stderr:
print(result.stderr, file=sys.stderr)
sys.exit(1)
return result
def ensure_git_repo(api_url, repo, token, pr_number):
"""确保当前目录是git仓库,并切换到PR源分支。
checkout脚本用tarball方式下载代码(PR merge后的commit),没有.git目录。
这里自动初始化git仓库,fetch PR源分支并强制checkout
使工作区变为PR源分支的代码,确保后续格式化修复基于源分支。
"""
if os.path.exists(".git"):
return
print("检测到tarball checkout(无.git目录),自动初始化git仓库...")
# 构造带认证的远端URL
server_url = api_url.rsplit("/api/v1", 1)[0]
remote_url = f"{server_url.replace('https://', f'https://x-access-token:{token}@')}/{repo}.git"
# 获取PR的源分支
pr_api_url = f"{api_url}/repos/{repo}/pulls/{pr_number}"
req_obj = urllib.request.Request(pr_api_url, headers={"Authorization": f"token {token}"})
with urllib.request.urlopen(req_obj) as resp:
pr = json.loads(resp.read())
head_branch = pr["head"]["ref"]
print(f"PR源分支: {head_branch}")
# 初始化git
run("git init -q")
run(f"git remote add origin {remote_url}")
run('git config user.name "CI Bot"')
run('git config user.email "ci-bot@xiaoxiajianji.com"')
# fetch源分支(浅克隆,只要最新commit)
print("fetch源分支...")
run(f"git fetch --depth=1 origin {head_branch}")
# 强制checkout到源分支(覆盖tarball内容)
# tarball是merge后的commit,源分支才是我们要修改并推送的目标
print("切换到源分支...")
run(f"git checkout -f -B {head_branch} FETCH_HEAD")
result = run("git status --porcelain")
if result.stdout.strip():
n = len(result.stdout.strip().splitlines())
print(f"⚠️ 工作区有 {n} 个未追踪文件")
else:
print("✅ git仓库就绪,工作区clean")
return head_branch
def ensure_git_repo_for_push(api_url, repo, token, branch_name):
"""push事件下确保git仓库可用,并切换到目标分支。
checkout脚本用tarball方式下载代码,没有.git目录。
这里自动初始化git仓库,fetch目标分支并checkout。
"""
if os.path.exists(".git"):
# 已有git,确认在正确分支
result = run("git rev-parse --abbrev-ref HEAD", check=False)
if result.stdout.strip() == branch_name:
return
# 不在目标分支,切换
run(f"git checkout {branch_name}", check=False)
return
print(f"检测到tarball checkout(无.git目录),初始化git仓库(push模式,分支: {branch_name}...")
server_url = api_url.rsplit("/api/v1", 1)[0]
remote_url = f"{server_url.replace('https://', f'https://x-access-token:{token}@')}/{repo}.git"
run("git init -q")
run(f"git remote add origin {remote_url}")
run('git config user.name "CI Bot"')
run('git config user.email "ci-bot@xiaoxiajianji.com"')
print(f"fetch {branch_name} 分支...")
run(f"git fetch --depth=1 origin {branch_name}")
print(f"切换到 {branch_name} 分支...")
run(f"git checkout -f -B {branch_name} FETCH_HEAD")
result = run("git status --porcelain")
if result.stdout.strip():
n = len(result.stdout.strip().splitlines())
print(f"⚠️ 工作区有 {n} 个未追踪文件")
else:
print("✅ git仓库就绪,工作区clean")
def get_changed_files(pr_number, api_url, token):
"""获取PR中变更的文件列表"""
url = f"{api_url}/pulls/{pr_number}/files?limit=100"
req = urllib.request.Request(url, headers={"Authorization": f"token {token}"})
with urllib.request.urlopen(req) as resp:
files = json.loads(resp.read())
return [f["filename"] for f in files if f["status"] != "removed"]
def get_pr_head_branch(pr_number, api_url, token):
"""获取PR的来源分支名"""
url = f"{api_url}/pulls/{pr_number}"
req = urllib.request.Request(url, headers={"Authorization": f"token {token}"})
with urllib.request.urlopen(req) as resp:
pr = json.loads(resp.read())
return pr["head"]["ref"]
def fix_python(target_py_files, scan_mode):
"""修复 Python 文件格式 (black + isort)"""
if not target_py_files:
print("没有需要修复的 Python 文件,跳过")
return
target_str = " ".join(target_py_files)
print()
print("--- black 格式化 ---")
result = run(f"python3 -m black {target_str}", check=False)
print(result.stdout[-500:] if result.stdout else "")
if result.returncode != 0:
print("black执行失败,但继续尝试isort", file=sys.stderr)
print()
print("--- isort 排序 ---")
result = run(f"python3 -m isort {target_str}", check=False)
print(result.stdout[-500:] if result.stdout else "")
if result.returncode != 0:
print("isort执行失败", file=sys.stderr)
def fix_frontend(target_fe_files, scan_mode, repo_root):
"""修复前端文件格式 (prettier)"""
if not target_fe_files:
print("没有需要修复的前端文件,跳过")
return
# 检查 prettier 是否可用
web_dir = os.path.join(repo_root, "apps", "web")
prettier_bin = os.path.join(web_dir, "node_modules", ".bin", "prettier")
if not os.path.exists(prettier_bin):
print()
print("--- 安装前端依赖 (prettier) ---")
result = run("npm install --no-audit --no-fund --prefer-offline", check=False, cwd=web_dir)
if result.returncode != 0:
print("npm install 失败,跳过 prettier 修复", file=sys.stderr)
return
print("依赖安装完成")
if not os.path.exists(prettier_bin):
print("prettier 仍不可用,跳过", file=sys.stderr)
return
print()
print("--- prettier 格式化 ---")
if scan_mode == "incremental":
# 增量模式:只格式化变更的前端文件
target_str = " ".join(target_fe_files)
cmd = f"{prettier_bin} --write {target_str}"
else:
# 全量模式:格式化整个前端目录
cmd = f"{prettier_bin} --write ."
result = run(cmd, check=False, cwd=web_dir if scan_mode != "incremental" else repo_root)
print(result.stdout[-800:] if result.stdout else "")
if result.stderr:
print(result.stderr[-500:], file=sys.stderr)
def main():
event_name = os.environ.get("GITHUB_EVENT_NAME", "")
github_ref = os.environ.get("GITHUB_REF", "")
api_url = os.environ.get("GITHUB_API_URL", "")
repo = os.environ.get("GITHUB_REPOSITORY", "")
token = os.environ.get("REVIEW_TOKEN", "") or os.environ.get("GITHUB_TOKEN", "")
scan_mode = os.environ.get("SCAN_MODE", "full")
changed_files_env = os.environ.get("CHANGED_FILES", "")
if not token:
print("缺少REVIEW_TOKEN或GITHUB_TOKEN,无法推送修复", file=sys.stderr)
sys.exit(1)
repo_root = os.getcwd()
# ====== Push事件处理(develop/main等受保护分支) ======
if event_name == "push":
# 从 refs/heads/xxx 提取分支名
if not github_ref.startswith("refs/heads/"):
print(f"push事件但refs格式异常: {github_ref},跳过")
return
branch_name = github_ref.replace("refs/heads/", "")
# 只在受保护分支(develop/main)上自动修复并推送
protected_branches = {"develop", "main", "master"}
if branch_name not in protected_branches:
print(f"push事件,分支 {branch_name} 不是受保护分支,跳过自动修复")
return
print("=== Push事件:检测到格式问题,自动修复并推送回原分支 ===")
print(f"分支: {branch_name}")
print(f"扫描模式: {scan_mode}")
# 初始化git仓库
ensure_git_repo_for_push(api_url, repo, token, branch_name)
head_branch = branch_name
fix_mode = "auto_fix_and_push"
# ====== PR事件处理 ======
elif event_name == "pull_request":
pr_number = github_ref.split("/")[2] if github_ref.startswith("refs/pull/") else ""
if not pr_number:
print("无法获取PR号,跳过自动修复")
return
# 获取PR信息
pr_info_url = f"{api_url}/repos/{repo}/pulls/{pr_number}"
req_pr = urllib.request.Request(pr_info_url, headers={"Authorization": f"token {token}"})
with urllib.request.urlopen(req_pr) as resp:
pr_info = json.loads(resp.read())
pr_author = pr_info.get("user", {}).get("login", "")
print(f"PR作者: {pr_author}")
# 防循环检测:检查最新commit是否已经是格式修复commit
# 修复commit message 带 [skip ci-format-check] 标记,检测到则跳过
head_branch_tmp = pr_info.get("head", {}).get("ref", "")
skip_marker = "[skip ci-format-check]"
try:
commits_url = f"{api_url}/repos/{repo}/pulls/{pr_number}/commits?limit=3"
req_commits = urllib.request.Request(commits_url, headers={"Authorization": f"token {token}"})
with urllib.request.urlopen(req_commits) as resp_commits:
commits = json.loads(resp_commits.read())
latest_msg = commits[0].get("commit", {}).get("message", "") if commits else ""
if skip_marker in latest_msg:
print(f"检测到最新commit包含 {skip_marker} 标记,跳过格式修复(防循环)")
print("本次格式检查失败是格式修复commit触发的CI回跑,属正常现象")
sys.exit(0)
except Exception as e:
print(f"⚠️ 防循环检测失败,继续执行: {e}")
# 所有PR都自动修复格式(不再区分人/Agent)
print("检测到格式问题,将自动修复并推送回分支")
fix_mode = "auto_fix_and_push"
print("=== 检测到代码格式问题,尝试自动修复 ===")
print(f"PR #{pr_number}")
print(f"扫描模式: {scan_mode}")
# 确保git仓库可用(tarball checkout模式下自动初始化)
head_branch = ensure_git_repo(api_url, repo, token, pr_number)
# ====== 其他事件跳过 ======
else:
print(f"事件 {event_name} 不支持自动修复,跳过")
return
# 前端文件扩展名
fe_extensions = (
".ts",
".tsx",
".js",
".jsx",
".css",
".scss",
".less",
".json",
".html",
".md",
".yaml",
".yml",
)
py_extensions = (".py",)
# 确定要修复的文件范围
if scan_mode == "incremental" and changed_files_env:
all_changed = changed_files_env.split()
target_py_files = [f for f in all_changed if f.endswith(py_extensions)]
target_fe_files = [f for f in all_changed if f.endswith(fe_extensions)]
print(f"增量模式: {len(target_py_files)} 个Python文件, {len(target_fe_files)} 个前端文件")
else:
target_py_files = ["alembic", "apps", "packages", "tests", "scripts"]
target_fe_files = ["apps/web"]
print("全量模式,修复所有文件")
# Python 格式化
fix_python(target_py_files, scan_mode)
# 前端格式化
if scan_mode != "incremental":
fix_frontend(["apps/web"], scan_mode, repo_root)
else:
fix_frontend(target_fe_files, scan_mode, repo_root)
# 检查是否有改动
result = run("git status --porcelain")
if not result.stdout.strip():
print()
print("没有需要提交的格式改动")
return
print()
print("变更文件:")
for line in result.stdout.strip().split("\n"):
print(f" {line}")
# 提交修复
run("git add -A")
run('git commit -m "style: auto-format with black + isort + prettier [skip ci-format-check]"')
# 推送(head_branch已从ensure_git_repo获取)
print(f"\nPR来源分支: {head_branch}")
print("推送格式修复到远端...")
# 推送前先 rebase 拉取远端最新,避免快进冲突
# 最多重试 3 次:rebase → push,失败则重新拉取再试
max_retries = 3
push_success = False
last_error = ""
for attempt in range(1, max_retries + 1):
print(f" 尝试 {attempt}/{max_retries}: 拉取最新代码并推送...")
# 先拉取远端最新 commit 并 rebase
fetch_result = run(f"git fetch origin {head_branch}", check=False)
if fetch_result.returncode != 0:
last_error = f"git fetch 失败: {fetch_result.stderr.strip()}"
print(f" {last_error}")
time.sleep(2)
continue
rebase_result = run(f"git rebase origin/{head_branch}", check=False)
if rebase_result.returncode != 0:
last_error = f"git rebase 失败,中止并重置: {rebase_result.stderr.strip()[:200]}"
print(f" {last_error}")
run("git rebase --abort", check=False)
# rebase 失败通常是冲突,重试没用,直接跳出
break
# 推送
push_result = run(f'git push origin "HEAD:{head_branch}"', check=False)
if push_result.returncode == 0:
push_success = True
break
last_error = push_result.stderr.strip() or push_result.stdout.strip()
print(f" push 失败: {last_error[:200]}")
time.sleep(3)
if not push_success:
print(f"\n❌ 推送失败(已重试 {max_retries} 次)", file=sys.stderr)
print(f"最后错误: {last_error}", file=sys.stderr)
sys.exit(1)
print()
print("✅ 格式已自动修复并推送回分支")
print("新的commit会重新触发CI检查")
if __name__ == "__main__":
main()
+148
View File
@@ -0,0 +1,148 @@
#!/usr/bin/env bash
# 自动合并:CI全绿+已审批后自动squash merge PR到develop
# 短作业模式:只检查一次,不满足条件就退出,由pr-auto-scan定时兜底
# 环境变量:GITHUB_TOKEN, MERGE_TOKEN, PR_NUMBER, PR_HEAD_SHA, BASE_REF, GITHUB_API_URL, GITHUB_REPOSITORY
set -eu
echo "PR #${PR_NUMBER} - 检查CI状态+审批并自动合并到${BASE_REF}"
echo
echo "模式: 短作业(只检查一次,不满足则退出,由pr-auto-scan定时兜底)"
echo
# 只合develop分支
if [ "$BASE_REF" != "develop" ]; then
echo "Skip: 目标分支不是develop"
exit 0
fi
# 判断是否纯前端改动
FILES=$(curl -s -H "Authorization: token ${GITHUB_TOKEN}" \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/files?limit=300" \
| python3 -c "import sys,json; [print(f['filename']) for f in json.load(sys.stdin)]")
TOTAL=$(echo "$FILES" | grep -cv '^$' || true)
FRONTEND_COUNT=$(echo "$FILES" | grep -c '^apps/web/' || true)
BACKEND_COUNT=$((TOTAL - FRONTEND_COUNT))
echo "变更文件: ${TOTAL} 个 (前端: ${FRONTEND_COUNT}, 后端/公共: ${BACKEND_COUNT})"
echo
# 使用统一的CI Gate门禁(单一检查点,自动处理前端/后端/全栈跳过逻辑)
CONTEXTS=(
"CI/CD Pipeline / CI Gate (pull_request)"
)
echo "检查CI Gate统一门禁"
echo
# 等待60秒,给CI启动写status的时间
echo "等待60秒让CI启动..."
sleep 60
# 405计数器(单次运行内重试)
MERGE_405_COUNT=0
MAX_405_RETRIES=3
check_and_merge() {
ALL_SUCCESS=true
ANY_FAILED=false
ANY_PENDING=false
echo "--- 检查CI状态 ($(date '+%H:%M:%S')) ---"
# 检查CI状态
for ctx in "${CONTEXTS[@]}"; do
STATE=$(python3 scripts/check_ci_status.py "$GITHUB_TOKEN" "$GITHUB_REPOSITORY" "$PR_HEAD_SHA" "$ctx")
echo " CI: ${ctx##*/}: $STATE"
if [ "$STATE" != "success" ]; then
ALL_SUCCESS=false
fi
if [ "$STATE" = "failure" ] || [ "$STATE" = "error" ]; then
ANY_FAILED=true
fi
if [ "$STATE" = "pending" ]; then
ANY_PENDING=true
fi
done
# CI有失败 → 不合并,直接退出
if [ "$ANY_FAILED" = "true" ]; then
echo
echo "❌ CI有失败项,不自动合并"
exit 0
fi
# CI未全绿(pending中)→ 退出,等下次触发
if [ "$ALL_SUCCESS" != "true" ]; then
echo
echo "⏳ CI尚未全绿(仍有pending),退出等待下次触发"
echo " pr-auto-scan每5分钟扫描一次,CI通过后会自动合并)"
exit 0
fi
# CI全绿 → 合并
echo
echo "✅ CI全绿,执行自动合并"
echo "等待30秒冷却,给Gitea内部状态同步时间..."
sleep 30
# 幂等检查:PR是否还是open
PR_STATE=$(curl -s -H "Authorization: token ${MERGE_TOKEN}" \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}" \
| python3 -c "import sys,json; print(json.load(sys.stdin).get('state',''))")
if [ "$PR_STATE" != "open" ]; then
echo "PR状态为 ${PR_STATE},无需合并"
exit 0
fi
# 执行squash merge
HTTP_CODE=$(curl -s -o /tmp/merge_resp.json -w "%{http_code}" \
-X POST \
-H "Authorization: token ${MERGE_TOKEN}" \
-H "Content-Type: application/json" \
-d '{"do":"squash","merge_title_field":"","merge_message_field":"","delete_branch_after_merge":true,"force_merge":false}' \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/merge")
echo "合并API HTTP状态: $HTTP_CODE"
if [ "$HTTP_CODE" = "200" ]; then
echo "✅ 自动合并成功"
exit 0
elif [ "$HTTP_CODE" = "405" ]; then
MERGE_405_COUNT=$((MERGE_405_COUNT + 1))
echo "⚠️ 合并返回405(第${MERGE_405_COUNT}次),可能CI状态尚未同步或有未解决的门禁"
cat /tmp/merge_resp.json 2>/dev/null || true
echo
if [ "$MERGE_405_COUNT" -ge "$MAX_405_RETRIES" ]; then
echo "⚠️ 连续${MAX_405_RETRIES}次合并返回405,放弃本次自动合并"
echo " pr-auto-scan会继续尝试,需人工确认是否有冲突或门禁问题)"
curl -s -X POST \
-H "Authorization: token ${MERGE_TOKEN}" \
-H "Content-Type: application/json" \
-d '{"body": "Auto merge skipped after multiple 405 errors: PR may have conflicts or unresolved checks. Please review manually. This is not a CI failure."}' \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/issues/${PR_NUMBER}/comments" > /dev/null 2>&1 || true
exit 0
fi
echo "30秒后重试..."
sleep 30
return 1 # 重试
else
echo "❌ 自动合并失败 (HTTP $HTTP_CODE)"
cat /tmp/merge_resp.json 2>/dev/null || true
curl -s -X POST \
-H "Authorization: token ${MERGE_TOKEN}" \
-H "Content-Type: application/json" \
-d "{\"body\": \"Auto merge failed (HTTP ${HTTP_CODE}), please check manually.\"}" \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/issues/${PR_NUMBER}/comments" > /dev/null 2>&1 || true
exit 1
fi
}
# 最多重试3次(用于405重试,非CI轮询)
for i in 1 2 3; do
if check_and_merge; then
exit 0
fi
done
echo
echo "本次检查未满足合并条件,退出。pr-auto-scan每5分钟会继续扫描。"
exit 0
+363
View File
@@ -0,0 +1,363 @@
#!/bin/bash
# ===========================================
# 金丝雀发布脚本 - 分阶段灰度到全量
# ===========================================
# 在 CI Runner 上执行,通过 SSH 控制生产服务器执行灰度发布。
# 流程:5%灰度 → 20%灰度 → 50%灰度 → 100%全量
# 每阶段自动健康检查,失败自动回滚。
#
# 用法:
# IMAGE_TAG=v0.1.130 ./scripts/ci/canary_release.sh
#
# 环境变量:
# IMAGE_TAG - 新版本镜像标签 (必填)
# CANARY_STAGES - 灰度阶段配置,格式: "百分比:等待秒数" 用逗号分隔
# 默认: "5:600,20:900,50:1200"
# PROD_API_URL - Production API 公网地址
# PROD_WEB_URL - Production Web 公网地址
# PRODUCTION_SSH_HOST - 生产服务器 SSH 地址
# PRODUCTION_SSH_USER - SSH 用户名
# PRODUCTION_SSH_PORT - SSH 端口
# PRODUCTION_SSH_KEY - SSH 私钥内容
# ACR_USERNAME - 容器镜像仓库用户名
# ACR_PASSWORD - 容器镜像仓库密码
# CI_NOTIFY_WEBHOOK - 通知 Webhook
# SKIP_ROLLBACK - 失败时不自动回滚 (调试用)
set -euo pipefail
SCRIPT_DIR="$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)"
REPO_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
# 配置
IMAGE_TAG="${IMAGE_TAG:-}"
CANARY_STAGES="${CANARY_STAGES:-5:600,20:900,50:1200}"
PROD_API_URL="${PROD_API_URL:-https://api.xiaoxiajianji.com}"
PROD_WEB_URL="${PROD_WEB_URL:-https://saas.xiaoxiajianji.com}"
PRODUCTION_SSH_HOST="${PRODUCTION_SSH_HOST:-47.98.113.167}"
PRODUCTION_SSH_USER="${PRODUCTION_SSH_USER:-root}"
PRODUCTION_SSH_PORT="${PRODUCTION_SSH_PORT:-22222}"
# gray_deploy.sh 的镜像命名格式是 ${REGISTRY}-component:tag
# 需要与 ACR 镜像名 xiaoxia-registry.../xiaoxiakeji/xiaoxia-saas-api:tag 匹配
GRAY_REGISTRY="${GRAY_REGISTRY:-xiaoxia-registry.cn-hangzhou.cr.aliyuncs.com/xiaoxiakeji/xiaoxia-saas}"
ACR_REGISTRY_HOST="${ACR_REGISTRY_HOST:-xiaoxia-registry.cn-hangzhou.cr.aliyuncs.com}"
ACR_USERNAME="${ACR_USERNAME:-}"
ACR_PASSWORD="${ACR_PASSWORD:-}"
SKIP_ROLLBACK="${SKIP_ROLLBACK:-false}"
if [[ -z "$IMAGE_TAG" ]]; then
echo "ERROR: IMAGE_TAG is required"
exit 1
fi
# 颜色
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m'
log_info() { echo -e "${GREEN}[INFO]${NC} $1"; }
log_warn() { echo -e "${YELLOW}[WARN]${NC} $1"; }
log_error() { echo -e "${RED}[ERROR]${NC} $1"; }
log_step() { echo -e "${BLUE}[STEP]${NC} $1"; }
# ===========================================
# SSH 配置
# ===========================================
SSH_KEY_PATH=""
setup_ssh() {
if [ -f /root/.ssh/xiaoxia_runtime_builder ]; then
SSH_KEY_PATH="/root/.ssh/xiaoxia_runtime_builder"
elif [ -f "$HOME/.ssh/xiaoxia_runtime_builder" ]; then
SSH_KEY_PATH="$HOME/.ssh/xiaoxia_runtime_builder"
elif [ -n "${PRODUCTION_SSH_KEY:-}" ]; then
SSH_KEY_PATH="$HOME/.ssh/canary_deploy_key"
mkdir -p "$HOME/.ssh"
printf '%s\n' "$PRODUCTION_SSH_KEY" > "$SSH_KEY_PATH"
chmod 600 "$SSH_KEY_PATH"
else
log_error "没有可用的 SSH 密钥"
return 1
fi
ssh-keyscan -p "$PRODUCTION_SSH_PORT" -H "$PRODUCTION_SSH_HOST" >> ~/.ssh/known_hosts 2>/dev/null || true
log_info "SSH 已配置: ${PRODUCTION_SSH_USER}@${PRODUCTION_SSH_HOST}:${PRODUCTION_SSH_PORT}"
}
run_ssh() {
local cmd="$1"
ssh -p "$PRODUCTION_SSH_PORT" -i "$SSH_KEY_PATH" -o StrictHostKeyChecking=no \
"${PRODUCTION_SSH_USER}@${PRODUCTION_SSH_HOST}" "$cmd"
}
# ===========================================
# 上传脚本 + Docker登录
# ===========================================
prepare_server() {
log_step "准备生产服务器环境"
# 创建临时目录
run_ssh "mkdir -p /tmp/canary-release"
# 上传 gray_deploy.sh
local gray_script="$REPO_ROOT/scripts/gray_deploy.sh"
if [[ -f "$gray_script" ]]; then
cat "$gray_script" | run_ssh "cat > /tmp/canary-release/gray_deploy.sh && chmod +x /tmp/canary-release/gray_deploy.sh"
log_info " gray_deploy.sh 已上传"
else
log_error "找不到 gray_deploy.sh: $gray_script"
return 1
fi
# 上传 rollback_gray.sh
local rollback_script="$REPO_ROOT/scripts/rollback_gray.sh"
if [[ -f "$rollback_script" ]]; then
cat "$rollback_script" | run_ssh "cat > /tmp/canary-release/rollback_gray.sh && chmod +x /tmp/canary-release/rollback_gray.sh"
log_info " rollback_gray.sh 已上传"
else
log_warn "找不到 rollback_gray.sh"
fi
# 上传 ci_production_deploy.sh
local prod_deploy="$REPO_ROOT/scripts/ci_production_deploy.sh"
if [[ -f "$prod_deploy" ]]; then
cat "$prod_deploy" | run_ssh "cat > /tmp/canary-release/ci_production_deploy.sh && chmod +x /tmp/canary-release/ci_production_deploy.sh"
log_info " ci_production_deploy.sh 已上传"
else
log_error "找不到 ci_production_deploy.sh: $prod_deploy"
return 1
fi
# Docker 登录到 ACR
if [[ -n "$ACR_USERNAME" && -n "$ACR_PASSWORD" ]]; then
log_info " Docker 登录到 ACR..."
run_ssh "docker login '$ACR_REGISTRY_HOST' -u '$ACR_USERNAME' -p '$ACR_PASSWORD' 2>/dev/null" || \
log_warn " Docker login 失败(可能已有凭证),将尝试直接 pull"
fi
log_info "✅ 服务器环境准备完成"
}
# ===========================================
# 健康检查(公网访问)
# ===========================================
health_check() {
local stage_name="$1"
local timeout="${2:-120}"
local interval=5
local elapsed=0
log_step "健康检查 - $stage_name (超时 ${timeout}s)"
while [ $elapsed -lt $timeout ]; do
local api_ok=false
local web_ok=false
# 检查 API
local api_code=$(curl -s -o /dev/null -w "%{http_code}" \
--connect-timeout 5 --max-time 10 \
"${PROD_API_URL}/health" 2>/dev/null || echo "000")
if [[ "$api_code" == "200" ]]; then
api_ok=true
fi
# 检查 Web
local web_code=$(curl -s -o /dev/null -w "%{http_code}" \
--connect-timeout 5 --max-time 10 \
"$PROD_WEB_URL" 2>/dev/null || echo "000")
if [[ "$web_code" == "200" || "$web_code" == "301" || "$web_code" == "302" ]]; then
web_ok=true
fi
if $api_ok && $web_ok; then
log_info "✅ 健康检查通过 (API=$api_code, Web=$web_code)"
return 0
fi
log_warn " 等待中... API=$api_code, Web=$web_code (${elapsed}s/${timeout}s)"
sleep $interval
elapsed=$((elapsed + interval))
done
log_error "❌ 健康检查超时"
return 1
}
# ===========================================
# 灰度发布
# ===========================================
gray_deploy() {
local pct="$1"
log_step "灰度发布 ${pct}% - $IMAGE_TAG"
run_ssh "cd /tmp/canary-release && \
REGISTRY='$GRAY_REGISTRY' \
./gray_deploy.sh '$IMAGE_TAG' '$pct'"
}
# ===========================================
# 全量部署
# ===========================================
full_deploy() {
log_step "全量部署 - $IMAGE_TAG"
run_ssh "cd /tmp/canary-release && \
IMAGE_TAG='$IMAGE_TAG' \
ACR_USERNAME='$ACR_USERNAME' \
ACR_PASSWORD='$ACR_PASSWORD' \
sh ./ci_production_deploy.sh"
}
# ===========================================
# 灰度回滚
# ===========================================
rollback_gray() {
log_error "执行灰度回滚..."
if [[ "$SKIP_ROLLBACK" == "true" ]]; then
log_warn "SKIP_ROLLBACK=true,跳过回滚"
return
fi
if run_ssh "test -f /tmp/canary-release/rollback_gray.sh"; then
run_ssh "cd /tmp/canary-release && ./rollback_gray.sh" || \
log_error "回滚脚本执行失败,请手动处理"
else
# 内联回滚逻辑
log_warn "使用内联回滚逻辑"
run_ssh '
NGINX_CONF="/etc/nginx/sites-enabled/00-xiaoxia-saas"
LATEST_BAK=$(ls -t "${NGINX_CONF}".bak.gray.* 2>/dev/null | head -1 || true)
if [[ -n "$LATEST_BAK" ]]; then
cp "$LATEST_BAK" "$NGINX_CONF"
else
sed -i "s|proxy_pass http://saas_api_backend|proxy_pass http://127.0.0.1:8001|g" "$NGINX_CONF"
sed -i "s|proxy_pass http://saas_web_backend/|proxy_pass http://127.0.0.1:3002/|g" "$NGINX_CONF"
fi
nginx -t && nginx -s reload
docker rm -f xiaoxia-api-canary xiaoxia-web-canary 2>/dev/null || true
' || log_error "回滚失败,请手动处理"
fi
}
# ===========================================
# 通知
# ===========================================
notify_status() {
local status="$1"
local message="$2"
if [ -n "${CI_NOTIFY_WEBHOOK:-}" ]; then
NOTIFY_MODE="$status" JOB_NAME="Canary Release - $message" \
python3 "$REPO_ROOT/scripts/ci_notify.py" 2>/dev/null || true
fi
}
# ===========================================
# 清理
# ===========================================
cleanup() {
log_step "清理生产服务器临时文件"
run_ssh "rm -rf /tmp/canary-release" 2>/dev/null || true
}
# ===========================================
# 主流程
# ===========================================
main() {
echo "==========================================="
echo " 🐦 金丝雀发布"
echo " 版本: $IMAGE_TAG"
echo " 阶段: $CANARY_STAGES"
echo "==========================================="
echo ""
setup_ssh
prepare_server
trap cleanup EXIT
# 解析灰度阶段
IFS=',' read -ra STAGES <<< "$CANARY_STAGES"
local total_stages=${#STAGES[@]}
local current_stage=0
# 逐阶段灰度
for stage in "${STAGES[@]}"; do
current_stage=$((current_stage + 1))
local pct=$(echo "$stage" | cut -d: -f1)
local wait_time=$(echo "$stage" | cut -d: -f2)
echo ""
echo "--- 阶段 $current_stage/$total_stages: ${pct}% 灰度 ---"
# 执行灰度发布
if ! gray_deploy "$pct"; then
log_error "灰度发布 ${pct}% 失败"
rollback_gray
notify_status "failure" "Stage ${pct}% Deploy Failed"
exit 1
fi
# 健康检查
if ! health_check "${pct}%灰度"; then
log_error "${pct}%灰度健康检查失败"
rollback_gray
notify_status "failure" "Stage ${pct}% Health Check Failed"
exit 1
fi
# 观察期
log_info "⏳ 观察期 ${wait_time}s,监控流量稳定性..."
local waited=0
local check_interval=60
while [ $waited -lt $wait_time ]; do
sleep $check_interval
waited=$((waited + check_interval))
# 每隔一段时间做一次快速健康检查
local api_code=$(curl -s -o /dev/null -w "%{http_code}" \
--connect-timeout 5 --max-time 10 \
"${PROD_API_URL}/health" 2>/dev/null || echo "000")
if [[ "$api_code" != "200" ]]; then
log_error "❌ 观察期内 API 异常 (HTTP $api_code),触发回滚"
rollback_gray
notify_status "failure" "Stage ${pct}% Watch Period Failed"
exit 1
fi
log_info " 观察中... ${waited}s/${wait_time}s (API=$api_code)"
done
log_info "${pct}%灰度阶段完成,稳定运行 ${wait_time}s"
done
# 全量部署
echo ""
echo "--- 最终阶段: 100% 全量部署 ---"
if ! full_deploy; then
log_error "全量部署失败"
notify_status "failure" "Full Deploy Failed"
exit 1
fi
# 最终健康检查
if ! health_check "全量部署" "180"; then
log_error "全量部署后健康检查失败"
notify_status "failure" "Full Deploy Health Check Failed"
exit 1
fi
# 清理 canary 容器
log_step "清理 Canary 容器"
run_ssh "docker rm -f xiaoxia-api-canary xiaoxia-web-canary 2>/dev/null || true" || true
echo ""
echo "==========================================="
echo " ✅ 金丝雀发布完成"
echo " 版本: $IMAGE_TAG"
echo " 状态: 100%全量运行"
echo "==========================================="
notify_status "success" "$IMAGE_TAG Fully Deployed"
}
main "$@"
+19
View File
@@ -0,0 +1,19 @@
"""CI ChatOps 工具包 - 飞书机器人对接 Gitea Actions
模块:
config - 配置管理(环境变量)
gitea_client - Gitea API 客户端封装
feishu_notify - 飞书通知(失败/恢复/E2E摘要)
ci_query - CI 状态查询
ci_trigger - CI 重跑触发
webhook_server - Gitea webhook 接收服务(FastAPI
"""
__all__ = [
"config",
"gitea_client",
"feishu_notify",
"ci_query",
"ci_trigger",
"webhook_server",
]
+296
View File
@@ -0,0 +1,296 @@
#!/usr/bin/env python3
"""
CI 状态查询模块 - 查询 run 列表、某分支/某 PR 的 CI 状态、失败详情
支持查询类型:
- list_runs: 列出最近的 workflow runs
- branch_status: 某分支最新 CI 状态
- pr_status: 某 PR 的 CI 状态
- failure_detail: 某次 run 的失败详情
用法:
python3 scripts/ci/chatops/ci_query.py --branch develop
python3 scripts/ci/chatops/ci_query.py --pr 123
python3 scripts/ci/chatops/ci_query.py --run-id 456 --detail
设计:
- 与飞书机器人 /ci status 命令对接
- 返回结构化数据,上层负责格式化输出
"""
import argparse
import sys
from . import config
from .gitea_client import GiteaClient
class CIQuery:
"""CI 状态查询器"""
def __init__(self, gitea_client=None):
self.gitea = gitea_client or GiteaClient()
# ── 查询方法 ──────────────────────────────────────
def get_branch_status(self, branch, limit=5):
"""获取指定分支最新的 CI 状态
Returns:
dict: {branch, latest_run, recent_runs, overall_status}
"""
runs, total = self.gitea.list_runs(branch=branch, limit=limit)
if not runs:
return {
"branch": branch,
"latest_run": None,
"recent_runs": [],
"overall_status": "no_runs",
"total_count": total,
}
latest = runs[0]
overall = self._derive_overall_status(runs)
return {
"branch": branch,
"latest_run": latest,
"recent_runs": runs,
"overall_status": overall,
"total_count": total,
}
def get_pr_status(self, pr_number):
"""获取指定 PR 的 CI 状态
Returns:
dict: {pr_number, pr_title, runs, overall_status}
"""
pr = self.gitea.get_pr(pr_number)
if not pr:
return {
"pr_number": pr_number,
"pr_title": "未知",
"runs": [],
"overall_status": "pr_not_found",
}
pr_title = pr.get("title", "")
runs = self.gitea.get_pr_ci_runs(pr_number, limit=10)
overall = self._derive_overall_status(runs) if runs else "no_runs"
return {
"pr_number": pr_number,
"pr_title": pr_title,
"runs": runs,
"overall_status": overall,
"head_sha": pr.get("head", {}).get("sha", ""),
}
def get_failure_detail(self, run_id):
"""获取某次 run 的失败详情
Returns:
dict: {run_info, failed_jobs, summary}
"""
run = self.gitea.get_run(run_id)
if not run:
return {"run_info": None, "failed_jobs": [], "summary": "Run not found"}
failed_jobs = self.gitea.get_failed_jobs_summary(run_id, max_lines_per_job=30)
summary_parts = []
for job in failed_jobs:
step = f"(步骤: {job['failed_step']}" if job["failed_step"] else ""
summary_parts.append(f"{job['name']}{step}")
summary = "\n".join(summary_parts) if summary_parts else "无失败 job(可能还在运行中)"
return {
"run_info": run,
"failed_jobs": failed_jobs,
"summary": summary,
"total_jobs": len(self.gitea.get_run_jobs(run_id)),
}
def list_recent_runs(self, status=None, branch=None, limit=10):
"""列出最近的 runs"""
runs, total = self.gitea.list_runs(status=status, branch=branch, limit=limit)
return {"runs": runs, "total_count": total}
# ── 辅助方法 ────────────────────────────────────
@staticmethod
def _derive_overall_status(runs):
"""根据最近 runs 推导整体状态
Returns:
success: 最近一次成功
failing: 最近一次失败(连续失败)
flaky: 有失败有成功(最近一次失败
running: 有正在运行的
unknown: 未知
"""
if not runs:
return "no_runs"
# 检查是否有运行中的
running = [r for r in runs if r.get("status") != "completed"]
if running:
return "running"
# 看最近一次
latest = runs[0]
latest_conclusion = latest.get("conclusion", "unknown")
if latest_conclusion == "success":
return "success"
if latest_conclusion == "failure":
# 检查是否连续失败
consecutive_failures = 0
for r in runs:
if r.get("conclusion") == "failure":
consecutive_failures += 1
else:
break
# 看之前有没有成功
has_success = any(r.get("conclusion") == "success" for r in runs)
if has_success:
return "flaky"
return "failing"
return "unknown"
# ── 格式化输出 ────────────────────────────────────
@staticmethod
def format_branch_status(status_data):
"""格式化分支状态为人类可读文本"""
branch = status_data["branch"]
latest = status_data["latest_run"]
overall = status_data["overall_status"]
status_emoji = {
"success": "",
"failing": "🔴",
"flaky": "🟡",
"running": "🔄",
"no_runs": "",
"unknown": "",
}.get(overall, "")
lines = [f"**CI 状态:{branch} 分支**", f"整体状态: {status_emoji} {overall}"]
if latest:
name = latest.get("name", "Unknown")
conclusion = latest.get("conclusion", latest.get("status", "unknown"))
run_id = latest.get("id", "")
created = latest.get("created_at", "")[:16].replace("T", " ")
run_url = f"{config.GITEA_URL}/{config.GITEA_REPO}/actions/runs/{run_id}"
lines.append(f"最新: [{name} #{run_id}]({run_url}) - {conclusion} ({created})")
recent = status_data["recent_runs"]
if len(recent) > 1:
lines.append(f"\n最近 {len(recent)} 次:")
for r in recent[:5]:
c = r.get("conclusion", r.get("status", "?"))
emoji = {"success": "", "failure": "", "skipped": "⏭️"}.get(c, "🔄")
lines.append(f" {emoji} #{r.get('id', '?')} {r.get('name', '?')[:30]} - {c}")
return "\n".join(lines)
@staticmethod
def format_pr_status(status_data):
"""格式化 PR 状态为人类可读文本"""
pr_num = status_data["pr_number"]
pr_title = status_data["pr_title"]
overall = status_data["overall_status"]
status_emoji = {
"success": "",
"failing": "🔴",
"flaky": "🟡",
"running": "🔄",
"no_runs": "",
"pr_not_found": "",
"unknown": "",
}.get(overall, "")
pr_url = f"{config.GITEA_URL}/{config.GITEA_REPO}/pulls/{pr_num}"
lines = [
f"**CI 状态:PR #{pr_num}**",
f"标题: [{pr_title}]({pr_url})",
f"状态: {status_emoji} {overall}",
]
runs = status_data["runs"]
if runs:
lines.append(f"\nCI Runs ({len(runs)}):")
for r in runs[:5]:
c = r.get("conclusion", r.get("status", "?"))
emoji = {"success": "", "failure": "", "skipped": "⏭️"}.get(c, "🔄")
run_id = r.get("id", "?")
run_url = f"{config.GITEA_URL}/{config.GITEA_REPO}/actions/runs/{run_id}"
lines.append(f" {emoji} [{r.get('name', '?')[:30]} #{run_id}]({run_url}) - {c}")
return "\n".join(lines)
# ── CLI 入口 ──────────────────────────────────────────
def main():
parser = argparse.ArgumentParser(description="CI 状态查询")
parser.add_argument("--branch", help="查询指定分支的 CI 状态")
parser.add_argument("--pr", type=int, help="查询指定 PR 的 CI 状态")
parser.add_argument("--run-id", help="查询指定 run 的详情")
parser.add_argument("--detail", action="store_true", help="显示失败详情")
parser.add_argument("--limit", type=int, default=5, help="返回数量限制")
parser.add_argument("--status", help="按状态过滤: success/failure/running")
args = parser.parse_args()
query = CIQuery()
if args.run_id:
if args.detail:
result = query.get_failure_detail(args.run_id)
print(f"Run #{args.run_id} 失败详情:")
print(result["summary"])
if result["failed_jobs"]:
print("\n详细日志尾部:")
for job in result["failed_jobs"]:
print(f"\n--- {job['name']} ---")
print(job["log_tail"][:500] if job["log_tail"] else "无日志")
else:
run = query.gitea.get_run(args.run_id)
if run:
print(f"Run #{args.run_id}: {run.get('name')} - {run.get('conclusion', run.get('status'))}")
print(f"分支: {run.get('head_branch', '?')}")
print(f"触发: {run.get('event', '?')}")
else:
print(f"Run {args.run_id} 不存在")
elif args.pr:
result = query.get_pr_status(args.pr)
print(CIQuery.format_pr_status(result))
elif args.branch:
result = query.get_branch_status(args.branch, limit=args.limit)
print(CIQuery.format_branch_status(result))
elif args.status:
result = query.list_recent_runs(status=args.status, limit=args.limit)
for r in result["runs"]:
print(
f"#{r.get('id')} {r.get('name')[:40]} - {r.get('conclusion', r.get('status'))} ({r.get('head_branch', '?')})"
)
else:
parser.print_help()
return 1
return 0
if __name__ == "__main__":
sys.exit(main())
+169
View File
@@ -0,0 +1,169 @@
#!/usr/bin/env python3
"""
CI 触发模块 - 重新运行失败 job、重跑整个 workflow、取消 run
支持操作:
- rerun_failed: 重跑失败的 jobs
- rerun_all: 重跑整个 workflow
- cancel: 取消运行中的 run
用法:
python3 scripts/ci/chatops/ci_trigger.py --run-id 456 --action rerun_failed
python3 scripts/ci/chatops/ci_trigger.py --run-id 456 --action rerun_all
python3 scripts/ci/chatops/ci_trigger.py --run-id 456 --action cancel
设计:
- 与飞书机器人 /ci rerun 命令对接
- 操作前自动校验 run 状态,避免无效操作
"""
import argparse
import sys
from . import config
from .gitea_client import GiteaClient
class CITrigger:
"""CI 操作触发器"""
def __init__(self, gitea_client=None):
self.gitea = gitea_client or GiteaClient()
# ── 触发操作 ─────────────────────────────────────
def rerun_failed(self, run_id):
"""重跑失败的 jobs
Returns:
dict: {success, message, new_run_id?}
"""
run = self.gitea.get_run(run_id)
if not run:
return {"success": False, "message": f"Run {run_id} 不存在"}
status = run.get("status", "")
if status != "completed":
return {
"success": False,
"message": f"Run {run_id} 当前状态为 {status},仅 completed 状态才能重跑",
}
result = self.gitea.rerun_failed_jobs(run_id)
if result is None:
return {"success": False, "message": "重跑请求失败"}
# Gitea rerun 后返回的 run id 通常不变(复用原 run)
return {
"success": True,
"message": f"已触发重跑失败 jobs: Run #{run_id}",
"run_id": run_id,
"run_url": f"{config.GITEA_URL}/{config.GITEA_REPO}/actions/runs/{run_id}",
}
def rerun_all(self, run_id):
"""重跑整个 workflow run
Returns:
dict: {success, message, run_id, run_url}
"""
run = self.gitea.get_run(run_id)
if not run:
return {"success": False, "message": f"Run {run_id} 不存在"}
status = run.get("status", "")
if status == "running" or status == "pending":
return {
"success": False,
"message": f"Run {run_id} 正在运行中,无需重跑",
}
result = self.gitea.rerun_run(run_id)
if result is None:
return {"success": False, "message": "重跑请求失败"}
return {
"success": True,
"message": f"已触发完整重跑: Run #{run_id}",
"run_id": run_id,
"run_url": f"{config.GITEA_URL}/{config.GITEA_REPO}/actions/runs/{run_id}",
}
def cancel_run(self, run_id):
"""取消运行中的 run
Returns:
dict: {success, message}
"""
run = self.gitea.get_run(run_id)
if not run:
return {"success": False, "message": f"Run {run_id} 不存在"}
status = run.get("status", "")
if status == "completed":
return {
"success": False,
"message": f"Run {run_id} 已完成,无需取消",
}
result = self.gitea.cancel_run(run_id)
if result is None:
return {"success": False, "message": "取消请求失败"}
return {
"success": True,
"message": f"已取消 Run #{run_id}",
"run_id": run_id,
}
def rerun_latest_failed(self, branch="develop", workflow_id=None):
"""重跑指定分支最近一次失败的 run
用于快速恢复场景,不需要先查 run_id
"""
runs, _ = self.gitea.list_runs(branch=branch, workflow_id=workflow_id, status="failure", limit=5)
if not runs:
return {"success": False, "message": f"{branch} 分支没有失败的 run"}
latest = runs[0]
run_id = latest.get("id")
return self.rerun_failed(run_id)
# ── CLI 入口 ──────────────────────────────────────────
def main():
parser = argparse.ArgumentParser(description="CI 触发操作")
parser.add_argument("--run-id", required=True, help="Workflow Run ID")
parser.add_argument(
"--action",
required=True,
choices=["rerun_failed", "rerun_all", "cancel"],
help="操作类型",
)
args = parser.parse_args()
trigger = CITrigger()
if args.action == "rerun_failed":
result = trigger.rerun_failed(args.run_id)
elif args.action == "rerun_all":
result = trigger.rerun_all(args.run_id)
elif args.action == "cancel":
result = trigger.cancel_run(args.run_id)
else:
print(f"未知操作: {args.action}")
return 1
status = "" if result["success"] else ""
print(f"{status} {result['message']}")
if result.get("run_url"):
print(f" {result['run_url']}")
return 0 if result["success"] else 1
if __name__ == "__main__":
sys.exit(main())
+55
View File
@@ -0,0 +1,55 @@
#!/usr/bin/env python3
"""
ChatOps 配置管理 - 统一从环境变量读取配置,不硬编码任何敏感信息
环境变量:
GITEA_URL Gitea 地址 (默认 https://git.xiaoxiajianji.com)
GITEA_REPO 仓库路径 (默认 xiaoxia/xiaoxia-saas)
GITEA_TOKEN Gitea API Token (优先使用)
GITEA_USERNAME Gitea 用户名 (密码认证时)
GITEA_PASSWORD Gitea 密码 (密码认证时)
FEISHU_WEBHOOK_URL 飞书自定义机器人 webhook 地址
FEISHU_APP_ID 飞书应用 App ID (应用机器人模式,预留)
FEISHU_APP_SECRET 飞书应用 App Secret (应用机器人模式,预留)
CHATOPS_NOTIFY_BRANCHES 触发通知的分支,逗号分隔 (默认 main,develop)
CHATOPS_WEBHOOK_PORT webhook 服务监听端口 (默认 8090)
CHATOPS_WEBHOOK_SECRET Gitea webhook 密钥 (校验签名,可选)
"""
import os
# ── Gitea 配置 ────────────────────────────────────────
GITEA_URL = os.environ.get("GITEA_URL", "https://git.xiaoxiajianji.com").rstrip("/")
GITEA_REPO = os.environ.get("GITEA_REPO", "xiaoxia/xiaoxia-saas")
GITEA_TOKEN = os.environ.get("GITEA_TOKEN", "")
GITEA_USERNAME = os.environ.get("GITEA_USERNAME", "")
GITEA_PASSWORD = os.environ.get("GITEA_PASSWORD", "")
# ── 飞书配置 ──────────────────────────────────────────
FEISHU_WEBHOOK_URL = os.environ.get("FEISHU_WEBHOOK_URL", "")
FEISHU_APP_ID = os.environ.get("FEISHU_APP_ID", "")
FEISHU_APP_SECRET = os.environ.get("FEISHU_APP_SECRET", "")
# ── 通知配置 ──────────────────────────────────────────
NOTIFY_BRANCHES = [b.strip() for b in os.environ.get("CHATOPS_NOTIFY_BRANCHES", "main,develop").split(",") if b.strip()]
# ── Webhook 服务配置 ──────────────────────────────────
CHATOPS_WEBHOOK_PORT = int(os.environ.get("CHATOPS_WEBHOOK_PORT", "8090"))
WEBHOOK_SECRET = os.environ.get("CHATOPS_WEBHOOK_SECRET", "")
# ── 常量 ──────────────────────────────────────────────
PAGE_LIMIT = 50 # Gitea API 每页最大数量
def has_gitea_auth() -> bool:
"""检查是否配置了 Gitea 认证信息"""
if GITEA_TOKEN:
return True
if GITEA_USERNAME and GITEA_PASSWORD:
return True
return False
def has_feishu_webhook() -> bool:
"""检查是否配置了飞书 webhook"""
return bool(FEISHU_WEBHOOK_URL)
+390
View File
@@ -0,0 +1,390 @@
#!/usr/bin/env python3
"""
飞书通知模块 - CI 关键事件推送
支持通知类型:
- branch_failure: main/develop 分支 CI 失败
- branch_recovery: main/develop 分支 CI 从失败恢复(绿色恢复)
- e2e_failure: E2E 测试失败摘要
- pr_failure: PR CI 失败(可选)
用法:
# 命令行直接调用(供 CI workflow 使用)
python3 -m scripts.ci.chatops.feishu_notify --mode failure --run-id 12345
python3 scripts/ci/chatops/feishu_notify.py --mode recovery --run-id 12345
# Python 模块调用
from scripts.ci.chatops.feishu_notify import FeishuNotifier
notifier = FeishuNotifier()
notifier.notify_branch_failure(run_id=12345, branch="develop")
设计原则:
1. 通知失败永远不阻断主流程(返回 0)
2. 卡片信息丰富,一键跳转 Gitea 详情页
3. 失败通知包含错误摘要,不用点进去就能判断严重程度
"""
import argparse
import json
import sys
import urllib.request
from datetime import datetime
from . import config
from .gitea_client import GiteaClient
class FeishuNotifier:
"""飞书通知发送器"""
def __init__(self, webhook_url=None, gitea_client=None):
self.webhook_url = webhook_url or config.FEISHU_WEBHOOK_URL
self.gitea = gitea_client or GiteaClient()
def _send_card(self, card_payload):
"""发送飞书卡片消息
Returns:
True 表示发送成功(飞书返回 code=0)
"""
if not self.webhook_url:
print("[INFO] 未配置 FEISHU_WEBHOOK_URL,跳过飞书通知")
return False
payload = {"msg_type": "interactive", "card": card_payload}
data = json.dumps(payload).encode("utf-8")
req = urllib.request.Request(
self.webhook_url,
data=data,
headers={"Content-Type": "application/json"},
method="POST",
)
try:
with urllib.request.urlopen(req, timeout=10) as resp:
resp_body = resp.read().decode("utf-8")
result = json.loads(resp_body)
if result.get("code", 0) != 0:
print(
f"[WARN] 飞书通知返回错误: {result.get('msg', resp_body)}",
file=sys.stderr,
)
return False
return True
except Exception as e:
print(f"[WARN] 飞书通知发送失败: {e}", file=sys.stderr)
return False
# ── 通知模板 ──────────────────────────────────────
def _run_url(self, run_id):
return f"{config.GITEA_URL}/{config.GITEA_REPO}/actions/runs/{run_id}"
def _pr_url(self, pr_number):
return f"{config.GITEA_URL}/{config.GITEA_REPO}/pulls/{pr_number}"
def notify_branch_failure(self, run_id, branch, run_data=None):
"""main/develop 分支 CI 失败通知
包含: 失败 job 列表、错误摘要、一键重跑链接
"""
run = run_data or self.gitea.get_run(run_id)
if not run:
print(f"[WARN] 无法获取 run {run_id} 详情", file=sys.stderr)
return False
workflow_name = run.get("name", "Unknown Workflow")
commit_msg = run.get("head_commit", {}).get("message", "未知").splitlines()[0][:60]
commit_sha = run.get("head_sha", "")[:8]
actor = (
run.get("trigger_event", {}).get("actor", {}).get("login", "unknown")
if isinstance(run.get("trigger_event"), dict)
else run.get("actor", "unknown")
)
run_url = self._run_url(run_id)
# 获取失败 job 摘要
failed_jobs = self.gitea.get_failed_jobs_summary(run_id, max_lines_per_job=15)
# 构建失败摘要
failure_summary = ""
if failed_jobs:
job_lines = []
for job in failed_jobs[:3]: # 最多显示 3 个
step_info = f"{job['failed_step']}" if job["failed_step"] else ""
job_lines.append(f"• **{job['name']}**{step_info}")
if job["log_tail"]:
# 取最后 3 行日志
tail_lines = job["log_tail"].strip().splitlines()[-3:]
for line in tail_lines:
clean_line = line.strip()[:100]
if clean_line:
job_lines.append(f" `{clean_line}`")
failure_summary = "\n".join(job_lines)
else:
failure_summary = "(获取失败详情中,点击查看日志)"
# 字段
fields = [
{"is_short": True, "text": {"tag": "lark_md", "content": f"**分支**\n{branch}"}},
{"is_short": True, "text": {"tag": "lark_md", "content": f"**Workflow**\n{workflow_name}"}},
{"is_short": True, "text": {"tag": "lark_md", "content": f"**提交**\n`{commit_sha}`"}},
{"is_short": True, "text": {"tag": "lark_md", "content": f"**触发者**\n{actor}"}},
{"is_short": False, "text": {"tag": "lark_md", "content": f"**提交信息**\n{commit_msg}"}},
{"is_short": False, "text": {"tag": "lark_md", "content": f"**失败详情**\n{failure_summary}"}},
]
card = {
"header": {
"title": {
"tag": "plain_text",
"content": f"❌ CI告警:{branch} 分支构建失败",
},
"status": "red",
},
"elements": [
{"tag": "div", "fields": fields},
{
"tag": "action",
"actions": [
{
"tag": "button",
"text": {"tag": "plain_text", "content": "查看失败日志"},
"url": run_url,
"type": "danger",
},
{
"tag": "button",
"text": {"tag": "plain_text", "content": "重跑失败Job"},
"url": f"{run_url}/rerun-failed-jobs",
"type": "default",
},
],
},
],
}
result = self._send_card(card)
print(f"[INFO] 分支失败通知已发送: {branch} run={run_id}")
return result
def notify_branch_recovery(self, run_id, branch, previous_failure_run_id=None):
"""分支 CI 恢复通知(从失败变成功)"""
run = self.gitea.get_run(run_id)
if not run:
print(f"[WARN] 无法获取 run {run_id} 详情", file=sys.stderr)
return False
workflow_name = run.get("name", "Unknown Workflow")
commit_sha = run.get("head_sha", "")[:8]
run_url = self._run_url(run_id)
# 计算恢复耗时
duration_text = "已恢复"
if previous_failure_run_id:
prev_run = self.gitea.get_run(previous_failure_run_id)
if prev_run:
# 简单计算两个 run 的时间差
prev_time = prev_run.get("created_at", "")
cur_time = run.get("created_at", "")
if prev_time and cur_time:
try:
t1 = datetime.fromisoformat(prev_time.replace("Z", "+00:00"))
t2 = datetime.fromisoformat(cur_time.replace("Z", "+00:00"))
diff = (t2 - t1).total_seconds() / 60
duration_text = f"故障时长约 {diff:.0f} 分钟"
except Exception:
pass
fields = [
{"is_short": True, "text": {"tag": "lark_md", "content": f"**分支**\n{branch}"}},
{"is_short": True, "text": {"tag": "lark_md", "content": f"**Workflow**\n{workflow_name}"}},
{"is_short": True, "text": {"tag": "lark_md", "content": f"**提交**\n`{commit_sha}`"}},
{"is_short": True, "text": {"tag": "lark_md", "content": "**状态**\n✅ 已恢复"}},
{"is_short": False, "text": {"tag": "lark_md", "content": f"**说明**\n{duration_text}"}},
]
card = {
"header": {
"title": {
"tag": "plain_text",
"content": f"✅ CI通知:{branch} 分支构建已恢复",
},
"status": "green",
},
"elements": [
{"tag": "div", "fields": fields},
{
"tag": "action",
"actions": [
{
"tag": "button",
"text": {"tag": "plain_text", "content": "查看详情"},
"url": run_url,
"type": "primary",
}
],
},
],
}
result = self._send_card(card)
print(f"[INFO] 分支恢复通知已发送: {branch} run={run_id}")
return result
def notify_e2e_failure(self, run_id, branch="develop", pr_number=None):
"""E2E 测试失败摘要通知"""
failed_jobs = self.gitea.get_failed_jobs_summary(run_id, max_lines_per_job=50)
e2e_jobs = [j for j in failed_jobs if "e2e" in j["name"].lower() or "test" in j["name"].lower()]
if not e2e_jobs:
# 没有明确的 e2e job,取所有失败的
e2e_jobs = failed_jobs
run_url = self._run_url(run_id)
title_suffix = f"PR #{pr_number}" if pr_number else f"{branch} 分支"
# 构建失败用例摘要
case_summary = ""
for job in e2e_jobs[:3]:
case_summary += f"**{job['name']}**\n"
if job["log_tail"]:
# 尝试提取 FAIL 行
fail_lines = [
line.strip()
for line in job["log_tail"].splitlines()
if "FAIL" in line or "fail" in line.lower() or "" in line or "" in line
][:5]
if fail_lines:
for line in fail_lines:
case_summary += f"{line[:120]}\n"
else:
tail = job["log_tail"].strip().splitlines()[-5:]
for line in tail:
case_summary += f" `{line.strip()[:100]}`\n"
case_summary += "\n"
if not case_summary:
case_summary = "(点击查看完整测试报告)"
fields = [
{"is_short": True, "text": {"tag": "lark_md", "content": f"**来源**\n{title_suffix}"}},
{"is_short": True, "text": {"tag": "lark_md", "content": f"**失败Job数**\n{len(e2e_jobs)}"}},
{"is_short": False, "text": {"tag": "lark_md", "content": f"**失败摘要**\n{case_summary}"}},
]
card = {
"header": {
"title": {
"tag": "plain_text",
"content": f"🧪 CI告警:E2E 测试失败 - {title_suffix}",
},
"status": "orange",
},
"elements": [
{"tag": "div", "fields": fields},
{
"tag": "action",
"actions": [
{
"tag": "button",
"text": {"tag": "plain_text", "content": "查看完整报告"},
"url": run_url,
"type": "danger",
}
],
},
],
}
result = self._send_card(card)
print(f"[INFO] E2E失败通知已发送: run={run_id}")
return result
def notify_pr_failure(self, run_id, pr_number, pr_title=""):
"""PR CI 失败通知(轻量版,可选开启)"""
run_url = self._run_url(run_id)
pr_url = self._pr_url(pr_number)
failed_jobs = self.gitea.get_failed_jobs_summary(run_id, max_lines_per_job=10)
failure_names = [j["name"] for j in failed_jobs[:3]]
failure_text = "".join(failure_names) if failure_names else "未知"
fields = [
{
"is_short": False,
"text": {"tag": "lark_md", "content": f"**PR**\n[#{pr_number} {pr_title[:50]}]({pr_url})"},
},
{"is_short": False, "text": {"tag": "lark_md", "content": f"**失败任务**\n{failure_text}"}},
]
card = {
"header": {
"title": {
"tag": "plain_text",
"content": f"⚠️ CI通知:PR #{pr_number} 构建失败",
},
"status": "yellow",
},
"elements": [
{"tag": "div", "fields": fields},
{
"tag": "action",
"actions": [
{
"tag": "button",
"text": {"tag": "plain_text", "content": "查看日志"},
"url": run_url,
"type": "default",
},
{
"tag": "button",
"text": {"tag": "plain_text", "content": "查看PR"},
"url": pr_url,
"type": "default",
},
],
},
],
}
result = self._send_card(card)
print(f"[INFO] PR失败通知已发送: PR #{pr_number} run={run_id}")
return result
# ── CLI 入口 ──────────────────────────────────────────
def main():
parser = argparse.ArgumentParser(description="飞书 CI 通知")
parser.add_argument(
"--mode",
required=True,
choices=["failure", "recovery", "e2e_failure", "pr_failure"],
help="通知模式",
)
parser.add_argument("--run-id", required=True, help="Workflow Run ID")
parser.add_argument("--branch", default="develop", help="分支名")
parser.add_argument("--pr-number", type=int, help="PR 编号")
parser.add_argument("--pr-title", default="", help="PR 标题")
parser.add_argument("--prev-run-id", help="上一个失败的 run ID(恢复通知用)")
parser.add_argument("--webhook", help="飞书 webhook URL(覆盖环境变量)")
args = parser.parse_args()
notifier = FeishuNotifier(webhook_url=args.webhook)
if args.mode == "failure":
notifier.notify_branch_failure(args.run_id, args.branch)
elif args.mode == "recovery":
notifier.notify_branch_recovery(args.run_id, args.branch, previous_failure_run_id=args.prev_run_id)
elif args.mode == "e2e_failure":
notifier.notify_e2e_failure(args.run_id, branch=args.branch, pr_number=args.pr_number)
elif args.mode == "pr_failure":
notifier.notify_pr_failure(args.run_id, args.pr_number, pr_title=args.pr_title)
return 0
if __name__ == "__main__":
sys.exit(main())
+243
View File
@@ -0,0 +1,243 @@
#!/usr/bin/env python3
"""
Gitea API 客户端封装 - Actions + PR + Webhook 相关接口
基于 urllib 实现,无第三方依赖,与 ci_dashboard.py 风格一致。
支持 token 和 basic auth 两种认证方式。
"""
import base64
import json
import sys
import urllib.error
import urllib.request
from . import config
class GiteaClient:
"""Gitea API 客户端"""
def __init__(
self,
base_url=None,
repo=None,
token=None,
username=None,
password=None,
):
self.base_url = (base_url or config.GITEA_URL).rstrip("/")
self.repo = repo or config.GITEA_REPO
self.token = token or config.GITEA_TOKEN
self.username = username or config.GITEA_USERNAME
self.password = password or config.GITEA_PASSWORD
self.api_base = f"{self.base_url}/api/v1/repos/{self.repo}"
def _request(self, path, method="GET", data=None):
"""通用 HTTP 请求
Args:
path: API 路径(相对于 /api/v1/repos/{repo}/
method: HTTP 方法
data: 请求体(dict 或 bytes
Returns:
解析后的 JSON 数据,失败返回 None
"""
url = f"{self.api_base}/{path}"
body = None
if data is not None:
if isinstance(data, (dict, list)):
body = json.dumps(data).encode("utf-8")
else:
body = data if isinstance(data, bytes) else str(data).encode()
req = urllib.request.Request(url, data=body, method=method)
req.add_header("Content-Type", "application/json")
if self.token:
req.add_header("Authorization", f"token {self.token}")
elif self.username and self.password:
auth = base64.b64encode(f"{self.username}:{self.password}".encode()).decode()
req.add_header("Authorization", f"Basic {auth}")
try:
with urllib.request.urlopen(req, timeout=30) as resp:
resp_body = resp.read().decode()
if not resp_body:
return {}
return json.loads(resp_body)
except urllib.error.HTTPError as e:
err_body = ""
try:
err_body = e.read().decode()
except Exception:
pass
print(
f"[WARN] HTTP {e.code}: {url} - {err_body[:200]}",
file=sys.stderr,
)
return None
except Exception as e:
print(f"[WARN] 请求失败 {url}: {e}", file=sys.stderr)
return None
# ── Actions: Workflow Runs ────────────────────────
def list_runs(
self,
status=None,
branch=None,
event=None,
workflow_id=None,
page=1,
limit=config.PAGE_LIMIT,
):
"""获取 workflow runs 列表
Returns:
(runs列表, 总数)
"""
params = []
if status:
params.append(f"status={status}")
if branch:
params.append(f"branch={branch}")
if event:
params.append(f"event={event}")
if workflow_id:
params.append(f"workflow_id={workflow_id}")
params.append(f"page={page}")
params.append(f"limit={limit}")
path = f"actions/runs?{'&'.join(params)}"
data = self._request(path)
if not data:
return [], 0
runs = data.get("workflow_runs", [])
total = data.get("total_count", 0)
return runs, total
def get_run(self, run_id):
"""获取单个 run 详情"""
return self._request(f"actions/runs/{run_id}")
def get_run_jobs(self, run_id):
"""获取 run 的 jobs 列表"""
data = self._request(f"actions/runs/{run_id}/jobs")
if not data:
return []
return data.get("jobs", [])
def get_job_log(self, run_id, job_id):
"""获取 job 日志(纯文本)"""
url = f"{self.api_base}/actions/runs/{run_id}/jobs/{job_id}/logs"
req = urllib.request.Request(url)
if self.token:
req.add_header("Authorization", f"token {self.token}")
elif self.username and self.password:
auth = base64.b64encode(f"{self.username}:{self.password}".encode()).decode()
req.add_header("Authorization", f"Basic {auth}")
try:
with urllib.request.urlopen(req, timeout=30) as resp:
return resp.read().decode("utf-8", errors="replace")
except Exception as e:
print(f"[WARN] 获取日志失败 job={job_id}: {e}", file=sys.stderr)
return ""
def rerun_run(self, run_id):
"""重新运行整个 workflow run"""
return self._request(f"actions/runs/{run_id}/rerun", method="POST")
def rerun_failed_jobs(self, run_id):
"""重新运行失败的 jobs"""
return self._request(f"actions/runs/{run_id}/rerun-failed-jobs", method="POST")
def cancel_run(self, run_id):
"""取消 run"""
return self._request(f"actions/runs/{run_id}/cancel", method="POST")
# ── Actions: Workflows ────────────────────────────
def list_workflows(self):
"""获取 workflow 列表"""
data = self._request("actions/workflows")
if not data:
return []
return data.get("workflows", [])
def get_workflow(self, workflow_id):
"""获取单个 workflow 详情"""
return self._request(f"actions/workflows/{workflow_id}")
# ── Pull Requests ─────────────────────────────────
def get_pr(self, pr_number):
"""获取 PR 详情"""
return self._request(f"pulls/{pr_number}")
def get_pr_ci_runs(self, pr_number, limit=20):
"""获取 PR 关联的 CI runs(通过 head_sha 查询)"""
pr = self.get_pr(pr_number)
if not pr:
return []
head_sha = pr.get("head", {}).get("sha", "")
if not head_sha:
return []
# 用 head_sha 过滤 runs
runs, _ = self.list_runs(limit=limit)
return [r for r in runs if r.get("head_sha", "") == head_sha]
# ── 便捷方法 ──────────────────────────────────────
def get_latest_run(self, branch, workflow_id=None, status=None):
"""获取指定分支最新的 run"""
runs, _ = self.list_runs(branch=branch, workflow_id=workflow_id, status=status, limit=5)
return runs[0] if runs else None
def get_failed_jobs_summary(self, run_id, max_lines_per_job=30):
"""获取失败 job 的摘要信息(用于通知)
Returns:
list[dict]: 每个失败 job 的 {name, conclusion, failed_step, log_tail}
"""
jobs = self.get_run_jobs(run_id)
if not jobs:
return []
failed = [j for j in jobs if j.get("status") == "completed" and j.get("conclusion") == "failure"]
if not failed:
# 运行中的也返回,方便定位
failed = [j for j in jobs if j.get("status") != "completed"]
result = []
for job in failed[:5]: # 最多取 5 个失败 job
job_id = job.get("id", "")
name = job.get("name", "Unknown")
conclusion = job.get("conclusion", job.get("status", "unknown"))
# 找失败的 step
failed_step = ""
steps = job.get("steps", [])
for step in steps:
if step.get("conclusion") == "failure":
failed_step = step.get("name", "")
break
# 取日志尾部
log_tail = ""
if job_id:
log = self.get_job_log(run_id, job_id)
if log:
lines = log.strip().splitlines()
log_tail = "\n".join(lines[-max_lines_per_job:])
result.append(
{
"name": name,
"conclusion": conclusion,
"failed_step": failed_step,
"log_tail": log_tail,
"job_id": job_id,
}
)
return result
+446
View File
@@ -0,0 +1,446 @@
#!/usr/bin/env python3
"""
Gitea Webhook 接收服务 - FastAPI 实现
功能:
1. 接收 Gitea Actions webhook 事件,触发飞书通知
2. 接收飞书机器人回调消息,处理 /ci 交互命令
3. 维护简单的状态缓存,检测分支恢复等状态变化
部署:
部署到构建服务器,监听 8090 端口(可配置)
Gitea webhook 指向: http://<server>:8090/webhook/gitea
飞书消息回调指向: http://<server>:8090/webhook/feishu
依赖:
fastapi + uvicorn(可选,未安装时仅模块可用,服务不可启动)
注意:
本文件为第一版骨架,通知逻辑已实现,飞书交互命令待后续完善。
"""
import hashlib
import hmac
import json
import sys
import threading
import time
from typing import Optional
from . import config
from .gitea_client import GiteaClient
# FastAPI 是可选依赖,未安装时仅导出类不启动服务
try:
from fastapi import FastAPI, Header, HTTPException, Request
from fastapi.responses import JSONResponse
FASTAPI_AVAILABLE = True
except ImportError:
FASTAPI_AVAILABLE = False
FastAPI = None # type: ignore
# ── 状态缓存 ──────────────────────────────────────────
class StateCache:
"""简单的内存状态缓存,用于检测状态变化
记录每个分支最后一次 run 的状态,用于判断:
- 是否从失败变成功(恢复通知)
- 是否连续失败(避免重复告警)
"""
def __init__(self, max_entries=100):
self._cache = {} # {branch: {last_status, last_run_id, last_notified_failure}}
self._lock = threading.Lock()
self._max = max_entries
def get(self, key):
with self._lock:
return self._cache.get(key)
def set(self, key, value):
with self._lock:
self._cache[key] = value
# 简单的淘汰策略
if len(self._cache) > self._max:
oldest_key = next(iter(self._cache))
del self._cache[oldest_key]
def check_and_update(self, branch, run_id, conclusion):
"""检查状态变化并更新缓存
Returns:
dict: {is_new_failure, is_recovery, previous_status, previous_run_id}
"""
prev = self.get(branch) or {}
prev_status = prev.get("last_status", "unknown")
prev_run_id = prev.get("last_run_id")
is_new_failure = False
is_recovery = False
if conclusion == "failure" and prev_status != "failure":
is_new_failure = True
if conclusion == "success" and prev_status == "failure":
is_recovery = True
self.set(
branch,
{
"last_status": conclusion,
"last_run_id": run_id,
"last_updated": time.time(),
"last_notified_failure": run_id if is_new_failure else prev.get("last_notified_failure"),
},
)
return {
"is_new_failure": is_new_failure,
"is_recovery": is_recovery,
"previous_status": prev_status,
"previous_run_id": prev_run_id,
}
state_cache = StateCache()
# ── Gitea Webhook 处理 ────────────────────────────────
def verify_gitea_signature(payload: bytes, signature: str) -> bool:
"""校验 Gitea webhook 签名(X-Gitea-Signature
Gitea 使用 HMAC-SHA256 签名,格式: sha256=xxx
"""
if not config.WEBHOOK_SECRET:
return True # 未配置密钥则跳过校验
if not signature:
return False
try:
algo, sig_hex = signature.split("=", 1)
if algo != "sha256":
return False
expected = hmac.new(config.WEBHOOK_SECRET.encode(), payload, hashlib.sha256).hexdigest()
return hmac.compare_digest(expected, sig_hex)
except Exception:
return False
def handle_gitea_webhook(payload: dict, event_type: str) -> dict:
"""处理 Gitea webhook 事件
Args:
payload: webhook 请求体
event_type: X-Gitea-Event 头
Returns:
dict: {handled, notifications_sent, message}
"""
if event_type != "create" and event_type != "push":
# 我们主要关心 push 和 actions 事件
# Gitea Actions 的 webhook 事件类型可能是 "push" 或专门的 actions 事件
pass
# 尝试提取 run 信息
run_info = _extract_run_info(payload)
if not run_info:
return {"handled": False, "notifications_sent": 0, "message": "非 CI 事件,跳过"}
branch = run_info["branch"]
run_id = run_info["run_id"]
status = run_info["status"]
conclusion = run_info.get("conclusion", "")
# 只处理已完成的 run
if status != "completed":
return {"handled": True, "notifications_sent": 0, "message": f"Run {run_id} 仍在运行中 ({status})"}
# 检查是否在通知分支列表中
if branch not in config.NOTIFY_BRANCHES:
return {
"handled": True,
"notifications_sent": 0,
"message": f"分支 {branch} 不在通知列表中",
}
# 检查状态变化
change_info = state_cache.check_and_update(branch, run_id, conclusion)
notifications = 0
# 延迟导入,避免循环依赖
from .feishu_notify import FeishuNotifier
notifier = FeishuNotifier()
if conclusion == "failure" and change_info["is_new_failure"]:
# 新失败 → 发失败通知
notifier.notify_branch_failure(run_id, branch)
notifications += 1
# 检查是否是 E2E 失败
gitea = GiteaClient()
failed_jobs = gitea.get_failed_jobs_summary(run_id)
has_e2e = any("e2e" in j["name"].lower() for j in failed_jobs)
if has_e2e:
notifier.notify_e2e_failure(run_id, branch=branch)
notifications += 1
elif conclusion == "success" and change_info["is_recovery"]:
# 从失败恢复 → 发恢复通知
prev_run_id = change_info.get("previous_run_id")
notifier.notify_branch_recovery(run_id, branch, previous_failure_run_id=prev_run_id)
notifications += 1
return {
"handled": True,
"notifications_sent": notifications,
"message": f"分支 {branch} run {run_id} {conclusion}",
}
def _extract_run_info(payload: dict) -> Optional[dict]:
"""从 webhook payload 中提取 run 信息
Gitea Actions webhook 的 payload 结构可能不同,这里做兼容处理。
如果 payload 不是 run 事件,返回 None。
"""
# 尝试多种可能的结构
if "workflow_run" in payload:
wr = payload["workflow_run"]
return {
"run_id": wr.get("id"),
"branch": wr.get("head_branch", ""),
"status": wr.get("status", ""),
"conclusion": wr.get("conclusion", ""),
"name": wr.get("name", ""),
}
if "action" in payload and "pull_request" in payload:
# PR 事件,暂不处理
return None
if "ref" in payload and "head_commit" in payload:
# push 事件,不是 run 事件
return None
return None
# ── 飞书消息处理 ──────────────────────────────────────
def handle_feishu_message(payload: dict) -> dict:
"""处理飞书机器人回调消息
支持命令:
/ci status [branch] - 查询分支 CI 状态
/ci rerun <run-id> - 重跑失败的 jobs
/ci help - 帮助
注意: 第一版骨架,仅解析命令,实际执行逻辑待完善。
"""
# 飞书消息回调格式
header = payload.get("header", {})
event_type = header.get("event_type", "")
if event_type == "url_verification":
# 飞书 URL 验证
return {"challenge": payload.get("challenge", "")}
if event_type != "im.message.receive_v1":
return {"handled": False, "message": f"非消息事件: {event_type}"}
event = payload.get("event", {})
message = event.get("message", {})
content_str = message.get("content", "{}")
try:
content = json.loads(content_str)
except json.JSONDecodeError:
content = {}
text = content.get("text", "")
if not text:
return {"handled": False, "message": "空消息"}
# 解析命令
text = text.strip()
if not text.startswith("/ci"):
return {"handled": False, "message": "非 CI 命令"}
parts = text.split()
if len(parts) < 2:
return _help_response()
cmd = parts[1].lower()
if cmd == "status":
branch = parts[2] if len(parts) > 2 else "develop"
return _handle_status_command(branch)
elif cmd == "rerun":
if len(parts) < 3:
return {"text": "用法: /ci rerun <run-id> 或 /ci rerun latest [branch]"}
arg = parts[2]
if arg == "latest":
branch = parts[3] if len(parts) > 3 else "develop"
return _handle_rerun_latest(branch)
return _handle_rerun_command(arg)
elif cmd == "help":
return _help_response()
else:
return {"text": f"未知命令: {cmd}\n输入 /ci help 查看帮助"}
def _handle_status_command(branch: str) -> dict:
"""处理 /ci status 命令"""
from .ci_query import CIQuery
query = CIQuery()
result = query.get_branch_status(branch)
reply = CIQuery.format_branch_status(result)
return {"text": reply}
def _handle_rerun_command(run_id: str) -> dict:
"""处理 /ci rerun 命令"""
from .ci_trigger import CITrigger
trigger = CITrigger()
try:
result = trigger.rerun_failed(int(run_id))
except ValueError:
return {"text": f"无效的 run id: {run_id}"}
if result["success"]:
return {"text": f"{result['message']}\n{result.get('run_url', '')}"}
return {"text": f"{result['message']}"}
def _handle_rerun_latest(branch: str) -> dict:
"""处理 /ci rerun latest 命令"""
from .ci_trigger import CITrigger
trigger = CITrigger()
result = trigger.rerun_latest_failed(branch=branch)
if result["success"]:
return {"text": f"{result['message']}\n{result.get('run_url', '')}"}
return {"text": f"{result['message']}"}
def _help_response() -> dict:
"""返回帮助信息"""
help_text = """**CI ChatOps 命令帮助**
`/ci status [branch]` 查询分支 CI 状态(默认 develop)
`/ci rerun <run-id>` 重跑指定 run 的失败 jobs
`/ci rerun latest [branch]` 重跑分支最近一次失败的 run
`/ci help` 显示此帮助
**环境变量配置:**
`GITEA_TOKEN` / `GITEA_USERNAME + GITEA_PASSWORD`
`FEISHU_WEBHOOK_URL`
`CHATOPS_NOTIFY_BRANCHES=main,develop`
"""
return {"text": help_text}
# ── FastAPI 应用 ──────────────────────────────────────
def create_app():
"""创建 FastAPI 应用
如果 FastAPI 未安装,返回 None
"""
if not FASTAPI_AVAILABLE:
print(
"[WARN] FastAPI 未安装,无法启动 webhook 服务。" " 请运行: pip install fastapi uvicorn",
file=sys.stderr,
)
return None
app = FastAPI(title="CI ChatOps Webhook", version="0.1.0")
@app.post("/webhook/gitea")
async def gitea_webhook(
request: Request,
x_gitea_event: str = Header(default=""),
x_gitea_signature: str = Header(default=""),
):
body = await request.body()
# 签名校验
if not verify_gitea_signature(body, x_gitea_signature):
raise HTTPException(status_code=401, detail="Invalid signature")
try:
payload = json.loads(body.decode())
except json.JSONDecodeError as e:
raise HTTPException(status_code=400, detail="Invalid JSON") from e
result = handle_gitea_webhook(payload, x_gitea_event)
return JSONResponse(content=result)
@app.post("/webhook/feishu")
async def feishu_webhook(request: Request):
body = await request.body()
try:
payload = json.loads(body.decode())
except json.JSONDecodeError as e:
raise HTTPException(status_code=400, detail="Invalid JSON") from e
result = handle_feishu_message(payload)
return JSONResponse(content=result)
@app.get("/health")
async def health():
return {"status": "ok", "service": "ci-chatops"}
return app
# ── CLI 入口 ──────────────────────────────────────────
def main():
"""启动 webhook 服务"""
import argparse
parser = argparse.ArgumentParser(description="CI ChatOps Webhook 服务")
parser.add_argument("--port", type=int, default=config.CHATOPS_WEBHOOK_PORT, help="监听端口")
parser.add_argument("--host", default="0.0.0.0", help="监听地址")
args = parser.parse_args()
app = create_app()
if not app:
print("[ERROR] FastAPI 不可用,请先安装: pip install fastapi uvicorn")
return 1
try:
import uvicorn
except ImportError:
print("[ERROR] uvicorn 未安装,请先安装: pip install uvicorn")
return 1
print(f"[INFO] CI ChatOps Webhook 服务启动: http://{args.host}:{args.port}")
print("[INFO] Gitea webhook: POST /webhook/gitea")
print("[INFO] 飞书 webhook: POST /webhook/feishu")
print("[INFO] 健康检查: GET /health")
print(f"[INFO] 通知分支: {', '.join(config.NOTIFY_BRANCHES)}")
uvicorn.run(app, host=args.host, port=args.port)
return 0
if __name__ == "__main__":
sys.exit(main())
+174
View File
@@ -0,0 +1,174 @@
#!/usr/bin/env python3
"""
检查 Alembic migration 编号连续性。
扫描 alembic/versions/ 下所有 migration 文件,提取 revision 和 down_revision
验证整条链是否完整——每个 down_revision(除了 baseline 的 None)都必须对应一个存在的 revision。
支持两种格式:
revision: str = "001" # 旧格式(带类型注解)
revision = "038_error_retry" # 新格式(带描述后缀)
匹配策略:提取 revision 名称的数字前缀(如 "001""038")作为唯一标识进行匹配,
兼容纯数字编号和"数字_描述"两种命名风格。
用法:
python3 scripts/ci/check_migration_chain.py [alembic_versions_dir]
默认目录: alembic/versions/
退出码:
0 - 链完整
1 - 有断链或其他错误
"""
from __future__ import annotations
import re
import sys
from pathlib import Path
# 匹配 revision / down_revision,支持带类型注解和不带类型注解两种格式
# revision: str = "xxx" 或 revision = "xxx"
REV_PATTERN = re.compile(
r'^\s*revision\s*(?::\s*str\s*)?=\s*["\']([^"\']+)["\']',
re.MULTILINE,
)
DOWN_PATTERN = re.compile(
r'^\s*down_revision\s*(?::\s*(?:Union\[str,\s*None\]|str\s*\|\s*None|None|str)\s*)?=\s*(["\']([^"\']+)["\']|None)',
re.MULTILINE,
)
# 提取 revision 名称的数字前缀,如 "001" 或 "038_error_retry" → "038"
NUM_PREFIX_PATTERN = re.compile(r"^(\d+)")
def num_prefix(name: str) -> str:
"""提取 revision 名称的数字前缀。"""
m = NUM_PREFIX_PATTERN.match(name)
return m.group(1) if m else name
def extract_migration_info(filepath: Path) -> tuple[str, str | None]:
"""从 migration 文件中提取 revision 和 down_revision(返回完整名称)。"""
content = filepath.read_text(encoding="utf-8")
rev_match = REV_PATTERN.search(content)
down_match = DOWN_PATTERN.search(content)
if not rev_match:
raise ValueError(f"{filepath.name}: 未找到 revision 定义")
revision = rev_match.group(1)
if not down_match:
raise ValueError(f"{filepath.name}: 未找到 down_revision 定义")
# down_match group(2) 是引号内的值,如果是 None 则 group(2) 为 None
down_revision = down_match.group(2)
return revision, down_revision
def check_chain(versions_dir: Path) -> list[str]:
"""检查 migration 链是否完整,返回错误列表。"""
errors: list[str] = []
if not versions_dir.is_dir():
return [f"目录不存在: {versions_dir}"]
py_files = sorted(versions_dir.glob("*.py"))
if not py_files:
return [f"目录下没有 migration 文件: {versions_dir}"]
# 收集所有 revision(用数字前缀做唯一标识)
revisions_by_num: dict[str, str] = {} # 数字前缀 -> 完整 revision 名
revision_files: dict[str, str] = {} # 数字前缀 -> 文件名
down_revisions: list[tuple[str, str | None]] = [] # (文件名, down_revision 数字前缀或None)
for f in py_files:
if f.name.startswith("__"):
continue
try:
rev, down = extract_migration_info(f)
except ValueError as e:
errors.append(str(e))
continue
rev_num = num_prefix(rev)
if rev_num in revisions_by_num:
errors.append(
f"编号重复: 编号 {rev_num} 同时出现在 "
f"{f.name} (revision={rev}) 和 {revision_files[rev_num]} (revision={revisions_by_num[rev_num]})"
)
else:
revisions_by_num[rev_num] = rev
revision_files[rev_num] = f.name
down_num = num_prefix(down) if down else None
down_revisions.append((f.name, down_num))
if errors:
return errors
# 检查每个 down_revision 是否存在
baselines = 0
for filename, down_num in down_revisions:
if down_num is None:
baselines += 1
continue
if down_num not in revisions_by_num:
errors.append(
f"断链: {filename} 的 down_revision 指向编号 '{down_num}',但没有任何 migration 的 revision 是这个编号"
)
if baselines == 0:
errors.append("没有找到 baseline migrationdown_revision = None 的文件)")
elif baselines > 1:
errors.append(f"发现 {baselines} 个 baseline migration,通常只能有 1 个")
# 额外检查:数字编号是否连续(只对能提取出数字的)
if revisions_by_num and not errors:
nums = sorted(int(n) for n in revisions_by_num if n.isdigit())
if nums:
expected = list(range(nums[0], nums[-1] + 1))
missing = [n for n in expected if n not in nums]
if missing:
missing_str = ", ".join(f"{n:03d}" for n in missing)
errors.append(f"编号不连续: 缺少编号 {missing_str}")
return errors
def main() -> int:
if len(sys.argv) > 1:
versions_dir = Path(sys.argv[1])
else:
versions_dir = Path("alembic/versions")
print(f"检查 migration 编号连续性: {versions_dir}")
print()
errors = check_chain(versions_dir)
py_files = [f for f in versions_dir.glob("*.py") if not f.name.startswith("__")]
if errors:
print(f"❌ Migration 链有问题(共 {len(py_files)} 个文件,{len(errors)} 个错误):")
for e in errors:
print(f" - {e}")
print()
print("请修复后再提交。常见原因:")
print(" 1. 新 migration 的 down_revision 编号写错了")
print(" 2. 多个 PR 同时加 migration,编号冲突")
print(" 3. 合并代码时漏了某个 migration 文件")
return 1
print(f"✅ Migration 链完整,共 {len(py_files)} 个版本")
return 0
if __name__ == "__main__":
sys.exit(main())
+137
View File
@@ -0,0 +1,137 @@
#!/usr/bin/env python3
"""
检查 Alembic migration 文件命名规范。
规则:
1. 文件名必须以数字前缀开头(3位补零),如 001_xxx.py、052_add_table.py
2. 数字前缀必须连续递增(与 check_migration_chain.py 一致,但只看文件名)
3. 数字前缀后必须跟有描述性后缀(不能只有数字)
4. 文件名使用小写+下划线(snake_case
5. revision 变量值必须与文件名数字前缀一致(可选带描述后缀)
用法:
python3 scripts/ci/check_migration_naming.py [alembic_versions_dir]
默认目录: alembic/versions/
退出码:
0 - 全部通过
1 - 有命名违规
"""
from __future__ import annotations
import re
import sys
from pathlib import Path
# 文件名格式: 3位数字_描述.py
FILE_NAME_PATTERN = re.compile(r"^(\d{3})_[a-z][a-z0-9_]*\.py$")
# 纯数字文件名(不允许)
PURE_NUM_PATTERN = re.compile(r"^\d{3}\.py$")
# revision 值的数字前缀
REV_NUM_PATTERN = re.compile(r"^(\d{3})")
# revision 变量行
REV_LINE_PATTERN = re.compile(
r'^\s*revision\s*(?::\s*str\s*)?=\s*["\']([^"\']+)["\']',
re.MULTILINE,
)
def check_naming(versions_dir: Path) -> list[str]:
"""检查 migration 文件命名,返回错误列表。"""
errors: list[str] = []
if not versions_dir.is_dir():
return [f"目录不存在: {versions_dir}"]
py_files = sorted(f for f in versions_dir.iterdir() if f.suffix == ".py")
if not py_files:
return [f"目录下没有 migration 文件: {versions_dir}"]
print(f"检查 migration 文件命名: {versions_dir}")
print(f"{len(py_files)} 个文件")
print()
# 1. 文件名格式检查
print("1. 文件名格式检查...")
file_nums: list[int] = []
for f in py_files:
name = f.name
if PURE_NUM_PATTERN.match(name):
errors.append(f"{name}: 只有数字编号,缺少描述性后缀")
continue
m = FILE_NAME_PATTERN.match(name)
if not m:
errors.append(f"{name}: 命名格式不规范,应为 NNN_description.py " f"3位数字前缀+下划线+小写描述)")
continue
file_nums.append(int(m.group(1)))
if not any("命名格式不规范" in e or "缺少描述性后缀" in e for e in errors):
print(f" ✅ 全部 {len(py_files)} 个文件名格式正确")
else:
for e in errors:
if "命名格式不规范" in e or "缺少描述性后缀" in e:
print(e)
# 2. 编号连续性检查(基于文件名数字前缀)
print()
print("2. 编号连续性检查...")
if file_nums:
expected = set(range(min(file_nums), max(file_nums) + 1))
actual = set(file_nums)
missing = sorted(expected - actual)
if missing:
errors.append(f" ❌ 编号不连续,缺少: {', '.join(f'{n:03d}' for n in missing)}")
print(f" ❌ 编号不连续,缺少 {len(missing)} 个: " f"{', '.join(f'{n:03d}' for n in missing)}")
else:
print(f" ✅ 编号连续({min(file_nums):03d} ~ {max(file_nums):03d}")
# 3. revision 变量与文件名前缀一致性检查
print()
print("3. revision变量与文件名一致性检查...")
rev_mismatch = 0
for f in py_files:
m = FILE_NAME_PATTERN.match(f.name)
if not m:
continue # 格式不对的已经报过了
file_num = m.group(1)
content = f.read_text(encoding="utf-8")
rev_match = REV_LINE_PATTERN.search(content)
if not rev_match:
errors.append(f"{f.name}: 未找到 revision 变量定义")
rev_mismatch += 1
continue
rev_value = rev_match.group(1)
rev_num_match = REV_NUM_PATTERN.match(rev_value)
if not rev_num_match or rev_num_match.group(1) != file_num:
errors.append(f"{f.name}: revision='{rev_value}' 与文件名前缀 {file_num} 不一致")
rev_mismatch += 1
if rev_mismatch == 0:
print(f" ✅ 全部 {len(py_files)} 个文件的 revision 与文件名一致")
return errors
def main() -> int:
versions_dir = Path(sys.argv[1]) if len(sys.argv) > 1 else Path("alembic/versions")
errors = check_naming(versions_dir)
print()
if errors:
print(f"❌ 发现 {len(errors)} 个命名问题")
print()
print("命名规范:")
print(" - 文件名格式: NNN_description.py3位数字前缀 + 下划线 + 小写描述)")
print(" - 编号必须连续,不能跳号")
print(" - revision 变量的数字前缀必须与文件名一致")
return 1
print("✅ 所有 migration 文件命名规范检查通过")
return 0
if __name__ == "__main__":
sys.exit(main())
+973
View File
@@ -0,0 +1,973 @@
#!/usr/bin/env python3
"""
CI 可观测性看板 - 从 Gitea Actions API 拉取数据并生成 Markdown/HTML 日报
用法:
python3 scripts/ci/ci_dashboard.py --days 7
python3 scripts/ci/ci_dashboard.py --days 30 --output ci_report.md
python3 scripts/ci/ci_dashboard.py --workflow ci-cd.yml --days 7
python3 scripts/ci/ci_dashboard.py --days 7 --html --html-output dashboard.html
环境变量:
GITEA_URL Gitea 地址 (默认 https://git.xiaoxiajianji.com)
GITEA_REPO 仓库 (默认 xiaoxia/xiaoxia-saas)
GITEA_TOKEN API Token (优先) 或 GITEA_USERNAME + GITEA_PASSWORD
"""
import argparse
import base64
import json
import math
import os
import statistics
import sys
import urllib.error
import urllib.request
from collections import defaultdict
from datetime import datetime, timedelta, timezone
# ── 配置 ──────────────────────────────────────────────
DEFAULT_GITEA_URL = "https://git.xiaoxiajianji.com"
DEFAULT_REPO = "xiaoxia/xiaoxia-saas"
DEFAULT_DAYS = 7
PAGE_LIMIT = 50 # 每页数量,最大50
# ── API 封装 ─────────────────────────────────────────
class GiteaActions:
def __init__(self, base_url, repo, token=None, username=None, password=None):
self.base_url = base_url.rstrip("/")
self.repo = repo
self.token = token
self.username = username
self.password = password
self.api_base = f"{self.base_url}/api/v1/repos/{self.repo}/actions"
def _request(self, path):
url = f"{self.api_base}/{path}"
req = urllib.request.Request(url)
if self.token:
req.add_header("Authorization", f"token {self.token}")
elif self.username and self.password:
auth = base64.b64encode(f"{self.username}:{self.password}".encode()).decode()
req.add_header("Authorization", f"Basic {auth}")
try:
with urllib.request.urlopen(req, timeout=30) as resp:
return json.loads(resp.read().decode())
except urllib.error.HTTPError as e:
print(f"[WARN] HTTP {e.code}: {url}", file=sys.stderr)
return None
except Exception as e:
print(f"[WARN] 请求失败 {url}: {e}", file=sys.stderr)
return None
def list_runs(self, status=None, branch=None, event=None, page=1, limit=PAGE_LIMIT):
"""获取 workflow runs 列表"""
params = []
if status:
params.append(f"status={status}")
if branch:
params.append(f"branch={branch}")
if event:
params.append(f"event={event}")
params.append(f"page={page}")
params.append(f"limit={limit}")
path = f"runs?{'&'.join(params)}"
data = self._request(path)
if not data:
return [], 0
runs = data.get("workflow_runs", [])
total = data.get("total_count", 0)
return runs, total
def get_run_jobs(self, run_id):
"""获取 run 的所有 job"""
data = self._request(f"runs/{run_id}/jobs")
if not data:
return []
return data.get("jobs", [])
def list_workflows(self):
"""获取所有 workflow"""
data = self._request("workflows")
if not data:
return []
return data.get("workflows", [])
# ── 工具函数 ─────────────────────────────────────────
def parse_datetime(s):
"""解析 ISO 格式时间字符串"""
if not s or s.startswith("1970") or s.startswith("0001"):
return None
try:
if s.endswith("Z"):
s = s[:-1] + "+00:00"
return datetime.fromisoformat(s)
except Exception:
return None
def to_shanghai(dt):
"""转换为上海时区"""
if dt is None:
return None
if dt.tzinfo is None:
dt = dt.replace(tzinfo=timezone.utc)
return dt.astimezone(timezone(timedelta(hours=8)))
def duration_seconds(start_str, end_str):
"""计算耗时(秒)"""
start = parse_datetime(start_str)
end = parse_datetime(end_str)
if not start or not end:
return None
return (end - start).total_seconds()
def fmt_duration(seconds):
"""格式化耗时显示"""
if seconds is None:
return "N/A"
seconds = int(seconds)
if seconds < 60:
return f"{seconds}s"
mins, secs = divmod(seconds, 60)
if mins < 60:
return f"{mins}m{secs:02d}s"
hours, mins = divmod(mins, 60)
return f"{hours}h{mins:02d}m"
def percentile(sorted_values, p):
"""计算百分位数"""
if not sorted_values:
return None
k = (len(sorted_values) - 1) * (p / 100)
f = math.floor(k)
c = math.ceil(k)
if f == c:
return sorted_values[int(k)]
return sorted_values[f] * (c - k) + sorted_values[c] * (k - f)
def classify_failure(job_name, step_name=None):
"""根据失败的 job/step 名称分类失败原因"""
name = f"{job_name} {step_name or ''}".lower()
if any(k in name for k in ["lint", "ruff", "flake8", "eslint", "prettier", "black", "mypy"]):
return "代码质量 / Lint"
if any(k in name for k in ["unit test", "pytest", "vitest", "jest"]):
return "单元测试失败"
if any(k in name for k in ["integration", "e2e"]):
return "集成测试 / E2E"
if any(k in name for k in ["build", "compile", "docker", "image"]):
return "构建失败"
if any(k in name for k in ["deploy", "preview", "release"]):
return "部署失败"
if any(k in name for k in ["setup", "checkout", "cache", "install", "deps"]):
return "环境 / 依赖"
if any(k in name for k in ["migrate", "migration", "schema"]):
return "数据库迁移"
return "其他"
# ── 数据收集 ─────────────────────────────────────────
def fetch_runs_in_range(ga, start_date, end_date, workflow_filter=None):
"""拉取指定日期范围内的所有 completed runs"""
all_runs = []
page = 1
print(f"[INFO] 拉取 {start_date} ~ {end_date} 的 CI runs...", file=sys.stderr)
while True:
runs, total = ga.list_runs(status="completed", page=page, limit=PAGE_LIMIT)
if not runs:
break
if workflow_filter:
runs = [r for r in runs if workflow_filter in r.get("path", "")]
in_range = []
out_range_old = False
for run in runs:
started = to_shanghai(parse_datetime(run.get("started_at")))
if not started:
continue
run_date = started.date()
if start_date <= run_date <= end_date:
in_range.append(run)
elif run_date < start_date:
out_range_old = True
all_runs.extend(in_range)
print(
f"[INFO] 第 {page} 页: {len(runs)} 条, 范围内 {len(in_range)} 条, 累计 {len(all_runs)}", file=sys.stderr
)
if out_range_old or len(runs) < PAGE_LIMIT:
break
page += 1
if page > 100:
print("[WARN] 超过100页,停止拉取", file=sys.stderr)
break
print(f"[INFO] 共获取 {len(all_runs)} 条 run 数据", file=sys.stderr)
return all_runs
def enrich_with_jobs(ga, runs, max_failures=50):
"""为 runs 补充 job 详情(失败原因分析 + runner 统计)
失败 run 按时间倒序取最近 N 个(避免 API 调用过多),
成功 run 采样用于 runner 分布统计。
"""
# 失败 run 取最近 N 个
failure_runs = [r for r in runs if r.get("conclusion") != "success"]
failure_runs = failure_runs[:max_failures] # 已经是时间倒序
print(f"[INFO] 为最近 {len(failure_runs)} 个失败 run 拉取 job 详情...", file=sys.stderr)
for i, run in enumerate(failure_runs):
jobs = ga.get_run_jobs(run["id"])
run["_jobs"] = jobs
if (i + 1) % 10 == 0:
print(f"[INFO] 已处理 {i+1}/{len(failure_runs)}", file=sys.stderr)
# 成功 run 采样用于 runner 分布
success_runs = [r for r in runs if r.get("conclusion") == "success"]
sample_size = min(50, len(success_runs))
if sample_size > 0:
sampled = success_runs[:: max(1, len(success_runs) // sample_size)]
print(f"[INFO] 采样 {len(sampled)} 个成功 run 用于 runner 统计...", file=sys.stderr)
for run in sampled:
if "_jobs" not in run:
jobs = ga.get_run_jobs(run["id"])
run["_jobs"] = jobs
return runs
# ── 统计分析 ─────────────────────────────────────────
def analyze_runs(runs):
"""对 runs 做全面统计分析"""
if not runs:
return {}
# 基础统计
total = len(runs)
success = sum(1 for r in runs if r.get("conclusion") == "success")
failure = sum(1 for r in runs if r.get("conclusion") == "failure")
cancelled = sum(1 for r in runs if r.get("conclusion") == "cancelled")
other = total - success - failure - cancelled
success_rate = (success / total * 100) if total > 0 else 0
# 耗时统计
durations = []
for r in runs:
d = duration_seconds(r.get("started_at"), r.get("completed_at"))
if d and d > 0:
durations.append(d)
durations.sort()
avg_dur = statistics.mean(durations) if durations else None
median_dur = percentile(durations, 50)
p95_dur = percentile(durations, 95)
# 按日期统计
daily_stats = defaultdict(lambda: {"total": 0, "success": 0, "failure": 0, "durations": []})
for r in runs:
started = to_shanghai(parse_datetime(r.get("started_at")))
if not started:
continue
day = started.date().isoformat()
daily_stats[day]["total"] += 1
if r.get("conclusion") == "success":
daily_stats[day]["success"] += 1
elif r.get("conclusion") == "failure":
daily_stats[day]["failure"] += 1
d = duration_seconds(r.get("started_at"), r.get("completed_at"))
if d and d > 0:
daily_stats[day]["durations"].append(d)
# 按 workflow 统计
wf_stats = defaultdict(lambda: {"total": 0, "success": 0, "failure": 0, "durations": []})
for r in runs:
path = r.get("path", "")
wf_name = path.split("@")[0] if "@" in path else path
wf_stats[wf_name]["total"] += 1
if r.get("conclusion") == "success":
wf_stats[wf_name]["success"] += 1
elif r.get("conclusion") == "failure":
wf_stats[wf_name]["failure"] += 1
d = duration_seconds(r.get("started_at"), r.get("completed_at"))
if d and d > 0:
wf_stats[wf_name]["durations"].append(d)
# 按触发事件统计
event_stats = defaultdict(lambda: {"total": 0, "success": 0, "failure": 0})
for r in runs:
evt = r.get("event", "unknown")
event_stats[evt]["total"] += 1
if r.get("conclusion") == "success":
event_stats[evt]["success"] += 1
elif r.get("conclusion") == "failure":
event_stats[evt]["failure"] += 1
# 失败原因 + runner + job 耗时(需要 _jobs 数据)
failure_categories = defaultdict(int)
failed_jobs_by_name = defaultdict(int)
job_success_stats = defaultdict(lambda: {"total": 0, "success": 0, "failure": 0})
runner_stats = defaultdict(lambda: {"jobs": 0, "success": 0, "failure": 0, "durations": []})
job_time_stats = defaultdict(list)
infra_failures = 0
business_failures = 0
other_failures_count = 0
# 基础设施关键词(与 ci_health_check.py 保持一致的分类逻辑)
infra_job_keywords = ["checkout", "build", "deploy", "cleanup", "setup", "cache", "install", "docker"]
business_job_keywords = [
"unit test",
"pytest",
"vitest",
"jest",
"lint",
"eslint",
"prettier",
"integration",
"e2e",
"validate",
"code quality",
"mypy",
"ruff",
"flake8",
]
for r in runs:
jobs = r.get("_jobs", [])
if not jobs:
continue
for job in jobs:
runner = job.get("runner_name", "unknown")
conclusion = job.get("conclusion", "unknown")
job_name = job.get("name", "unknown")
job_name_lower = job_name.lower()
runner_stats[runner]["jobs"] += 1
job_success_stats[job_name]["total"] += 1
if conclusion == "success":
runner_stats[runner]["success"] += 1
job_success_stats[job_name]["success"] += 1
elif conclusion == "failure":
runner_stats[runner]["failure"] += 1
job_success_stats[job_name]["failure"] += 1
jd = duration_seconds(job.get("started_at"), job.get("completed_at"))
if jd and jd > 0:
runner_stats[runner]["durations"].append(jd)
job_time_stats[job_name].append(jd)
if conclusion == "failure":
failed_jobs_by_name[job_name] += 1
failed_step = None
for step in job.get("steps", []):
if step.get("conclusion") == "failure":
failed_step = step.get("name")
break
category = classify_failure(job_name, failed_step)
failure_categories[category] += 1
# 基础设施 vs 业务代码分类
is_infra = any(k in job_name_lower for k in infra_job_keywords) and not any(
k in job_name_lower for k in business_job_keywords
)
is_business = any(k in job_name_lower for k in business_job_keywords)
if is_infra:
infra_failures += 1
elif is_business:
business_failures += 1
else:
other_failures_count += 1
return {
"total": total,
"success": success,
"failure": failure,
"cancelled": cancelled,
"other": other,
"success_rate": success_rate,
"avg_duration": avg_dur,
"median_duration": median_dur,
"p95_duration": p95_dur,
"durations": durations,
"daily_stats": dict(sorted(daily_stats.items())),
"workflow_stats": dict(wf_stats),
"event_stats": dict(event_stats),
"failure_categories": dict(failure_categories),
"failed_jobs_top": dict(sorted(failed_jobs_by_name.items(), key=lambda x: -x[1])[:15]),
"runner_stats": dict(runner_stats),
"job_time_stats": dict(job_time_stats),
"job_success_stats": dict(job_success_stats),
"infra_failures": infra_failures,
"business_failures": business_failures,
"other_failures_combined": other_failures_count,
}
# ── Markdown 报表生成 ────────────────────────────────
def generate_markdown(stats, start_date, end_date, repo):
"""生成 Markdown 格式的日报"""
lines = []
lines.append("# CI 运行状态看板")
lines.append("")
lines.append(f"> 统计周期: **{start_date} ~ {end_date}**")
lines.append(f"> 仓库: `{repo}`")
lines.append(f"> 生成时间: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}")
lines.append("")
# 概览
lines.append("## 📊 整体概览")
lines.append("")
lines.append("| 指标 | 数值 |")
lines.append("|------|------|")
lines.append(f"| 总构建次数 | **{stats['total']}** |")
lines.append(f"| ✅ 成功 | {stats['success']} |")
lines.append(f"| ❌ 失败 | {stats['failure']} |")
lines.append(f"| ⏹️ 取消 | {stats['cancelled']} |")
lines.append(f"| 📈 成功率 | **{stats['success_rate']:.1f}%** |")
lines.append(f"| ⏱️ 平均耗时 | {fmt_duration(stats['avg_duration'])} |")
lines.append(f"| ⏱️ P50 耗时 | {fmt_duration(stats['median_duration'])} |")
lines.append(f"| ⏱️ P95 耗时 | {fmt_duration(stats['p95_duration'])} |")
lines.append("")
# 每日趋势
lines.append("## 📈 每日趋势")
lines.append("")
lines.append("| 日期 | 总次数 | 成功 | 失败 | 成功率 | 平均耗时 | P95 耗时 |")
lines.append("|------|--------|------|------|--------|----------|----------|")
for day, s in stats["daily_stats"].items():
rate = (s["success"] / s["total"] * 100) if s["total"] > 0 else 0
durations = sorted(s["durations"])
avg = statistics.mean(durations) if durations else None
p95 = percentile(durations, 95) if durations else None
lines.append(
f"| {day} | {s['total']} | {s['success']} | {s['failure']} | {rate:.1f}% | {fmt_duration(avg)} | {fmt_duration(p95)} |"
)
lines.append("")
# 成功率趋势图
lines.append("### 成功率趋势图")
lines.append("")
lines.append("```")
max_bar = 40
days = list(stats["daily_stats"].keys())
if len(days) > 14:
days = days[-14:]
for day in days:
s = stats["daily_stats"][day]
rate = (s["success"] / s["total"] * 100) if s["total"] > 0 else 0
bar_len = int(rate / 100 * max_bar)
bar = "" * bar_len + "" * (max_bar - bar_len)
lines.append(f"{day} {bar} {rate:5.1f}% ({s['total']}次)")
lines.append("```")
lines.append("")
# 按 Workflow 统计
lines.append("## 🧩 各 Workflow 统计")
lines.append("")
wf_sorted = sorted(stats["workflow_stats"].items(), key=lambda x: -x[1]["total"])
lines.append("| Workflow | 次数 | 成功 | 失败 | 成功率 | 平均耗时 | P95 耗时 |")
lines.append("|----------|------|------|------|--------|----------|----------|")
for wf, s in wf_sorted:
rate = (s["success"] / s["total"] * 100) if s["total"] > 0 else 0
durations = sorted(s["durations"])
avg = statistics.mean(durations) if durations else None
p95 = percentile(durations, 95) if durations else None
wf_short = wf.split("/")[-1] if "/" in wf else wf
lines.append(
f"| `{wf_short}` | {s['total']} | {s['success']} | {s['failure']} | {rate:.1f}% | {fmt_duration(avg)} | {fmt_duration(p95)} |"
)
lines.append("")
# 失败原因分析
if stats["failure_categories"]:
lines.append("## ❌ 失败原因分析")
lines.append("")
lines.append("> ⚠️ 基于最近 N 个失败 run 采样分析,用于趋势参考")
lines.append("")
lines.append("### 按分类统计")
lines.append("")
total_failures = sum(stats["failure_categories"].values())
fc_sorted = sorted(stats["failure_categories"].items(), key=lambda x: -x[1])
lines.append("| 分类 | 次数 | 占比 |")
lines.append("|------|------|------|")
for cat, cnt in fc_sorted:
pct = (cnt / total_failures * 100) if total_failures > 0 else 0
lines.append(f"| {cat} | {cnt} | {pct:.1f}% |")
lines.append("")
lines.append("### Top 失败 Job")
lines.append("")
lines.append("| Job 名称 | 失败次数 |")
lines.append("|----------|----------|")
for job, cnt in stats["failed_jobs_top"].items():
lines.append(f"| `{job}` | {cnt} |")
lines.append("")
# Runner 利用率
if stats["runner_stats"]:
lines.append("## 🏃 Runner 利用率")
lines.append("")
runner_sorted = sorted(stats["runner_stats"].items(), key=lambda x: -x[1]["jobs"])
lines.append("| Runner | Job 数 | 成功 | 失败 | 成功率 | 平均耗时 |")
lines.append("|--------|--------|------|------|--------|----------|")
for runner, s in runner_sorted:
rate = (s["success"] / s["jobs"] * 100) if s["jobs"] > 0 else 0
avg = statistics.mean(s["durations"]) if s["durations"] else None
lines.append(
f"| `{runner}` | {s['jobs']} | {s['success']} | {s['failure']} | {rate:.1f}% | {fmt_duration(avg)} |"
)
lines.append("")
# Job 耗时排行
if stats["job_time_stats"]:
lines.append("## ⏱️ Job 耗时排行 (Top 20 by P95)")
lines.append("")
job_stats = []
for name, durs in stats["job_time_stats"].items():
if not durs:
continue
durs_sorted = sorted(durs)
job_stats.append(
{
"name": name,
"count": len(durs_sorted),
"avg": statistics.mean(durs_sorted),
"p50": percentile(durs_sorted, 50),
"p95": percentile(durs_sorted, 95),
}
)
job_stats.sort(key=lambda x: -x["p95"])
top_n = min(20, len(job_stats))
lines.append("| Job 名称 | 次数 | 平均 | P50 | P95 |")
lines.append("|----------|------|------|-----|-----|")
for j in job_stats[:top_n]:
lines.append(
f"| `{j['name']}` | {j['count']} | {fmt_duration(j['avg'])} | {fmt_duration(j['p50'])} | {fmt_duration(j['p95'])} |"
)
lines.append("")
# 触发事件分布
lines.append("## 📋 触发事件分布")
lines.append("")
evt_sorted = sorted(stats["event_stats"].items(), key=lambda x: -x[1]["total"])
lines.append("| 事件类型 | 次数 | 成功 | 失败 | 成功率 |")
lines.append("|----------|------|------|------|--------|")
for evt, s in evt_sorted:
rate = (s["success"] / s["total"] * 100) if s["total"] > 0 else 0
lines.append(f"| `{evt}` | {s['total']} | {s['success']} | {s['failure']} | {rate:.1f}% |")
lines.append("")
return "\n".join(lines)
# ── HTML 看板生成 ────────────────────────────────────
def generate_html(stats, start_date, end_date, repo):
"""生成 HTML 格式的可视化看板(内嵌 ECharts)"""
# 准备图表数据
# 1. 每日成功率趋势
daily_dates = list(stats["daily_stats"].keys())
daily_success_rates = []
daily_run_counts = []
for day in daily_dates:
s = stats["daily_stats"][day]
rate = (s["success"] / s["total"] * 100) if s["total"] > 0 else 0
daily_success_rates.append(round(rate, 1))
daily_run_counts.append(s["total"])
# 2. 各 Workflow 耗时对比
wf_sorted = sorted(stats["workflow_stats"].items(), key=lambda x: -x[1]["total"])
wf_names = []
wf_avg_durations = []
for wf, s in wf_sorted:
wf_short = wf.split("/")[-1] if "/" in wf else wf
wf_names.append(wf_short)
avg = statistics.mean(s["durations"]) if s["durations"] else 0
wf_avg_durations.append(round(avg / 60, 1)) # 转为分钟
# 3. 失败原因分布(饼图数据 - 基础设施 vs 业务 vs 其他)
total_infra_biz = stats["infra_failures"] + stats["business_failures"] + stats["other_failures_combined"]
infra_rate = (stats["infra_failures"] / total_infra_biz * 100) if total_infra_biz > 0 else 0
failure_pie_data = [
{"value": stats["infra_failures"], "name": "基础设施问题"},
{"value": stats["business_failures"], "name": "业务代码问题"},
{"value": stats["other_failures_combined"], "name": "其他"},
]
# 4. 各 Job 成功率排行(横向柱状图,取成功率最低的 Top 15)
job_stats_list = []
for name, s in stats["job_success_stats"].items():
if s["total"] >= 3: # 至少有3次才统计
rate = (s["success"] / s["total"] * 100) if s["total"] > 0 else 0
job_stats_list.append(
{
"name": name,
"rate": round(rate, 1),
"total": s["total"],
"success": s["success"],
}
)
job_stats_list.sort(key=lambda x: x["rate"])
job_stats_list = job_stats_list[:15] # 取成功率最低的15个
job_names = [j["name"] for j in job_stats_list]
job_rates = [j["rate"] for j in job_stats_list]
# 核心指标
total_runs = stats["total"]
success_rate = round(stats["success_rate"], 1)
avg_dur_min = round(stats["avg_duration"] / 60, 1) if stats["avg_duration"] else 0
infra_fail_rate = round(infra_rate, 1)
now_str = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
# 序列化数据为 JSON(供 JS 使用)
data_json = json.dumps(
{
"daily_dates": daily_dates,
"daily_success_rates": daily_success_rates,
"daily_run_counts": daily_run_counts,
"wf_names": wf_names,
"wf_avg_durations": wf_avg_durations,
"failure_pie_data": failure_pie_data,
"job_names": job_names,
"job_rates": job_rates,
},
ensure_ascii=False,
)
# HTML 模板(注意:不使用 f-string,避免与 CSS/JS 的大括号冲突)
html_parts = []
html_parts.append("<!DOCTYPE html>")
html_parts.append('<html lang="zh-CN">')
html_parts.append("<head>")
html_parts.append(' <meta charset="UTF-8">')
html_parts.append(' <meta name="viewport" content="width=device-width, initial-scale=1.0">')
html_parts.append(f" <title>CI 健康度看板 - {repo}</title>")
html_parts.append(' <script src="https://cdn.jsdelivr.net/npm/echarts/dist/echarts.min.js"></script>')
html_parts.append(" <style>")
html_parts.append(" * { margin: 0; padding: 0; box-sizing: border-box; }")
html_parts.append(" body {")
html_parts.append(
' font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", "PingFang SC", "Hiragino Sans GB",'
)
html_parts.append(' "Microsoft YaHei", sans-serif;')
html_parts.append(" background: #f0f2f5;")
html_parts.append(" color: #333;")
html_parts.append(" padding: 20px;")
html_parts.append(" }")
html_parts.append(" .container { max-width: 1400px; margin: 0 auto; }")
html_parts.append(" .header {")
html_parts.append(" background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);")
html_parts.append(" color: white;")
html_parts.append(" padding: 24px 32px;")
html_parts.append(" border-radius: 12px;")
html_parts.append(" margin-bottom: 20px;")
html_parts.append(" }")
html_parts.append(" .header h1 { font-size: 24px; margin-bottom: 8px; }")
html_parts.append(" .header .subtitle { font-size: 14px; opacity: 0.9; }")
html_parts.append(" .header .meta { font-size: 12px; opacity: 0.8; margin-top: 8px; }")
html_parts.append(" .metrics-row {")
html_parts.append(" display: grid;")
html_parts.append(" grid-template-columns: repeat(4, 1fr);")
html_parts.append(" gap: 16px;")
html_parts.append(" margin-bottom: 20px;")
html_parts.append(" }")
html_parts.append(" .metric-card {")
html_parts.append(" background: white;")
html_parts.append(" border-radius: 12px;")
html_parts.append(" padding: 20px;")
html_parts.append(" box-shadow: 0 2px 8px rgba(0,0,0,0.06);")
html_parts.append(" transition: transform 0.2s;")
html_parts.append(" }")
html_parts.append(
" .metric-card:hover { transform: translateY(-2px); box-shadow: 0 4px 12px rgba(0,0,0,0.1); }"
)
html_parts.append(" .metric-card .label { font-size: 13px; color: #8c8c8c; margin-bottom: 8px; }")
html_parts.append(" .metric-card .value { font-size: 28px; font-weight: 600; }")
html_parts.append(" .metric-card .unit { font-size: 14px; color: #8c8c8c; margin-left: 4px; }")
html_parts.append(" .metric-card.success .value { color: #52c41a; }")
html_parts.append(" .metric-card.warning .value { color: #faad14; }")
html_parts.append(" .metric-card.danger .value { color: #ff4d4f; }")
html_parts.append(" .metric-card.info .value { color: #1890ff; }")
html_parts.append(" .charts-grid {")
html_parts.append(" display: grid;")
html_parts.append(" grid-template-columns: 1fr 1fr;")
html_parts.append(" gap: 16px;")
html_parts.append(" margin-bottom: 20px;")
html_parts.append(" }")
html_parts.append(" .chart-card {")
html_parts.append(" background: white;")
html_parts.append(" border-radius: 12px;")
html_parts.append(" padding: 20px;")
html_parts.append(" box-shadow: 0 2px 8px rgba(0,0,0,0.06);")
html_parts.append(" }")
html_parts.append(" .chart-card.full-width { grid-column: 1 / -1; }")
html_parts.append(" .chart-card h3 {")
html_parts.append(" font-size: 16px;")
html_parts.append(" margin-bottom: 12px;")
html_parts.append(" color: #262626;")
html_parts.append(" font-weight: 600;")
html_parts.append(" }")
html_parts.append(" .chart-container { width: 100%; height: 320px; }")
html_parts.append(" .chart-container.tall { height: 400px; }")
html_parts.append(" .footer {")
html_parts.append(" text-align: center;")
html_parts.append(" color: #8c8c8c;")
html_parts.append(" font-size: 12px;")
html_parts.append(" padding: 16px 0;")
html_parts.append(" }")
html_parts.append(" @media (max-width: 900px) {")
html_parts.append(" .metrics-row { grid-template-columns: repeat(2, 1fr); }")
html_parts.append(" .charts-grid { grid-template-columns: 1fr; }")
html_parts.append(" }")
html_parts.append(" @media (max-width: 600px) {")
html_parts.append(" .metrics-row { grid-template-columns: 1fr; }")
html_parts.append(" body { padding: 12px; }")
html_parts.append(" }")
html_parts.append(" </style>")
html_parts.append("</head>")
html_parts.append("<body>")
html_parts.append(' <div class="container">')
html_parts.append(' <div class="header">')
html_parts.append(" <h1>📊 CI 健康度看板</h1>")
html_parts.append(f' <div class="subtitle">仓库: {repo}</div>')
html_parts.append(f' <div class="meta">统计周期: {start_date} ~ {end_date} | 生成时间: {now_str}</div>')
html_parts.append(" </div>")
html_parts.append(' <div class="metrics-row">')
html_parts.append(' <div class="metric-card success">')
html_parts.append(' <div class="label">总成功率</div>')
html_parts.append(f' <div class="value">{success_rate}<span class="unit">%</span></div>')
html_parts.append(" </div>")
html_parts.append(' <div class="metric-card info">')
html_parts.append(' <div class="label">总 Run 数</div>')
html_parts.append(f' <div class="value">{total_runs}<span class="unit">次</span></div>')
html_parts.append(" </div>")
html_parts.append(' <div class="metric-card warning">')
html_parts.append(' <div class="label">平均耗时</div>')
html_parts.append(f' <div class="value">{avg_dur_min}<span class="unit">分钟</span></div>')
html_parts.append(" </div>")
html_parts.append(' <div class="metric-card danger">')
html_parts.append(' <div class="label">基础设施故障率</div>')
html_parts.append(f' <div class="value">{infra_fail_rate}<span class="unit">%</span></div>')
html_parts.append(" </div>")
html_parts.append(" </div>")
html_parts.append(' <div class="charts-grid">')
html_parts.append(' <div class="chart-card full-width">')
html_parts.append(" <h3>📈 CI 成功率趋势</h3>")
html_parts.append(' <div id="chart-success-rate" class="chart-container"></div>')
html_parts.append(" </div>")
html_parts.append(" </div>")
html_parts.append(' <div class="charts-grid">')
html_parts.append(' <div class="chart-card">')
html_parts.append(" <h3>⏱️ 各 Workflow 平均耗时</h3>")
html_parts.append(' <div id="chart-wf-duration" class="chart-container"></div>')
html_parts.append(" </div>")
html_parts.append(' <div class="chart-card">')
html_parts.append(" <h3>❌ 失败原因分布</h3>")
html_parts.append(' <div id="chart-failure-pie" class="chart-container"></div>')
html_parts.append(" </div>")
html_parts.append(" </div>")
html_parts.append(' <div class="charts-grid">')
html_parts.append(' <div class="chart-card full-width">')
html_parts.append(" <h3>📋 各 Job 成功率排行(最低 15 名)</h3>")
html_parts.append(' <div id="chart-job-success" class="chart-container tall"></div>')
html_parts.append(" </div>")
html_parts.append(" </div>")
html_parts.append(' <div class="charts-grid">')
html_parts.append(' <div class="chart-card full-width">')
html_parts.append(" <h3>📊 每日 Run 数量趋势</h3>")
html_parts.append(' <div id="chart-run-count" class="chart-container"></div>')
html_parts.append(" </div>")
html_parts.append(" </div>")
html_parts.append(' <div class="footer">')
html_parts.append(" 由 ci_dashboard.py 自动生成 | ECharts 可视化")
html_parts.append(" </div>")
html_parts.append(" </div>")
html_parts.append(" <script>")
html_parts.append(f" const DATA = {data_json};")
html_parts.append("")
# 图表 1: 成功率趋势
html_parts.append(" (function() {")
html_parts.append(' const chart = echarts.init(document.getElementById("chart-success-rate"));')
html_parts.append(" chart.setOption({")
html_parts.append(' tooltip: { trigger: "axis", formatter: function(params) {')
html_parts.append(' const p = params[0]; return p.name + "<br/>成功率: <b>" + p.value + "%</b>";')
html_parts.append(" }},")
html_parts.append(' grid: { left: "3%", right: "4%", bottom: "3%", containLabel: true },')
html_parts.append(' xAxis: { type: "category", boundaryGap: false, data: DATA.daily_dates,')
html_parts.append(" axisLabel: { rotate: 30, fontSize: 11 } },")
html_parts.append(' yAxis: { type: "value", min: 0, max: 100, axisLabel: { formatter: "{value}%" } },')
html_parts.append(
' series: [{ name: "成功率", type: "line", smooth: true, data: DATA.daily_success_rates,'
)
html_parts.append(' itemStyle: { color: "#52c41a" },')
html_parts.append(" areaStyle: { color: new echarts.graphic.LinearGradient(0, 0, 0, 1, [")
html_parts.append(' { offset: 0, color: "rgba(82, 196, 26, 0.3)" },')
html_parts.append(' { offset: 1, color: "rgba(82, 196, 26, 0.05)" }')
html_parts.append(" ])},")
html_parts.append(' markLine: { silent: true, data: [{ type: "average", name: "平均值",')
html_parts.append(' label: { formatter: "均值 {c}%" } }] }')
html_parts.append(" }]")
html_parts.append(" });")
html_parts.append(' window.addEventListener("resize", () => chart.resize());')
html_parts.append(" })();")
html_parts.append("")
# 图表 2: Workflow 耗时对比
html_parts.append(" (function() {")
html_parts.append(' const chart = echarts.init(document.getElementById("chart-wf-duration"));')
html_parts.append(" chart.setOption({")
html_parts.append(' tooltip: { trigger: "axis", formatter: function(params) {')
html_parts.append(
' const p = params[0]; return p.name + "<br/>平均耗时: <b>" + p.value + " 分钟</b>";'
)
html_parts.append(" }},")
html_parts.append(' grid: { left: "3%", right: "4%", bottom: "15%", containLabel: true },')
html_parts.append(' xAxis: { type: "category", data: DATA.wf_names,')
html_parts.append(" axisLabel: { rotate: 30, fontSize: 10, interval: 0 } },")
html_parts.append(' yAxis: { type: "value", name: "分钟", axisLabel: { formatter: "{value} min" } },')
html_parts.append(' series: [{ name: "平均耗时", type: "bar", data: DATA.wf_avg_durations,')
html_parts.append(" itemStyle: { color: new echarts.graphic.LinearGradient(0, 0, 0, 1, [")
html_parts.append(' { offset: 0, color: "#1890ff" },')
html_parts.append(' { offset: 1, color: "#096dd9" }')
html_parts.append(" ]), borderRadius: [4, 4, 0, 0] },")
html_parts.append(" barMaxWidth: 40")
html_parts.append(" }]")
html_parts.append(" });")
html_parts.append(' window.addEventListener("resize", () => chart.resize());')
html_parts.append(" })();")
html_parts.append("")
# 图表 3: 失败原因饼图
html_parts.append(" (function() {")
html_parts.append(' const chart = echarts.init(document.getElementById("chart-failure-pie"));')
html_parts.append(" chart.setOption({")
html_parts.append(' tooltip: { trigger: "item", formatter: "{b}: {c} 次 ({d}%)" },')
html_parts.append(' legend: { orient: "vertical", right: "5%", top: "center" },')
html_parts.append(
' series: [{ name: "失败原因", type: "pie", radius: ["40%", "70%"], center: ["35%", "50%"],'
)
html_parts.append(" avoidLabelOverlap: false,")
html_parts.append(' itemStyle: { borderRadius: 6, borderColor: "#fff", borderWidth: 2 },')
html_parts.append(' label: { show: false, position: "center" },')
html_parts.append(' emphasis: { label: { show: true, fontSize: 16, fontWeight: "bold" } },')
html_parts.append(" labelLine: { show: false },")
html_parts.append(" data: DATA.failure_pie_data,")
html_parts.append(' color: ["#ff4d4f", "#faad14", "#8c8c8c"]')
html_parts.append(" }]")
html_parts.append(" });")
html_parts.append(' window.addEventListener("resize", () => chart.resize());')
html_parts.append(" })();")
html_parts.append("")
# 图表 4: Job 成功率排行(横向柱状图)
html_parts.append(" (function() {")
html_parts.append(' const chart = echarts.init(document.getElementById("chart-job-success"));')
html_parts.append(" const barData = DATA.job_rates.map(function(rate, i) {")
html_parts.append(" return { value: rate, itemStyle: {")
html_parts.append(' color: rate >= 90 ? "#52c41a" : (rate >= 70 ? "#faad14" : "#ff4d4f")')
html_parts.append(" }};")
html_parts.append(" });")
html_parts.append(" chart.setOption({")
html_parts.append(' tooltip: { trigger: "axis", formatter: function(params) {')
html_parts.append(' const p = params[0]; return p.name + "<br/>成功率: <b>" + p.value + "%</b>";')
html_parts.append(" }},")
html_parts.append(' grid: { left: "3%", right: "8%", bottom: "3%", top: "3%", containLabel: true },')
html_parts.append(' xAxis: { type: "value", min: 0, max: 100, axisLabel: { formatter: "{value}%" } },')
html_parts.append(' yAxis: { type: "category", data: DATA.job_names, axisLabel: { fontSize: 11 } },')
html_parts.append(' series: [{ name: "成功率", type: "bar", data: barData, barWidth: "60%",')
html_parts.append(' label: { show: true, position: "right", formatter: "{c}%", fontSize: 11 }')
html_parts.append(" }]")
html_parts.append(" });")
html_parts.append(' window.addEventListener("resize", () => chart.resize());')
html_parts.append(" })();")
html_parts.append("")
# 图表 5: 每日 Run 数量趋势(面积图)
html_parts.append(" (function() {")
html_parts.append(' const chart = echarts.init(document.getElementById("chart-run-count"));')
html_parts.append(" chart.setOption({")
html_parts.append(' tooltip: { trigger: "axis", formatter: function(params) {')
html_parts.append(
' const p = params[0]; return p.name + "<br/>Run 数量: <b>" + p.value + " 次</b>";'
)
html_parts.append(" }},")
html_parts.append(' grid: { left: "3%", right: "4%", bottom: "3%", containLabel: true },')
html_parts.append(' xAxis: { type: "category", boundaryGap: false, data: DATA.daily_dates,')
html_parts.append(" axisLabel: { rotate: 30, fontSize: 11 } },")
html_parts.append(' yAxis: { type: "value", name: "次数" },')
html_parts.append(
' series: [{ name: "Run 数量", type: "line", smooth: true, data: DATA.daily_run_counts,'
)
html_parts.append(' itemStyle: { color: "#722ed1" },')
html_parts.append(" areaStyle: { color: new echarts.graphic.LinearGradient(0, 0, 0, 1, [")
html_parts.append(' { offset: 0, color: "rgba(114, 46, 209, 0.3)" },')
html_parts.append(' { offset: 1, color: "rgba(114, 46, 209, 0.05)" }')
html_parts.append(" ])},")
html_parts.append(' markLine: { silent: true, data: [{ type: "average", name: "平均值",')
html_parts.append(' label: { formatter: "均值 {c}" } }] }')
html_parts.append(" }]")
html_parts.append(" });")
html_parts.append(' window.addEventListener("resize", () => chart.resize());')
html_parts.append(" })();")
html_parts.append(" </script>")
html_parts.append("</body>")
html_parts.append("</html>")
return "\n".join(html_parts)
# ── 主函数 ───────────────────────────────────────────
def main():
parser = argparse.ArgumentParser(description="CI 可观测性看板 - 生成 Gitea Actions 运行状态报表")
parser.add_argument("--days", type=int, default=DEFAULT_DAYS, help=f"统计最近 N 天 (默认 {DEFAULT_DAYS})")
parser.add_argument("--output", "-o", type=str, help="输出文件路径 (默认输出到 stdout)")
parser.add_argument("--workflow", type=str, help="只统计指定 workflow (如 ci-cd.yml)")
parser.add_argument("--gitea-url", type=str, default=os.environ.get("GITEA_URL", DEFAULT_GITEA_URL))
parser.add_argument("--repo", type=str, default=os.environ.get("GITEA_REPO", DEFAULT_REPO))
parser.add_argument("--token", type=str, default=os.environ.get("GITEA_TOKEN"))
parser.add_argument("--username", type=str, default=os.environ.get("GITEA_USERNAME"))
parser.add_argument("--password", type=str, default=os.environ.get("GITEA_PASSWORD"))
parser.add_argument("--no-job-detail", action="store_true", help="不拉取 job 详情")
parser.add_argument("--max-failures", type=int, default=50, help="最多分析多少个失败 run 的 job 详情 (默认 50)")
# HTML 输出相关参数
parser.add_argument("--html", action="store_true", help="生成 HTML 可视化看板")
parser.add_argument("--html-output", type=str, help="HTML 输出文件路径 (默认 ci_dashboard.html)")
args = parser.parse_args()
ga = GiteaActions(
base_url=args.gitea_url,
repo=args.repo,
token=args.token,
username=args.username,
password=args.password,
)
end_date = datetime.now().date()
start_date = end_date - timedelta(days=args.days - 1)
runs = fetch_runs_in_range(ga, start_date, end_date, args.workflow)
if not runs:
print("[ERROR] 未获取到任何数据", file=sys.stderr)
sys.exit(1)
if not args.no_job_detail:
runs = enrich_with_jobs(ga, runs, max_failures=args.max_failures)
stats = analyze_runs(runs)
# HTML 模式
if args.html:
html = generate_html(stats, start_date, end_date, args.repo)
html_output = args.html_output or args.output or "ci_dashboard.html"
with open(html_output, "w", encoding="utf-8") as f:
f.write(html)
print(f"[INFO] HTML 看板已保存到 {html_output}", file=sys.stderr)
return
# 默认 Markdown 模式(向后兼容)
md = generate_markdown(stats, start_date, end_date, args.repo)
if args.output:
with open(args.output, "w", encoding="utf-8") as f:
f.write(md)
print(f"[INFO] 报表已保存到 {args.output}", file=sys.stderr)
else:
print(md)
if __name__ == "__main__":
main()
+14
View File
@@ -0,0 +1,14 @@
#!/bin/bash
# CI共享环境变量与常量定义
# 所有CI脚本source此文件获取统一的配置,避免硬编码分散
# === 共享常驻PG实例(CI_USE_SHARED_PG=true时使用)===
export CI_SHARED_PG_PORT="${CI_SHARED_PG_PORT:-5433}"
export CI_SHARED_PG_USER="${CI_SHARED_PG_USER:-postgres}"
export CI_SHARED_PG_PASSWORD="${CI_SHARED_PG_PASSWORD:-ci_pg_2026!}"
# === 本地PG默认端口(CI_USE_SHARED_PG=false时容器映射或本地PG===
export CI_LOCAL_PG_PORT="${CI_LOCAL_PG_PORT:-5432}"
# === 默认数据库名 ===
export CI_DEFAULT_DB="${CI_DEFAULT_DB:-xiaoxia_saas}"
+447
View File
@@ -0,0 +1,447 @@
#!/usr/bin/env python3
"""CI失败诊断增强脚本:自动分类失败原因 + 提取关键错误 + 给出修复建议。
# Trigger CI after auto-format fix
支持的失败类型:
1. Lint/格式问题 (ruff/black/eslint/prettier)
2. 单元测试失败
3. Docker构建失败
4. 依赖安装失败 (pip/npm)
5. 超时
6. 缓存问题
7. 数据库/迁移问题
8. 网络问题
9. 其他
用法:
python3 scripts/ci/ci_failure_diagnosis.py [--job-name "Job Name"] [--log-file /path/to/log]
如果不传--log-file,会尝试从Gitea API获取失败job的日志。
"""
import json
import os
import re
import sys
import urllib.request
from dataclasses import dataclass, field
from typing import List, Optional
@dataclass
class FailureDiagnosis:
"""失败诊断结果"""
category: str # 失败分类
category_cn: str # 中文分类名
severity: str # 严重程度: high / medium / low
summary: str # 一句话摘要
error_lines: List[str] = field(default_factory=list) # 关键错误行
suggestions: List[str] = field(default_factory=list) # 修复建议
auto_fixable: bool = False # 是否可以自动修复
related_docs: str = "" # 相关文档链接
# ============================================================
# 失败模式定义
# ============================================================
FAILURE_PATTERNS = [
# ===== Lint / 格式问题 =====
{
"pattern": r"(ruff|black|isort)\b.*(error|failed|Error)",
"category": "lint_python",
"category_cn": "Python代码质量检查",
"severity": "low",
"summary_contains": ["ruff", "black", "isort"],
"suggestions": [
"本地运行 `black . && isort . && ruff check --fix .` 自动修复",
"使用 `scripts/agent-commit.sh` 提交(自动格式化)",
"如确认无误,可加 `# noqa: xxx` 忽略特定规则",
],
"auto_fixable": True,
},
{
"pattern": r"ESLint|prettier|eslint",
"category": "lint_frontend",
"category_cn": "前端代码检查",
"severity": "low",
"summary_contains": ["eslint", "prettier"],
"suggestions": [
"本地运行 `cd apps/web && npm run lint:fix` 自动修复",
"Prettier问题: `cd apps/web && npx prettier --write .`",
],
"auto_fixable": True,
},
{
"pattern": r"F\d{3}|E\d{3}|W\d{3}.*ruff|ruff.*F\d{3}",
"category": "lint_python",
"category_cn": "Python代码质量检查",
"severity": "low",
"suggestions": [
"F401: 删除未使用的import",
"F841: 删除未使用的变量或加下划线前缀",
"E501: 行超长,加 `# noqa: E501`",
"F811: 删重复import",
"运行 `ruff check --fix .` 自动修复大部分问题",
],
"auto_fixable": True,
},
# ===== 单元测试失败 =====
{
"pattern": r"FAILED|assert.*Error|AssertionError",
"category": "unit_test",
"category_cn": "单元测试失败",
"severity": "high",
"suggestions": [
"检查相关测试文件,确认是代码问题还是测试用例问题",
"本地运行对应测试:`pytest path/to/test.py -v`",
"如测试依赖外部服务,检查mock是否正确",
],
"auto_fixable": False,
},
{
"pattern": r"pytest.*failed|\d+ failed.*\d+ passed",
"category": "unit_test",
"category_cn": "单元测试失败",
"severity": "high",
"suggestions": [
"查看上方日志中的FAILED测试用例",
"检查失败断言的期望值 vs 实际值",
"新代码影响了现有测试行为,确认是预期内变更吗?",
],
"auto_fixable": False,
},
# ===== Docker 构建失败 =====
{
"pattern": r"Dockerfile.*not found|docker build.*failed|ERROR: failed to solve",
"category": "docker_build",
"category_cn": "Docker构建失败",
"severity": "high",
"suggestions": [
"检查Dockerfile语法是否正确",
"检查引用的基础镜像是否存在",
"本地运行 `docker build -f path/to/Dockerfile .` 复现",
],
"auto_fixable": False,
},
{
"pattern": r"manifest.*not found|no such image|image.*not found",
"category": "docker_build",
"category_cn": "镜像不存在",
"severity": "medium",
"suggestions": [
"检查基础镜像名称和tag是否正确",
"确认镜像仓库可访问,登录是否有效",
"如为新基础镜像,需先手动构建一次基础镜像",
],
"auto_fixable": False,
},
{
"pattern": r"ETXTBSY|text file busy",
"category": "docker_build",
"category_cn": "文件锁冲突(ETXTBSY",
"severity": "low",
"summary": "esbuild并发构建冲突,重试即可",
"suggestions": ["偶发问题,点击Rerun重新运行即可", "如频繁出现,检查是否有多个job并发写入同一文件"],
"auto_fixable": True,
},
# ===== 依赖安装失败 =====
{
"pattern": r"pip install.*error|Could not find a version|No matching distribution",
"category": "dependency",
"category_cn": "pip依赖安装失败",
"severity": "medium",
"suggestions": [
"检查requirements.txt中的版本号是否正确",
"如为新版本刚发布,可能源还没同步,稍后重试",
"检查网络连接,可尝试切换pip镜像源",
],
"auto_fixable": False,
},
{
"pattern": r"npm.*ERR|npm install.*failed|E404|ECONNREFUSED.*npm",
"category": "dependency",
"category_cn": "npm依赖安装失败",
"severity": "medium",
"suggestions": [
"检查package.json中的版本号是否存在",
"网络问题:检查npm registry是否可访问",
"国内网络建议配置npmmirror镜像源",
],
"auto_fixable": False,
},
{
"pattern": r"Connection refused|timed out|network.*unreachable",
"category": "network",
"category_cn": "网络问题",
"severity": "medium",
"summary": "网络连接失败,可能是源站问题或DNS问题",
"suggestions": [
"点击Rerun重试,网络问题通常是临时的",
"如持续失败,检查对应服务是否正常",
"检查Runner网络配置",
],
"auto_fixable": True,
},
# ===== 超时 =====
{
"pattern": r"timeout|timed out|exceeded.*time limit|job.*cancelled.*timeout",
"category": "timeout",
"category_cn": "执行超时",
"severity": "medium",
"suggestions": [
"如首次出现:重试一次,可能是临时性能波动",
"频繁出现:检查构建是否变慢了,最近是否加了新依赖",
"可适当增加timeout-minutes配置",
],
"auto_fixable": False,
},
# ===== 数据库/迁移 =====
{
"pattern": r"alembic.*error|migration.*failed|relation.*does not exist|column.*does not exist",
"category": "migration",
"category_cn": "数据库迁移失败",
"severity": "high",
"suggestions": [
"检查迁移脚本是否正确,down_revision是否对",
"确认数据库中是否有脏数据或残留表",
"迁移脚本合并冲突时,重新生成迁移文件",
],
"auto_fixable": False,
},
# ===== 缓存问题 =====
{
"pattern": r"cache.*corrupt|cache.*invalid|snapshot.*not found|failed to compute cache key",
"category": "cache",
"category_cn": "缓存损坏",
"severity": "low",
"suggestions": ["构建系统会自动清理损坏缓存并重试,通常无需干预", "如持续失败,手动清理Runner上的缓存目录"],
"auto_fixable": True,
},
# ===== Checkout 失败 =====
{
"pattern": r"Could not resolve host|fatal:.*repository|SSL.*problem",
"category": "checkout",
"category_cn": "代码拉取失败",
"severity": "low",
"suggestions": ["临时网络问题,点击Rerun重试", "如持续失败,检查Gitea服务状态"],
"auto_fixable": True,
},
]
def analyze_log(log_text: str, job_name: str = "") -> FailureDiagnosis:
"""分析日志,返回诊断结果"""
lines = log_text.strip().split("\n")
# 收集所有匹配的模式
matched = []
error_lines = []
for line in lines:
line_stripped = line.strip()
# 收集ERROR/FAILED/Failed等错误行(最多20行)
if re.search(r"(ERROR|FAILED|Error|error:|FAIL:|Traceback)", line_stripped):
if len(error_lines) < 20:
error_lines.append(line_stripped)
for pattern_info in FAILURE_PATTERNS:
if re.search(pattern_info["pattern"], line_stripped, re.IGNORECASE):
matched.append(pattern_info)
break # 一行只匹配一个模式
if not matched:
# 未识别的失败类型
return FailureDiagnosis(
category="unknown",
category_cn="未知错误",
severity="medium",
summary="未识别的失败类型,需要人工查看日志",
error_lines=error_lines[:10],
suggestions=[
"点击'查看失败日志'查看完整日志",
"如为偶发问题,可先重试一次",
"常见原因:环境问题、配置问题、新增逻辑引入的bug",
],
auto_fixable=False,
)
# 选最严重、最具体的那个
severity_order = {"high": 3, "medium": 2, "low": 1}
matched.sort(key=lambda x: severity_order.get(x["severity"], 0), reverse=True)
best_match = matched[0]
# 生成摘要
if "summary" in best_match:
summary = best_match["summary"]
else:
summary = f"{best_match['category_cn']}检查失败"
if job_name:
summary = f"[{job_name}] {summary}"
# 从error_lines中过滤出与该分类相关的
relevant_errors = error_lines[:10]
return FailureDiagnosis(
category=best_match["category"],
category_cn=best_match["category_cn"],
severity=best_match["severity"],
summary=summary,
error_lines=relevant_errors,
suggestions=best_match["suggestions"],
auto_fixable=best_match.get("auto_fixable", False),
)
def fetch_failed_job_log(run_id: str, job_id: str, token: str, repo: str) -> Optional[str]:
"""从Gitea API获取失败job的日志"""
api_base = f"https://git.xiaoxiajianji.com/api/v1/repos/{repo}"
# 尝试获取job的日志
url = f"{api_base}/actions/runs/{run_id}/jobs/{job_id}/log"
req = urllib.request.Request(url)
req.add_header("Authorization", f"token {token}")
try:
with urllib.request.urlopen(req, timeout=15) as resp:
return resp.read().decode("utf-8", errors="replace")
except Exception as e:
print(f"获取日志失败: {e}", file=sys.stderr)
return None
def format_diagnosis_markdown(d: FailureDiagnosis, job_name: str = "", run_url: str = "") -> str:
"""将诊断结果格式化为飞书卡片markdown"""
severity_emoji = {"high": "🔴", "medium": "🟡", "low": "🟢"}
emoji = severity_emoji.get(d.severity, "")
lines = []
lines.append(f"**分类**: {emoji} {d.category_cn}")
lines.append(f"**问题**: {d.summary}")
if d.error_lines:
lines.append("")
lines.append("**关键错误行**:")
for err in d.error_lines[:5]:
# 截断过长的行
if len(err) > 150:
err = err[:147] + "..."
lines.append(f" `{err}`")
lines.append("")
lines.append("**修复建议**:")
for i, s in enumerate(d.suggestions[:5], 1):
lines.append(f" {i}. {s}")
if d.auto_fixable:
lines.append("")
lines.append("💡 **可自动修复**:如格式问题,可尝试点击Rerun让auto-fix自动处理")
if run_url:
lines.append("")
lines.append(f"[查看完整日志]({run_url})")
return "\n".join(lines)
def main():
job_name = os.environ.get("FAILED_JOB", "")
run_id = os.environ.get("GITHUB_RUN_ID", "")
repo = os.environ.get("GITHUB_REPOSITORY", "xiaoxia/xiaoxia-saas")
token = os.environ.get("GITHUB_TOKEN", "")
# 1. 尝试获取日志
log_text = ""
# 优先从环境变量或文件读取
log_file = os.environ.get("CI_LOG_FILE", "")
if log_file and os.path.exists(log_file):
with open(log_file) as f:
log_text = f.read()
elif run_id and token:
# 尝试从API获取(需要job_id,这里简化处理)
pass
# 如果没有日志,用job_name做粗略分类
if not log_text:
# 基于job名做初始判断
if any(k in job_name.lower() for k in ["validate", "lint", "quality"]):
d = FailureDiagnosis(
category="lint_general",
category_cn="代码质量检查",
severity="low",
summary=f"{job_name} 检查失败(日志不可用,基于job名初步诊断)",
suggestions=["点击查看日志获取具体错误信息", "格式类问题通常可自动修复"],
auto_fixable=True,
)
elif "build" in job_name.lower():
d = FailureDiagnosis(
category="build_general",
category_cn="构建失败",
severity="high",
summary=f"{job_name} 构建失败(日志不可用)",
suggestions=["点击查看日志获取具体构建错误", "常见原因:Dockerfile错误、依赖安装失败、网络问题"],
auto_fixable=False,
)
elif "test" in job_name.lower():
d = FailureDiagnosis(
category="test_general",
category_cn="测试失败",
severity="high",
summary=f"{job_name} 测试失败(日志不可用)",
suggestions=["点击查看日志获取具体失败的测试用例", "检查最近代码改动是否影响了测试"],
auto_fixable=False,
)
elif "deploy" in job_name.lower():
d = FailureDiagnosis(
category="deploy_general",
category_cn="部署失败",
severity="high",
summary=f"{job_name} 部署失败(日志不可用)",
suggestions=["检查目标服务器状态和网络", "检查镜像是否正确推送", "查看服务器上的容器日志"],
auto_fixable=False,
)
else:
d = FailureDiagnosis(
category="unknown",
category_cn="未知错误",
severity="medium",
summary=f"{job_name} 失败",
suggestions=["点击查看日志获取详细信息"],
auto_fixable=False,
)
else:
d = analyze_log(log_text, job_name)
# 输出诊断结果
run_url = f"https://git.xiaoxiajianji.com/{repo}/actions/runs/{run_id}" if run_id else ""
print("=" * 60)
print(" CI 失败诊断报告")
print("=" * 60)
print()
print(format_diagnosis_markdown(d, job_name, run_url))
print()
print("=" * 60)
# 将诊断结果写入文件(供通知脚本读取)
output_file = os.environ.get("DIAGNOSIS_OUTPUT", "/tmp/ci_diagnosis.json")
result = {
"category": d.category,
"category_cn": d.category_cn,
"severity": d.severity,
"summary": d.summary,
"error_lines": d.error_lines,
"suggestions": d.suggestions,
"auto_fixable": d.auto_fixable,
}
with open(output_file, "w") as f:
json.dump(result, f, ensure_ascii=False, indent=2)
print(f"\n诊断结果已保存到: {output_file}")
if __name__ == "__main__":
main()
+298
View File
@@ -0,0 +1,298 @@
#!/usr/bin/env python3
"""
CI 健康度快速检查脚本
- 统计最近 N 条 run 的成功率(按 workflow 分类)
- 列出失败的 run 和失败的 job/step
- 区分基础设施问题 vs 业务代码问题
- 输出简洁的健康度报告
用法:
python3 scripts/ci/ci_health_check.py [--limit 20] [--workflow ci-pipeline.yml] [--json]
环境变量:
GITEA_TOKEN API token(必需)
GITEA_API_URL Gitea API 地址,默认 https://git.xiaoxiajianji.com/api/v1
GITEA_REPO 仓库,默认 xiaoxia/xiaoxia-saas
"""
import argparse
import json
import os
import sys
import urllib.request
from datetime import datetime, timedelta, timezone
# ---- 基础设施问题关键词(命中即判定为基础设施问题)----
INFRA_KEYWORDS = [
# 网络/连接
"Couldn't connect to server",
"Connection refused",
"Connection reset",
"Connection timed out",
"Failed to connect to",
"network is unreachable",
"TLS handshake timeout",
"SSL certificate problem",
# 容器/Runner
"No such container",
"container already exists",
"docker: not found",
"no space left on device",
"out of memory",
"OOMKilled",
"pull access denied",
"manifest unknown",
"Error response from daemon",
"runner",
"runner is not online",
"no matching runners",
# Checkout/Git
"Could not resolve host",
"fatal: unable to access",
"The remote end hung up unexpectedly",
"early EOF",
"index-pack failed",
"git fetch",
"checkout failed",
"ETXTBSY",
"text file busy",
# 镜像/环境
"No module named pip",
"pip: not found",
"command not found: python",
"python3: not found",
"node: not found",
"npm: not found",
"exec format error",
"standard_init_linux.go",
# 系统/资源
"Input/output error",
"device or resource busy",
"No space left on device",
"Disk full",
# 鉴权/配置
"401 Unauthorized",
"403 Forbidden",
"404 Not Found",
"identity_sign: private key",
"Permission denied",
]
def api_get(path: str) -> dict:
base = os.environ.get("GITEA_API_URL", "https://git.xiaoxiajianji.com/api/v1")
repo = os.environ.get("GITEA_REPO", "xiaoxia/xiaoxia-saas")
token = os.environ.get("GITEA_TOKEN", "")
url = f"{base}/repos/{repo}/{path}"
req = urllib.request.Request(url, headers={"Authorization": f"token {token}"})
with urllib.request.urlopen(req, timeout=30) as resp:
return json.loads(resp.read().decode())
def get_run_jobs(run_id: int) -> list:
return api_get(f"actions/runs/{run_id}/jobs").get("jobs", [])
def get_job_log(job_id: int) -> str:
try:
return api_get(f"actions/jobs/{job_id}/logs")
except Exception:
return ""
def classify_failure(job: dict) -> str:
"""判断失败原因类型: infra / business / unknown"""
name = job.get("name", "")
# 仅根据 job 名称做初步分类(更精确需读日志,但代价高)
infra_jobs = ["Checkout", "Build", "Deploy", "Cleanup"]
business_jobs = [
"Unit Tests",
"Integration Tests",
"Frontend Lint",
"Frontend Unit Tests",
"Staging E2E",
"E2E",
"Validate Code Quality",
]
name_lower = name.lower()
if (
any(k.lower() in name_lower for k in infra_jobs)
and "Test" not in name
and "Lint" not in name
and "Validate" not in name
):
return "infra"
if any(k.lower() in name_lower for k in business_jobs):
return "business"
return "unknown"
def analyze_with_log(job_id: int) -> str:
"""通过日志关键词精确分类"""
log = get_job_log(job_id)
log_lower = log.lower()
for kw in INFRA_KEYWORDS:
if kw.lower() in log_lower:
return "infra"
return "business"
def fmt_time(t: str) -> str:
if not t or t.startswith("1970"):
return "-"
try:
dt = datetime.fromisoformat(t.replace("Z", "+00:00"))
bj = dt.astimezone(timezone(timedelta(hours=8)))
return bj.strftime("%m-%d %H:%M")
except Exception:
return t[:16]
def main():
parser = argparse.ArgumentParser(description="CI 健康度快速检查")
parser.add_argument("--limit", type=int, default=20, help="最近多少条 run")
parser.add_argument("--workflow", type=str, default="", help="只看某个 workflow")
parser.add_argument("--json", action="store_true", help="JSON 输出")
parser.add_argument("--deep", action="store_true", help="深度检查(读日志,较慢)")
args = parser.parse_args()
if not os.environ.get("GITEA_TOKEN"):
print("错误: 请设置 GITEA_TOKEN 环境变量", file=sys.stderr)
sys.exit(1)
# 1. 获取最近 run
runs = api_get(f"actions/runs?limit={args.limit}").get("workflow_runs", [])
if args.workflow:
runs = [r for r in runs if args.workflow in r.get("path", "")]
if not runs:
print("没有找到匹配的 run")
return
# 按 workflow 分组统计
wf_stats = {}
failed_runs = []
for r in runs:
path = r.get("path", "unknown")
# 提取 workflow 文件名,兼容各种 path 格式
if ".yml" in path or ".yaml" in path:
# ci-pipeline.yml@refs/heads/develop -> ci-pipeline.yml
wf = path.split("@")[0].split("/")[-1]
else:
wf = path.split("/")[-1] if "/" in path else path
if wf not in wf_stats:
wf_stats[wf] = {"total": 0, "success": 0, "failure": 0, "cancelled": 0, "others": 0}
wf_stats[wf]["total"] += 1
status = r.get("status", "")
conc = r.get("conclusion", "")
if status != "completed":
wf_stats[wf]["others"] += 1
continue
if conc == "success":
wf_stats[wf]["success"] += 1
elif conc == "failure":
wf_stats[wf]["failure"] += 1
failed_runs.append(r)
elif conc == "cancelled":
wf_stats[wf]["cancelled"] += 1
else:
wf_stats[wf]["others"] += 1
# 2. 失败 run 详情
failed_details = []
for r in failed_runs[:10]: # 最多看10个失败的
jobs = get_run_jobs(r["id"])
failed_jobs = [j for j in jobs if j.get("conclusion") == "failure"]
job_infos = []
for j in failed_jobs:
cat = classify_failure(j)
if args.deep and cat == "unknown":
cat = analyze_with_log(j["id"])
# 找失败的 step
failed_steps = []
for step in j.get("steps", []):
if step.get("conclusion") == "failure":
failed_steps.append(step.get("name", "?"))
job_infos.append(
{
"name": j.get("name", ""),
"category": cat,
"failed_steps": failed_steps,
"runner": j.get("runner_name", ""),
}
)
failed_details.append(
{
"id": r["id"],
"title": r.get("display_title", ""),
"branch": r.get("head_branch", ""),
"time": fmt_time(r.get("updated_at", "")),
"jobs": job_infos,
}
)
# 3. 输出
if args.json:
result = {"workflows": wf_stats, "failed_runs": failed_details}
print(json.dumps(result, ensure_ascii=False, indent=2))
return
# 文本报告
print("=" * 60)
print(" CI 健康度报告")
print("=" * 60)
print(f"统计范围: 最近 {len(runs)} 条 run")
print(f"时间: {datetime.now(timezone(timedelta(hours=8))).strftime('%Y-%m-%d %H:%M:%S')}")
print()
print("📊 各 Workflow 成功率:")
print("-" * 60)
for wf, s in sorted(wf_stats.items()):
total = s["total"]
succ = s["success"]
rate = (succ / total * 100) if total > 0 else 0
bar = "" * int(rate / 5) + "" * (20 - int(rate / 5))
icon = "🟢" if rate >= 90 else ("🟡" if rate >= 70 else "🔴")
print(f" {icon} {wf:35s} {rate:5.1f}% {bar} ({succ}/{total})")
if s["failure"]:
print(f" 失败: {s['failure']} 取消: {s['cancelled']} 进行中: {s['others']}")
if failed_details:
print()
print("❌ 失败详情:")
print("-" * 60)
for d in failed_details:
print(f" #{d['id']} [{d['time']}] {d['title'][:45]}")
print(f" 分支: {d['branch']}")
for j in d["jobs"]:
cat_icon = "🏗️" if j["category"] == "infra" else ("🐛" if j["category"] == "business" else "")
steps = ", ".join(j["failed_steps"][:3]) if j["failed_steps"] else "未知"
print(f" {cat_icon} {j['name'][:30]:30s} 失败步骤: {steps}")
if j["runner"]:
print(f" runner: {j['runner']}")
else:
print()
print("✅ 最近没有失败的 run")
# 总结
total_all = sum(s["total"] for s in wf_stats.values())
succ_all = sum(s["success"] for s in wf_stats.values())
fail_all = sum(s["failure"] for s in wf_stats.values())
infra_fail = sum(1 for d in failed_details for j in d["jobs"] if j["category"] == "infra")
biz_fail = sum(1 for d in failed_details for j in d["jobs"] if j["category"] == "business")
rate_all = (succ_all / total_all * 100) if total_all > 0 else 0
print()
print("=" * 60)
print(f" 总结: 总成功率 {rate_all:.1f}% ({succ_all}/{total_all})")
if fail_all > 0:
print(f" 失败job分类: 基础设施 {infra_fail} 个 | 业务代码 {biz_fail}")
if infra_fail > biz_fail:
print(" ⚠️ 主要是基础设施问题,建议优先排查 CI 环境")
else:
print(" 💡 主要是业务代码问题,建议关注业务侧修复")
print("=" * 60)
if __name__ == "__main__":
main()
+251
View File
@@ -0,0 +1,251 @@
#!/usr/bin/env python3
"""
CI健康度每日巡检报告脚本
- 调用ci_health_check.py获取数据
- 有失败时生成飞书卡片通知并发送
- 无失败时静默退出(不打扰)
- 用于每日定时巡检
用法:
python3 scripts/ci/ci_health_report.py [--limit 30] [--dry-run]
环境变量:
GITEA_TOKEN API token(必需)
CI_NOTIFY_WEBHOOK 飞书webhook地址(必需,用于发报告)
GITEA_API_URL Gitea API 地址
GITEA_REPO 仓库
"""
import argparse
import json
import os
import subprocess
import sys
import urllib.request
from datetime import datetime, timedelta, timezone
def run_health_check(limit: int) -> dict:
"""调用ci_health_check.py获取JSON结果"""
script_dir = os.path.dirname(os.path.abspath(__file__))
cmd = [
sys.executable,
os.path.join(script_dir, "ci_health_check.py"),
"--json",
"--limit",
str(limit),
]
env = os.environ.copy()
# 确保GITEA_TOKEN传递
if not env.get("GITEA_TOKEN") and env.get("GITHUB_TOKEN"):
env["GITEA_TOKEN"] = env["GITHUB_TOKEN"]
result = subprocess.run(cmd, capture_output=True, text=True, env=env)
if result.returncode != 0:
print(f"health check failed: {result.stderr}")
return {"workflows": {}, "failed_runs": []}
try:
return json.loads(result.stdout)
except json.JSONDecodeError:
print(f"failed to parse health check output: {result.stdout[:200]}")
return {"workflows": {}, "failed_runs": []}
def build_feishu_card(data: dict) -> dict:
"""构建飞书卡片消息"""
wf_stats = data.get("workflows", {})
failed_runs = data.get("failed_runs", [])
# 统计数据
total_all = sum(s["total"] for s in wf_stats.values())
succ_all = sum(s["success"] for s in wf_stats.values())
fail_all = sum(s["failure"] for s in wf_stats.values())
rate_all = (succ_all / total_all * 100) if total_all > 0 else 0
# 失败分类
infra_fail = 0
biz_fail = 0
unknown_fail = 0
for run in failed_runs:
for job in run.get("jobs", []):
cat = job.get("category", "unknown")
if cat == "infra":
infra_fail += 1
elif cat == "business":
biz_fail += 1
else:
unknown_fail += 1
now = datetime.now(timezone(timedelta(hours=8))).strftime("%Y-%m-%d %H:%M")
# 各workflow成功率行
wf_lines = []
for wf, s in sorted(wf_stats.items()):
total = s["total"]
succ = s["success"]
fail = s["failure"]
rate = (succ / total * 100) if total > 0 else 0
icon = "🟢" if rate >= 90 else ("🟡" if rate >= 70 else "🔴")
wf_name = (
wf.replace("ci-pipeline.yml", "CI Pipeline")
.replace("code-review.yml", "Code Review")
.replace("daily-check.yml", "Daily Check")
.replace("preview-deploy.yml", "Preview Deploy")
)
wf_lines.append(f"{icon} **{wf_name}**: {rate:.0f}% ({succ}/{total},失败{fail})")
# 失败详情(最多显示5条)
fail_detail_lines = []
for _i, run in enumerate(failed_runs[:5]):
run_id = run["id"]
title = run.get("title", "")[:35]
branch = run.get("branch", "")
jobs_str = ", ".join(j["name"][:15] for j in run.get("jobs", [])[:3])
fail_detail_lines.append(f"• **#{run_id}** {title}\n 分支: {branch} | 失败: {jobs_str}")
if len(failed_runs) > 5:
fail_detail_lines.append(f"... 还有 {len(failed_runs) - 5} 条失败记录")
# 整体状态
if fail_all == 0:
status_text = "✅ 全部通过"
status_color = "green"
elif infra_fail > biz_fail:
status_text = "⚠️ 基础设施问题为主"
status_color = "yellow"
else:
status_text = "🔴 存在业务失败"
status_color = "red"
card = {
"config": {"wide_screen_mode": True},
"header": {
"title": {"tag": "plain_text", "content": f"CI告警 - 每日健康度巡检 ({now})"},
"template": status_color,
},
"elements": [
{
"tag": "div",
"text": {
"tag": "lark_md",
"content": f"**统计范围**: 最近 {total_all} 条 run\n**整体状态**: {status_text}\n**总成功率**: {rate_all:.1f}% ({succ_all}/{total_all})",
},
},
{"tag": "hr"},
{
"tag": "div",
"text": {
"tag": "lark_md",
"content": "**📊 各Workflow成功率**\n" + "\n".join(wf_lines) if wf_lines else "暂无数据",
},
},
],
}
# 失败分类统计
if fail_all > 0:
card["elements"].append({"tag": "hr"})
card["elements"].append(
{
"tag": "div",
"text": {
"tag": "lark_md",
"content": f"**失败原因分类**\n🏗️ 基础设施: {infra_fail}\n🐛 业务代码: {biz_fail}\n❓ 待确认: {unknown_fail}",
},
}
)
# 失败详情
if fail_detail_lines:
card["elements"].append({"tag": "hr"})
card["elements"].append(
{
"tag": "div",
"text": {
"tag": "lark_md",
"content": "**❌ 失败详情**\n" + "\n\n".join(fail_detail_lines),
},
}
)
# 查看更多
card["elements"].append({"tag": "hr"})
base_url = os.environ.get("GITEA_BASE_URL", "https://git.xiaoxiajianji.com")
repo = os.environ.get("GITEA_REPO", "xiaoxia/xiaoxia-saas")
card["elements"].append(
{
"tag": "action",
"actions": [
{
"tag": "button",
"text": {"tag": "plain_text", "content": "查看CI面板"},
"type": "primary",
"url": f"{base_url}/{repo}/actions",
}
],
}
)
return {"msg_type": "interactive", "card": card}
def send_feishu(webhook: str, payload: dict) -> bool:
"""发送飞书webhook"""
data = json.dumps(payload).encode()
req = urllib.request.Request(
webhook,
data=data,
headers={"Content-Type": "application/json"},
)
try:
with urllib.request.urlopen(req, timeout=10) as resp:
result = json.loads(resp.read().decode())
return result.get("code", -1) == 0 or result.get("StatusCode", -1) == 0
except Exception as e:
print(f"send feishu failed: {e}")
return False
def main():
parser = argparse.ArgumentParser(description="CI健康度每日巡检报告")
parser.add_argument("--limit", type=int, default=30, help="统计最近N条run")
parser.add_argument("--dry-run", action="store_true", help="只打印不发送")
parser.add_argument("--always-notify", action="store_true", help="即使全部通过也发送通知")
args = parser.parse_args()
webhook = os.environ.get("CI_NOTIFY_WEBHOOK", "")
if not webhook and not args.dry_run:
print("未配置 CI_NOTIFY_WEBHOOK,跳过通知")
# 还是执行健康检查输出到日志,方便排查
data = run_health_check(args.limit)
print(f"health check done: {len(data.get('failed_runs', []))} failed")
return 0
# 执行健康检查
data = run_health_check(args.limit)
failed_count = len(data.get("failed_runs", []))
# 无失败且不强制通知 → 静默退出
if failed_count == 0 and not args.always_notify:
print("✅ 全部通过,静默退出")
return 0
# 构建并发送卡片
card = build_feishu_card(data)
if args.dry_run:
print(json.dumps(card, ensure_ascii=False, indent=2))
return 0
success = send_feishu(webhook, card)
if success:
print(f"📤 已发送健康度报告,失败 {failed_count}")
else:
print("❌ 发送飞书通知失败")
# 通知失败不阻断流程
return 0
if __name__ == "__main__":
sys.exit(main())
+417
View File
@@ -0,0 +1,417 @@
#!/usr/bin/env python3
"""
CI重复失败检测脚本
- 扫描最近N天的CI失败
- 按job名称分组统计失败率
- 识别高失败率job(系统性故障)
- 飞书通知告警
"""
import json
import os
import sys
import time
import urllib.error
import urllib.request
from collections import defaultdict
from datetime import datetime, timedelta, timezone
def get_env(name, default=None, required=False):
val = os.environ.get(name, default)
if required and not val:
print(f"❌ 缺少环境变量: {name}")
sys.exit(1)
return val
GITEA_URL = get_env("GITEA_URL", "https://git.xiaoxiajianji.com")
GITEA_TOKEN = get_env("GITEA_API_TOKEN", required=False) or get_env("GITHUB_TOKEN", "")
REPO = get_env("GITEA_REPO", "xiaoxia/xiaoxia-saas")
DAYS = int(get_env("FAIL_CHECK_DAYS", "7"))
FAIL_THRESHOLD = int(get_env("FAIL_THRESHOLD", 3)) # 失败次数阈值
FAIL_RATE_THRESHOLD = float(get_env("FAIL_RATE_THRESHOLD", "30")) # 失败率阈值%
CONSECUTIVE_FAIL_THRESHOLD = int(get_env("CONSECUTIVE_FAIL_THRESHOLD", "3")) # 连续失败阈值
WEBHOOK = get_env("CI_NOTIFY_WEBHOOK", "")
def api_get(path):
"""调用Gitea API"""
url = f"{GITEA_URL}/api/v1{path}"
req = urllib.request.Request(url)
if GITEA_TOKEN:
req.add_header("Authorization", f"token {GITEA_TOKEN}")
try:
with urllib.request.urlopen(req, timeout=30) as resp:
return json.loads(resp.read())
except urllib.error.HTTPError as e:
print(f" HTTP {e.code}: {path}")
return None
except Exception as e:
print(f" 错误: {e}")
return None
def fetch_recent_runs(days=7, per_page=50, max_pages=10):
"""获取最近N天的runs"""
since = (datetime.now(timezone.utc) - timedelta(days=days)).isoformat()
all_runs = []
for page in range(1, max_pages + 1):
path = f"/repos/{REPO}/actions/runs?page={page}&limit={per_page}"
data = api_get(path)
if not data:
break
runs = data.get("workflow_runs", data.get("runs", []))
if not runs:
break
# 检查时间范围(Gitea用started_at,格式2026-07-22T10:58:10+08:00
oldest = None
for r in runs:
started = r.get("started_at", r.get("created_at", ""))
if started and started >= since:
all_runs.append(r)
else:
oldest = started
if oldest and oldest < since:
break
if len(runs) < per_page:
break
return all_runs
def fetch_run_jobs(run_id):
"""获取run的所有jobs"""
path = f"/repos/{REPO}/actions/runs/{run_id}/jobs"
data = api_get(path)
if not data:
return []
return data.get("jobs", [])
def analyze_failures(runs):
"""
分析失败情况
返回:
- job_stats: {job_name: {total, success, failure, skipped, failure_rate, failures: [...]}}
- consecutive_failures: {job_name: current_streak, max_streak, last_status}
"""
job_stats = defaultdict(
lambda: {
"total": 0,
"success": 0,
"failure": 0,
"error": 0,
"skipped": 0,
"cancelled": 0,
"failures": [],
}
)
# 按时间正序排列(旧→新)用于连续失败计算
sorted_runs = sorted(runs, key=lambda r: r.get("started_at", r.get("created_at", "")))
# 连续失败跟踪 {job_name: streak}
consecutive = defaultdict(lambda: {"current": 0, "max": 0, "last_run": None})
for run in sorted_runs:
run_id = run.get("id")
run.get("status", "")
run.get("conclusion", "")
run_started = run.get("started_at", run.get("created_at", ""))
event = run.get("event", "")
# 只统计pull_request和push事件的CI
if event not in ("pull_request", "push"):
continue
jobs = fetch_run_jobs(run_id)
for job in jobs:
name = job.get("name", "")
job.get("status", "")
conclusion = job.get("conclusion", "")
# 跳过非CI核心job(如AI Code Review、Preview等)
skip_prefixes = ("AI Code Review", "Preview", "PR Automation", "Auto")
if any(name.startswith(p) for p in skip_prefixes):
continue
stats = job_stats[name]
stats["total"] += 1
if conclusion == "success":
stats["success"] += 1
consecutive[name]["current"] = 0
elif conclusion == "failure":
stats["failure"] += 1
stats["failures"].append(
{
"run_id": run_id,
"time": run_started,
"event": event,
}
)
consecutive[name]["current"] += 1
if consecutive[name]["current"] > consecutive[name]["max"]:
consecutive[name]["max"] = consecutive[name]["current"]
consecutive[name]["last_run"] = run_id
elif conclusion == "error":
stats["error"] += 1
# error也算失败的一种
consecutive[name]["current"] += 1
if consecutive[name]["current"] > consecutive[name]["max"]:
consecutive[name]["max"] = consecutive[name]["current"]
elif conclusion == "skipped":
stats["skipped"] += 1
# skipped不算也不打断连续失败
elif conclusion == "cancelled":
stats["cancelled"] += 1
# cancelled不算失败也不打断
# 计算失败率
for _name, stats in job_stats.items():
total_actual = stats["total"] - stats["skipped"] - stats["cancelled"]
if total_actual > 0:
stats["failure_rate"] = round((stats["failure"] + stats["error"]) / total_actual * 100, 1)
else:
stats["failure_rate"] = 0.0
return dict(job_stats), dict(consecutive)
def find_high_failures(job_stats, consecutive):
"""
找出高风险job
告警级别:
- critical: 连续失败 >= CONSECUTIVE_FAIL_THRESHOLD,或 失败率>=50%且失败次数>=5
- warning: 失败率>=FAIL_RATE_THRESHOLD且失败次数>=FAIL_THRESHOLD
- info: 失败次数>=2
"""
critical = []
warning = []
info = []
for name, stats in job_stats.items():
fail_count = stats["failure"] + stats["error"]
rate = stats["failure_rate"]
streak = consecutive.get(name, {}).get("current", 0)
max_streak = consecutive.get(name, {}).get("max", 0)
issue = {
"name": name,
"fail_count": fail_count,
"total": stats["total"],
"failure_rate": rate,
"current_streak": streak,
"max_streak": max_streak,
"recent_failures": stats["failures"][-5:], # 最近5次
}
if streak >= CONSECUTIVE_FAIL_THRESHOLD or (rate >= 50 and fail_count >= 5):
critical.append(issue)
elif rate >= FAIL_RATE_THRESHOLD and fail_count >= FAIL_THRESHOLD:
warning.append(issue)
elif fail_count >= 2:
info.append(issue)
# 按失败次数倒序
critical.sort(key=lambda x: x["fail_count"], reverse=True)
warning.sort(key=lambda x: x["fail_count"], reverse=True)
info.sort(key=lambda x: x["fail_count"], reverse=True)
return critical, warning, info
def generate_report(critical, warning, info, days, total_runs):
"""生成Markdown报告"""
lines = []
lines.append("# CI重复失败检测报告")
lines.append("")
lines.append(f"**统计周期**: 最近{days}")
lines.append(f"**扫描Runs**: {total_runs}")
lines.append(f"**生成时间**: {datetime.now(timezone.utc).strftime('%Y-%m-%d %H:%M UTC')}")
lines.append("")
lines.append("## 概览")
lines.append("")
lines.append("| 级别 | 数量 |")
lines.append("|------|------|")
lines.append(f"| 🔴 严重 (连续失败≥{CONSECUTIVE_FAIL_THRESHOLD}次 或 失败率≥50%) | {len(critical)} |")
lines.append(f"| 🟡 警告 (失败率≥{FAIL_RATE_THRESHOLD}% 且 失败≥{FAIL_THRESHOLD}次) | {len(warning)} |")
lines.append(f"| 🔵 关注 (失败≥2次) | {len(info)} |")
lines.append("")
if critical:
lines.append("## 🔴 严重问题")
lines.append("")
for item in critical:
lines.append(f"### {item['name']}")
lines.append("")
lines.append(f"- 失败次数: **{item['fail_count']}** / {item['total']} 次运行")
lines.append(f"- 失败率: **{item['failure_rate']}%**")
lines.append(f"- 当前连续失败: **{item['current_streak']}** 次 (历史最高: {item['max_streak']} 次)")
lines.append("")
if item["recent_failures"]:
lines.append("最近失败:")
lines.append("")
for f in item["recent_failures"]:
lines.append(f"- [{f['time'][:16]}] run #{f['run_id']} ({f['event']})")
lines.append("")
if warning:
lines.append("## 🟡 警告")
lines.append("")
for item in warning:
lines.append(
f"- **{item['name']}**: {item['fail_count']}次失败 / {item['total']}次运行 ({item['failure_rate']}%)"
)
lines.append("")
if info:
lines.append("## 🔵 关注列表")
lines.append("")
lines.append("| Job名称 | 失败次数 | 总次数 | 失败率 | 当前连续 |")
lines.append("|---------|----------|--------|--------|----------|")
for item in info[:20]: # 最多显示20个
lines.append(
f"| {item['name']} | {item['fail_count']} | {item['total']} | {item['failure_rate']}% | {item['current_streak']} |"
)
lines.append("")
return "\n".join(lines)
def send_feishu_notification(critical, warning, info, days):
"""发送飞书通知"""
if not WEBHOOK:
print(" ⚠️ 未配置WEBHOOK,跳过飞书通知")
return False
total_issues = len(critical) + len(warning) + len(info)
if total_issues == 0:
print(" ✅ 无异常,不发送通知")
return True
level = "🔴 严重告警" if critical else "🟡 警告" if warning else "🔵 关注"
title = f"CI重复失败检测 - {level}"
text = f"统计周期: 最近{days}\n\n"
if critical:
text += "【严重问题】\n"
for item in critical[:5]:
text += f"{item['name']}\n"
text += f" 失败 {item['fail_count']}/{item['total']} ({item['failure_rate']}%) 连续{item['current_streak']}\n"
if len(critical) > 5:
text += f" ...还有{len(critical)-5}\n"
text += "\n"
if warning:
text += "【警告】\n"
for item in warning[:5]:
text += f"{item['name']}: {item['fail_count']}次失败 ({item['failure_rate']}%)\n"
if len(warning) > 5:
text += f" ...还有{len(warning)-5}\n"
text += "\n"
if info and not critical and not warning:
text += "【关注列表】\n"
for item in info[:10]:
text += f"{item['name']}: {item['fail_count']}次失败\n"
text += "\n"
text += f"共发现 {total_issues} 个异常job"
payload = {"msg_type": "text", "content": {"text": f"{title}\n\n{text}"}}
data = json.dumps(payload).encode()
req = urllib.request.Request(WEBHOOK, data=data, headers={"Content-Type": "application/json"})
try:
with urllib.request.urlopen(req, timeout=10) as resp:
result = json.loads(resp.read())
if result.get("code") == 0 or result.get("StatusCode") == 0:
print(" ✅ 飞书通知已发送")
return True
else:
print(f" ⚠️ 飞书返回: {result}")
return False
except Exception as e:
print(f" ❌ 飞书通知失败: {e}")
return False
def main():
print("=== CI重复失败检测 ===")
print(f"统计周期: 最近{DAYS}")
print(f"仓库: {REPO}")
print()
print("1. 获取最近的Runs...")
runs = fetch_recent_runs(days=DAYS)
print(f" 找到 {len(runs)} 个runs")
if not runs:
print("⚠️ 没有找到runs,退出")
return
print()
print("2. 分析job失败情况(可能需要点时间)...")
job_stats, consecutive = analyze_failures(runs)
print(f" 共统计 {len(job_stats)} 个job")
print()
print("3. 识别高风险job...")
critical, warning, info = find_high_failures(job_stats, consecutive)
print(f" 🔴 严重: {len(critical)}")
print(f" 🟡 警告: {len(warning)}")
print(f" 🔵 关注: {len(info)}")
print()
print("4. 生成报告...")
report = generate_report(critical, warning, info, DAYS, len(runs))
# 保存报告
report_path = os.environ.get("REPORT_PATH", f"/tmp/ci_failure_report_{int(time.time())}.md")
with open(report_path, "w") as f:
f.write(report)
print(f" 报告已保存: {report_path}")
# 打印摘要
print()
print("=== 摘要 ===")
if critical:
print("🔴 严重问题:")
for item in critical[:5]:
print(
f" {item['name']}: {item['fail_count']}次失败, {item['failure_rate']}%, 连续{item['current_streak']}"
)
if warning:
print("🟡 警告:")
for item in warning[:5]:
print(f" {item['name']}: {item['fail_count']}次失败, {item['failure_rate']}%")
print()
print("5. 发送飞书通知...")
send_feishu_notification(critical, warning, info, DAYS)
print()
print("✅ 检测完成")
# 有严重问题时退出码非零,方便workflow标记
if critical:
sys.exit(2)
elif warning:
sys.exit(1)
if __name__ == "__main__":
main()
+375
View File
@@ -0,0 +1,375 @@
#!/usr/bin/env python3
"""
CI Trace Report Script - Reports CI Trace data to AgentLoop from Gitea Actions workflows.
Usage in CI workflow jobs:
- At start: python3 scripts/ci/ci_trace_report.py --status running
- At end: python3 scripts/ci/ci_trace_report.py --status ok --start-time $CI_TRACE_START_TIME
Environment variables (built-in Gitea Actions):
GITEA_REPOSITORY / GITHUB_REPOSITORY - repository (owner/repo)
GITEA_WORKFLOW / GITHUB_WORKFLOW - workflow name
GITEA_JOB / GITHUB_JOB - job ID
GITEA_SHA / GITHUB_SHA - commit SHA
GITEA_REF_NAME / GITHUB_REF_NAME - branch name
GITEA_RUN_ID / GITHUB_RUN_ID - run ID
GITEA_ACTOR / GITHUB_ACTOR - trigger actor
GITEA_EVENT_NAME / GITHUB_EVENT_NAME - event type
PR_NUMBER / GITEA_PR_NUMBER - PR number (if PR triggered)
AgentLoop configuration (injected via Secrets):
AGENTLOOP_LICENSE_KEY - LicenseKey (required)
AGENTLOOP_ENDPOINT - Trace endpoint (optional, has default)
AGENTLOOP_PROJECT - SLS Project name (optional)
AGENTLOOP_WORKSPACE - CMS Workspace name (optional)
"""
import argparse
import json
import os
import sys
import time
import urllib.error
import urllib.request
import uuid
# ========== Default Configuration ==========
DEFAULT_ENDPOINT = "https://proj-xtrace-495e81719a1fd9a2c5fd671eefafbe-cn-hangzhou.cn-hangzhou.log.aliyuncs.com/apm/trace/opentelemetry/v1/traces"
DEFAULT_PROJECT = "proj-xtrace-495e81719a1fd9a2c5fd671eefafbe-cn-hangzhou"
DEFAULT_WORKSPACE = "agentloop-13b8d6efb7fde6e9b193eb982ade68e2"
# ========== OTLP Protobuf Manual Encoding ==========
def _encode_varint(value):
result = bytearray()
while value > 0x7F:
result.append((value & 0x7F) | 0x80)
value >>= 7
result.append(value & 0x7F)
return bytes(result)
def _encode_tag(field_number, wire_type):
return _encode_varint((field_number << 3) | wire_type)
def _encode_string_field(field_number, value):
value_bytes = value.encode("utf-8")
return _encode_tag(field_number, 2) + _encode_varint(len(value_bytes)) + value_bytes
def _encode_bytes_field(field_number, value_bytes):
return _encode_tag(field_number, 2) + _encode_varint(len(value_bytes)) + value_bytes
def _encode_int_field(field_number, value):
return _encode_tag(field_number, 0) + _encode_varint(value & 0xFFFFFFFFFFFFFFFF)
def _encode_message_field(field_number, message_bytes):
return _encode_tag(field_number, 2) + _encode_varint(len(message_bytes)) + message_bytes
def _encode_key_value(key, value_str):
any_value = _encode_string_field(1, value_str)
return _encode_string_field(1, key) + _encode_message_field(2, any_value)
def _encode_status(status_code, status_msg=""):
data = _encode_int_field(1, status_code)
if status_msg:
data += _encode_string_field(2, status_msg)
return data
def _encode_span(
trace_id_bytes,
span_id_bytes,
parent_span_id_bytes,
name,
start_time_unix_nano,
end_time_unix_nano,
span_kind,
attributes,
status_code,
status_msg="",
):
data = b""
data += _encode_bytes_field(1, trace_id_bytes)
data += _encode_bytes_field(2, span_id_bytes)
if parent_span_id_bytes:
data += _encode_bytes_field(3, parent_span_id_bytes)
data += _encode_string_field(4, name)
data += _encode_int_field(5, span_kind)
data += _encode_int_field(6, start_time_unix_nano)
data += _encode_int_field(7, end_time_unix_nano)
for key, value in attributes.items():
kv = _encode_key_value(key, str(value))
data += _encode_message_field(9, kv)
status = _encode_status(status_code, status_msg)
data += _encode_message_field(12, status)
return data
def _encode_resource_spans(service_name, scope_spans_bytes):
svc_kv = _encode_key_value("service.name", service_name)
resource = _encode_message_field(1, svc_kv)
data = _encode_message_field(1, resource)
data += _encode_message_field(2, scope_spans_bytes)
return data
def _encode_scope_spans(scope_name, spans_bytes_list):
scope = _encode_string_field(1, scope_name)
data = _encode_message_field(1, scope)
for span_bytes in spans_bytes_list:
data += _encode_message_field(2, span_bytes)
return data
def _encode_traces_data(resource_spans_bytes_list):
data = b""
for rs_bytes in resource_spans_bytes_list:
data += _encode_message_field(1, rs_bytes)
return data
# ========== Helper Functions ==========
def _gen_trace_id():
return uuid.uuid4().bytes
def _gen_span_id():
return uuid.uuid4().bytes[:8]
def _env(name, default=""):
"""Get env var with GITEA_/GITHUB_ prefix fallback."""
val = os.getenv(name, "")
if val:
return val
if name.startswith("GITEA_"):
alt = "GITHUB_" + name[6:]
return os.getenv(alt, default)
if name.startswith("GITHUB_"):
alt = "GITEA_" + name[7:]
return os.getenv(alt, default)
return default
def _get_pr_number():
"""Get PR number from environment or event file."""
pr = os.getenv("PR_NUMBER", "") or os.getenv("GITEA_PR_NUMBER", "")
if pr:
return pr
event_path = os.getenv("GITHUB_EVENT_PATH", "") or os.getenv("GITEA_EVENT_PATH", "")
if event_path and os.path.isfile(event_path):
try:
with open(event_path, "r") as f:
event = json.load(f)
if "pull_request" in event and "number" in event["pull_request"]:
return str(event["pull_request"]["number"])
except Exception:
pass
return ""
def _get_ci_attributes():
"""Collect attributes from CI environment variables."""
attrs = {
"ci.repo": _env("GITEA_REPOSITORY") or _env("GITHUB_REPOSITORY") or "unknown",
"ci.workflow": _env("GITEA_WORKFLOW") or _env("GITHUB_WORKFLOW") or "unknown",
"ci.job": _env("GITEA_JOB") or _env("GITHUB_JOB") or "unknown",
"ci.commit_sha": _env("GITEA_SHA") or _env("GITHUB_SHA") or "unknown",
"ci.branch": _env("GITEA_REF_NAME") or _env("GITHUB_REF_NAME") or "unknown",
"ci.run_id": _env("GITEA_RUN_ID") or _env("GITHUB_RUN_ID") or "unknown",
"ci.actor": _env("GITEA_ACTOR") or _env("GITHUB_ACTOR") or "unknown",
"ci.event": _env("GITEA_EVENT_NAME") or _env("GITHUB_EVENT_NAME") or "unknown",
}
pr = _get_pr_number()
if pr:
attrs["ci.pr_number"] = pr
return attrs
# ========== Trace Building & Reporting ==========
def build_trace(service_name, trace_name, status, duration_ms, attributes=None):
"""Build an OTLP trace payload (protobuf bytes). No external dependencies."""
trace_id = _gen_trace_id()
end_time = int(time.time() * 1e9)
start_time = end_time - int(duration_ms * 1e6)
status_code = 1 if status in ("ok", "running") else 2
status_msg = "" if status in ("ok", "running") else "Job failed"
main_attrs = {
"agent.trace_name": trace_name,
"agent.service": service_name,
"ci.trace_status": status,
}
if attributes:
main_attrs.update(attributes)
main_span = _encode_span(
trace_id_bytes=trace_id,
span_id_bytes=_gen_span_id(),
parent_span_id_bytes=b"",
name=trace_name,
start_time_unix_nano=start_time,
end_time_unix_nano=end_time,
span_kind=1,
attributes=main_attrs,
status_code=status_code,
status_msg=status_msg,
)
scope_spans = _encode_scope_spans("ci-trace", [main_span])
resource_spans = _encode_resource_spans(service_name, scope_spans)
return _encode_traces_data([resource_spans])
def report_ci_trace(
service_name,
trace_name,
status="ok",
duration_ms=1000,
endpoint=None,
license_key=None,
project=None,
workspace=None,
extra_attributes=None,
):
"""
Report CI Trace data. Returns (success: bool, message: str).
Never raises exceptions; returns False on failure.
"""
try:
endpoint = endpoint or os.getenv("AGENTLOOP_ENDPOINT", DEFAULT_ENDPOINT)
license_key = license_key or os.getenv("AGENTLOOP_LICENSE_KEY", "")
project = project or os.getenv("AGENTLOOP_PROJECT", DEFAULT_PROJECT)
workspace = workspace or os.getenv("AGENTLOOP_WORKSPACE", DEFAULT_WORKSPACE)
if not license_key:
return False, "[Trace] skipped: AGENTLOOP_LICENSE_KEY not configured"
attrs = _get_ci_attributes()
if extra_attributes:
attrs.update(extra_attributes)
payload = build_trace(
service_name=service_name,
trace_name=trace_name,
status=status,
duration_ms=duration_ms,
attributes=attrs,
)
headers = {
"Content-Type": "application/x-protobuf",
"x-arms-license-key": license_key,
"x-arms-project": project,
"x-cms-workspace": workspace,
}
req = urllib.request.Request(endpoint, data=payload, headers=headers, method="POST")
try:
with urllib.request.urlopen(req, timeout=10) as resp:
status_code = resp.status
resp_body = resp.read().decode("utf-8", errors="replace")
except urllib.error.HTTPError as e:
status_code = e.code
resp_body = e.read().decode("utf-8", errors="replace")
if status_code in (200, 202):
return True, (f"[Trace] success: {service_name} / {trace_name} " f"({status}, {duration_ms}ms)")
else:
return False, (f"[Trace] failed: HTTP {status_code} - {resp_body[:200]}")
except Exception as e:
return False, f"[Trace] error: {type(e).__name__}: {str(e)}"
def main():
parser = argparse.ArgumentParser(description="CI AgentLoop Trace Reporter")
parser.add_argument(
"--service",
dest="service_name",
default=os.getenv("TRACE_SERVICE", ""),
help="Service name (also via TRACE_SERVICE env)",
)
parser.add_argument(
"--name",
dest="trace_name",
default=os.getenv("TRACE_NAME", ""),
help="Trace name (also via TRACE_NAME env)",
)
parser.add_argument(
"--status",
default=os.getenv("TRACE_STATUS", "ok"),
choices=["ok", "error", "running"],
help="Status: ok / error / running (default ok)",
)
parser.add_argument(
"--start-time",
dest="start_time",
default=os.getenv("TRACE_START_TIME", ""),
help="Start timestamp (seconds) for duration calculation",
)
parser.add_argument(
"--duration-ms",
dest="duration_ms",
type=int,
default=0,
help="Direct duration in ms; takes precedence over --start-time",
)
parser.add_argument("--attrs", default="", help="Extra attributes (JSON string)")
args = parser.parse_args()
if not args.service_name:
print("[Trace] skipped: no service specified (--service or TRACE_SERVICE)")
sys.exit(0)
duration_ms = args.duration_ms
if duration_ms <= 0 and args.start_time:
try:
start_ts = float(args.start_time)
duration_ms = int((time.time() - start_ts) * 1000)
except (ValueError, TypeError):
duration_ms = 1000
if duration_ms <= 0:
duration_ms = 1000
extra_attrs = {}
if args.attrs:
try:
extra_attrs = json.loads(args.attrs)
except json.JSONDecodeError:
pass
trace_name = args.trace_name
if not trace_name:
wf = _env("GITEA_WORKFLOW") or _env("GITHUB_WORKFLOW") or "CI"
job = _env("GITEA_JOB") or _env("GITHUB_JOB") or "job"
trace_name = f"{wf} / {job}"
success, msg = report_ci_trace(
service_name=args.service_name,
trace_name=trace_name,
status=args.status,
duration_ms=duration_ms,
extra_attributes=extra_attrs,
)
print(msg)
sys.exit(0)
if __name__ == "__main__":
main()
+43
View File
@@ -0,0 +1,43 @@
#!/bin/bash
# PR构建专用:只构建不输出,验证Dockerfile能否正常构建
# 无本地缓存(12个runner不共享,反而添乱),只用ACR远程缓存
set -eu
NO_CACHE_FLAG=""
if [ "$1" = "--no-cache" ]; then
NO_CACHE_FLAG="--no-cache"
shift
fi
DOCKERFILE="$1"
IMAGE_TAG="$2"
CACHE_REF="$3"
shift 3
BUILD_ARGS=""
for arg in "$@"; do
BUILD_ARGS="$BUILD_ARGS --build-arg $arg"
done
BUILDER_NAME="ci-pr-builder-${GITHUB_RUN_ID:-local}"
if ! docker buildx inspect "$BUILDER_NAME" > /dev/null 2>&1; then
docker buildx create --use --name "$BUILDER_NAME" --driver docker-container
else
docker buildx use "$BUILDER_NAME"
fi
docker buildx inspect --bootstrap
echo "=== PR Build: build only, no output, remote cache only ==="
echo "Dockerfile: ${DOCKERFILE}"
echo "Image tag: ${IMAGE_TAG}"
echo ""
docker buildx build \
$NO_CACHE_FLAG \
$BUILD_ARGS \
--cache-from "type=registry,ref=${CACHE_REF}" \
-f "${DOCKERFILE}" \
-t "${IMAGE_TAG}" \
.
echo ""
echo "PR build OK (build only, no output): ${IMAGE_TAG}"
+106
View File
@@ -0,0 +1,106 @@
#!/bin/bash
# 通用Docker镜像构建+推送脚本(local cache为主 + registry cache共享)
# 用法: docker_build_push.sh [--no-cache] <Dockerfile> <image_tag> <cache_ref> [build_arg...]
set -eu
NO_CACHE_FLAG=""
if [ "$1" = "--no-cache" ]; then
NO_CACHE_FLAG="--no-cache"
shift
echo "模式: --no-cache (不使用缓存,全新构建)"
fi
DOCKERFILE="$1"
IMAGE_TAG="$2"
CACHE_REF="$3"
shift 3
BUILD_ARGS=""
for arg in "$@"; do
BUILD_ARGS="$BUILD_ARGS --build-arg $arg"
done
if ! docker buildx inspect ci-builder > /dev/null 2>&1; then
docker buildx create --use --name ci-builder --driver docker-container
echo "Created ci-builder"
else
docker buildx use ci-builder
echo "Using existing ci-builder"
fi
docker buildx inspect --bootstrap
# 从cache_ref中提取缓存名称(如 api-cache:develop -> api-cache-develop
CACHE_NAME=$(echo "$CACHE_REF" | tr '/' '_' | tr ':' '-')
LOCAL_CACHE_DIR="/tmp/buildx-cache/${CACHE_NAME}"
mkdir -p "$LOCAL_CACHE_DIR"
# 缓存源:local优先(带自动修复),registry兜底读写
# 本地缓存损坏时自动清理后重试,避免snapshot not found导致构建全挂
build_with_cache_retry() {
local attempt=1
local max_attempts=2
while [ $attempt -le $max_attempts ]; do
local build_output
local exit_code
set +e
build_output=$(docker buildx build \
$NO_CACHE_FLAG \
$BUILD_ARGS \
--cache-from "type=local,src=${LOCAL_CACHE_DIR}" \
--cache-from "type=registry,ref=${CACHE_REF}" \
--cache-to "type=local,dest=${LOCAL_CACHE_DIR},mode=max" \
--cache-to "type=registry,ref=${CACHE_REF},mode=max,ignore-error=true" \
-f "${DOCKERFILE}" \
-t "${IMAGE_TAG}" \
--push \
. 2>&1)
exit_code=$?
set -e
if [ $exit_code -eq 0 ]; then
echo "$build_output"
return 0
fi
# 检测到缓存损坏类错误,清掉本地缓存重试
if echo "$build_output" | grep -qE "parent snapshot.*not found|snapshot.*does not exist|cache.*corrupt|failed to compute cache key"; then
echo "$build_output"
echo ""
echo "⚠️ Local cache appears corrupted, cleaning up and retrying (attempt $attempt/$max_attempts)..."
rm -rf "${LOCAL_CACHE_DIR}"
mkdir -p "${LOCAL_CACHE_DIR}"
# 清理buildx builder的内部snapshot状态
docker buildx prune -f -a >/dev/null 2>&1 || true
attempt=$((attempt + 1))
else
# 非缓存类错误,直接输出并返回
echo "$build_output"
return $exit_code
fi
done
# 重试完还是失败,不用本地缓存最后试一次(只从registry读)
echo "⚠️ All cached attempts failed, building without local cache..."
docker buildx build \
$NO_CACHE_FLAG \
$BUILD_ARGS \
--cache-from "type=registry,ref=${CACHE_REF}" \
--cache-to "type=local,dest=${LOCAL_CACHE_DIR},mode=max" \
--cache-to "type=registry,ref=${CACHE_REF},mode=max,ignore-error=true" \
-f "${DOCKERFILE}" \
-t "${IMAGE_TAG}" \
--push \
.
}
echo "=== Step 1: Build & push image (local cache + registry cache, with auto-repair) ==="
echo "Local cache: ${LOCAL_CACHE_DIR}"
echo "Registry cache: ${CACHE_REF}"
echo ""
build_with_cache_retry
echo ""
echo "Image pushed: ${IMAGE_TAG}"
echo "Local cache updated"
echo "Registry cache updated (if supported)"
echo ""
echo "Build completed: ${IMAGE_TAG}"
+97
View File
@@ -0,0 +1,97 @@
#!/usr/bin/env bash
#
# CI 健康度看板一键生成脚本
# - 从 Gitea Actions API 拉取数据
# - 生成 HTML 可视化看板
# - 输出文件路径
#
# 用法:
# bash scripts/ci/generate_ci_dashboard.sh [--days 7] [--output ci_dashboard.html]
#
# 环境变量:
# GITEA_TOKEN API Token(必需)
# GITEA_URL Gitea 地址(可选,默认 https://git.xiaoxiajianji.com
# GITEA_REPO 仓库(可选,默认 xiaoxia/xiaoxia-saas
#
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"
# 默认参数
DAYS=7
OUTPUT="ci_dashboard.html"
# 解析参数
while [[ $# -gt 0 ]]; do
case "$1" in
--days)
DAYS="$2"
shift 2
;;
--output|-o)
OUTPUT="$2"
shift 2
;;
--help|-h)
echo "用法: bash scripts/ci/generate_ci_dashboard.sh [--days 7] [--output ci_dashboard.html]"
echo ""
echo "选项:"
echo " --days N 统计最近 N 天 (默认 7)"
echo " --output PATH HTML 输出路径 (默认 ci_dashboard.html)"
echo " --help 显示帮助"
echo ""
echo "环境变量:"
echo " GITEA_TOKEN API Token(必需)"
echo " GITEA_URL Gitea 地址"
echo " GITEA_REPO 仓库"
exit 0
;;
*)
echo "未知参数: $1"
exit 1
;;
esac
done
# 检查 Python
if ! command -v python3 &> /dev/null; then
echo "[ERROR] 未找到 python3,请先安装 Python 3"
exit 1
fi
# 检查 Token
if [[ -z "${GITEA_TOKEN:-}" ]]; then
echo "[ERROR] 请设置 GITEA_TOKEN 环境变量"
exit 1
fi
echo "========================================"
echo " CI 健康度看板生成器"
echo "========================================"
echo ""
echo "统计天数: ${DAYS}"
echo "输出文件: ${OUTPUT}"
echo ""
# 生成 HTML 看板
echo "[INFO] 正在拉取数据并生成看板..."
python3 "${SCRIPT_DIR}/ci_dashboard.py" \
--days "${DAYS}" \
--html \
--html-output "${OUTPUT}"
echo ""
echo "========================================"
echo " ✅ 看板生成完成!"
echo "========================================"
echo ""
echo "文件路径: $(realpath "${OUTPUT}")"
echo ""
# 如果在 macOS 上,尝试打开
if [[ "$(uname)" == "Darwin" ]]; then
echo "[INFO] 正在打开浏览器..."
open "${OUTPUT}"
fi
+48
View File
@@ -0,0 +1,48 @@
#!/bin/bash
# mypyå¢žé‡æ‰«æè„šæœ¬ - CI中调用
# 环境å˜é‡: SCAN_MODE, CHANGED_PY_FILES
set -e
echo "=== Installing mypy ==="
python3 -m pip install -q mypy
mypy --version
echo ""
echo "=== Running mypy type check (hard gate mode) ==="
echo "告警模å¼ï¼Œä¸Í阻断CI"
echo ""
MYPY_COMMON_ARGS="--ignore-missing-imports --no-site-packages --no-strict-optional --explicit-package-bases --exclude tests/|test_|migrations/|alembic/ --no-error-summary --incremental --cache-dir .mypy_cache"
EXIT_CODE=0
if [ "$SCAN_MODE" = "incremental" ] && [ -n "$CHANGED_PY_FILES" ]; then
echo "=== Incremental mypy scan (PR mode) ==="
echo "Changed files: $(echo $CHANGED_PY_FILES | wc -w) files"
MYPY_FILES=""
for f in $CHANGED_PY_FILES; do
case "$f" in
apps/*|packages/*)
MYPY_FILES="$MYPY_FILES $f"
;;
esac
done
if [ -n "$MYPY_FILES" ]; then
echo "Checking: $MYPY_FILES"
mypy $MYPY_FILES $MYPY_COMMON_ARGS 2>&1 | head -80 || EXIT_CODE=$?
else
echo "No mypy-checkable files changed, skipping"
fi
else
echo "=== Full mypy scan ==="
mypy apps/api/app packages $MYPY_COMMON_ARGS 2>&1 | head -60 || EXIT_CODE=$?
fi
echo ""
if [ "$EXIT_CODE" != "0" ]; then
echo "mypy å‘现类型问题(告警模å¼ï¼Œä¸Í阻断)"
echo "建议åŽç»­é€æ­¥ä¿®å¤"
else
echo "mypy 类型检查通过"
fi
+408
View File
@@ -0,0 +1,408 @@
#!/usr/bin/env python3
"""
PR自动扫描器:扫描所有open PR,对CI全绿的进行自动审批/合并
作为短作业模式的兜底机制,每5分钟运行一次
新增:AI审查联动 - AI代码审查发现严重问题时,不自动审批
"""
import argparse
import json
import os
import re
import sys
import time
import urllib.error
import urllib.request
def api_request(token, repo, endpoint, method="GET", data=None):
"""Gitea API请求"""
url = f"https://git.xiaoxiajianji.com/api/v1/repos/{repo}/{endpoint}"
headers = {"Authorization": f"token {token}", "Content-Type": "application/json"}
body = json.dumps(data).encode() if data else None
req = urllib.request.Request(url, data=body, headers=headers, method=method)
# 跳过SSL验证
import ssl
ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE
try:
resp = urllib.request.urlopen(req, context=ctx)
return json.loads(resp.read().decode()), resp.status
except urllib.error.HTTPError as e:
return json.loads(e.read().decode()) if e.read() else {"error": str(e)}, e.code
def get_open_prs(token, repo, base="develop"):
"""获取所有open的PR"""
prs = []
page = 1
while True:
data, code = api_request(token, repo, f"pulls?state=open&base={base}&sort=recentupdate&per_page=50&page={page}")
if code != 200 or not isinstance(data, list) or len(data) == 0:
break
prs.extend(data)
if len(data) < 50:
break
page += 1
return prs
def get_commit_status(token, repo, sha):
"""获取commit的CI状态汇总"""
data, code = api_request(token, repo, f"commits/{sha}/status")
if code != 200:
return {}, "error"
return data, data.get("state", "unknown")
def check_required_contexts(token, repo, sha, contexts):
"""检查指定的context是否都通过"""
data, _ = get_commit_status(token, repo, sha)
statuses = {s["context"]: s["status"] for s in data.get("statuses", [])}
all_success = True
any_pending = False
any_failed = False
for ctx in contexts:
state = statuses.get(ctx, "pending")
if state != "success":
all_success = False
if state == "pending":
any_pending = True
if state in ("failure", "error"):
any_failed = True
return all_success, any_pending, any_failed, statuses
def get_pr_files(token, repo, pr_number):
"""获取PR变更文件"""
files = []
page = 1
while True:
data, code = api_request(token, repo, f"pulls/{pr_number}/files?per_page=300&page={page}")
if code != 200 or not isinstance(data, list) or len(data) == 0:
break
files.extend(data)
if len(data) < 300:
break
page += 1
return [f["filename"] for f in files]
def is_frontend_only(files):
"""判断是否纯前端改动"""
if not files:
return False
frontend_count = sum(1 for f in files if f.startswith("apps/web/"))
backend_count = len(files) - frontend_count
return backend_count == 0 and frontend_count > 0
def has_approval(token, repo, pr_number):
"""检查PR是否已有审批"""
reviews, code = api_request(token, repo, f"pulls/{pr_number}/reviews")
if code != 200:
return False
return any(r.get("state") == "APPROVED" for r in reviews if isinstance(r, dict))
def get_ai_review_result(token, repo, pr_number):
"""
检查AI代码审查结果,返回 (has_critical, review_body)
has_critical: 是否有严重问题(需修改的问题 > 0)
review_body: 最新的AI审查评论文本
"""
# AI审查评论标记
AI_REVIEW_MARKER = "AI_CODE_REVIEW_AUTO_COMMENT"
comments, code = api_request(token, repo, f"issues/{pr_number}/comments")
if code != 200:
return False, None
# 找最新的AI审查评论
ai_comments = [c for c in comments if isinstance(c, dict) and AI_REVIEW_MARKER in c.get("body", "")]
if not ai_comments:
return False, None
# 按时间排序,取最新的
latest = max(ai_comments, key=lambda c: c.get("created_at", ""))
body = latest.get("body", "")
# 解析严重问题数量
# 匹配 "严重问题数量:X 个" 或 "需修改的问题(严重)" 下的列表
critical_count = 0
# 方式1:直接匹配数字
match = re.search(r"严重问题数量[:]\s*(\d+)\s*个", body)
if match:
critical_count = int(match.group(1))
else:
# 方式2:数 "需修改的问题" 章节下的条目数
critical_section = re.search(
r"###\s*[❌⚠️].*?(?:需修改|问题).*?\n(.*?)(?=\n###|\Z)",
body,
re.DOTALL,
)
if critical_section:
section_text = critical_section.group(1)
# 数编号条目 1. 2. 3.
items = re.findall(r"^\d+\.\s+\*\*", section_text, re.MULTILINE)
critical_count = len(items)
has_critical = critical_count > 0
return has_critical, body
def approve_pr(token, repo, pr_number, reason="CI全绿,自动审批通过。"):
"""审批PR"""
# 创建review
data, code = api_request(
token,
repo,
f"pulls/{pr_number}/reviews",
method="POST",
data={"event": "PENDING", "body": reason},
)
if code not in (200, 201):
return False, f"创建review失败: HTTP {code}"
review_id = data.get("id")
if data.get("state") == "APPROVED":
return True, "直接创建APPROVED成功"
if not review_id:
return False, "未获取到review ID"
# submit为APPROVED
data2, code2 = api_request(
token,
repo,
f"pulls/{pr_number}/reviews/{review_id}/events",
method="POST",
data={"event": "APPROVED", "body": reason},
)
if code2 in (200, 201):
return True, "审批提交成功"
else:
# 尝试另一个端点
data3, code3 = api_request(
token,
repo,
f"pulls/{pr_number}/reviews/{review_id}",
method="POST",
data={"event": "APPROVED", "body": reason},
)
if code3 in (200, 201):
return True, "审批提交成功(备用端点)"
return False, f"审批提交失败: HTTP {code2}/{code3}"
def add_pr_label(token, repo, pr_number, label):
"""给PR添加标签"""
data, code = api_request(
token,
repo,
f"issues/{pr_number}/labels",
method="POST",
data={"labels": [label]},
)
return code in (200, 201)
def merge_pr(token, repo, pr_number):
"""合并PRsquash merge"""
# 等待几秒让状态同步
time.sleep(30)
# 检查PR状态
pr_data, code = api_request(token, repo, f"pulls/{pr_number}")
if code != 200:
return False, f"获取PR状态失败: HTTP {code}"
if pr_data.get("state") != "open":
return False, f"PR状态不是open: {pr_data.get('state')}"
# 执行squash merge
data, code = api_request(
token,
repo,
f"pulls/{pr_number}/merge",
method="POST",
data={
"do": "squash",
"merge_title_field": "",
"merge_message_field": "",
"delete_branch_after_merge": True,
"force_merge": False,
},
)
if code == 200:
return True, "合并成功"
elif code == 405:
return False, "合并返回405(门禁未满足或冲突)"
else:
return False, f"合并失败: HTTP {code}"
def main():
parser = argparse.ArgumentParser(description="PR自动扫描器")
parser.add_argument("--token", required=True, help="Gitea API token")
parser.add_argument("--repo", default="xiaoxia/xiaoxia-saas", help="仓库")
parser.add_argument("--base", default="develop", help="目标分支")
parser.add_argument("--approve", action="store_true", help="执行自动审批")
parser.add_argument("--merge", action="store_true", help="执行自动合并")
parser.add_argument("--dry-run", default="false", help="试运行模式")
parser.add_argument("--max-prs", type=int, default=20, help="最多处理的PR数")
parser.add_argument("--skip-ai-review", action="store_true", help="跳过AI审查检查(强制审批)")
args = parser.parse_args()
dry_run = args.dry_run.lower() == "true"
# required contexts(与分支保护一致)
REQUIRED_CONTEXTS_FULL = [
"CI/CD Pipeline / Validate - Code Quality (pull_request)",
"CI/CD Pipeline / Validate - Type Check (mypy) (pull_request)",
"CI/CD Pipeline / Validate - Migration (alembic) (pull_request)",
"CI/CD Pipeline / Frontend Lint (pull_request)",
"CI/CD Pipeline / PR Build API Image (pull_request)",
"CI/CD Pipeline / PR Build Worker Image (pull_request)",
"CI/CD Pipeline / PR Build Web Image (pull_request)",
]
REQUIRED_CONTEXTS_APPROVE = [
"CI/CD Pipeline / Validate - Code Quality (pull_request)",
"CI/CD Pipeline / Validate - Type Check (mypy) (pull_request)",
"CI/CD Pipeline / Validate - Migration (alembic) (pull_request)",
"CI/CD Pipeline / Frontend Lint (pull_request)",
]
FRONTEND_ONLY_CONTEXT = [
"CI/CD Pipeline / Frontend Lint (pull_request)",
]
# 获取所有open PR
print(f"获取 {args.base} 分支的open PR...")
prs = get_open_prs(args.token, args.repo, args.base)
print(f"找到 {len(prs)} 个open PR")
approved_count = 0
merged_count = 0
skipped_count = 0
ai_blocked_count = 0
for pr in prs[: args.max_prs]:
pr_num = pr["number"]
pr_title = pr["title"]
head_sha = pr["head"]["sha"]
base_ref = pr.get("base", {}).get("re", "")
# 跳过draft
if pr.get("draft"):
print(f"\n⏭️ #{pr_num} {pr_title[:50]} - draft,跳过")
skipped_count += 1
continue
# 跳过目标分支不对的
if base_ref != args.base:
skipped_count += 1
continue
print(f"\n--- #{pr_num} {pr_title[:60]} ---")
# 判断是否纯前端
files = get_pr_files(args.token, args.repo, pr_num)
frontend_only = is_frontend_only(files)
if frontend_only:
approve_contexts = FRONTEND_ONLY_CONTEXT
merge_contexts = FRONTEND_ONLY_CONTEXT
print(f" 类型: 纯前端改动 ({len(files)}个文件)")
else:
approve_contexts = REQUIRED_CONTEXTS_APPROVE
merge_contexts = REQUIRED_CONTEXTS_FULL
print(f" 类型: 全栈/后端改动 ({len(files)}个文件)")
# 检查审批用的CI状态
all_ok, pending, failed, _ = check_required_contexts(args.token, args.repo, head_sha, approve_contexts)
# === AI审查检查 ===
ai_has_critical = False
if not args.skip_ai_review and all_ok and not failed and args.approve:
ai_has_critical, ai_body = get_ai_review_result(args.token, args.repo, pr_num)
if ai_has_critical:
print(" ⚠️ AI审查发现严重问题,阻止自动审批")
ai_blocked_count += 1
# 给PR打标签便于人工识别
if not dry_run:
add_pr_label(args.token, args.repo, pr_num, "ai-review/需修改")
# === 自动审批 ===
if args.approve and all_ok and not failed and not ai_has_critical:
if has_approval(args.token, args.repo, pr_num):
print(" ✅ 已有审批,跳过")
else:
if dry_run:
print(" 🎯 [DRY-RUN] 将自动审批")
else:
print(" 🎯 执行自动审批...")
ok, msg = approve_pr(args.token, args.repo, pr_num)
if ok:
print(f" ✅ 审批成功: {msg}")
approved_count += 1
else:
print(f" ❌ 审批失败: {msg}")
elif ai_has_critical:
print(" 🚫 AI审查阻止审批(人工可手动审批覆盖)")
elif failed:
print(" ❌ CI有失败项,跳过审批")
elif pending:
print(" ⏳ CI仍在运行,跳过")
# === 自动合并 ===
if args.merge:
# 检查合并用的CI状态
merge_ok, merge_pending, merge_failed, _ = check_required_contexts(
args.token, args.repo, head_sha, merge_contexts
)
# 检查审批
approved = has_approval(args.token, args.repo, pr_num)
if merge_ok and approved and not merge_failed:
if dry_run:
print(" 🎯 [DRY-RUN] 将自动合并")
else:
print(" 🎯 执行自动合并...")
ok, msg = merge_pr(args.token, args.repo, pr_num)
if ok:
print(f" ✅ 合并成功: {msg}")
merged_count += 1
else:
print(f" ⚠️ 合并失败: {msg}")
elif merge_pending:
print(" ⏳ 合并条件未满足: CI运行中")
elif merge_failed:
print(" ❌ 合并条件未满足: CI有失败")
elif not approved:
print(" ⏳ 合并条件未满足: 无审批")
print("\n=== 扫描结果 ===")
print(f" 处理PR数: {min(len(prs), args.max_prs)}")
print(f" 自动审批: {approved_count}")
print(f" 自动合并: {merged_count}")
print(f" AI审查阻止: {ai_blocked_count}")
print(f" 跳过: {skipped_count}")
print(" 模式: {'DRY-RUN' if dry_run else '正式执行'}")
if __name__ == "__main__":
main()
+54
View File
@@ -0,0 +1,54 @@
#!/usr/bin/env python3
"""生成预览环境PR评论内容"""
import json
import os
import sys
def generate_deploy_comment(pr_number, preview_url):
"""生成部署成功的评论内容"""
return f"""🚀 **预览环境已部署**
| 项目 | 详情 |
|------|------|
| PR号 | #{pr_number} |
| 预览链接 | [{preview_url}]({preview_url}) |
| API环境 | staging |
> 💡 预览环境使用 staging API 数据,请勿在预览环境中操作重要数据。
>
> 🔄 每次提交新代码后预览环境会自动更新。
>
> 🗑️ PR 关闭或合并后,预览环境会自动清理。
"""
def generate_cleanup_comment(pr_number):
"""生成清理完成的评论内容"""
return f"""🗑️ **预览环境已清理**
PR #{pr_number} 已关闭或合并,对应的预览环境已被清理。
> 如有需要,可以重新打开 PR 来重新生成预览环境。
"""
def main():
mode = sys.argv[1] if len(sys.argv) > 1 else "deploy"
pr_number = os.environ.get("PR_NUMBER", "")
preview_url = os.environ.get("PREVIEW_URL", "")
if mode == "deploy":
body = generate_deploy_comment(pr_number, preview_url)
elif mode == "cleanup":
body = generate_cleanup_comment(pr_number)
else:
print(f"Unknown mode: {mode}", file=sys.stderr)
sys.exit(1)
print(json.dumps({"body": body}))
if __name__ == "__main__":
main()
+264
View File
@@ -0,0 +1,264 @@
#!/bin/bash
# ============================================================
# 预览环境服务器初始化脚本
# 用途:在业务服务器上创建预览环境所需的目录和配置
# 使用方式:bash scripts/ci/preview_init_server.sh
# ============================================================
set -eu
PREVIEW_ROOT="/var/www/preview"
NGINX_CONF_PATH="/etc/nginx/conf.d/preview.conf"
DOMAIN="xiaoxiajianji.com"
STAGING_API="https://staging-api.xiaoxiajianji.com"
echo "=========================================="
echo " 预览环境服务器初始化"
echo "=========================================="
echo ""
# 1. 创建预览根目录
echo "[1/4] 创建预览根目录..."
if [ -d "$PREVIEW_ROOT" ]; then
echo " 目录已存在: $PREVIEW_ROOT"
else
mkdir -p "$PREVIEW_ROOT"
echo " 已创建: $PREVIEW_ROOT"
fi
chown -R root:root "$PREVIEW_ROOT"
chmod -R 755 "$PREVIEW_ROOT"
echo ""
# 2. 创建测试页面(验证Nginx配置用)
echo "[2/4] 创建测试页面..."
TEST_DIR="${PREVIEW_ROOT}/pr-demo"
mkdir -p "$TEST_DIR"
cat > "$TEST_DIR/index.html" <<'EOF'
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>预览环境测试页</title>
<style>
body { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
display: flex; align-items: center; justify-content: center;
min-height: 100vh; margin: 0; background: #f5f5f5; }
.card { background: white; padding: 40px; border-radius: 12px;
box-shadow: 0 4px 6px rgba(0,0,0,0.1); text-align: center; max-width: 400px; }
h1 { color: #2d3748; margin-top: 0; }
.success { color: #38a169; font-size: 48px; margin: 20px 0; }
p { color: #718096; line-height: 1.6; }
code { background: #edf2f7; padding: 2px 6px; border-radius: 4px; font-size: 0.9em; }
</style>
</head>
<body>
<div class="card">
<div class="success">✅</div>
<h1>预览环境配置成功!</h1>
<p>如果你能看到这个页面,说明 Nginx 预览环境配置正确。</p>
<p>当前站点通过子域名 <code>pr-demo.preview</code> 路由到 <code>/var/www/preview/pr-demo/</code> 目录。</p>
</div>
</body>
</html>
EOF
echo " 测试页面已创建: $TEST_DIR/index.html"
echo ""
# 3. 检查Nginx是否安装
echo "[3/4] 检查Nginx环境..."
if command -v nginx > /dev/null 2>&1; then
echo " Nginx 已安装: $(nginx -v 2>&1)"
NGINX_INSTALLED=true
else
echo " ⚠️ Nginx 未安装,请先安装 Nginx"
NGINX_INSTALLED=false
fi
echo ""
# 4. 输出Nginx配置建议
echo "[4/4] Nginx 配置建议"
echo ""
echo "----------------------------------------"
echo " 请将以下配置保存到: $NGINX_CONF_PATH"
echo " 或复制到 Nginx 配置目录中"
echo "----------------------------------------"
echo ""
cat <<'NGINX_CONF'
# ============================================================
# 预览环境 Nginx 配置
# 支持 *.preview.xiaoxiajianji.com 通配符子域名
# ============================================================
# 从子域名中提取 PR 号(如 pr-123.preview -> pr-123
map $host $preview_pr {
default "";
~^(?<pr>pr-\d+)\.preview\.xiaoxiajianji\.com$ $pr;
}
# HTTP 服务器(80端口)
server {
listen 80;
server_name *.preview.xiaoxiajianji.com;
# 根目录根据子域名动态映射
root /var/www/preview/$preview_pr;
# 索引文件
index index.html;
# 字符集
charset utf-8;
# 访问日志
access_log /var/log/nginx/preview_access.log;
error_log /var/log/nginx/preview_error.log warn;
# 如果子域名格式不正确,返回404
if ($preview_pr = "") {
return 404;
}
# 如果预览目录不存在,返回404
if (!-d $document_root) {
return 404;
}
# API 反向代理到 staging 环境
location /api/ {
proxy_pass https://staging-api.xiaoxiajianji.com/api/;
proxy_http_version 1.1;
proxy_set_header Host staging-api.xiaoxiajianji.com;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
# 超时设置
proxy_connect_timeout 30s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;
# 缓冲设置
proxy_buffering on;
proxy_buffer_size 4k;
proxy_buffers 8 4k;
# WebSocket 支持(如需要)
# proxy_set_header Upgrade $http_upgrade;
# proxy_set_header Connection "upgrade";
}
# 静态资源缓存
location /assets/ {
expires 7d;
add_header Cache-Control "public, max-age=604800, immutable";
try_files $uri =404;
}
# SPA 路由支持
location / {
try_files $uri $uri/ /index.html;
}
# 安全相关响应头
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-Content-Type-Options "nosniff" always;
add_header X-XSS-Protection "1; mode=block" always;
# 禁止隐藏文件访问
location ~ /\. {
deny all;
access_log off;
log_not_found off;
}
}
# HTTPS 服务器(443端口)
# 注意:需要先配置 SSL 证书
# 建议使用 certbot 或手动配置证书
#
# server {
# listen 443 ssl http2;
# server_name *.preview.xiaoxiajianji.com;
#
# # SSL 证书配置(请替换为实际证书路径)
# ssl_certificate /path/to/fullchain.pem;
# ssl_certificate_key /path/to/privkey.pem;
#
# # SSL 安全配置
# ssl_protocols TLSv1.2 TLSv1.3;
# ssl_ciphers HIGH:!aNULL:!MD5;
# ssl_prefer_server_ciphers on;
# ssl_session_cache shared:SSL:10m;
# ssl_session_timeout 10m;
#
# # 其余配置与 HTTP 相同
# root /var/www/preview/$preview_pr;
# index index.html;
# charset utf-8;
#
# access_log /var/log/nginx/preview_ssl_access.log;
# error_log /var/log/nginx/preview_ssl_error.log warn;
#
# if ($preview_pr = "") {
# return 404;
# }
#
# if (!-d $document_root) {
# return 404;
# }
#
# location /api/ {
# proxy_pass https://staging-api.xiaoxiajianji.com/api/;
# proxy_http_version 1.1;
# proxy_set_header Host staging-api.xiaoxiajianji.com;
# proxy_set_header X-Real-IP $remote_addr;
# proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# proxy_set_header X-Forwarded-Proto $scheme;
# proxy_set_header X-Forwarded-Host $host;
# proxy_connect_timeout 30s;
# proxy_send_timeout 60s;
# proxy_read_timeout 60s;
# }
#
# location /assets/ {
# expires 7d;
# add_header Cache-Control "public, max-age=604800, immutable";
# try_files $uri =404;
# }
#
# location / {
# try_files $uri $uri/ /index.html;
# }
#
# add_header X-Frame-Options "SAMEORIGIN" always;
# add_header X-Content-Type-Options "nosniff" always;
# add_header X-XSS-Protection "1; mode=block" always;
#
# location ~ /\. {
# deny all;
# access_log off;
# log_not_found off;
# }
# }
NGINX_CONF
echo ""
echo "----------------------------------------"
echo " 配置完成后的操作步骤:"
echo "----------------------------------------"
echo ""
echo "1. 将上面的 Nginx 配置保存到合适的位置(如 /etc/nginx/conf.d/preview.conf"
echo "2. 测试配置: nginx -t"
echo "3. 重载配置: nginx -s reload"
echo "4. 配置 DNS 解析: 将 *.preview.xiaoxiajianji.com 指向服务器 IP"
echo "5. 配置 SSL 证书(推荐使用 Let's Encrypt 通配符证书)"
echo ""
echo "测试方式:"
echo " 访问 http://pr-demo.preview.xiaoxiajianji.com 验证配置"
echo ""
echo "=========================================="
echo " 初始化完成"
echo "=========================================="

Some files were not shown because too many files have changed in this diff Show More