Compare commits

..

1 Commits

Author SHA1 Message Date
xiaoxia 8437d4060b feat(products): 成片库视频内联播放,去掉弹窗 + 自适应比例
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 1s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 3m35s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 4m7s
CI/CD Pipeline / Validate - Type Check (mypy) (pull_request) Successful in 4m5s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 4m40s
CI/CD Pipeline / Validate - Migration (alembic) (pull_request) Successful in 5m18s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 6m13s
AI Code Review / AI Code Review (pull_request) Successful in 10m1s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Successful in 2m9s
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 2m16s
CI/CD Pipeline / Validate - Code Quality (pull_request) Successful in 10m52s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 3m7s
CI/CD Pipeline / CI Gate (pull_request) Successful in 43s
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 4m1s
ACR Cleanup / ACR Image Cleanup (pull_request_target) Waiting to run
CI/CD Pipeline / Production Browser E2E (pull_request) Failing after 382h33m48s
CI/CD Pipeline / Canary Release to Production (pull_request) Failing after 382h33m49s
CI/CD Pipeline / Build Production Worker Image (pull_request) Failing after 382h33m50s
CI/CD Pipeline / Build Production Web Image (pull_request) Failing after 382h33m50s
CI/CD Pipeline / Build Production API Image (pull_request) Failing after 382h33m50s
CI/CD Pipeline / Integration Tests (pull_request) Failing after 382h33m55s
CI/CD Pipeline / PR Build Worker Image (pull_request) Failing after 382h36m49s
CI/CD Pipeline / Unit Tests (pull_request) Failing after 382h37m0s
CI/CD Pipeline / ACR Image Cleanup (pull_request) Failing after 382h44m35s
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Failing after 382h44m43s
CI/CD Pipeline / Staging API Integration Tests (pull_request) Failing after 382h44m37s
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Failing after 382h44m45s
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 382h44m39s
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Failing after 382h44m47s
CI/CD Pipeline / Build Staging Worker Image (pull_request) Failing after 382h44m49s
CI/CD Pipeline / Build Staging API Image (pull_request) Failing after 382h44m49s
CI/CD Pipeline / Check push changed paths (pull_request) Failing after 382h44m51s
CI/CD Pipeline / Deploy Production (pull_request) Failing after 383h8m1s
CI/CD Pipeline / PR Build API Image (pull_request) Failing after 383h11m3s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Failing after 383h18m53s
CI/CD Pipeline / Build Staging Web Image (pull_request) Failing after 383h19m1s
- ProductCard: 缩略图区域改为 <video> 内联播放,点击播放按钮原生 controls
- 视频比例通过 onLoadedMetadata 动态获取,获取不到时默认 16:9
- ProductLibrary: 移除 playingProduct 状态和 VideoPlayer 弹窗组件
- products.css: .xx-product-thumb 去掉固定 9:16 aspect-ratio,
  新增 .xx-product-thumb-video 样式(object-fit: contain)
- 卡片网格保持 4 列,卡片高度自适应视频内容
2026-08-31 16:40:15 +08:00
124 changed files with 853 additions and 7489 deletions
-1
View File
@@ -1 +0,0 @@
CI re-trigger after runner add-host/DNS fix. This file is harmless and not referenced.
-105
View File
@@ -1,105 +0,0 @@
name: CI Base Image Build
on:
push:
branches:
- develop
- main
paths:
- 'requirements-base.txt'
- 'requirements-dev.txt'
- 'infra/docker/ci.Dockerfile'
workflow_dispatch:
inputs:
reason:
description: "触发原因"
required: false
default: "手动触发 - ci-base 镜像重建"
concurrency:
group: ci-base-image-build
cancel-in-progress: false
jobs:
build-ci-base:
name: Build CI Base Image
runs-on: runtime-builder
timeout-minutes: 60
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sH "Authorization: token $GITHUB_TOKEN" \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" \
| bash
- name: Docker login to Gitea Registry
shell: sh
env:
GITEA_REGISTRY_USER: xiaoxia
GITEA_REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
set -eu
for i in 1 2 3; do
echo "=== Docker login 尝试 $i/3 ==="
if docker login git.xiaoxiajianji.com -u "${GITEA_REGISTRY_USER}" -p "${GITEA_REGISTRY_TOKEN}"; then
echo "✅ Docker login successful"
break
fi
echo "❌ Docker login 失败(尝试 $i/3),5s 后重试..."
sleep 5
done
- name: Build and push CI base image
shell: sh
run: |
set -eu
IMAGE="git.xiaoxiajianji.com/xiaoxia/xiaoxia-saas/ci-base"
VERSION_TAG="deps-$(date +%Y%m%d-%H%M)-${GITHUB_SHA::8}"
echo "=== Building CI base image (tags: latest, ${VERSION_TAG}) ==="
docker build --progress=plain \
-f infra/docker/ci.Dockerfile \
-t "${IMAGE}:latest" \
-t "${IMAGE}:${VERSION_TAG}" \
.
echo "✅ Image built successfully"
echo "=== Pushing ${VERSION_TAG} ==="
docker push "${IMAGE}:${VERSION_TAG}"
echo "=== Pushing latest ==="
docker push "${IMAGE}:latest"
echo "✅ Pushed to Gitea Registry"
- name: Verify image
shell: sh
run: |
set -eu
IMAGE="git.xiaoxiajianji.com/xiaoxia/xiaoxia-saas/ci-base:latest"
echo "=== Verifying pinned deps in fresh image ==="
docker run --rm "${IMAGE}" /opt/xiaoxia-ci-venv/bin/python -c \
"import httpcore, h2, numpy, httpx; print('VERSIONS:', httpcore.__version__, h2.__version__, numpy.__version__, httpx.__version__)"
- name: Notify result
if: always()
continue-on-error: true
shell: sh
env:
CI_NOTIFY_WEBHOOK: ${{ secrets.CI_NOTIFY_WEBHOOK }}
run: |
set +e
if [ "${{ job.status }}" = "success" ]; then
NOTIFY_MODE=success JOB_NAME="CI Base Image Build" python3 scripts/ci_notify.py
else
NOTIFY_MODE=failure JOB_NAME="CI Base Image Build" python3 scripts/ci_notify.py
fi
- name: Cleanup
if: always()
shell: sh
run: |
IMAGE="git.xiaoxiajianji.com/xiaoxia/xiaoxia-saas/ci-base"
docker rmi "${IMAGE}:latest" 2>/dev/null || true
echo "Cleanup done"
-51
View File
@@ -1,51 +0,0 @@
name: CI Canary Check
on:
schedule:
- cron: '*/30 * * * *'
workflow_dispatch:
jobs:
canary:
runs-on: ci-l2
timeout-minutes: 10
steps:
- name: Canary (runner -> docker -> network -> gitea)
run: |
set -e
echo "== runner/container basic =="
date; hostname; whoami
echo "== gitea api reachability =="
code=$(curl -s -o /tmp/v.json -w '%{http_code}' -m 15 "$GITHUB_API_URL/version")
echo "gitea api http_code=$code"
[ "$code" = "200" ] || { echo "::error::Gitea API unreachable, http_code=$code"; exit 1; }
cat /tmp/v.json; echo
echo "== external egress =="
ext=$(curl -s -o /dev/null -w '%{http_code}' -m 15 https://www.baidu.com || echo 000)
echo "external http_code=$ext"
echo "== gitea domain resolves NOT to loopback =="
set -o pipefail
ip=$(getent hosts git.xiaoxiajianji.com | awk '{print $1}' | head -1)
echo "git.xiaoxiajianji.com -> $ip"
if [ -z "$ip" ]; then
echo "::error::DNS resolution failed, git.xiaoxiajianji.com unresolvable"; exit 1
fi
if [ "$ip" = "127.0.0.1" ] || [ "$ip" = "::1" ]; then
echo "::error::Gitea domain resolves to loopback inside job container (hosts/DNS leak)"; exit 1
fi
echo "CANARY OK"
- name: Notify failure
if: failure()
env:
CI_NOTIFY_WEBHOOK: ${{ secrets.CI_NOTIFY_WEBHOOK }}
run: |
set +e
if [ -n "$CI_NOTIFY_WEBHOOK" ]; then
MSG="🚨 CI 金丝雀失败:runner->docker->网络->Gitea 链路异常,时间 $(date '+%Y-%m-%d %H:%M:%S'),请立即检查构建服务器"
python3 - "$CI_NOTIFY_WEBHOOK" "$MSG" <<'PY'
import json,sys,urllib.request
hook,msg=sys.argv[1],sys.argv[2]
data=json.dumps({"msg_type":"text","content":{"text":msg}}).encode()
urllib.request.urlopen(urllib.request.Request(hook,data=data,headers={"Content-Type":"application/json"}),timeout=10)
PY
fi
exit 0
+161 -212
View File
@@ -81,6 +81,12 @@ jobs:
skip_backend: ${{ steps.check.outputs.skip_backend }}
skip_frontend: ${{ steps.check.outputs.skip_frontend }}
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sfH "Authorization: token $GITHUB_TOKEN" -o /tmp/_ci_checkout.sh "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" && bash /tmp/_ci_checkout.sh
- name: Check changed files
id: check
shell: bash
@@ -119,14 +125,14 @@ jobs:
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
curl -sfH "Authorization: token ${GITHUB_TOKEN:-$GITEA_TOKEN}" -o /tmp/_ci_trace.py "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/ci_trace_report.py?ref=${GITHUB_SHA}" 2>/dev/null && python3 /tmp/_ci_trace.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
validate-style:
validate-code-quality:
needs: dedupe-check
if: always() && needs.dedupe-check.outputs.skip_tests != 'true'
name: Validate - Style
name: Validate - Code Quality
runs-on: ci-l2
timeout-minutes: 6
timeout-minutes: 8
env:
PIP_CACHE_DIR: /root/.cache/pip
PIP_NO_CACHE_DIR: ''
@@ -150,12 +156,11 @@ jobs:
run: bash scripts/ci/step_timer_start.sh
- name: Cache pip dependencies
uses: actions/cache@v4
continue-on-error: true
with:
path: /root/.cache/pip
key: ${{ runner.os }}-pip-style-${{ hashFiles('requirements*.txt') }}
key: ${{ runner.os }}-pip-codequality-${{ hashFiles('requirements*.txt') }}
restore-keys: |
${{ runner.os }}-pip-style-
${{ runner.os }}-pip-codequality-
${{ runner.os }}-pip-
- name: Install dependencies
shell: sh
@@ -179,9 +184,17 @@ jobs:
[ $i -eq 3 ] && exit 1
sleep 5
done
- name: Run style checks
for i in 1 2 3; do
python3 -m pip install --no-binary :all: black==26.5.1 isort==8.0.1 && break
echo "pip install black/isort 失败,重试 $i/3..."
[ $i -eq 3 ] && exit 1
sleep 5
done
- name: Run code quality and security checks
shell: bash
run: bash scripts/ci/validate_style.sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: bash scripts/ci/validate_code_quality.sh
- name: Auto-fix formatting (black + isort)
if: failure()
shell: sh
@@ -197,7 +210,7 @@ jobs:
CI_WEBHOOK_URL: ${{ secrets.CI_WEBHOOK_URL }}
run: |
set +e
FAILED_JOB="Validate - Style" python3 scripts/ci_notify_failure.py
FAILED_JOB="Validate - Code Quality" python3 scripts/ci_notify_failure.py
- name: Job duration summary
if: always()
shell: sh
@@ -210,7 +223,7 @@ jobs:
CI_NOTIFY_WEBHOOK: ${{ secrets.CI_NOTIFY_WEBHOOK }}
run: |
set +e
NOTIFY_MODE=failure JOB_NAME="Validate - Style" python3 scripts/ci_notify.py
NOTIFY_MODE=failure JOB_NAME="Validate - Code Quality" python3 scripts/ci_notify.py
- name: Report CI trace
if: always()
shell: sh
@@ -223,16 +236,12 @@ jobs:
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
validate-security:
validate-type-check:
needs: dedupe-check
if: always() && needs.dedupe-check.outputs.skip_tests != 'true'
name: Validate - Security
name: Validate - Type Check (mypy)
runs-on: ci-l2
timeout-minutes: 8
env:
PIP_CACHE_DIR: /root/.cache/pip
PIP_NO_CACHE_DIR: ''
permissions:
contents: read
steps:
@@ -251,117 +260,6 @@ jobs:
- name: Record job start time
shell: sh
run: bash scripts/ci/step_timer_start.sh
- name: Cache pip dependencies
uses: actions/cache@v4
continue-on-error: true
with:
path: /root/.cache/pip
key: ${{ runner.os }}-pip-security-${{ hashFiles('requirements*.txt') }}
restore-keys: |
${{ runner.os }}-pip-security-
${{ runner.os }}-pip-
- name: Install dependencies
shell: sh
run: |
set -eu
for i in 1 2 3; do
python3 -m pip install -q -r requirements-base.txt && break
echo "pip install requirements-base.txt 失败,重试 $i/3..."
[ $i -eq 3 ] && exit 1
sleep 5
done
for i in 1 2 3; do
python3 -m pip install -q -r requirements.txt && break
echo "pip install requirements.txt 失败,重试 $i/3..."
[ $i -eq 3 ] && exit 1
sleep 5
done
for i in 1 2 3; do
python3 -m pip install -q -r requirements-dev.txt && break
echo "pip install requirements-dev.txt 失败,重试 $i/3..."
[ $i -eq 3 ] && exit 1
sleep 5
done
- name: Run security checks
continue-on-error: true # Security scan is advisory; runner failure must not block deploy
shell: bash
env:
GITHUB_TOKEN: ${{ github.token }}
run: bash scripts/ci/validate_security.sh
- name: CI failure notification
if: failure()
shell: sh
env:
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
CI_WEBHOOK_URL: ${{ secrets.CI_WEBHOOK_URL }}
run: |
set +e
FAILED_JOB="Validate - Security" python3 scripts/ci_notify_failure.py
- name: Job duration summary
if: always()
shell: sh
run: bash scripts/ci/step_timer_end.sh
- name: Notify on failure
continue-on-error: true
if: failure()
shell: sh
env:
CI_NOTIFY_WEBHOOK: ${{ secrets.CI_NOTIFY_WEBHOOK }}
run: |
set +e
NOTIFY_MODE=failure JOB_NAME="Validate - Security" python3 scripts/ci_notify.py
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
validate-python:
needs: dedupe-check
if: always() && needs.dedupe-check.outputs.skip_tests != 'true'
name: Validate - Python (mypy + alembic)
runs-on: ci-l2
timeout-minutes: 10
env:
PIP_CACHE_DIR: /root/.cache/pip
PIP_NO_CACHE_DIR: ''
DATABASE_URL: postgresql+psycopg://postgres:postgres@host.docker.internal:5432/xiaoxia_saas
USE_IN_MEMORY_DB: 'false'
CI_USE_SHARED_PG: 'false'
permissions:
contents: read
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sfH "Authorization: token $GITHUB_TOKEN" -o /tmp/_ci_checkout.sh "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" && bash /tmp/_ci_checkout.sh
- name: Zombie run selfcheck
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
GITHUB_RUN_ID: ${{ github.run_id }}
run: bash scripts/ci/ci_run_selfcheck.sh
- name: Record job start time
shell: sh
run: bash scripts/ci/step_timer_start.sh
- name: Cache pip dependencies
uses: actions/cache@v4
continue-on-error: true
with:
path: /root/.cache/pip
key: ${{ runner.os }}-pip-python-${{ hashFiles('requirements*.txt') }}
restore-keys: |
${{ runner.os }}-pip-python-
${{ runner.os }}-pip-
- name: Install dependencies
shell: sh
run: |
@@ -387,9 +285,6 @@ jobs:
- name: Run mypy type check
shell: bash
run: bash scripts/ci/validate_mypy.sh
- name: Run alembic migration validation
shell: bash
run: bash scripts/ci/validate_migration.sh
- name: CI failure notification
if: failure()
shell: sh
@@ -398,7 +293,7 @@ jobs:
CI_WEBHOOK_URL: ${{ secrets.CI_WEBHOOK_URL }}
run: |
set +e
FAILED_JOB="Validate - Python (mypy + alembic)" python3 scripts/ci_notify_failure.py
FAILED_JOB="Validate - Type Check (mypy)" python3 scripts/ci_notify_failure.py
- name: Job duration summary
if: always()
shell: sh
@@ -411,7 +306,7 @@ jobs:
CI_NOTIFY_WEBHOOK: ${{ secrets.CI_NOTIFY_WEBHOOK }}
run: |
set +e
NOTIFY_MODE=failure JOB_NAME="Validate - Python (mypy + alembic)" python3 scripts/ci_notify.py
NOTIFY_MODE=failure JOB_NAME="Validate - Type Check (mypy)" python3 scripts/ci_notify.py
- name: Report CI trace
if: always()
shell: sh
@@ -424,6 +319,92 @@ jobs:
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
validate-migration:
needs: dedupe-check
if: always() && needs.dedupe-check.outputs.skip_tests != 'true'
name: Validate - Migration (alembic)
runs-on: ci-l2
timeout-minutes: 8
permissions:
contents: read
env:
DATABASE_URL: postgresql+psycopg://postgres:postgres@host.docker.internal:5432/xiaoxia_saas
USE_IN_MEMORY_DB: 'false'
CI_USE_SHARED_PG: 'true'
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sfH "Authorization: token $GITHUB_TOKEN" -o /tmp/_ci_checkout.sh "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" && bash /tmp/_ci_checkout.sh
- name: Zombie run selfcheck
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
GITHUB_RUN_ID: ${{ github.run_id }}
run: bash scripts/ci/ci_run_selfcheck.sh
- name: Record job start time
shell: sh
run: bash scripts/ci/step_timer_start.sh
- name: Install dependencies
shell: sh
run: |
set -eu
for i in 1 2 3; do
python3 -m pip install -q -r requirements-base.txt && break
echo "pip install requirements-base.txt 失败,重试 $i/3..."
[ $i -eq 3 ] && exit 1
sleep 5
done
for i in 1 2 3; do
python3 -m pip install -q -r requirements.txt && break
echo "pip install requirements.txt 失败,重试 $i/3..."
[ $i -eq 3 ] && exit 1
sleep 5
done
for i in 1 2 3; do
python3 -m pip install -q -r requirements-dev.txt && break
echo "pip install requirements-dev.txt 失败,重试 $i/3..."
[ $i -eq 3 ] && exit 1
sleep 5
done
- name: Run alembic migration validation
shell: bash
run: bash scripts/ci/validate_migration.sh
- name: CI failure notification
if: failure()
shell: sh
env:
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
CI_WEBHOOK_URL: ${{ secrets.CI_WEBHOOK_URL }}
run: |
set +e
FAILED_JOB="Validate - Migration (alembic)" python3 scripts/ci_notify_failure.py
- name: Job duration summary
if: always()
shell: sh
run: bash scripts/ci/step_timer_end.sh
- name: Notify on failure
continue-on-error: true
if: failure()
shell: sh
env:
CI_NOTIFY_WEBHOOK: ${{ secrets.CI_NOTIFY_WEBHOOK }}
run: |
set +e
NOTIFY_MODE=failure JOB_NAME="Validate - Migration (alembic)" python3 scripts/ci_notify.py
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
unit-tests:
needs: [check-frontend-only, dedupe-check]
@@ -460,7 +441,6 @@ jobs:
run: bash scripts/ci/step_install_ffmpeg.sh
- name: Cache pip dependencies
uses: actions/cache@v4
continue-on-error: true
with:
path: /root/.cache/pip
key: ${{ runner.os }}-pip-unittests-${{ hashFiles('requirements*.txt') }}
@@ -515,10 +495,13 @@ jobs:
needs:
- check-frontend-only
- dedupe-check
- validate-code-quality
- validate-type-check
- validate-migration
env:
DATABASE_URL: postgresql+psycopg://postgres:postgres@host.docker.internal:5432/xiaoxia_saas
USE_IN_MEMORY_DB: 'false'
CI_USE_SHARED_PG: 'false'
CI_USE_SHARED_PG: 'true'
OSS_ACCESS_KEY_ID: placeholder
OSS_ACCESS_KEY_SECRET: placeholder
OSS_BUCKET_NAME: xiaoxia-autocut
@@ -669,7 +652,6 @@ jobs:
run: bash scripts/ci/step_timer_start.sh
- name: Cache npm dependencies
uses: actions/cache@v4
continue-on-error: true
with:
path: /root/.npm
key: ${{ runner.os }}-npm-vitest-${{ hashFiles('apps/web/package-lock.json') }}
@@ -876,13 +858,19 @@ jobs:
skip_backend: ${{ steps.check.outputs.skip_backend }}
skip_frontend: ${{ steps.check.outputs.skip_frontend }}
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sfH "Authorization: token $GITHUB_TOKEN" -o /tmp/_ci_checkout.sh "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" && bash /tmp/_ci_checkout.sh
- name: Check changed paths
id: check
shell: bash
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sfH "Authorization: token $GITHUB_TOKEN" -o /tmp/_ci_push_paths.sh "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/ci_push_paths.sh?ref=${GITHUB_SHA}" && bash /tmp/_ci_push_paths.sh
bash scripts/ci/ci_push_paths.sh
- name: Report CI trace
if: always()
shell: sh
@@ -893,7 +881,7 @@ jobs:
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
curl -sfH "Authorization: token ${GITHUB_TOKEN:-$GITEA_TOKEN}" -o /tmp/_ci_trace.py "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/ci_trace_report.py?ref=${GITHUB_SHA}" 2>/dev/null && python3 /tmp/_ci_trace.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
build-staging:
name: Build Staging ${{ matrix.service_display }} Image
@@ -1151,7 +1139,9 @@ jobs:
- check-push-paths
- build-staging
- retag-staging-skipped
if: (!cancelled()) && github.event_name == 'push' && (github.ref_name == 'main' || github.ref_name == 'develop')
# 显式 success() 状态检查:上游 build/retag 被路径过滤 if 跳过(skipped)时不阻塞本 job;
# 上游真正失败时仍然阻断(act_runner 对无状态函数的 if 隐式包 success(),纯 skipped 也会连带跳过)
if: success() && github.event_name == 'push' && (github.ref_name == 'main' || github.ref_name == 'develop')
steps:
- name: Checkout code
shell: sh
@@ -1170,31 +1160,6 @@ jobs:
run: |
set +e
NOTIFY_MODE=start JOB_NAME="Deploy Staging" python3 scripts/ci_notify.py
- name: Render .env from template
shell: sh
env:
STAGING_DATABASE_URL: ${{ secrets.STAGING_DATABASE_URL }}
STAGING_REDIS_URL: ${{ secrets.STAGING_REDIS_URL }}
STAGING_CELERY_BROKER_URL: ${{ secrets.STAGING_CELERY_BROKER_URL }}
STAGING_CELERY_RESULT_BACKEND: ${{ secrets.STAGING_CELERY_RESULT_BACKEND }}
STAGING_JWT_SECRET_KEY: ${{ secrets.STAGING_JWT_SECRET_KEY }}
STAGING_MINIO_ENDPOINT: ${{ secrets.STAGING_MINIO_ENDPOINT }}
STAGING_MINIO_ACCESS_KEY: ${{ secrets.STAGING_MINIO_ACCESS_KEY }}
STAGING_MINIO_SECRET_KEY: ${{ secrets.STAGING_MINIO_SECRET_KEY }}
STAGING_MINIO_BUCKET: ${{ secrets.STAGING_MINIO_BUCKET }}
OSS_ACCESS_KEY_ID: ${{ secrets.OSS_ACCESS_KEY_ID }}
OSS_ACCESS_KEY_SECRET: ${{ secrets.OSS_ACCESS_KEY_SECRET }}
COSYVOICE_API_KEY: ${{ secrets.COSYVOICE_API_KEY }}
DASHSCOPE_API_KEY: ${{ secrets.DASHSCOPE_API_KEY }}
MEDIAKIT_API_KEY: ${{ secrets.MEDIAKIT_API_KEY }}
run: |
set -eu
echo "Rendering .env from template + secrets..."
bash scripts/render_env.sh staging
echo "✅ .env rendered (file contains secrets, not printed to log)"
# 验证文件存在且非空
test -s .env.rendered
echo "✅ .env.rendered validated ($(wc -l < .env.rendered) lines)"
- name: Docker login to Registry
shell: sh
env:
@@ -1267,31 +1232,9 @@ jobs:
ssh -p "$staging_port" -i "$key_path" -o StrictHostKeyChecking=no "${staging_user}@${staging_host}" "echo SSH_CONNECTION_OK && hostname"
echo "SSH connection verified"
# 配置 Diff 检查:下载服务器当前 .env,对比渲染结果,检测漂移
echo "Running config diff check..."
scp -P "$staging_port" -i "$key_path" -o StrictHostKeyChecking=no \
"${staging_user}@${staging_host}:/var/lib/xiaoxia-saas-staging/.env" .env.current 2>/dev/null \
|| touch .env.current # 首次部署时文件不存在,创建空文件
bash scripts/config_diff_check.sh .env.rendered .env.current
rm -f .env.current
echo "Config diff check done"
# 上传渲染后的 .env 到服务器(替代服务器上旧的 .env)
echo "Uploading rendered .env to staging server..."
# 备份旧 .env
ssh -p "$staging_port" -i "$key_path" -o StrictHostKeyChecking=no "${staging_user}@${staging_host}" \
"cp -f /var/lib/xiaoxia-saas-staging/.env /var/lib/xiaoxia-saas-staging/.env.bak.\$(date +%Y%m%d%H%M%S) 2>/dev/null || true"
# 上传新 .env
scp -P "$staging_port" -i "$key_path" -o StrictHostKeyChecking=no .env.rendered \
"${staging_user}@${staging_host}:/var/lib/xiaoxia-saas-staging/.env"
echo "✅ .env uploaded to staging server"
# 通过环境变量传递凭证,避免命令行引号转义问题
cat scripts/ci_staging_deploy.sh | ssh -p "$staging_port" -i "$key_path" -o StrictHostKeyChecking=no "${staging_user}@${staging_host}" "IMAGE_TAG=${GITHUB_SHA} ACR_USERNAME=${ACR_USERNAME} ACR_PASSWORD=${ACR_PASSWORD} sh"
# 清理 CI runner 上的渲染文件
rm -f .env.rendered
- name: Staging health check + auto rollback
if: success()
shell: sh
@@ -1456,13 +1399,12 @@ jobs:
runs-on: runtime-builder
timeout-minutes: ${{ matrix.timeout }}
needs:
- validate-style
- validate-security
- validate-python
- validate-code-quality
- validate-type-check
- unit-tests
- frontend-lint
- frontend-unit-test
if: github.event_name == 'push' && github.ref_name == 'main' && !failure() && !cancelled()
if: startsWith(github.ref, 'refs/tags/v') || (github.event_name == 'push' && github.ref_name == 'main')
strategy:
fail-fast: false
matrix:
@@ -1546,7 +1488,11 @@ jobs:
set -eu
REGISTRY="xiaoxia-registry.cn-hangzhou.cr.aliyuncs.com/xiaoxiakeji"
# 根据ref类型设置镜像标签:tag用版本号,分支用分支名+sha
TAG_NAME="${GITHUB_SHA}"
if [[ "$GITHUB_REF" == refs/tags/* ]]; then
TAG_NAME="${GITHUB_REF_NAME}"
else
TAG_NAME="${GITHUB_REF_NAME}-${GITHUB_SHA::8}"
fi
IMAGE_TAG="${REGISTRY}/${{ matrix.image_name }}:${TAG_NAME}"
CACHE_REF="${REGISTRY}/${{ matrix.cache_name }}:main"
@@ -1608,7 +1554,7 @@ jobs:
concurrency:
group: deploy-production-${{ gitea.ref }}
cancel-in-progress: false
if: github.event_name == 'push' && github.ref_name == 'main'
if: startsWith(github.ref, 'refs/tags/v')
needs:
- build-production
steps:
@@ -1681,7 +1627,7 @@ jobs:
echo "SSH connection verified"
# 通过环境变量传递凭证,避免命令行引号转义问题
cat scripts/ci_production_deploy.sh | ssh -p "$production_port" -i "$key_path" -o StrictHostKeyChecking=no "${production_user}@${production_host}" "IMAGE_TAG=${GITHUB_SHA} ACR_USERNAME=${ACR_USERNAME} ACR_PASSWORD=${ACR_PASSWORD} sh"
cat scripts/ci_production_deploy.sh | ssh -p "$production_port" -i "$key_path" -o StrictHostKeyChecking=no "${production_user}@${production_host}" "IMAGE_TAG=${GITHUB_REF_NAME} ACR_USERNAME=${ACR_USERNAME} ACR_PASSWORD=${ACR_PASSWORD} sh"
- name: Production health check + auto rollback
if: success()
@@ -1740,7 +1686,7 @@ jobs:
name: Production Browser E2E
runs-on: runtime-builder
timeout-minutes: 15
# if: removed - runs after deploy-production succeeds
if: startsWith(github.ref, 'refs/tags/v')
needs: deploy-production
steps:
- name: Checkout code
@@ -1940,9 +1886,9 @@ jobs:
if: always() && github.event_name == 'pull_request'
needs:
- check-frontend-only
- validate-style
- validate-security
- validate-python
- validate-code-quality
- validate-type-check
- validate-migration
- unit-tests
- integration-tests
- frontend-lint
@@ -1950,6 +1896,14 @@ jobs:
- build-pr
timeout-minutes: 3
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sfH "Authorization: token $GITHUB_TOKEN" -o /tmp/_ci_checkout.sh \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" && bash /tmp/_ci_checkout.sh
- name: Evaluate CI Gate
id: gate
shell: bash
@@ -1958,9 +1912,9 @@ jobs:
PR_NUMBER: ${{ github.event.pull_request.number }}
PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
RESULT_CHECK_FRONTEND: ${{ needs.check-frontend-only.result }}
RESULT_STYLE: ${{ needs.validate-style.result }}
RESULT_SECURITY: ${{ needs.validate-security.result }}
RESULT_PYTHON: ${{ needs.validate-python.result }}
RESULT_CODE_QUALITY: ${{ needs.validate-code-quality.result }}
RESULT_TYPE_CHECK: ${{ needs.validate-type-check.result }}
RESULT_MIGRATION: ${{ needs.validate-migration.result }}
RESULT_UNIT_TESTS: ${{ needs.unit-tests.result }}
RESULT_INTEGRATION: ${{ needs.integration-tests.result }}
RESULT_FRONTEND_LINT: ${{ needs.frontend-lint.result }}
@@ -1972,9 +1926,9 @@ jobs:
echo ""
echo "各job结果:"
echo " check-frontend-only: $RESULT_CHECK_FRONTEND"
echo " validate-style: $RESULT_STYLE"
echo " validate-security: $RESULT_SECURITY"
echo " validate-python: $RESULT_PYTHON"
echo " validate-code-quality: $RESULT_CODE_QUALITY"
echo " validate-type-check: $RESULT_TYPE_CHECK"
echo " validate-migration: $RESULT_MIGRATION"
echo " unit-tests: $RESULT_UNIT_TESTS"
echo " integration-tests: $RESULT_INTEGRATION"
echo " frontend-lint: $RESULT_FRONTEND_LINT"
@@ -2014,9 +1968,9 @@ jobs:
# 必填检查项(根据PR类型决定)
# 通用检查(所有PR都必须过)
REQUIRED_GENERAL=(
"validate-style:$RESULT_STYLE"
"validate-security:$RESULT_SECURITY"
"validate-python:$RESULT_PYTHON"
"validate-code-quality:$RESULT_CODE_QUALITY"
"validate-type-check:$RESULT_TYPE_CHECK"
"validate-migration:$RESULT_MIGRATION"
"frontend-lint:$RESULT_FRONTEND_LINT"
"build-pr:$RESULT_BUILD_PR"
"ai-code-review:$AI_REVIEW_STATUS"
@@ -2060,11 +2014,6 @@ jobs:
echo " ⏳ $name: pending(审查中,暂不阻塞)"
continue
fi
# Security scan cancelled/failed时不阻塞部署(runner故障不应卡住流水线)
if [ "$name" = "validate-security" ] && { [ "$result" = "cancelled" ] || [ "$result" = "failure" ]; }; then
echo " ⚠️ $name: $result(安全扫描为非阻塞项,不卡住部署)"
continue
fi
check_job "$name" "$result"
done
@@ -2116,4 +2065,4 @@ jobs:
[ "${{ steps.gate.outputs.gate_result }}" = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
curl -sfH "Authorization: token ${GITHUB_TOKEN:-$GITEA_TOKEN}" -o /tmp/_ci_trace.py "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/ci_trace_report.py?ref=${GITHUB_SHA}" 2>/dev/null && python3 /tmp/_ci_trace.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
@@ -1,59 +0,0 @@
name: Playwright Base Image Build
on:
workflow_dispatch:
inputs:
reason:
description: "触发原因"
required: false
default: "构建 playwright 基础镜像"
jobs:
build-playwright:
name: Build Playwright Base Image
runs-on: runtime-builder
timeout-minutes: 30
steps:
- name: Docker login to Gitea Registry
shell: sh
env:
GITEA_REGISTRY_USER: xiaoxia
GITEA_REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
set -eu
for i in 1 2 3; do
echo "=== Docker login attempt $i/3 ==="
if printf '%s' "${GITEA_REGISTRY_TOKEN}" | docker login git.xiaoxiajianji.com -u "${GITEA_REGISTRY_USER}" --password-stdin; then
echo "Docker login successful"
break
fi
echo "Docker login failed (attempt $i/3), retrying in 5s..."
sleep 5
[ $i -eq 3 ] && exit 1
done
- name: Pull, retag and push Playwright image
shell: sh
run: |
set -eu
OFFICIAL_IMAGE="mcr.microsoft.com/playwright:v1.45.0-jammy"
GITEA_IMAGE="git.xiaoxiajianji.com/xiaoxia/base/playwright:v1.45.0-jammy"
echo "=== Pulling official Playwright image ==="
docker pull "${OFFICIAL_IMAGE}"
echo "=== Tagging ==="
docker tag "${OFFICIAL_IMAGE}" "${GITEA_IMAGE}"
echo "=== Pushing to Gitea Registry ==="
docker push "${GITEA_IMAGE}"
echo "Done: ${GITEA_IMAGE}"
- name: Cleanup
if: always()
shell: sh
run: |
docker rmi "mcr.microsoft.com/playwright:v1.45.0-jammy" 2>/dev/null || true
docker rmi "git.xiaoxiajianji.com/xiaoxia/base/playwright:v1.45.0-jammy" 2>/dev/null || true
echo "Cleanup done"
+1 -1
View File
@@ -3,7 +3,7 @@ name: PR Auto Scan
# 作为短作业模式的兜底,防止事件驱动遗漏
on:
schedule:
# - cron: "*/15 * * * *" # DISABLED: temporarily to stop failure spam (2026-09-02) # 每10分钟扫描一次(脚本自带240s墙钟上限,降频减负)
- cron: "*/10 * * * *" # 每10分钟扫描一次(脚本自带240s墙钟上限,降频减负)
workflow_dispatch:
permissions:
+2 -3
View File
@@ -18,7 +18,7 @@ jobs:
name: Auto Approve on CI Green
runs-on: ci-check
if: github.event_name == 'pull_request' && !github.event.pull_request.draft
timeout-minutes: 10 # 等待CI全绿+审批,需要充足时间
timeout-minutes: 3 # 长等待模式:等CI全绿后自动合并,不遗漏任何PR
steps:
- name: Checkout code
shell: sh
@@ -61,8 +61,7 @@ jobs:
name: Auto Merge on CI Green + Approved
runs-on: ci-check
if: github.event_name == 'pull_request' && !github.event.pull_request.draft && github.event.pull_request.base.ref == 'develop'
needs: [auto-approve] # 修复竞态:必须等审批完成后再尝试合并
timeout-minutes: 15 # 等待审批+CI就绪+合并,需要充足时间
timeout-minutes: 3 # 短作业模式:检查一次,不满足就退出,由pr-auto-scan每5分钟定时兜底
steps:
- name: Checkout code
shell: sh
-6
View File
@@ -24,11 +24,6 @@ ruff_cache/
.env.production
.env.staging
!.env.example
# 配置模板不受忽略规则限制
!deploy/configs/.env.staging
!deploy/configs/.env.production
# 渲染后的 env 文件包含真实密钥,绝不能提交
.env.rendered
# OS / editor
.DS_Store
@@ -59,4 +54,3 @@ frontend-v21-ui-prototype-final.html
!.vscode/settings.json
.vscode/extensions.json
.coverage
.env.current
@@ -1,23 +0,0 @@
"""add duplicate_rate to generated_videos
Revision ID: 059_duplicate_rate
Revises: 058_uq_asset_lib_project_kind
Create Date: 2026-08-31
"""
import sqlalchemy as sa
from alembic import op
revision = "059_duplicate_rate"
down_revision = "058_uq_asset_lib_project_kind"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column("generated_videos", sa.Column("duplicate_rate", sa.Float(), nullable=True))
def downgrade() -> None:
op.drop_column("generated_videos", "duplicate_rate")
@@ -1,57 +0,0 @@
"""migrate template_segments data to template_clip_configs
Revision ID: 060_migrate_segments
Revises: 059_duplicate_rate
Create Date: 2026-08-31
"""
import sqlalchemy as sa
from alembic import op
revision = "060_migrate_segments"
down_revision = "059_duplicate_rate"
branch_labels = None
depends_on = None
def upgrade() -> None:
dialect = op.get_bind().dialect.name
if dialect == "postgresql":
config_expr = (
"CASE WHEN s.material_type IS NOT NULL AND s.material_type != '' "
"THEN json_build_object('material_type', s.material_type)::jsonb "
"ELSE '{}'::jsonb END"
)
empty_json = "'{}'::jsonb"
else:
config_expr = (
"CASE WHEN s.material_type IS NOT NULL AND s.material_type != '' "
"THEN JSON_OBJECT('material_type', s.material_type) "
"ELSE '{}' END"
)
empty_json = "'{}'"
sql_str = (
"INSERT INTO template_clip_configs "
'(id, template_id, clip_type, "order", min_duration, max_duration, '
"text_template, material_requirements, transition_effect, config, "
"created_at, updated_at) "
"SELECT "
"s.id, s.template_id, 'main', s.segment_order, "
"s.duration_min, s.duration_max, "
"'', " + empty_json + ", "
"'cut', " + config_expr + ", "
"s.created_at, s.updated_at "
"FROM template_segments s "
"WHERE NOT EXISTS ("
" SELECT 1 FROM template_clip_configs c "
" WHERE c.template_id = s.template_id"
")"
)
op.execute(sa.text(sql_str))
def downgrade() -> None:
pass
@@ -1,26 +0,0 @@
"""add sort_order to template_categories
Revision ID: 061_sort_order
Revises: 060_migrate_segments
Create Date: 2026-09-02
"""
import sqlalchemy as sa
from alembic import op
revision = "061_sort_order"
down_revision = "060_migrate_segments"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column(
"template_categories",
sa.Column("sort_order", sa.Integer, nullable=False, server_default="0"),
)
def downgrade() -> None:
op.drop_column("template_categories", "sort_order")
@@ -1,28 +0,0 @@
"""re-add edit_plan_id to generation_tasks (align staging with production)
Revision ID: 062_edit_plan_id
Revises: 061_sort_order
Create Date: 2026-09-02
"""
import sqlalchemy as sa
from alembic import op
revision = "062_edit_plan_id"
down_revision = "061_sort_order"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column(
"generation_tasks",
sa.Column("edit_plan_id", sa.String(36), nullable=True),
)
op.create_index("ix_generation_tasks_edit_plan_id_2", "generation_tasks", ["edit_plan_id"])
def downgrade() -> None:
op.drop_index("ix_generation_tasks_edit_plan_id_2", table_name="generation_tasks")
op.drop_column("generation_tasks", "edit_plan_id")
+22 -68
View File
@@ -26,7 +26,7 @@ from app.schemas.asset import (
UpdateAssetReviewRequest,
)
from app.schemas.tag import TagAssetsRequest
from app.services.asset_segment_tracker import compute_asset_availability, get_asset_recent_use_counts
from app.services.asset_segment_tracker import compute_asset_availability
from fastapi import APIRouter, Depends, HTTPException, Query, Response
from packages.domain.smart_match import smart_select_assets
@@ -290,7 +290,7 @@ def list_assets(
else:
total = asset_repository.count_by_project_ids(project_ids, status=status_list)
# 跨项目分页:逐项目累积直到凑够一页
paged_items = []
paged_items: list = []
offset = skip
remaining = limit
for pid in project_ids:
@@ -579,80 +579,34 @@ def smart_match_assets(
filtered_assets = asset_repository.find_by_library(request.library_id, status=["ready"], limit=10000)
total_candidates = len(filtered_assets)
# ── 过滤前置:余量 + 高频使用,过滤在评分/截取 limit 之前完成 ──────────
# 旧实现先 smart_select_assets(limit=N) 再对这 N 条做过滤,过滤后不回补,
# 当排名靠前的素材恰好都被排除时返回空 items(前端回退全选,smart-match 名存实亡)。
# 现在先过滤全量候选,每级过滤后为空/不足则回退上一级,最后才评分截取。
# 调用统一智能选素材算法(kind 已在 DB 层过滤,无需重复过滤)
results = smart_select_assets(
filtered_assets,
limit=request.limit,
kind=None,
)
# 1) 余量过滤:usable=False(零重复可切区间耗尽且历史区间均达复用上限)的素材排除
usable_assets = []
exhausted_assets = []
for a in filtered_assets:
# 结果层过滤:usable=false(零重复可切区间耗尽且历史区间均达复用上限)的素材
# 不返回给前端;不动 smart_select_assets 评分逻辑本身
filtered_results = []
for r in results:
try:
avail = compute_asset_availability(a)
avail = compute_asset_availability(r.asset)
except Exception:
logger.warning(
"smart-match 余量计算失败,按可用处理: asset_id=%s",
getattr(a, "id", "?"),
getattr(r.asset, "id", "?"),
exc_info=True,
)
avail = None
if avail is not None and not avail["usable"]:
exhausted_assets.append(a)
else:
usable_assets.append(a)
if exhausted_assets:
logger.info(
"smart-match 余量过滤: 候选 %d,可切区间耗尽 %d",
len(filtered_assets), len(exhausted_assets),
)
# 回退策略:余量过滤后为空(全部耗尽)时,保留全部候选,不返回空结果。
# 宁可让用户在已耗尽素材上复用,也比 smart-match 空结果回退全选更可控
# (全选同样会选到这些素材,且不经过评分排序)。
pool = usable_assets if usable_assets else filtered_assets
# 2) 高频使用排除:同一素材在最近 5 个视频中出现超过 3 次则排除
MAX_RECENT_USE_COUNT = 3
high_freq_assets = set()
if pool:
asset_ids = [getattr(a, "id", "") for a in pool if getattr(a, "id", "")]
if asset_ids:
try:
use_counts = get_asset_recent_use_counts(
db=asset_repository.session,
asset_ids=asset_ids,
recent_video_count=5,
)
for a in pool:
aid = getattr(a, "id", "")
count = use_counts.get(aid, 0)
if count > MAX_RECENT_USE_COUNT:
high_freq_assets.add(aid)
logger.info(
"smart-match 排除高频使用素材: asset_id=%s use_count=%d limit=%d",
aid, count, MAX_RECENT_USE_COUNT,
)
# 回退策略:排除后剩余素材不足(为空或不够 limit)时,
# 不再全部排除,保留全部可用素材
if high_freq_assets:
remaining_count = len(pool) - len(high_freq_assets)
enough = request.limit is None or remaining_count >= request.limit
if remaining_count > 0 and enough:
pool = [a for a in pool if getattr(a, "id", "") not in high_freq_assets]
else:
logger.info(
"smart-match 高频排除后素材不足(%d<%s),保留全部 %d 条",
remaining_count,
request.limit if request.limit is not None else "不限",
len(pool),
)
except Exception:
logger.warning("smart-match 高频使用查询失败,跳过排除", exc_info=True)
# 3) 调用统一智能选素材算法(kind 已在 DB 层过滤,无需重复过滤)
results = smart_select_assets(pool, limit=request.limit, kind=None)
logger.info(
"smart-match 排除已用尽素材: asset_id=%s name=%s",
getattr(r.asset, "id", "?"),
getattr(r.asset, "name", ""),
)
continue
filtered_results.append(r)
# 扁平结构:SmartMatchItem 继承 AssetResponse,素材字段直接在条目顶层,
# 前端无需解析 item.asset 包装层,item.id / item.usable / 余量字段直接可读
@@ -662,7 +616,7 @@ def smart_match_assets(
score=r.score,
breakdown=r.breakdown,
)
for r in results
for r in filtered_results
]
return SmartMatchResponse(items=items, total_candidates=total_candidates)
+1 -1
View File
@@ -456,7 +456,7 @@ async def wechat_callback(
user = user_repository.find_by_id(response.user_id)
binding_complete = False
if user:
binding_complete = bool(
binding_complete = (
user.phone_verified and user.email_verified and user.email and "@wechat.local" not in user.email
)
+3 -3
View File
@@ -513,7 +513,7 @@ def generate_cover(
if generation_task_id:
try:
task = gen_task_repo.get(generation_task_id)
if task and getattr(task, "cover_url", ""): # type: ignore[arg-type]
if task and getattr(task, "cover_url", ""):
cover_url_from_task = task.cover_url
logger.info(
"[封面生成] 统一管道封面(步骤A-direct): plan_id=%s task_id=%s url=%s",
@@ -538,7 +538,7 @@ def generate_cover(
gv_task_id = getattr(gv, "generation_task_id", "") or ""
if gv_task_id:
task_a2 = gen_task_repo.get(gv_task_id)
if task_a2 and getattr(task_a2, "cover_url", ""): # type: ignore[arg-type]
if task_a2 and getattr(task_a2, "cover_url", ""):
cover_url_from_task = task_a2.cover_url
logger.info(
"[封面生成] 封面(步骤A2-video-task): plan_id=%s video_id=%s url=%s",
@@ -747,7 +747,7 @@ def generate_cover(
if cover_url_from_task:
# 标题已在预览视频渲染时烧录(ASS字幕),封面帧自然包含标题
cover_data: dict[str, object] = { # type: ignore[no-redef]
cover_data = {
"type": "ai_frame",
"image_url": cover_url_from_task,
"frame_time": 0.0,
@@ -98,7 +98,7 @@ def _resolve_strategy_id_from_template(template_id: str, db: Session, user_id: s
try:
new_repo = SQLAlchemyEditTemplateRepository(db)
new_template = new_repo.get(template_id)
if new_template and getattr(new_template, "editing_mode", ""): # type: ignore[arg-type]
if new_template and getattr(new_template, "editing_mode", ""):
mode = new_template.editing_mode.strip()
if mode:
logger.info(
+1 -1
View File
@@ -385,7 +385,7 @@ def create_generation_task(
use_case = CreateGenerationTaskUseCase(generation_task_repository)
count = request.count
created_tasks: list = []
created_tasks = []
failed_tasks = []
user_id = authenticated_user.user.id
# 同批次任务共享 batch_id,用于视频查重时批次内比对
+110 -373
View File
@@ -23,10 +23,6 @@ import re
from app.auth import AuthenticatedUser, get_current_user
from app.core.storage import get_storage_service
from app.dependencies import get_asset_repository, get_db_session
# 默认转场时长(与 worker 端保持一致)
_DEFAULT_TRANSITION_DURATION = 0.5
from app.services.asset_segment_tracker import (
REUSE_RATIO_LIMIT,
SEGMENT_EDGE_GAP,
@@ -41,20 +37,10 @@ from fastapi import APIRouter, BackgroundTasks, Depends, HTTPException, Query, s
from sqlalchemy.orm import Session
from packages.adapters.sqlalchemy_impl.asset_repository import SQLAlchemyAssetRepository
from packages.adapters.sqlalchemy_impl.template_clip_config_repository import (
SQLAlchemyTemplateClipConfigRepository,
)
from packages.adapters.sqlalchemy_impl.template_repository import (
SQLAlchemyTemplateRepository,
)
from packages.domain.plan_generator_utils import (
_calc_random_start_time,
build_scene_segments,
extract_scene_points_from_metadata,
pick_scene_aware_start,
pick_start_in_scene_segment,
)
from packages.domain.smart_match import SCORE_RANDOM_NOISE_MAX, score_asset
from packages.domain.plan_generator_utils import _calc_random_start_time
from packages.shared.mediakit_client import get_mediakit_client
from .dependencies import get_draft_plan_id, get_editor_services
@@ -425,25 +411,7 @@ def _get_template_segments(
result.append((cc.order, dur_min, dur_max))
return sorted(result, key=lambda x: x[0])
except Exception:
logger.warning("新模板系统查询clip_configs失败(主表可能不存在),直接查clip_configs表", exc_info=True)
# 兜底:直接查 template_clip_configs 表(片段表有 template_id 外键,不依赖模板主表)
try:
direct_repo = SQLAlchemyTemplateClipConfigRepository(db)
direct_configs = direct_repo.list_by_template(template_id)
if direct_configs:
result = []
for cc in direct_configs:
dur_min = _safe_segment_duration(cc.min_duration, _DEFAULT_EDITOR_CLIP_DURATION)
dur_max = _safe_segment_duration(
cc.max_duration or cc.min_duration,
_DEFAULT_EDITOR_CLIP_DURATION,
)
dur_min, dur_max = min(dur_min, dur_max), max(dur_min, dur_max)
result.append((cc.order, dur_min, dur_max))
return sorted(result, key=lambda x: x[0])
except Exception:
logger.warning("直接查clip_configs表也失败,继续回退旧系统", exc_info=True)
logger.warning("新模板系统查询clip_configs失败,回退到旧系统", exc_info=True)
# 回退到旧模板系统(template_segments表)
try:
@@ -481,12 +449,6 @@ def _recommended_time_conflicts(
return False
# 向后兼容别名:镜头段构建/段内取点逻辑已下沉到 packages.domain.plan_generator_utils,
# 旧测试与历史代码仍按 clips._build_scene_segments / _pick_start_in_scene_segment 导入
_build_scene_segments = build_scene_segments
_pick_start_in_scene_segment = pick_start_in_scene_segment
def _get_mediakit_recommendations(
asset_ids: list[str],
asset_repo,
@@ -599,47 +561,6 @@ def _get_mediakit_recommendations(
return {}
def _calc_plan_internal_duplicate_rate(clips_data: list[dict]) -> float:
"""估算单条成片内部重复率(%).
检查本条成片中同一素材是否有重叠的时间区间。
重叠时长 / 成片总时长 * 100 = 内部重复率。
这是一个轻量估算,不依赖视频指纹;完整查重由 worker 异步完成。
"""
if not clips_data:
return 0.0
# 按素材分组
by_asset: dict[str, list[tuple[float, float]]] = {}
total_duration = 0.0
for c in clips_data:
aid = c.get("asset_id", "")
if not aid:
continue
start = c.get("start_time", 0.0)
end = start + c.get("duration", 0.0)
by_asset.setdefault(aid, []).append((start, end))
total_duration += c.get("duration", 0.0)
if total_duration <= 0:
return 0.0
# 检查同素材内的区间重叠
overlap_duration = 0.0
for segments in by_asset.values():
if len(segments) < 2:
continue
segments_sorted = sorted(segments, key=lambda s: s[0])
last_end = segments_sorted[0][1]
for start, end in segments_sorted[1:]:
overlap = max(0.0, min(end, last_end) - start)
if overlap > 0:
overlap_duration += overlap
last_end = max(last_end, end)
return round(overlap_duration / total_duration * 100, 1)
@router.post("/clips/from-assets", response_model=ClipsFromAssetsResponse)
def create_clips_from_assets_editor(
template_id: str,
@@ -684,26 +605,10 @@ def create_clips_from_assets_editor(
# 2. 获取素材实际时长(去重查询)
unique_asset_ids = list(dict.fromkeys(asset_ids))
asset_durations: dict[str, float] = {}
asset_smart_scores: dict[str, float] = {}
# 素材 metadata 中缓存的场景切换点(由后台 MediaKit SceneChange 检测写入):
# 有缓存时片段起点从随机镜头段中选取(不同片段来自不同镜头),无缓存回退随机起点
asset_scene_points: dict[str, list[float]] = {}
for asset_id in unique_asset_ids:
asset = asset_repo.get(asset_id)
if asset and hasattr(asset, "duration"):
asset_durations[asset_id] = float(asset.duration or 0.0)
# 计算 smart_match 综合评分,用于候选排序
smart_score, _ = score_asset(asset)
asset_smart_scores[asset_id] = smart_score
# 读取场景切换点缓存(新素材未检测过时为 None,走随机起点兜底)
cached_points = extract_scene_points_from_metadata(getattr(asset, "metadata", None))
if cached_points:
asset_scene_points[asset_id] = cached_points
logger.info(
"from-assets 场景缓存命中: %d/%d 个素材有场景切换点",
len(asset_scene_points),
len(unique_asset_ids),
)
# 3. 在内存中计算所有片段数据(使用随机起始时间,不调用MediaKit)
# 读取素材 metadata 中持久化的历史已用区间(跨任务/跨调用去重),
@@ -736,52 +641,19 @@ def create_clips_from_assets_editor(
return False
return reused_durations.get(aid, 0.0) / assigned > REUSE_RATIO_LIMIT
# 素材耗尽标志:某轮循环中所有素材均被跳过时为 True
all_assets_exhausted = False
# 计算转场重叠补偿:每个 clip 需要额外增加的时长
# 目标:渲染后视频总时长 = 模板设定的各片段时长之和
# 公式:每 clip 增加 (n_segments - 1) * td / n_segments
n_segments = len(segments)
if n_segments > 1:
transition_compensation = (n_segments - 1) * _DEFAULT_TRANSITION_DURATION / n_segments
else:
transition_compensation = 0.0
# 打乱 segments 的处理顺序(分配素材的顺序随机化),但最终 clips_data 按原始 order 排序
shuffled_indices = list(range(len(segments)))
random.shuffle(shuffled_indices)
for idx in shuffled_indices:
_seg_order, dur_min, dur_max = segments[idx]
for i, (_seg_order, dur_min, dur_max) in enumerate(segments):
# 在 segment 的 duration_min ~ duration_max 之间随机取值(保留一位小数)
raw_duration = random.uniform(dur_min, dur_max)
# 加上转场补偿,确保最终输出时长 = 模板设定总时长
raw_duration += transition_compensation
# 贪心分配素材:按"已使用次数"升序排列候选素材(使用最少的优先),
# 同次数随机打散,避免"A-B-C-D"的固定组合反复出现。
# 跳过时长缺失、复用占比已超 10% 阈值的素材;
# 轮询分配素材:跳过时长缺失、复用占比已超 15% 阈值的素材;
# 选中后计算起点,若该素材可用区间耗尽且复用被闸门拒绝(calc 返回 None),
# 继续尝试下一个素材
# 继续轮询下一个素材
asset_id = ""
clip_duration = 0.0
start_time: float | None = None
# 动态按使用次数排序:优先选使用最少的素材,同次数随机打散
asset_use_counts = {aid: len(used_segments.get(aid, [])) for aid in asset_ids}
# 排序键:smart_match 评分(注入随机噪声)→ 使用次数 → 纯随机。
# 噪声让得分接近的素材排名每次浮动,避免同一批素材反复选出相同组合,
# 从素材组合层面降低成片查重率;分差 > SCORE_RANDOM_NOISE_MAX 时排名稳定,
# 质量差距显著的素材仍保持优先级。
sorted_candidates = sorted(
asset_ids,
key=lambda aid: (
-(asset_smart_scores.get(aid, 0.0) + random.uniform(0.0, SCORE_RANDOM_NOISE_MAX)),
asset_use_counts.get(aid, 0),
random.random(),
),
)
for candidate in sorted_candidates:
n_assets = len(asset_ids)
for offset in range(n_assets):
candidate = asset_ids[(i + offset) % n_assets]
candidate_total = asset_durations.get(candidate, 0.0)
if candidate_total <= 0:
continue
@@ -795,30 +667,16 @@ def create_clips_from_assets_editor(
candidate,
)
continue
# 起始时间选取(不调用 MediaKit,保证接口快速返回):
# 1) 素材有场景切换点缓存时,优先从随机镜头段中选起点(不同片段来自不同镜头,
# 画面内容本质不同),与 used_segments 做冲突避让(含 1.5s 边缘间隙)
# 2) 无缓存 / 镜头段全冲突 → _calc_random_start_time 随机起点兜底;
# 100 次避不开历史区间时走受控复用回调(复用片段累加 reused_durations,
# 回调内部预判复用后占比超 10% 则拒绝并返回 None)
candidate_start = None
if candidate in asset_scene_points:
candidate_start = pick_scene_aware_start(
candidate,
candidate_duration,
asset_durations,
asset_scene_points,
used_segments,
edge_gap=SEGMENT_EDGE_GAP,
)
if candidate_start is None:
candidate_start = _calc_random_start_time(
candidate,
candidate_duration,
asset_durations,
used_segments,
on_exhausted=reuse_cb,
)
# 随机起始时间(不调用 MediaKit,保证接口快速返回);100 次避不开
# 历史区间时走受控复用回调(复用片段累加 reused_durations,回调内部
# 预判复用后占比超 15% 则拒绝并返回 None)
candidate_start = _calc_random_start_time(
candidate,
candidate_duration,
asset_durations,
used_segments,
on_exhausted=reuse_cb,
)
if candidate_start is None:
# 该素材可用区间耗尽且复用被闸门/use_count 上限拒绝 → 尝试下一素材
logger.info(
@@ -833,7 +691,6 @@ def create_clips_from_assets_editor(
if not asset_id or start_time is None:
# 所有素材时长缺失、复用占比超阈值,或区间耗尽且复用被拒 → 素材可切区间不足
all_assets_exhausted = True
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="素材可切区间不足,请补充新素材",
@@ -849,7 +706,7 @@ def create_clips_from_assets_editor(
clips_data.append(
{
"order": _seg_order,
"order": i,
"asset_id": asset_id,
"start_time": start_time,
"duration": clip_duration,
@@ -857,9 +714,6 @@ def create_clips_from_assets_editor(
}
)
# 按原始 segment order 排序,确保 clips_data 的 order 字段有序(0,1,2,3...)
clips_data.sort(key=lambda c: c["order"])
# 4. 事务性替换:清空旧片段 → 创建新片段 → 标记ready(单事务,失败自动回滚)
created_count = plan_svc.replace_all_clips_transactional(plan_id, clips_data)
@@ -879,31 +733,11 @@ def create_clips_from_assets_editor(
unique_asset_ids,
)
# 6. 估算成片内部重复率(本条成片中同一素材的重叠片段时长占比)
dup_rate = _calc_plan_internal_duplicate_rate(clips_data)
duplicate_warning = None
if dup_rate > 50:
duplicate_warning = f"查重率 {dup_rate:.1f}% 超过50%,建议更换素材或模板"
logger.warning(
"from-assets 成片查重率超标: plan_id=%s dup_rate=%.1f%%",
plan_id,
dup_rate,
)
# 7. 素材耗尽提示
exhaustion_warning = None
if all_assets_exhausted and created_count < len(segments):
exhaustion_warning = (
"素材可切区间不足,部分片段使用了复用素材。" "建议:1) 补充更多素材到素材库 2) 使用不同的素材组合生成"
)
# 8. 立即返回响应
# 6. 立即返回响应
return ClipsFromAssetsResponse(
created_count=created_count,
plan_id=plan_id,
clip_ids=[],
duplicate_warning=duplicate_warning,
exhaustion_warning=exhaustion_warning,
)
@@ -911,15 +745,7 @@ def _update_mediakit_recommendations_async( # pragma: no cover
plan_id: str,
asset_ids: list[str],
) -> None:
"""后台任务:使用 SceneChange 智能选帧并更新片段的起始时间.
优先使用 SceneChange 策略检测视频镜头切换点,将每个素材按镜头段拆分,
各片段优先从不同镜头段中选取起始时间,实现「不同片段展示不同场景」的效果。
降级策略:
1. SceneChange 优先 → detect_scene_changes 内部已含 TimeInterval 降级
2. 若 detect_scene_changes 仍返回 None → 回退到旧的 analyze_videos 方式
3. 所有方式都失败 → 保持现有随机 start_time,不影响视频生成
"""后台任务:调用 MediaKit 智能选片并更新片段的起始时间.
此函数在后台异步执行,不影响接口响应时间。
失败时静默处理,不影响已创建的片段。
@@ -941,6 +767,12 @@ def _update_mediakit_recommendations_async( # pragma: no cover
asset_repo = SQLAlchemyAssetRepository(db)
plan_svc = EditPlanService(db)
# 调用 MediaKit 获取推荐时间
recommendations = _get_mediakit_recommendations(asset_ids, asset_repo)
if not recommendations:
logger.info("后台任务: MediaKit 无推荐结果,跳过更新")
return
# 查询该 plan 的所有片段(分批获取,避免硬编码 limit 截断)
batch_size = 500
all_clips = []
@@ -963,16 +795,15 @@ def _update_mediakit_recommendations_async( # pragma: no cover
unique_asset_ids = list({getattr(c, "asset_id", "") or "" for c in clips} - {""})
assets_map: dict[str, object] = {a.id: a for a in asset_repo.find_by_ids(unique_asset_ids)}
# 按 asset_id 预分组片段对象(按 order 排序,保证按模板顺序分配镜头段)
clips_by_asset: dict[str, list] = defaultdict(list)
# 按 asset_id 预分组片段时间段(消除 O(N^2) 嵌套循环)
clips_by_asset: dict[str, list[tuple[str, float, float]]] = defaultdict(list)
for clip in clips:
aid = getattr(clip, "asset_id", "") or ""
if aid:
clips_by_asset[aid].append(clip)
for aid in clips_by_asset:
clips_by_asset[aid].sort(key=lambda c: c.order)
if aid and clip.start_time is not None:
clips_by_asset[aid].append((clip.id, clip.start_time, clip.start_time + clip.duration))
# 读取素材全部历史已用区间(跨任务/跨 plan 持久化记录)
# 读取素材全部历史已用区间(跨任务/跨 plan 持久化记录):
# MediaKit 挪点必须与随机选片一样避让历史区间,否则会把片段挪回已用过的画面
historical_segments = get_used_segments(db, unique_asset_ids)
# 已更新的片段ID(用于排除已移动的旧时间段)
@@ -981,22 +812,16 @@ def _update_mediakit_recommendations_async( # pragma: no cover
updated_segments: dict[str, list[tuple[float, float]]] = {}
updated_count = 0
# 尝试获取存储服务(用于生成视频 URL)
try:
storage = get_storage_service()
except Exception:
logger.warning("后台任务: 获取存储服务失败,跳过 SceneChange 更新")
return
# 获取 MediaKit 客户端
client = get_mediakit_client()
# 对每个素材,检测场景切换点并分配镜头段
for asset_id in unique_asset_ids:
asset_clips = clips_by_asset.get(asset_id, [])
if not asset_clips:
# 遍历片段,按 asset_id 匹配推荐时间
for clip in clips:
asset_id = getattr(clip, "asset_id", "") or ""
if not asset_id or asset_id not in recommendations:
continue
recommended_start = recommendations[asset_id]
clip_duration = clip.duration
# 从预加载字典获取素材(O(1) 查找)
asset = assets_map.get(asset_id)
if not asset:
continue
@@ -1004,177 +829,89 @@ def _update_mediakit_recommendations_async( # pragma: no cover
if asset_total <= 0:
continue
# 获取素材视频 URL
video_url: str | None = None
storage_key = getattr(asset, "storage_key", None) or ""
mime = getattr(asset, "mime_type", "") or ""
if storage_key and mime.startswith("video/"):
try:
video_url = storage.get_download_url(storage_key)
except Exception as e:
logger.warning("后台任务: 获取素材URL失败: asset_id=%s error=%s", asset_id, e)
# 构建该素材的占用区间列表(排除已更新片段)
def _get_other_segments(asset_id_inner, clip_id_inner):
segs: list[tuple[float, float]] = []
for c in clips_by_asset.get(asset_id_inner, []):
cid = c.id
if cid != clip_id_inner and cid not in updated_clip_ids:
segs.append((c.start_time, c.start_time + c.duration))
segs.extend(updated_segments.get(asset_id_inner, []))
# 并入历史已用区间
def _norm(segs_in):
return {(round(float(a), 3), round(float(b), 3)) for a, b in segs_in}
return list(_norm(segs) | _norm(historical_segments.get(asset_id_inner, [])))
# 优先使用 SceneChange 策略
scene_segments: list[tuple[float, float]] = []
# 先查素材 metadata 中的场景点缓存:命中则直接复用,跳过 MediaKit 检测
# (缓存由本任务首次检测后写入,跨任务/跨 plan 复用)
cached_points = extract_scene_points_from_metadata(getattr(asset, "metadata", None))
if cached_points:
scene_segments = build_scene_segments(cached_points, asset_total)
# 推荐时间 + 片段时长不能超过素材总时长
if recommended_start + clip_duration > asset_total:
logger.info(
"后台任务: 命中场景点缓存: asset_id=%s scenes=%d",
asset_id,
len(scene_segments),
)
if not scene_segments and client.is_available and video_url:
scene_changes = client.detect_scene_changes(video_url)
if scene_changes is not None:
scene_segments = build_scene_segments(scene_changes, asset_total)
logger.info(
"后台任务: 素材场景检测完成: asset_id=%s scenes=%d",
asset_id,
len(scene_segments),
)
# 检测结果写入素材 metadata 缓存:首次生成用随机起点,
# 检测完成后后续生成的渲染前同步路径即可读缓存选镜头段
try:
existing_meta = dict(getattr(asset, "metadata", None) or {})
existing_meta["scene_change_points"] = scene_changes
asset.metadata = existing_meta # type: ignore[attr-defined]
asset_repo.update(asset) # type: ignore[arg-type]
logger.info(
"后台任务: 场景点已写入素材缓存: asset_id=%s points=%d",
asset_id,
len(scene_changes),
)
except Exception as cache_err:
# 缓存写入失败不影响本次片段更新
logger.warning(
"后台任务: 场景点缓存写入失败: asset_id=%s error=%s",
asset_id,
cache_err,
)
# SceneChange 未获得有效结果 → 尝试 analyze_videos 作为 fallback
if not scene_segments and video_url:
fallback_recs = _get_mediakit_recommendations([asset_id], asset_repo)
if fallback_recs and asset_id in fallback_recs:
# analyze_videos 只返回单个推荐点,转为单镜头段
rec_start = fallback_recs[asset_id]
scene_segments = [(rec_start, asset_total)]
logger.info(
"后台任务: 使用 analyze_videos fallback: asset_id=%s start=%.2f",
asset_id,
rec_start,
)
if not scene_segments:
# 所有方式都失败 → 保持现有随机 start_time
logger.info(
"后台任务: SceneChange 与 analyze_videos 均无结果,保持随机起点: asset_id=%s",
"后台任务: 推荐时间越界,跳过: asset_id=%s recommended=%.2f duration=%.1f total=%.1f",
asset_id,
recommended_start,
clip_duration,
asset_total,
)
continue
# 为每个片段分配不同的镜头段
scene_segments_pool = list(scene_segments) # 可消费的镜头段池
for clip in asset_clips:
clip_duration = clip.duration
recommended_start: float | None = None
# 构建排除当前片段及已更新片段后的占用列表(O(M),M=同素材片段数)
other_segments: list[tuple[float, float]] = [
(cs, ce)
for cid, cs, ce in clips_by_asset.get(asset_id, [])
if cid != clip.id and cid not in updated_clip_ids
]
other_segments.extend(updated_segments.get(asset_id, []))
# 从镜头段池中依次尝试,选一个不冲突的
for seg_idx, (seg_start, seg_end) in enumerate(scene_segments_pool):
candidate_start = pick_start_in_scene_segment(seg_start, seg_end, clip_duration)
if candidate_start is None:
continue # 镜头段太短,跳过
# 并入该素材全部历史已用区间(含其他 plan/其他任务),set 去重:
# 本 plan 片段创建时已写入历史记录
# 并入该素材全部历史已用区间(含其他 plan/其他任务)。
# set 去重前先归一化精度(round 3 位),避免浮点尾差导致逻辑相同的
# 区间(如 1.0 与 1.0000000001)被误判为不同区间
def _norm(segs):
return {(round(float(a), 3), round(float(b), 3)) for a, b in segs}
# 检查越界
if candidate_start + clip_duration > asset_total:
continue
other_segments = list(_norm(other_segments) | _norm(historical_segments.get(asset_id, [])))
# 检查与已用区间冲突
other_segs = _get_other_segments(asset_id, clip.id)
if _recommended_time_conflicts(candidate_start, clip_duration, other_segs):
continue
# 检查推荐时间是否与同 plan 片段或历史已用区间冲突(含 0.3s 边缘间隙):
# 冲突时放弃该推荐、保留原随机起点(不硬挪到已用过的画面)
if _recommended_time_conflicts(recommended_start, clip_duration, other_segments):
logger.info(
"后台任务: 推荐时间与同片/历史区间冲突,保留原起点: asset_id=%s recommended=%.2f",
asset_id,
recommended_start,
)
continue
recommended_start = candidate_start
# 消费该镜头段(从池中移除,下一个片段用不同镜头段)
scene_segments_pool.pop(seg_idx)
break
if recommended_start is None:
# 镜头段用完或都冲突 → 尝试 _calc_random_start_time 兜底
used_segs_for_calc: dict[str, list[tuple[float, float]]] = {
asset_id: _get_other_segments(asset_id, clip.id)
}
fallback_start = _calc_random_start_time(
asset_id,
clip_duration,
{asset_id: asset_total},
used_segs_for_calc,
)
if fallback_start is None:
continue # 完全无法分配,保持原起点
recommended_start = fallback_start
# 更新片段起始时间
# 逐个更新并捕获异常(单点失败不影响其他片段)
try:
old_start = clip.start_time
old_end = old_start + clip_duration
# MediaKit 移动片段起点 + 同步素材 metadata 区间记录放在同一事务:
# 删旧区间记录(按 plan_id + 旧 start 匹配,兼容无 plan_id 的旧数据)、
# 写新区间,最后统一 commit;任一步失败整体 rollback,
# 保证 clip.start_time 与 metadata.used_time_ranges 不出现不一致。
plan_svc.update_clip(clip.id, start_time=recommended_start)
try:
old_start = clip.start_time
old_end = old_start + clip_duration
plan_svc.update_clip(clip.id, start_time=recommended_start)
try:
if remove_used_segment(db, asset_id, old_start, old_end, plan_id=plan_id):
record_used_segments(
db,
asset_id,
recommended_start,
recommended_start + clip_duration,
plan_id,
)
except Exception as me:
logger.warning(
"后台任务: 同步素材区间记录失败,回滚本次片段更新: clip_id=%s error=%s",
clip.id,
me,
if remove_used_segment(db, asset_id, old_start, old_end, plan_id=plan_id):
record_used_segments(
db,
asset_id,
recommended_start,
recommended_start + clip_duration,
plan_id,
)
db.rollback()
continue
db.commit()
updated_count += 1
updated_clip_ids.add(clip.id)
updated_segments.setdefault(asset_id, []).append(
(recommended_start, recommended_start + clip_duration)
)
logger.info(
"后台任务: 更新片段起始时间(场景选帧): clip_id=%s asset_id=%s start_time=%.2f",
except Exception as me:
logger.warning(
"后台任务: 同步素材区间记录失败,回滚本次片段更新: clip_id=%s error=%s",
clip.id,
asset_id,
recommended_start,
me,
)
except Exception as ue:
logger.warning("后台任务: 单个片段更新失败: clip_id=%s error=%s", clip.id, ue)
try:
db.rollback()
except Exception:
pass
db.rollback()
continue
db.commit()
updated_count += 1
updated_clip_ids.add(clip.id)
except Exception as ue:
logger.warning("后台任务: 单个片段更新失败: clip_id=%s error=%s", clip.id, ue)
try:
db.rollback()
except Exception:
pass
continue
updated_segments.setdefault(asset_id, []).append((recommended_start, recommended_start + clip_duration))
logger.info(
"后台任务: 更新片段起始时间: clip_id=%s asset_id=%s start_time=%.2f",
clip.id,
asset_id,
recommended_start,
)
logger.info("后台任务完成: plan_id=%s 成功更新 %d 个片段", plan_id, updated_count)
@@ -41,17 +41,17 @@ def list_editor_transition_presets(
_: AuthenticatedUser = Depends(get_current_user),
) -> TransitionPresetListResponse:
"""获取转场预设列表"""
from packages.domain.transition_presets import TRANSITION_PRESET_LIBRARY
from packages.domain.transition_presets import TRANSITION_PRESETS
items = [
{
"id": p.id,
"name": p.name,
"category": p.category,
"duration": p.default_duration,
"description": p.description,
"id": p["id"],
"name": p["name"],
"category": p.get("category", "通用"),
"duration": p.get("default_duration", 0.5),
"description": p.get("description", ""),
}
for p in TRANSITION_PRESET_LIBRARY
for p in TRANSITION_PRESETS
]
return TransitionPresetListResponse(items=items, total=len(items))
@@ -123,17 +123,17 @@ def list_editor_filter_presets(
_: AuthenticatedUser = Depends(get_current_user),
) -> FilterPresetListResponse:
"""获取滤镜预设列表"""
from packages.domain.filter_presets import FILTER_PRESET_LIBRARY
from packages.domain.filter_presets import FILTER_PRESETS
items = [
{
"id": p.id,
"name": p.name,
"category": p.category,
"thumbnail": p.lut_url,
"description": p.description,
"id": p["id"],
"name": p["name"],
"category": p.get("category", "通用"),
"thumbnail": p.get("thumbnail", ""),
"description": p.get("description", ""),
}
for p in FILTER_PRESET_LIBRARY
for p in FILTER_PRESETS
]
return FilterPresetListResponse(items=items, total=len(items))
@@ -189,8 +189,6 @@ class ClipsFromAssetsResponse(BaseModel):
plan_id: str = ""
message: str = ""
clip_ids: List[str] = Field(default_factory=list, description="创建的片段ID列表")
duplicate_warning: Optional[str] = Field(default=None, description="查重率超标警告")
exhaustion_warning: Optional[str] = Field(default=None, description="素材耗尽警告")
# ── 封面配置 ────────────────────────────────────────────────────────────────
+1 -22
View File
@@ -2,9 +2,7 @@
from __future__ import annotations
import json
import logging
import subprocess
import tempfile
from pathlib import Path
from typing import Any, Optional
@@ -432,8 +430,6 @@ def save_tts_job_to_library(
storage_key = f"uploads/voice/tts/{job.id}.{audio_format}"
tmp_path: Path | None = None
audio_duration: float | None = None
file_size = 0
try:
with tempfile.NamedTemporaryFile(suffix=f".{audio_format}", delete=False) as tmp:
tmp_path = Path(tmp.name)
@@ -449,23 +445,6 @@ def save_tts_job_to_library(
)
file_size = tmp_path.stat().st_size
storage_service.upload_file(tmp_path, storage_key, content_type=content_type)
# 从音频文件提取时长(ffprobe),作为 job.duration 的兜底
try:
proc = subprocess.run(
[
"ffprobe", "-v", "quiet", "-print_format", "json",
"-show_format", str(tmp_path),
],
capture_output=True, text=True, timeout=10,
)
if proc.returncode == 0:
fmt = json.loads(proc.stdout).get("format", {})
dur = float(fmt.get("duration", 0))
if dur > 0:
audio_duration = dur
except Exception:
logger.warning("ffprobe 提取时长失败: job_id=%s", job.id, exc_info=True)
except HTTPException:
raise
except Exception as e:
@@ -503,7 +482,7 @@ def save_tts_job_to_library(
mime_type=content_type,
metadata=metadata_,
file_size=file_size,
duration=job.duration or audio_duration or None,
duration=job.duration or None,
status=AssetStatus.READY,
classification_status=ClassificationStatus.PENDING, # 音频不参与内容分类,保持 pending 与 ingest 链路一致
uploaded_by_user_id=user_id,
+1 -68
View File
@@ -23,7 +23,6 @@ from app.schemas.upload import (
from fastapi import APIRouter, Depends, File, Form, HTTPException, UploadFile, status
from packages.application import SubmitIngestJobCommand, SubmitIngestJobUseCase
from packages.domain import Asset, AssetStatus
logger = logging.getLogger(__name__)
@@ -81,40 +80,6 @@ def _validate_mime_type(content_type: str | None) -> str:
return base_type
def _infer_mime_type_from_storage_key(storage_key: str) -> str:
"""从 storage_key 推断 MIME 类型(与 worker 端保持一致)。"""
lower_filename = storage_key.rsplit("/", 1)[-1].lower()
_MIME_MAP = {
".mov": "video/quicktime", ".mp4": "video/mp4", ".avi": "video/x-msvideo",
".mkv": "video/x-matroska", ".webm": "video/webm",
".png": "image/png", ".gif": "image/gif", ".bmp": "image/bmp",
".svg": "image/svg+xml", ".jpg": "image/jpeg", ".jpeg": "image/jpeg",
".mp3": "audio/mpeg", ".wav": "audio/wav", ".ogg": "audio/ogg",
".flac": "audio/flac", ".m4a": "audio/x-m4a",
}
for ext, mime in _MIME_MAP.items():
if lower_filename.endswith(ext):
return mime
return "video/mp4" # default
def _create_pending_asset(
asset_repository, project_id, library_id, storage_key, filename, mime_type, user_id, file_hash=""
):
"""立即创建一条 PROCESSING 状态的 Asset 记录,使前端能马上看到新素材。"""
asset = Asset.create(
project_id=project_id,
library_id=library_id,
name=filename,
storage_key=storage_key,
mime_type=mime_type,
status=AssetStatus.PROCESSING,
uploaded_by_user_id=user_id,
file_hash=file_hash,
)
return asset_repository.create(asset)
def _submit_ingest_job(
project_id: str,
library_id: str,
@@ -244,20 +209,6 @@ async def complete_direct_upload(
url=storage_service.get_url(normalized_key),
)
# 立即创建 Asset 记录(PROCESSING 状态),使前端刷新后即可看到新素材
filename = normalized_key.rsplit("/", 1)[-1]
mime_type = _infer_mime_type_from_storage_key(normalized_key)
pending_asset = _create_pending_asset(
asset_repository=asset_repository,
project_id=request.project_id,
library_id=request.library_id,
storage_key=normalized_key,
filename=filename,
mime_type=mime_type,
user_id=authenticated_user.user.id,
file_hash=request.file_hash,
)
job = _submit_ingest_job(
project_id=request.project_id,
library_id=request.library_id,
@@ -265,12 +216,7 @@ async def complete_direct_upload(
ingest_job_repository=ingest_job_repository,
file_hash=request.file_hash,
)
return DirectUploadCompleteResponse(
storage_key=normalized_key,
ingest_job_id=job.id,
asset_id=pending_asset.id,
url=storage_service.get_url(normalized_key),
)
return DirectUploadCompleteResponse(storage_key=normalized_key, ingest_job_id=job.id, url=storage_service.get_url(normalized_key))
@router.post(
@@ -338,18 +284,6 @@ async def upload_asset(
detail=f"Failed to upload file: {type(error).__name__}",
) from error
# 立即创建 Asset 记录(PROCESSING 状态),使前端刷新后即可看到新素材
pending_asset = _create_pending_asset(
asset_repository=asset_repository,
project_id=project_id,
library_id=library_id,
storage_key=storage_key,
filename=safe_filename,
mime_type=validated_content_type,
user_id=authenticated_user.user.id,
file_hash=file_hash,
)
job = _submit_ingest_job(
project_id=project_id,
library_id=library_id,
@@ -361,6 +295,5 @@ async def upload_asset(
return UploadAssetResponse(
storage_key=storage_key,
ingest_job_id=job.id,
asset_id=pending_asset.id,
url=file_url,
)
-1
View File
@@ -52,7 +52,6 @@ def _to_video_response(item, storage: OSSStorageService | None = None) -> VideoI
generation_params=item.generation_params,
download_url=download_url,
generated_at=format_utc_datetime(item.generated_at) if hasattr(item, "generated_at") else "",
duplicate_rate=getattr(item, "duplicate_rate", None),
)
+2 -260
View File
@@ -6,26 +6,12 @@
from __future__ import annotations
import logging
import shutil
import subprocess
import tempfile
import time
from pathlib import Path
from typing import Literal, Optional
from uuid import uuid4
from app.api.routes._helpers import get_user_plan
from app.auth import AuthenticatedUser, get_current_user
from app.core.storage import get_storage_service
from app.dependencies import (
get_asset_library_repository,
get_asset_repository,
get_audio_url_signer,
get_cosyvoice_service,
get_db_session,
get_project_repository,
get_user_repository,
)
from app.dependencies import get_audio_url_signer, get_cosyvoice_service, get_db_session, get_user_repository
from app.schemas.voice import (
PresetVoiceItemResponse,
PresetVoiceListResponse,
@@ -38,7 +24,7 @@ from app.schemas.voice_library import (
UpdateVoiceLibraryRequest,
VoiceLibraryItemResponse,
)
from fastapi import APIRouter, Depends, File, Form, HTTPException, Query, Response, UploadFile, status
from fastapi import APIRouter, Depends, HTTPException, Query, Response, status
from sqlalchemy.orm import Session
from packages.adapters.sqlalchemy_impl.voice_clone_profile_repository import SQLAlchemyVoiceCloneProfileRepository
@@ -54,12 +40,8 @@ from packages.application.voice_library.use_cases import (
QuotaExceededError,
UpdateVoiceLibraryUseCase,
)
from packages.domain import Asset, AssetStatus
from packages.domain.classification import AssetLibraryKind, ClassificationStatus
from packages.domain.entities import AssetLibrary
from packages.domain.preset_voices import PRESET_VOICES, get_preset_voice_by_id
from packages.ports.user_repository import UserRepository
from packages.shared.storage import SharedStorageService
router = APIRouter()
logger = logging.getLogger(__name__)
@@ -525,243 +507,3 @@ def delete_voice(
if not deleted:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Voice not found")
return
# ── 提取视频配音 ─────────────────────────────────────────────────────
# 支持的视频格式
EXTRACT_VIDEO_MIMES = frozenset({"video/mp4", "video/quicktime", "video/webm", "video/x-msvideo"})
MAX_EXTRACT_SIZE = 500 * 1024 * 1024 # 500MB
@router.post(
"/extract-voice",
status_code=status.HTTP_201_CREATED,
)
def extract_voice_from_video(
file: UploadFile = File(...),
project_id: str = Form(...),
authenticated_user: AuthenticatedUser = Depends(get_current_user),
project_repository=Depends(get_project_repository),
asset_library_repository=Depends(get_asset_library_repository),
asset_repository=Depends(get_asset_repository),
storage_service: SharedStorageService = Depends(get_storage_service),
sign_url=Depends(get_audio_url_signer),
):
"""从上传的视频中提取人声配音。
流程:
1. 接收视频文件(mp4/mov/webm)
2. ffmpeg 提取音频 + 降噪 + 编码为 mp3
3. 上传到 OSS,创建 Asset 记录到配音素材库
4. 返回素材信息(时长、文件大小、URL)
"""
user_id = authenticated_user.user.id
# 校验文件类型
content_type = file.content_type or ""
if content_type and content_type not in EXTRACT_VIDEO_MIMES:
# 兜底:按扩展名判断
ext = (file.filename or "").rsplit(".", 1)[-1].lower()
ext_to_mime = {"mp4": "video/mp4", "mov": "video/quicktime", "webm": "video/webm", "avi": "video/x-msvideo"}
if ext not in ext_to_mime:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="仅支持 mp4/mov/webm/avi 格式的视频文件",
)
content_type = ext_to_mime[ext]
# 找到(或自动创建)用户 voice 素材库(复用 TTS 的逻辑)
library = _find_or_create_voice_library_for_extract(
user_id=user_id,
project_repository=project_repository,
asset_library_repository=asset_library_repository,
)
tmp_dir = None
try:
tmp_dir = Path(tempfile.mkdtemp(prefix="voice_extract_"))
video_path = tmp_dir / f"input_{uuid4().hex[:8]}_{file.filename or 'video.mp4'}"
audio_path = tmp_dir / f"output_{uuid4().hex[:8]}.mp3"
# 保存上传的视频到临时文件
with open(video_path, "wb") as f:
total = 0
while chunk := file.file.read(1024 * 1024): # 1MB chunks
total += len(chunk)
if total > MAX_EXTRACT_SIZE:
raise HTTPException(
status_code=status.HTTP_413_REQUEST_ENTITY_TOO_LARGE,
detail="视频文件过大,最大支持 500MB",
)
f.write(chunk)
if video_path.stat().st_size == 0:
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="视频文件为空")
# ffmpeg: 提取音频 + 降噪 + 编码 mp3
# 滤镜链:highpass(去低频噪声) → afftdn(FFT降噪) → lowpass(去高频噪声)
ffmpeg_cmd = [
"ffmpeg",
"-y",
"-i",
str(video_path),
"-vn", # 不要视频
"-af",
"highpass=f=80,afftdn=nf=-25:tn=1,lowpass=f=8000",
"-acodec",
"libmp3lame",
"-ab",
"192k",
"-ar",
"44100",
"-ac",
"1", # 单声道(人声足够)
str(audio_path),
]
result = subprocess.run(
ffmpeg_cmd,
capture_output=True,
timeout=300, # 5 分钟超时
)
if result.returncode != 0:
stderr_text = result.stderr.decode("utf-8", errors="replace")[-500:]
logger.error("ffmpeg 提取配音失败: %s", stderr_text)
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
detail="视频音频提取失败,可能该视频没有音轨或格式不支持",
)
if not audio_path.exists() or audio_path.stat().st_size == 0:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
detail="音频提取结果为空",
)
# 获取音频时长
duration = _get_audio_duration(audio_path)
file_size = audio_path.stat().st_size
# 上传到 OSS
audio_ext = "mp3"
storage_key = f"uploads/voice/extracted/{uuid4().hex}.{audio_ext}"
storage_service.upload_file(audio_path, storage_key, content_type="audio/mpeg")
# 创建 Asset 记录
original_name = (file.filename or "video").rsplit(".", 1)[0]
asset_name = f"{original_name}-配音"
asset = Asset.create(
project_id=library.project_id,
library_id=library.id,
name=asset_name,
storage_key=storage_key,
mime_type="audio/mpeg",
metadata={
"source": "video_extract",
"original_video": file.filename or "unknown",
},
file_size=file_size,
duration=duration,
status=AssetStatus.READY,
classification_status=ClassificationStatus.PENDING,
uploaded_by_user_id=user_id,
)
asset = asset_repository.create(asset)
return {
"id": asset.id,
"name": asset.name,
"audio_url": sign_url(storage_key),
"duration": duration,
"file_size": file_size,
"status": "completed",
"source": "video_extract",
}
except HTTPException:
raise
except subprocess.TimeoutExpired:
raise HTTPException(
status_code=status.HTTP_504_GATEWAY_TIMEOUT,
detail="视频处理超时,请尝试较短的视频",
)
except Exception as e:
logger.exception("提取视频配音失败: %s", e)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="提取配音失败,请稍后重试",
)
finally:
# 清理临时文件
if tmp_dir and Path(tmp_dir).exists():
shutil.rmtree(tmp_dir, ignore_errors=True)
def _find_or_create_voice_library_for_extract(*, user_id, project_repository, asset_library_repository):
"""为用户找到或创建 voice 素材库(与 TTS 保存逻辑一致)。"""
projects = project_repository.find_accessible_projects(user_id)
if not projects:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="没有可用的项目,请先创建项目",
)
for project in projects:
for lib in asset_library_repository.find_by_project(project.id):
kind = lib.kind.value if hasattr(lib.kind, "value") else lib.kind
if kind == AssetLibraryKind.VOICE.value:
return lib
# 自动创建
from sqlalchemy.exc import IntegrityError
project = projects[0]
library = AssetLibrary.create(
project_id=project.id,
name="配音素材库",
kind=AssetLibraryKind.VOICE,
)
try:
return asset_library_repository.create(library)
except IntegrityError:
session = getattr(asset_library_repository, "session", None)
if session is not None:
try:
session.rollback()
except Exception:
pass
for lib in asset_library_repository.find_by_project(project.id):
kind = lib.kind.value if hasattr(lib.kind, "value") else lib.kind
if kind == AssetLibraryKind.VOICE.value:
return lib
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="配音素材库创建失败",
)
def _get_audio_duration(audio_path: Path) -> float:
"""用 ffprobe 获取音频时长(秒)。"""
try:
result = subprocess.run(
[
"ffprobe",
"-v",
"quiet",
"-show_entries",
"format=duration",
"-of",
"csv=p=0",
str(audio_path),
],
capture_output=True,
timeout=10,
)
if result.returncode == 0 and result.stdout.strip():
return float(result.stdout.strip())
except (ValueError, subprocess.TimeoutExpired):
pass
return 0.0
-1
View File
@@ -22,7 +22,6 @@ class VideoItemResponse(BaseModel):
generation_params: dict = Field(default_factory=dict)
download_url: str | None = None
generated_at: str = ""
duplicate_rate: float | None = None
class ListVideosResponse(BaseModel):
+7 -61
View File
@@ -3,7 +3,7 @@
在素材 metadata(assets.classification_result JSON)中持久化已使用的片段时间区间,
供 from-assets 创建片段时避开历史区间,实现跨任务/跨调用的片段去重;
素材可用区间耗尽后进入受控复用:允许有限次数(MAX_RANGE_USE_COUNT)复用最久未用
的历史区间,配合调用方的成片复用占比控制(MAX_REUSE_RATIO = 10%),把任意两条
的历史区间,配合调用方的成片复用占比控制(MAX_REUSE_RATIO = 15%),把任意两条
成片的画面重复率控制在阈值内。
metadata 中的记录字段 ``used_time_ranges``::
@@ -40,14 +40,14 @@ logger = logging.getLogger(__name__)
USED_RANGES_KEY = "used_time_ranges"
# ── 受控复用配置常量 ─────────────────────────────────────────────────────────
MAX_RANGE_USE_COUNT = 2
MAX_RANGE_USE_COUNT = 3
"""单条历史区间最多被使用次数(含首次),达到后不再参与复用。"""
REUSE_RATIO_LIMIT = 0.10
"""单条成片中,单个素材的复用片段累计时长 / 该素材在成片中的总时长上限(10%)。
REUSE_RATIO_LIMIT = 0.15
"""单条成片中,单个素材的复用片段累计时长 / 该素材在成片中的总时长上限(15%)。
超过则该素材不再分配新片段(调用方在轮询分配时跳过)。"""
SEGMENT_EDGE_GAP = 1.5
SEGMENT_EDGE_GAP = 0.3
"""冲突判定边缘间隙(秒):历史区间按 [start-gap, end+gap] 扩边后参与冲突检测,
避免两条片段首尾紧贴导致画面观感重复;记录仍存实际值。"""
@@ -397,12 +397,12 @@ def make_reuse_callback(
db: SQLAlchemy session
asset_durations: 素材 ID -> 总时长(回调需要素材总时长做边界约束)
reused_tracker: 可选的 ``{asset_id: 累计复用时长}``,回调成功返回复用区间时
会把本次片段时长累加进去,供调用方统计成片复用占比(10% 阈值)。
会把本次片段时长累加进去,供调用方统计成片复用占比(15% 阈值)。
assigned_tracker: 可选的 ``{asset_id: 已分配片段总时长}``,配合 ratio_limit
在复用前预判:若复用本片段后占比 (reused + clip_duration) /
(assigned + clip_duration) 超过 ratio_limit,则拒绝复用、返回 None
(保证成片复用占比不超阈值)。
ratio_limit: 单条成片复用时长占比上限,默认 10%。
ratio_limit: 单条成片复用时长占比上限,默认 15%。
Returns:
回调函数 ``(asset_id, clip_duration) -> (start, end) | None``。
@@ -438,57 +438,3 @@ def make_reuse_callback(
return result
return _reuse
def get_asset_recent_use_counts(
db: Session,
asset_ids: list[str],
recent_video_count: int = 5,
) -> dict[str, int]:
"""统计每个素材在最近 N 个不同 plan_id 中的使用次数。
遍历素材 metadata 中的 used_time_ranges,统计有多少个不同的 plan_id(去重),
返回 {asset_id: count}。只统计最近 recent_video_count 个不同 plan_id 的使用次数。
Args:
db: 数据库会话
asset_ids: 素材 ID 列表
recent_video_count: 统计最近多少个不同 plan_id
Returns:
{asset_id: 在最近 recent_video_count 个 plan 中的使用次数}
"""
if not asset_ids:
return {}
result: dict[str, int] = {}
models = db.query(AssetModel).filter(AssetModel.id.in_(asset_ids)).all()
for model in models:
meta = _read_meta(model)
ranges = meta.get(USED_RANGES_KEY) or []
if not ranges:
result[model.id] = 0
continue
# 按 created_at 倒序收集不同 plan_id
sorted_ranges = sorted(
ranges,
key=lambda r: r.get("created_at") or "",
reverse=True,
)
recent_plan_ids: set[str] = set()
for r in sorted_ranges:
plan_id = r.get("plan_id")
if plan_id:
recent_plan_ids.add(plan_id)
if len(recent_plan_ids) >= recent_video_count:
break
result[model.id] = len(recent_plan_ids)
# 未找到的素材计为 0
for aid in asset_ids:
if aid not in result:
result[aid] = 0
return result
@@ -13,7 +13,6 @@
from __future__ import annotations
import logging
import random
from typing import Any, List
from sqlalchemy.orm import Session
@@ -30,11 +29,9 @@ from packages.domain.editing_mode import EditingMode
from packages.domain.plan_generator_utils import (
create_clips_from_configs,
distribute_assets,
extract_scene_points_from_metadata,
generate_default_clips,
map_clip_types_for_mode,
)
from packages.domain.smart_match import SCORE_RANDOM_NOISE_MAX, score_asset
from packages.domain.template_clip_config import TemplateClipConfig
logger = logging.getLogger(__name__)
@@ -221,69 +218,16 @@ class PlanGeneratorService:
) -> None:
"""按 editing_mode 将素材分配到 clips(就地修改,未持久化).
先用 smart_match 评分对素材排序(高分优先),再委托给
plan_generator_utils.distribute_assets 纯函数完成分配。
委托给 plan_generator_utils.distribute_assets 纯函数。
"""
# 预览随机模式:素材顺序已 shuffle,纯随机起点即可,不读 DB 评分/缓存
asset_scene_points: dict[str, list[float]] = {}
if not random_selection:
# 正式生成:smart_match 评分排序(高分优先)+ 场景切换点缓存
if self._asset_repo:
asset_ids = self._sort_assets_by_smart_score(asset_ids)
# 读取素材 metadata 中的场景切换点缓存(后台 SceneChange 检测写入):
# 有缓存的素材片段起点从随机镜头段选取,无缓存走随机起点兜底
asset_scene_points = self._fetch_asset_scene_points(asset_ids)
# 正式生成也随机重排片段顺序(降重,默认开启无开关)
# smart_match 决定选哪些素材,shuffle 只改变分配到 clips 的顺序
asset_ids = list(asset_ids) # 复制避免修改调用方原列表
random.shuffle(asset_ids)
distribute_assets(
clips,
asset_ids,
editing_mode,
random_selection=random_selection,
asset_durations=asset_durations,
asset_scene_points=asset_scene_points,
)
def _fetch_asset_scene_points(self, asset_ids: List[str]) -> dict[str, list[float]]:
"""从素材 metadata 读取场景切换点缓存(无缓存的素材不包含在结果中)。"""
points_map: dict[str, list[float]] = {}
if not self._asset_repo:
return points_map
for asset_id in asset_ids:
asset = self._asset_repo.get(asset_id)
if asset:
points = extract_scene_points_from_metadata(getattr(asset, "metadata", None))
if points:
points_map[asset_id] = points
return points_map
def _sort_assets_by_smart_score(self, asset_ids: List[str]) -> List[str]:
"""按 smart_match 综合评分降序排列素材 ID(注入随机噪声)。
评分高的素材(质量好、时长合适、新鲜、使用次数少)倾向排在前面;
排序时给每个素材的得分注入 0~SCORE_RANDOM_NOISE_MAX 的随机噪声,
使得分接近的素材排名每次浮动,避免一键生成反复选出相同素材组合,
从素材组合层面降低成片查重率。分差大于噪声上限时排名保持稳定。
"""
scored: list[tuple[str, float]] = []
for asset_id in asset_ids:
asset = self._asset_repo.get(asset_id)
if asset:
score, _ = score_asset(asset)
scored.append((asset_id, score))
else:
scored.append((asset_id, 0.0))
# 评分 + 随机噪声后按降序排列
scored.sort(
key=lambda x: x[1] + random.uniform(0.0, SCORE_RANDOM_NOISE_MAX),
reverse=True,
)
return [aid for aid, _ in scored]
def _fetch_asset_durations(self, asset_ids: List[str]) -> dict[str, float]:
"""从数据库获取素材时长信息.
File diff suppressed because one or more lines are too long
+14 -7
View File
@@ -1848,9 +1848,10 @@
},
"node_modules/@testing-library/dom": {
"version": "10.4.1",
"resolved": "https://registry.npmjs.org/@testing-library/dom/-/dom-10.4.1.tgz",
"resolved": "https://registry.npmmirror.com/@testing-library/dom/-/dom-10.4.1.tgz",
"integrity": "sha512-o4PXJQidqJl82ckFaXUeoAW+XysPLauYI43Abki5hABd853iMhitooc6znOnczgbTYmEP6U6/y1ZyKAIsvMKGg==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@babel/code-frame": "^7.10.4",
@@ -1937,9 +1938,10 @@
},
"node_modules/@types/aria-query": {
"version": "5.0.4",
"resolved": "https://registry.npmjs.org/@types/aria-query/-/aria-query-5.0.4.tgz",
"resolved": "https://registry.npmmirror.com/@types/aria-query/-/aria-query-5.0.4.tgz",
"integrity": "sha512-rfT93uj5s0PRL7EzccGMs3brplhcrghnDoV26NqKhCAS1hVo+WdNsPvE/yb6ilfr5hi2MEk6d5EWJTKdxg8jVw==",
"dev": true,
"license": "MIT",
"peer": true
},
"node_modules/@types/babel__core": {
@@ -3111,9 +3113,10 @@
},
"node_modules/dom-accessibility-api": {
"version": "0.5.16",
"resolved": "https://registry.npmjs.org/dom-accessibility-api/-/dom-accessibility-api-0.5.16.tgz",
"resolved": "https://registry.npmmirror.com/dom-accessibility-api/-/dom-accessibility-api-0.5.16.tgz",
"integrity": "sha512-X7BJ2yElsnOJ30pZF4uIIDfBEVgF4XEBxL9Bxhy6dnrm5hkzqmsWHGTiHqRiITNhMyFLyAiWndIJP7Z1NTteDg==",
"dev": true,
"license": "MIT",
"peer": true
},
"node_modules/dunder-proto": {
@@ -4454,9 +4457,10 @@
},
"node_modules/lz-string": {
"version": "1.5.0",
"resolved": "https://registry.npmjs.org/lz-string/-/lz-string-1.5.0.tgz",
"resolved": "https://registry.npmmirror.com/lz-string/-/lz-string-1.5.0.tgz",
"integrity": "sha512-h5bgJWpxJNswbU7qCrV0tIKQCaS3blPDrqKWx+QxzuzL1zGUzij9XCWLrSLsJPu5t+eWA/ycetzYAO5IOMcWAQ==",
"dev": true,
"license": "MIT",
"peer": true,
"bin": {
"lz-string": "bin/bin.js"
@@ -5004,9 +5008,10 @@
},
"node_modules/pretty-format": {
"version": "27.5.1",
"resolved": "https://registry.npmjs.org/pretty-format/-/pretty-format-27.5.1.tgz",
"resolved": "https://registry.npmmirror.com/pretty-format/-/pretty-format-27.5.1.tgz",
"integrity": "sha512-Qb1gy5OrP5+zDf2Bvnzdl3jsTf1qXVMazbvCoKhtKqVs4/YK4ozX4gKQJJVyNe+cajNPn0KoC0MC3FUmaHWEmQ==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"ansi-regex": "^5.0.1",
@@ -5019,9 +5024,10 @@
},
"node_modules/pretty-format/node_modules/ansi-styles": {
"version": "5.2.0",
"resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-5.2.0.tgz",
"resolved": "https://registry.npmmirror.com/ansi-styles/-/ansi-styles-5.2.0.tgz",
"integrity": "sha512-Cxwpt2SfTzTtXcfOlzGEee8O+c+MmUgGrNiBcXnuWxuFJHe6a5Hz7qwhwe5OgaSYI0IJvkLqWX1ASG+cJOkEiA==",
"dev": true,
"license": "MIT",
"peer": true,
"engines": {
"node": ">=10"
@@ -5729,9 +5735,10 @@
},
"node_modules/react-is": {
"version": "17.0.2",
"resolved": "https://registry.npmjs.org/react-is/-/react-is-17.0.2.tgz",
"resolved": "https://registry.npmmirror.com/react-is/-/react-is-17.0.2.tgz",
"integrity": "sha512-w2GsyukL62IJnlaff/nRegPQR94C/XXamvMWmSHRJ4y7Ts/4ocGRmTHvOs8PSE6pB3dWOrD/nueuU5sduBsQ4w==",
"dev": true,
"license": "MIT",
"peer": true
},
"node_modules/react-refresh": {
-1
View File
@@ -71,5 +71,4 @@ export interface VideoItem {
generation_params: Record<string, unknown>
download_url: string
generated_at: string
duplicate_rate?: number
}
+2 -1
View File
@@ -29,6 +29,7 @@ export function mapVideoToProductItem(video: VideoItem): ProductItem {
// 后端字段名为 generated_at,映射为 created_at 供前端统一使用
created_at: video.generated_at,
updated_at: video.generated_at,
duplicate_rate: video.duplicate_rate,
// 后端 /videos 接口暂无 duplicate_rate 字段
duplicate_rate: undefined,
}
}
-1
View File
@@ -28,5 +28,4 @@ export {
deleteTTSJob,
getTtsVoices,
previewTts,
extractVideoVoice,
} from "./jobs"
-53
View File
@@ -70,56 +70,3 @@ export const previewTts = async (data: TTSPreviewRequest): Promise<TTSPreviewRes
const response = await apiClient.post<TTSPreviewResponse>("/tts/preview", data)
return response.data
}
/**
* 从视频中提取配音(上传视频 → 后端提取人声 → 保存到配音素材库)
* 支持 mp4/mov/webm 格式
*/
export const extractVideoVoice = async (
file: File,
onProgress?: (percent: number) => void,
): Promise<{ asset_id: string; duration: number }> => {
const formData = new FormData()
formData.append("file", file)
return new Promise((resolve, reject) => {
const xhr = new XMLHttpRequest()
xhr.open("POST", "/api/v1/tts/extract-video-voice")
// 携带认证 token(从 localStorage 获取,与 apiClient 拦截器一致)
const token = localStorage.getItem("access_token")
if (token) {
xhr.setRequestHeader("Authorization", `Bearer ${token}`)
}
xhr.timeout = 10 * 60 * 1000 // 10 分钟超时
xhr.upload.onprogress = (e) => {
if (e.lengthComputable && onProgress) {
onProgress(Math.round((e.loaded / e.total) * 100))
}
}
xhr.onload = () => {
if (xhr.status >= 200 && xhr.status < 300) {
try {
resolve(JSON.parse(xhr.responseText))
} catch {
reject(new Error("服务器返回数据解析失败"))
}
} else {
try {
const err = JSON.parse(xhr.responseText)
reject(new Error(err.detail || err.message || `提取失败: HTTP ${xhr.status}`))
} catch {
reject(new Error(`提取失败: HTTP ${xhr.status}`))
}
}
}
xhr.onerror = () => reject(new Error("网络错误,请检查网络连接"))
xhr.ontimeout = () => reject(new Error("上传超时(10分钟),请检查网络或尝试更小的文件"))
xhr.send(formData)
})
}
@@ -297,7 +297,7 @@ const CloneModal: React.FC<CloneModalProps> = ({ open, onClose, onSuccess }) =>
buttonSize="sm"
onClick={() => {
handleClose()
navigate("/app/voices?tab=material&upload=1")
navigate("/app/voice-materials")
}}
>
去配音库上传
@@ -3,11 +3,9 @@ import { useQuery } from "@tanstack/react-query"
import {
getAssetLibraries,
getAssets,
ensureDefaultLibrary,
type AssetLibraryItem,
type AssetItem as ApiAssetItem,
} from "@/api/assets"
import { getOrCreateDefaultProject } from "@/api/projects"
import { mapLibrary, mapAsset, type AssetItem, type LibraryItem } from "../types"
/**
@@ -18,18 +16,7 @@ export function useAssetsData() {
/* ── 视频库列表查询 ── */
const { data: apiLibraries = [], isLoading: libLoading } = useQuery<AssetLibraryItem[], Error>({
queryKey: ["asset-libraries"],
queryFn: async () => {
const libs = await getAssetLibraries()
// 如果没有 video 类型的库,自动创建默认视频素材库(与 useVoiceMaterials 保持一致)
const hasVideoLib = libs.some((lib) => lib.kind === "video")
if (!hasVideoLib) {
const project = await getOrCreateDefaultProject()
await ensureDefaultLibrary({ project_id: project.id, kind: "video" })
// 创建后重新拉取最新列表
return getAssetLibraries()
}
return libs
},
queryFn: getAssetLibraries,
staleTime: 60_000,
})
+1 -4
View File
@@ -338,15 +338,12 @@ const GeneratePage: React.FC = () => {
size: titleSettings.size,
font: titleSettings.font,
color: titleSettings.color,
position: titleSettings.position as "top" | "center" | "bottom" | "custom",
position: titleSettings.position as "top" | "center" | "bottom",
bold: titleSettings.bold,
italic: titleSettings.italic,
stroke: titleSettings.stroke,
shadow: titleSettings.shadow,
posX: titleSettings.posX,
posY: titleSettings.posY,
}}
onTitlePositionChange={styleUpdaters.updateTitlePosition}
/>
)}
{currentStep >= 5 && generated && finalVideo && (
@@ -32,15 +32,12 @@ interface FrontendPreviewPlayerProps {
size: number
font: string
color: string
position: "top" | "center" | "bottom" | "custom"
position: "top" | "center" | "bottom"
bold?: boolean
italic?: boolean
stroke?: boolean
shadow?: boolean
posX?: number | null
posY?: number | null
}
onTitlePositionChange?: (posX: number, posY: number) => void
}
function formatTime(seconds: number): string {
@@ -108,7 +105,6 @@ const FrontendPreviewPlayer: React.FC<FrontendPreviewPlayerProps> = ({
serverClips,
voiceAudioUrl,
titleSettings,
onTitlePositionChange,
}) => {
const segments = useMemo(
() => buildPlaybackSegments(assets, template, serverClips),
@@ -130,69 +126,6 @@ const FrontendPreviewPlayer: React.FC<FrontendPreviewPlayerProps> = ({
return { width: 1080, height: 1920 }
}
})()
const customTitleXPct =
titleSettings?.posX != null && playRes.width > 0
? (titleSettings.posX / playRes.width) * 100
: null
const customTitleYPct =
titleSettings?.posY != null && playRes.height > 0
? (titleSettings.posY / playRes.height) * 100
: null
// ── 拖拽状态(用 ref 避免在每帧渲染中触发重渲染)──
const draggingTitleRef = useRef(false)
const titleDragRef = useRef<HTMLDivElement>(null)
const handleTitlePointerDown = useCallback(
(e: React.PointerEvent<HTMLDivElement>) => {
if (!onTitlePositionChange || !playerContainerRef.current) return
e.preventDefault()
e.stopPropagation()
;(e.target as Element).setPointerCapture(e.pointerId)
draggingTitleRef.current = true
;(e.currentTarget as HTMLDivElement).style.cursor = "grabbing"
},
[onTitlePositionChange],
)
const handleTitlePointerMove = useCallback((e: React.PointerEvent<HTMLDivElement>) => {
if (!draggingTitleRef.current || !playerContainerRef.current) return
e.preventDefault()
e.stopPropagation()
// 拖拽过程中直接修改 DOM,不触发 React 渲染(避免频繁重渲染导致换行)
if (titleDragRef.current) {
const rect = playerContainerRef.current.getBoundingClientRect()
const relX = Math.max(0, Math.min(rect.width, e.clientX - rect.left))
const relY = Math.max(0, Math.min(rect.height, e.clientY - rect.top))
const xpct = (relX / rect.width) * 100
const ypct = (relY / rect.height) * 100
titleDragRef.current.style.left = `${xpct}%`
titleDragRef.current.style.top = `${ypct}%`
}
}, [])
const handleTitlePointerUp = useCallback(
(e: React.PointerEvent<HTMLDivElement>) => {
if (!draggingTitleRef.current) return
draggingTitleRef.current = false
// 拖拽结束时才调用 onTitlePositionChange 保存最终位置
if (onTitlePositionChange && playerContainerRef.current) {
const rect = playerContainerRef.current.getBoundingClientRect()
const relX = Math.max(0, Math.min(rect.width, e.clientX - rect.left))
const relY = Math.max(0, Math.min(rect.height, e.clientY - rect.top))
const posX = Math.round((relX / rect.width) * playRes.width)
const posY = Math.round((relY / rect.height) * playRes.height)
onTitlePositionChange(posX, posY)
}
;(e.currentTarget as HTMLDivElement).style.cursor = "grab"
try {
if ((e.currentTarget as Element).hasPointerCapture(e.pointerId)) {
;(e.currentTarget as Element).releasePointerCapture(e.pointerId)
}
} catch {
/* ignore */
}
},
[onTitlePositionChange, playRes.width, playRes.height],
)
const playerContainerRef = useRef<HTMLDivElement>(null)
const [containerHeight, setContainerHeight] = useState(0)
useEffect(() => {
@@ -591,36 +524,15 @@ const FrontendPreviewPlayer: React.FC<FrontendPreviewPlayerProps> = ({
<div
style={{
position: "absolute",
...(customTitleXPct != null && customTitleYPct != null
? {
left: `${customTitleXPct}%`,
top: `${customTitleYPct}%`,
transform: "translate(-50%, -50%)",
textAlign: "center" as const,
}
: {
left: `${titleSidePct}%`,
right: `${titleSidePct}%`,
textAlign: "center" as const,
...(titleSettings.position === "top"
? { top: `${titleTopPct}%` }
: titleSettings.position === "center"
? { top: "50%", transform: "translateY(-50%)" }
: { bottom: `${titleBottomPct}%` }),
}),
pointerEvents: "auto",
cursor: onTitlePositionChange ? "grab" : "default",
touchAction: "none",
userSelect: "none",
WebkitUserSelect: "none",
padding: "8px 12px",
boxShadow: "inset 0 0 0 16px transparent",
left: `${titleSidePct}%`,
right: `${titleSidePct}%`,
textAlign: "center",
...(titleSettings.position === "top"
? { top: `${titleTopPct}%` }
: titleSettings.position === "center"
? { top: "50%", transform: "translateY(-50%)" }
: { bottom: `${titleBottomPct}%` }),
}}
ref={titleDragRef}
onPointerDown={handleTitlePointerDown}
onPointerMove={handleTitlePointerMove}
onPointerUp={handleTitlePointerUp}
onPointerCancel={handleTitlePointerUp}
>
<span
style={{
@@ -134,7 +134,7 @@ const Step5VoiceSelect: React.FC<Step5VoiceSelectProps> = ({
/** 跳转到配音库上传 */
const handleGoToUpload = useCallback(() => {
navigate("/app/voices?tab=material&upload=1")
navigate("/app/voices")
}, [navigate])
// 加载中状态
@@ -23,7 +23,7 @@ const MaterialModeTabs: React.FC<MaterialModeTabsProps> = ({ mode, onModeChange
onClick={() => onModeChange("auto")}
type="button"
>
AI智能匹配
选择视频库自动匹配
</button>
</div>
)
@@ -1,7 +1,5 @@
/**
* 标题预设样式网格
* 双图层渲染:底层=描边轮廓(text-shadow模拟),上层=填充色
* 避免 -webkit-text-stroke 在 Chromium 中吞掉填充色的问题
*/
import React from "react"
import { getFontFamily } from "../../constants"
@@ -9,10 +7,7 @@ import { getFontFamily } from "../../constants"
interface TitlePresetItem {
key: string
label: string
previewStyle: React.CSSProperties & {
_strokeColor?: string
_strokeWidth?: number
}
previewStyle: React.CSSProperties
}
interface TitlePresetsGridProps {
@@ -22,32 +17,6 @@ interface TitlePresetsGridProps {
fontFamily?: string
}
/**
* 用 text-shadow 模拟描边轮廓(8方向 + 4对角 = 12层阴影)
*/
function buildStrokeShadow(color: string, width: number): string {
const w = width
const parts: string[] = []
// 4 cardinal directions
parts.push(`${w}px 0 ${color}`)
parts.push(`${-w}px 0 ${color}`)
parts.push(`0 ${w}px ${color}`)
parts.push(`0 ${-w}px ${color}`)
// 4 diagonal directions
const d = Math.round(w * 0.71 * 10) / 10 // 0.71 ≈ sqrt(2)/2
parts.push(`${d}px ${d}px ${color}`)
parts.push(`${-d}px ${d}px ${color}`)
parts.push(`${d}px ${-d}px ${color}`)
parts.push(`${-d}px ${-d}px ${color}`)
// 4 extra mid-points for smoother stroke
const h = Math.round(w * 0.5 * 10) / 10
parts.push(`${w}px ${h}px ${color}`)
parts.push(`${w}px ${-h}px ${color}`)
parts.push(`${-w}px ${h}px ${color}`)
parts.push(`${-w}px ${-h}px ${color}`)
return parts.join(", ")
}
const TitlePresetsGrid: React.FC<TitlePresetsGridProps> = ({
presets,
activePreset,
@@ -58,53 +27,20 @@ const TitlePresetsGrid: React.FC<TitlePresetsGridProps> = ({
<div className="xx-title-presets-grid">
{presets.map((p) => {
const isActive = activePreset === p.key
const { _strokeColor, _strokeWidth, ...fillStyle } = p.previewStyle
const ff = getFontFamily(fontFamily || "思源黑体")
// 底层:描边轮廓(用 text-shadow 模拟粗描边)
const strokeStyle: React.CSSProperties = {
color: _strokeColor || "transparent",
textShadow:
_strokeColor && _strokeWidth
? buildStrokeShadow(_strokeColor, _strokeWidth)
: undefined,
fontWeight: fillStyle.fontWeight,
fontSize: fillStyle.fontSize,
lineHeight: 1,
}
// 上层:仅填充色 + 可选 textShadow(发光/投影效果)
const topStyle: React.CSSProperties = {
color: fillStyle.color,
textShadow: fillStyle.textShadow,
fontWeight: fillStyle.fontWeight,
fontSize: fillStyle.fontSize,
lineHeight: 1,
}
return (
<button
key={p.key}
className={`xx-title-preset-card${isActive ? " active" : ""}`}
onClick={() => onApply(p.key)}
title={p.label}
>
<span className="xx-title-preset-preview-text" style={{ position: "relative" }}>
{/* 底层:描边轮廓 */}
<span
aria-hidden
style={{
...strokeStyle,
fontFamily: ff,
position: "absolute",
top: 0,
left: 0,
}}
>
T
</span>
{/* 上层:填充色 */}
<span style={{ ...topStyle, fontFamily: ff, position: "relative" }}>T</span>
<span
className="xx-title-preset-preview-text"
style={{ ...p.previewStyle, fontFamily: getFontFamily(fontFamily || "思源黑体") }}
>
标题
</span>
<span className="xx-title-preset-card-label">{p.label}</span>
</button>
)
})}
+16 -219
View File
@@ -42,7 +42,6 @@ export const POSITION_OPTIONS = [
{ value: "top", label: "顶部" },
{ value: "center", label: "居中" },
{ value: "bottom", label: "底部" },
{ value: "custom", label: "自定义" },
]
/* ── 标题字体选项 ── */
@@ -78,11 +77,10 @@ export const TITLE_PRESETS = [
label: "经典白字",
style: { size: 28, color: "#ffffff", bold: true, italic: false, stroke: true, shadow: false },
previewStyle: {
color: "#ffffff",
_strokeColor: "#000000",
_strokeWidth: 2,
fontWeight: 700,
fontSize: "32px",
color: "#ffffff",
WebkitTextStroke: "1px #000000",
fontSize: "20px",
},
},
{
@@ -90,54 +88,45 @@ export const TITLE_PRESETS = [
label: "黑金质感",
style: { size: 32, color: "#d4a843", bold: true, italic: false, stroke: false, shadow: true },
previewStyle: {
color: "#d4a843",
fontWeight: 700,
fontSize: "32px",
color: "#d4a843",
textShadow: "1px 1px 3px rgba(0,0,0,0.8)",
fontSize: "20px",
},
},
{
key: "fresh_minimal",
label: "清新简约",
style: { size: 24, color: "#333333", bold: false, italic: false, stroke: false, shadow: false },
previewStyle: {
color: "#e8e8e8",
fontWeight: 400,
fontSize: "32px",
},
previewStyle: { fontWeight: 400, color: "#333333", fontSize: "18px" },
},
{
key: "variety_show",
label: "综艺花字",
style: { size: 36, color: "#ff4081", bold: true, italic: false, stroke: true, shadow: true },
previewStyle: {
color: "#ff4081",
_strokeColor: "#ffffff",
_strokeWidth: 2,
fontWeight: 900,
fontSize: "32px",
color: "#ff4081",
WebkitTextStroke: "1.5px #ffffff",
textShadow: "2px 2px 4px rgba(0,0,0,0.5)",
fontSize: "22px",
},
},
{
key: "business",
label: "商务极简",
style: { size: 24, color: "#1a1a1a", bold: false, italic: false, stroke: false, shadow: false },
previewStyle: {
color: "#e0e0e0",
fontWeight: 400,
fontSize: "32px",
},
previewStyle: { fontWeight: 400, color: "#1a1a1a", fontSize: "17px" },
},
{
key: "retro_film",
label: "复古胶片",
style: { size: 28, color: "#e8d5b7", bold: false, italic: false, stroke: false, shadow: true },
previewStyle: {
color: "#e8d5b7",
fontWeight: 400,
fontSize: "32px",
color: "#e8d5b7",
textShadow: "2px 2px 6px rgba(0,0,0,0.7)",
fontSize: "18px",
},
},
{
@@ -145,10 +134,10 @@ export const TITLE_PRESETS = [
label: "霓虹发光",
style: { size: 32, color: "#00e5ff", bold: true, italic: false, stroke: false, shadow: true },
previewStyle: {
color: "#00e5ff",
fontWeight: 700,
fontSize: "32px",
color: "#00e5ff",
textShadow: "0 0 4px #00e5ff, 0 0 8px #00e5ff, 0 0 16px rgba(0,229,255,0.5)",
fontSize: "20px",
},
},
{
@@ -156,202 +145,10 @@ export const TITLE_PRESETS = [
label: "手写字",
style: { size: 28, color: "#333333", bold: false, italic: false, stroke: false, shadow: true },
previewStyle: {
color: "#e0e0e0",
fontWeight: 400,
fontSize: "32px",
color: "#333333",
textShadow: "1px 1px 2px rgba(0,0,0,0.3)",
},
},
{
key: "outline_yellow",
label: "黄色描边",
style: { size: 28, color: "#ffd54f", bold: true, italic: false, stroke: true, shadow: false },
previewStyle: {
color: "#ffd54f",
_strokeColor: "#000000",
_strokeWidth: 2,
fontWeight: 700,
fontSize: "32px",
},
},
{
key: "outline_pink",
label: "粉色描边",
style: { size: 28, color: "#ff80ab", bold: true, italic: false, stroke: true, shadow: false },
previewStyle: {
color: "#ff80ab",
_strokeColor: "#000000",
_strokeWidth: 2,
fontWeight: 700,
fontSize: "32px",
},
},
{
key: "outline_blue",
label: "蓝色描边",
style: { size: 28, color: "#82b1ff", bold: true, italic: false, stroke: true, shadow: false },
previewStyle: {
color: "#82b1ff",
_strokeColor: "#000000",
_strokeWidth: 2,
fontWeight: 700,
fontSize: "32px",
},
},
{
key: "outline_green",
label: "绿色描边",
style: { size: 28, color: "#69f0ae", bold: true, italic: false, stroke: true, shadow: false },
previewStyle: {
color: "#69f0ae",
_strokeColor: "#000000",
_strokeWidth: 2,
fontWeight: 700,
fontSize: "32px",
},
},
{
key: "outline_gray",
label: "灰色描边",
style: { size: 28, color: "#bdbdbd", bold: true, italic: false, stroke: true, shadow: false },
previewStyle: {
color: "#bdbdbd",
_strokeColor: "#000000",
_strokeWidth: 2,
fontWeight: 700,
fontSize: "32px",
},
},
{
key: "bg_white",
label: "白底黑字",
style: { size: 28, color: "#1a1a1a", bold: true, italic: false, stroke: false, shadow: false },
previewStyle: {
color: "#1a1a1a",
fontWeight: 700,
fontSize: "32px",
background: "#ffffff",
borderRadius: "4px",
padding: "2px 6px",
},
},
{
key: "bg_yellow",
label: "黄底黑字",
style: { size: 28, color: "#1a1a1a", bold: true, italic: false, stroke: false, shadow: false },
previewStyle: {
color: "#1a1a1a",
fontWeight: 700,
fontSize: "32px",
background: "#ffd54f",
borderRadius: "4px",
padding: "2px 6px",
},
},
{
key: "bg_pink",
label: "粉底黑字",
style: { size: 28, color: "#1a1a1a", bold: true, italic: false, stroke: false, shadow: false },
previewStyle: {
color: "#1a1a1a",
fontWeight: 700,
fontSize: "32px",
background: "#ff80ab",
borderRadius: "4px",
padding: "2px 6px",
},
},
{
key: "bg_red",
label: "红底白字",
style: { size: 28, color: "#ffffff", bold: true, italic: false, stroke: false, shadow: false },
previewStyle: {
color: "#ffffff",
fontWeight: 700,
fontSize: "32px",
background: "#ef5350",
borderRadius: "4px",
padding: "2px 6px",
},
},
{
key: "neon_orange",
label: "橙色发光",
style: { size: 32, color: "#ff9100", bold: true, italic: false, stroke: false, shadow: true },
previewStyle: {
color: "#ff9100",
fontWeight: 700,
fontSize: "32px",
textShadow: "0 0 4px #ff9100, 0 0 8px #ff9100, 0 0 16px rgba(255,145,0,0.5)",
},
},
{
key: "neon_purple",
label: "紫色发光",
style: { size: 32, color: "#d500f9", bold: true, italic: false, stroke: false, shadow: true },
previewStyle: {
color: "#d500f9",
fontWeight: 700,
fontSize: "32px",
textShadow: "0 0 4px #d500f9, 0 0 8px #d500f9, 0 0 16px rgba(213,0,249,0.5)",
},
},
{
key: "bordered_white",
label: "白字绿框",
style: { size: 28, color: "#ffffff", bold: true, italic: false, stroke: false, shadow: false },
previewStyle: {
color: "#ffffff",
fontWeight: 700,
fontSize: "32px",
background: "#1a1a1a",
border: "2px solid #69f0ae",
borderRadius: "4px",
padding: "2px 6px",
},
},
{
key: "gradient_warm",
label: "暖色渐变",
style: { size: 32, color: "#ff6d00", bold: true, italic: false, stroke: false, shadow: true },
previewStyle: {
color: "#ff6d00",
fontWeight: 700,
fontSize: "32px",
textShadow: "0 0 6px rgba(255,109,0,0.6), 1px 1px 2px rgba(0,0,0,0.5)",
},
},
{
key: "gradient_cool",
label: "冷色渐变",
style: { size: 32, color: "#00b0ff", bold: true, italic: false, stroke: false, shadow: true },
previewStyle: {
color: "#00b0ff",
fontWeight: 700,
fontSize: "32px",
textShadow: "0 0 6px rgba(0,176,255,0.6), 1px 1px 2px rgba(0,0,0,0.5)",
},
},
{
key: "shadow_deep",
label: "深影白字",
style: { size: 28, color: "#ffffff", bold: true, italic: false, stroke: false, shadow: true },
previewStyle: {
color: "#ffffff",
fontWeight: 700,
fontSize: "32px",
textShadow: "2px 2px 4px rgba(0,0,0,0.8), 0 0 8px rgba(0,0,0,0.4)",
},
},
{
key: "soft_gold",
label: "柔光金",
style: { size: 28, color: "#ffd54f", bold: true, italic: false, stroke: false, shadow: true },
previewStyle: {
color: "#ffd54f",
fontWeight: 700,
fontSize: "32px",
textShadow: "0 0 6px rgba(255,213,79,0.5), 1px 1px 2px rgba(0,0,0,0.4)",
fontSize: "20px",
},
},
]
+23 -13
View File
@@ -1733,40 +1733,50 @@
/* 标题预设卡片网格 */
.xx-title-presets-grid {
display: grid;
grid-template-columns: repeat(6, 52px);
gap: 1px;
grid-template-columns: repeat(4, 1fr);
gap: 10px;
}
.xx-title-preset-card {
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
width: 52px;
height: 52px;
padding: 0;
background: #404040;
padding: 14px 8px;
background: var(--bg-secondary);
border: 2px solid transparent;
border-radius: 8px;
border-radius: var(--radius-sm);
cursor: pointer;
transition: all 0.15s;
text-align: center;
}
.xx-title-preset-card:hover {
border-color: #666;
background: #4d4d4d;
border-color: var(--primary-200);
background: var(--bg-primary);
}
.xx-title-preset-card.active {
border-color: #409eff;
background: #4d4d4d;
border-color: var(--primary-color);
background: var(--primary-50);
}
.xx-title-preset-preview-text {
font-size: 32px;
line-height: 1;
line-height: 1.4;
margin-bottom: 6px;
user-select: none;
}
.xx-title-preset-card-label {
font-size: 11px;
color: var(--text-secondary);
}
.xx-title-preset-card.active .xx-title-preset-card-label {
color: var(--primary-color);
font-weight: 500;
}
/* 样式按钮组 */
.xx-style-btns {
display: flex;
@@ -20,14 +20,13 @@ const SECONDS_PER_ASSET = 15
/**
* 根据模板 segments 计算所需素材数量上限。
* 取每个 segment 的 duration_min 之和作为目标视频总时长,
* 再按 15 秒/素材估算需要多少个素材,且保证不少于片段数(每个片段至少 1 个素材);
* 结果钳制到 [1, 200] 区间(后端 limit 上限 200)。
* 再按 15 秒/素材估算需要多少个素材;结果钳制到 [1, 200] 区间(后端 limit 上限 200)。
*/
function computeLimitFromSegments(segments?: TemplateSegment[]): number {
if (!segments || segments.length === 0) return DEFAULT_LIMIT
const totalSeconds = segments.reduce((sum, seg) => sum + (seg.duration_min || 0), 0)
if (totalSeconds <= 0) return DEFAULT_LIMIT
const limit = Math.max(segments.length, Math.ceil(totalSeconds / SECONDS_PER_ASSET))
const limit = Math.ceil(totalSeconds / SECONDS_PER_ASSET)
return Math.max(1, Math.min(limit, 200))
}
@@ -233,7 +233,7 @@ export function useCanvasPlayer(
fontSize: number
fontFamily: string
color: string
position: "top" | "center" | "bottom" | "custom"
position: "top" | "center" | "bottom"
bold?: boolean
stroke?: boolean
shadow?: boolean
@@ -27,8 +27,6 @@ const DEFAULT_TITLE_SETTINGS: TitleSettings = {
stroke: true,
shadow: false,
color: "#ffffff",
posX: null,
posY: null,
}
export interface GenerateFormState {
@@ -117,14 +117,6 @@ export function useGenerateVideo(props: UseGenerateVideoProps) {
font_size: props.titleSettings.size,
font_color: props.titleSettings.color,
position: props.titleSettings.position,
...(props.titleSettings.position === "custom" &&
props.titleSettings.posX != null &&
props.titleSettings.posY != null
? {
pos_x: Math.round(props.titleSettings.posX),
pos_y: Math.round(props.titleSettings.posY),
}
: {}),
bold: props.titleSettings.bold,
stroke: props.titleSettings.stroke,
shadow: props.titleSettings.shadow,
@@ -46,16 +46,7 @@ export function useTitleStyleUpdaters({
const updatePosition = useCallback(
(position: string) => {
// 切回三档预设时清掉自定义坐标
onTitleSettingsChange({ ...titleSettings, position, posX: null, posY: null })
},
[titleSettings, onTitleSettingsChange],
)
/** 拖拽更新自定义位置(由预览播放器调用) */
const updateTitlePosition = useCallback(
(posX: number, posY: number) => {
onTitleSettingsChange({ ...titleSettings, position: "custom", posX, posY })
onTitleSettingsChange({ ...titleSettings, position })
},
[titleSettings, onTitleSettingsChange],
)
@@ -120,7 +111,6 @@ export function useTitleStyleUpdaters({
updateTitle,
toggleAiAutoSelect,
updatePosition,
updateTitlePosition,
updateFont,
updateSize,
updateColor,
-3
View File
@@ -16,9 +16,6 @@ export interface TitleSettings {
stroke: boolean
shadow: boolean
color: string
/** 自由位置坐标(PlayRes 像素),仅当 position="custom" 时有效 */
posX: number | null
posY: number | null
}
/* ── 智能匹配结果 ── */
@@ -11,7 +11,7 @@ import {
} from "@ant-design/icons"
import type { ProductItem } from "../types"
import { statusConfig, reviewStatusConfig } from "../constants"
import { formatTime, formatSize } from "../utils"
import { formatTime } from "../utils"
interface ProductCardProps {
product: ProductItem
@@ -210,20 +210,11 @@ export const ProductCard: React.FC<ProductCardProps> = ({
<span className={`xx-product-status ${st.className}`}>{st.text}</span>
<span className="xx-product-date">{product.date}</span>
</div>
<div className="xx-product-meta-grid">
<span className="xx-product-meta-item">分辨率:{product.resolution || "-"}</span>
<span className="xx-product-meta-item">
时长:{product.duration > 0 ? formatTime(product.duration) : "-"}
{product.duplicateRate > 0 && (
<span className={`xx-product-dup-rate ${dupClass}`}>
查重率:{product.duplicateRate.toFixed(1)}%
</span>
<span className="xx-product-meta-item">大小:{formatSize(product.fileSize)}</span>
<span
className={`xx-product-meta-item xx-product-dup-rate${
product.duplicateRate > 0 ? ` ${dupClass}` : ""
}`}
>
查重率:{product.duplicateRate != null ? `${product.duplicateRate.toFixed(1)}%` : "-"}
</span>
</div>
)}
</div>
{/* 操作按钮 */}
+29 -35
View File
@@ -91,6 +91,8 @@
height: 18px;
border: 2px solid var(--border-color);
border-radius: var(--radius-xs);
display: grid;
place-items: center;
transition: var(--transition-all);
background: var(--bg-primary);
flex-shrink: 0;
@@ -135,7 +137,7 @@
============================================================ */
.xx-products-grid {
display: grid;
grid-template-columns: repeat(auto-fill, minmax(220px, 1fr));
grid-template-columns: repeat(4, 1fr);
gap: var(--space-md);
}
@@ -185,6 +187,8 @@
height: 22px;
border: 2px solid rgba(255, 255, 255, 0.8);
border-radius: var(--radius-xs);
display: grid;
place-items: center;
background: rgba(0, 0, 0, 0.3);
backdrop-filter: blur(4px);
cursor: pointer;
@@ -258,8 +262,11 @@
.xx-product-thumb {
position: relative;
overflow: hidden;
display: grid;
place-items: center;
color: var(--text-inverse);
background: var(--color-gray-950);
max-height: 320px;
}
.xx-product-thumb-bg {
@@ -269,11 +276,11 @@
background-position: center;
}
/* 内联视频播放器(cover 填满容器,竖屏视频不留左右空白) */
/* 内联视频播放器 */
.xx-product-thumb-video {
width: 100%;
height: 100%;
object-fit: cover;
object-fit: contain;
display: block;
}
@@ -282,28 +289,22 @@
}
.xx-product-play {
position: absolute;
top: 50%;
left: 50%;
transform: translate(-50%, -50%);
z-index: 2;
width: 52px;
height: 52px;
border-radius: 50%;
position: relative;
z-index: 1;
width: 44px;
height: 44px;
border-radius: var(--radius-full);
background: rgba(0, 0, 0, 0.45);
backdrop-filter: blur(4px);
display: flex;
align-items: center;
justify-content: center;
font-size: 28px;
color: #fff;
transition: all 0.2s;
cursor: pointer;
display: grid;
place-items: center;
font-size: var(--font-size-md);
transition: var(--transition-all);
}
.xx-product-card:hover .xx-product-play {
background: var(--primary-color);
transform: translate(-50%, -50%) scale(1.1);
transform: scale(1.1);
}
/* 时长标签 */
@@ -323,7 +324,7 @@
/* 卡片信息区 */
.xx-product-info {
padding: 8px 10px;
padding: 14px;
display: flex;
flex-direction: column;
gap: var(--space-sm);
@@ -346,21 +347,6 @@
gap: var(--space-xs);
}
/* 卡片信息网格:分辨率/时长 一行,大小/查重率 一行 */
.xx-product-meta-grid {
display: grid;
grid-template-columns: 1fr 1fr;
gap: 4px var(--space-sm);
}
.xx-product-meta-item {
font-size: var(--font-size-xs);
color: var(--text-secondary);
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
/* 状态标签 */
.xx-product-status {
padding: 2px 10px;
@@ -505,6 +491,8 @@
z-index: 1000;
background: rgba(0, 0, 0, 0.75);
backdrop-filter: blur(8px);
display: grid;
place-items: center;
animation: player-fade-in 0.25s ease-out;
}
@@ -543,6 +531,8 @@
background: var(--color-gray-950);
aspect-ratio: 9 / 16;
max-height: 60vh;
display: grid;
place-items: center;
}
.xx-player-video-wrap video {
@@ -564,6 +554,8 @@
backdrop-filter: blur(4px);
color: var(--text-inverse);
font-size: 28px;
display: grid;
place-items: center;
cursor: pointer;
border: none;
transition: var(--transition-all);
@@ -660,6 +652,8 @@
backdrop-filter: blur(4px);
color: var(--text-inverse);
font-size: var(--font-size-md);
display: grid;
place-items: center;
cursor: pointer;
border: none;
transition: var(--transition-all);
@@ -1,11 +1,6 @@
import { useMemo, useEffect } from "react"
import { useQuery, useMutation, useQueryClient } from "@tanstack/react-query"
import {
getAssetsByKind,
getAssetLibraries,
createAssetLibrary,
type AssetItem,
} from "@/api/assets"
import { getAssetsByKind, getAssetLibraries, createAssetLibrary } from "@/api/assets"
import { type VoiceMaterial, mapAssetToMaterial } from "../../types"
interface UseVoiceMaterialDataOptions {
@@ -49,15 +44,6 @@ export function useVoiceMaterialData({ keyword, gender, tagIds }: UseVoiceMateri
queryKey: ["assets", "voice", { keyword, gender, tag_ids: tagIds }],
queryFn: () => getAssetsByKind("voice", { keyword, gender, tag_ids: tagIds }),
staleTime: 30_000,
// 列表中存在上传中/处理中素材时每 3s 轮询;全部就绪后自动停止
refetchInterval: (query) => {
const items = (query.state.data as AssetItem[] | undefined) ?? []
const processing = items.some((a) => {
const st = a.status ?? ""
return st === "uploading" || st === "ingesting" || st === "processing" || st === "pending"
})
return processing ? 3000 : false
},
})
const materials: VoiceMaterial[] = useMemo(() => assets.map(mapAssetToMaterial), [assets])
+1 -1
View File
@@ -41,7 +41,7 @@ export const mapAssetToMaterial = (asset: AssetItem): VoiceMaterial => {
tagIds: Array.isArray(asset.tag_ids) ? asset.tag_ids : [],
fileName: asset.storage_key?.split("/").pop() || asset.name,
fileSize: asset.file_size || 0,
duration: asset.duration || (meta.duration as number) || 0,
duration: (meta.duration as number) || 0,
mimeType: asset.mime_type || "audio/mpeg",
createdAt: asset.created_at || new Date().toISOString(),
fileUrl: asset.file_url,
+3 -56
View File
@@ -14,14 +14,8 @@
* 弹窗集合 → components/VoiceModals
* Toast 提示 → components/VoiceToasts
*/
import React, { useCallback, useEffect, useState } from "react"
import { useSearchParams } from "react-router-dom"
import {
UploadOutlined,
AudioOutlined,
RobotOutlined,
VideoCameraOutlined,
} from "@ant-design/icons"
import React, { useCallback, useState } from "react"
import { UploadOutlined, AudioOutlined, RobotOutlined } from "@ant-design/icons"
import { Button } from "@/components/ui"
import PageHead from "@/components/layout/PageHead"
import { type AssetItem } from "@/api/assets"
@@ -40,8 +34,6 @@ import { useTtsSynthesize } from "./hooks/useTtsSynthesize"
import { useVoiceUpload } from "./hooks/useVoiceUpload"
import { useMaterialDelete } from "./hooks/useMaterialDelete"
import { useMaterialBatchDelete } from "./hooks/useMaterialBatchDelete"
import { useVideoExtract } from "./hooks/useVideoExtract"
import VideoExtractModal from "./components/VideoExtractModal"
import "./voices.css"
let toastIdSeq = 0
@@ -166,32 +158,6 @@ const VoiceLibrary: React.FC = () => {
handleUploadClose,
} = useVoiceUpload({ showToast })
// ── 提取视频配音 ──────────────────────────────────────
const {
extractOpen,
extractFile,
extractProgress,
isExtracting,
setExtractOpen,
handleFileSelect: handleExtractFileSelect,
handleExtract,
handleExtractClose,
} = useVideoExtract({ showToast })
// ── URL 参数自动打开上传弹窗 ────────────────────────────
const [searchParams, setSearchParams] = useSearchParams()
useEffect(() => {
if (searchParams.get("upload") === "1") {
setActiveTab("material")
setUploadOpen(true)
// 一次性触发器:清理 upload 参数,避免切换 Tab 时重复触发
const next = new URLSearchParams(searchParams)
next.delete("upload")
setSearchParams(next, { replace: true })
}
}, [searchParams, setActiveTab, setUploadOpen, setSearchParams])
// ── 切换 Tab 时停止播放 ───────────────────────────────
const handleTabChange = useCallback(
(tab: VoiceTabKey) => {
@@ -219,14 +185,6 @@ const VoiceLibrary: React.FC = () => {
>
上传音频
</Button>
<Button
buttonType="primary"
buttonSize="sm"
icon={<VideoCameraOutlined />}
onClick={() => setExtractOpen(true)}
>
提取视频配音
</Button>
<Button
buttonType="ghost"
buttonSize="sm"
@@ -327,18 +285,7 @@ const VoiceLibrary: React.FC = () => {
/>
)}
{/* ── 视频提取配音弹窗 ─────────────────────────────── */}
<VideoExtractModal
open={extractOpen}
file={extractFile}
progress={extractProgress}
isExtracting={isExtracting}
onClose={handleExtractClose}
onFileSelect={handleExtractFileSelect}
onExtract={handleExtract}
/>
{/* ── 弹窗集合 ─────────────────────────────────── */}
{/* ── 弹窗集合 ──────────────────────────────────── */}
<VoiceModals
cloneModalOpen={cloneModalOpen}
onCloneClose={() => setCloneModalOpen(false)}
@@ -136,8 +136,6 @@ export const MaterialVoiceTab: React.FC<MaterialVoiceTabProps> = ({
const material = mapAssetToMaterial(asset)
// duration 优先取顶层(后端从 metadata 提取),兜底 metadata
const cardDuration = asset.duration || material.duration || 0
// AI 生成素材标识(metadata.source === "tts_job")
const isAiMaterial = (asset.metadata as Record<string, unknown>)?.source === "tts_job"
const isPlaying = playingId === asset.id
const isSelected = selectedIds.has(asset.id)
// 播放中以 audio 真实时长为准,未播放显示卡片时长
@@ -186,7 +184,6 @@ export const MaterialVoiceTab: React.FC<MaterialVoiceTabProps> = ({
<div className="xx-voice-info vmat-info">
<div className="xx-voice-name" title={asset.name}>
{asset.name}
{isAiMaterial && <span className="vmat-ai-badge">AI</span>}
</div>
<div className="xx-voice-subtitle">
{asset.file_size ? `${formatFileSize(asset.file_size)}` : "--"}
@@ -1,207 +0,0 @@
import React, { useRef } from "react"
import { Modal } from "antd"
import { InboxOutlined, CloseOutlined } from "@ant-design/icons"
interface VideoExtractModalProps {
open: boolean
file: File | null
progress: number | null
isExtracting: boolean
onClose: () => void
onFileSelect: (file: File | null) => void
onExtract: () => void
}
const ACCEPT_TYPES = ".mp4,.mov,.webm"
const VideoExtractModal: React.FC<VideoExtractModalProps> = ({
open,
file,
progress,
isExtracting,
onClose,
onFileSelect,
onExtract,
}) => {
const inputRef = useRef<HTMLInputElement>(null)
return (
<Modal
title={<span style={{ fontSize: 16, fontWeight: 600 }}>提取视频配音</span>}
open={open}
onCancel={() => {
if (isExtracting) return
onClose()
}}
footer={null}
width={480}
maskClosable={!isExtracting}
>
{!file ? (
<div
className="vmat-upload-dropzone"
onClick={() => inputRef.current?.click()}
style={{
border: "2px dashed #d9d9d9",
borderRadius: 8,
padding: "40px 20px",
textAlign: "center",
cursor: "pointer",
transition: "border-color 0.3s",
}}
onMouseEnter={(e) => (e.currentTarget.style.borderColor = "#7c3aed")}
onMouseLeave={(e) => (e.currentTarget.style.borderColor = "#d9d9d9")}
>
<InboxOutlined style={{ fontSize: 32, color: "#7c3aed", marginBottom: 12 }} />
<p style={{ margin: "0 0 8px", fontSize: 14, color: "#333" }}>点击选择视频文件</p>
<span style={{ fontSize: 12, color: "#999" }}>支持 MP4、MOV、WebM 格式</span>
<input
ref={inputRef}
type="file"
accept={ACCEPT_TYPES}
style={{ display: "none" }}
onChange={(e) => {
const f = e.target.files?.[0]
if (f) onFileSelect(f)
}}
/>
</div>
) : (
<div>
<div
style={{
display: "flex",
alignItems: "center",
justifyContent: "space-between",
padding: "12px 16px",
background: "#fafafa",
borderRadius: 8,
marginBottom: 16,
}}
>
<span
style={{
flex: 1,
overflow: "hidden",
textOverflow: "ellipsis",
whiteSpace: "nowrap",
fontSize: 14,
fontWeight: 500,
}}
title={file.name}
>
{file.name}
</span>
<span style={{ fontSize: 12, color: "#999", marginLeft: 8, flexShrink: 0 }}>
{(file.size / (1024 * 1024)).toFixed(1)} MB
</span>
{!isExtracting && (
<button
type="button"
onClick={() => {
if (inputRef.current) inputRef.current.value = ""
onFileSelect(null)
}}
style={{
border: "none",
background: "none",
cursor: "pointer",
color: "#999",
marginLeft: 8,
fontSize: 14,
}}
aria-label="移除文件"
>
<CloseOutlined />
</button>
)}
</div>
{progress !== null && (
<div style={{ marginBottom: 12 }}>
<div
style={{
height: 6,
background: "#f0f0f0",
borderRadius: 3,
overflow: "hidden",
}}
>
<div
style={{
height: "100%",
width: `${progress}%`,
background: "linear-gradient(90deg, #7c3aed, #a78bfa)",
borderRadius: 3,
transition: "width 0.3s",
}}
/>
</div>
<div
style={{
textAlign: "right",
fontSize: 12,
color: "#999",
marginTop: 4,
}}
>
{progress}%
</div>
</div>
)}
{isExtracting && (
<p style={{ textAlign: "center", fontSize: 13, color: "#7c3aed", margin: "12px 0 0" }}>
{progress === 100 ? "正在提取人声,请稍候..." : "正在上传视频..."}
</p>
)}
</div>
)}
<div
style={{
display: "flex",
justifyContent: "flex-end",
gap: 8,
marginTop: 24,
}}
>
<button
type="button"
onClick={onClose}
disabled={isExtracting}
style={{
padding: "6px 16px",
borderRadius: 6,
border: "1px solid #d9d9d9",
background: "#fff",
cursor: isExtracting ? "not-allowed" : "pointer",
fontSize: 14,
opacity: isExtracting ? 0.5 : 1,
}}
>
取消
</button>
<button
type="button"
onClick={onExtract}
disabled={!file || isExtracting}
style={{
padding: "6px 16px",
borderRadius: 6,
border: "none",
background: !file || isExtracting ? "#d9d9d9" : "#7c3aed",
color: "#fff",
cursor: !file || isExtracting ? "not-allowed" : "pointer",
fontSize: 14,
fontWeight: 500,
}}
>
{isExtracting ? "提取中..." : "开始提取"}
</button>
</div>
</Modal>
)
}
export default VideoExtractModal
@@ -1,74 +0,0 @@
import { useState, useCallback } from "react"
import { useQueryClient } from "@tanstack/react-query"
import { extractVideoVoice } from "@/api/tts"
/**
* 视频提取配音 Hook
* 封装视频上传弹窗状态、提取进度、提取 mutation 逻辑
*/
interface UseVideoExtractProps {
showToast: (message: string, type: "success" | "error") => void
}
export function useVideoExtract({ showToast }: UseVideoExtractProps) {
const queryClient = useQueryClient()
const [extractOpen, setExtractOpen] = useState(false)
const [extractFile, setExtractFile] = useState<File | null>(null)
const [extractProgress, setExtractProgress] = useState<number | null>(null)
const [isExtracting, setIsExtracting] = useState(false)
const handleExtractClose = useCallback(() => {
setExtractOpen(false)
setExtractFile(null)
setExtractProgress(null)
setIsExtracting(false)
}, [])
const handleExtract = useCallback(async () => {
if (!extractFile) return
setIsExtracting(true)
setExtractProgress(0)
try {
await extractVideoVoice(extractFile, (p) => setExtractProgress(p))
// 刷新素材列表
queryClient.invalidateQueries({ queryKey: ["assets", "voice"] })
queryClient.invalidateQueries({ queryKey: ["voice-materials"] })
showToast("视频配音提取成功", "success")
handleExtractClose()
} catch (err: unknown) {
const msg = err instanceof Error ? err.message : "提取失败,请重试"
showToast(msg, "error")
} finally {
setIsExtracting(false)
setExtractProgress(null)
}
}, [extractFile, queryClient, showToast, handleExtractClose])
const handleFileSelect = useCallback(
(file: File | null) => {
if (!file) {
setExtractFile(null)
return
}
const validTypes = ["video/mp4", "video/quicktime", "video/webm"]
if (!validTypes.includes(file.type)) {
showToast("仅支持 MP4、MOV、WebM 格式的视频文件", "error")
return
}
setExtractFile(file)
},
[showToast],
)
return {
extractOpen,
setExtractOpen,
extractFile,
extractProgress,
isExtracting,
handleFileSelect,
handleExtract,
handleExtractClose,
}
}
-18
View File
@@ -193,24 +193,6 @@
overflow: hidden;
text-overflow: ellipsis;
flex: 1;
display: flex;
align-items: center;
}
/* AI 配音标识 */
.vmat-ai-badge {
display: inline-block;
margin-left: 6px;
padding: 1px 6px;
font-size: 11px;
font-weight: 600;
color: #7c3aed;
background: #f3f0ff;
border: 1px solid #ddd6fe;
border-radius: 4px;
line-height: 16px;
vertical-align: middle;
flex-shrink: 0;
}
.xx-voice-star {
-79
View File
@@ -308,85 +308,6 @@ class VideoDeduplicator:
return sum(similarities) / len(similarities) if similarities else 0.0
def compute_duplicate_rate(
self,
fingerprint: VideoFingerprint,
project_id: str,
current_video_id: str | None,
session: Session,
*,
user_id: str = "",
) -> float:
"""计算当前视频与用户库内已有视频的最高相似度百分比。
优先按 user_id 全局比较(跨项目),user_id 为空时回退到项目级比较。
遍历最近 200 个其他有指纹的视频,对每个计算相似度:
- MD5 精确匹配 → 100%
- pHash 相似度 → (1.0 - avg_distance / 64) * 100
取最高值作为 duplicate_rate(0~100)。
如果没有其他视频可比较,返回 0.0。
Args:
fingerprint: 当前视频的指纹
project_id: 项目 ID(user_id 为空时的回退范围)
current_video_id: 当前视频 ID(排除自身,可为 None)
session: 数据库会话
user_id: 用户 ID(优先按用户全局比较)
Returns:
duplicate_rate: 0~100 的浮点数
"""
# 限制查询最近 200 个视频,避免大库内存溢出
from packages.adapters.sqlalchemy_impl.models import GeneratedVideoModel
# 优先按 user_id 全局比较(跨项目),否则回退到项目级
if user_id:
query = session.query(GeneratedVideoModel).filter(
GeneratedVideoModel.user_id == user_id,
)
logger.debug("compute_duplicate_rate: user-level scope user_id=%s", user_id)
else:
query = session.query(GeneratedVideoModel).filter(
GeneratedVideoModel.project_id == project_id,
)
logger.debug("compute_duplicate_rate: project-level fallback project_id=%s", project_id)
# 排除当前视频自身(记录可能已写入 DB,必须在查询层排除)
if current_video_id:
query = query.filter(GeneratedVideoModel.id != current_video_id)
recent_models = query.order_by(GeneratedVideoModel.generated_at.desc()).limit(200).all()
video_repo = SQLAlchemyGeneratedVideoRepository(session)
existing_videos = [video_repo._to_domain(m) for m in recent_models]
max_similarity = 0.0
for existing in existing_videos:
if current_video_id and existing.id == current_video_id:
continue
if not existing.video_fingerprint:
continue
ef = existing.video_fingerprint
# MD5 精确匹配 → 100%
if fingerprint.md5 == ef.get("md5"):
return 100.0
# pHash 相似度
existing_phashes = ef.get("keyframe_phashes", [])
if not existing_phashes or not fingerprint.keyframe_phashes:
continue
min_distances = []
for phash in fingerprint.keyframe_phashes:
distances = [hamming_distance(phash, ep) for ep in existing_phashes]
min_distances.append(min(distances))
avg_distance = sum(min_distances) / len(min_distances) if min_distances else 64
similarity = (1.0 - avg_distance / 64) * 100
max_similarity = max(max_similarity, similarity)
return round(max(max_similarity, 0.0), 2)
@celery_app.task(bind=True, max_retries=3, name="worker.check_duplicate")
def check_duplicate_task(self: Task, generated_video_id: str) -> dict:
@@ -121,21 +121,6 @@ def create_video_record_and_dedup(
generated_video.is_duplicate = False
generated_video.duplicate_of = None
# 计算重复率百分比(与项目内所有已有视频对比取最高相似度)
try:
dup_rate = deduplicator.compute_duplicate_rate(
fingerprint,
project_id,
video_id,
session,
user_id=user_id,
)
generated_video.duplicate_rate = dup_rate
logger.info("Duplicate rate for %s: %.2f%%", video_id, dup_rate)
except Exception as rate_err:
logger.warning("Failed to compute duplicate_rate for %s: %s", video_id, rate_err)
generated_video.duplicate_rate = None
video_repo.update(generated_video)
session.commit()
logger.info(
@@ -200,22 +200,6 @@ class UnifiedRenderService:
# 3. 计算视频总时长(用于字幕显示时长)
video_duration = self._estimate_total_duration(layers)
# Debug: 输出各图层时长明细
for layer in layers:
layer_total = sum(UnifiedRenderService._clip_adjusted_duration(c) for c in layer.clips)
clip_details = [
f"{c.clip_id}(dur={c.duration:.3f},actual={c.actual_duration:.3f},speed={getattr(c, 'playback_speed', 1.0):.4f})"
for c in layer.clips
]
logger.info(
"[debug] layer=%s clips=%d total=%.3f transition_duration=%.3f details=%s",
layer.role,
len(layer.clips),
layer_total,
self.transition_duration,
", ".join(clip_details),
)
logger.info("[debug] estimated video_duration=%.3f", video_duration)
# 3.5 TTS 配音生成(如果配置了)
self._maybe_add_voiceover_layer(layers, video_duration=video_duration)
@@ -1438,7 +1422,6 @@ class UnifiedRenderService:
if trim_segments and len(trim_segments) > 1:
# 多段裁剪:展开为多个 clip
resolved_segments = TrimEngine.resolve_segments(trim_segments, actual_duration)
configured_speed = getattr(clip, "playback_speed", 1.0) or 1.0
for i, seg in enumerate(resolved_segments):
# 每个段生成一个独立的 ResolvedClip
seg_clip_id = f"{clip.id}_seg_{seg.segment_id}"
@@ -1446,19 +1429,6 @@ class UnifiedRenderService:
seg_start = seg.trim.start_time
seg_duration = seg.trim.duration
# 多段裁剪:如果段的时长超过素材实际时长,减速补偿
seg_speed = configured_speed
if actual_duration > 0 and seg_duration > actual_duration + 0.05:
seg_speed = max(0.25, round(configured_speed * actual_duration / seg_duration, 4))
logger.info(
"[debug] multi-seg clip=%s seg=%s duration=%.3f actual=%.3f → speed=%.4f",
clip.id,
seg.segment_id,
seg_duration,
actual_duration,
seg_speed,
)
rc = ResolvedClip(
clip_id=seg_clip_id,
asset_id=asset_id,
@@ -1469,7 +1439,7 @@ class UnifiedRenderService:
duration=seg_duration,
transition_effect=clip.transition_effect or "cut",
transition_duration=getattr(clip, "transition_duration", 0.0) or 0.0,
playback_speed=seg_speed,
playback_speed=getattr(clip, "playback_speed", 1.0) or 1.0,
config={**clip_config, "_segment_id": seg.segment_id},
actual_duration=actual_duration,
trim_config=seg.trim,
@@ -1491,7 +1461,6 @@ class UnifiedRenderService:
effective_trim: TrimConfig | None = None
final_start = clip.start_time
final_duration = clip.duration
configured_speed = getattr(clip, "playback_speed", 1.0) or 1.0
if trim_config is not None and actual_duration > 0:
effective_trim = trim_config.validate_and_resolve(actual_duration)
@@ -1505,25 +1474,6 @@ class UnifiedRenderService:
final_start = 0.0
final_duration = actual_duration
# 素材实际时长不足以覆盖配置的时长时,降低播放速度来补偿
# 例如:配置4s但素材只有3s → speed=0.75x,用满3s素材达到4s输出
if actual_duration > 0 and final_duration > actual_duration + 0.05:
compensated_speed = actual_duration / final_duration
# 保留用户设置的速度(如果已减速则叠加)
final_speed = configured_speed * compensated_speed
# 下限 0.25x
final_speed = max(0.25, round(final_speed, 4))
logger.info(
"[debug] clip=%s duration=%.3f actual=%.3f → 减速补偿 speed=%.4f (configured=%.3f)",
clip.id,
final_duration,
actual_duration,
final_speed,
configured_speed,
)
else:
final_speed = configured_speed
rc = ResolvedClip(
clip_id=clip.id,
asset_id=asset_id,
@@ -1534,25 +1484,13 @@ class UnifiedRenderService:
duration=final_duration,
transition_effect=clip.transition_effect or "cut",
transition_duration=getattr(clip, "transition_duration", 0.0) or 0.0,
playback_speed=final_speed,
playback_speed=getattr(clip, "playback_speed", 1.0) or 1.0,
config=clip_config,
actual_duration=actual_duration,
trim_config=effective_trim,
)
resolved.append(rc)
# Debug日志:记录每个clip的时长信息
eff_dur = _clip_effective_duration_pure(final_duration, actual_duration)
logger.info(
"[debug] resolved clip=%s duration=%.3f actual=%.3f effective=%.3f speed=%.4f start=%.3f",
clip.id,
final_duration,
actual_duration,
eff_dur,
final_speed,
final_start,
)
# 按 order 排序
resolved.sort(key=lambda c: c.order)
return resolved
@@ -1745,7 +1683,7 @@ class UnifiedRenderService:
if d > 0:
layer_dur = d
break
xfade_filter, xfade_estimated_dur = self._transition_engine.build_xfade_chain(
xfade_filter, _ = self._transition_engine.build_xfade_chain(
clip_durations=layer_durations,
clip_video_labels=layer_labels,
transitions=layer_transitions,
@@ -1754,13 +1692,6 @@ class UnifiedRenderService:
)
if xfade_filter:
filter_parts.append(xfade_filter)
logger.info(
"[unified-render] layer=%s xfade: clips=%d durations=%s estimated_dur=%.3f",
layer.role,
len(layer_labels),
[round(d, 3) for d in layer_durations],
xfade_estimated_dur,
)
layer_output_labels[layer.role] = out_label
# Step 3: 合成各层
@@ -1984,14 +1915,8 @@ class UnifiedRenderService:
def _clip_effective_duration(clip: ResolvedClip) -> float:
"""计算 clip 的有效时长(原速 trim 后时长)。
如果 playback_speed < 1(为补偿素材不足而减速),返回配置的 duration,
而非 min(duration, actual_duration)。
实际实现移至 packages.domain.render_layer_utils.clip_effective_duration。
"""
speed = getattr(clip, "playback_speed", 1.0) or 1.0
# 减速场景:duration 已通过降低 playback_speed 补偿,返回配置的 duration
if speed < 1.0 - 1e-6 and clip.duration > 0:
return clip.duration
return _clip_effective_duration_pure(clip.duration, clip.actual_duration)
# ── 画中画(PiP)相关方法 ──────────────────────────────────────────────────
+20 -75
View File
@@ -630,7 +630,7 @@ def ingest_asset(job_id: str) -> dict:
name=filename,
storage_key=job.storage_key,
mime_type=mime_type,
metadata={"source": "upload", "ingest_error": error_reason},
metadata={"ingest_error": error_reason},
file_size=int(metadata.get("size_bytes", 0)),
duration=float(metadata.get("duration", 0)),
width=int(metadata.get("width", 0)),
@@ -656,57 +656,24 @@ def ingest_asset(job_id: str) -> dict:
"error": error_reason,
}
# 查找已存在的 Asset 记录(由 API 端在上传完成时立即创建为 PROCESSING 状态)
existing_asset = None
try:
existing_asset = asset_repo.find_by_storage_key(job.storage_key)
except Exception:
logger.warning("find_by_storage_key not available, trying fallback lookup")
if existing_asset is None:
# 兜底:如果 API 端没有预先创建 Asset(旧版本兼容),则创建新记录
logger.info("No pre-created asset found for storage_key=%s, creating new", job.storage_key)
metadata["source"] = "upload"
asset = Asset.create(
project_id=job.project_id,
library_id=job.library_id,
name=filename,
storage_key=job.storage_key,
mime_type=mime_type,
metadata=metadata,
file_size=int(metadata.get("size_bytes", 0)),
duration=float(metadata.get("duration", 0)),
width=int(metadata.get("width", 0)),
height=int(metadata.get("height", 0)),
codec=metadata.get("codec") or None,
status=AssetStatus.READY,
file_hash=job.file_hash,
thumbnail_url=thumbnail_url,
)
asset_repo.create(asset)
else:
# 更新已有的 Asset 记录,补充元数据并将状态改为 READY
asset = existing_asset
asset.mime_type = mime_type
metadata["source"] = "upload"
asset.metadata = metadata
asset.file_size = int(metadata.get("size_bytes", 0))
asset.duration = float(metadata.get("duration", 0))
asset.width = int(metadata.get("width", 0))
asset.height = int(metadata.get("height", 0))
codec_val = metadata.get("codec")
if codec_val:
asset.codec = str(codec_val)
fps_val = metadata.get("fps")
if fps_val:
try:
asset.fps = float(fps_val)
except (ValueError, TypeError):
pass
asset.status = AssetStatus.READY
asset.thumbnail_url = thumbnail_url
asset.updated_at = datetime.now(timezone.utc)
asset_repo.update(asset)
# Create Asset
asset = Asset.create(
project_id=job.project_id,
library_id=job.library_id,
name=filename,
storage_key=job.storage_key,
mime_type=mime_type,
metadata=metadata,
file_size=int(metadata.get("size_bytes", 0)),
duration=float(metadata.get("duration", 0)),
width=int(metadata.get("width", 0)),
height=int(metadata.get("height", 0)),
codec=metadata.get("codec") or None,
status=AssetStatus.READY,
file_hash=job.file_hash,
thumbnail_url=thumbnail_url,
)
asset_repo.create(asset)
# Update job status to COMPLETED
job.status = IngestJobStatus.COMPLETED
@@ -725,37 +692,15 @@ def ingest_asset(job_id: str) -> dict:
db.rollback()
logger.error(f"Failed to ingest asset {job_id}: {e}")
# Update job status to FAILED and mark pre-created Asset as ERROR
# Update job status to FAILED
try:
job_repo = SQLAlchemyIngestJobRepository(db)
asset_repo = SQLAlchemyAssetRepository(db)
job = job_repo.get(job_id)
if job:
job.status = IngestJobStatus.FAILED
job.error_message = str(e)
job.updated_at = datetime.now(timezone.utc)
job_repo.update(job)
# 将上传时创建的占位 Asset(PROCESSING/UPLOADING)标记为 ERROR,
# 避免素材永远卡在中间状态
try:
existing = asset_repo.find_by_storage_key(job.storage_key)
if existing and existing.status in (
AssetStatus.PROCESSING,
AssetStatus.UPLOADING,
):
existing.status = AssetStatus.ERROR
existing.metadata = {**(existing.metadata or {}), "ingest_error": str(e)}
existing.updated_at = datetime.now(timezone.utc)
asset_repo.update(existing)
logger.info(
"Marked asset as ERROR due to ingest failure: asset_id=%s job_id=%s",
existing.id,
job_id,
)
except Exception as asset_err:
logger.warning("Failed to mark asset as ERROR: %s", asset_err)
db.commit()
except Exception:
db.rollback()
-216
View File
@@ -1,216 +0,0 @@
# ============================================================
# 小虾 SaaS — Production 环境配置模板
# ============================================================
# 使用方式:复制为 /var/lib/xiaoxia-saas-production/.env 并填入实际密钥
# 敏感值标记为 ${PLACEHOLDER},部署前必须替换为真实值
# ============================================================
# ==================== 应用基本配置 ====================
# 应用名称
APP_NAME=xiaoxia-saas
# 环境标识
APP_ENV=production
# 关闭 Debug 模式
DEBUG=false
# 应用基础 URL(前端页面地址)
APP_BASE_URL=https://xiaoxiajianji.com
# 对外公开的 API 基础 URL(用于生成回调链接等)
PUBLIC_API_BASE_URL=https://api.xiaoxiajianji.com
# API 服务监听地址
API_HOST=0.0.0.0
# API 服务监听端口
API_PORT=8001
# 生产环境关闭自动建表,使用 alembic migration
AUTO_CREATE_SCHEMA=false
# ==================== 数据库配置 ====================
# 数据库连接串(格式:postgresql+psycopg://user:password@host:port/dbname)
# ${DATABASE_URL} — 替换为实际的 Production PostgreSQL 连接串
DATABASE_URL=${DATABASE_URL}
# 连接池大小(常驻连接数)
DATABASE_POOL_SIZE=20
# 连接池最大溢出连接数(pool_size + max_overflow = 最大并发连接数)
DATABASE_MAX_OVERFLOW=10
# 获取连接超时时间(秒)
DATABASE_POOL_TIMEOUT=30
# 连接回收时间(秒),防止数据库端主动断开导致的死连接
DATABASE_POOL_RECYCLE=3600
# 不使用内存数据库
USE_IN_MEMORY_DB=false
# ==================== Redis 配置 ====================
# Redis 连接 URL(格式:redis://[:password@]host:port/db)
# ${REDIS_URL} — 替换为实际的 Production Redis 连接串
REDIS_URL=${REDIS_URL}
# 启用 Redis Session 存储(多实例部署必须开启)
ENABLE_REDIS_SESSIONS=true
# ==================== Celery 任务队列 ====================
# Celery Broker(任务分发),使用 Redis db0
CELERY_BROKER_URL=${CELERY_BROKER_URL}
# Celery Result Backend(任务结果存储),使用 Redis db1
CELERY_RESULT_BACKEND=${CELERY_RESULT_BACKEND}
# ==================== Worker 配置 ====================
# Worker 进程名称
WORKER_NAME=xiaoxia-saas-worker
# Worker 并发数(同时执行的任务数)
WORKER_CONCURRENCY=4
# 每个子进程最多处理多少任务后重启(防止内存泄漏)
WORKER_MAX_TASKS_PER_CHILD=1000
# ==================== JWT 认证配置 ====================
# JWT 签名密钥 — 必须设置为强随机字符串(至少32字符)
# ${JWT_SECRET_KEY} — 替换为实际的随机密钥
JWT_SECRET_KEY=${JWT_SECRET_KEY}
# JWT 签名算法
JWT_ALGORITHM=HS256
# Access Token 过期时间(分钟)
JWT_ACCESS_TOKEN_EXPIRE_MINUTES=30
# Refresh Token 过期时间(天)
JWT_REFRESH_TOKEN_EXPIRE_DAYS=30
# ==================== 邮件配置 ====================
# 邮件功能尚未上线,暂时关闭
ENABLE_EMAIL_DELIVERY=false
# SMTP 服务器地址
SMTP_HOST=
# SMTP 端口
SMTP_PORT=587
# SMTP 用户名(邮件功能上线后配置)
SMTP_USER=
# SMTP 密码(邮件功能上线后配置)
SMTP_PASSWORD=
# 发件人邮箱(邮件功能上线后配置)
SMTP_FROM_EMAIL=
# 发件人显示名称
SMTP_FROM_NAME=小虾 SaaS
# 启用 TLS
SMTP_USE_TLS=true
# ==================== 阿里云 OSS 配置 ====================
# OSS 区域 endpoint
OSS_ENDPOINT=oss-cn-hangzhou.aliyuncs.com
# OSS Access Key ID
# ${OSS_ACCESS_KEY_ID} — 替换为实际的 OSS Access Key ID
OSS_ACCESS_KEY_ID=${OSS_ACCESS_KEY_ID}
# OSS Access Key Secret
# ${OSS_ACCESS_KEY_SECRET} — 替换为实际的 OSS Access Key Secret
OSS_ACCESS_KEY_SECRET=${OSS_ACCESS_KEY_SECRET}
# OSS Bucket 名称
OSS_BUCKET_NAME=xiaoxia-autocut
# 直传最大文件大小(MB)
OSS_DIRECT_UPLOAD_MAX_MB=2000
# 直传签名有效期(秒)
OSS_DIRECT_UPLOAD_EXPIRE_SECONDS=900
# ==================== CORS 配置 ====================
# 允许跨域的前端域名列表,逗号分隔
CORS_ORIGINS_RAW=https://xiaoxiajianji.com,https://api.xiaoxiajianji.com
# ==================== 生成文件路径 ====================
# 容器内生成文件目录(固定值,勿改)
GENERATED_FILES_DIR=/app/generated
# 生成文件 URL 前缀
GENERATED_FILES_URL_PREFIX=/generated-files
# 主机上生成文件目录(供 Docker volume bind mount 使用)
GENERATED_FILES_HOST_DIR=/var/lib/xiaoxia-saas-production/generated
# ==================== 渲染引擎配置 ====================
# 渲染引擎选择:legacy(旧引擎,稳定)/ unified(新架构)
RENDER_ENGINE=legacy
# ==================== CosyVoice 语音合成 ====================
# 阿里云百灵语音合成服务 API Key
# ${COSYVOICE_API_KEY} — 替换为实际的 CosyVoice API Key
COSYVOICE_API_KEY=${COSYVOICE_API_KEY}
# API 基础 URL
COSYVOICE_BASE_URL=https://dashscope.aliyuncs.com/api/v1
# 模型选择:cosyvoice-v3-flash(推荐)/ cosyvoice-v3-plus
COSYVOICE_MODEL=cosyvoice-v3-flash
# 音色:v3 系列系统音色带 _v3 后缀
COSYVOICE_VOICE=longxiaoxia_v3
# 采样率
COSYVOICE_SAMPLE_RATE=22050
# 输出格式
COSYVOICE_FORMAT=wav
# 音色克隆模型名(固定值)
COSYVOICE_CLONE_MODEL=voice-enrollment
# DashScope 通用 API Key(与 CosyVoice 共用)
DASHSCOPE_API_KEY=${DASHSCOPE_API_KEY}
# ==================== MediaKit 视频理解(火山引擎)====================
MEDIAKIT_API_KEY=${MEDIAKIT_API_KEY}
MEDIAKIT_BASE_URL=https://mediakit.cn-beijing.volces.com/api/v1
MEDIAKIT_TIMEOUT=60
# ==================== 监控(可选)====================
# Sentry DSN(取消注释并填入实际值以启用错误追踪)
# SENTRY_DSN=${SENTRY_DSN}
-233
View File
@@ -1,233 +0,0 @@
# ============================================================
# 小虾 SaaS — Staging 环境配置模板
# ============================================================
# 使用方式:复制为 /var/lib/xiaoxia-saas-staging/.env 并填入实际密钥
# 敏感值标记为 ${PLACEHOLDER},部署前必须替换为真实值
# ============================================================
# ==================== 应用基本配置 ====================
# 应用名称
APP_NAME=xiaoxia-saas
# 环境标识
APP_ENV=staging
# Staging 开启 Debug 模式便于排查问题
DEBUG=true
# 应用基础 URL(前端页面地址)
APP_BASE_URL=https://staging.xiaoxiajianji.com
# 对外公开的 API 基础 URL(用于生成回调链接等)
PUBLIC_API_BASE_URL=https://staging-api.xiaoxiajianji.com
# API 服务监听地址
API_HOST=0.0.0.0
# API 服务监听端口
API_PORT=8000
# 生产/预发布环境关闭自动建表,使用 alembic migration
AUTO_CREATE_SCHEMA=false
# ==================== 数据库配置 ====================
# 数据库连接串(格式:postgresql+psycopg://user:password@host:port/dbname)
# ${DATABASE_URL} — 替换为实际的 Staging PostgreSQL 连接串
DATABASE_URL=${DATABASE_URL}
# 连接池大小(常驻连接数)
DATABASE_POOL_SIZE=20
# 连接池最大溢出连接数(pool_size + max_overflow = 最大并发连接数)
DATABASE_MAX_OVERFLOW=10
# 获取连接超时时间(秒)
DATABASE_POOL_TIMEOUT=30
# 连接回收时间(秒),防止数据库端主动断开导致的死连接
DATABASE_POOL_RECYCLE=3600
# 不使用内存数据库
USE_IN_MEMORY_DB=false
# ==================== Redis 配置 ====================
# Redis 连接 URL(格式:redis://[:password@]host:port/db)
# ${REDIS_URL} — 替换为实际的 Staging Redis 连接串
REDIS_URL=${REDIS_URL}
# 启用 Redis Session 存储(多实例部署必须开启)
ENABLE_REDIS_SESSIONS=true
# ==================== Celery 任务队列 ====================
# Celery Broker(任务分发),使用 Redis db0
CELERY_BROKER_URL=${CELERY_BROKER_URL}
# Celery Result Backend(任务结果存储),使用 Redis db1
CELERY_RESULT_BACKEND=${CELERY_RESULT_BACKEND}
# ==================== Worker 配置 ====================
# Worker 进程名称
WORKER_NAME=xiaoxia-saas-worker
# Worker 并发数(同时执行的任务数)
WORKER_CONCURRENCY=1
# 每个子进程最多处理多少任务后重启(防止内存泄漏)
WORKER_MAX_TASKS_PER_CHILD=1000
# ==================== JWT 认证配置 ====================
# JWT 签名密钥 — 必须设置为强随机字符串(至少32字符)
# ${JWT_SECRET_KEY} — 替换为实际的随机密钥
JWT_SECRET_KEY=${JWT_SECRET_KEY}
# JWT 签名算法
JWT_ALGORITHM=HS256
# Access Token 过期时间(分钟)
JWT_ACCESS_TOKEN_EXPIRE_MINUTES=1440
# Refresh Token 过期时间(天)
JWT_REFRESH_TOKEN_EXPIRE_DAYS=30
# ==================== 邮件配置 ====================
# 邮件功能尚未上线,暂时关闭
ENABLE_EMAIL_DELIVERY=false
# SMTP 服务器地址
SMTP_HOST=smtp.gmail.com
# SMTP 端口
SMTP_PORT=587
# SMTP 用户名(邮件功能上线后配置)
SMTP_USER=
# SMTP 密码(邮件功能上线后配置)
SMTP_PASSWORD=
# 发件人邮箱(邮件功能上线后配置)
SMTP_FROM_EMAIL=
# 发件人显示名称
SMTP_FROM_NAME=小虾 SaaS
# 启用 TLS
SMTP_USE_TLS=true
# ==================== 阿里云 OSS 配置 ====================
# OSS 区域 endpoint
OSS_ENDPOINT=oss-cn-hangzhou.aliyuncs.com
# OSS Access Key ID
# ${OSS_ACCESS_KEY_ID} — 替换为实际的 OSS Access Key ID
OSS_ACCESS_KEY_ID=${OSS_ACCESS_KEY_ID}
# OSS Access Key Secret
# ${OSS_ACCESS_KEY_SECRET} — 替换为实际的 OSS Access Key Secret
OSS_ACCESS_KEY_SECRET=${OSS_ACCESS_KEY_SECRET}
# OSS Bucket 名称
OSS_BUCKET_NAME=xiaoxia-autocut
# 直传最大文件大小(MB)
OSS_DIRECT_UPLOAD_MAX_MB=2000
# 直传签名有效期(秒)
OSS_DIRECT_UPLOAD_EXPIRE_SECONDS=900
# ==================== MinIO 配置(Staging 独有)====================
# Staging 环境使用 MinIO 替代 OSS 进行文件存储测试
# MinIO 服务 Endpoint
# ${MINIO_ENDPOINT} — 替换为实际的 MinIO 地址
MINIO_ENDPOINT=${MINIO_ENDPOINT}
# MinIO Access Key
# ${MINIO_ACCESS_KEY} — 替换为实际的 MinIO Access Key
MINIO_ACCESS_KEY=${MINIO_ACCESS_KEY}
# MinIO Secret Key
# ${MINIO_SECRET_KEY} — 替换为实际的 MinIO Secret Key
MINIO_SECRET_KEY=${MINIO_SECRET_KEY}
# MinIO Bucket 名称
MINIO_BUCKET_NAME=${MINIO_BUCKET_NAME}
# 是否使用 SSL 连接 MinIO
MINIO_USE_SSL=false
# ==================== CORS 配置 ====================
# 允许跨域的前端域名列表,逗号分隔
CORS_ORIGINS_RAW=https://staging.xiaoxiajianji.com,https://staging-api.xiaoxiajianji.com
# ==================== 生成文件路径 ====================
# 容器内生成文件目录(固定值,勿改)
GENERATED_FILES_DIR=/app/generated
# 生成文件 URL 前缀
GENERATED_FILES_URL_PREFIX=/generated-files
# 主机上生成文件目录(供 Docker volume bind mount 使用)
GENERATED_FILES_HOST_DIR=/var/lib/xiaoxia-saas-staging/generated
# ==================== 渲染引擎配置 ====================
# 渲染引擎选择:legacy(旧引擎,稳定)/ unified(新架构)
RENDER_ENGINE=legacy
# ==================== CosyVoice 语音合成 ====================
# 阿里云百灵语音合成服务 API Key
# ${COSYVOICE_API_KEY} — 替换为实际的 CosyVoice API Key
COSYVOICE_API_KEY=${COSYVOICE_API_KEY}
# API 基础 URL
COSYVOICE_BASE_URL=https://dashscope.aliyuncs.com/api/v1
# 模型选择:cosyvoice-v3-flash(推荐)/ cosyvoice-v3-plus
COSYVOICE_MODEL=cosyvoice-v3-flash
# 音色:v3 系列系统音色带 _v3 后缀
COSYVOICE_VOICE=longxiaoxia_v3
# 采样率
COSYVOICE_SAMPLE_RATE=22050
# 输出格式
COSYVOICE_FORMAT=wav
# 音色克隆模型名(固定值)
COSYVOICE_CLONE_MODEL=voice-enrollment
# DashScope 通用 API Key(与 CosyVoice 共用)
DASHSCOPE_API_KEY=${DASHSCOPE_API_KEY}
# ==================== MediaKit 视频理解(火山引擎)====================
MEDIAKIT_API_KEY=${MEDIAKIT_API_KEY}
MEDIAKIT_BASE_URL=https://mediakit.cn-beijing.volces.com/api/v1
MEDIAKIT_TIMEOUT=60
-51
View File
@@ -1,51 +0,0 @@
server {
listen 80;
server_name _;
root /usr/share/nginx/html;
index index.html;
gzip on;
gzip_vary on;
gzip_min_length 1024;
gzip_types text/plain text/css text/xml text/javascript application/javascript application/json application/xml+rss;
client_max_body_size 800m;
# SPA routing - index.html 禁止缓存,确保每次获取最新版本
location / {
try_files $uri /index.html;
}
# API proxy — Production 环境代理到 production API 容器
resolver 127.0.0.11 valid=10s;
resolver_timeout 5s;
location /api/ {
proxy_pass http://xiaoxia-api-production:8000/api/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 300s;
proxy_send_timeout 300s;
proxy_request_buffering off;
}
# Generated files — 通过 alias 映射容器内 /app/generated/ 目录
location /generated-files/ {
alias /app/generated/;
}
# Assets with legacy fallback — 部署期间兼容旧版缓存的 hash 文件名
# 先在当前镜像中找,找不到去 legacy-assets 目录找(从旧版本容器中备份的)
location ^~ /assets/ {
expires 1y;
add_header Cache-Control "public, immutable";
try_files $uri /assets-legacy$uri =404;
}
# 静态资源长缓存
location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg|woff|woff2|ttf|eot)$ {
expires 1y;
add_header Cache-Control "public, immutable";
}
}
-50
View File
@@ -1,50 +0,0 @@
server {
listen 80;
server_name _;
root /usr/share/nginx/html;
index index.html;
gzip on;
gzip_vary on;
gzip_min_length 1024;
gzip_types text/plain text/css text/xml text/javascript application/javascript application/json application/xml+rss;
client_max_body_size 800m;
# SPA routing - index.html 禁止缓存,确保每次获取最新版本
location = /index.html {
add_header Cache-Control "no-cache, no-store, must-revalidate";
add_header Pragma "no-cache";
expires 0;
}
# SPA fallback
location / {
try_files $uri /index.html;
}
# API proxy — Staging 环境代理到 staging API 容器
resolver 127.0.0.11 valid=10s;
resolver_timeout 5s;
location /api/ {
proxy_pass http://xiaoxia-api-staging:8000/api/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 300s;
proxy_send_timeout 300s;
proxy_request_buffering off;
}
# Generated files — 通过 alias 映射容器内 /app/generated/ 目录
location /generated-files/ {
alias /app/generated/;
}
# 静态资源长缓存
location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg|woff|woff2|ttf|eot)$ {
expires 1y;
add_header Cache-Control "public, immutable";
}
}
-8
View File
@@ -30,14 +30,6 @@ RUN apt-get update \
&& apt-get install -y --no-install-recommends docker-ce-cli docker-buildx-plugin \
&& rm -rf /var/lib/apt/lists/*
# Pre-bake ffmpeg: unit-tests run in fresh containers each time; installing ffmpeg
# on every job cost ~24 min (apt update + hundreds of codec deps). Bake it into the
# image so step_install_ffmpeg.sh detects it and exits instantly.
RUN apt-get update \
&& apt-get install -y --no-install-recommends ffmpeg \
&& ffmpeg -version | head -1 \
&& rm -rf /var/lib/apt/lists/*
# Pre-install base deps (layer cache)
COPY requirements-base.txt ./
RUN python -m venv "$VIRTUAL_ENV" \
+3 -8
View File
@@ -175,14 +175,9 @@ services:
- xiaoxia-net
# =========================================
# Nginx 配置运行时覆盖
# 确保容器使用正确环境的 nginx 配置,即使镜像构建时使用了默认配置
# 注意: 只覆盖 /etc/nginx/conf.d/default.conf,不挂载 /usr/share/nginx/html
# 重要: 生产环境不要添加任何 volume 挂载到 /usr/share/nginx/html
# 这会导致静态文件被覆盖,返回 403 错误
# =========================================
environment:
- NGINX_ENV=${ENV:-staging}
volumes:
- ./nginx-${ENV:-staging}.conf:/etc/nginx/conf.d/default.conf:ro
healthcheck:
test: ["CMD", "wget", "--spider", "-q", "http://127.0.0.1:80"]
@@ -213,7 +208,7 @@ volumes:
# 重要: 确保主机目录存在且有正确权限
# Staging: /var/lib/xiaoxia-saas-staging/generated
# Production: /var/lib/xiaoxia-saas-production/generated
device: ${GENERATED_FILES_HOST_DIR:?GENERATED_FILES_HOST_DIR must be set in .env}
device: ${GENERATED_FILES_HOST_DIR:-/var/lib/xiaoxia-saas-staging/generated}
# ===========================================
# 网络配置
-3
View File
@@ -1,6 +1,5 @@
# Build stage
FROM git.xiaoxiajianji.com/xiaoxia/base/node:20 AS builder
ARG SOURCE_HASH=""
WORKDIR /app
ARG VITE_API_URL=https://saas-api.xiaoxiajianji.com
ENV VITE_API_URL=$VITE_API_URL
@@ -19,10 +18,8 @@ COPY apps/web/ ./
# 构建:TS增量编译 + Vite构建,tsbuildinfo用cache mount持久化
# node_modules直接使用镜像中已安装的(layer缓存保证完整性)
# SOURCE_HASH 变化时强制重新执行(防止 buildkit 幽灵缓存命中)
RUN --mount=type=cache,target=/app/apps/web/.tscache,sharing=locked \
mkdir -p .tscache \
&& echo "SOURCE_HASH=${SOURCE_HASH}" > .cache_bust \
&& ./node_modules/.bin/tsc --incremental --tsBuildInfoFile .tscache/tsconfig.tsbuildinfo \
&& ./node_modules/.bin/vite build
@@ -127,13 +127,6 @@ class InMemoryAssetRepository:
items = [a for a in self._assets.values() if tag_set.issubset(set(a.tag_ids))]
return items[skip : skip + limit]
def find_by_storage_key(self, storage_key: str) -> Asset | None:
"""按 storage_key 查找素材。"""
for asset in self._assets.values():
if asset.storage_key == storage_key:
return asset
return None
def find_by_library_and_file_hash(
self,
library_id: str,
+2 -2
View File
@@ -99,8 +99,8 @@ class SessionStore(SessionStorePort):
session_id: str,
user_id: str,
refresh_token: str,
device_info: str = "",
ip_address: str = "",
device_info: str,
ip_address: str,
expires_in_seconds: int = 30 * 24 * 60 * 60, # 30 天
) -> bool:
"""
+2 -2
View File
@@ -96,7 +96,7 @@ class EmailService(EmailServicePort):
except Exception as e:
return False, str(e)
def send_verification_email( # type: ignore[override]
def send_verification_email(
self,
to_email: str,
username: str,
@@ -165,7 +165,7 @@ class EmailService(EmailServicePort):
return self.send_email(to_email, subject, html_body, text_body)
def send_password_reset_email( # type: ignore[override]
def send_password_reset_email(
self,
to_email: str,
username: str,
@@ -426,13 +426,6 @@ class SQLAlchemyAssetRepository:
models = self.session.query(AssetModel).filter(AssetModel.id.in_(ids)).offset(skip).limit(limit).all()
return [self._to_domain(m) for m in models]
def find_by_storage_key(self, storage_key: str) -> Asset | None:
"""按 storage_key(对应 DB 中的 file_url)查找素材。"""
model = self.session.query(AssetModel).filter(AssetModel.file_url == storage_key).first()
if model is None:
return None
return self._to_domain(model)
def find_by_library_and_file_hash(
self,
library_id: str,
@@ -30,7 +30,6 @@ class SQLAlchemyGeneratedVideoRepository:
video_fingerprint=json.dumps(video.video_fingerprint) if video.video_fingerprint else None,
is_duplicate=video.is_duplicate,
duplicate_of=video.duplicate_of,
duplicate_rate=video.duplicate_rate,
generated_at=video.generated_at,
created_at=video.created_at,
)
@@ -61,7 +60,6 @@ class SQLAlchemyGeneratedVideoRepository:
video_fingerprint=json.loads(getattr(model, "video_fingerprint", "null") or "null"),
is_duplicate=getattr(model, "is_duplicate", False),
duplicate_of=getattr(model, "duplicate_of", None),
duplicate_rate=getattr(model, "duplicate_rate", None),
generated_at=model.generated_at,
created_at=model.created_at,
)
@@ -76,7 +74,6 @@ class SQLAlchemyGeneratedVideoRepository:
model.video_fingerprint = json.dumps(video.video_fingerprint) if video.video_fingerprint else None
model.is_duplicate = video.is_duplicate
model.duplicate_of = video.duplicate_of
model.duplicate_rate = video.duplicate_rate
self.session.add(model)
self.session.commit()
return video
@@ -207,7 +204,6 @@ class SQLAlchemyGeneratedVideoRepository:
video_fingerprint=json.loads(getattr(model, "video_fingerprint", "null") or "null"),
is_duplicate=getattr(model, "is_duplicate", False),
duplicate_of=getattr(model, "duplicate_of", None),
duplicate_rate=getattr(model, "duplicate_rate", None),
generated_at=model.generated_at,
created_at=model.created_at,
)
@@ -289,7 +289,6 @@ class GenerationTaskModel(Base):
completed_at = Column(DateTime, nullable=True)
created_by_user_id = Column(String(36), nullable=False, default="", index=True)
source_edit_plan_id = Column(String(36), nullable=True, index=True)
edit_plan_id = Column(String(36), nullable=True, index=True)
asset_select_mode = Column(String(20), nullable=False, default="")
batch_id = Column(String(36), nullable=False, default="", index=True)
video_title = Column(String(255), nullable=False, default="")
@@ -339,7 +338,6 @@ class GeneratedVideoModel(Base):
video_fingerprint = Column(Text, nullable=True)
is_duplicate = Column(Boolean, nullable=False, default=False)
duplicate_of = Column(String(36), nullable=True)
duplicate_rate = Column(Float, nullable=True)
class TitleLibraryModel(Base):
@@ -498,7 +496,6 @@ class TemplateCategoryModel(Base):
id = Column(String(36), primary_key=True)
user_id = Column(String(36), nullable=False, index=True)
name = Column(String(100), nullable=False)
sort_order = Column(Integer, nullable=False, default=0)
created_at = Column(DateTime, nullable=False, default=lambda: datetime.now(timezone.utc))
@@ -1,9 +1,4 @@
"""SQLAlchemy implementation of TemplateRepository.
模板 segments 数据源已统一为 template_clip_configs 表。
读取时优先 template_clip_configs,回退 template_segments(兼容历史数据)。
写入全部走 template_clip_configs。
"""
"""SQLAlchemy implementation of TemplateRepository."""
from __future__ import annotations
@@ -15,7 +10,6 @@ from sqlalchemy.orm import Session
from packages.adapters.sqlalchemy_impl.models import (
EditPlanModel,
TemplateCategoryModel,
TemplateClipConfigModel,
TemplateModel,
TemplateSegmentModel,
)
@@ -53,38 +47,27 @@ class SQLAlchemyTemplateRepository:
like_pattern = f"%{keyword}%"
query = query.filter(TemplateModel.name.like(like_pattern))
if tag:
query = query.filter(TemplateModel.tags.like(f'"%{tag}"%'))
# JSON 数组包含指定标签(MySQL JSON_CONTAINS / SQLite json_each 兼容写法用 LIKE)
query = query.filter(TemplateModel.tags.like(f'%"{tag}"%'))
models = query.order_by(TemplateModel.created_at.desc()).offset(skip).limit(limit).all()
templates = [self._model_to_entity(m) for m in models]
# 批量加载 segments —— 优先 template_clip_configs
# 批量加载所有 segments,避免 N+1 查询
if templates:
template_ids = [t.id for t in templates]
clip_models = (
self.session.query(TemplateClipConfigModel)
.filter(TemplateClipConfigModel.template_id.in_(template_ids))
.order_by(TemplateClipConfigModel.order)
seg_models = (
self.session.query(TemplateSegmentModel)
.filter(TemplateSegmentModel.template_id.in_(template_ids))
.order_by(TemplateSegmentModel.segment_order)
.all()
)
clip_map: dict[str, list] = {}
for cm in clip_models:
clip_map.setdefault(cm.template_id, []).append(
self._clip_config_to_segment(cm),
# 按 template_id 分组
seg_map: dict[str, list] = {}
for sm in seg_models:
seg_map.setdefault(sm.template_id, []).append(
self._segment_model_to_entity(sm),
)
# 对没有 clip_configs 的模板,回退读 template_segments
missing_ids = [t.id for t in templates if t.id not in clip_map]
if missing_ids:
old_models = (
self.session.query(TemplateSegmentModel)
.filter(TemplateSegmentModel.template_id.in_(missing_ids))
.order_by(TemplateSegmentModel.segment_order)
.all()
)
for om in old_models:
clip_map.setdefault(om.template_id, []).append(
self._segment_model_to_entity(om),
)
for t in templates:
t.segments = clip_map.get(t.id, [])
t.segments = seg_map.get(t.id, [])
return templates
def get(self, template_id: str, user_id: str) -> Optional[Template]:
@@ -117,6 +100,7 @@ class SQLAlchemyTemplateRepository:
is_active=template.is_active,
)
self.session.add(model)
# flush 而非 commit,让 create + create_segments 在同一事务中提交
self.session.flush()
self.session.refresh(model)
result = self._model_to_entity(model)
@@ -161,8 +145,11 @@ class SQLAlchemyTemplateRepository:
if model is None:
return False
model.is_active = False
# 复用 delete_segments_by_template 清理两张表的关联数据
self.delete_segments_by_template(template_id)
# 级联清理关联的 segments,避免孤儿数据
self.session.query(TemplateSegmentModel).filter(
TemplateSegmentModel.template_id == template_id,
).delete(synchronize_session=False)
self.session.commit()
return True
def count_by_user(
@@ -185,7 +172,7 @@ class SQLAlchemyTemplateRepository:
if keyword:
query = query.filter(TemplateModel.name.like(f"%{keyword}%"))
if tag:
query = query.filter(TemplateModel.tags.like(f'"%{tag}"%'))
query = query.filter(TemplateModel.tags.like(f'%"{tag}"%'))
return query.count()
def copy_template(self, template_id: str, user_id: str, new_name: str) -> Template:
@@ -194,8 +181,9 @@ class SQLAlchemyTemplateRepository:
if source is None:
raise ValueError(f"Template {template_id} not found")
new_id = str(uuid.uuid4())
new_template = Template(
id=str(uuid.uuid4()),
id=new_id,
user_id=user_id,
name=new_name,
mode=source.mode,
@@ -209,22 +197,28 @@ class SQLAlchemyTemplateRepository:
)
created = self.create(new_template)
# 复用 create_segments 写入 template_clip_configs
# 复制 segments
new_segments: List[TemplateSegment] = []
for seg in source.segments:
new_segments.append(
TemplateSegment(
id=str(uuid.uuid4()),
template_id=created.id,
segment_order=seg.segment_order,
duration_min=seg.duration_min,
duration_max=seg.duration_max,
material_type=seg.material_type,
)
new_seg = TemplateSegment(
id=str(uuid.uuid4()),
template_id=new_id,
segment_order=seg.segment_order,
duration_min=seg.duration_min,
duration_max=seg.duration_max,
material_type=seg.material_type,
)
new_segments.append(new_seg)
model = TemplateSegmentModel(
id=new_seg.id,
template_id=new_seg.template_id,
segment_order=new_seg.segment_order,
duration_min=new_seg.duration_min,
duration_max=new_seg.duration_max,
material_type=new_seg.material_type,
)
self.session.add(model)
if new_segments:
self.create_segments(new_segments)
else:
self.session.commit()
created.segments = new_segments
@@ -233,58 +227,34 @@ class SQLAlchemyTemplateRepository:
# ── Segments ──
def list_segments(self, template_id: str) -> List[TemplateSegment]:
"""优先从 template_clip_configs 读取,回退读 template_segments。"""
clips = (
self.session.query(TemplateClipConfigModel)
.filter(TemplateClipConfigModel.template_id == template_id)
.order_by(TemplateClipConfigModel.order)
.all()
)
if clips:
return [self._clip_config_to_segment(m) for m in clips]
# 回退:旧表
old = (
models = (
self.session.query(TemplateSegmentModel)
.filter(TemplateSegmentModel.template_id == template_id)
.order_by(TemplateSegmentModel.segment_order)
.all()
)
return [self._segment_model_to_entity(m) for m in old]
return [self._segment_model_to_entity(m) for m in models]
def create_segments(self, segments: List[TemplateSegment]) -> List[TemplateSegment]:
"""写入 template_clip_configs 表。material_type 存入 config JSON。"""
for seg in segments:
config = {"material_type": seg.material_type} if seg.material_type else {}
model = TemplateClipConfigModel(
model = TemplateSegmentModel(
id=seg.id,
template_id=seg.template_id,
clip_type="main",
order=seg.segment_order,
min_duration=seg.duration_min,
max_duration=seg.duration_max,
text_template="",
material_requirements={},
transition_effect="cut",
config=config,
segment_order=seg.segment_order,
duration_min=seg.duration_min,
duration_max=seg.duration_max,
material_type=seg.material_type,
)
self.session.add(model)
self.session.commit()
return segments
def delete_segments_by_template(self, template_id: str) -> int:
"""删除两张表中的 segments 数据,返回删除总数。"""
c1 = (
self.session.query(TemplateClipConfigModel)
.filter(TemplateClipConfigModel.template_id == template_id)
.delete(synchronize_session=False)
)
c2 = (
self.session.query(TemplateSegmentModel)
.filter(TemplateSegmentModel.template_id == template_id)
.delete(synchronize_session=False)
count = (
self.session.query(TemplateSegmentModel).filter(TemplateSegmentModel.template_id == template_id).delete()
)
self.session.commit()
return c1 + c2
return count
# ── Categories ──
@@ -396,23 +366,6 @@ class SQLAlchemyTemplateRepository:
updated_at=model.updated_at,
)
@staticmethod
def _clip_config_to_segment(model: TemplateClipConfigModel) -> TemplateSegment:
"""将 TemplateClipConfigModel 转换为 TemplateSegment 域实体。"""
material_type = None
if model.config and isinstance(model.config, dict):
material_type = model.config.get("material_type")
return TemplateSegment(
id=model.id,
template_id=model.template_id,
segment_order=model.order,
duration_min=model.min_duration,
duration_max=model.max_duration,
material_type=material_type,
created_at=model.created_at,
updated_at=model.updated_at,
)
@staticmethod
def _category_model_to_entity(model: TemplateCategoryModel) -> TemplateCategory:
return TemplateCategory(
+1 -1
View File
@@ -51,7 +51,7 @@ def parse_titles_from_response(content: str) -> list[str]:
pass
# 尝试按行解析
titles = []
titles: list[str] = []
for line in content.strip().split("\n"):
line = line.strip()
if not line:
+1 -50
View File
@@ -256,42 +256,6 @@ def _wrap_title_text(
return "\\N".join(wrapped_segments)
def _parse_title_position(
title_config: dict[str, Any],
video_width: int,
video_height: int,
) -> tuple[int, int] | None:
"""解析标题自由拖拽坐标 pos_x/pos_y(PlayRes 像素坐标系)。
要求两个字段同时存在、可转 int,且落在 [0, video_width] × [0, video_height]
闭区间内。任一条件不满足返回 None,调用方回退 position 三档逻辑。
Args:
title_config: 标题配置 dict
video_width: PlayResX(视频宽度像素)
video_height: PlayResY(视频高度像素)
Returns:
(x, y) 整数坐标,或 None 表示不使用自由位置
"""
if "pos_x" not in title_config or "pos_y" not in title_config:
return None
raw_x = title_config["pos_x"]
raw_y = title_config["pos_y"]
# 坐标必须是 PlayRes 像素整数:bool 是 int 子类(isinstance(True,int)=True)
# 但 True/False 作坐标无意义;float 静默截断会造成拖拽位置偏差,一律按非法回退
if isinstance(raw_x, bool) or isinstance(raw_y, bool):
return None
if not isinstance(raw_x, int) or not isinstance(raw_y, int):
return None
x, y = raw_x, raw_y
if video_width <= 0 or video_height <= 0:
return None
if not (0 <= x <= video_width and 0 <= y <= video_height):
return None
return (x, y)
def build_ass_content(
*,
video_width: int,
@@ -379,16 +343,7 @@ def build_ass_content(
title_shadow.get("offset_y", 2) if title_shadow.get("enabled", False) else 0,
)
# ── 自由位置拖拽(工单 #1405 方案 B)────────────────────────────
# pos_x/pos_y 为 PlayRes 坐标系像素整数(PlayResX/Y = video_width/height)。
# 合法时:TitleStyle Alignment 固定 5(\an5 中对齐,使 \pos 锚点为文本块中心),
# Dialogue 文本前注入 {\pos(x,y)}。字段缺失/非法/越界时一律回退
# position → alignment 三档逻辑,现有输出保持一字节不变。
title_pos = _parse_title_position(title_config, video_width, video_height)
title_alignment = 5 if title_pos is not None else position_to_ass_alignment(
title_config.get("position", "top")
)
title_alignment = position_to_ass_alignment(title_config.get("position", "top"))
styles.append(
build_ass_style(
@@ -415,10 +370,6 @@ def build_ass_content(
safe_title_text_raw = escape_ass_text(title_text)
safe_title_text = _wrap_title_text(safe_title_text_raw, video_width, title_font_size)
# 自由位置:在文本前注入 \pos override tag(锚点为文本块中心,配合 \an5)
if title_pos is not None:
safe_title_text = f"{{\\pos({title_pos[0]},{title_pos[1]})}}{safe_title_text}"
events.append(
"Dialogue: 0,0:00:00.00," f"{format_ass_time(video_duration)}," "TitleStyle,,0,0,0,," f"{safe_title_text}"
)
-1
View File
@@ -26,7 +26,6 @@ class GeneratedVideo:
video_fingerprint: dict[str, Any] | None = None
is_duplicate: bool = False
duplicate_of: str | None = None
duplicate_rate: float | None = None
generated_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))
created_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))
+13 -200
View File
@@ -27,137 +27,6 @@ DEFAULT_INTRO_DURATION = 3.0
DEFAULT_OUTRO_DURATION = 3.0
# ── SceneChange 镜头段工具 ────────────────────────────────────────────────────
def build_scene_segments(
scene_changes: list[float],
asset_duration: float,
) -> list[tuple[float, float]]:
"""根据场景切换点构建镜头段列表.
Args:
scene_changes: 场景切换点时间戳列表(已排序,首位为 0.0)
asset_duration: 素材总时长
Returns:
镜头段列表 [(start, end), ...],仅保留长度 >= 0.5s 的段
"""
segments: list[tuple[float, float]] = []
for i, ts in enumerate(scene_changes):
end = scene_changes[i + 1] if i + 1 < len(scene_changes) else asset_duration
# 只保留有效长度的镜头段(至少 0.5 秒)
if end - ts >= 0.5:
segments.append((ts, end))
return segments
def pick_start_in_scene_segment(
seg_start: float,
seg_end: float,
clip_duration: float,
) -> float | None:
"""在镜头段内随机选取一个起始时间点.
确保 start + clip_duration <= seg_end。
若镜头段长度不足以容纳片段,返回 None。
"""
available = seg_end - seg_start - clip_duration
if available < 0:
return None
max_start = seg_start + available
return random.uniform(seg_start, max_start)
def _segments_overlap(
start: float,
duration: float,
used: list[tuple[float, float]],
edge_gap: float = 0.0,
) -> bool:
"""候选区间 [start, start+duration] 是否与已用区间冲突(含边缘间隙扩边)。"""
end = start + duration
for used_start, used_end in used:
if start < used_end + edge_gap and end > used_start - edge_gap:
return True
return False
def pick_scene_aware_start(
asset_id: str,
clip_duration: float,
asset_durations: dict[str, float],
asset_scene_points: dict[str, list[float]] | None,
used_segments: dict[str, list[tuple[float, float]]],
*,
edge_gap: float = 0.0,
) -> float | None:
"""基于缓存的场景切换点,从随机镜头段中选取不冲突的起始时间.
流程:
1. 读取 asset_scene_points 中该素材的场景切换点缓存 → 构建镜头段
2. random.shuffle 镜头段(保证同一素材多次生成选不同镜头,而非固定第N段)
3. 依次尝试:段内随机取点 → 越界检查 → 与 used_segments 冲突检查
4. 全部冲突/无缓存 → 返回 None,由调用方回退 _calc_random_start_time
Args:
asset_id: 素材 ID
clip_duration: 片段时长(秒)
asset_durations: 素材 ID -> 总时长
asset_scene_points: 素材 ID -> 场景切换点列表(metadata 缓存)
used_segments: 素材 ID -> 已用区间列表(冲突避让)
edge_gap: 冲突判定的边缘间隙(秒),已用区间按 [s-gap, e+gap] 扩边
"""
asset_total = (asset_durations or {}).get(asset_id)
if not asset_total or asset_total <= 0:
return None
scene_points = (asset_scene_points or {}).get(asset_id)
if not scene_points:
return None
used = used_segments.get(asset_id, []) if used_segments else []
scene_segments = build_scene_segments(scene_points, asset_total)
if not scene_segments:
return None
random.shuffle(scene_segments)
for seg_start, seg_end in scene_segments:
candidate = pick_start_in_scene_segment(seg_start, seg_end, clip_duration)
if candidate is None:
continue
# 越界检查(防御:场景点末尾段理论上不越界,metadata 脏数据兜底)
if candidate + clip_duration > asset_total:
continue
# 与已用区间冲突检查
if _segments_overlap(candidate, clip_duration, used, edge_gap):
continue
return candidate
return None
def extract_scene_points_from_metadata(metadata: object) -> list[float] | None:
"""从素材 metadata 中提取并校验场景切换点缓存.
合法缓存:list 类型、至少 2 个数值点、单调非负;否则返回 None(按未缓存处理)。
"""
if not isinstance(metadata, dict):
return None
points = metadata.get("scene_change_points")
if not isinstance(points, list) or len(points) < 2:
return None
try:
cleaned = [float(p) for p in points]
except (TypeError, ValueError):
return None
if any(p < 0 for p in cleaned):
return None
cleaned = sorted(cleaned)
if cleaned[0] != 0.0:
cleaned.insert(0, 0.0)
return cleaned
# ── 素材分配 ────────────────────────────────────────────────────────────────
@@ -168,7 +37,6 @@ def distribute_assets(
*,
random_selection: bool = False,
asset_durations: dict[str, float] | None = None,
asset_scene_points: dict[str, list[float]] | None = None,
) -> None:
"""按 editing_mode 将素材分配到 clips(就地修改).
@@ -178,16 +46,12 @@ def distribute_assets(
- VOICE_OVER: 素材→main clips (B-roll)
- VOICE_PIP: 第1个→background, 第2个→corner_voice, 其余→b_roll
start_time 选取:素材 metadata 中有场景切换点缓存时,优先从随机镜头段
取起点(不同片段来自不同镜头);无缓存或镜头段都冲突时回退随机起点。
Args:
clips: 剪辑片段列表(就地修改 asset_id)
asset_ids: 素材 ID 列表
editing_mode: 剪辑模式字符串
random_selection: 是否随机选择素材(用于预览生成)
asset_durations: 素材 ID -> 时长(秒)映射,用于设置 start_time
asset_scene_points: 素材 ID -> 场景切换点列表(metadata 缓存)
asset_durations: 素材 ID -> 时长(秒)映射,用于设置随机 start_time
"""
if not asset_ids or not clips:
return
@@ -198,56 +62,22 @@ def distribute_assets(
random.shuffle(asset_ids)
if editing_mode == EditingMode.ONE_TAKE.value:
_distribute_one_take(clips, asset_ids, asset_durations, asset_scene_points)
_distribute_one_take(clips, asset_ids, asset_durations)
elif editing_mode == EditingMode.PIP.value:
_distribute_pip(clips, asset_ids, asset_durations, asset_scene_points)
_distribute_pip(clips, asset_ids, asset_durations)
elif editing_mode == EditingMode.VOICE_OVER.value:
_distribute_voice_over(clips, asset_ids, asset_durations, asset_scene_points)
_distribute_voice_over(clips, asset_ids, asset_durations)
elif editing_mode == EditingMode.VOICE_PIP.value:
_distribute_voice_pip(clips, asset_ids, asset_durations, asset_scene_points)
_distribute_voice_pip(clips, asset_ids, asset_durations)
else:
# 未知模式,退化为 one_take
_distribute_one_take(clips, asset_ids, asset_durations, asset_scene_points)
def _resolve_start_time(
asset_id: str,
clip_duration: float,
asset_durations: dict[str, float] | None,
used_segments: dict[str, list[tuple[float, float]]],
asset_scene_points: dict[str, list[float]] | None = None,
on_exhausted: Callable[[str, float], tuple[float, float] | None] | None = None,
) -> float | None:
"""选取片段起点:场景缓存优先(随机镜头段),无缓存/全冲突回退随机起点.
场景路径与随机路径共享 used_segments 冲突避让;场景路径返回 None 时
(无缓存、镜头段全冲突)回退 _calc_random_start_time,其受控复用逻辑
(on_exhausted)不受影响。
"""
if asset_scene_points and asset_scene_points.get(asset_id):
scene_start = pick_scene_aware_start(
asset_id,
clip_duration,
asset_durations or {},
asset_scene_points,
used_segments,
)
if scene_start is not None:
return scene_start
return _calc_random_start_time(
asset_id,
clip_duration,
asset_durations,
used_segments,
on_exhausted=on_exhausted,
)
_distribute_one_take(clips, asset_ids, asset_durations)
def _distribute_one_take(
clips: List[EditPlanClip],
asset_ids: List[str],
asset_durations: dict[str, float] | None = None,
asset_scene_points: dict[str, list[float]] | None = None,
) -> None:
"""ONE_TAKE: 素材按顺序依次分配给 main 类型 clips."""
used_segments: dict[str, list[tuple[float, float]]] = {}
@@ -255,9 +85,7 @@ def _distribute_one_take(
for i, clip in enumerate(main_clips):
if i < len(asset_ids):
asset_id = asset_ids[i]
start_time = _resolve_start_time(
asset_id, clip.duration, asset_durations, used_segments, asset_scene_points
)
start_time = _calc_random_start_time(asset_id, clip.duration, asset_durations, used_segments)
clip.assign_asset(asset_id, start_time=start_time)
# Record used segment
if start_time is not None and asset_durations is not None:
@@ -270,7 +98,6 @@ def _distribute_pip(
clips: List[EditPlanClip],
asset_ids: List[str],
asset_durations: dict[str, float] | None = None,
asset_scene_points: dict[str, list[float]] | None = None,
) -> None:
"""PIP: 第1个素材→main(全屏背景),其余→overlay clips."""
used_segments: dict[str, list[tuple[float, float]]] = {}
@@ -278,9 +105,7 @@ def _distribute_pip(
main_clips = [c for c in clips if c.clip_type == ClipType.MAIN.value]
if main_clips and asset_ids:
asset_id = asset_ids[0]
start_time = _resolve_start_time(
asset_id, main_clips[0].duration, asset_durations, used_segments, asset_scene_points
)
start_time = _calc_random_start_time(asset_id, main_clips[0].duration, asset_durations, used_segments)
main_clips[0].assign_asset(asset_id, start_time=start_time)
# Record used segment
if start_time is not None and asset_durations is not None:
@@ -294,9 +119,7 @@ def _distribute_pip(
for i, clip in enumerate(overlay_clips):
if i < len(remaining):
asset_id = remaining[i]
start_time = _resolve_start_time(
asset_id, clip.duration, asset_durations, used_segments, asset_scene_points
)
start_time = _calc_random_start_time(asset_id, clip.duration, asset_durations, used_segments)
clip.assign_asset(asset_id, start_time=start_time)
# Record used segment
if start_time is not None and asset_durations is not None:
@@ -309,7 +132,6 @@ def _distribute_voice_over(
clips: List[EditPlanClip],
asset_ids: List[str],
asset_durations: dict[str, float] | None = None,
asset_scene_points: dict[str, list[float]] | None = None,
) -> None:
"""VOICE_OVER: 素材→main clips (B-roll)."""
used_segments: dict[str, list[tuple[float, float]]] = {}
@@ -317,9 +139,7 @@ def _distribute_voice_over(
for i, clip in enumerate(main_clips):
if i < len(asset_ids):
asset_id = asset_ids[i]
start_time = _resolve_start_time(
asset_id, clip.duration, asset_durations, used_segments, asset_scene_points
)
start_time = _calc_random_start_time(asset_id, clip.duration, asset_durations, used_segments)
clip.assign_asset(asset_id, start_time=start_time)
# Record used segment
if start_time is not None and asset_durations is not None:
@@ -332,7 +152,6 @@ def _distribute_voice_pip(
clips: List[EditPlanClip],
asset_ids: List[str],
asset_durations: dict[str, float] | None = None,
asset_scene_points: dict[str, list[float]] | None = None,
) -> None:
"""VOICE_PIP: 第1个→background, 第2个→corner_voice, 其余→b_roll."""
used_segments: dict[str, list[tuple[float, float]]] = {}
@@ -345,9 +164,7 @@ def _distribute_voice_pip(
# 第1个 → background
if idx < len(asset_ids) and bg_clips:
asset_id = asset_ids[idx]
start_time = _resolve_start_time(
asset_id, bg_clips[0].duration, asset_durations, used_segments, asset_scene_points
)
start_time = _calc_random_start_time(asset_id, bg_clips[0].duration, asset_durations, used_segments)
bg_clips[0].assign_asset(asset_id, start_time=start_time)
# Record used segment
if start_time is not None and asset_durations is not None:
@@ -359,9 +176,7 @@ def _distribute_voice_pip(
# 第2个 → corner_voice
if idx < len(asset_ids) and voice_clips:
asset_id = asset_ids[idx]
start_time = _resolve_start_time(
asset_id, voice_clips[0].duration, asset_durations, used_segments, asset_scene_points
)
start_time = _calc_random_start_time(asset_id, voice_clips[0].duration, asset_durations, used_segments)
voice_clips[0].assign_asset(asset_id, start_time=start_time)
# Record used segment
if start_time is not None and asset_durations is not None:
@@ -375,9 +190,7 @@ def _distribute_voice_pip(
for i, clip in enumerate(broll_clips):
if i < len(remaining):
asset_id = remaining[i]
start_time = _resolve_start_time(
asset_id, clip.duration, asset_durations, used_segments, asset_scene_points
)
start_time = _calc_random_start_time(asset_id, clip.duration, asset_durations, used_segments)
clip.assign_asset(asset_id, start_time=start_time)
# Record used segment
if start_time is not None and asset_durations is not None:
-7
View File
@@ -14,13 +14,6 @@ from dataclasses import dataclass, field
from datetime import datetime, timezone
from typing import Any
# 素材选取排序时注入的随机噪声上限(分)。
# score_asset 综合得分范围为 0-100,噪声 0~20 意味着:
# - 素材间得分差距 > 20 分时,排名不受影响(质量差异显著的素材保持稳定优先级)
# - 得分接近(差距 <= 20 分)的素材排名会随机浮动,使每次生成选出的素材组合不同,
# 从素材组合层面降低成片重复率;排名靠后的低分素材也有机会入选。
SCORE_RANDOM_NOISE_MAX = 20.0
@dataclass
class SmartMatchResult:
+2 -5
View File
@@ -150,11 +150,8 @@ def build_xfade_filter_chain(
else:
first_input_dur = cumulative - total_transition
# 正确的 offset 计算:offset 应相对于累积输出时长
# offset = 累积输出中,转场开始的时间点
# = first_input_dur - transition_duration
# 这样每个转场之间的"纯内容"时长等于原始 clip 时长
offset = max(0.0, first_input_dur - transition_duration)
# 原始 offset 计算
offset = max(0.0, cumulative - transition_duration * i)
# 安全钳制:offset + td 不能超过第一个输入的时长
available = max(0.0, first_input_dur - offset)
-5
View File
@@ -112,11 +112,6 @@ class AssetRepository(ABC):
"""查找包含所有指定标签的素材。"""
pass
@abstractmethod
def find_by_storage_key(self, storage_key: str) -> Asset | None:
"""按 storage_key 查找素材(用于异步处理时更新已创建的记录)。"""
pass
@abstractmethod
def find_by_library_and_file_hash(
self,
-64
View File
@@ -119,70 +119,6 @@ class MediaKitClient:
return None
def detect_scene_changes(
self,
video_url: str,
max_frames: int = 20,
poll_interval: float = 2.0,
max_poll_attempts: int = 30,
) -> Optional[List[float]]:
"""检测视频场景切换点,返回时间戳列表.
降级策略:
1. 先尝试 SceneChange 策略
2. SceneChange 失败(OOM等)→ 退回 TimeInterval(5秒间隔)
3. MediaKit 不可用 → 返回 None
Returns:
场景切换点时间戳列表,如 [0.0, 3.2, 7.8, 12.5]
失败返回 None
"""
if not self.is_available:
logger.warning("MediaKit 未配置,跳过场景检测")
return None
# 策略1:尝试 SceneChange
frames = self.extract_frames(
video_url=video_url,
strategy="SceneChange",
max_frames=max_frames,
poll_interval=poll_interval,
max_poll_attempts=max_poll_attempts,
)
# 策略2:SceneChange 失败 → 退回 TimeInterval(5秒间隔)
if frames is None:
logger.info("SceneChange 策略失败,降级为 TimeInterval(5秒间隔)")
# 估算帧数:假设视频最长60秒,每5秒一帧
ti_max_frames = max(max_frames, 12)
frames = self.extract_frames(
video_url=video_url,
strategy="TimeInterval",
max_frames=ti_max_frames,
poll_interval=poll_interval,
max_poll_attempts=max_poll_attempts,
)
if frames is None:
return None
# 从帧列表中提取 timestamp,排序
timestamps = sorted({float(f.get("timestamp", 0.0)) for f in frames if "timestamp" in f})
if not timestamps:
return None
# 始终在列表开头加 0.0(素材起始点)
if timestamps[0] != 0.0:
timestamps.insert(0, 0.0)
logger.info(
"场景检测完成: video_url=%s scene_changes=%s",
video_url[:80],
timestamps,
)
return timestamps
def _submit_extract_task(
self,
video_url: str,
+1 -1
View File
@@ -100,7 +100,7 @@ def _check_ssrf_domain(hostname: str) -> None:
raise UrlSecurityError(f"域名解析失败: {hostname}")
for info in infos:
ip_str = str(info[4][0])
ip_str = info[4][0]
try:
_check_ssrf_ip_base(ip_str)
except ValueError:
+1 -3
View File
@@ -24,10 +24,8 @@ celery==5.4.0
# 对象存储
oss2==2.18.4
# HTTP 客户端(pin 间接依赖防止版本漂移)
# HTTP 客户端
httpx==0.27.2
httpcore==1.0.7
h2==4.1.0
# Prometheus monitoring
prometheus-client==0.21.1
-1
View File
@@ -15,6 +15,5 @@ pytest-xdist==3.6.1
diff-cover==8.0.3
# 资产质量评分依赖(与 requirements-worker.txt 保持一致)
numpy==1.26.4
scipy==1.13.1
Pillow==10.4.0
+9 -6
View File
@@ -28,12 +28,15 @@ if [ "$SKIP_BACKEND" = "true" ]; then
CONTEXTS=("CI/CD Pipeline / Frontend Lint (pull_request)")
else
CONTEXTS=(
# CI 优化后 job 名称(2026-08):Code Quality 拆分为 Style+Security,Type Check+Migration 合并为 Python
# 与 pr_auto_scan.py 的 REQUIRED_CONTEXTS_APPROVE 保持一致
"CI/CD Pipeline / Validate - Style (pull_request)"
"CI/CD Pipeline / Validate - Security (pull_request)"
"CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request)"
"CI/CD Pipeline / Validate - Code Quality (pull_request)"
"CI/CD Pipeline / Validate - Type Check (mypy) (pull_request)"
"CI/CD Pipeline / Validate - Migration (alembic) (pull_request)"
"CI/CD Pipeline / Frontend Lint (pull_request)"
"CI/CD Pipeline / Unit Tests (pull_request)"
"CI/CD Pipeline / Frontend Unit Tests (pull_request)"
"CI/CD Pipeline / PR Build API Image (pull_request)"
"CI/CD Pipeline / PR Build Web Image (pull_request)"
"CI/CD Pipeline / PR Build Worker Image (pull_request)"
)
fi
@@ -150,4 +153,4 @@ done
echo
echo "⏰ 快速检查超时(2分钟),CI尚未完成,退出等待下次触发(workflow_run事件或5分钟定时扫描)"
exit 0
exit 0
+9 -24
View File
@@ -32,9 +32,9 @@ CONTEXTS=(
echo "检查CI Gate统一门禁"
echo
# 等待30秒后开始轮询,最多10分钟
echo "等待30秒让CI启动..."
sleep 30
# 等待60秒,给CI启动写status的时间
echo "等待60秒让CI启动..."
sleep 60
# 405计数器(单次运行内重试)
MERGE_405_COUNT=0
@@ -72,9 +72,9 @@ check_and_merge() {
# CI未全绿(pending中)→ 退出,等下次触发
if [ "$ALL_SUCCESS" != "true" ]; then
echo
echo "⏳ CI尚未全绿(仍有pending),等待重试..."
echo " (当前第${attempt}次轮询,最多${MAX_ATTEMPTS}次)"
return 1
echo "⏳ CI尚未全绿(仍有pending),退出等待下次触发"
echo " (pr-auto-scan每5分钟扫描一次,CI通过后会自动合并)"
exit 0
fi
# CI全绿 → 合并
@@ -136,28 +136,13 @@ check_and_merge() {
fi
}
# 轮询等待CI就绪+审批完成,最多10分钟(60次x10秒)
MAX_ATTEMPTS=60
for attempt in $(seq 1 $MAX_ATTEMPTS); do
# 最多重试3次(用于405重试,非CI轮询)
for i in 1 2 3; do
if check_and_merge; then
exit 0
fi
# 检查PR是否还open(可能已被手动合并或关闭)
PR_STATE=$(curl -s -H "Authorization: token ${MERGE_TOKEN}" \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}" \
| python3 -c "import sys,json; print(json.load(sys.stdin).get('state',''))" 2>/dev/null || echo "?")
if [ "$PR_STATE" != "open" ]; then
echo "PR状态为 ${PR_STATE},无需继续等待"
exit 0
fi
if [ $attempt -lt $MAX_ATTEMPTS ]; then
sleep 10
fi
done
echo
echo "⏰ 等待10分钟后仍未满足合并条件,退出。pr-auto-scan定时扫描会继续重试。"
echo "本次检查未满足合并条件,退出。pr-auto-scan每5分钟会继续扫描。"
exit 0
+1 -70
View File
@@ -13,7 +13,6 @@ before="${GITHUB_EVENT_BEFORE:-}"
after="${GITHUB_SHA:-}"
repo="${GITHUB_REPOSITORY:-}"
base="${GITHUB_API_URL:-}"
ZERO="0000000000000000000000000000000000000000"
# Gitea Actions 中 push 事件的前一个 SHA 在 event payload 的 before 字段
if [ -z "$before" ] && [ -n "${GITHUB_EVENT_PATH:-}" ] && [ -f "$GITHUB_EVENT_PATH" ]; then
@@ -29,76 +28,8 @@ fi
echo "改动范围检测: before=${before:-<empty>} after=${after}"
# ── 安全回溯:确保 diff 基准覆盖所有未构建的改动 ──
# 问题:concurrency 取消机制会导致前端改动被跳过。被取消的 push 的改动不会被
# 后续 push 的 diff 覆盖到,因为 GITHUB_EVENT_BEFORE 只指向上一次 push 的 SHA。
# 修复:查询最近一次**实际构建了 web 镜像**的成功 push run,用其 head_sha 作为
# diff 基准。这样被取消/跳过的 run 的改动都会被包含在当前 diff 中。
if [ -n "$before" ] && [ "$before" != "$ZERO" ] && [ -n "${GITHUB_TOKEN:-}" ]; then
BRANCH="${GITHUB_REF_NAME:-}"
if [ -n "$BRANCH" ]; then
SAFE_BASE=$(python3 -c "
import json, subprocess, sys
base = '${base}'
repo = '${repo}'
token = '${GITHUB_TOKEN}'
branch = '${BRANCH}'
cur_sha = '${after}'
def check_run(run_id):
\"\"\"Check if this run actually built the web image.\"\"\"
try:
r = subprocess.run(
['curl', '-sf', '--max-time', '10',
'-H', f'Authorization: token {token}',
f'{base}/repos/{repo}/actions/runs/{run_id}/jobs'],
capture_output=True, text=True, timeout=15)
if r.returncode != 0:
return False
jobs = json.loads(r.stdout).get('jobs', [])
return any(
'Build Staging Web' in j.get('name', '')
and j.get('conclusion') == 'success'
for j in jobs
)
except Exception:
return False
try:
r = subprocess.run(
['curl', '-sf', '--max-time', '15',
'-H', f'Authorization: token {token}',
f'{base}/repos/{repo}/actions/runs?status=success&event=push&branch={branch}&per_page=30'],
capture_output=True, text=True, timeout=20)
if r.returncode != 0:
sys.exit(0)
d = json.loads(r.stdout)
runs = d.get('workflow_runs', []) if isinstance(d, dict) else d
for run in runs:
sha = run.get('head_sha', '')
if sha and sha != cur_sha:
if check_run(run['id']):
print(sha)
break
except Exception:
pass
" 2>/dev/null || true)
if [ -n "$SAFE_BASE" ] && [ "$SAFE_BASE" != "$before" ]; then
echo "🔒 安全回溯: 使用最近实际构建 web 的 commit ${SAFE_BASE:0:8} 替代 before=${before:0:8}"
before="$SAFE_BASE"
elif [ -z "$SAFE_BASE" ]; then
echo "⚠️ 未找到历史成功构建 web 的 push run,保守走全量构建"
echo "skip_backend=false" >> "$OUTPUT"
echo "skip_frontend=false" >> "$OUTPUT"
exit 0
fi
fi
fi
FILES=""
if [ -n "$before" ] && [ "$before" != "$ZERO" ]; then
if [ -n "$before" ] && [ "$before" != "0000000000000000000000000000000000000000" ]; then
# Gitea 1.26.x compare API 的顶层 files 字段不填充(始终为空),
# 但响应里每个 commit 条目自带的 files 完整可用;聚合区间内所有提交的 files 即可。
API_URL="${base}/repos/${repo}/compare/${before}...${after}?per_page=300"
-12
View File
@@ -19,18 +19,6 @@ for arg in "$@"; do
BUILD_ARGS="$BUILD_ARGS --build-arg $arg"
done
# Web 镜像 cache bust:计算 apps/web/ 的 git tree hash
# 当源码变化时 hash 变化,buildx 的 ARG 缓存键失效 → vite build 必定重新执行
if [ "${DOCKERFILE##*/}" = "web.Dockerfile" ]; then
SOURCE_HASH=$(git rev-parse HEAD:apps/web 2>/dev/null || echo "")
if [ -n "$SOURCE_HASH" ]; then
echo "Web cache bust: SOURCE_HASH=${SOURCE_HASH}"
BUILD_ARGS="$BUILD_ARGS --build-arg SOURCE_HASH=${SOURCE_HASH}"
else
echo "⚠️ 无法计算 apps/web tree hash,跳过 cache bust"
fi
fi
BUILDER_NAME="ci-builder-persist"
if ! docker buildx inspect "$BUILDER_NAME" > /dev/null 2>&1; then
echo "持久 builder 不存在,创建中..."
-12
View File
@@ -31,18 +31,6 @@ for arg in "$@"; do
BUILD_ARGS="$BUILD_ARGS --build-arg $arg"
done
# Web 镜像 cache bust:计算 apps/web/ 的 git tree hash
# 当源码变化时 hash 变化,buildx 的 ARG 缓存键失效 → vite build 必定重新执行
if [ "${DOCKERFILE##*/}" = "web.Dockerfile" ]; then
SOURCE_HASH=$(git rev-parse HEAD:apps/web 2>/dev/null || echo "")
if [ -n "$SOURCE_HASH" ]; then
echo "Web cache bust: SOURCE_HASH=${SOURCE_HASH}"
BUILD_ARGS="$BUILD_ARGS --build-arg SOURCE_HASH=${SOURCE_HASH}"
else
echo "⚠️ 无法计算 apps/web tree hash,跳过 cache bust"
fi
fi
# 确保持久 builder 存在并使用(幂等)
if ! docker buildx inspect "$BUILDER_NAME" > /dev/null 2>&1; then
echo "持久 builder 不存在,创建中..."
-24
View File
@@ -4,7 +4,6 @@
# - docker-container driver, host 网络
# - 层缓存保存在 buildkit 容器及其 _state 命名卷中,job 结束不清理
# - 宿主机 ci-docker-cleanup.sh 已豁免该 builder
# - 每次执行自动同步宿主机 docker config 到 BuildKit 容器(确保 registry 认证)
# 用法: bash scripts/ci/ensure_persistent_builder.sh
set -eu
@@ -35,26 +34,3 @@ docker buildx use "$BUILDER"
docker buildx inspect "$BUILDER" --bootstrap
echo "✅ builder ready"
docker buildx ls | head -5
# === 同步宿主机 docker config 到 BuildKit 容器(确保 registry 认证) ===
# BuildKit 容器名遵循 docker buildx 命名规则: buildx_buildkit_<builder-name>_0
BUILDKIT_CONTAINER="buildx_buildkit_${BUILDER}_0"
if docker inspect "$BUILDKIT_CONTAINER" >/dev/null 2>&1; then
# 宿主机 docker config 路径
HOST_DOCKER_CONFIG="/root/.docker/config.json"
if [ -f "$HOST_DOCKER_CONFIG" ]; then
echo "=== 同步 docker config 到 BuildKit 容器 ==="
# 确保容器内 .docker 目录存在
docker exec "$BUILDKIT_CONTAINER" mkdir -p /root/.docker
# 拷贝 config.json
docker cp "$HOST_DOCKER_CONFIG" "$BUILDKIT_CONTAINER:/root/.docker/config.json"
echo "✅ docker config 已同步到 BuildKit 容器"
else
echo "⚠️ 宿主机 docker config 不存在: $HOST_DOCKER_CONFIG(跳过同步)"
fi
else
echo "⚠️ BuildKit 容器不存在: $BUILDKIT_CONTAINER(跳过 config 同步)"
fi
+3 -4
View File
@@ -303,10 +303,9 @@ def main():
"CI/CD Pipeline / CI Gate (pull_request)",
]
REQUIRED_CONTEXTS_APPROVE = [
# CI 优化后 job 名称(2026-08):Code Quality 拆分为 Style+Security,Type Check+Migration 合并为 Python
"CI/CD Pipeline / Validate - Style (pull_request)",
"CI/CD Pipeline / Validate - Security (pull_request)",
"CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request)",
"CI/CD Pipeline / Validate - Code Quality (pull_request)",
"CI/CD Pipeline / Validate - Type Check (mypy) (pull_request)",
"CI/CD Pipeline / Validate - Migration (alembic) (pull_request)",
"CI/CD Pipeline / Frontend Lint (pull_request)",
]
FRONTEND_ONLY_CONTEXT = [
+52 -27
View File
@@ -52,45 +52,70 @@ bash scripts/ci/step_install_ffmpeg.sh
# 需要用宿主机IP访问映射端口
# 检测策略:host.docker.internal -> docker0桥接IP -> 容器IP直连 -> 默认网关 -> 127.0.0.1
detect_docker_host() {
# 目标:找到宿主机IP(DooD模式下CI容器访问宿主机上其他容器用)
# 不依赖特定端口TCP探测,直接用网络拓扑信息
local test_port="${1:-${CI_LOCAL_PG_PORT}}"
# 候选IP列表
local candidates=()
# 1. host.docker.internal(runner配置了--add-host时可用)
if python3 -c "import socket; socket.gethostbyname('host.docker.internal')" 2>/dev/null; then
echo "host.docker.internal"
return 0
candidates+=("host.docker.internal")
fi
# 2. 默认网关(Docker bridge模式下网关即宿主机)
# 2. docker0 桥接网关 (172.17.0.1)
candidates+=("172.17.0.1")
# 3. 默认网关(容器网络的网关即宿主机)
local gw=""
gw=$(ip route 2>/dev/null | grep default | awk '{print $3}' | head -1)
if [ -n "$gw" ] && [ "$gw" != "127.0.0.1" ]; then
echo "$gw"
return 0
candidates+=("$gw")
fi
# 3. docker0 桥接网关
if [ -n "$(ip addr show docker0 2>/dev/null)" ]; then
echo "172.17.0.1"
return 0
# 4. 宿主机可能的IP:容器同网段的.1或.254
local my_ip=""
my_ip=$(hostname -I 2>/dev/null | awk '{print $1}')
if [ -n "$my_ip" ]; then
# 尝试同网段的常见宿主机IP
local subnet=$(echo "$my_ip" | cut -d. -f1-3)
candidates+=("${subnet}.1")
candidates+=("${subnet}.254")
fi
# 4. 通过 git server hostname 反查(runner 配置了 ExtraHosts host-gateway)
local git_host_ip=""
git_host_ip=$(python3 -c "import socket; print(socket.gethostbyname('git.xiaoxiajianji.com'))" 2>/dev/null || true)
if [ -n "$git_host_ip" ] && [ "$git_host_ip" != "127.0.0.1" ]; then
echo "$git_host_ip"
return 0
fi
# 5. 127.0.0.1 最后尝试
candidates+=("127.0.0.1")
# 5. 最终 fallback
# 测试每个候选IP
for candidate in "${candidates[@]}"; do
if python3 -c "
import socket
s = socket.socket()
s.settimeout(2)
try:
s.connect(('$candidate', $test_port))
s.close()
print('ok')
except:
pass
" 2>/dev/null | grep -q ok; then
echo "$candidate"
return 0
fi
done
# 都失败则返回127.0.0.1
echo "127.0.0.1"
return 0
return 1
}
# 获取宿主机IP(先尝试用共享PG端口5433测试,再回退到其他端口)
if [ -S /var/run/docker.sock ]; then
DOCKER_HOST_IP=$(detect_docker_host)
# 先用共享PG端口5433探测
DOCKER_HOST_IP=$(detect_docker_host "${CI_SHARED_PG_PORT}")
if [ "$DOCKER_HOST_IP" = "127.0.0.1" ]; then
# 如果共享PG端口探测失败,说明不在DooD或共享PG不可用,再试其他端口
DOCKER_HOST_IP=$(detect_docker_host 22)
fi
echo "检测到DooD模式(/var/run/docker.sock已挂载),宿主机地址: $DOCKER_HOST_IP"
else
DOCKER_HOST_IP="127.0.0.1"
@@ -172,8 +197,8 @@ if [ "$USE_SHARED_PG" = "true" ]; then
# 创建主数据库(xdist 模式下各 worker 会创建自己的数据库,主库作为 fallback)
echo "创建主测试数据库: $CI_DB_NAME"
PGPASSWORD="$SHARED_PG_PASSWORD" python3 -c "
import psycopg
conn = psycopg.connect(host='$SHARED_PG_HOST', port=$SHARED_PG_PORT, user='$SHARED_PG_USER', password='$SHARED_PG_PASSWORD', dbname='postgres')
import psycopg2
conn = psycopg2.connect(host='$SHARED_PG_HOST', port=$SHARED_PG_PORT, user='$SHARED_PG_USER', password='$SHARED_PG_PASSWORD', dbname='postgres')
conn.autocommit = True
cur = conn.cursor()
cur.execute(f'DROP DATABASE IF EXISTS \"$CI_DB_NAME\" WITH (FORCE)')
@@ -202,7 +227,7 @@ else
postgres:16
PG_PORT=$(docker port "$PG_CONTAINER" ${CI_LOCAL_PG_PORT}/tcp | cut -d: -f2)
echo "PostgreSQL port: $PG_PORT"
export DATABASE_URL="postgresql+psycopg://postgres:postgres@${PG_HOST}:${PG_PORT}/${CI_DEFAULT_DB}"
export DATABASE_URL="postgresql+psycopg://${CI_SHARED_PG_USER}:${CI_SHARED_PG_PASSWORD}@${PG_HOST}:${PG_PORT}/${CI_DEFAULT_DB}"
# 等待容器健康
for i in $(seq 1 30); do
@@ -279,8 +304,8 @@ if [ "$USE_SHARED_PG" = "true" ]; then
# 清理所有以 CI_DB_NAME 开头的数据库(主库 + worker 库)
PGPASSWORD="${SHARED_PG_PASSWORD}" python3 -c "
import psycopg
conn = psycopg.connect(host='${SHARED_PG_HOST}', port=${SHARED_PG_PORT}, user='${SHARED_PG_USER}', password='${SHARED_PG_PASSWORD}', dbname='postgres')
import psycopg2
conn = psycopg2.connect(host='${SHARED_PG_HOST}', port=${SHARED_PG_PORT}, user='${SHARED_PG_USER}', password='${SHARED_PG_PASSWORD}', dbname='postgres')
conn.autocommit = True
cur = conn.cursor()
@@ -319,4 +344,4 @@ python3 scripts/ci_coverage_summary.py
set -e
echo ""
echo "=== CI Integration Tests 全部通过 ✅ ==="
echo "=== CI Integration Tests 全部通过 ✅ ==="
+47 -106
View File
@@ -1,136 +1,77 @@
#!/bin/bash
# CI Unit Tests Job 主脚本
# 包含:依赖缓存、增量测试选择、覆盖率测试、diff覆盖率门禁
# 包含:依赖安装、增量测试选择、覆盖率测试、diff覆盖率门禁
set -eu
JOB_NAME="${1:-Unit Tests}"
echo "=== CI Unit Tests 开始 ==="
# --- 依赖缓存检查 ---
# 如果 requirements 文件未变化且依赖已安装,跳过 pip install(持久 runner 优化)
REQ_HASH_FILE="/tmp/.ci_unit_tests_req_hash"
CURRENT_REQ_HASH=""
if [ -f requirements-base.txt ] && [ -f requirements.txt ] && [ -f requirements-dev.txt ]; then
CURRENT_REQ_HASH=$(cat requirements-base.txt requirements.txt requirements-dev.txt | md5sum | cut -d' ' -f1)
fi
SKIP_PIP_INSTALL=false
if [ -n "$CURRENT_REQ_HASH" ] && [ -f "$REQ_HASH_FILE" ]; then
CACHED_HASH=$(cat "$REQ_HASH_FILE")
if [ "$CACHED_HASH" = "$CURRENT_REQ_HASH" ]; then
# 验证关键包是否还在
if python3 -c "import pytest; import celery" 2>/dev/null; then
echo "✅ 依赖无变化 (hash=$CURRENT_REQ_HASH),跳过 pip install"
SKIP_PIP_INSTALL=true
else
echo "⚠️ 依赖 hash 匹配但关键包缺失,重新安装"
fi
fi
fi
# --- 安装依赖 ---
if [ "$SKIP_PIP_INSTALL" = "false" ]; then
echo ""
echo "=== 安装 Python 依赖 ==="
# pip install 带重试(网络不稳定时自动重试),合并为一次调用减少开销
for i in 1 2 3; do
python3 -m pip install -q -r requirements-base.txt -r requirements.txt -r requirements-dev.txt && break
echo "pip install 失败,重试 $i/3..."
[ $i -eq 3 ] && exit 1
sleep 5
done
# 保存 hash 标记
if [ -n "$CURRENT_REQ_HASH" ]; then
echo "$CURRENT_REQ_HASH" > "$REQ_HASH_FILE"
fi
fi
echo ""
echo "=== 安装 Python 依赖 ==="
# pip install 带重试(网络不稳定时自动重试)
for i in 1 2 3; do
python3 -m pip install -q -r requirements-base.txt && break
echo "pip install requirements-base.txt 失败,重试 $i/3..."
[ $i -eq 3 ] && exit 1
sleep 5
done
for i in 1 2 3; do
python3 -m pip install -q -r requirements.txt && break
echo "pip install requirements.txt 失败,重试 $i/3..."
[ $i -eq 3 ] && exit 1
sleep 5
done
for i in 1 2 3; do
python3 -m pip install -q -r requirements-dev.txt && break
echo "pip install requirements-dev.txt 失败,重试 $i/3..."
[ $i -eq 3 ] && exit 1
sleep 5
done
pytest --version
# 双保险:确保numpy已安装
echo "=== 验证 numpy 安装 ==="
SKIP_NUMPY_TESTS=0
if python3 -c "import numpy; assert numpy.__version__ == '1.26.4'" 2>/dev/null; then
echo "✅ numpy 1.26.4 已就绪(缓存命中)"
else
echo "需要安装 numpy 1.26.4..."
python3 -m pip install numpy==1.26.4 || {
echo "❌ numpy 首次安装失败,尝试不使用缓存重新安装..."
python3 -m pip install --no-cache-dir numpy==1.26.4 || {
echo "⚠️ numpy 安装失败,跳过需要 numpy 的测试"
SKIP_NUMPY_TESTS=1
}
python3 -m pip install numpy==1.26.4 || {
echo "❌ numpy 首次安装失败,尝试不使用缓存重新安装..."
python3 -m pip install --no-cache-dir numpy==1.26.4 || {
echo "⚠️ numpy 安装失败,跳过需要 numpy 的测试"
SKIP_NUMPY_TESTS=1
}
fi
}
if [ "$SKIP_NUMPY_TESTS" = "0" ]; then
python3 -c "import numpy; print(f'✅ numpy {numpy.__version__} 就绪')" || {
python3 -c "import numpy; print(f'✅ numpy {numpy.__version__} 安装成功')" || {
echo "⚠️ numpy 导入失败,跳过需要 numpy 的测试"
SKIP_NUMPY_TESTS=1
}
fi
# --- 增量测试选择(PR + push 均支持) ---
# --- 增量测试选择(仅PR) ---
UNIT_TEST_MODE="full"
SELECTED_TEST_FILES="tests/unit"
IS_PULL_REQUEST=false
IS_PUSH=false
[ "${GITHUB_EVENT_NAME:-}" = "pull_request" ] && IS_PULL_REQUEST=true
[ "${GITHUB_EVENT_NAME:-}" = "push" ] && IS_PUSH=true
if ($IS_PULL_REQUEST || $IS_PUSH) && [ -n "${GITHUB_TOKEN:-}" ]; then
if [ "${GITHUB_EVENT_NAME:-}" = "pull_request" ] && [ -n "${GITHUB_TOKEN:-}" ]; then
echo ""
echo "=== 增量测试选择 ==="
CHANGED_FILES=""
if $IS_PULL_REQUEST; then
PR_NUMBER=$(echo "$GITHUB_REF" | sed 's|refs/pull/||; s|/.*||')
API_URL="${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/files?limit=300"
CHANGED_FILES=$(curl -s -H "Authorization: token ${GITHUB_TOKEN}" "$API_URL" \
| python3 -c "import sys,json; [print(f['filename']) for f in json.load(sys.stdin) if f['status'] != 'removed']")
elif $IS_PUSH && [ -n "${GITHUB_SHA:-}" ]; then
# Push 事件:通过 GitHub API 获取本次 push 改动的文件
API_URL="${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/commits/${GITHUB_SHA}"
RESPONSE=$(curl -s -H "Authorization: token ${GITHUB_TOKEN}" \
-H "Accept: application/vnd.github.v3.diff" "$API_URL" 2>/dev/null || echo "")
if [ -n "$RESPONSE" ]; then
CHANGED_FILES=$(echo "$RESPONSE" | grep '^diff --git' | sed 's|diff --git a/\(.*\) b/.*|\1|' || echo "")
fi
# 备用方案:获取 previous commit SHA 再查 API
if [ -z "$CHANGED_FILES" ]; then
PREV_SHA=$(curl -s -H "Authorization: token ${GITHUB_TOKEN}" \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/commits?sha=${GITHUB_SHA}&per_page=2" \
| python3 -c "import sys,json; commits=json.load(sys.stdin); print(commits[1]['sha'] if len(commits)>1 else '')" 2>/dev/null || echo "")
if [ -n "$PREV_SHA" ]; then
COMPARE_URL="${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/compare/${PREV_SHA}...${GITHUB_SHA}"
CHANGED_FILES=$(curl -s -H "Authorization: token ${GITHUB_TOKEN}" "$COMPARE_URL" \
| python3 -c "import sys,json; data=json.load(sys.stdin); [print(f['filename']) for f in data.get('files',[]) if f['status'] != 'removed']" 2>/dev/null || echo "")
fi
fi
fi
PR_NUMBER=$(echo "$GITHUB_REF" | sed 's|refs/pull/||; s|/.*||')
API_URL="${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/files?limit=300"
CHANGED_FILES=$(curl -s -H "Authorization: token ${GITHUB_TOKEN}" "$API_URL" | python3 -c "import sys,json; [print(f['filename']) for f in json.load(sys.stdin) if f['status'] != 'removed']")
echo "改动文件数: $(echo "$CHANGED_FILES" | grep -c . || echo 0)"
if [ -n "$CHANGED_FILES" ]; then
set +e
CHANGED_FILES="$CHANGED_FILES" \
SELECTED_TESTS_OUTPUT=/tmp/selected_tests.txt \
python3 scripts/ci/select_unit_tests.py
SELECT_EXIT=$?
set -e
if [ $SELECT_EXIT -eq 0 ]; then
UNIT_TEST_MODE="incremental"
TEST_FILES=$(cat /tmp/selected_tests.txt | tr '\n' ' ')
SELECTED_TEST_FILES="$TEST_FILES"
echo "增量模式: $(cat /tmp/selected_tests.txt | wc -l) 个测试文件"
else
echo "全量模式(增量选择失败)"
fi
set +e
CHANGED_FILES="$CHANGED_FILES" \
SELECTED_TESTS_OUTPUT=/tmp/selected_tests.txt \
python3 scripts/ci/select_unit_tests.py
SELECT_EXIT=$?
set -e
if [ $SELECT_EXIT -eq 0 ]; then
UNIT_TEST_MODE="incremental"
TEST_FILES=$(cat /tmp/selected_tests.txt | tr '\n' ' ')
SELECTED_TEST_FILES="$TEST_FILES"
echo "增量模式: $(cat /tmp/selected_tests.txt | wc -l) 个测试文件"
else
echo "无法获取改动文件列表,使用全量模式"
echo "全量模式"
fi
fi
@@ -175,7 +116,7 @@ if [ "${GITHUB_EVENT_NAME:-}" = "pull_request" ] && [ -n "${GITHUB_TOKEN:-}" ];
PR_CODE_DIR="/tmp/pr-code-$$"
mkdir -p "$PR_CODE_DIR"
# 备份PR代码(含coverage.xml,diff-cover需要用到)
# 备份PR代码(含coverage.xml,diff-cover需要用到
find . -maxdepth 1 -mindepth 1 ! -name 'diff_coverage.html' -exec cp -r {} "$PR_CODE_DIR/" \;
rm -rf .git
git init > /dev/null 2>&1
+4 -4
View File
@@ -409,8 +409,8 @@ except:
echo "创建测试数据库: $CI_DB_NAME"
PGPASSWORD="$SHARED_PG_PASSWORD" python3 -c "
import psycopg
conn = psycopg.connect(host='$SHARED_PG_HOST', port=$SHARED_PG_PORT, user='$SHARED_PG_USER', password='$SHARED_PG_PASSWORD', dbname='postgres')
import psycopg2
conn = psycopg2.connect(host='$SHARED_PG_HOST', port=$SHARED_PG_PORT, user='$SHARED_PG_USER', password='$SHARED_PG_PASSWORD', dbname='postgres')
conn.autocommit = True
cur = conn.cursor()
cur.execute(f'DROP DATABASE IF EXISTS \"$CI_DB_NAME\" WITH (FORCE)')
@@ -431,8 +431,8 @@ conn.close()
# 清理数据库
echo "清理测试数据库: $CI_DB_NAME"
PGPASSWORD="$SHARED_PG_PASSWORD" python3 -c "
import psycopg
conn = psycopg.connect(host='$SHARED_PG_HOST', port=$SHARED_PG_PORT, user='$SHARED_PG_USER', password='$SHARED_PG_PASSWORD', dbname='postgres')
import psycopg2
conn = psycopg2.connect(host='$SHARED_PG_HOST', port=$SHARED_PG_PORT, user='$SHARED_PG_USER', password='$SHARED_PG_PASSWORD', dbname='postgres')
conn.autocommit = True
cur = conn.cursor()
cur.execute(f'DROP DATABASE IF EXISTS \"$CI_DB_NAME\" WITH (FORCE)')
+14 -26
View File
@@ -1,39 +1,18 @@
#!/bin/sh
# CI 公共步骤:Checkout 代码(流式下载+解压,带重试)
# CI 公共步骤:Checkout 代码(带重试)
# 用法:直接 source 或调用,需要 GITHUB_TOKEN 环境变量
set -eu
python3 - <<'PY'
import io, os, sys, tarfile, time, urllib.request, urllib.error
import io, os, tarfile, time, urllib.request, urllib.error
url = f"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz"
request = urllib.request.Request(url, headers={"Authorization": f"token {os.environ['GITHUB_TOKEN']}"})
last_err = None
for attempt in range(5):
try:
with urllib.request.urlopen(request, timeout=120) as response:
# 流式读取:先读少量数据确认连接成功,再大块读取
first_chunk = response.read(8192)
buf = io.BytesIO()
buf.write(first_chunk)
while True:
chunk = response.read(65536)
if not chunk:
break
buf.write(chunk)
buf.seek(0)
with tarfile.open(fileobj=buf, mode='r:gz') as tar:
root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'
for member in tar.getmembers():
name = member.name
if name == root_prefix[:-1]:
continue
if name.startswith(root_prefix):
member.name = name[len(root_prefix):]
if member.name:
tar.extract(member, '.')
break
archive = response.read()
break
except urllib.error.HTTPError as e:
last_err = e
if e.code >= 500 and attempt < 4:
@@ -52,5 +31,14 @@ for attempt in range(5):
raise
else:
raise last_err
with tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:
root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'
for member in tar.getmembers():
name = member.name
if name == root_prefix[:-1]:
continue
if name.startswith(root_prefix):
member.name = name[len(root_prefix):]
if member.name:
tar.extract(member, '.')
PY
# CI pipeline speedup batch 1
+54 -38
View File
@@ -1,20 +1,14 @@
#!/bin/bash
# CI Validate: 安全扫描(validate-security)
# 包含:密钥扫描、bandit 安全扫描(仅告警)、pip-audit 依赖漏洞(仅告警)、CI 脚本语法校验
# CI Validate: 代码质量与安全扫描(并行Job 1/3)
# 包含:密钥扫描、格式检查、安全扫描、依赖漏洞、死代码检测、脚本语法校验
set -eu
echo "=== CI Validate: 安全扫描 ==="
echo "=== CI Validate: 代码质量与安全扫描 ==="
# --- 密钥检测 ---
echo ""
echo "=== [1/4] Secret detection (detect-secrets) ==="
python3 -m pip install -q --no-cache-dir detect-secrets || {
echo "⚠️ detect-secrets install failed, retrying without cache..."
python3 -m pip install -q --no-cache-dir --no-binary :all: detect-secrets || {
echo "❌ detect-secrets install failed after retry"
exit 1
}
}
echo "=== [1/6] Secret detection (detect-secrets) ==="
python3 -m pip install -q detect-secrets
detect-secrets --version
detect-secrets scan \
@@ -62,9 +56,23 @@ for fpath, items in data.get('results', {}).items():
fi
echo "✅ Secret scan passed"
# --- Bandit 安全扫描(仅告警)---
# --- 代码质量检查(全量,PR 和 push 统一标准)---
# 历史:PR 侧用增量检查以加速,但会导致 push 侧全量检查失败时 PR 侧感知不到
# 现在统一全量检查,确保 CI 真正保护主分支(black/isort/ruff 全量仅多几十秒)
echo ""
echo "=== [2/4] Security scan (bandit, advisory only) ==="
echo "=== [2/6] Code quality checks (full scan) ==="
SCAN_MODE="full"
echo "Full scan mode"
python3 -m compileall -q alembic apps packages tests scripts
python3 -m black --check --fast alembic apps packages tests scripts
python3 -m isort --check-only alembic apps packages tests scripts
python3 -m ruff check apps packages tests --statistics
echo "✅ Code quality checks passed"
# --- Bandit 安全扫描(仅告警) ---
echo ""
echo "=== [3/6] Security scan (bandit, advisory only) ==="
set +e
bandit -r apps packages -q -ll
BANDIT_EXIT=$?
@@ -75,34 +83,42 @@ else
echo "✅ Bandit security scan passed"
fi
# --- Pip-audit 依赖漏洞扫描(仅告警)---
# --- Pip-audit 依赖漏洞扫描(仅告警) ---
echo ""
echo "=== [3/4] Python dependency vulnerability scan (pip-audit, advisory only) ==="
python3 -m pip install -q --no-cache-dir pip-audit || {
echo "⚠️ pip-audit install failed (cache issue?), retrying..."
python3 -m pip install -q --no-cache-dir pip-audit || {
echo "⚠️ pip-audit unavailable, skipping dependency vulnerability scan (advisory)"
pip-audit --version 2>/dev/null || true
}
}
if command -v pip-audit >/dev/null 2>&1 || python3 -m pip show pip-audit >/dev/null 2>&1; then
pip-audit --version
EXIT_CODE=0
for req_file in requirements.txt requirements-base.txt requirements-dev.txt; do
if [ -f "$req_file" ]; then
echo "--- Scanning $req_file ---"
pip-audit -r "$req_file" --desc on 2>&1 | head -40 || EXIT_CODE=$?
echo ""
fi
done
echo "pip-audit scan completed (advisory mode - warnings only, not blocking CI)"
else
echo "⚠️ pip-audit not available, skipping dependency vulnerability scan (advisory)"
fi
echo "=== [4/6] Python dependency vulnerability scan (pip-audit, advisory only) ==="
python3 -m pip install -q pip-audit
pip-audit --version
EXIT_CODE=0
for req_file in requirements.txt requirements-base.txt requirements-dev.txt; do
if [ -f "$req_file" ]; then
echo "--- Scanning $req_file ---"
pip-audit -r "$req_file" --desc on 2>&1 | head -40 || EXIT_CODE=$?
echo ""
fi
done
echo "pip-audit scan completed (advisory mode - warnings only, not blocking CI)"
# --- Vulture 死代码检测(仅告警) ---
echo ""
echo "=== [5/6] Dead code detection (vulture, advisory only) ==="
set +e
python3 -m pip install -q vulture
vulture --version
echo "告警模式,不阻断CI。置信度>=90%建议尽快确认。"
echo ""
vulture apps packages scripts \
--exclude "tests,test,migrations,.gitea,docs,node_modules,site-packages,*/test_*.py,*/conftest.py" \
--min-confidence 70 \
2>&1 | sort -t'(' -k2 -rn | head -80
echo ""
echo "=== vulture scan summary ==="
echo "发现潜在死代码(可能包含框架装饰器注册的函数,为误报)"
echo "建议:定期人工审查高置信度(>=90%)条目"
set -e
# --- CI脚本语法校验 ---
echo ""
echo "=== [4/4] CI & shell scripts syntax validation ==="
echo "=== [6/6] CI & shell scripts syntax validation ==="
SYNTAX_ERROR=0
# 检查所有 CI shell 脚本
for script in scripts/ci/*.sh; do
@@ -138,4 +154,4 @@ fi
echo "✅ All CI scripts syntax OK"
echo ""
echo "=== CI Validate: 安全扫描 全部通过 ✅ ==="
echo "=== CI Validate: 代码质量与安全扫描 全部通过 ✅ ==="
+46 -27
View File
@@ -59,40 +59,56 @@ echo ""
# ============================================================
detect_docker_host() {
# 目标:找到宿主机IP(DooD模式下CI容器访问宿主机上其他容器用)
# 不依赖特定端口TCP探测,直接用网络拓扑信息
local test_port="${1:-${CI_LOCAL_PG_PORT}}"
# 1. host.docker.internal(runner配置了--add-host时可用)
local candidates=()
# 1. host.docker.internal
if python3 -c "import socket; socket.gethostbyname('host.docker.internal')" 2>/dev/null; then
echo "host.docker.internal"
return 0
candidates+=("host.docker.internal")
fi
# 2. 默认网关(Docker bridge模式下网关即宿主机)
# 2. docker0 桥接网关
candidates+=("172.17.0.1")
# 3. 默认网关
local gw=""
gw=$(ip route 2>/dev/null | grep default | awk '{print $3}' | head -1)
if [ -n "$gw" ] && [ "$gw" != "127.0.0.1" ]; then
echo "$gw"
return 0
candidates+=("$gw")
fi
# 3. docker0 桥接网关
if [ -n "$(ip addr show docker0 2>/dev/null)" ]; then
echo "172.17.0.1"
return 0
# 4. 宿主机同网段的.1或.254
local my_ip=""
my_ip=$(hostname -I 2>/dev/null | awk '{print $1}')
if [ -n "$my_ip" ]; then
local subnet=$(echo "$my_ip" | cut -d. -f1-3)
candidates+=("${subnet}.1")
candidates+=("${subnet}.254")
fi
# 4. 通过 git server hostname 反查(runner 配置了 ExtraHosts host-gateway)
local git_host_ip=""
git_host_ip=$(python3 -c "import socket; print(socket.gethostbyname('git.xiaoxiajianji.com'))" 2>/dev/null || true)
if [ -n "$git_host_ip" ] && [ "$git_host_ip" != "127.0.0.1" ]; then
echo "$git_host_ip"
return 0
fi
# 5. 127.0.0.1 最后尝试
candidates+=("127.0.0.1")
for candidate in "${candidates[@]}"; do
if python3 -c "
import socket
s = socket.socket()
s.settimeout(2)
try:
s.connect(('$candidate', $test_port))
s.close()
print('ok')
except:
pass
" 2>/dev/null | grep -q ok; then
echo "$candidate"
return 0
fi
done
# 5. 最终 fallback
echo "127.0.0.1"
return 0
return 1
}
# 指数退避TCP连接检查
@@ -116,7 +132,10 @@ wait_tcp_ready() {
# 获取宿主机IP
if [ -S /var/run/docker.sock ]; then
DOCKER_HOST_IP=$(detect_docker_host)
DOCKER_HOST_IP=$(detect_docker_host "${CI_SHARED_PG_PORT}")
if [ "$DOCKER_HOST_IP" = "127.0.0.1" ]; then
DOCKER_HOST_IP=$(detect_docker_host 22)
fi
echo "检测到DooD模式,宿主机地址: $DOCKER_HOST_IP"
else
DOCKER_HOST_IP="127.0.0.1"
@@ -150,8 +169,8 @@ if [ "$USE_SHARED_PG" = "true" ]; then
echo "创建测试数据库: $CI_DB_NAME"
PGPASSWORD="$SHARED_PG_PASSWORD" python3 -c "
import psycopg
conn = psycopg.connect(host='$SHARED_PG_HOST', port=$SHARED_PG_PORT, user='$SHARED_PG_USER', password='$SHARED_PG_PASSWORD', dbname='postgres')
import psycopg2
conn = psycopg2.connect(host='$SHARED_PG_HOST', port=$SHARED_PG_PORT, user='$SHARED_PG_USER', password='$SHARED_PG_PASSWORD', dbname='postgres')
conn.autocommit = True
cur = conn.cursor()
cur.execute(f'DROP DATABASE IF EXISTS \"$CI_DB_NAME\" WITH (FORCE)')
@@ -166,8 +185,8 @@ conn.close()
echo ""
echo "清理测试数据库: $CI_DB_NAME"
PGPASSWORD="$SHARED_PG_PASSWORD" python3 -c "
import psycopg
conn = psycopg.connect(host='$SHARED_PG_HOST', port=$SHARED_PG_PORT, user='$SHARED_PG_USER', password='$SHARED_PG_PASSWORD', dbname='postgres')
import psycopg2
conn = psycopg2.connect(host='$SHARED_PG_HOST', port=$SHARED_PG_PORT, user='$SHARED_PG_USER', password='$SHARED_PG_PASSWORD', dbname='postgres')
conn.autocommit = True
cur = conn.cursor()
cur.execute(f'DROP DATABASE IF EXISTS \"$CI_DB_NAME\" WITH (FORCE)')
@@ -194,7 +213,7 @@ else
postgres:16-alpine
PG_PORT=$(docker port "$PG_CONTAINER" ${CI_LOCAL_PG_PORT}/tcp | cut -d: -f2)
echo "PostgreSQL port: $PG_PORT"
export DATABASE_URL="postgresql+psycopg://postgres:postgres@${PG_HOST}:${PG_PORT}/${CI_DEFAULT_DB}"
export DATABASE_URL="postgresql+psycopg://${CI_SHARED_PG_USER}:${CI_SHARED_PG_PASSWORD}@${PG_HOST}:${PG_PORT}/${CI_DEFAULT_DB}"
# 等待容器健康
for i in $(seq 1 30); do
-42
View File
@@ -1,42 +0,0 @@
#!/bin/bash
# CI Validate: 代码风格检查(validate-style)
# 包含:Python 字节码编译、black 格式、isort 排序、ruff lint、vulture 死代码(仅告警)
set -eu
echo "=== CI Validate: 代码风格检查 ==="
# --- Python 字节码编译 ---
echo ""
echo "=== [1/3] Python bytecode compilation ==="
python3 -m compileall -q alembic apps packages tests scripts
echo "✅ Bytecode compilation passed"
# --- 代码格式检查(全量)---
echo ""
echo "=== [2/3] Code formatting (black + isort + ruff) ==="
echo "Full scan mode"
python3 -m black --check --fast alembic apps packages tests scripts
python3 -m isort --check-only alembic apps packages tests scripts
python3 -m ruff check apps packages tests --statistics
echo "✅ Code formatting checks passed"
# --- Vulture 死代码检测(仅告警)---
echo ""
echo "=== [3/3] Dead code detection (vulture, advisory only) ==="
set +e
python3 -m pip install -q --no-cache-dir vulture || echo "⚠️ vulture install failed, skipping dead code detection"
vulture --version
echo "告警模式,不阻断CI。置信度>=90%建议尽快确认。"
echo ""
vulture apps packages scripts \
--exclude "tests,test,migrations,.gitea,docs,node_modules,site-packages,*/test_*.py,*/conftest.py" \
--min-confidence 70 \
2>&1 | sort -t'(' -k2 -rn | head -80
echo ""
echo "=== vulture scan summary ==="
echo "发现潜在死代码(可能包含框架装饰器注册的函数,为误报)"
echo "建议:定期人工审查高置信度(>=90%)条目"
set -e
echo ""
echo "=== CI Validate: 代码风格检查 全部通过 ✅ ==="

Some files were not shown because too many files have changed in this diff Show More