Compare commits

...

40 Commits

Author SHA1 Message Date
xiaoxia c613f35662 Merge pull request 'fix(ai-avatar): 修复三个bug——封面重影/标题字号缩放/对口型音频截断' (#1876) from fix/ai-avatar-three-bugs-0913 into develop
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 2s
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / Check push changed paths (push) Successful in 10s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 17s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 19s
CI/CD Pipeline / Build Staging API Image (push) Successful in 1m0s
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been skipped
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Successful in 1m53s
CI/CD Pipeline / Integration Tests (push) Successful in 1m59s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 1m0s
CI/CD Pipeline / Validate - Style (push) Successful in 2m39s
CI/CD Pipeline / ACR Image Cleanup (push) Successful in 1m35s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1m35s
CI/CD Pipeline / Frontend Unit Tests (push) Successful in 5m21s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 3m15s
CI/CD Pipeline / Validate - Security (push) Successful in 5m37s
CI/CD Pipeline / Unit Tests (push) Successful in 8m32s
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / CI Gate (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Canary Release to Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
2026-09-13 14:15:31 +08:00
CI Bot 9be89484e6 style: auto-format with black + isort + ruff + prettier [skip ci-format-check]
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 5s
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 26s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 32s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 47s
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 49s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m50s
AI Code Review / AI Code Review (pull_request) Successful in 6m16s
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Failing after 12m8s
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Successful in 1m2s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 10m34s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m36s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 1m43s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 2m4s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 5m28s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 9m34s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Successful in 4s
ACR Cleanup / ACR Image Cleanup (pull_request_target) Successful in 10s
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 17s
2026-09-13 05:32:51 +00:00
灵应 5f128d175d fix(ai-avatar): 修复三个bug——封面重影/标题字号缩放/对口型音频截断
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 1s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 1m12s
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 1m28s
CI/CD Pipeline / Frontend Lint (pull_request) Failing after 1m33s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m49s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m48s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 2m16s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Successful in 2m28s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 2m54s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m8s
CI/CD Pipeline / Validate - Style (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Security (pull_request) Has been cancelled
CI/CD Pipeline / Unit Tests (pull_request) Has been cancelled
CI/CD Pipeline / Build Production API Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Web Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Production (pull_request) Has been cancelled
CI/CD Pipeline / Production Browser E2E (pull_request) Has been cancelled
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
CI/CD Pipeline / CI Gate (pull_request) Has been cancelled
AI Code Review / AI Code Review (pull_request) Has been cancelled
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been cancelled
1. 删除封面展示区重复叠加的标题div(成片帧已含标题,不再CSS叠字)
2. 标题字号按视频分辨率等比缩放,预览容器动态测量宽度计算scale
3. enable_video_loop默认改为true,防止TTS音频长于出镜视频时被截断
2026-09-13 13:28:51 +08:00
xiaoxia 053b00634a feat(ai-avatar): 配音前置 (#1875)
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 3s
CI/CD Pipeline / Check push changed paths (push) Successful in 4s
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / Build Staging API Image (push) Successful in 23s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 23s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 26s
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 49s
CI/CD Pipeline / ACR Image Cleanup (push) Successful in 1m32s
CI/CD Pipeline / Frontend Unit Tests (push) Successful in 3m38s
CI/CD Pipeline / Integration Tests (push) Successful in 3m50s
CI/CD Pipeline / Validate - Style (push) Successful in 4m2s
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Successful in 4m9s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 3m43s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 4m26s
CI/CD Pipeline / Unit Tests (push) Successful in 8m55s
CI/CD Pipeline / Validate - Security (push) Successful in 9m7s
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / CI Gate (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Canary Release to Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
Co-authored-by: xiaoxia <dev@xiaoxiajianji.com>
Co-committed-by: xiaoxia <dev@xiaoxiajianji.com>
2026-09-13 04:22:21 +08:00
xiaoxia 3858acf377 Merge pull request 'fix(ai-avatar): B-roll时间戳根因——逗号分句+优先后端时间戳' (#1874) from fix/ai-avatar-broll-comma-split into develop
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 1s
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / Check push changed paths (push) Successful in 15s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m37s
CI/CD Pipeline / Build Staging API Image (push) Successful in 2m6s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 2m32s
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been skipped
CI/CD Pipeline / Integration Tests (push) Successful in 3m25s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 59s
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Successful in 4m22s
CI/CD Pipeline / Validate - Style (push) Successful in 4m25s
CI/CD Pipeline / ACR Image Cleanup (push) Successful in 2m1s
CI/CD Pipeline / Frontend Unit Tests (push) Successful in 6m28s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 3m25s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 4m7s
CI/CD Pipeline / Validate - Security (push) Successful in 9m30s
CI/CD Pipeline / Unit Tests (push) Successful in 9m39s
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / CI Gate (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Canary Release to Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
fix(ai-avatar): B-roll逗号分句+标题PNG图层+TTS竞态修复+死代码清理

- B-roll分句正则加入中文逗号,放宽后端时间戳校验直接使用
- 标题改为Canvas渲染透明PNG,FFmpeg overlay图片图层替代drawtext,所见即所得
- Celery事务竞态修复:先commit后发任务+worker侧retry防御
- TTS任务超时防护(180s soft/200s hard)+入口日志
- 删除封面死代码(POST /smart-cover裸视频抽帧,所有封面从成片获取)
- 删除DRAWTEXT_BOLD_FONT_SEARCH_PATHS冗余代码
2026-09-13 02:49:04 +08:00
xiaoxia f607b0cec9 fix(test): mock storage.get_download_url,修复_sign_media_url内部调用mock泄漏问题
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 1s
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 1s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 46s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 48s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 53s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 1m20s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m35s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Successful in 1m32s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m46s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 3m51s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 4m26s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 4m28s
AI Code Review / AI Code Review (pull_request) Successful in 6m17s
CI/CD Pipeline / Unit Tests (pull_request) Failing after 9m9s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 11m12s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Failing after 1s
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 8m35s
ACR Cleanup / ACR Image Cleanup (pull_request_target) Successful in 8s
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 32s
2026-09-13 02:21:08 +08:00
xiaoxia 688b35efa8 fix(test): 统一test_lipsync_tts持久化测试upload_url参数与key一致,消除flaky
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 2s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 27s
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 26s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 28s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 49s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Successful in 1m0s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m42s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m40s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 3m34s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 4m3s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 4m8s
AI Code Review / AI Code Review (pull_request) Successful in 6m19s
CI/CD Pipeline / Unit Tests (pull_request) Failing after 8m28s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 9m29s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Failing after 4s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 7m4s
2026-09-13 02:05:30 +08:00
xiaoxia ce6c831cf3 style: prettier format sentences.ts & titleCanvas.ts [skip ci-format-check]
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m1s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m51s
AI Code Review / AI Code Review (pull_request) Successful in 6m47s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 7m42s
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 1s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 1s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 34s
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 34s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 34s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 1m6s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Successful in 1m14s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 2m49s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 3m58s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 3m58s
CI/CD Pipeline / Unit Tests (pull_request) Failing after 8m0s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 9m27s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Failing after 5s
2026-09-13 01:34:47 +08:00
xiaoxia 3267b24433 fix(ai-avatar): 删除未使用变量 startY(TS6133修复)
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 1s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 36s
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 39s
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 1m13s
CI/CD Pipeline / Frontend Lint (pull_request) Failing after 1m20s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Successful in 1m31s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m45s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m46s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 3m45s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 3m50s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 4m16s
AI Code Review / AI Code Review (pull_request) Successful in 6m21s
CI/CD Pipeline / Unit Tests (pull_request) Failing after 8m8s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 9m51s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Failing after 3s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 7m16s
2026-09-13 01:14:15 +08:00
CI Bot 222c4d15a9 style: auto-format with black + isort + ruff + prettier [skip ci-format-check]
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 1s
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 2s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Failing after 2m26s
Preview Deploy / Deploy Preview Environment (pull_request) Failing after 2m35s
CI/CD Pipeline / PR Build Web Image (pull_request) Failing after 2m40s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 2m42s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m59s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Successful in 3m9s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 3m52s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 4m28s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 5m3s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 5m31s
AI Code Review / AI Code Review (pull_request) Successful in 6m41s
CI/CD Pipeline / Unit Tests (pull_request) Failing after 8m37s
CI/CD Pipeline / Validate - Security (pull_request) Has been cancelled
CI/CD Pipeline / Build Production API Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Web Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Production (pull_request) Has been cancelled
CI/CD Pipeline / Production Browser E2E (pull_request) Has been cancelled
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
CI/CD Pipeline / CI Gate (pull_request) Has been cancelled
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been cancelled
2026-09-12 17:05:14 +00:00
xiaoxia 63fb0508be refactor(ai-avatar): 删除封面标题叠加逻辑+废弃路由(封面一律从成片抽帧)
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 1s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 50s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 53s
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 2m8s
CI/CD Pipeline / Frontend Lint (pull_request) Failing after 43s
Preview Deploy / Deploy Preview Environment (pull_request) Failing after 3m12s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Successful in 1m17s
CI/CD Pipeline / PR Build Web Image (pull_request) Failing after 3m56s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 4m51s
CI/CD Pipeline / Validate - Style (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Security (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Has been cancelled
CI/CD Pipeline / Unit Tests (pull_request) Has been cancelled
CI/CD Pipeline / Integration Tests (pull_request) Has been cancelled
CI/CD Pipeline / Build Production API Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Web Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Production (pull_request) Has been cancelled
CI/CD Pipeline / Production Browser E2E (pull_request) Has been cancelled
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
CI/CD Pipeline / CI Gate (pull_request) Has been cancelled
AI Code Review / AI Code Review (pull_request) Has been cancelled
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been cancelled
- 删除前端 generateSmartCover(0处调用的死代码)
- 删除后端 POST /ai-avatar/render/smart-cover(对口型阶段抽封面入口,已废弃)
- 删除 SmartCoverRequest schema(无引用)
- 精简 ai_avatar_cover_service:删除 apply_title_to_cover / _overlay_title_png_on_image,
  封面一律从最终成片(已叠加标题/B-roll)抽帧,不再额外叠加
- 删除 persist_cover_to_oss / generate_smart_cover 的 title_config 参数
2026-09-13 00:58:02 +08:00
xiaoxia 577ec83636 feat(ai-avatar): 标题改为Canvas渲染PNG图层叠加,所见即所得 2026-09-13 00:41:38 +08:00
xiaoxia 6503a74a7c fix(ai-avatar): B-roll时间戳为0根因——逗号分句+优先后端时间戳
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 1s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Failing after 1m27s
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 1m33s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m56s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m56s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 2m57s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Successful in 3m0s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 2m55s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 5m50s
CI/CD Pipeline / Unit Tests (pull_request) Failing after 5m56s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 5m59s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 6m11s
AI Code Review / AI Code Review (pull_request) Successful in 6m44s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 11m28s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Failing after 1s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 8m23s
根因(用户截图文案:'卖花的叫花无缺,卖姜的叫姜子牙,卖菜的蔡文姬,那我修脚阔头的呢,应该叫什么呢'):
1. 分句正则没包含中文逗号「,」,整段被识别为1句;后端静音检测按TTS音频停顿
   切出多句,前端校验 sentenceTimings.length===rawParts.length 条数不等 →
   整个后端精确时间戳被丢弃走降级
2. 降级路径 outputDuration=0(对口型预览阶段最终视频未渲染)→ 所有句子时间估算为0

修复:
- 前端 sentences.ts:
  a. 分句正则加上中英文逗号「,,」
  b. sentenceTimings 校验放宽:只要是有效数组就直接用后端句子列表,
     不再强制条数相等(后端按音频停顿的切法才是真实边界)
- 后端 _split_script_into_sentences:正则同步加逗号,前后端一致
- 补单测验证逗号分隔文案分句
2026-09-12 23:55:07 +08:00
xiaoxia 4a93aaaf4c Merge pull request 'fix(ai-avatar): B-roll时间戳/标题重影/TTS卡死防护' (#1873) from fix/ai-avatar-broll-title-worker-issues into develop
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 2s
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Check push changed paths (push) Successful in 17s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 51s
CI/CD Pipeline / Build Staging API Image (push) Successful in 1m4s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 1m25s
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 1m51s
CI/CD Pipeline / Frontend Unit Tests (push) Successful in 3m52s
CI/CD Pipeline / ACR Image Cleanup (push) Successful in 1m26s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 2m2s
CI/CD Pipeline / Integration Tests (push) Successful in 6m0s
CI/CD Pipeline / Validate - Style (push) Successful in 6m13s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 3m12s
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Successful in 7m59s
CI/CD Pipeline / Unit Tests (push) Successful in 11m28s
CI/CD Pipeline / Validate - Security (push) Successful in 22m15s
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / CI Gate (push) Has been skipped
CI/CD Pipeline / Canary Release to Production (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
fix(ai-avatar): B-roll时间戳/标题重影/Celery事务竞态/TTS超时防护

- 问题1:B-roll时间戳全0,sentences.ts参数顺序错位导致后端精确时间戳被忽略
- 问题2:标题粗体同色描边borderw=3导致字形重影,改为黑色细描边
- 问题3P0:Celery事务竞态(先发任务后commit),worker查不到job静默return永远卡tts_processing;改为先commit再发任务+worker侧self.retry防御
- 补充:TTS任务加soft_time_limit=180s、autoretry_for网络错误、入口INFO日志

单测覆盖:新增198+相关用例通过,全量15054 passed
2026-09-12 22:37:11 +08:00
CI Bot 1a4f475fbf style: auto-format with black + isort + ruff + prettier [skip ci-format-check]
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 2s
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 1s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 42s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 43s
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 45s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Successful in 1m4s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 1m9s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m3s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m41s
AI Code Review / AI Code Review (pull_request) Successful in 6m31s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 7m40s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 7m55s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 8m8s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 9m43s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 10m31s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 23m32s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Successful in 1s
ACR Cleanup / ACR Image Cleanup (pull_request_target) Successful in 8s
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 58s
2026-09-12 14:10:26 +00:00
xiaoxia 2fa6de29bc fix(lipsync): 修复Celery事务竞态导致job永远卡在tts_processing(P0)
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 1s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 30s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 33s
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 1m39s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 1m39s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Successful in 1m47s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m44s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m55s
AI Code Review / AI Code Review (pull_request) Successful in 6m31s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 8m15s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 8m17s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 10m28s
CI/CD Pipeline / Validate - Style (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Security (pull_request) Has been cancelled
CI/CD Pipeline / Build Production API Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Web Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Production (pull_request) Has been cancelled
CI/CD Pipeline / Production Browser E2E (pull_request) Has been cancelled
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
CI/CD Pipeline / CI Gate (pull_request) Has been cancelled
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been cancelled
根因:create_job() 先 apply_async() 发送 Celery 任务,再 db.commit() 提交事务。
worker 是独立进程+独立DB连接,任务<4ms就被消费,但此时 API 事务还未 commit,
worker 查询 job 返回 None → 静默 return 不重试 → job 永远卡在 tts_processing。

修复:
1. API侧(lipsync_service.py):先 db.commit()+db.refresh(job),再 apply_async() 发任务;
   投递失败/MediaKit提交失败分支也各自 commit,确保状态及时落库。
2. Worker侧(lipsync_tts.py):job not found 时改用 self.retry() 递增重试3次
   (1s/3s/7s退避),作为竞态场景的第二道防线;
   增加 autoretry_for=(OSError, ConnectionError) 自动重试网络抖动;
   max_retries 从2调整到5。

子agent在staging实锤:受影响job共5个,手工重投递后全部3秒内完成TTS+提交MediaKit,
证实TTS本身只需要2秒,卡顿完全是因为竞态。

单测:新增 TestCreateJobCommitOrder 验证 commit 在 apply_async 之前。
2026-09-12 21:50:02 +08:00
xiaoxia 831075a9c0 test(ai-avatar): 补充B-roll/标题单测,修正断言
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 2s
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 1m29s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 1m31s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 1m49s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Successful in 2m6s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m20s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 2m49s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m22s
AI Code Review / AI Code Review (pull_request) Successful in 6m28s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 6m40s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 7m5s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 7m13s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 8m24s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 10m38s
CI/CD Pipeline / Validate - Security (pull_request) Has been cancelled
CI/CD Pipeline / Build Production API Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Web Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Production (pull_request) Has been cancelled
CI/CD Pipeline / Production Browser E2E (pull_request) Has been cancelled
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
CI/CD Pipeline / CI Gate (pull_request) Has been cancelled
2026-09-12 21:39:00 +08:00
xiaoxia a83b53ae58 fix(ai-avatar): B-roll时间戳、标题粗体重影、TTS卡死防护
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 2s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 1m23s
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 1m19s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 2m18s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m38s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 2m40s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m11s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Successful in 3m45s
AI Code Review / AI Code Review (pull_request) Successful in 6m59s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 6m58s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 7m11s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 7m14s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 9m16s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 10m40s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 25m22s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Successful in 3s
1. B-roll时间戳全0修复:
   - sentences.ts 新增 sentenceTimings 参数优先使用后端精确时间戳,
     降级才按字数比例估算
   - 修正参数顺序,与 ModalBRollEditor 现有调用 (scriptText, timings, duration) 对齐
   - 前端 AiAvatarPage 已通过 props 传 sentenceTimings(旧版已传但因顺序错位被忽略)

2. 标题粗体重影修复:
   - 之前 bold=true 时使用 borderw=3 + font_color 同色描边模拟粗体,
     会在小字号/竖屏视频上造成字形边缘偏移,视觉上文字像被打印了两次
     (用户截图中的'曝光曝光…'重影)
   - 改为黑色细描边(borderw=2, 黑色),既保留清晰加粗效果又不重影
   - 用户显式开启 stroke 时仍按用户配置走
   - 新增 _resolve_font_path(bold=True) 预留粗体字体查找能力(当前镜像
     无独立 Bold 字体文件,沿用 VF 常规字重)

3. TTS 任务防卡死:
   - 给 tts_synthesize_and_submit 加 soft_time_limit=180s / time_limit=200s,
     避免因网络/上游问题导致 Celery 任务永久挂起(用户之前卡10+分钟
     tts_processing 不失败)
   - 任务开头加 INFO 日志(job_id/voice_id/text_len),方便排查 worker
     是否真的收到任务

相关:staging 用户反馈 5 问题中的 1/2/3 项(B-roll时间、标题重影、对口型卡死)
2026-09-12 21:07:34 +08:00
frontend-dev e250132ace perf(ai-avatar): 对口型加速+封面流程重构 (#1872)
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 3s
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / Check push changed paths (push) Successful in 18s
CI/CD Pipeline / Build Staging API Image (push) Successful in 15s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 18s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 42s
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been skipped
CI/CD Pipeline / Integration Tests (push) Successful in 1m51s
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Successful in 1m56s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 1m1s
CI/CD Pipeline / Validate - Style (push) Successful in 2m13s
CI/CD Pipeline / ACR Image Cleanup (push) Successful in 1m39s
CI/CD Pipeline / Validate - Security (push) Successful in 4m42s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 2m56s
CI/CD Pipeline / Frontend Unit Tests (push) Successful in 5m6s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 4m5s
CI/CD Pipeline / Unit Tests (push) Failing after 7m43s
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / CI Gate (push) Has been skipped
CI/CD Pipeline / Canary Release to Production (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
Co-authored-by: frontend-dev <frontend-dev@xiaoxiajianji.com>
Co-committed-by: frontend-dev <frontend-dev@xiaoxiajianji.com>
2026-09-12 19:10:03 +08:00
frontend-dev 774dd27844 fix(ai-avatar): 修复B-roll文案时长全为0.0s的问题 (#1871)
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 1s
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / Check push changed paths (push) Successful in 4s
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Build Staging Web Image (push) Successful in 23s
CI/CD Pipeline / Build Staging API Image (push) Successful in 24s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 24s
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 59s
CI/CD Pipeline / ACR Image Cleanup (push) Successful in 1m35s
CI/CD Pipeline / Integration Tests (push) Successful in 3m26s
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Successful in 3m28s
CI/CD Pipeline / Validate - Style (push) Successful in 3m51s
CI/CD Pipeline / Frontend Unit Tests (push) Successful in 4m42s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 3m28s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 3m49s
CI/CD Pipeline / Validate - Security (push) Successful in 8m11s
CI/CD Pipeline / Unit Tests (push) Successful in 8m48s
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / CI Gate (push) Has been skipped
CI/CD Pipeline / Canary Release to Production (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
Co-authored-by: frontend-dev <frontend-dev@xiaoxiajianji.com>
Co-committed-by: frontend-dev <frontend-dev@xiaoxiajianji.com>
2026-09-12 15:00:56 +08:00
frontend-dev ed7af0642d fix(ai-avatar): B-roll文案显示时长+标题字号调大 (#1870)
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 1s
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Check push changed paths (push) Successful in 2s
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Build Staging API Image (push) Successful in 24s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 25s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 27s
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 53s
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Successful in 2m21s
CI/CD Pipeline / Integration Tests (push) Successful in 2m19s
CI/CD Pipeline / Validate - Style (push) Successful in 3m0s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 2m14s
CI/CD Pipeline / ACR Image Cleanup (push) Successful in 2m48s
CI/CD Pipeline / Validate - Security (push) Successful in 4m33s
CI/CD Pipeline / Frontend Unit Tests (push) Successful in 5m17s
CI/CD Pipeline / Unit Tests (push) Successful in 8m8s
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / CI Gate (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 23m3s
CI/CD Pipeline / Canary Release to Production (push) Has been skipped
Co-authored-by: frontend-dev <frontend-dev@xiaoxiajianji.com>
Co-committed-by: frontend-dev <frontend-dev@xiaoxiajianji.com>
2026-09-12 10:02:34 +08:00
frontend-dev 938ef0b8cc fix(ai-avatar): 端到端一致性修复(标题/封面/B-roll位置/B-roll时长) (#1869)
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 2s
CI/CD Pipeline / Check push changed paths (push) Successful in 4s
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / Build Staging API Image (push) Successful in 21s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 24s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 51s
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been skipped
CI/CD Pipeline / Integration Tests (push) Successful in 3m26s
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Successful in 3m37s
CI/CD Pipeline / Validate - Style (push) Successful in 4m6s
CI/CD Pipeline / Frontend Unit Tests (push) Successful in 4m29s
CI/CD Pipeline / Validate - Security (push) Successful in 8m25s
CI/CD Pipeline / Unit Tests (push) Successful in 8m59s
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / CI Gate (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 18m30s
CI/CD Pipeline / ACR Image Cleanup (push) Successful in 1m43s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 3m7s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 3m42s
CI/CD Pipeline / Canary Release to Production (push) Has been skipped
Co-authored-by: frontend-dev <frontend-dev@xiaoxiajianji.com>
Co-committed-by: frontend-dev <frontend-dev@xiaoxiajianji.com>
2026-09-12 02:41:11 +08:00
frontend-dev 982daac6e5 Merge pull request 'fix(ai-avatar): 全盘修复 FFmpeg 渲染滤镜链路(exit 234 P0)' (#1868) from fix/ffmpeg-filter-comprehensive-fix into develop
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 1s
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / Check push changed paths (push) Successful in 6s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m29s
CI/CD Pipeline / Frontend Unit Tests (push) Successful in 4m2s
CI/CD Pipeline / Integration Tests (push) Successful in 7m24s
CI/CD Pipeline / Validate - Style (push) Successful in 9m16s
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Successful in 9m56s
CI/CD Pipeline / Unit Tests (push) Successful in 12m12s
CI/CD Pipeline / Validate - Security (push) Successful in 23m10s
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / CI Gate (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Build Staging API Image (push) Successful in 39m39s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 41m2s
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 3m10s
CI/CD Pipeline / ACR Image Cleanup (push) Successful in 1m43s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 2m50s
CI/CD Pipeline / Canary Release to Production (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Failing after 3m27s
2026-09-11 23:40:21 +08:00
CI Bot a7067c8171 style: auto-format with black + isort + ruff + prettier [skip ci-format-check]
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 1s
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 2s
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 32s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 33s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m48s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m42s
AI Code Review / AI Code Review (pull_request) Successful in 6m37s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 7m55s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 8m12s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 8m22s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 10m50s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 11m0s
ACR Cleanup / ACR Image Cleanup (pull_request_target) Successful in 5s
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 2m55s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 28m58s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Successful in 1s
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
2026-09-11 15:20:35 +00:00
LingYing Agent 32c3d2f263 fix(ai-avatar): 全盘修复 FFmpeg 渲染滤镜链路(exit 234 P0)
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 1s
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 1s
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 27s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 1m43s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m55s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m44s
AI Code Review / AI Code Review (pull_request) Successful in 6m34s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 8m17s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 8m45s
CI/CD Pipeline / Validate - Style (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Security (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Has been cancelled
CI/CD Pipeline / Build Production API Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Web Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Production (pull_request) Has been cancelled
CI/CD Pipeline / Production Browser E2E (pull_request) Has been cancelled
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
CI/CD Pipeline / CI Gate (pull_request) Has been cancelled
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been cancelled
根因分析(退出码 234 = Invalid argument):
1) drawtext 使用了无效参数 font=bold,导致整个 filter_complex 解析失败
   —— FFmpeg drawtext 没有 bold 参数;改为通过 borderw 模拟粗体视觉效果
2) DRAWTEXT_FONT_SEARCH_PATHS 未包含 Dockerfile 中 COPY 的
   NotoSansSC-VF.ttf 路径,且把不支持中文的 DejaVuSans 放在 fallback 首位,
   导致字体 fallback 到拉丁字体,中文渲染乱码/方框
3) build_broll_overlay_filter 硬编码 1280:720 横屏尺寸,AI 数字人是 9:16 竖屏
4) B-roll PIP 输入索引错误(用 len(sorted_segments) 而非原始下标映射),
   PIP 标签链断裂([pip0]→[vout0] 而非 [vout])
5) 渲染命令缺 -map 0:a?,合成后音频丢失
6) 标题滤镜与 B-roll 输出标签拼接用分号有缺陷,末尾分号处理偶发问题

修复内容(packages/domain/video_filter_builder.py):
- DRAWTEXT_FONT_SEARCH_PATHS: VF 字体置顶、移除 DejaVuSans、加 Bold.ttc 路径
- DRAWTEXT_FONT_MAP: 思源黑体等关键字改为 NotoSansSC(匹配 VF 文件名)
- 删除 font=bold 无效参数;bold 无显式描边时自动用 borderw=3+同色描边模拟粗体
- build_broll_overlay_filter 返回 (filter_str, final_label) 元组,解决标签问题
- 重写 fullscreen/PIP 拆分:原始列表下标决定 -i 输入序号,sorted 只用于时序处理
- PIP overlay 正确基于 fullscreen 输出([vout_fs])或主视频([0:v])链接
- output_width/output_height 贯穿所有 scale/pad/overlay,默认 1280x720 兼容旧调用

修复内容(apps/api/app/services/ai_avatar_render_service.py):
- 新增 _probe_video_resolution() 用 ffprobe 探测输入视频实际分辨率
- AI 数字人默认竖屏 720x1280,探测失败兜底不阻断渲染
- build_broll_overlay_filter 调用传实际 output_width/output_height
- 标题滤镜传实际尺寸,保证位置/坐标计算正确
- filter_complex 拼接重写:broll+title/broll-only/title-only/无滤镜四分支清晰
- _build_ffmpeg_command 补 -map 0:a? + -c:a aac,音频不再丢失
- 清理冗余内联 import

测试:
- 更新 test_ai_avatar_render_routes.py / test_video_filter_builder.py 适配新元组签名
- 新增 test_bold_true_does_not_use_font_bold_param 防回归
- 本地 ffmpeg 实测中文标题+B-roll+竖屏720x1280合成成功
- 相关 3524 个单测全通过
2026-09-11 23:09:22 +08:00
xiaoxia 7198cfe980 Merge pull request 'fix(ai-avatar, P0): 数字人成片入库兜底project_id + 封面抽帧加速 + cv2依赖' (#1864) from fix/ai-avatar-generated-video-and-cover-speed into develop
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 1s
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / Check push changed paths (push) Successful in 7s
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Build Staging Web Image (push) Successful in 20s
CI/CD Pipeline / Build Staging API Image (push) Successful in 24s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 23s
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 1m43s
CI/CD Pipeline / ACR Image Cleanup (push) Successful in 1m32s
CI/CD Pipeline / Frontend Unit Tests (push) Successful in 3m52s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1m54s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 3m2s
CI/CD Pipeline / Integration Tests (push) Successful in 8m35s
CI/CD Pipeline / Validate - Style (push) Successful in 9m15s
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Successful in 11m32s
CI/CD Pipeline / Unit Tests (push) Successful in 14m10s
CI/CD Pipeline / Validate - Security (push) Successful in 31m22s
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / CI Gate (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Canary Release to Production (push) Has been skipped
API Base Image Build / Build API Base Image (push) Successful in 58m15s
Worker Base Image Build / Build Worker Base Image (push) Successful in 1h11m23s
P0: AI数字人视频不入成品库+封面抽帧加速 (#1864)

- GeneratedVideo.create() 允许 project_id/generation_task_id 为空字符串,支持 AI数字人无项目场景
- ai_avatar_render_service: project_id空兜底ai_avatar, generation_task_id空兜底job.id; except升error+exc_info
- 封面抽帧轮询 poll_interval=1s × max_attempts=15 → 最长15s(原60s)
- requirements.txt 追加 numpy==1.26.4 + opencv-python-headless==4.10.0.84,恢复cv2帧评分能力
- 单测适配:允许空project_id、poll参数更新、cv2评分阈值+patch方式调整
2026-09-11 20:06:57 +08:00
xiaoxia b0cfa98e20 Merge pull request 'fix(ai-avatar): 封面title_config字段名与后端契约对齐' (#1865) from fix/ai-avatar-cover-title-contract into develop
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 1s
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Check push changed paths (push) Successful in 14s
CI/CD Pipeline / Build Staging API Image (push) Successful in 14s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 16s
CI/CD Pipeline / Integration Tests (push) Successful in 1m12s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m30s
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been skipped
CI/CD Pipeline / Validate - Style (push) Successful in 1m56s
CI/CD Pipeline / Frontend Unit Tests (push) Successful in 3m47s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 2m4s
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Successful in 4m30s
CI/CD Pipeline / ACR Image Cleanup (push) Successful in 1m35s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 2m56s
CI/CD Pipeline / Unit Tests (push) Successful in 7m18s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 3m34s
CI/CD Pipeline / Validate - Security (push) Has been cancelled
CI/CD Pipeline / Build Production API Image (push) Has been cancelled
CI/CD Pipeline / Build Production Web Image (push) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (push) Has been cancelled
CI/CD Pipeline / Deploy Production (push) Has been cancelled
CI/CD Pipeline / Production Browser E2E (push) Has been cancelled
CI/CD Pipeline / Canary Release to Production (push) Has been cancelled
CI/CD Pipeline / CI Gate (push) Has been cancelled
2026-09-11 19:58:52 +08:00
xiaoying-agent e905989695 test: 适配 cv2 可用后的单测阈值与 mock 方式
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 1s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 1s
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 14s
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 14s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 1m50s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m38s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m40s
AI Code Review / AI Code Review (pull_request) Successful in 6m25s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 6m26s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 8m39s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 11m33s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 11m0s
ACR Cleanup / ACR Image Cleanup (pull_request_target) Successful in 8s
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 1m16s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 25m55s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Successful in 4s
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
- test_cover_frame_scorer: 渐变图高分阈值 50→40(实测41.27,纯渐变拉普拉斯方差中等);全黑阈值 5→10(兼容cv2浮点/直方图微小差异)
- test_dedup_v2: cv2.VideoCapture 改为 patch.object 方式 mock,确保在真实 cv2 可用环境下mock生效(之前直接赋值 cv2_mock.VideoCapture.return_value 在sys.modules恢复后可能引用丢失)
2026-09-11 19:42:21 +08:00
xiaoxia 3dcf1079a9 fix(ai-avatar): 封面title_config字段名与后端契约对齐
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 2s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
CI/CD Pipeline / Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Has been skipped
CI/CD Pipeline / PR Build Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Successful in 1m17s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 1m43s
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 1m48s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 1m50s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m20s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 2m19s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 2m39s
AI Code Review / AI Code Review (pull_request) Successful in 6m45s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 10m33s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 17m33s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Successful in 3s
ACR Cleanup / ACR Image Cleanup (pull_request_target) Successful in 7s
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 1m13s
封面接口传入的titleConfig用buildTitleConfigPayload转换:
- 前端字段 title -> 后端 text
- 前端字段 color -> 后端 font_color (#开头保留)
- 前端字段 size -> 后端 font_size
- 前端字段 position 默认bottom

之前直接传原始AiAvatarTitleConfig对象,后端只认text/font_color/font_size,
导致封面标题内容、颜色、字号、位置全部取默认值(白色居中36px),与对口型预览不一致。
渲染提交路径已经用了buildTitleConfigPayload,是封面路径漏了转换。
2026-09-11 19:38:28 +08:00
xiaoxia da22c2e834 Merge pull request 'fix(ai-avatar): 素材预览不叠标题+封面传title_config+渲染后封面避免双标题' (#1863) from fix/ai-avatar-title-cover-source into develop
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 1s
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / Check push changed paths (push) Successful in 5s
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Build Staging API Image (push) Successful in 26s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 28s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m6s
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been skipped
CI/CD Pipeline / Integration Tests (push) Successful in 1m41s
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Successful in 1m58s
CI/CD Pipeline / Validate - Style (push) Successful in 2m22s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 1m7s
CI/CD Pipeline / ACR Image Cleanup (push) Successful in 1m54s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1m56s
CI/CD Pipeline / Frontend Unit Tests (push) Successful in 5m21s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 3m5s
CI/CD Pipeline / Validate - Security (push) Successful in 7m37s
CI/CD Pipeline / Unit Tests (push) Successful in 8m17s
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / CI Gate (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Canary Release to Production (push) Has been skipped
2026-09-11 19:05:40 +08:00
xiaoxia c49c855533 Merge branch 'fix/ai-avatar-title-cover-source' into develop 2026-09-11 19:05:25 +08:00
xiaoying-agent baed0c6431 fix(ai-avatar, P0): 数字人成片入库兜底project_id + 封面抽帧加速 + cv2依赖
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 1s
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 22s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 56s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 55s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m45s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 2m50s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 3m12s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 3m26s
AI Code Review / AI Code Review (pull_request) Successful in 6m25s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 8m4s
CI/CD Pipeline / Unit Tests (pull_request) Failing after 9m55s
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Failing after 1s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 7m15s
P0 Bug:AI数字人视频渲染成功但成片库看不到
- packages/domain/generated_video.py: GeneratedVideo.create 去掉 project_id/generation_task_id 非空必填校验(AI数字人无项目概念,project_id为空;lipsync路径下task_id也可能为空),name/file_url仍强制非空
- apps/api/app/services/ai_avatar_render_service.py: 第7步保存成片时,project_id为空兜底为'ai_avatar',generation_task_id为空兜底为job.id;except块日志从logger.warning改为logger.error+exc_info=True,避免异常被吞

优化:封面抽帧加速(解决60s超时)
- apps/api/app/services/ai_avatar_cover_service.py: COVER_POLL_INTERVAL 3s→1s,COVER_MAX_POLL_ATTEMPTS 20→15(最长15s,之前60s);max_frames默认已是5
- requirements.txt: 加 numpy==1.26.4 + opencv-python-headless==4.10.0.84(cover_frame_scorer 用cv2做清晰度/亮度/色彩评分,API镜像之前缺cv2只能回退默认分50.0)

测试适配:
- 4处GeneratedVideo单测:从期望ValueError改为允许空串/空白归一化
- test_ai_avatar_emotion_tts_lipsync.py: poll_interval断言1.0,max_poll_attempts断言15
2026-09-11 18:45:21 +08:00
xiaoxia 3c817a2ffe fix(ai-avatar): 素材预览不叠加标题 + 封面接口传title_config + 渲染后封面避免双标题
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 4s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 33s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 33s
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 1m14s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m32s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m56s
AI Code Review / AI Code Review (pull_request) Successful in 6m37s
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Failing after 10m16s
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m18s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Successful in 1m20s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 1m22s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 1m42s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 1m49s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 10m36s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 6m39s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Successful in 1s
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
ACR Cleanup / ACR Image Cleanup (pull_request_target) Successful in 55s
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 1m11s
1. PanelVideoSelector: 移除素材视频上的标题叠加预览,标题只在对口型预览和最终成片上展示
2. generateSmartCover API: 传入title_config参数,让后端在对口型视频帧上用drawtext叠加标题
3. AiAvatarPage: 调用generateSmartCover时传入state.titleConfig;移除传给PanelVideoSelector的titleConfig prop
4. ai_avatar_render_service: 渲染完成后自动抽封面时不传title_config,避免最终视频已有标题再叠加导致双重标题
2026-09-11 18:29:23 +08:00
xiaoxia 96bf62b00c fix(P0): 成品库分页加载更多 + AI数字人渲染传 project_id 入库 (#1862)
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 0s
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / Check push changed paths (push) Successful in 4s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 22s
CI/CD Pipeline / Validate - Style (push) Successful in 1m26s
CI/CD Pipeline / Build Staging API Image (push) Successful in 1m23s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 1m46s
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been skipped
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Successful in 2m24s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 59s
CI/CD Pipeline / Integration Tests (push) Successful in 3m31s
CI/CD Pipeline / ACR Image Cleanup (push) Successful in 1m43s
CI/CD Pipeline / Frontend Unit Tests (push) Successful in 5m33s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 3m36s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 4m9s
CI/CD Pipeline / Validate - Security (push) Successful in 7m14s
CI/CD Pipeline / Unit Tests (push) Successful in 8m47s
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / CI Gate (push) Has been skipped
CI/CD Pipeline / Canary Release to Production (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
Co-authored-by: xiaoxia <dev@xiaoxiajianji.com>
Co-committed-by: xiaoxia <dev@xiaoxiajianji.com>
2026-09-11 18:23:32 +08:00
xiaoxia 0b16e08d09 chore(ai-avatar): generateSmartCover 接口超时 60s → 120s (#1861)
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 2s
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Check push changed paths (push) Successful in 15s
CI/CD Pipeline / Build Staging API Image (push) Successful in 17s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 16s
CI/CD Pipeline / Integration Tests (push) Successful in 1m16s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m27s
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been skipped
CI/CD Pipeline / Validate - Style (push) Successful in 1m52s
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Successful in 2m2s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 1m7s
CI/CD Pipeline / Frontend Unit Tests (push) Successful in 3m36s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1m24s
CI/CD Pipeline / ACR Image Cleanup (push) Successful in 1m27s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 3m9s
CI/CD Pipeline / Unit Tests (push) Successful in 6m34s
CI/CD Pipeline / Validate - Security (push) Successful in 10m41s
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / CI Gate (push) Has been skipped
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Canary Release to Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
Co-authored-by: xiaoxia <dev@xiaoxiajianji.com>
Co-committed-by: xiaoxia <dev@xiaoxiajianji.com>
2026-09-11 15:58:19 +08:00
xiaoxia 33510b8dbf Merge pull request 'fix(ai-avatar, P0): FFmpeg subprocess list修复32512 + Celery异常raise + 投递失败写DB + 封面drawtext标题 + audio/x-wav白名单' (#1859) from fix/ai-avatar-ffmpeg-cover-wav into develop
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 1s
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Check push changed paths (push) Successful in 19s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 20s
CI/CD Pipeline / Build Staging API Image (push) Successful in 37s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m4s
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been skipped
CI/CD Pipeline / Integration Tests (push) Successful in 1m58s
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Successful in 2m14s
CI/CD Pipeline / Validate - Style (push) Successful in 2m37s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 1m12s
CI/CD Pipeline / ACR Image Cleanup (push) Successful in 1m35s
CI/CD Pipeline / Validate - Security (push) Has been cancelled
CI/CD Pipeline / Unit Tests (push) Has been cancelled
CI/CD Pipeline / Frontend Unit Tests (push) Has been cancelled
CI/CD Pipeline / Staging E2E Tests (push) Has been cancelled
CI/CD Pipeline / Staging API Integration Tests (push) Has been cancelled
CI/CD Pipeline / Build Production API Image (push) Has been cancelled
CI/CD Pipeline / Build Production Web Image (push) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (push) Has been cancelled
CI/CD Pipeline / Deploy Production (push) Has been cancelled
CI/CD Pipeline / Production Browser E2E (push) Has been cancelled
CI/CD Pipeline / Canary Release to Production (push) Has been cancelled
CI/CD Pipeline / CI Gate (push) Has been cancelled
2026-09-11 15:52:55 +08:00
xiaoying-agent ec2fb1c241 test(ai-avatar): 适配 subprocess list 形式的单测 mock 与 preset 断言
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 2s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 28s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 28s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m4s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m1s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 1m42s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 1m57s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m43s
CI/CD Pipeline / Unit Tests (pull_request) Failing after 6m3s
AI Code Review / AI Code Review (pull_request) Successful in 6m20s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 9m58s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Failing after 2s
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 7m29s
ACR Cleanup / ACR Image Cleanup (pull_request_target) Successful in 7s
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 32s
- execute_render 两个用例: os.system mock → subprocess.run mock,返回 CompletedProcess(returncode=0)
- preset 三个用例: 断言从 '-preset veryfast' 字符串包含改为 list 元素相邻校验,兼容 list[str] 形式
- 42/42 相关测试通过
2026-09-11 15:12:55 +08:00
xiaoxia 3cd8910f73 feat(ai-avatar): 封面预览实时显示标题叠加效果 + getRenderJob 60s 超时 (#1860)
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 2s
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Check push changed paths (push) Successful in 7s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 20s
CI/CD Pipeline / Build Staging API Image (push) Successful in 57s
CI/CD Pipeline / Integration Tests (push) Successful in 1m23s
CI/CD Pipeline / Validate - Style (push) Successful in 2m11s
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Successful in 2m14s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 3m13s
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 1m2s
CI/CD Pipeline / Frontend Unit Tests (push) Successful in 5m10s
CI/CD Pipeline / ACR Image Cleanup (push) Successful in 1m38s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 3m0s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 3m3s
CI/CD Pipeline / Unit Tests (push) Successful in 8m1s
CI/CD Pipeline / Validate - Security (push) Successful in 9m20s
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / CI Gate (push) Has been skipped
CI/CD Pipeline / Canary Release to Production (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
Co-authored-by: xiaoxia <dev@xiaoxiajianji.com>
Co-committed-by: xiaoxia <dev@xiaoxiajianji.com>
2026-09-11 15:00:22 +08:00
CI Bot 1b76821307 style: auto-format with black + isort + ruff + prettier [skip ci-format-check]
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 1s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 3s
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 29s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 30s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 1m6s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m14s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m35s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 1m41s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m52s
CI/CD Pipeline / Unit Tests (pull_request) Failing after 6m24s
AI Code Review / AI Code Review (pull_request) Successful in 6m29s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 7m39s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Failing after 4s
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 4m58s
2026-09-11 06:27:13 +00:00
xiaoxia-agent 2c76d55d2b fix(ai-avatar, P0): FFmpeg命令改为subprocess list形式修复32512 + Celery异常raise + 投递失败写DB + 封面drawtext标题 + TTS audio/x-wav白名单
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 1s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 29s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 28s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 1m19s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m29s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m47s
CI/CD Pipeline / Validate - Style (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Security (pull_request) Has been cancelled
CI/CD Pipeline / Unit Tests (pull_request) Has been cancelled
CI/CD Pipeline / Build Production API Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Web Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Production (pull_request) Has been cancelled
CI/CD Pipeline / Production Browser E2E (pull_request) Has been cancelled
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
CI/CD Pipeline / CI Gate (pull_request) Has been cancelled
AI Code Review / AI Code Review (pull_request) Has been cancelled
PR Automation / Auto Approve on CI Green (pull_request) Has been cancelled
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been cancelled
P0:
- ai_avatar_render_service._build_ffmpeg_command返回list, subprocess.run(shell=False)
  修复OSS预签名URL含&被sh解释为命令分隔符导致exit 127(32512)
- cover_extract命令同步改为list+subprocess.run, 超时60s, 非致命失败
- execute_render两个except块末尾加raise, 确保Celery标记任务FAILURE而非SUCCESS
- ffmpeg stderr写入error_message末尾800字符, 前端可见具体错误
- routes两处delay()异常: job.status=failed + error_message='任务提交失败:...' + db.commit
- lipsync_tts.safe_download_bytes白名单补audio/x-wav(CosyVoice可能返回该MIME)

P1:
- ai_avatar_cover_service新增apply_title_to_cover: ffmpeg drawtext叠加标题
  竖屏封面720x1280, 失败回退无标题原始帧
- generate_smart_cover/persist_cover_to_oss新增title_config参数
- SmartCoverRequest schema新增可选title_config字段
- smart-cover接口透传body.title_config; execute_render管线透传job.title_config

仅修改5个后端文件, 不改其他功能.
2026-09-11 14:22:45 +08:00
45 changed files with 3510 additions and 766 deletions
@@ -0,0 +1,27 @@
"""add sentence_timings to lipsync_jobs
Revision ID: 075_add_sentence_timings
Revises: 074_ai_avatar_render_script_id_optional
Create Date: 2026-09-12
"""
import sqlalchemy as sa
from alembic import op
revision = "075_add_sentence_timings"
down_revision = "074_render_script_id_optional"
branch_labels = None
depends_on = None
def upgrade() -> None:
with op.batch_alter_table("lipsync_jobs") as batch:
batch.add_column(
sa.Column("sentence_timings", sa.JSON(), nullable=True),
)
def downgrade() -> None:
with op.batch_alter_table("lipsync_jobs") as batch:
batch.drop_column("sentence_timings")
+66 -25
View File
@@ -11,13 +11,13 @@
from __future__ import annotations from __future__ import annotations
import logging import logging
from datetime import datetime, timezone
from app.auth import AuthenticatedUser, get_current_user from app.auth import AuthenticatedUser, get_current_user
from app.dependencies import get_db_session from app.dependencies import get_db_session
from app.schemas.ai_avatar_render import ( from app.schemas.ai_avatar_render import (
AiAvatarRenderJobResponse, AiAvatarRenderJobResponse,
CreateAiAvatarRenderRequest, CreateAiAvatarRenderRequest,
SmartCoverRequest,
SmartCoverResponse, SmartCoverResponse,
) )
from app.services.ai_avatar_cover_service import generate_smart_cover from app.services.ai_avatar_cover_service import generate_smart_cover
@@ -77,10 +77,16 @@ def create_render_job(
from app.tasks.ai_avatar_render import execute_ai_avatar_render from app.tasks.ai_avatar_render import execute_ai_avatar_render
execute_ai_avatar_render.delay(job.id) execute_ai_avatar_render.delay(job.id)
except Exception: except Exception as exc:
logger.warning("Celery 任务提交失败,渲染任务已创建但未触发执行: %s", job.id) logger.exception("Celery 任务投递失败(创建): job_id=%s err=%s", job.id, exc)
job.status = "failed"
job.error_message = f"任务提交失败:{exc}"
job.updated_at = datetime.now(timezone.utc)
svc.db.commit()
svc.db.refresh(job)
return AiAvatarRenderJobResponse.model_validate(job)
return job return AiAvatarRenderJobResponse.model_validate(job)
# ── GET /jobs — 任务列表 ───────────────────────────────────────────────── # ── GET /jobs — 任务列表 ─────────────────────────────────────────────────
@@ -172,38 +178,54 @@ def retry_render_job(
from app.tasks.ai_avatar_render import execute_ai_avatar_render from app.tasks.ai_avatar_render import execute_ai_avatar_render
execute_ai_avatar_render.delay(job.id) execute_ai_avatar_render.delay(job.id)
except Exception: except Exception as exc:
logger.warning("Celery 任务提交失败重试任务已重置但未触发执行: %s", job.id) logger.exception("Celery 任务投递失败重试: job_id=%s err=%s", job.id, exc)
job.status = "failed"
job.error_message = f"任务提交失败:{exc}"
job.updated_at = datetime.now(timezone.utc)
svc.db.commit()
svc.db.refresh(job)
return AiAvatarRenderJobResponse.model_validate(job)
return job return AiAvatarRenderJobResponse.model_validate(job)
# ── POST /{job_id}/smart-cover — 从最终成片智能抽封面(步骤②)────────
# ── POST /smart-cover — 智能获取封面(MediaKit 抽帧 + 评分选帧)────────
@router.post("/smart-cover", response_model=SmartCoverResponse) @router.post("/{job_id}/smart-cover", response_model=SmartCoverResponse)
def generate_avatar_smart_cover( def generate_render_smart_cover(
body: SmartCoverRequest, job_id: str,
current_user: AuthenticatedUser = Depends(get_current_user), current_user: AuthenticatedUser = Depends(get_current_user),
) -> SmartCoverResponse: db: Session = Depends(get_db_session),
"""智能获取数字人视频封面. ):
"""从最终渲染成片智能抽帧生成封面(MediaKit 抽帧 + 评分选最佳帧 + 转存 OSS).
复用智能剪辑的 MediaKit 抽帧 + 质量评分选最佳帧逻辑(非 FFmpeg 简单截帧), - 必须等渲染任务 completed 后才可调用(否则返回 400)
并将选中帧转存到自家 OSS,返回非临时的封面公网 URL。 - 生成成功后自动更新 render_job 的 cover_config 与 output_cover_url
前端「智能获取封面」按钮可直接调用本接口;不依赖渲染任务完成。
""" """
video_url = (body.video_url or "").strip() from app.services.ai_avatar_render_service import AiAvatarRenderService
if not video_url.startswith(("http://", "https://")):
raise HTTPException(status_code=400, detail="video_url 必须是合法的 HTTP/HTTPS URL") svc = AiAvatarRenderService(db)
job = svc.get_render_job(job_id, current_user.user.id)
if job is None:
raise HTTPException(status_code=404, detail="渲染任务不存在")
if job.status != "completed":
raise HTTPException(status_code=400, detail="请先完成视频生成")
video_url = (job.output_video_url or "").strip()
if not video_url:
raise HTTPException(status_code=400, detail="渲染成片视频 URL 为空")
try: try:
cover_url = generate_smart_cover(video_url, max_frames=body.max_frames) # 从最终成片抽帧,帧本身已含标题/B-roll,直接转存 OSS
cover_url = generate_smart_cover(video_url, job_id=job_id, max_frames=5)
except Exception as exc: except Exception as exc:
logger.error( logger.error(
"智能封面生成异常: user=%s video_url=%s err=%s", "渲染成片智能封面生成异常: user=%s render_id=%s video_url=%s err=%s",
current_user.user.id, video_url[:80], exc, current_user.user.id,
job_id,
video_url[:80],
exc,
exc_info=True, exc_info=True,
) )
cover_url = "" cover_url = ""
@@ -214,5 +236,24 @@ def generate_avatar_smart_cover(
status="fallback_failed", status="fallback_failed",
message="智能抽帧失败(MediaKit 不可用或抽帧异常),请稍后重试", message="智能抽帧失败(MediaKit 不可用或抽帧异常),请稍后重试",
) )
logger.info("智能封面生成成功: user=%s cover_url=%s", current_user.user.id, cover_url[:120])
# 更新 render_job 的封面字段(异步写入 DB;失败不影响返回)
try:
job.cover_config = {
**(job.cover_config if isinstance(job.cover_config, dict) else {}),
"mode": "auto_frame",
"url": cover_url,
}
job.output_cover_url = cover_url
job.updated_at = datetime.now(timezone.utc)
db.commit()
except Exception as exc:
logger.warning("更新 render_job 封面字段失败(不影响返回): job_id=%s err=%s", job_id, exc)
logger.info(
"渲染成片智能封面生成成功: user=%s render_id=%s cover_url=%s",
current_user.user.id,
job_id,
cover_url[:120],
)
return SmartCoverResponse(cover_url=cover_url, status="completed") return SmartCoverResponse(cover_url=cover_url, status="completed")
+64 -15
View File
@@ -1,11 +1,12 @@
"""对口型 API 路由 — #1796 MediaKit 对口型, #1809 参数调整. """对口型 API 路由 — #1796 MediaKit 对口型, #1809 参数调整, #1845 配音前置.
接口: 接口:
POST /api/v1/lipsync/jobs 提交对口型任务 POST /api/v1/lipsync/jobs 提交对口型任务(支持 TTS/直传/预合成 三种模式)
GET /api/v1/lipsync/jobs 任务列表 GET /api/v1/lipsync/jobs 任务列表
GET /api/v1/lipsync/jobs/{id} 任务详情 GET /api/v1/lipsync/jobs/{id} 任务详情
POST /api/v1/lipsync/jobs/{id}/refresh 刷新任务状态 POST /api/v1/lipsync/jobs/{id}/refresh 刷新任务状态
POST /api/v1/lipsync/jobs/{id}/cancel 取消任务 POST /api/v1/lipsync/jobs/{id}/cancel 取消任务
POST /api/v1/lipsync/tts-preview #1845 步骤1 TTS 预合成(同步 HTTP~2-3s
""" """
from __future__ import annotations from __future__ import annotations
@@ -17,7 +18,12 @@ from app.dependencies import (
get_db_session, get_db_session,
get_voice_clone_profile_repository, get_voice_clone_profile_repository,
) )
from app.schemas.lipsync import CreateLipsyncJobRequest, LipsyncJobResponse from app.schemas.lipsync import (
AiAvatarTtsPreviewRequest,
AiAvatarTtsPreviewResponse,
CreateLipsyncJobRequest,
LipsyncJobResponse,
)
from app.services.lipsync_service import LipsyncService from app.services.lipsync_service import LipsyncService
from app.services.mediakit_client import MediaKitError from app.services.mediakit_client import MediaKitError
from fastapi import APIRouter, BackgroundTasks, Depends, HTTPException, Query from fastapi import APIRouter, BackgroundTasks, Depends, HTTPException, Query
@@ -33,7 +39,6 @@ def _get_service(
voice_clone_repo=Depends(get_voice_clone_profile_repository), voice_clone_repo=Depends(get_voice_clone_profile_repository),
) -> LipsyncService: ) -> LipsyncService:
# voice_clone_repo 用于克隆音色 profile 解析 # voice_clone_repo 用于克隆音色 profile 解析
# TTS 合成已移至 Celery 异步任务,无需同步注入 cosyvoice_service
return LipsyncService( return LipsyncService(
db, db,
voice_clone_repo=voice_clone_repo, voice_clone_repo=voice_clone_repo,
@@ -51,15 +56,20 @@ def create_lipsync_job(
): ):
"""提交对口型任务. """提交对口型任务.
#1809/#1822: 前端传 {video_url, voice_id, script_text, speed?, emotion?} 三种模式:
后端创建任务记录(状态 tts_processing),dispatch Celery 异步任务执行 TTS 合成 + MediaKit 提交; - TTS 直生(旧版/降级):传 {video_url, voice_id, script_text, speed?, emotion?}
也支持直接传 {video_url, audio_url}(同步提交 MediaKit 后端 dispatch Celery 异步任务
- 直接音频:传 {video_url, audio_url},后端同步下载+算timings+提交MediaKit。
- 预合成音频(#1845 新主路径):传 {video_url, audio_url, audio_duration, sentence_timings}
后端同步ffprobe+写入timings+直接提交MediaKit~2-3s)。
""" """
try: try:
job = svc.create_job( job = svc.create_job(
user_id=current_user.user.id, user_id=current_user.user.id,
video_url=body.video_url, video_url=body.video_url,
audio_url=body.audio_url, audio_url=body.audio_url,
audio_duration=body.audio_duration,
sentence_timings=body.sentence_timings,
voice_id=body.voice_id, voice_id=body.voice_id,
script_text=body.script_text, script_text=body.script_text,
speed=body.speed, speed=body.speed,
@@ -68,10 +78,8 @@ def create_lipsync_job(
project_id=body.project_id, project_id=body.project_id,
) )
except ValueError as exc: except ValueError as exc:
# 参数无效(如 voice_id 格式不对、文本过长等)
raise HTTPException(status_code=400, detail=str(exc)) from exc raise HTTPException(status_code=400, detail=str(exc)) from exc
except MediaKitError as exc: except MediaKitError as exc:
# 音色无权访问 → 403;参数无效 → 400MediaKit 提交失败 → 502
status_code = 502 status_code = 502
if exc.code in ("VoiceForbidden",): if exc.code in ("VoiceForbidden",):
status_code = 403 status_code = 403
@@ -86,7 +94,6 @@ def create_lipsync_job(
}, },
) from exc ) from exc
except Exception as exc: except Exception as exc:
# 兜底:任何未预期的错误返回 400 而非 500
logger.error("创建对口型任务异常: %s", exc, exc_info=True) logger.error("创建对口型任务异常: %s", exc, exc_info=True)
raise HTTPException( raise HTTPException(
status_code=400, status_code=400,
@@ -96,6 +103,52 @@ def create_lipsync_job(
return job return job
# ── POST /tts-preview — #1845 步骤1 TTS 预合成 ──────────────────────────
@router.post("/tts-preview", response_model=AiAvatarTtsPreviewResponse)
def preview_tts(
body: AiAvatarTtsPreviewRequest,
current_user: AuthenticatedUser = Depends(get_current_user),
svc: LipsyncService = Depends(_get_service),
):
"""步骤1「生成配音」同步 TTS 预合成.
同步执行 TTS 合成 → 下载音频 → ffprobe 时长 → 句子时间戳计算,
不创建 LipsyncJob、不转存 OSS,直接返回 CosyVoice 临时 URL~24h 有效)。
耗时约 2-3 秒。
"""
try:
result = svc.preview_tts(
user_id=current_user.user.id,
voice_id=body.voice_id,
script_text=body.script_text,
speed=body.speed,
emotion=body.emotion,
)
except MediaKitError as exc:
status_code = 400
if exc.code in ("VoiceForbidden",):
status_code = 403
elif exc.code in ("TTSNoAudio",):
status_code = 502
raise HTTPException(
status_code=status_code,
detail={
"code": exc.code,
"message": str(exc),
},
) from exc
except Exception as exc:
logger.error("TTS 预合成异常: %s", exc, exc_info=True)
raise HTTPException(
status_code=400,
detail=f"TTS 合成失败: {exc}",
) from exc
return result
# ── GET /jobs — 任务列表 ───────────────────────────────────────────────── # ── GET /jobs — 任务列表 ─────────────────────────────────────────────────
@@ -134,11 +187,7 @@ def get_lipsync_job(
current_user: AuthenticatedUser = Depends(get_current_user), current_user: AuthenticatedUser = Depends(get_current_user),
svc: LipsyncService = Depends(_get_service), svc: LipsyncService = Depends(_get_service),
): ):
"""获取对口型任务详情. """获取对口型任务详情."""
非终态任务:先返回 DB 缓存,挂后台刷新(下次轮询拿到新状态),
避免 MediaKit 慢响应阻塞前端轮询。
"""
job = svc.get_job(job_id, current_user.user.id) job = svc.get_job(job_id, current_user.user.id)
if job is None: if job is None:
raise HTTPException(status_code=404, detail="任务不存在") raise HTTPException(status_code=404, detail="任务不存在")
+4 -10
View File
@@ -52,7 +52,9 @@ class CreateAiAvatarRenderRequest(BaseModel):
lipsync_job_id: str = Field(..., description="对口型任务 ID") lipsync_job_id: str = Field(..., description="对口型任务 ID")
script_id: str = Field("", description="文案 ID(选自文案库时传;手动输入文案直生场景可留空)") script_id: str = Field("", description="文案 ID(选自文案库时传;手动输入文案直生场景可留空)")
b_roll_segments: list[BRollSegment] = Field(default_factory=list, description="B-roll 片段列表") b_roll_segments: list[BRollSegment] = Field(default_factory=list, description="B-roll 片段列表")
title_config: dict[str, Any] = Field(default_factory=dict, description="标题配置") title_config: dict[str, Any] = Field(
default_factory=dict, description="标题配置(可含 title_image_dataurl:前端 Canvas 渲染的标题 PNG dataURL"
)
cover_config: dict[str, Any] = Field(default_factory=dict, description="封面配置") cover_config: dict[str, Any] = Field(default_factory=dict, description="封面配置")
project_id: str = Field("", description="项目 ID") project_id: str = Field("", description="项目 ID")
@@ -67,7 +69,6 @@ class CreateAiAvatarRenderRequest(BaseModel):
@field_validator("script_id") @field_validator("script_id")
@classmethod @classmethod
def validate_script_id(cls, v: str) -> str: def validate_script_id(cls, v: str) -> str:
# script_id 可选:手动输入文案(TTS 直生)场景不关联文案库条目
return (v or "").strip() return (v or "").strip()
@@ -109,15 +110,8 @@ class AiAvatarRenderProgressResponse(BaseModel):
error_message: str error_message: str
class SmartCoverRequest(BaseModel):
"""智能封面请求 — MediaKit 抽帧 + 质量评分选最佳帧."""
video_url: str = Field(..., description="数字人视频 URL(对口型/渲染成片)")
max_frames: int = Field(5, ge=1, le=10, description="抽帧数量(默认 5")
class SmartCoverResponse(BaseModel): class SmartCoverResponse(BaseModel):
"""智能封面响应.""" """智能封面响应(封面从最终成片抽帧,不再叠加标题)."""
cover_url: str = Field("", description="封面图公网 URL(OSS,非临时);失败为空") cover_url: str = Field("", description="封面图公网 URL(OSS,非临时);失败为空")
status: str = Field("completed", description="completed / fallback_failed") status: str = Field("completed", description="completed / fallback_failed")
+39 -9
View File
@@ -1,9 +1,12 @@
"""对口型 API Schema 定义 — #1796 / #1809 / #1822. """对口型 API Schema 定义 — #1796 / #1809 / #1822 / #1845(配音前置).
支持种输入模式(二选一) 支持种输入模式:
1. TTS 直生模式(推荐):传 voice_id + script_text+ speed/emotion), 1. TTS 直生模式(兼容旧版前端):传 voice_id + script_text+ speed/emotion),
后端内部先调 CosyVoice 合成音频,再提交 MediaKit 对口型 后端 Celery 异步做 TTS 合成 + MediaKit 提交
2. 直接音频模式:传 video_url + audio_url(音频已由调用方准备好)。 2. 直接音频模式:传 video_url + audio_url(音频已由调用方准备好)。
3. 预合成音频模式(#1845 配音前置新主路径):前端先调 POST /lipsync/tts-preview
拿到 audio_url + sentence_timings,再在 create_job 时传 audio_url + audio_duration
+ sentence_timings,后端跳过 TTS 和时间戳计算,直接 ffprobe 校验后提交 MediaKit。
""" """
from __future__ import annotations from __future__ import annotations
@@ -33,6 +36,7 @@ class LipsyncJobResponse(BaseModel):
output_duration: float output_duration: float
error_message: str error_message: str
error_code: str error_code: str
sentence_timings: Optional[list] = None
submitted_at: Optional[datetime] = None submitted_at: Optional[datetime] = None
completed_at: Optional[datetime] = None completed_at: Optional[datetime] = None
created_at: datetime created_at: datetime
@@ -45,15 +49,19 @@ class LipsyncJobResponse(BaseModel):
class CreateLipsyncJobRequest(BaseModel): class CreateLipsyncJobRequest(BaseModel):
"""创建对口型任务请求. """创建对口型任务请求.
种模式(选一): 种模式(选一):
- TTS 直生:voice_id + script_text 必填+ 可选 speed/emotionaudio_url 留空。 - TTS 直生(旧版/降级)voice_id + script_text 必填;audio_url 留空。
- 直接音频:video_url + audio_url 必填。 - 直接音频:video_url + audio_url 必填。
- 预合成音频(#1845 新主路径):audio_url 必填 + 可选 audio_duration/sentence_timings
后端同步 ffprobe 校验时长、写入 timings,直接提交 MediaKit。
""" """
video_url: str = Field(..., description="人物视频 URL(MP4,≤30min,单人真人)") video_url: str = Field(..., description="人物视频 URL(MP4,≤30min,单人真人)")
# 模式 2:直接音频 # 模式 2/3:直接/预合成音频
audio_url: str = Field("", description="驱动音频 URLmp3/aac/wav/m4a/flac);直生模式留空") audio_url: str = Field("", description="驱动音频 URLmp3/aac/wav/m4a/flac);直生模式留空")
audio_duration: Optional[float] = Field(None, ge=0, description="预合成音频时长(秒),可选;后端会 ffprobe 校验")
sentence_timings: Optional[list] = Field(None, description="预合成接口返回的句子时间戳,可选;若传入则直接写入 job")
# 模式 1TTS 直生 # 模式 1TTS 直生
voice_id: str = Field("", description="音色 ID(预置音色或克隆音色 profile UUID") voice_id: str = Field("", description="音色 ID(预置音色或克隆音色 profile UUID")
@@ -61,7 +69,9 @@ class CreateLipsyncJobRequest(BaseModel):
speed: float = Field(1.0, ge=0.5, le=2.0, description="语速(0.5-2.0),默认 1.0") speed: float = Field(1.0, ge=0.5, le=2.0, description="语速(0.5-2.0),默认 1.0")
emotion: str = Field("", description="情绪(natural/excited/calm/friendly 或中文 自然/兴奋/沉稳/亲切)") emotion: str = Field("", description="情绪(natural/excited/calm/friendly 或中文 自然/兴奋/沉稳/亲切)")
enable_video_loop: bool = Field(False, description="音频长于视频时是否循环画面") enable_video_loop: bool = Field(
True, description="音频长于视频时是否循环画面(AI数字人默认开启,防止音频长于视频被截断)"
)
project_id: str = Field("", description="项目 ID(可选)") project_id: str = Field("", description="项目 ID(可选)")
@model_validator(mode="after") @model_validator(mode="after")
@@ -80,7 +90,7 @@ class CreateLipsyncJobRequest(BaseModel):
if not has_audio and not has_tts: if not has_audio and not has_tts:
raise ValueError( raise ValueError(
"必须提供驱动音频:要么传 audio_url(直接音频模式)," "必须提供驱动音频:要么传 audio_url(直接/预合成音频模式),"
"要么同时传 voice_id + script_textTTS 直生模式)" "要么同时传 voice_id + script_textTTS 直生模式)"
) )
@@ -98,3 +108,23 @@ class CreateLipsyncJobRequest(BaseModel):
self.audio_url = au self.audio_url = au
return self return self
# ── #1845 TTS 预合成接口 ────────────────────────────────────────────────
class AiAvatarTtsPreviewRequest(BaseModel):
"""步骤1「生成配音」预合成请求(同步 HTTP,~2-3s)."""
voice_id: str = Field(..., min_length=1, max_length=128, description="音色 ID")
script_text: str = Field(..., min_length=1, max_length=5000, description="要合成的文案")
speed: float = Field(1.0, ge=0.5, le=2.0, description="语速(0.5-2.0),默认 1.0")
emotion: str = Field("natural", max_length=32, description="情绪")
class AiAvatarTtsPreviewResponse(BaseModel):
"""TTS 预合成响应(临时 URL,24h 内有效,足够当前会话使用)."""
audio_url: str = Field(..., description="CosyVoice 临时音频 URL")
duration: float = Field(..., ge=0, description="音频总时长(秒),ffprobe 测得")
sentence_timings: list[dict] = Field(..., description="句子级精确时间戳")
@@ -1,10 +1,13 @@
"""AI 数字人封面服务 — 复用智能剪辑的 MediaKit 抽帧 + 质量评分选最佳帧. """AI 数字人封面服务 — MediaKit 抽帧 + 质量评分选最佳帧 + 转存 OSS.
与 generation_cover.py 的智能选帧能力对齐(不再用 FFmpeg 简单截帧): 与 generation_cover.py 的智能选帧能力对齐(不再用 FFmpeg 简单截帧):
1. MediaKit extract_frames 抽取多帧(默认 5 帧,SpecifiedFrames 策略) 1. MediaKit extract_frames 抽取多帧(默认 5 帧,SpecifiedFrames 策略)
2. cover_frame_scorer.score_frames 按清晰度/亮度/色彩评分选最佳 2. cover_frame_scorer.score_frames 按清晰度/亮度/色彩评分选最佳
3. 下载最佳帧并转存 OSS,返回公网封面 URL 3. 下载最佳帧并转存 OSS,返回公网封面 URL
设计原则:封面一律从最终成片(已叠加标题/B-roll)抽帧,帧本身已含标题,
本服务**不再叠加标题**。对口型阶段的裸视频封面入口已删除(废弃)。
降级:MediaKit 不可用或抽帧失败时返回空字符串,由调用方决定回退策略。 降级:MediaKit 不可用或抽帧失败时返回空字符串,由调用方决定回退策略。
""" """
@@ -19,9 +22,9 @@ from urllib.parse import urlparse
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
# MediaKit 抽帧轮询参数(与 MediaKit API timeout=60s 对齐) # MediaKit 抽帧轮询参数poll_interval=1s × max_poll=15 → 最长 15s,配合前端 120s 超时足够
COVER_POLL_INTERVAL = 3.0 COVER_POLL_INTERVAL = 1.0
COVER_MAX_POLL_ATTEMPTS = 20 # 最多等 60 秒 COVER_MAX_POLL_ATTEMPTS = 15
# 帧图片下载超时(秒) # 帧图片下载超时(秒)
FRAME_DOWNLOAD_TIMEOUT = 20 FRAME_DOWNLOAD_TIMEOUT = 20
@@ -49,7 +52,6 @@ def _sign_video_url_for_mediakit(video_url: str) -> str:
own_host = urlparse(public_base).netloc.lower() own_host = urlparse(public_base).netloc.lower()
url_host = urlparse(video_url).netloc.lower() url_host = urlparse(video_url).netloc.lower()
if own_host and url_host == own_host: if own_host and url_host == own_host:
# 是自家 OSS URL,重签 7 天有效期供 MediaKit 拉取
signed = storage.get_download_url(video_url, expires_seconds=MEDIAKIT_URL_TTL_SECONDS) signed = storage.get_download_url(video_url, expires_seconds=MEDIAKIT_URL_TTL_SECONDS)
if signed: if signed:
logger.info("[数字人封面] video_url 已重签(自家 OSS 私有桶)") logger.info("[数字人封面] video_url 已重签(自家 OSS 私有桶)")
@@ -60,19 +62,10 @@ def _sign_video_url_for_mediakit(video_url: str) -> str:
def select_best_cover_frame(video_url: str, *, max_frames: int = 5) -> str: def select_best_cover_frame(video_url: str, *, max_frames: int = 5) -> str:
"""从视频抽取多帧并评分选最佳帧,返回最佳帧的临时 URL. """从视频抽取多帧并评分选最佳帧,返回最佳帧的临时 URL."""
Args:
video_url: 可公网访问的视频 URL
max_frames: 抽帧数量
Returns:
最佳帧图片 URL;失败返回空字符串
"""
if not video_url: if not video_url:
return "" return ""
# 确保 MediaKit 能访问 video_url(自家 OSS 私有桶需重签)
video_url = _sign_video_url_for_mediakit(video_url) video_url = _sign_video_url_for_mediakit(video_url)
try: try:
@@ -85,11 +78,9 @@ def select_best_cover_frame(video_url: str, *, max_frames: int = 5) -> str:
return "" return ""
logger.info( logger.info(
"[数字人封面] 开始抽帧: video_url=%s max_frames=%d poll_interval=%.1f max_poll=%d", "[数字人封面] 开始抽帧: video_url=%s max_frames=%d",
video_url[:80], video_url[:80],
max_frames, max_frames,
COVER_POLL_INTERVAL,
COVER_MAX_POLL_ATTEMPTS,
) )
snapshots = mk.extract_frames( snapshots = mk.extract_frames(
@@ -107,7 +98,6 @@ def select_best_cover_frame(video_url: str, *, max_frames: int = 5) -> str:
if len(snapshots) == 1: if len(snapshots) == 1:
return snapshots[0].get("image_url") or snapshots[0].get("url") or "" return snapshots[0].get("image_url") or snapshots[0].get("url") or ""
# 使用连接池下载各帧(复用 TCP 连接,减少延迟)
import httpx import httpx
candidates = [] candidates = []
@@ -135,7 +125,6 @@ def select_best_cover_frame(video_url: str, *, max_frames: int = 5) -> str:
best = scored[0] if scored else None best = scored[0] if scored else None
best_url = best.get("url", "") if best else "" best_url = best.get("url", "") if best else ""
# 清理临时文件
for c in candidates: for c in candidates:
p = c.get("image_path") p = c.get("image_path")
if p: if p:
@@ -156,16 +145,15 @@ def select_best_cover_frame(video_url: str, *, max_frames: int = 5) -> str:
return "" return ""
def persist_cover_to_oss(frame_url: str, *, job_id: str = "", prefix: str = "ai-avatar/covers") -> str: def persist_cover_to_oss(
"""下载帧图并转存到 OSS,返回公网封面 URL. frame_url: str,
*,
job_id: str = "",
prefix: str = "ai-avatar/covers",
) -> str:
"""下载最佳帧图并转存到 OSS,返回公网封面 URL(预签名).
Args: 封面来自最终成片抽帧,帧本身已含标题,本函数不再做任何文字/图片叠加。
frame_url: MediaKit 返回的临时帧图 URL
job_id: 关联任务 ID(用于 OSS key 命名)
prefix: OSS key 前缀
Returns:
OSS 公网 URL;失败回退原始 frame_url
""" """
if not frame_url: if not frame_url:
return "" return ""
@@ -189,13 +177,13 @@ def persist_cover_to_oss(frame_url: str, *, job_id: str = "", prefix: str = "ai-
storage = get_shared_storage_service() storage = get_shared_storage_service()
token = job_id or uuid.uuid4().hex[:12] token = job_id or uuid.uuid4().hex[:12]
cover_key = f"{prefix}/{token}/cover_{uuid.uuid4().hex[:8]}.jpg" cover_key = f"{prefix}/{token}/cover_{uuid.uuid4().hex[:8]}.jpg"
public_url = storage.upload_file( public_url = storage.upload_file(
file_or_path=tmp_path, file_or_path=tmp_path,
storage_key=cover_key, storage_key=cover_key,
content_type="image/jpeg", content_type="image/jpeg",
) )
logger.info("[数字人封面] 封面已转存 OSS: key=%s", cover_key) logger.info("[数字人封面] 封面已转存 OSS: key=%s", cover_key)
# 私有桶:返回预签名 URL(前端才能加载)
if public_url: if public_url:
signed = storage.get_download_url(cover_key, expires_seconds=86400) signed = storage.get_download_url(cover_key, expires_seconds=86400)
return signed return signed
@@ -211,10 +199,15 @@ def persist_cover_to_oss(frame_url: str, *, job_id: str = "", prefix: str = "ai-
pass pass
def generate_smart_cover(video_url: str, *, job_id: str = "", max_frames: int = 5) -> str: def generate_smart_cover(
"""一站式:MediaKit 智能抽帧选最佳 → 转存 OSS,返回封面公网 URL. video_url: str,
*,
job_id: str = "",
max_frames: int = 5,
) -> str:
"""一站式:MediaKit 智能抽帧选最佳 → 转存 OSS。失败返回空字符串。
供独立封面接口与渲染管线复用。失败返回空字符串 封面从最终成片抽帧,不再叠加任何标题(帧本身已含)
""" """
best_frame = select_best_cover_frame(video_url, max_frames=max_frames) best_frame = select_best_cover_frame(video_url, max_frames=max_frames)
if not best_frame: if not best_frame:
+257 -74
View File
@@ -9,8 +9,11 @@
from __future__ import annotations from __future__ import annotations
import base64
import binascii
import logging import logging
import os import os
import subprocess
import tempfile import tempfile
import uuid import uuid
from datetime import datetime, timezone from datetime import datetime, timezone
@@ -24,8 +27,9 @@ from packages.adapters.sqlalchemy_impl.models import (
ScriptModel, ScriptModel,
) )
from packages.domain.video_filter_builder import ( from packages.domain.video_filter_builder import (
build_cover_extract_command, build_broll_overlay_filter,
build_title_drawtext_filter, build_title_drawtext_filter,
build_title_overlay_filter,
) )
from packages.shared.storage import get_shared_storage_service from packages.shared.storage import get_shared_storage_service
@@ -196,9 +200,8 @@ class AiAvatarRenderService:
1. 下载对口型输出视频 (20%) 1. 下载对口型输出视频 (20%)
2. 构建 FFmpeg 滤镜链 (40%) 2. 构建 FFmpeg 滤镜链 (40%)
3. 执行 FFmpeg 渲染 (80%) 3. 执行 FFmpeg 渲染 (80%)
4. 提取封面 (90%) 4. 上传到 OSS (95%) — 封面不再自动生成,改由前端主动抽帧
5. 上传到 OSS (95%) 5. 更新任务状态 (100%)
6. 更新任务状态 (100%)
""" """
job = self.db.query(AiAvatarRenderJob).filter(AiAvatarRenderJob.id == job_id).first() job = self.db.query(AiAvatarRenderJob).filter(AiAvatarRenderJob.id == job_id).first()
if job is None: if job is None:
@@ -228,27 +231,32 @@ class AiAvatarRenderService:
self.db.commit() self.db.commit()
# 2. 构建 FFmpeg 滤镜链 (40%) # 2. 构建 FFmpeg 滤镜链 (40%)
from packages.domain.video_filter_builder import build_broll_overlay_filter # 用 ffprobe 探测输入视频分辨率,确保 B-roll 缩放与标题位置与实际输出一致。
# AI 数字人对口型输出为 9:16 竖屏,默认兜底 720x1280;探测失败时使用默认值不阻断渲染。
output_width, output_height = self._probe_video_resolution(input_video_path)
if output_width <= 0 or output_height <= 0:
output_width, output_height = 720, 1280
logger.info(
"[数字人渲染] ffprobe 探测分辨率失败或无效,使用默认竖屏尺寸 %sx%s",
output_width,
output_height,
)
else:
logger.info("[数字人渲染] 探测输入视频分辨率: %sx%s", output_width, output_height)
filter_complex = build_broll_overlay_filter( broll_filter, broll_label = build_broll_overlay_filter(
b_roll_segments=job.b_roll_segments, b_roll_segments=job.b_roll_segments,
video_duration=lipsync_job.output_duration, video_duration=lipsync_job.output_duration,
output_width=output_width,
output_height=output_height,
) )
# 标题叠加 # 标题叠加路径:优先前端 Canvas 渲染的 PNG 图层(所见即所得),
title_filter = build_title_drawtext_filter(job.title_config) # 无 title_image_dataurl 时降级到 drawtext 重画文字。
if title_filter: title_cfg = job.title_config if isinstance(job.title_config, dict) else {}
if filter_complex: title_dataurl = (title_cfg or {}).get("title_image_dataurl") if title_cfg else None
filter_complex += f"[vout]{title_filter}[vout_titled];" use_title_png = isinstance(title_dataurl, str) and title_dataurl.startswith("data:image/")
else: title_input_index = 1 + len(job.b_roll_segments or []) if use_title_png else None
filter_complex = f"[0:v]{title_filter}[vout_titled];"
# 清理末尾分号
if filter_complex.endswith(";"):
filter_complex = filter_complex[:-1]
# 最终输出标签
final_label = "vout_titled" if title_filter else ("vout" if filter_complex else None)
job.progress = 40 job.progress = 40
self.db.commit() self.db.commit()
@@ -257,57 +265,126 @@ class AiAvatarRenderService:
with tempfile.TemporaryDirectory() as tmpdir: with tempfile.TemporaryDirectory() as tmpdir:
output_video_path = os.path.join(tmpdir, "output.mp4") output_video_path = os.path.join(tmpdir, "output.mp4")
cmd = self._build_ffmpeg_command( # 在临时目录里解码保存标题 PNG(with 退出自动清理)
title_png_path: Optional[str] = None
extra_inputs: list[str] = []
title_filter = None
if use_title_png:
try:
title_png_path = os.path.join(tmpdir, f"title_{job.id}.png")
self._save_title_dataurl_to_file(title_dataurl, dst_path=title_png_path)
extra_inputs.append(title_png_path)
logger.info(
"[数字人渲染] 标题 PNG 已保存: %s (input index %d)", title_png_path, title_input_index
)
except Exception as exc:
logger.warning("[数字人渲染] 标题 PNG 解码/保存失败,降级 drawtext: %s", exc)
title_png_path = None
extra_inputs = []
# 构建标题滤镜
final_label = None
if title_png_path and title_input_index is not None:
title_input_label = f"[{title_input_index}:v]"
base_label = f"[{broll_label}]" if broll_label else "[0:v]"
title_filter = build_title_overlay_filter(
title_cfg,
output_width=output_width,
output_height=output_height,
title_png_path=title_png_path,
title_input_label=title_input_label,
base_label=base_label,
output_label="vout_titled",
)
if not title_filter:
# build 返回 None → 文件不存在(极端并发情况),降级 drawtext
title_png_path = None
extra_inputs = []
if title_png_path:
# overlay 路径
if broll_filter and title_filter:
filter_complex = broll_filter + f";{title_filter}"
elif broll_filter:
filter_complex = broll_filter
final_label = broll_label
elif title_filter:
filter_complex = title_filter
else:
filter_complex = ""
if title_filter:
final_label = "vout_titled"
elif not final_label:
final_label = None
else:
# 降级:drawtext 重画文字
title_filter = build_title_drawtext_filter(
title_cfg,
output_width=output_width,
output_height=output_height,
)
if broll_filter and title_filter:
filter_complex = broll_filter + f";[{broll_label}]{title_filter}[vout_titled]"
final_label = "vout_titled"
elif broll_filter:
filter_complex = broll_filter
final_label = broll_label
elif title_filter:
filter_complex = f"[0:v]{title_filter}[vout_titled]"
final_label = "vout_titled"
else:
filter_complex = ""
final_label = None
cmd_list = self._build_ffmpeg_command(
input_video=input_video_path, input_video=input_video_path,
b_roll_segments=job.b_roll_segments, b_roll_segments=job.b_roll_segments,
extra_inputs=extra_inputs,
filter_complex=filter_complex, filter_complex=filter_complex,
final_label=final_label, final_label=final_label,
output_path=output_video_path, output_path=output_video_path,
) )
exit_code = os.system(cmd) try:
if exit_code != 0: render_result = subprocess.run(
raise AiAvatarRenderError(f"FFmpeg 渲染失败,退出码: {exit_code}", code="FFmpegFailed") cmd_list,
capture_output=True,
text=True,
timeout=600,
)
except subprocess.TimeoutExpired as exc:
raise AiAvatarRenderError(
"FFmpeg 渲染超时(600s",
code="FFmpegTimeout",
) from exc
if render_result.returncode != 0:
stderr_tail = (render_result.stderr or "").strip()[-800:]
raise AiAvatarRenderError(
f"FFmpeg 渲染失败,退出码: {render_result.returncode}, stderr: {stderr_tail}",
code="FFmpegFailed",
)
job.progress = 80 job.progress = 80
self.db.commit() self.db.commit()
# 4. 提取封面 (90%) # 4/5. 上传成片到 OSS (95%) —— 已砍掉自动抽封面逻辑(步骤⑤);
cover_path = "" # 封面由前端在渲染完成后通过 /smart-cover 接口主动从成片抽帧,不阻塞渲染链路。
if job.cover_config:
cover_path = os.path.join(tmpdir, "cover.jpg")
cover_cmd = build_cover_extract_command(job.cover_config, cover_path)
cover_cmd = cover_cmd.replace("INPUT_VIDEO", output_video_path)
cover_exit = os.system(cover_cmd)
if cover_exit != 0:
logger.warning("封面提取失败,跳过: %s", cover_cmd)
cover_path = ""
job.progress = 90
self.db.commit()
# 5. 上传到 OSS (95%)
output_video_url = self._upload_to_oss(output_video_path, f"ai-avatar/{job_id}/output.mp4") output_video_url = self._upload_to_oss(output_video_path, f"ai-avatar/{job_id}/output.mp4")
job.output_video_url = output_video_url job.output_video_url = output_video_url
# 封面:优先复用智能剪辑的 MediaKit 抽帧 + 质量评分选最佳帧; # 封面透传:如果用户已在 cover_config 中选定封面 URLmode=upload 的自定义上传 或
# MediaKit 不可用时回退到 FFmpeg 已按 cover_config 抽取的 cover_path # mode=auto_frame 已有的智能封面结果),直接透传到 output_cover_url,不再重新截帧。
smart_cover_url = "" if isinstance(job.cover_config, dict):
if output_video_url: _pre_cover_url = (
try: job.cover_config.get("url")
from app.services.ai_avatar_cover_service import ( or job.cover_config.get("imageUrl")
generate_smart_cover, or job.cover_config.get("cover_url")
) or ""
)
smart_cover_url = generate_smart_cover(output_video_url, job_id=job_id, max_frames=5) if _pre_cover_url:
except Exception: job.output_cover_url = _pre_cover_url
logger.warning("智能封面(MediaKit)失败,回退 FFmpeg 封面 job_id=%s", job_id, exc_info=True) logger.info("[数字人渲染] 使用用户已选定封面 URL: job_id=%s", job_id)
if smart_cover_url:
job.output_cover_url = smart_cover_url
elif cover_path:
output_cover_url = self._upload_to_oss(cover_path, f"ai-avatar/{job_id}/cover.jpg")
job.output_cover_url = output_cover_url
# 获取输出视频时长 # 获取输出视频时长
job.output_duration = lipsync_job.output_duration job.output_duration = lipsync_job.output_duration
@@ -331,9 +408,13 @@ class AiAvatarRenderService:
from packages.domain.generated_video import GeneratedVideo from packages.domain.generated_video import GeneratedVideo
clip_name = f"AI数字人_{job_id[:8]}" clip_name = f"AI数字人_{job_id[:8]}"
# AI数字人入口是独立页面,前端可能不传 project_id(无项目概念),
# 兜底为 "ai_avatar" 避免 DB 非空约束/查询问题;generation_task_id 同样兜底用 render_job_id
clip_project_id = (job.project_id or "").strip() or "ai_avatar"
clip_generation_task_id = (job.lipsync_job_id or "").strip() or job_id
clip = GeneratedVideo.create( clip = GeneratedVideo.create(
project_id=job.project_id, project_id=clip_project_id,
generation_task_id=job.lipsync_job_id, generation_task_id=clip_generation_task_id,
name=clip_name, name=clip_name,
file_url=job.output_video_url, file_url=job.output_video_url,
user_id=job.user_id, user_id=job.user_id,
@@ -347,11 +428,11 @@ class AiAvatarRenderService:
video_repo = SQLAlchemyGeneratedVideoRepository(self.db) video_repo = SQLAlchemyGeneratedVideoRepository(self.db)
video_repo.create(clip) video_repo.create(clip)
logger.info("成片记录已保存到成片库: clip_id=%s, render_job=%s", clip.id, job_id) logger.info("成片记录已保存到成片库: clip_id=%s, render_job=%s", clip.id, job_id)
except Exception as clip_err: except Exception:
logger.warning( logger.error(
"自动保存成片记录失败(不影响渲染任务状态): render_job=%s, error=%s", "自动保存成片记录失败(不影响渲染任务状态): render_job=%s",
job_id, job_id,
clip_err, exc_info=True,
) )
except AiAvatarRenderError as exc: except AiAvatarRenderError as exc:
@@ -360,12 +441,14 @@ class AiAvatarRenderService:
job.updated_at = datetime.now(timezone.utc) job.updated_at = datetime.now(timezone.utc)
self.db.commit() self.db.commit()
logger.error("渲染任务失败 [%s]: %s", job_id, exc) logger.error("渲染任务失败 [%s]: %s", job_id, exc)
raise
except Exception as exc: except Exception as exc:
job.status = "failed" job.status = "failed"
job.error_message = f"渲染异常: {str(exc)}" job.error_message = f"渲染异常: {str(exc)}"
job.updated_at = datetime.now(timezone.utc) job.updated_at = datetime.now(timezone.utc)
self.db.commit() self.db.commit()
logger.exception("渲染任务异常 [%s]", job_id) logger.exception("渲染任务异常 [%s]", job_id)
raise
def _download_video(self, url: str) -> str: def _download_video(self, url: str) -> str:
"""下载视频到临时文件.""" """下载视频到临时文件."""
@@ -383,32 +466,132 @@ class AiAvatarRenderService:
os.unlink(tmp.name) os.unlink(tmp.name)
raise raise
@staticmethod
def _save_title_dataurl_to_file(dataurl: str, *, dst_path: str | None = None, job_id: str = "") -> str:
"""解码前端传来的 data:image/png;base64,... 并保存为本地 PNG 文件。
Args:
dataurl: 完整 dataURL 字符串
dst_path: 指定输出路径;为 None 时创建临时文件并返回路径
job_id: 仅在 dst_path 为空时用于临时文件命名
Returns:
保存后的本地文件路径
"""
if not isinstance(dataurl, str) or not dataurl.startswith("data:image/"):
raise ValueError("title_image_dataurl 不是合法的 data:image URL")
# 拆分 data:image/png;base64,<payload>
try:
header, b64 = dataurl.split(",", 1)
except ValueError as exc:
raise ValueError("title_image_dataurl 缺少 base64 payload") from exc
if "base64" not in header:
raise ValueError("title_image_dataurl 不是 base64 编码")
try:
png_bytes = base64.b64decode(b64, validate=True)
except (binascii.Error, ValueError) as exc:
raise ValueError(f"title_image_dataurl base64 解码失败: {exc}") from exc
if not png_bytes:
raise ValueError("title_image_dataurl 解码后为空")
if dst_path:
out_path = dst_path
with open(out_path, "wb") as f:
f.write(png_bytes)
return out_path
suffix = f"_title_{job_id}.png" if job_id else "_title.png"
with tempfile.NamedTemporaryFile(suffix=suffix, delete=False) as tmp:
tmp.write(png_bytes)
return tmp.name
@staticmethod
def _probe_video_resolution(video_path: str) -> tuple[int, int]:
"""用 ffprobe 探测视频分辨率,返回 (width, height);失败返回 (0, 0)。"""
try:
result = subprocess.run(
[
"ffprobe",
"-v",
"error",
"-select_streams",
"v:0",
"-show_entries",
"stream=width,height",
"-of",
"csv=p=0:s=x",
video_path,
],
capture_output=True,
text=True,
timeout=15,
)
if result.returncode == 0 and result.stdout.strip():
parts = result.stdout.strip().split("x")
if len(parts) == 2:
w, h = int(parts[0]), int(parts[1])
if w > 0 and h > 0:
return w, h
except Exception as exc:
logger.warning("[数字人渲染] ffprobe 探测分辨率失败: %s", exc)
return 0, 0
def _build_ffmpeg_command( def _build_ffmpeg_command(
self, self,
*, *,
input_video: str, input_video: str,
b_roll_segments: list[dict[str, Any]], b_roll_segments: list[dict[str, Any]],
extra_inputs: list[str] | None = None,
filter_complex: str, filter_complex: str,
final_label: Optional[str], final_label: Optional[str],
output_path: str, output_path: str,
) -> str: ) -> list[str]:
"""构建 FFmpeg 命令.""" """构建 FFmpeg 命令list 形式,shell=False.
# 输入文件
inputs = f"-i {input_video}" 根因修复 #1798 P0OSS 预签名 URL 含 `&Expires=...&Signature=...` 特殊字符,
os.system(shell=True) 会把 `&` 解释为后台命令分隔符,导致 -filter_complex 被
当成独立命令报 sh: -filter_complex: not foundexit 127 → Python 32512)。
list + shell=False 彻底规避 shell 转义问题。
"""
cmd: list[str] = ["ffmpeg", "-i", input_video]
for seg in b_roll_segments: for seg in b_roll_segments:
asset_url = seg.get("asset_url", "") asset_url = seg.get("asset_url", "")
if asset_url: if asset_url:
inputs += f" -i {asset_url}" cmd.extend(["-i", asset_url])
# 额外输入(例如前端 Canvas 渲染的标题 PNG)
for extra in extra_inputs or []:
cmd.extend(["-i", extra])
# 滤镜
if filter_complex and final_label: if filter_complex and final_label:
filter_arg = f'-filter_complex "{filter_complex}" -map "[{final_label}]"' cmd.extend(
[
"-filter_complex",
filter_complex,
"-map",
f"[{final_label}]",
"-map",
"0:a?",
]
)
elif filter_complex: elif filter_complex:
filter_arg = f'-filter_complex "{filter_complex}"' cmd.extend(["-filter_complex", filter_complex])
else:
filter_arg = ""
return f"ffmpeg {inputs} {filter_arg} -c:v libx264 -preset veryfast -crf 23 -y {output_path}" cmd.extend(
[
"-c:v",
"libx264",
"-preset",
"veryfast",
"-crf",
"23",
"-c:a",
"aac",
"-b:a",
"128k",
"-y",
output_path,
]
)
return cmd
def _upload_to_oss(self, local_path: str, oss_key: str) -> str: def _upload_to_oss(self, local_path: str, oss_key: str) -> str:
"""上传文件到 OSS,返回 URL. """上传文件到 OSS,返回 URL.
+247 -64
View File
@@ -1,8 +1,12 @@
"""对口型 Service — #1796 MediaKit 对口型业务逻辑, #1809 参数调整. """对口型 Service — #1796 MediaKit 对口型业务逻辑, #1809 参数调整, #1845 配音前置.
职责: 职责:
- 创建/查询对口型任务 - 创建/查询对口型任务
- 输入模式:TTS 直生(voice_id + script_text,内部先合成音频转存 OSS)或直接音频(audio_url - 输入模式:
1. TTS 直生(voice_id + script_text)→ 走 Celery 异步(降级路径)
2. 直接音频(audio_url,前端未传 timings)→ 同步下载 + 算 timings + 提交 MediaKit
3. 预合成音频(audio_url + sentence_timings#1845 新主路径)→ 同步 ffprobe 校验时长 +
写入前端传来的 timings → 直接提交 MediaKit~2-3s
- 调用 MediaKit 客户端提交异步任务 - 调用 MediaKit 客户端提交异步任务
- 轮询更新任务状态(中间状态同步 DB,成片转存自家 OSS) - 轮询更新任务状态(中间状态同步 DB,成片转存自家 OSS)
- 用户隔离(每个用户只能操作自己的任务) - 用户隔离(每个用户只能操作自己的任务)
@@ -26,19 +30,22 @@ from app.services.mediakit_client import (
get_mediakit_client, get_mediakit_client,
) )
# Celery 异步任务:TTS 合成 + MediaKit 提交(#lipsync-speed-optimization # Celery 异步任务:TTS 合成 + MediaKit 提交(降级路径
from app.tasks.lipsync_tts import tts_synthesize_and_submit from app.tasks.lipsync_tts import tts_synthesize_and_submit
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from packages.adapters.sqlalchemy_impl.models import LipsyncJobModel from packages.adapters.sqlalchemy_impl.models import LipsyncJobModel
from packages.application.cosyvoice_service import CosyVoiceError, normalize_emotion from packages.application.cosyvoice_service import CosyVoiceError, normalize_emotion
from packages.domain.sentence_timings import (
compute_sentence_timings,
probe_audio_duration,
)
from packages.shared.storage import get_shared_storage_service from packages.shared.storage import get_shared_storage_service
from packages.shared.url_security import ALLOWED_AUDIO_MIME_TYPES, safe_download_bytes from packages.shared.url_security import ALLOWED_AUDIO_MIME_TYPES, safe_download_bytes
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
# 传给 MediaKit GPU worker / 回给前端播放的 OSS 预签名有效期:7 天。 # 传给 MediaKit GPU worker / 回给前端播放的 OSS 预签名有效期:7 天。
# MediaKit 排队 + 拉取可能延迟,私有桶裸 URL 或 1 小时短预签名都会 403,故统一重签长有效期。
MEDIAKIT_URL_TTL_SECONDS = 7 * 24 * 3600 MEDIAKIT_URL_TTL_SECONDS = 7 * 24 * 3600
@@ -142,6 +149,100 @@ class LipsyncService:
logger.warning("TTS 音频转存 OSS 失败,回退临时 URL: job_id=%s err=%s", job_id, exc) logger.warning("TTS 音频转存 OSS 失败,回退临时 URL: job_id=%s err=%s", job_id, exc)
return temp_url return temp_url
def _submit_audio_direct(
self,
*,
job: LipsyncJobModel,
supplied_timings: Optional[list] = None,
supplied_duration: Optional[float] = None,
) -> None:
"""音频直传模式(包含 #1845 预合成路径):同步下载 → ffprobe → timings → 提交 MediaKit.
直接在 HTTP 请求内完成,不走 Celery。job.status 成功后置为 submitted。
失败时把 job 标成 failed 并 commit,然后抛 MediaKitError。
Args:
job: 已 commit 的 LipsyncJobModelaudio_url / video_url 已写入)
supplied_timings: 前端传来的预合成 timings(可选,可信时直接用)
supplied_duration: 前端传来的预合成时长(可选,用于优先避免重复探测)
"""
# 1. 下载音频
audio_data: bytes | None = None
try:
audio_data = safe_download_bytes(
job.audio_url,
purpose="lipsync_direct_audio",
allowed_mime_types=ALLOWED_AUDIO_MIME_TYPES,
timeout=60.0,
)
logger.info(
"[lipsync] 直传音频下载完成: job_id=%s size=%d",
job.id,
len(audio_data) if audio_data else 0,
)
except Exception as exc:
logger.warning("[lipsync] 直传音频下载失败,跳过 timings 计算: job_id=%s err=%s", job.id, exc)
# 2. ffprobe 探测时长(优先用前端传入的预合成时长,但以 ffprobe 为准做兜底校验)
audio_duration = 0.0
if audio_data:
audio_duration = probe_audio_duration(audio_data)
if audio_duration <= 0 and supplied_duration and supplied_duration > 0:
audio_duration = supplied_duration
logger.info(
"[lipsync] ffprobe 失败,使用前端传入的预合成时长: job_id=%s duration=%.2f", job.id, audio_duration
)
# 3. 句子时间戳:优先用前端预合成传入的 timings(后端预合成接口已经算过,可信);
# 否则若音频下载成功则重算;否则不设置(不阻塞主流程)
timings: Optional[list] = None
if supplied_timings:
timings = supplied_timings
logger.info("[lipsync] 使用前端预合成句子时间戳: job_id=%s sentences=%d", job.id, len(timings))
elif audio_data and audio_duration > 0 and job.script_text:
try:
timings = compute_sentence_timings(audio_data, job.script_text, audio_duration)
logger.info(
"[lipsync] 后端重算句子时间戳: job_id=%s sentences=%d duration=%.2f",
job.id,
len(timings) if timings else 0,
audio_duration,
)
except Exception as exc:
logger.warning("[lipsync] 句子时间戳计算失败(不阻塞): job_id=%s err=%s", job.id, exc)
if timings:
job.sentence_timings = timings
# 4. 签名 URL 并提交 MediaKit
video_url = self._sign_media_url(job.video_url)
signed_audio_url = self._sign_media_url(job.audio_url)
job.audio_url = signed_audio_url
try:
result = self.client.submit_lipsync(
video_url=video_url,
audio_url=signed_audio_url,
enable_video_loop=job.enable_video_loop,
client_token=job.id,
)
job.mediakit_task_id = result["task_id"]
job.status = "submitted"
job.submitted_at = datetime.now(timezone.utc)
self.db.commit()
logger.info(
"[lipsync] 直传音频已提交 MediaKit: job_id=%s task_id=%s",
job.id,
result["task_id"],
)
except MediaKitError as exc:
job.status = "failed"
job.error_message = str(exc)
job.error_code = exc.code
logger.error("[lipsync] 直传音频提交 MediaKit 失败: job_id=%s err=%s", job.id, exc)
self.db.commit()
raise
# ── 创建任务 ────────────────────────────────────────────────────────── # ── 创建任务 ──────────────────────────────────────────────────────────
def create_job( def create_job(
@@ -150,27 +251,35 @@ class LipsyncService:
user_id: str, user_id: str,
video_url: str, video_url: str,
audio_url: str = "", audio_url: str = "",
audio_duration: Optional[float] = None,
sentence_timings: Optional[list] = None,
voice_id: str = "", voice_id: str = "",
script_text: str = "", script_text: str = "",
speed: float = 1.0, speed: float = 1.0,
emotion: str = "", emotion: str = "",
enable_video_loop: bool = False, enable_video_loop: bool = True,
project_id: str = "", project_id: str = "",
) -> LipsyncJobModel: ) -> LipsyncJobModel:
"""创建对口型任务. """创建对口型任务.
种输入模式: 种输入模式:
- TTS 直生:voice_id + script_textaudio_url 留空) - TTS 直生:voice_id + script_textaudio_url 留空)
创建 DB 记录(状态 tts_processing),dispatch Celery 异步任务 → 创建 DB 记录(状态 tts_processing),dispatch Celery 异步任务(降级路径)。
执行 TTS 合成 + MediaKit 提交。API 响应 <1s。 API 响应 <1s。
- 直接音频:提供 audio_url - 直接音频:audio_url 非空 + 无 sentence_timings
→ 同步提交 MediaKit,状态直接设为 submitted → 同步下载音频 + 重算 timings + 提交 MediaKit(几秒完成)
- 预合成音频(#1845 新主路径):audio_url 非空 + 传 sentence_timings
→ 同步 ffprobe 校验时长 + 写入 timings + 提交 MediaKit~2-3s)。
Raises: Raises:
MediaKitError: 参数校验失败或 MediaKit 提交失败(仅直接音频模式) MediaKitError: 参数校验失败或 MediaKit 提交失败
""" """
# 0. 输入校验 # 0. 输入校验
if not audio_url: is_pre_synth = bool(audio_url) and bool(sentence_timings)
bool(audio_url) and not is_pre_synth
is_tts_mode = not bool(audio_url)
if is_tts_mode:
if not (voice_id and script_text): if not (voice_id and script_text):
raise MediaKitError( raise MediaKitError(
"必须提供 audio_url 或 voice_id+script_text", "必须提供 audio_url 或 voice_id+script_text",
@@ -178,10 +287,13 @@ class LipsyncService:
) )
# TTS 模式:在 HTTP 请求中同步校验音色归属,快速失败 # TTS 模式:在 HTTP 请求中同步校验音色归属,快速失败
self._resolve_voice_id(voice_id, user_id) self._resolve_voice_id(voice_id, user_id)
elif is_pre_synth:
# 预合成模式:script_text 可空(因为 timings 已自带句子文本),但仍建议传
if not isinstance(sentence_timings, list) or len(sentence_timings) == 0:
raise MediaKitError("预合成模式 sentence_timings 不能为空", code="InvalidInput")
# 1. 创建数据库记录 # 1. 创建数据库记录
job_id = str(uuid.uuid4()) job_id = str(uuid.uuid4())
is_tts_mode = not bool(audio_url)
job = LipsyncJobModel( job = LipsyncJobModel(
id=job_id, id=job_id,
user_id=user_id, user_id=user_id,
@@ -192,14 +304,19 @@ class LipsyncService:
voice_id=voice_id or "", voice_id=voice_id or "",
script_text=script_text or "", script_text=script_text or "",
speed=speed, speed=speed,
emotion=normalize_emotion(emotion), emotion=normalize_emotion(emotion) if is_tts_mode else (emotion or ""),
# 音频直传(含预合成)直接进入 pending(后续同步改为 submitted);TTS 模式进入 tts_processing
status="tts_processing" if is_tts_mode else "pending", status="tts_processing" if is_tts_mode else "pending",
) )
self.db.add(job) self.db.add(job)
self.db.flush() self.db.flush()
# ⚠️ 必须先 commit 再发 Celery 任务 / 后续同步操作,避免事务竞态
self.db.commit()
self.db.refresh(job)
if is_tts_mode: if is_tts_mode:
# 2a. TTS 模式:dispatch Celery 异步任务处理 TTS 合成 + MediaKit 提交 # 2a. TTS 模式:dispatch Celery 异步任务处理 TTS 合成 + MediaKit 提交(降级路径)
try: try:
tts_synthesize_and_submit.apply_async( tts_synthesize_and_submit.apply_async(
args=( args=(
@@ -212,8 +329,6 @@ class LipsyncService:
) )
) )
except Exception as exc: except Exception as exc:
# 投递失败时立即把 job 标成 failed 并写入 error_message
# 前端轮询时能直接看到失败原因,不会无限卡在 tts_processing。
logger.exception( logger.exception(
"Celery 任务提交失败,TTS 任务已创建但未触发执行: job_id=%s err=%s", "Celery 任务提交失败,TTS 任务已创建但未触发执行: job_id=%s err=%s",
job_id, job_id,
@@ -223,34 +338,102 @@ class LipsyncService:
job.error_message = f"Celery 任务投递失败: {exc}" job.error_message = f"Celery 任务投递失败: {exc}"
job.error_code = "AsyncDispatchFailed" job.error_code = "AsyncDispatchFailed"
job.updated_at = datetime.now(timezone.utc) job.updated_at = datetime.now(timezone.utc)
self.db.commit()
else: else:
# 2b. 直接音频模式:同步签名并提交 MediaKit # 2b/2c. 直接音频 / 预合成音频:同步路径
video_url = self._sign_media_url(video_url) self._submit_audio_direct(
if audio_url: job=job,
audio_url = self._sign_media_url(audio_url) supplied_timings=sentence_timings,
job.audio_url = audio_url supplied_duration=audio_duration,
)
self.db.refresh(job)
try:
result = self.client.submit_lipsync(
video_url=video_url,
audio_url=audio_url,
enable_video_loop=enable_video_loop,
client_token=job_id,
)
job.mediakit_task_id = result["task_id"]
job.status = "submitted"
job.submitted_at = datetime.now(timezone.utc)
except MediaKitError as exc:
job.status = "failed"
job.error_message = str(exc)
job.error_code = exc.code
logger.error("提交对口型任务失败: %s", exc)
raise
self.db.commit()
self.db.refresh(job)
return job return job
# ── TTS 预合成(#1845 步骤1「生成配音」同步接口使用) ──────────────────
def preview_tts(
self,
*,
user_id: str,
voice_id: str,
script_text: str,
speed: float = 1.0,
emotion: str = "natural",
) -> dict:
"""同步做 TTS 合成 + 下载 + ffprobe + 句子时间戳计算.
不创建 LipsyncJob、不转存 OSS,直接返回 CosyVoice 临时 URL~24h 有效期)。
耗时约 2-3 秒,由前端在步骤1点「生成配音」时同步等待。
Returns:
{"audio_url": str, "duration": float, "sentence_timings": list[dict]}
Raises:
MediaKitError: TTS 合成失败 / 下载失败 / ffprobe 失败
"""
# 1. 音色解析(校验克隆音色归属)
actual_voice_id = self._resolve_voice_id(voice_id, user_id)
cosyvoice = self._get_cosyvoice()
# 2. TTS 合成(同步,~2-3s
try:
result = cosyvoice.submit_synthesize_task(
text=script_text,
voice_id=actual_voice_id,
speed=speed,
emotion=normalize_emotion(emotion),
)
except CosyVoiceError as exc:
raise MediaKitError(f"TTS 合成失败: {exc}", code="TTSSynthesisFailed") from exc
except ValueError as exc:
raise MediaKitError(f"TTS 参数错误: {exc}", code="TTSInvalidParam") from exc
temp_url = result.get("audio_url", "")
if not temp_url:
raise MediaKitError("TTS 未返回音频 URL", code="TTSNoAudio")
# 3. 下载音频到内存(用于 ffprobe + 静音检测)
try:
audio_data = safe_download_bytes(
temp_url,
purpose="tts_preview_audio",
allowed_mime_types=ALLOWED_AUDIO_MIME_TYPES,
timeout=60.0,
)
except Exception as exc:
logger.warning("[tts-preview] TTS 音频下载失败,仍返回 audio_url: user_id=%s err=%s", user_id, exc)
return {
"audio_url": temp_url,
"duration": 0.0,
"sentence_timings": [],
}
# 4. ffprobe 时长
duration = probe_audio_duration(audio_data)
if duration <= 0:
logger.warning("[tts-preview] ffprobe 未返回有效时长,timings 留空: user_id=%s", user_id)
return {
"audio_url": temp_url,
"duration": 0.0,
"sentence_timings": [],
}
# 5. 句子时间戳
timings = compute_sentence_timings(audio_data, script_text, duration)
logger.info(
"[tts-preview] TTS 预合成完成: user_id=%s duration=%.2f sentences=%d",
user_id,
duration,
len(timings),
)
return {
"audio_url": temp_url,
"duration": round(duration, 2),
"sentence_timings": timings,
}
# ── 查询任务 ────────────────────────────────────────────────────────── # ── 查询任务 ──────────────────────────────────────────────────────────
def get_job(self, job_id: str, user_id: str) -> Optional[LipsyncJobModel]: def get_job(self, job_id: str, user_id: str) -> Optional[LipsyncJobModel]:
@@ -284,11 +467,7 @@ class LipsyncService:
# ── 更新任务状态(轮询) ────────────────────────────────────────────── # ── 更新任务状态(轮询) ──────────────────────────────────────────────
def refresh_job_status(self, job_id: str, user_id: str) -> Optional[LipsyncJobModel]: def refresh_job_status(self, job_id: str, user_id: str) -> Optional[LipsyncJobModel]:
"""从 MediaKit 拉取最新状态并更新本地记录. """从 MediaKit 拉取最新状态并更新本地记录."""
Returns:
更新后的 Job,或 None(任务不存在/不属于该用户)
"""
job = self.get_job(job_id, user_id) job = self.get_job(job_id, user_id)
if job is None: if job is None:
return None return None
@@ -313,11 +492,25 @@ class LipsyncService:
if mk_status == STATUS_COMPLETED: if mk_status == STATUS_COMPLETED:
result = status_data.get("result", {}) result = status_data.get("result", {})
job.status = STATUS_COMPLETED job.status = STATUS_COMPLETED
output_url = result.get("video_url", "") temp_url = result.get("video_url", "")
# MediaKit 输出为临时 URL,转存自家 OSS 防止过期(失败则回退临时 URL) job.output_video_url = temp_url
job.output_video_url = self._persist_output_video(output_url, job_id, user_id)
job.output_duration = result.get("duration", 0.0) job.output_duration = result.get("duration", 0.0)
job.completed_at = datetime.now(timezone.utc) job.completed_at = datetime.now(timezone.utc)
job.updated_at = datetime.now(timezone.utc)
self.db.commit()
# 异步转存自家 OSS
try:
from app.tasks.lipsync_tts import persist_output_video_task
persist_output_video_task.apply_async(args=(job_id, user_id, temp_url))
except Exception as exc:
logger.warning(
"提交输出视频异步转存任务失败,保留临时 URL: job_id=%s err=%s",
job_id,
exc,
)
self.db.refresh(job)
return job
elif mk_status == STATUS_FAILED: elif mk_status == STATUS_FAILED:
error = status_data.get("error", {}) error = status_data.get("error", {})
job.status = "failed" job.status = "failed"
@@ -325,7 +518,6 @@ class LipsyncService:
job.error_code = error.get("code", "TaskFailed") job.error_code = error.get("code", "TaskFailed")
job.completed_at = datetime.now(timezone.utc) job.completed_at = datetime.now(timezone.utc)
else: else:
# 中间状态(running/processing/queued 等)同步到 DB,避免前端永远卡在 submitted
if isinstance(mk_status, str) and mk_status: if isinstance(mk_status, str) and mk_status:
job.status = mk_status job.status = mk_status
job.updated_at = datetime.now(timezone.utc) job.updated_at = datetime.now(timezone.utc)
@@ -334,10 +526,7 @@ class LipsyncService:
return job return job
def _persist_output_video(self, temp_url: str, job_id: str, user_id: str) -> str: def _persist_output_video(self, temp_url: str, job_id: str, user_id: str) -> str:
"""将 MediaKit 输出的临时视频 URL 转存到自家 OSS. """将 MediaKit 输出的临时视频 URL 转存到自家 OSS. 失败时回退返回原始临时 URL."""
失败时回退返回原始临时 URL,不影响任务完成。
"""
if not temp_url: if not temp_url:
return "" return ""
try: try:
@@ -357,27 +546,21 @@ class LipsyncService:
return temp_url return temp_url
def _sign_media_url(self, url: str) -> str: def _sign_media_url(self, url: str) -> str:
"""对自家 OSS 私有桶 URL 重签长有效期预签名,供 MediaKit 拉取 / 前端播放。 """对自家 OSS 私有桶 URL 重签长有效期预签名."""
- 裸 public_urlupload_file 返回,不带签名)→ 私有桶匿名访问 403,重签。
- 已带签名但即将过期的 URL(如前端 1h 预签名)→ 抽 storage_key 后重签。
- 外部 URLCosyVoice/MediaKit 临时链接,非本桶 host)→ 原样透传。
- 任何异常都降级原样返回,不阻断主流程。
"""
if not url: if not url:
return url return url
try: try:
storage = get_shared_storage_service() storage = get_shared_storage_service()
public_base = getattr(storage, "public_url", "") public_base = getattr(storage, "public_url", "")
if not isinstance(public_base, str) or not public_base: if not isinstance(public_base, str) or not public_base:
return url # 无法判定归属,保守透传 return url
own_host = urlparse(public_base).netloc.lower() own_host = urlparse(public_base).netloc.lower()
host = urlparse(url).netloc.lower() host = urlparse(url).netloc.lower()
if not own_host or host != own_host: if not own_host or host != own_host:
return url # 非自家 OSS外部临时链接),不处理 return url # 外部临时链接原样透传
signed = storage.get_download_url(url, expires_seconds=MEDIAKIT_URL_TTL_SECONDS) signed = storage.get_download_url(url, expires_seconds=MEDIAKIT_URL_TTL_SECONDS)
return signed or url return signed or url
except Exception as exc: # noqa: BLE001 - 签名失败不阻断,降级原 URL except Exception as exc:
logger.warning("对口型 URL 重签失败,原样返回: url_prefix=%s err=%s", url[:80], exc) logger.warning("对口型 URL 重签失败,原样返回: url_prefix=%s err=%s", url[:80], exc)
return url return url
+2 -3
View File
@@ -75,7 +75,7 @@ class MediaKitClient:
*, *,
video_url: str, video_url: str,
audio_url: str, audio_url: str,
enable_video_loop: bool = False, enable_video_loop: bool = True,
callback_url: Optional[str] = None, callback_url: Optional[str] = None,
callback_args: Optional[str] = None, callback_args: Optional[str] = None,
client_token: Optional[str] = None, client_token: Optional[str] = None,
@@ -103,8 +103,7 @@ class MediaKitClient:
"video_url": video_url, "video_url": video_url,
"audio_url": audio_url, "audio_url": audio_url,
} }
if enable_video_loop: payload["enable_video_loop"] = bool(enable_video_loop)
payload["enable_video_loop"] = True
if callback_url: if callback_url:
payload["callback_url"] = callback_url payload["callback_url"] = callback_url
if callback_args: if callback_args:
+148 -15
View File
@@ -12,6 +12,10 @@
注意:使用 @shared_task 而非绑定到某个 celery_app 实例, 注意:使用 @shared_task 而非绑定到某个 celery_app 实例,
确保任务能被 Worker 侧 celery_app 正确注册,同时 API 侧 send_task/apply_async 仍可正常调用。 确保任务能被 Worker 侧 celery_app 正确注册,同时 API 侧 send_task/apply_async 仍可正常调用。
#1845:句子时间戳计算已提取至 packages/domain/sentence_timings.py,本模块保留
_ 开头别名兼容历史导入,但 _compute_sentence_timings/_split_script_into_sentences/
_estimate_sentence_timings_by_chars 等内部函数已复用共享实现,避免重复代码。
""" """
import io import io
@@ -21,6 +25,12 @@ from urllib.parse import urlparse
from celery import shared_task from celery import shared_task
# 复用共享的句子时间戳工具(#1845 配音前置)
from packages.domain.sentence_timings import compute_sentence_timings as _compute_sentence_timings
from packages.domain.sentence_timings import (
probe_audio_duration,
)
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
# MediaKit 预签名 URL 有效期(7天,秒),与 LipsyncService._sign_media_url 保持一致 # MediaKit 预签名 URL 有效期(7天,秒),与 LipsyncService._sign_media_url 保持一致
@@ -57,8 +67,13 @@ def _sign_media_url(url: str) -> str:
@shared_task( @shared_task(
bind=True, bind=True,
name="lipsync_tts.synthesize_and_submit", name="lipsync_tts.synthesize_and_submit",
max_retries=2, max_retries=5, # 事务竞态重试3次(job not found+ TTS偶发错误2次
default_retry_delay=30, default_retry_delay=30,
autoretry_for=(OSError, ConnectionError), # 网络/连接错误自动重试
retry_backoff=True,
retry_backoff_max=30,
soft_time_limit=180,
time_limit=200,
) )
def tts_synthesize_and_submit( def tts_synthesize_and_submit(
self, self,
@@ -71,7 +86,8 @@ def tts_synthesize_and_submit(
): ):
"""异步执行 TTS 合成 + OSS 转存 + MediaKit 提交. """异步执行 TTS 合成 + OSS 转存 + MediaKit 提交.
在 Celery worker 中运行,不阻塞 HTTP 请求。 在 Celery worker 中运行,不阻塞 HTTP 请求。保留作为降级路径
(预合成失败 / 旧版前端未传 audio_url 时走此路径)。
""" """
from app.services.mediakit_client import MediaKitError, get_mediakit_client from app.services.mediakit_client import MediaKitError, get_mediakit_client
from sqlalchemy.orm import Session as DBSession from sqlalchemy.orm import Session as DBSession
@@ -103,7 +119,25 @@ def tts_synthesize_and_submit(
) )
if job is None: if job is None:
logger.error("[lipsync_tts] Job not found: job_id=%s", job_id) # 事务竞态防御:API 在 commit 前投递了任务,worker 消费时事务尚未提交。
retries = getattr(self.request, "retries", 0)
max_retries = 3
if retries < max_retries:
backoff = (2**retries) + (retries * 1) # 1s, 3s, 7s
logger.warning(
"[lipsync_tts] Job not found yet (retry %d/%d, backoff %ds): job_id=%s",
retries + 1,
max_retries,
backoff,
job_id,
)
self.db.close()
raise self.retry(countdown=backoff, max_retries=max_retries)
logger.error(
"[lipsync_tts] Job not found after %d retries, giving up: job_id=%s",
max_retries,
job_id,
)
return return
# 已取消的任务不再处理 # 已取消的任务不再处理
@@ -112,6 +146,13 @@ def tts_synthesize_and_submit(
return return
# 1. TTS 合成 # 1. TTS 合成
logger.info(
"[lipsync_tts] 开始 TTS 合成: job_id=%s voice_id=%s text_len=%d speed=%.2f",
job_id,
voice_id,
len(script_text),
speed,
)
try: try:
cosyvoice = CosyVoiceService() cosyvoice = CosyVoiceService()
result = cosyvoice.submit_synthesize_task( result = cosyvoice.submit_synthesize_task(
@@ -147,37 +188,74 @@ def tts_synthesize_and_submit(
db.commit() db.commit()
return return
# 2. 下载转存自家 OSS # 2. 下载 TTS 音频到内存(用于 2.5 静音检测;不转存自家 OSS,直接使用 CosyVoice 临时 URL
audio_data: bytes | None = None
try: try:
audio_data = safe_download_bytes( audio_data = safe_download_bytes(
temp_url, temp_url,
purpose="lipsync_tts_audio", purpose="lipsync_tts_audio",
allowed_mime_types=( allowed_mime_types={
"audio/mpeg", "audio/mpeg",
"audio/mp3", "audio/mp3",
"audio/wav", "audio/wav",
"audio/x-wav", # CosyVoice 部分接口返回 audio/x-wav
"audio/mp4", "audio/mp4",
"audio/x-m4a", "audio/x-m4a",
), },
timeout=60.0, timeout=60.0,
) )
from packages.shared.storage import get_shared_storage_service logger.info(
"[lipsync_tts] TTS 音频已下载到内存: job_id=%s size=%d",
storage = get_shared_storage_service() job_id,
storage_key = f"lipsync-tts/{user_id}/{job_id}.mp3" len(audio_data) if audio_data else 0,
permanent_url = storage.upload_file(io.BytesIO(audio_data), storage_key, content_type="audio/mpeg") )
logger.info("[lipsync_tts] TTS 音频已转存 OSS: job_id=%s key=%s", job_id, storage_key)
job.audio_url = permanent_url
except Exception as exc: except Exception as exc:
logger.warning( logger.warning(
"[lipsync_tts] TTS 音频转存 OSS 失败,回退临时 URL: job_id=%s err=%s", "[lipsync_tts] TTS 音频下载失败,跳过静音检测,直接使用临时 URL 提交: job_id=%s err=%s",
job_id, job_id,
exc, exc,
) )
job.audio_url = temp_url # TTS 音频使用 CosyVoice 临时 URL,跳过自家 OSS 转存(加速,步骤⑥)
job.audio_url = temp_url
logger.info("[lipsync_tts] TTS 音频使用 CosyVoice 临时 URL(跳过 OSS 转存): job_id=%s", job_id)
db.commit() db.commit()
# 2.5 计算精确句子时间戳(基于 TTS 音频静音检测)—— 复用共享工具
try:
if not audio_data:
logger.warning("[lipsync_tts] 无音频数据,跳过句子时间戳计算: job_id=%s", job_id)
else:
_audio_duration = probe_audio_duration(audio_data)
logger.info(
"[lipsync_tts] 音频时长探测: job_id=%s duration=%.2f",
job_id,
_audio_duration,
)
if _audio_duration > 0:
_timings = _compute_sentence_timings(audio_data, script_text, _audio_duration)
if _timings:
job.sentence_timings = _timings
logger.info(
"[lipsync_tts] 句子时间戳已计算: job_id=%s sentences=%d duration=%.1f",
job_id,
len(_timings),
_audio_duration,
)
else:
logger.warning("[lipsync_tts] 句子时间戳计算返回空结果: job_id=%s", job_id)
else:
logger.warning(
"[lipsync_tts] ffprobe 未获取到有效时长,跳过句子时间戳: job_id=%s",
job_id,
)
db.commit()
except Exception as _st_err:
logger.warning(
"[lipsync_tts] 句子时间戳计算失败(不影响主流程): job_id=%s err=%s", job_id, _st_err, exc_info=True
)
# 3. 签名 URL 并提交到 MediaKit(复用模块内 _sign_media_url,避免对 LipsyncService 的耦合) # 3. 签名 URL 并提交到 MediaKit(复用模块内 _sign_media_url,避免对 LipsyncService 的耦合)
audio_url = _sign_media_url(job.audio_url) audio_url = _sign_media_url(job.audio_url)
video_url = _sign_media_url(job.video_url) video_url = _sign_media_url(job.video_url)
@@ -220,3 +298,58 @@ def tts_synthesize_and_submit(
logger.exception("[lipsync_tts] 回写失败状态时异常: job_id=%s", job_id) logger.exception("[lipsync_tts] 回写失败状态时异常: job_id=%s", job_id)
finally: finally:
db.close() db.close()
@shared_task(
name="lipsync_tts.persist_output_video",
max_retries=2,
default_retry_delay=30,
)
def persist_output_video_task(job_id: str, user_id: str, temp_url: str):
"""异步转存对口型输出视频到自家 OSS(步骤⑦ — 将同步阻塞挪到后台,加速前端响应)."""
try:
from worker_app.db import SessionLocal # type: ignore
except Exception: # noqa: BLE001
from app.db import SessionLocal # type: ignore
from packages.adapters.sqlalchemy_impl.models import LipsyncJobModel
from packages.shared.storage import get_shared_storage_service
db = SessionLocal()
try:
job = db.query(LipsyncJobModel).filter(LipsyncJobModel.id == job_id, LipsyncJobModel.user_id == user_id).first()
if job is None:
logger.error("[lipsync_tts.persist] Job not found: job_id=%s", job_id)
return
if not temp_url:
logger.warning("[lipsync_tts.persist] temp_url 为空,跳过转存: job_id=%s", job_id)
return
try:
import httpx
with httpx.Client(timeout=180.0, follow_redirects=True) as client:
resp = client.get(temp_url)
resp.raise_for_status()
data = resp.content
storage = get_shared_storage_service()
storage_key = f"lipsync-outputs/{user_id}/{job_id}.mp4"
permanent_url = storage.upload_file(io.BytesIO(data), storage_key, content_type="video/mp4")
final_url = _sign_media_url(permanent_url) if permanent_url else temp_url
job.output_video_url = final_url
job.updated_at = datetime.now(timezone.utc)
db.commit()
logger.info("[lipsync_tts.persist] 输出视频已转存 OSS: job_id=%s key=%s", job_id, storage_key)
except Exception as exc:
logger.warning(
"[lipsync_tts.persist] 输出视频转存失败,保留临时 URL: job_id=%s err=%s",
job_id,
exc,
)
except Exception:
logger.exception("[lipsync_tts.persist] 未预期异常: job_id=%s", job_id)
finally:
db.close()
+34 -11
View File
@@ -1,6 +1,6 @@
/** /**
* 成品 / 视频相关 API 函数 * 成品 / 视频相关 API 函数
* 后端实际接口:/videos * 后端实际接口:/videos(分页:page/page_size,返回 {items, total, page, page_size}
*/ */
import apiClient from "../client" import apiClient from "../client"
import type { import type {
@@ -12,16 +12,39 @@ import type {
} from "./types" } from "./types"
import { mapVideoToProductItem } from "./utils" import { mapVideoToProductItem } from "./utils"
/** 获取成品列表(支持分页和筛选 */ /** 分页列表响应(前端消费用 */
export const getProducts = async (params?: ProductListParams): Promise<ProductItem[]> => { export interface ProductListResult {
const response = await apiClient.get("/videos", { params }) items: ProductItem[]
const data = response.data total: number
const videos: VideoItem[] = Array.isArray(data?.items) page: number
? data.items page_size: number
: Array.isArray(data) }
? data
: [] /**
return videos.map(mapVideoToProductItem) * 获取成品列表(分页)
* @param params 分页与筛选参数:page 默认 1page_size 默认 20
*/
export const getProducts = async (params?: ProductListParams): Promise<ProductListResult> => {
const response = await apiClient.get("/videos", {
params: {
page: 1,
page_size: 20,
...params,
},
})
const data = response.data as {
items?: VideoItem[]
total?: number
page?: number
page_size?: number
}
const items: VideoItem[] = Array.isArray(data?.items) ? data.items : []
return {
items: items.map(mapVideoToProductItem),
total: data.total ?? items.length,
page: data.page ?? params?.page ?? 1,
page_size: data.page_size ?? params?.page_size ?? 20,
}
} }
/** 获取单个成品详情 */ /** 获取单个成品详情 */
+16 -1
View File
@@ -552,7 +552,7 @@
max-width: 240px; max-width: 240px;
aspect-ratio: 9/16; aspect-ratio: 9/16;
background: #f0f0f5; background: #f0f0f5;
border-radius: 8px; border-radius: 12px;
overflow: hidden; overflow: hidden;
display: flex; display: flex;
align-items: center; align-items: center;
@@ -564,8 +564,10 @@
.aa-cover-preview img { .aa-cover-preview img {
width: 100%; width: 100%;
height: 100%; height: 100%;
aspect-ratio: 9/16;
object-fit: cover; object-fit: cover;
display: block; display: block;
border-radius: 12px;
} }
.aa-cover-preview__placeholder { .aa-cover-preview__placeholder {
@@ -573,6 +575,19 @@
color: #8c8ca1; color: #8c8ca1;
} }
.aa-cover-preview__loading {
position: absolute;
inset: 0;
display: flex;
align-items: center;
justify-content: center;
background: rgba(0, 0, 0, 0.45);
color: #fff;
font-size: 13px;
backdrop-filter: blur(4px);
-webkit-backdrop-filter: blur(4px);
}
.aa-cover-actions { .aa-cover-actions {
display: flex; display: flex;
gap: 8px; gap: 8px;
+389 -74
View File
@@ -1,7 +1,7 @@
/** /**
* AI数字人 — 主页面(v3 两步骤版) * AI数字人 — 主页面(v3 两步骤版 + #1845 配音前置
* 步骤1:出镜视频 / 配音库 / 文案 * 步骤1:出镜视频 / 配音库 / 文案 → 点击「🎵 生成配音」做 TTS 预合成(同步,~2-3s)
* 步骤2:对口型预览(含插入画面)/ 标题配置 / 封面&生成 * 步骤2:对口型预览(音频已就绪、B-roll 句子时间戳立即可用)/ 标题配置 / 封面&生成
*/ */
import React, { useState, useCallback, useEffect, useRef } from "react" import React, { useState, useCallback, useEffect, useRef } from "react"
import { message } from "antd" import { message } from "antd"
@@ -21,15 +21,19 @@ import {
getAssetById, getAssetById,
createLipsyncJob, createLipsyncJob,
getLipsyncJob, getLipsyncJob,
previewTts,
submitRender, submitRender,
getRenderJob, getRenderJob,
generateSmartCover, generateRenderSmartCover,
} from "./api/aiAvatar" } from "./api/aiAvatar"
import { getOrCreateDefaultProject } from "@/api/projects"
import type { RenderJob, SentenceTiming } from "./types"
import { import {
normalizeEmotion, normalizeEmotion,
buildTitleConfigPayload, buildTitleConfigPayload,
buildCoverConfigPayload, buildCoverConfigPayload,
} from "./utils/contract" } from "./utils/contract"
import { renderTitleToPngDataUrl, getVideoResolution } from "./utils/titleCanvas"
/** 面板折叠状态 */ /** 面板折叠状态 */
type PanelKey = "video" | "voice" | "script" | "lipsync" | "title" | "cover" type PanelKey = "video" | "voice" | "script" | "lipsync" | "title" | "cover"
@@ -47,14 +51,18 @@ const AiAvatarPage: React.FC = () => {
cover: false, cover: false,
}) })
/* ── #1845 TTS 预合成弹窗 ── */
const [showTtsModal, setShowTtsModal] = useState(false)
const [ttsProgress, setTtsProgress] = useState(0)
const [ttsErrorMessage, setTtsErrorMessage] = useState("")
const ttsProgressTimerRef = useRef<ReturnType<typeof setInterval> | null>(null)
/* ── 对口型生成弹窗 ── */ /* ── 对口型生成弹窗 ── */
const [showLipsyncModal, setShowLipsyncModal] = useState(false) const [showLipsyncModal, setShowLipsyncModal] = useState(false)
const [lipsyncStatus, setLipsyncStatus] = useState<"generating" | "completed" | "failed">( const [lipsyncStatus, setLipsyncStatus] = useState<"generating" | "completed" | "failed">(
"generating", "generating",
) )
const [lipsyncErrorMessage, setLipsyncErrorMessage] = useState("") const [lipsyncErrorMessage, setLipsyncErrorMessage] = useState("")
/* ── 智能封面加载态 ── */
const [smartCoverLoading, setSmartCoverLoading] = useState(false)
/* ── 渲染进度弹窗 ── */ /* ── 渲染进度弹窗 ── */
const [showRenderModal, setShowRenderModal] = useState(false) const [showRenderModal, setShowRenderModal] = useState(false)
const [renderStatus, setRenderStatus] = useState<"generating" | "completed" | "failed">( const [renderStatus, setRenderStatus] = useState<"generating" | "completed" | "failed">(
@@ -62,6 +70,8 @@ const AiAvatarPage: React.FC = () => {
) )
const [renderProgress, setRenderProgress] = useState(0) const [renderProgress, setRenderProgress] = useState(0)
const [renderErrorMessage, setRenderErrorMessage] = useState("") const [renderErrorMessage, setRenderErrorMessage] = useState("")
/* ── 当前渲染任务对象 ── */
const [currentRenderJob, setCurrentRenderJob] = useState<RenderJob | null>(null)
/* ── 对口型轮询 ── */ /* ── 对口型轮询 ── */
const lipsyncTimerRef = useRef<ReturnType<typeof setInterval> | null>(null) const lipsyncTimerRef = useRef<ReturnType<typeof setInterval> | null>(null)
@@ -72,8 +82,29 @@ const AiAvatarPage: React.FC = () => {
setCollapsed((prev) => ({ ...prev, [key]: !prev[key] })) setCollapsed((prev) => ({ ...prev, [key]: !prev[key] }))
}, []) }, [])
/* ── 步骤切换 ── */ /* ── #1845 文案/音色/语速变更时重置 TTS 预合成状态,避免音频与文案不一致 ── */
const handleNextStep = useCallback(() => { useEffect(() => {
if (state.ttsPreview.status !== "idle") {
state.resetTtsPreview()
}
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [state.scriptText, state.selectedVoice?.voice_id, state.speed, state.emotion])
const _clearTtsProgressTimer = useCallback(() => {
if (ttsProgressTimerRef.current) {
clearInterval(ttsProgressTimerRef.current)
ttsProgressTimerRef.current = null
}
}, [])
useEffect(() => {
return () => {
_clearTtsProgressTimer()
}
}, [_clearTtsProgressTimer])
/* ── #1845 步骤1:点击「🎵 生成配音」→ 同步 TTS 预合成 ── */
const handleGenerateTts = useCallback(async () => {
const missing: string[] = [] const missing: string[] = []
if (!state.selectedVideo) missing.push("出镜视频") if (!state.selectedVideo) missing.push("出镜视频")
if (!state.selectedVoice) missing.push("配音") if (!state.selectedVoice) missing.push("配音")
@@ -82,45 +113,120 @@ const AiAvatarPage: React.FC = () => {
message.warning(`请先完成${missing.join("、")}`) message.warning(`请先完成${missing.join("、")}`)
return return
} }
setCurrentStep(2)
}, [state.selectedVideo, state.selectedVoice, state.scriptText])
// 打开弹窗 & 启动模拟进度条
setShowTtsModal(true)
setTtsProgress(0)
setTtsErrorMessage("")
state.setTtsPreview({
audioUrl: null,
duration: 0,
sentenceTimings: [],
status: "generating",
error: null,
})
// 模拟进度:每 300ms +10%,到 90% 停住,真完成后瞬间到 100%
_clearTtsProgressTimer()
let fake = 0
ttsProgressTimerRef.current = setInterval(() => {
fake = Math.min(fake + 10, 90)
setTtsProgress(fake)
if (fake >= 90) {
_clearTtsProgressTimer()
}
}, 300)
try {
const res = await previewTts({
voice_id: state.selectedVoice!.voice_id,
script_text: state.scriptText,
speed: state.speed,
emotion: normalizeEmotion(state.emotion),
})
_clearTtsProgressTimer()
setTtsProgress(100)
state.setTtsPreview({
audioUrl: res.audio_url,
duration: res.duration,
sentenceTimings: res.sentence_timings as SentenceTiming[],
status: "done",
error: null,
})
message.success("配音合成完成")
} catch (err) {
_clearTtsProgressTimer()
const errMsg =
(err as { response?: { data?: { message?: string; detail?: unknown } } })?.response?.data
?.message || (err instanceof Error ? err.message : "配音合成失败,请重试")
setTtsErrorMessage(typeof errMsg === "string" ? errMsg : "配音合成失败,请重试")
state.setTtsPreview({
audioUrl: null,
duration: 0,
sentenceTimings: [],
status: "failed",
error: typeof errMsg === "string" ? errMsg : "配音合成失败",
})
}
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [state.selectedVideo, state.selectedVoice, state.scriptText, state.speed, state.emotion])
const handleRetryTts = useCallback(() => {
handleGenerateTts()
}, [handleGenerateTts])
const handleTtsNext = useCallback(() => {
setShowTtsModal(false)
setTtsProgress(0)
setCurrentStep(2)
}, [])
const handleCancelTts = useCallback(() => {
_clearTtsProgressTimer()
setShowTtsModal(false)
setTtsProgress(0)
setTtsErrorMessage("")
// 若用户在生成中途关闭,把状态重置回 idle,允许重新点击
if (state.ttsPreview.status === "generating") {
state.resetTtsPreview()
}
}, [_clearTtsProgressTimer, state])
/* ── 上一步(返回步骤1,不会丢失 TTS 预合成结果) ── */
const handlePrevStep = useCallback(() => { const handlePrevStep = useCallback(() => {
setCurrentStep(1) setCurrentStep(1)
}, []) }, [])
/* ── 对口型 ── */ /* ── 对口型 ── */
const handleGenerateLipsync = useCallback(async () => { const handleGenerateLipsync = useCallback(async () => {
// ② 缺项明确提示(#1809):不再静默 return
const video = state.selectedVideo const video = state.selectedVideo
const voice = state.selectedVoice
const text = state.scriptText.trim() const text = state.scriptText.trim()
const missing: string[] = [] const missing: string[] = []
if (!video) missing.push("出镜视频") if (!video) missing.push("出镜视频")
if (!voice) missing.push("音色")
if (!text) missing.push("文案") if (!text) missing.push("文案")
if (missing.length > 0 || !video || !voice) { if (missing.length > 0 || !video) {
message.warning(`请先选择${missing.join("、")}`) message.warning(`请先选择${missing.join("、")}`)
return return
} }
// #1845:预合成模式下必须要有 audioUrl(理论上到了步骤2肯定有,兜底防御)
const isPreSynth = state.ttsPreview.status === "done" && !!state.ttsPreview.audioUrl
if (!isPreSynth && !state.selectedVoice) {
message.warning("请先选择音色或完成配音合成")
return
}
try { try {
// 显示生成弹窗
setShowLipsyncModal(true) setShowLipsyncModal(true)
setLipsyncStatus("generating") setLipsyncStatus("generating")
setLipsyncErrorMessage("") setLipsyncErrorMessage("")
// ① 先按素材 id 拿 file_url(#1809 补充:对齐后端新参数 video_url)
console.log("[对口型] 开始生成:", { console.log("[对口型] 开始生成:", {
videoId: video.id, videoId: video.id,
voiceId: voice.voice_id, mode: isPreSynth ? "pre-synth" : "tts-direct",
voiceType: voice.type,
textLen: state.scriptText.length, textLen: state.scriptText.length,
}) })
const asset = await getAssetById(video.id) const asset = await getAssetById(video.id)
console.log("[对口型] getAssetById 响应:", {
id: asset?.id,
file_url: asset?.file_url?.substring(0, 100),
})
const videoUrl = asset?.file_url const videoUrl = asset?.file_url
if (!videoUrl) { if (!videoUrl) {
console.error("[对口型] file_url 为空,asset:", asset) console.error("[对口型] file_url 为空,asset:", asset)
@@ -128,19 +234,35 @@ const AiAvatarPage: React.FC = () => {
message.error("获取出镜视频播放地址失败,请重新选择素材") message.error("获取出镜视频播放地址失败,请重新选择素材")
return return
} }
// ② 模式A TTS直生:video_url + voice_id + script_text,语速/情绪英文枚举透传(#1822)
const payload = { type LipsyncPayload = Parameters<typeof createLipsyncJob>[0]
voice_id: voice.voice_id, let payload: LipsyncPayload
script_text: state.scriptText, if (isPreSynth) {
video_url: videoUrl, // 预合成模式:传 audio_url + audio_duration + sentence_timings(后端直接提交 MediaKit~2-3s
speed: state.speed, // 语速 0.5~2.0 payload = {
emotion: normalizeEmotion(state.emotion), // natural/excited/calm/friendly video_url: videoUrl,
audio_url: state.ttsPreview.audioUrl!,
audio_duration: state.ttsPreview.duration,
sentence_timings: state.ttsPreview.sentenceTimings,
enable_video_loop: true,
}
} else {
// 降级:TTS 直生(旧路径,前端未预合成时)
payload = {
voice_id: state.selectedVoice!.voice_id,
script_text: state.scriptText,
video_url: videoUrl,
speed: state.speed,
emotion: normalizeEmotion(state.emotion),
}
} }
console.log("[对口型] createLipsyncJob 请求:", payload) console.log("[对口型] createLipsyncJob 请求:", payload)
const job = await createLipsyncJob(payload) const job = await createLipsyncJob(payload)
console.log("[对口型] createLipsyncJob 响应:", { id: job.id, status: job.status }) console.log("[对口型] createLipsyncJob 响应:", { id: job.id, status: job.status })
state.setLipsyncJob(job) state.setLipsyncJob(job)
// 开始轮询
// 如果是预合成模式,后端会同步把状态置为 submitted(甚至可能已返回 running),
// 但仍需轮询等 completed
if (lipsyncTimerRef.current) clearInterval(lipsyncTimerRef.current) if (lipsyncTimerRef.current) clearInterval(lipsyncTimerRef.current)
lipsyncTimerRef.current = setInterval(async () => { lipsyncTimerRef.current = setInterval(async () => {
try { try {
@@ -177,7 +299,14 @@ const AiAvatarPage: React.FC = () => {
message.error(err instanceof Error ? err.message : "对口型任务提交失败,请重试") message.error(err instanceof Error ? err.message : "对口型任务提交失败,请重试")
} }
// eslint-disable-next-line react-hooks/exhaustive-deps // eslint-disable-next-line react-hooks/exhaustive-deps
}, [state.selectedVideo, state.selectedVoice, state.scriptText, state.speed, state.emotion]) }, [
state.selectedVideo,
state.selectedVoice,
state.scriptText,
state.speed,
state.emotion,
state.ttsPreview,
])
// 取消对口型生成 // 取消对口型生成
const handleCancelLipsync = useCallback(() => { const handleCancelLipsync = useCallback(() => {
@@ -198,6 +327,14 @@ const AiAvatarPage: React.FC = () => {
} }
}, []) }, [])
/* ── B-roll 弹窗可用的句子时间戳:优先 lipsyncJob.sentence_timings,否则用 ttsPreview.sentenceTimings ── */
const bRollSentenceTimings: SentenceTiming[] | undefined =
(state.lipsyncJob?.sentence_timings as SentenceTiming[] | undefined) ??
(state.ttsPreview.status === "done" ? state.ttsPreview.sentenceTimings : undefined)
/* ── B-roll 可用的总时长:优先 lipsyncJob.output_duration,否则用 ttsPreview.duration ── */
const bRollDuration = state.lipsyncJob?.output_duration || state.ttsPreview.duration || 0
/* ── 生成视频(含实时进度轮询) ── */ /* ── 生成视频(含实时进度轮询) ── */
const handleGenerate = useCallback(async () => { const handleGenerate = useCallback(async () => {
if (!state.lipsyncJob || state.lipsyncJob.status !== "completed") { if (!state.lipsyncJob || state.lipsyncJob.status !== "completed") {
@@ -206,9 +343,28 @@ const AiAvatarPage: React.FC = () => {
} }
state.setIsGenerating(true) state.setIsGenerating(true)
try { try {
const defaultProject = await getOrCreateDefaultProject()
// 用 Canvas 预渲染标题为 PNG dataURL
let titleImageDataUrl: string | null = null
if (state.titleConfig.title?.trim()) {
try {
const res = await getVideoResolution(state.lipsyncJob.output_video_url || "")
titleImageDataUrl = renderTitleToPngDataUrl({
titleConfig: state.titleConfig,
videoWidth: res.width,
videoHeight: res.height,
})
} catch (canvasErr) {
console.warn("[渲染] 标题 Canvas 渲染失败,降级 drawtext:", canvasErr)
titleImageDataUrl = null
}
}
const job = await submitRender({ const job = await submitRender({
lipsync_job_id: state.lipsyncJob.id, lipsync_job_id: state.lipsyncJob.id,
script_id: state.script?.id, script_id: state.script?.id,
project_id: defaultProject.id,
b_roll_segments: state.bRollSegments.map((seg) => ({ b_roll_segments: state.bRollSegments.map((seg) => ({
script_segment_index: seg.script_segment_index, script_segment_index: seg.script_segment_index,
asset_url: seg.asset.file_url || "", asset_url: seg.asset.file_url || "",
@@ -218,25 +374,38 @@ const AiAvatarPage: React.FC = () => {
pip_position: seg.pip_position, pip_position: seg.pip_position,
pip_scale: seg.pip_scale, pip_scale: seg.pip_scale,
})) as never, })) as never,
title_config: buildTitleConfigPayload(state.titleConfig), title_config: buildTitleConfigPayload(state.titleConfig, titleImageDataUrl),
cover_config: buildCoverConfigPayload(state.coverConfig, state.coverConfig.smart_cover_url), cover_config:
state.coverConfig.smart_cover_url ||
(state.coverConfig.upload_url && !state.coverConfig.upload_url.startsWith("blob:"))
? buildCoverConfigPayload(state.coverConfig, state.coverConfig.smart_cover_url)
: {},
}) })
// 打开渲染进度弹窗,启动轮询
setShowRenderModal(true) setShowRenderModal(true)
setRenderStatus("generating") setRenderStatus("generating")
setRenderProgress(job.progress ?? 0) setRenderProgress(job.progress ?? 0)
setRenderErrorMessage("") setRenderErrorMessage("")
setCurrentRenderJob(job as RenderJob)
if (renderTimerRef.current) clearInterval(renderTimerRef.current) if (renderTimerRef.current) clearInterval(renderTimerRef.current)
renderTimerRef.current = setInterval(async () => { renderTimerRef.current = setInterval(async () => {
try { try {
const updated = await getRenderJob(job.id) const updated = await getRenderJob(job.id)
setRenderProgress(updated.progress ?? 0) setRenderProgress(updated.progress ?? 0)
setCurrentRenderJob(updated)
if (updated.status === "completed") { if (updated.status === "completed") {
if (renderTimerRef.current) clearInterval(renderTimerRef.current) if (renderTimerRef.current) clearInterval(renderTimerRef.current)
renderTimerRef.current = null renderTimerRef.current = null
setRenderStatus("completed") setRenderStatus("completed")
if (updated.output_cover_url) {
state.setCoverConfig((prev) => ({
...prev,
mode: "auto_frame",
smart_cover_url: updated.output_cover_url,
thumbnail_url: updated.output_cover_url,
}))
}
message.success("视频已生成并保存到成片库") message.success("视频已生成并保存到成片库")
} else if (updated.status === "failed") { } else if (updated.status === "failed") {
if (renderTimerRef.current) clearInterval(renderTimerRef.current) if (renderTimerRef.current) clearInterval(renderTimerRef.current)
@@ -269,38 +438,47 @@ const AiAvatarPage: React.FC = () => {
setRenderErrorMessage("") setRenderErrorMessage("")
}, []) }, [])
/* ── 智能封面:调后端 MediaKit 选帧接口(#1822 ── */ /* ── 智能封面 ── */
const handleSmartCover = useCallback(async () => { const handleGenerateRenderSmartCover = useCallback(
// 基于对口型成片抽帧,必须先完成对口型 async (renderId: string): Promise<{ cover_url: string; message?: string }> => {
const videoUrl = state.lipsyncJob?.output_video_url try {
if (state.lipsyncJob?.status !== "completed" || !videoUrl) { const res = await generateRenderSmartCover(renderId)
message.warning("请先生成对口型视频,完成后再智能获取封面") if (res.cover_url) {
return state.setCoverConfig((prev) => ({
} ...prev,
setSmartCoverLoading(true) mode: "auto_frame",
try { smart_cover_url: res.cover_url,
const res = await generateSmartCover(videoUrl, 5) thumbnail_url: res.cover_url,
if (res.cover_url) { }))
state.setCoverConfig((prev) => ({ message.success("智能封面已生成")
...prev, return { cover_url: res.cover_url }
mode: "auto_frame", }
smart_cover_url: res.cover_url, const errMsg = res.message || "智能封面生成失败,请稍后重试"
thumbnail_url: res.cover_url, message.error(errMsg)
})) return { cover_url: "", message: errMsg }
message.success("智能封面已生成") } catch (err) {
} else { console.error("智能封面生成失败:", err)
message.error(res.message || "智能封面生成失败,请稍后重试") const errMsg = err instanceof Error ? err.message : "智能封面生成失败,请重试"
message.error(errMsg)
return { cover_url: "", message: errMsg }
} }
} catch (err) { },
console.error("智能封面生成失败:", err)
message.error(err instanceof Error ? err.message : "智能封面生成失败,请重试")
} finally {
setSmartCoverLoading(false)
}
// eslint-disable-next-line react-hooks/exhaustive-deps // eslint-disable-next-line react-hooks/exhaustive-deps
}, [state.lipsyncJob]) [],
)
/* ── 配置汇总 ── */ /* ── 配置汇总 ── */
const coverStatus: "not_ready" | "pending" | "selected" = (() => {
if (
state.coverConfig.smart_cover_url ||
state.coverConfig.thumbnail_url ||
(state.coverConfig.upload_url && !state.coverConfig.upload_url.startsWith("blob:"))
) {
return "selected"
}
if (currentRenderJob?.status === "completed") return "pending"
return "not_ready"
})()
const summary = { const summary = {
videoName: state.selectedVideo?.name || null, videoName: state.selectedVideo?.name || null,
voiceName: state.selectedVoice?.name || null, voiceName: state.selectedVoice?.name || null,
@@ -308,7 +486,7 @@ const AiAvatarPage: React.FC = () => {
lipsyncStatus: state.lipsyncJob?.status || null, lipsyncStatus: state.lipsyncJob?.status || null,
brollCount: state.bRollSegments.length, brollCount: state.bRollSegments.length,
hasTitle: state.titleConfig.title.length > 0, hasTitle: state.titleConfig.title.length > 0,
hasCover: state.coverConfig.enabled, coverStatus,
} }
return ( return (
@@ -342,7 +520,6 @@ const AiAvatarPage: React.FC = () => {
selectedVideo={state.selectedVideo} selectedVideo={state.selectedVideo}
onSelectVideo={() => state.setShowAssetPicker(true)} onSelectVideo={() => state.setShowAssetPicker(true)}
onRemoveVideo={state.removeVideo} onRemoveVideo={state.removeVideo}
titleConfig={state.titleConfig}
/> />
</div> </div>
</div> </div>
@@ -382,19 +559,33 @@ const AiAvatarPage: React.FC = () => {
onOpenScriptModal={() => state.setShowScriptModal(true)} onOpenScriptModal={() => state.setShowScriptModal(true)}
/> />
<div className="aa-step-btn-row"> <div className="aa-step-btn-row">
<button type="button" className="aa-btn aa-btn--primary" onClick={handleNextStep}> <button
type="button"
className="aa-btn aa-btn--primary"
onClick={handleGenerateTts}
disabled={state.ttsPreview.status === "generating"}
>
{state.ttsPreview.status === "done" ? "🎵 重新生成配音" : "🎵 生成配音"}
</button> </button>
{state.ttsPreview.status === "done" && (
<button
type="button"
className="aa-btn aa-btn--primary"
onClick={() => setCurrentStep(2)}
style={{ marginLeft: 12 }}
>
</button>
)}
</div> </div>
</div> </div>
</div> </div>
</> </>
)} )}
{/* ════ 步骤 2:对口型预览(含插入画面)/ 标题配置 / 封面&生成 ════ */} {/* ════ 步骤 2:对口型预览 / 标题配置 / 封面&生成 ════ */}
{currentStep === 2 && ( {currentStep === 2 && (
<> <>
{/* 面板:对口型预览 + 插入画面 */}
<div className={`aa-panel aa-panel--s2-wide${collapsed.lipsync ? " collapsed" : ""}`}> <div className={`aa-panel aa-panel--s2-wide${collapsed.lipsync ? " collapsed" : ""}`}>
<div className="aa-panel__header" onClick={() => togglePanel("lipsync")}> <div className="aa-panel__header" onClick={() => togglePanel("lipsync")}>
<span className="aa-panel__title"></span> <span className="aa-panel__title"></span>
@@ -444,9 +635,8 @@ const AiAvatarPage: React.FC = () => {
onCoverConfigChange={(partial) => onCoverConfigChange={(partial) =>
state.setCoverConfig((prev) => ({ ...prev, ...partial })) state.setCoverConfig((prev) => ({ ...prev, ...partial }))
} }
onSmartCover={handleSmartCover} renderJob={currentRenderJob}
smartCoverLoading={smartCoverLoading} onGenerateRenderSmartCover={handleGenerateRenderSmartCover}
canSmartCover={state.lipsyncJob?.status === "completed"}
resolution={state.resolution} resolution={state.resolution}
onResolutionChange={state.setResolution} onResolutionChange={state.setResolution}
isGenerating={state.isGenerating} isGenerating={state.isGenerating}
@@ -478,19 +668,144 @@ const AiAvatarPage: React.FC = () => {
/> />
)} )}
{/* B-roll 编辑器弹窗 */} {/* B-roll 编辑器弹窗 — #1845:timings 在对口型完成前就可用(来自 TTS 预合成) */}
{state.showBRollModal && ( {state.showBRollModal && (
<ModalBRollEditor <ModalBRollEditor
open={state.showBRollModal} open={state.showBRollModal}
onClose={() => state.setShowBRollModal(false)} onClose={() => state.setShowBRollModal(false)}
existingSegments={state.bRollSegments} existingSegments={state.bRollSegments}
scriptText={state.scriptText} scriptText={state.lipsyncJob?.script_text || state.scriptText}
outputDuration={state.lipsyncJob?.output_duration ?? 0} outputDuration={bRollDuration}
sentenceTimings={bRollSentenceTimings}
onConfirm={state.addBRollSegment} onConfirm={state.addBRollSegment}
onRemove={state.removeBRollSegment} onRemove={state.removeBRollSegment}
/> />
)} )}
{/* #1845 TTS 预合成弹窗 */}
{showTtsModal && (
<div className="aa-modal-overlay">
<div className="aa-modal" onClick={(e) => e.stopPropagation()}>
<div className="aa-modal__header">
<span className="aa-modal__title"></span>
{state.ttsPreview.status !== "generating" && (
<button className="aa-modal__close" onClick={handleCancelTts}>
</button>
)}
</div>
<div
className="aa-modal__body"
style={{
display: "flex",
flexDirection: "column",
alignItems: "center",
padding: "40px 20px",
}}
>
{state.ttsPreview.status === "generating" && (
<>
<div className="aa-lipsync-spinner" />
<div style={{ marginTop: 20, fontSize: 15, color: "#1a1a2e" }}>
</div>
<div
style={{
marginTop: 20,
fontSize: 32,
fontWeight: 700,
color: "#1890ff",
}}
>
{ttsProgress}%
</div>
<div
style={{
marginTop: 12,
width: "80%",
height: 8,
backgroundColor: "#f0f0f0",
borderRadius: 4,
overflow: "hidden",
}}
>
<div
style={{
width: `${ttsProgress}%`,
height: "100%",
backgroundColor: "#1890ff",
borderRadius: 4,
transition: "width 0.3s ease",
}}
/>
</div>
<div style={{ marginTop: 12, fontSize: 13, color: "#8c8ca1" }}>
</div>
</>
)}
{state.ttsPreview.status === "done" && (
<>
<div style={{ fontSize: 48 }}></div>
<div style={{ marginTop: 16, fontSize: 15, color: "#1a1a2e" }}>
</div>
<div style={{ marginTop: 8, fontSize: 13, color: "#8c8ca1" }}>
{state.ttsPreview.duration.toFixed(1)}s{" "}
{state.ttsPreview.sentenceTimings.length}
</div>
</>
)}
{state.ttsPreview.status === "failed" && (
<>
<div style={{ fontSize: 48 }}></div>
<div style={{ marginTop: 16, fontSize: 15, color: "#1a1a2e" }}></div>
{ttsErrorMessage && (
<div
style={{
marginTop: 8,
fontSize: 13,
color: "#ff4d4f",
textAlign: "center",
padding: "0 20px",
}}
>
{ttsErrorMessage}
</div>
)}
</>
)}
</div>
<div className="aa-modal__footer">
{state.ttsPreview.status === "generating" && (
<button className="aa-btn aa-btn--danger" onClick={handleCancelTts}>
</button>
)}
{state.ttsPreview.status === "done" && (
<button className="aa-btn aa-btn--primary" onClick={handleTtsNext}>
</button>
)}
{state.ttsPreview.status === "failed" && (
<>
<button className="aa-btn" onClick={handleCancelTts}>
</button>
<button
className="aa-btn aa-btn--primary"
onClick={handleRetryTts}
style={{ marginLeft: 12 }}
>
</button>
</>
)}
</div>
</div>
</div>
)}
{/* 对口型生成弹窗 */} {/* 对口型生成弹窗 */}
{showLipsyncModal && ( {showLipsyncModal && (
<div className="aa-modal-overlay"> <div className="aa-modal-overlay">
+53 -22
View File
@@ -2,7 +2,7 @@
* AI数字人 — API 封装(#1822 契约对齐) * AI数字人 — API 封装(#1822 契约对齐)
*/ */
import apiClient from "@/api/client" import apiClient from "@/api/client"
import type { Script, LipsyncJob, RenderJob, BRollSegment } from "../types" import type { Script, LipsyncJob, RenderJob, BRollSegment, SentenceTiming } from "../types"
/* ── 文案库 ── */ /* ── 文案库 ── */
export const getScripts = async (): Promise<Script[]> => { export const getScripts = async (): Promise<Script[]> => {
@@ -34,17 +34,28 @@ export const getAssetById = async (id: string): Promise<{ file_url?: string; id:
return response.data return response.data
} }
/* ── 对口型(模式A:TTS 直生,后端内部合成音频;不要先调 TTS 拿 audio_url ── */ /* ── 对口型(支持三种模式) ──
* 1. TTS 直生(降级/旧版):传 voice_id + script_text+speed/emotion),后端 Celery 异步合成
* 2. 直接音频:传 video_url + audio_url,后端同步下载+算timings+提交MediaKit
* 3. 预合成音频(#1845 新主路径):先调 previewTts 拿 audio_url+sentence_timings
* 再把 audio_url + audio_duration + sentence_timings 一起传过来,后端直接提交 MediaKit
*/
export const createLipsyncJob = async (data: { export const createLipsyncJob = async (data: {
/** 人物视频 URLMP4);由素材 id 经 getAssetById 拿 file_url,禁止传 video_asset_id */ /** 人物视频 URLMP4);由素材 id 经 getAssetById 拿 file_url */
video_url: string video_url: string
/** 音色 ID(预置音色 或 克隆音色 profile UUID,后端会解析 */ /** 预合成/直接音频模式:音频 URL(#1845 步骤1 预合成的 CosyVoice 临时 URL,或外部音频 URL */
voice_id: string audio_url?: string
/** 合成的文案(手动输入或文案库内容) */ /** 合成音频时长(秒),由 previewTts 返回 */
script_text: string audio_duration?: number
/** 语速 0.5~2.0,默认 1.0 */ /** 预合成接口返回的句子时间戳(精确),后端直接写入 job */
sentence_timings?: SentenceTiming[]
/** 音色 IDTTS 直生模式用) */
voice_id?: string
/** 要合成的文案(TTS 直生模式用) */
script_text?: string
/** 语速 0.5~2.0,默认 1.0TTS 直生模式用) */
speed?: number speed?: number
/** 情绪英文枚举:natural/excited/calm/friendly */ /** 情绪英文枚举:natural/excited/calm/friendlyTTS 直生模式用) */
emotion?: string emotion?: string
enable_video_loop?: boolean enable_video_loop?: boolean
project_id?: string project_id?: string
@@ -53,21 +64,27 @@ export const createLipsyncJob = async (data: {
return response.data return response.data
} }
export const getLipsyncJob = async (id: string): Promise<LipsyncJob> => { /* ── #1845 TTS 预合成(步骤1「生成配音」同步接口,~2-3s) ── */
const response = await apiClient.get<LipsyncJob>(`/lipsync/jobs/${id}`, { timeout: 60000 }) export const previewTts = async (data: {
voice_id: string
script_text: string
speed?: number
emotion?: string
}): Promise<{
audio_url: string
duration: number
sentence_timings: SentenceTiming[]
}> => {
const response = await apiClient.post<{
audio_url: string
duration: number
sentence_timings: SentenceTiming[]
}>("/lipsync/tts-preview", data, { timeout: 30000 })
return response.data return response.data
} }
/* ── 智能封面(MediaKit 抽帧 + 质量评分选最佳帧,独立于渲染任务) ── */ export const getLipsyncJob = async (id: string): Promise<LipsyncJob> => {
export const generateSmartCover = async ( const response = await apiClient.get<LipsyncJob>(`/lipsync/jobs/${id}`, { timeout: 60000 })
video_url: string,
max_frames = 5,
): Promise<{ cover_url: string; status: string; message: string }> => {
const response = await apiClient.post<{ cover_url: string; status: string; message: string }>(
"/ai-avatar/render/smart-cover",
{ video_url, max_frames },
{ timeout: 60000 },
)
return response.data return response.data
} }
@@ -80,15 +97,29 @@ export const submitRender = async (data: {
cover_config?: Record<string, unknown> cover_config?: Record<string, unknown>
project_id?: string project_id?: string
}): Promise<RenderJob> => { }): Promise<RenderJob> => {
// title_config 内可含 title_image_dataurl(前端 Canvas 渲染的 PNG dataURL
const response = await apiClient.post<RenderJob>("/ai-avatar/render", data) const response = await apiClient.post<RenderJob>("/ai-avatar/render", data)
return response.data return response.data
} }
export const getRenderJob = async (jobId: string): Promise<RenderJob> => { export const getRenderJob = async (jobId: string): Promise<RenderJob> => {
const response = await apiClient.get<RenderJob>(`/ai-avatar/render/${jobId}`) const response = await apiClient.get<RenderJob>(`/ai-avatar/render/${jobId}`, { timeout: 60000 })
return response.data return response.data
} }
export const cancelRenderJob = async (jobId: string): Promise<void> => { export const cancelRenderJob = async (jobId: string): Promise<void> => {
await apiClient.post(`/ai-avatar/render/${jobId}/cancel`) await apiClient.post(`/ai-avatar/render/${jobId}/cancel`)
} }
/* ── 从最终渲染成片智能抽封面(POST /ai-avatar/renders/{job_id}/smart-cover ── */
export const generateRenderSmartCover = async (
jobId: string,
): Promise<{ cover_url: string; status: string; message: string }> => {
const response = await apiClient.post<{ cover_url: string; status: string; message: string }>(
`/ai-avatar/render/${jobId}/smart-cover`,
{},
// 抽帧+评分+转存 OSS 链路较长,120s 超时
{ timeout: 120000 },
)
return response.data
}
@@ -5,12 +5,12 @@
* - 左侧:先选素材库(video 库)→ 再选该库视频素材(已被其他 segment 使用的素材 * - 左侧:先选素材库(video 库)→ 再选该库视频素材(已被其他 segment 使用的素材
* 标灰 + "已选择" 遮罩,pointer-events:none 防重复选择) * 标灰 + "已选择" 遮罩,pointer-events:none 防重复选择)
* - 右侧:文案句子列表(点选对应段落,替代原数字索引框)/ 全屏 or 画中画 / 四角位置+大小 * - 右侧:文案句子列表(点选对应段落,替代原数字索引框)/ 全屏 or 画中画 / 四角位置+大小
* (开始/结束时间已删除,按句子字数占比 × 口播总时长自动估算 * (开始/结束时间来自后端精确句子时间戳,基于 TTS 音频静音检测
* - 底部:已配置的画面插入列表(可删除) * - 底部:已配置的画面插入列表(可删除)
*/ */
import React, { useEffect, useMemo, useState } from "react" import React, { useEffect, useMemo, useState } from "react"
import { getAssets, getAssetLibraries, type AssetItem, type AssetLibraryItem } from "@/api/assets" import { getAssets, getAssetLibraries, type AssetItem, type AssetLibraryItem } from "@/api/assets"
import type { BRollSegment, BRollInsertMode, PipPosition } from "../types" import type { BRollSegment, BRollInsertMode, PipPosition, SentenceTiming } from "../types"
import { splitScriptIntoSentences, type ScriptSentence } from "../utils/sentences" import { splitScriptIntoSentences, type ScriptSentence } from "../utils/sentences"
interface ModalBRollEditorProps { interface ModalBRollEditorProps {
@@ -18,10 +18,12 @@ interface ModalBRollEditorProps {
onClose: () => void onClose: () => void
/** 当前已有的 B-roll segments(用于标灰已选素材) */ /** 当前已有的 B-roll segments(用于标灰已选素材) */
existingSegments: BRollSegment[] existingSegments: BRollSegment[]
/** 当前文案全文(用于分句 */ /** 文案全文(优先使用对口型时锁定的 scriptText */
scriptText: string scriptText: string
/** 对口型成片总时长(秒),用于时间自动估算 */ /** 对口型成片总时长(秒) */
outputDuration: number outputDuration: number
/** 后端精确句子时间戳(来自 lipsyncJob.sentence_timings */
sentenceTimings?: SentenceTiming[] | null
onConfirm: (segment: BRollSegment) => void onConfirm: (segment: BRollSegment) => void
onRemove: (id: string) => void onRemove: (id: string) => void
} }
@@ -43,7 +45,8 @@ const ModalBRollEditor: React.FC<ModalBRollEditorProps> = ({
onClose, onClose,
existingSegments, existingSegments,
scriptText, scriptText,
outputDuration, outputDuration: _outputDuration,
sentenceTimings,
onConfirm, onConfirm,
onRemove, onRemove,
}) => { }) => {
@@ -62,10 +65,10 @@ const ModalBRollEditor: React.FC<ModalBRollEditorProps> = ({
const [pipPosition, setPipPosition] = useState<PipPosition>("top-right") const [pipPosition, setPipPosition] = useState<PipPosition>("top-right")
const [pipScale, setPipScale] = useState(0.3) const [pipScale, setPipScale] = useState(0.3)
/** 文案分句( */ /** 文案分句(优先使用后端精确时间戳,降级为字数比例估算 */
const sentences = useMemo( const sentences = useMemo(
() => splitScriptIntoSentences(scriptText, outputDuration), () => splitScriptIntoSentences(scriptText, sentenceTimings, _outputDuration),
[scriptText, outputDuration], [scriptText, sentenceTimings, _outputDuration],
) )
/** 已被现有 segments 占用的素材 id 集合(标灰、禁止重复选择) */ /** 已被现有 segments 占用的素材 id 集合(标灰、禁止重复选择) */
@@ -142,7 +145,7 @@ const ModalBRollEditor: React.FC<ModalBRollEditorProps> = ({
setSelectedAsset(asset) setSelectedAsset(asset)
} }
/** 确认添加一段 B-roll(⑥ 时间取所选句子的估算起止 */ /** 确认添加一段 B-roll(⑥ 时间取所选句子的精确起止,后端静音检测 / 前端字数比例降级 */
const handleConfirm = () => { const handleConfirm = () => {
if (!selectedAsset || !selectedSentence) return if (!selectedAsset || !selectedSentence) return
const startTime = selectedSentence.startTime const startTime = selectedSentence.startTime
@@ -264,11 +267,9 @@ const ModalBRollEditor: React.FC<ModalBRollEditorProps> = ({
> >
<span className="aa-sentence-item__idx">{sent.index + 1}</span> <span className="aa-sentence-item__idx">{sent.index + 1}</span>
<span className="aa-sentence-item__text">{sent.text}</span> <span className="aa-sentence-item__text">{sent.text}</span>
{outputDuration > 0 && ( <span className="aa-sentence-item__time">
<span className="aa-sentence-item__time"> {sent.startTime.toFixed(1)}-{sent.endTime.toFixed(1)}s
{sent.startTime.toFixed(1)}-{sent.endTime.toFixed(1)}s </span>
</span>
)}
</button> </button>
) )
})} })}
@@ -349,7 +350,7 @@ const ModalBRollEditor: React.FC<ModalBRollEditorProps> = ({
selectedSentence.endTime, selectedSentence.endTime,
selectedSentence.startTime + 0.5, selectedSentence.startTime + 0.5,
).toFixed(1)} ).toFixed(1)}
s s
</div> </div>
</> </>
) : ( ) : (
@@ -1,14 +1,15 @@
/** /**
* AI数字人 — 面板5:封面 & 生成 * AI数字人 — 面板5分辨率/配置摘要/生成按钮/封面
* - 竖屏 9:16 封面预览(从视频截取 / 自定义上传) * v3 调整(步骤③④):
* - 分辨率选择(720p / 1080p / 4K * - 布局顺序:分辨率 → 配置摘要卡片 → 🔘「开始生成视频」按钮 → (渲染完成后)封面区域
* - 配置汇总卡片(出镜视频/音色/文案/对口型/B-roll/标题/封面) * - 渲染未完成时封面区域显示占位态,按钮 disabled
* - 渐变紫色生成按钮 * - 「智能获取封面」从最终成片抽帧(调用 POST /renders/{id}/smart-cover),不再依赖 lipsync 状态
* - 修复点 2 次 bug:内部维护 smartCoverLoading,不依赖外层异步 state 更新
* *
* 注意:v3 已删除"画面插入模式",本面板不包含该选项。 * 注意:v3 已删除"画面插入模式",本面板不包含该选项。
*/ */
import React, { useRef } from "react" import React, { useRef, useState } from "react"
import type { AiAvatarCoverConfig } from "../types" import type { AiAvatarCoverConfig, RenderJob } from "../types"
interface PanelCoverAndGenerateProps { interface PanelCoverAndGenerateProps {
coverConfig: AiAvatarCoverConfig coverConfig: AiAvatarCoverConfig
@@ -17,10 +18,12 @@ interface PanelCoverAndGenerateProps {
onResolutionChange: (r: string) => void onResolutionChange: (r: string) => void
isGenerating: boolean isGenerating: boolean
onGenerate: () => void onGenerate: () => void
/** 智能获取封面(MediaKit 选帧 */ /** 当前渲染任务(渲染完成后才有 output_video_url,才能抽封面 */
onSmartCover: () => void renderJob: RenderJob | null
smartCoverLoading: boolean /** 从最终成片智能抽帧(参数 renderId),返回 { cover_url } */
canSmartCover: boolean onGenerateRenderSmartCover: (renderId: string) => Promise<{ cover_url: string; message?: string }>
/** 自定义上传封面(选择本地文件后由父组件处理实际上传) */
onUploadCover?: (file: File) => void
/** 配置汇总信息 */ /** 配置汇总信息 */
summary: { summary: {
videoName: string | null videoName: string | null
@@ -29,7 +32,8 @@ interface PanelCoverAndGenerateProps {
lipsyncStatus: string | null lipsyncStatus: string | null
brollCount: number brollCount: number
hasTitle: boolean hasTitle: boolean
hasCover: boolean /** 封面状态:'not_ready'(视频未生成) / 'pending'(视频生成了但未选) / 'selected'(已选) */
coverStatus: "not_ready" | "pending" | "selected"
} }
} }
@@ -54,12 +58,14 @@ const PanelCoverAndGenerate: React.FC<PanelCoverAndGenerateProps> = ({
onResolutionChange, onResolutionChange,
isGenerating, isGenerating,
onGenerate, onGenerate,
onSmartCover, renderJob,
smartCoverLoading, onGenerateRenderSmartCover,
canSmartCover, onUploadCover,
summary, summary,
}) => { }) => {
const uploadInputRef = useRef<HTMLInputElement>(null) const uploadInputRef = useRef<HTMLInputElement>(null)
// 内部维护智能封面加载态(修复点 2 次 bug:不依赖外层异步 setState 顺序)
const [smartCoverLoading, setSmartCoverLoading] = useState(false)
/** 自定义上传封面 */ /** 自定义上传封面 */
const handleUploadClick = () => { const handleUploadClick = () => {
@@ -69,60 +75,66 @@ const PanelCoverAndGenerate: React.FC<PanelCoverAndGenerateProps> = ({
const handleFileChange = (e: React.ChangeEvent<HTMLInputElement>) => { const handleFileChange = (e: React.ChangeEvent<HTMLInputElement>) => {
const file = e.target.files?.[0] const file = e.target.files?.[0]
if (!file) return if (!file) return
// 本地预览:生成 object URL(实际上传由父级/后端链路处理) if (onUploadCover) {
const url = URL.createObjectURL(file) onUploadCover(file)
onCoverConfigChange({ mode: "upload", upload_url: url, thumbnail_url: url }) } else {
// 允许重复选择同一文件 // 本地预览兜底(实际上传由父级处理;blob URL 仅作本地展示)
const url = URL.createObjectURL(file)
onCoverConfigChange({ mode: "upload", upload_url: url, thumbnail_url: url })
}
e.target.value = "" e.target.value = ""
} }
/** 智能获取封面(调后端 MediaKit 抽帧评分选最佳帧,#1822 */ /** 智能获取封面(从最终成片抽帧;必须等 render 完成 */
const handleSmartCover = () => { const handleSmartCover = async () => {
onCoverConfigChange({ mode: "auto_frame" }) if (!renderJob || renderJob.status !== "completed" || !renderJob.id) return
onSmartCover() setSmartCoverLoading(true)
try {
const res = await onGenerateRenderSmartCover(renderJob.id)
if (res.cover_url) {
onCoverConfigChange({
mode: "auto_frame",
smart_cover_url: res.cover_url,
thumbnail_url: res.cover_url,
})
} else {
// 失败由父组件 message 提示,这里不重复弹窗
console.warn("[智能封面] 返回空 cover_url:", res.message)
}
} catch (err) {
console.error("[智能封面] 调用失败:", err)
} finally {
setSmartCoverLoading(false)
}
} }
const lipsync = summary.lipsyncStatus ? LIPSYNC_STATUS_LABEL[summary.lipsyncStatus] : null const lipsync = summary.lipsyncStatus ? LIPSYNC_STATUS_LABEL[summary.lipsyncStatus] : null
const canGenerate = summary.lipsyncStatus === "completed" && !isGenerating const canGenerate = summary.lipsyncStatus === "completed" && !isGenerating
// 渲染已完成 → 封面区可用
const isRenderCompleted = renderJob?.status === "completed"
const canSmartCover = isRenderCompleted && !smartCoverLoading
/** 封面图实际展示的 url:智能封面 > 自定义上传 > 空 */
const coverUrl =
coverConfig.smart_cover_url || coverConfig.thumbnail_url || coverConfig.upload_url
const hasCoverImage = Boolean(coverUrl)
/** 封面区占位文字 */
const coverPlaceholder = isRenderCompleted ? "暂无封面" : "视频生成后可选择封面"
/** 封面摘要状态文本 */
const coverSummaryNode = (() => {
if (summary.coverStatus === "selected") {
return <span className="aa-config-summary__value"></span>
}
if (summary.coverStatus === "pending") {
return <span className="aa-config-summary__value"></span>
}
return <span className="aa-config-summary__empty"></span>
})()
return ( return (
<div className="aa-cover-generate"> <div className="aa-cover-generate">
{/* 封面预览(竖屏 9:16 */}
<div className="aa-cover-preview">
{coverConfig.thumbnail_url ? (
<img src={coverConfig.thumbnail_url} alt="封面预览" />
) : (
<span className="aa-cover-preview__placeholder"></span>
)}
</div>
<div className="aa-cover-actions">
<button
type="button"
className={`aa-btn aa-btn--ghost${coverConfig.mode === "auto_frame" ? " active" : ""}`}
onClick={handleSmartCover}
disabled={smartCoverLoading || !canSmartCover}
title={canSmartCover ? "基于对口型成片智能选帧" : "请先完成对口型生成"}
>
{smartCoverLoading ? "⏳ 智能选帧中…" : "🎬 智能获取封面"}
</button>
<button
type="button"
className={`aa-btn aa-btn--ghost${coverConfig.mode === "upload" ? " active" : ""}`}
onClick={handleUploadClick}
>
📷
</button>
<input
ref={uploadInputRef}
type="file"
accept="image/*"
style={{ display: "none" }}
onChange={handleFileChange}
/>
</div>
{/* 分辨率选择 */} {/* 分辨率选择 */}
<div className="aa-form-field"> <div className="aa-form-field">
<label className="aa-label"></label> <label className="aa-label"></label>
@@ -130,6 +142,7 @@ const PanelCoverAndGenerate: React.FC<PanelCoverAndGenerateProps> = ({
className="aa-select" className="aa-select"
value={resolution} value={resolution}
onChange={(e) => onResolutionChange(e.target.value)} onChange={(e) => onResolutionChange(e.target.value)}
disabled={isGenerating}
> >
{RESOLUTION_OPTIONS.map((opt) => ( {RESOLUTION_OPTIONS.map((opt) => (
<option key={opt.value} value={opt.value}> <option key={opt.value} value={opt.value}>
@@ -190,11 +203,7 @@ const PanelCoverAndGenerate: React.FC<PanelCoverAndGenerateProps> = ({
</div> </div>
<div className="aa-config-summary__row"> <div className="aa-config-summary__row">
<span></span> <span></span>
{summary.hasCover ? ( {coverSummaryNode}
<span className="aa-config-summary__value"></span>
) : (
<span className="aa-config-summary__empty"></span>
)}
</div> </div>
</div> </div>
@@ -212,6 +221,55 @@ const PanelCoverAndGenerate: React.FC<PanelCoverAndGenerateProps> = ({
</div> </div>
)} )}
{isGenerating && (
<div style={{ marginTop: 8, fontSize: 11, color: "#8c8ca1", textAlign: "center" }}>
</div>
)}
</div>
{/* 封面区域(视频生成后才激活;步骤③④要求:按钮在封面上方,完成后再显示封面区) */}
<div className="aa-cover-section" style={{ marginTop: 16 }}>
<div className="aa-label" style={{ marginBottom: 8 }}>
</div>
{/* 封面预览(竖屏 9:16)——成片帧已经通过 Canvas PNG overlay 带有标题,直接展示原图即可 */}
<div className="aa-cover-preview" style={{ opacity: isRenderCompleted ? 1 : 0.5 }}>
{hasCoverImage ? (
<img src={coverUrl!} alt="封面预览" draggable={false} />
) : (
<span className="aa-cover-preview__placeholder">{coverPlaceholder}</span>
)}
{smartCoverLoading && <div className="aa-cover-preview__loading"> </div>}
</div>
<div className="aa-cover-actions">
<button
type="button"
className={`aa-btn aa-btn--ghost${coverConfig.mode === "auto_frame" ? " active" : ""}`}
onClick={handleSmartCover}
disabled={!canSmartCover}
title={isRenderCompleted ? "从成片智能选帧" : "请先生成视频"}
>
{smartCoverLoading ? "⏳ 智能选帧中…" : "🎬 智能获取封面"}
</button>
<button
type="button"
className={`aa-btn aa-btn--ghost${coverConfig.mode === "upload" ? " active" : ""}`}
onClick={handleUploadClick}
disabled={!isRenderCompleted || smartCoverLoading}
title={isRenderCompleted ? "自定义上传封面" : "请先生成视频"}
>
📷
</button>
<input
ref={uploadInputRef}
type="file"
accept="image/*"
style={{ display: "none" }}
onChange={handleFileChange}
/>
</div>
</div> </div>
</div> </div>
) )
@@ -1,9 +1,9 @@
/** /**
* AI数字人 — 对口型预览面板(步骤2用) * AI数字人 — 对口型预览面板(步骤2用)
* B-roll 画面插入 + 对口型视频预览 + 生成/重新生成按钮 * B-roll 画面插入 + 对口型视频预览 + 生成/重新生成按钮
* v3.1: 预览容器按 1/2 缩放、标题实时叠加预览 * v3.1: 标题字号按预览容器实际宽度动态计算 previewScale(基准 720p),与成片一致
*/ */
import React, { useRef } from "react" import React, { useCallback, useEffect, useRef, useState } from "react"
import type { LipsyncJob, BRollSegment, AiAvatarTitleConfig } from "../types" import type { LipsyncJob, BRollSegment, AiAvatarTitleConfig } from "../types"
interface PanelLipsyncPreviewProps { interface PanelLipsyncPreviewProps {
@@ -14,8 +14,8 @@ interface PanelLipsyncPreviewProps {
onRemoveBRoll: (id: string) => void onRemoveBRoll: (id: string) => void
/** 标题配置(实时叠加预览用) */ /** 标题配置(实时叠加预览用) */
titleConfig?: AiAvatarTitleConfig titleConfig?: AiAvatarTitleConfig
/** 标题位置变更回调(拖拽结束时调用) */ /** 标题位置变更回调(拖拽结束时调用,发送百分比坐标 + position:"custom" */
onTitlePositionChange?: (pos: { pos_x: number; pos_y: number }) => void onTitlePositionChange?: (pos: { pos_x: number; pos_y: number; position: string }) => void
} }
const BROLL_MODE_LABEL: Record<BRollSegment["mode"], string> = { const BROLL_MODE_LABEL: Record<BRollSegment["mode"], string> = {
@@ -29,6 +29,16 @@ function formatTime(seconds: number): string {
return `${m}:${s.toString().padStart(2, "0")}` return `${m}:${s.toString().padStart(2, "0")}`
} }
/** 字体名 → CSS font-family 映射(与 titleCanvas 字体链对齐) */
const FONT_FAMILY_MAP: Record<string, string> = {
: "'Noto Sans CJK SC', 'Source Han Sans CN', 'PingFang SC', 'Microsoft YaHei', sans-serif",
: "'Noto Serif SC', 'Source Han Serif SC', 'SimSun', serif",
: "KaiTi, 'STKaiti', serif",
: "'Heiti SC', 'SimHei', 'Microsoft YaHei', sans-serif",
}
const getFontFamily = (font: string): string =>
FONT_FAMILY_MAP[font] || FONT_FAMILY_MAP["思源黑体"]
export function PanelLipsyncPreview({ export function PanelLipsyncPreview({
lipsyncJob, lipsyncJob,
onGenerateLipsync, onGenerateLipsync,
@@ -41,6 +51,8 @@ export function PanelLipsyncPreview({
const titleDragRef = useRef<HTMLDivElement>(null) const titleDragRef = useRef<HTMLDivElement>(null)
const draggingTitleRef = useRef(false) const draggingTitleRef = useRef(false)
const previewContainerRef = useRef<HTMLDivElement>(null) const previewContainerRef = useRef<HTMLDivElement>(null)
// 预览容器实际宽度(通过 ResizeObserver 监听),用于动态计算 previewScale
const [containerWidth, setContainerWidth] = useState(0)
const isGenerating = lipsyncJob?.status === "pending" || lipsyncJob?.status === "processing" const isGenerating = lipsyncJob?.status === "pending" || lipsyncJob?.status === "processing"
const isDone = lipsyncJob?.status === "completed" const isDone = lipsyncJob?.status === "completed"
const isFailed = lipsyncJob?.status === "failed" const isFailed = lipsyncJob?.status === "failed"
@@ -52,29 +64,84 @@ export function PanelLipsyncPreview({
? "排队中…" ? "排队中…"
: "对口型生成中…" : "对口型生成中…"
/** 标题叠加样式 */ // 监听预览容器尺寸变化,动态测量宽度以计算 previewScale(基准 720p
const titleOverlayStyle: React.CSSProperties | null = titleConfig?.title useEffect(() => {
? { const el = previewContainerRef.current
position: "absolute", if (!el) return
left: "50%", const update = () => setContainerWidth(el.clientWidth || 0)
transform: "translateX(-50%)", update()
color: titleConfig.color || "#ffffff", if (typeof ResizeObserver !== "undefined") {
fontFamily: titleConfig.font || "思源黑体", const ro = new ResizeObserver(update)
fontSize: `${(titleConfig.size || 36) * 0.55}px`, // 预览等比缩 ro.observe(el)
fontWeight: titleConfig.bold ? 700 : 400, return () => ro.disconnect()
fontStyle: titleConfig.italic ? "italic" : "normal", }
textAlign: "center", window.addEventListener("resize", update)
width: "90%", return () => window.removeEventListener("resize", update)
padding: "4px 8px", }, [])
textShadow: titleConfig.shadow ? "0 2px 4px rgba(0,0,0,0.8)" : undefined,
WebkitTextStroke: titleConfig.stroke ? "1.5px #000" : undefined, // 预览缩放比:预览宽度 / 720(基准宽度)
...(titleConfig.position === "top" const previewScale = containerWidth > 0 ? containerWidth / 720 : 0.35
? { top: 8 } const ps = useCallback((v: number) => Math.round(v * previewScale * 100) / 100, [previewScale])
: titleConfig.position === "bottom"
? { bottom: 8 } /** 标题叠加样式(字号/padding/描边/阴影均按 previewScale 缩放,保持与成片视觉一致) */
: { top: "50%", transform: "translateX(-50%) translateY(-50%)" }), const titleOverlayStyle: React.CSSProperties | null =
} titleConfig?.title && containerWidth > 0
: null ? (() => {
const baseSize = titleConfig.size || 48
const fontSize = ps(baseSize)
// 描边宽度基准 ≈ size * 0.06,最小 1.5px @720p
const strokeW = Math.max(ps(1.5), +(baseSize * 0.06 * previewScale).toFixed(2))
// 阴影按比例缩放
const shadowBlur = ps(4)
const shadowOffsetY = ps(2)
// padding / top 边距按比例(基准 8px 对应预览小窗,成片基准 16px,这里 8px 对应约 0.33 缩放)
const padV = ps(16) * 0.5 // ≈ 8px in ~240px container
const padH = ps(24) * 0.5
const style: React.CSSProperties = {
position: "absolute",
color: titleConfig.color || "#ffffff",
fontFamily: getFontFamily(titleConfig.font || "思源黑体"),
fontSize: `${fontSize}px`,
fontWeight: titleConfig.bold ? 700 : 400,
fontStyle: titleConfig.italic ? "italic" : "normal",
textAlign: "center",
width: "90%",
lineHeight: 1.2,
padding: `${ps(4)}px ${padH}px`,
textShadow: titleConfig.shadow
? `0 ${shadowOffsetY}px ${shadowBlur}px rgba(0,0,0,0.8), 0 0 ${ps(2)}px rgba(0,0,0,0.5)`
: undefined,
WebkitTextStroke: titleConfig.stroke ? `${strokeW}px #000` : undefined,
boxSizing: "border-box",
wordBreak: "break-word",
whiteSpace: "pre-wrap",
}
if (
titleConfig.position === "custom" &&
titleConfig.pos_x != null &&
titleConfig.pos_y != null
) {
style.left = `${titleConfig.pos_x}%`
style.top = `${titleConfig.pos_y}%`
style.transform = "translateX(-50%) translateY(-50%)"
} else if (titleConfig.position === "top") {
style.left = "50%"
style.top = padV
style.transform = "translateX(-50%)"
} else if (titleConfig.position === "bottom") {
style.left = "50%"
style.bottom = padV
style.transform = "translateX(-50%)"
} else {
style.left = "50%"
style.top = "50%"
style.transform = "translateX(-50%) translateY(-50%)"
}
return style
})()
: null
const handleTitlePointerDown = (e: React.PointerEvent<HTMLDivElement>) => { const handleTitlePointerDown = (e: React.PointerEvent<HTMLDivElement>) => {
if (!onTitlePositionChange || !previewContainerRef.current) return if (!onTitlePositionChange || !previewContainerRef.current) return
@@ -105,7 +172,10 @@ export function PanelLipsyncPreview({
const rect = previewContainerRef.current.getBoundingClientRect() const rect = previewContainerRef.current.getBoundingClientRect()
const relX = Math.max(0, Math.min(rect.width, e.clientX - rect.left)) const relX = Math.max(0, Math.min(rect.width, e.clientX - rect.left))
const relY = Math.max(0, Math.min(rect.height, e.clientY - rect.top)) const relY = Math.max(0, Math.min(rect.height, e.clientY - rect.top))
onTitlePositionChange({ pos_x: relX, pos_y: relY }) // 发送百分比坐标(0-100),与后端 drawtext 百分比表达式对齐
const xpct = Math.round((relX / rect.width) * 1000) / 10
const ypct = Math.round((relY / rect.height) * 1000) / 10
onTitlePositionChange({ pos_x: xpct, pos_y: ypct, position: "custom" })
} }
;(e.currentTarget as HTMLDivElement).style.cursor = "grab" ;(e.currentTarget as HTMLDivElement).style.cursor = "grab"
} }
@@ -174,7 +244,7 @@ export function PanelLipsyncPreview({
)} )}
</div> </div>
{/* ── 对口型预览(v3.1: 缩放1/2 + 标题叠加 ─ */} {/* ── 对口型预览(标题字号按 previewScale 动态缩放 ─ */}
<div className="aa-lipsync-section"> <div className="aa-lipsync-section">
<div className="aa-lipsync-section__title"></div> <div className="aa-lipsync-section__title"></div>
@@ -2,17 +2,16 @@
* AI数字人 — 出镜视频选择面板 * AI数字人 — 出镜视频选择面板
* - 未选视频:虚线上传区,点击打开素材库弹窗 * - 未选视频:虚线上传区,点击打开素材库弹窗
* - 已选视频:竖屏 9:16 预览播放器 + 视频信息卡片 + 移除按钮 * - 已选视频:竖屏 9:16 预览播放器 + 视频信息卡片 + 移除按钮
*
* 注意:本面板只展示原始素材视频,不叠加标题(标题在对口型预览和最终成片上展示)
*/ */
import type { AssetItem } from "@/api/assets" import type { AssetItem } from "@/api/assets"
import type { AiAvatarTitleConfig } from "../types"
import { getFontFamily } from "@/pages/generate/constants"
export interface PanelVideoSelectorProps { export interface PanelVideoSelectorProps {
selectedVideo: AssetItem | null selectedVideo: AssetItem | null
/** 触发打开素材库弹窗 */ /** 触发打开素材库弹窗 */
onSelectVideo: () => void onSelectVideo: () => void
onRemoveVideo: () => void onRemoveVideo: () => void
titleConfig?: AiAvatarTitleConfig
} }
/** 格式化时长(秒 → mm:ss */ /** 格式化时长(秒 → mm:ss */
@@ -27,7 +26,6 @@ export function PanelVideoSelector({
selectedVideo, selectedVideo,
onSelectVideo, onSelectVideo,
onRemoveVideo, onRemoveVideo,
titleConfig,
}: PanelVideoSelectorProps) { }: PanelVideoSelectorProps) {
/* 未选视频:虚线上传区,点击打开素材库弹窗 */ /* 未选视频:虚线上传区,点击打开素材库弹窗 */
if (!selectedVideo) { if (!selectedVideo) {
@@ -57,42 +55,13 @@ export function PanelVideoSelector({
return ( return (
<div> <div>
{/* 竖屏 9:16 视频预览播放器 + 标题实时预览 */} {/* 竖屏 9:16 视频预览播放器(纯素材预览,不叠加标题) */}
<div className="aa-video-preview" style={{ position: "relative" }}> <div className="aa-video-preview">
{fileUrl ? ( {fileUrl ? (
<video src={fileUrl} poster={selectedVideo.thumbnail_url} controls playsInline /> <video src={fileUrl} poster={selectedVideo.thumbnail_url} controls playsInline />
) : ( ) : (
<div className="aa-video-preview__placeholder"></div> <div className="aa-video-preview__placeholder"></div>
)} )}
{titleConfig?.title && (
<div
style={{
position: "absolute",
left: "50%",
transform: "translateX(-50%)",
...(titleConfig.position === "top"
? { top: "10%" }
: titleConfig.position === "bottom"
? { bottom: "10%" }
: { top: "50%", transform: "translate(-50%, -50%)" }),
fontSize: Math.max(titleConfig.size, 32),
fontFamily: getFontFamily(titleConfig.font),
color: titleConfig.color,
fontWeight: titleConfig.bold ? 700 : 400,
fontStyle: titleConfig.italic ? "italic" : "normal",
textShadow: "0 2px 4px rgba(0,0,0,0.5)",
WebkitTextStroke: "2px #000",
pointerEvents: "none",
zIndex: 10,
maxWidth: "90%",
textAlign: "center",
whiteSpace: "pre-wrap",
lineHeight: 1.3,
}}
>
{titleConfig.title}
</div>
)}
</div> </div>
{/* 视频信息卡片:文件名 / 时长 / 分辨率 */} {/* 视频信息卡片:文件名 / 时长 / 分辨率 */}
@@ -1,5 +1,5 @@
/** /**
* AI数字人 — 页面全局状态管理 hook(v3) * AI数字人 — 页面全局状态管理 hook(v3 + #1845 配音前置
*/ */
import { useState, useCallback } from "react" import { useState, useCallback } from "react"
import type { AssetItem } from "@/api/assets" import type { AssetItem } from "@/api/assets"
@@ -13,10 +13,19 @@ import {
type BRollSegment, type BRollSegment,
type AiAvatarTitleConfig, type AiAvatarTitleConfig,
type AiAvatarCoverConfig, type AiAvatarCoverConfig,
type TtsPreviewResult,
DEFAULT_TITLE_CONFIG, DEFAULT_TITLE_CONFIG,
DEFAULT_COVER_CONFIG, DEFAULT_COVER_CONFIG,
} from "../types" } from "../types"
const DEFAULT_TTS_PREVIEW: TtsPreviewResult = {
audioUrl: null,
duration: 0,
sentenceTimings: [],
status: "idle",
error: null,
}
export function useAiAvatar() { export function useAiAvatar() {
/* ── 面板1:出镜视频 ── */ /* ── 面板1:出镜视频 ── */
const [selectedVideo, setSelectedVideo] = useState<AssetItem | null>(null) const [selectedVideo, setSelectedVideo] = useState<AssetItem | null>(null)
@@ -36,6 +45,9 @@ export function useAiAvatar() {
const [showScriptModal, setShowScriptModal] = useState(false) const [showScriptModal, setShowScriptModal] = useState(false)
const [showBRollModal, setShowBRollModal] = useState(false) const [showBRollModal, setShowBRollModal] = useState(false)
/* ── #1845 TTS 预合成(步骤1「生成配音」) ── */
const [ttsPreview, setTtsPreview] = useState<TtsPreviewResult>(DEFAULT_TTS_PREVIEW)
/* ── 面板3.5B-roll ── */ /* ── 面板3.5B-roll ── */
const [bRollSegments, setBRollSegments] = useState<BRollSegment[]>([]) const [bRollSegments, setBRollSegments] = useState<BRollSegment[]>([])
@@ -81,6 +93,7 @@ export function useAiAvatar() {
setScript(null) setScript(null)
setScriptText("") setScriptText("")
setLipsyncJob(null) setLipsyncJob(null)
setTtsPreview(DEFAULT_TTS_PREVIEW)
setBRollSegments([]) setBRollSegments([])
setTitleConfig(DEFAULT_TITLE_CONFIG) setTitleConfig(DEFAULT_TITLE_CONFIG)
setCoverConfig(DEFAULT_COVER_CONFIG) setCoverConfig(DEFAULT_COVER_CONFIG)
@@ -118,6 +131,10 @@ export function useAiAvatar() {
showBRollModal, showBRollModal,
setShowBRollModal, setShowBRollModal,
selectScript, selectScript,
// #1845 TTS 预合成
ttsPreview,
setTtsPreview,
resetTtsPreview: useCallback(() => setTtsPreview(DEFAULT_TTS_PREVIEW), []),
// B-roll // B-roll
bRollSegments, bRollSegments,
addBRollSegment, addBRollSegment,
+26 -3
View File
@@ -28,6 +28,17 @@ export const VOICE_LANGUAGE_OPTIONS: { value: VoiceLanguage; label: string }[] =
/* ── 对口型任务状态 ── */ /* ── 对口型任务状态 ── */
export type LipsyncStatus = "idle" | "pending" | "processing" | "completed" | "failed" export type LipsyncStatus = "idle" | "pending" | "processing" | "completed" | "failed"
/* ── TTS 预合成(#1845 配音前置:步骤1「生成配音」状态) ── */
export type TtsPreviewStatus = "idle" | "generating" | "done" | "failed"
export interface TtsPreviewResult {
audioUrl: string | null
duration: number
sentenceTimings: SentenceTiming[]
status: TtsPreviewStatus
error: string | null
}
/* ── 文案 ── */ /* ── 文案 ── */
export interface Script { export interface Script {
id: string id: string
@@ -44,12 +55,23 @@ export interface LipsyncJob {
status: LipsyncStatus status: LipsyncStatus
progress: number progress: number
output_video_url: string | null output_video_url: string | null
/** 对口型成片总时长(秒),后端返回;用于 B-roll 时间自动估算(#1809 ⑥) */ /** 对口型成片总时长(秒),后端返回 */
script_text: string
output_duration?: number output_duration?: number
/** 精确句子时间戳(后端基于 TTS 音频静音检测计算) */
sentence_timings?: SentenceTiming[] | null
error_message: string | null error_message: string | null
created_at: string created_at: string
} }
/* ── 句子时间戳(后端精确计算) ── */
export interface SentenceTiming {
index: number
text: string
start_time: number
end_time: number
}
/* ── B-roll 画面插入 ── */ /* ── B-roll 画面插入 ── */
export type BRollInsertMode = "fullscreen" | "pip" export type BRollInsertMode = "fullscreen" | "pip"
export type PipPosition = "top-left" | "top-right" | "bottom-left" | "bottom-right" export type PipPosition = "top-left" | "top-right" | "bottom-left" | "bottom-right"
@@ -77,7 +99,7 @@ export interface AiAvatarTitleConfig {
shadow: boolean shadow: boolean
color: string color: string
auto_subtitle: boolean auto_subtitle: boolean
/** 自定义位置坐标(position=custom 时生效,像素 */ /** 自定义位置坐标(position=custom 时生效,百分比 0-100 */
pos_x?: number pos_x?: number
pos_y?: number pos_y?: number
} }
@@ -101,6 +123,7 @@ export interface RenderJob {
status: RenderStatus status: RenderStatus
progress: number progress: number
output_video_url: string | null output_video_url: string | null
output_cover_url: string | null
error_message: string | null error_message: string | null
created_at: string created_at: string
} }
@@ -110,7 +133,7 @@ export const DEFAULT_TITLE_CONFIG: AiAvatarTitleConfig = {
title: "", title: "",
position: "bottom", position: "bottom",
font: "思源黑体", font: "思源黑体",
size: 28, size: 48,
bold: true, bold: true,
italic: false, italic: false,
stroke: false, stroke: false,
+16 -4
View File
@@ -28,10 +28,13 @@ export function normalizeEmotion(raw: string | undefined | null): VoiceEmotion {
* 后端真实字段:text(或content)、font(或font_preset)、font_size(或size)、 * 后端真实字段:text(或content)、font(或font_preset)、font_size(或size)、
* font_color(或color,可传 #RRGGBB)、position(top/center/bottom/custom)、 * font_color(或color,可传 #RRGGBB)、position(top/center/bottom/custom)、
* enabled、bold、stroke{enabled,width,color}、shadow{enabled,color,offset_x,offset_y}、 * enabled、bold、stroke{enabled,width,color}、shadow{enabled,color,offset_x,offset_y}、
* pos_x/pos_y(custom 时)。 * pos_x/pos_y(custom 时)、title_image_dataurl(前端 Canvas 渲染的 PNG dataURLWYSIWYG 路径优先)
* 口播标题默认 position=bottom(不传后端会默认 top 跑到画面顶部)。 * 口播标题默认 position=bottom(不传后端会默认 top 跑到画面顶部)。
*/ */
export function buildTitleConfigPayload(cfg: AiAvatarTitleConfig): Record<string, unknown> { export function buildTitleConfigPayload(
cfg: AiAvatarTitleConfig,
titleImageDataUrl?: string | null,
): Record<string, unknown> {
const text = (cfg.title || "").trim() const text = (cfg.title || "").trim()
if (!text) return {} if (!text) return {}
const position = cfg.position || "bottom" const position = cfg.position || "bottom"
@@ -39,7 +42,7 @@ export function buildTitleConfigPayload(cfg: AiAvatarTitleConfig): Record<string
text, text,
enabled: true, enabled: true,
font: cfg.font || "思源黑体", font: cfg.font || "思源黑体",
font_size: Math.round(cfg.size) || 36, font_size: Math.round(cfg.size) || 48,
font_color: cfg.color || "#ffffff", font_color: cfg.color || "#ffffff",
position, position,
bold: !!cfg.bold, bold: !!cfg.bold,
@@ -53,6 +56,10 @@ export function buildTitleConfigPayload(cfg: AiAvatarTitleConfig): Record<string
payload.pos_x = cfg.pos_x payload.pos_x = cfg.pos_x
payload.pos_y = cfg.pos_y payload.pos_y = cfg.pos_y
} }
// 前端 Canvas 渲染好的 PNG dataURL(所见即所得,后端优先 overlay 此图片图层)
if (titleImageDataUrl) {
payload.title_image_dataurl = titleImageDataUrl
}
return payload return payload
} }
@@ -67,9 +74,14 @@ export function buildCoverConfigPayload(
// build_cover_extract_command 读取 timestamp(截帧秒数) // build_cover_extract_command 读取 timestamp(截帧秒数)
timestamp: cfg.frame_time || 0, timestamp: cfg.frame_time || 0,
} }
if (smartCoverUrl) payload.cover_url = smartCoverUrl // 智能封面 URL(后端字段名为 url/imageUrl/cover_url 都兼容,优先 url
if (smartCoverUrl) {
payload.url = smartCoverUrl
payload.cover_url = smartCoverUrl
}
// 自定义上传:blob: 本地预览地址无法给后端,仅 OSS URL 可用 // 自定义上传:blob: 本地预览地址无法给后端,仅 OSS URL 可用
if (cfg.mode === "upload" && cfg.upload_url && !cfg.upload_url.startsWith("blob:")) { if (cfg.mode === "upload" && cfg.upload_url && !cfg.upload_url.startsWith("blob:")) {
payload.url = cfg.upload_url
payload.upload_url = cfg.upload_url payload.upload_url = cfg.upload_url
} }
return payload return payload
@@ -1,5 +1,10 @@
/** /**
* AI数字人 — 文案分句 & B-roll 时间自动估算(#1809 ⑤⑥) * AI数字人 — 文案分句 & B-roll 时间计算
*
* 数据来源优先级:
* 1. 后端 sentence_timings(基于 TTS 音频静音检测,精确到句子边界)—— 直接使用,不重新分句
* 2. 后端 output_duration(最终渲染视频时长) + 本地分句 —— 按字数比例估算
* 3. 两者都没有(对口型还在生成中)—— 返回分句文本但 startTime/endTime 全部 0,等数据到位重算
*/ */
export interface ScriptSentence { export interface ScriptSentence {
@@ -11,25 +16,67 @@ export interface ScriptSentence {
charCount: number charCount: number
/** 累计起始字数(用于时间估算) */ /** 累计起始字数(用于时间估算) */
startChar: number startChar: number
/** 估算的对口型视频内起始时间(秒) */ /** 对口型视频内起始时间(秒)——后端精确值或前端估算 */
startTime: number startTime: number
/** 估算的对口型视频内结束时间(秒) */ /** 对口型视频内结束时间(秒)——后端精确值或前端估算 */
endTime: number endTime: number
} }
/** 句子分隔符:中英文句号/问号/感叹号/分号/逗号/换行(覆盖中文短视频常用断句) */
const SENTENCE_SPLIT_RE = /[。!?!??!;,\n\r]+/
/** /**
* 按句号/问号/感叹号/分号/换行分句(兼容中英文标点) * 分句并计算每句的起止时间
* 空文案返回空数组。时间按「该句字数 ÷ 全文总字数 × 口播总时长」线性估算。 *
* @param sentenceTimings 后端返回的精确句子时间戳(来自 lipsync_job.sentence_timings)。
* 非空时直接按后端返回的句子列表渲染,不再本地分句(避免前后端分句不一致导致时间错位)。
* @param outputDuration 最终视频时长(秒)。对口型预览阶段可能为 0,此时降级估算只能给 0。
*/ */
export function splitScriptIntoSentences( export function splitScriptIntoSentences(
scriptText: string, scriptText: string,
outputDuration: number, sentenceTimings?:
{ index?: number; text?: string; start_time: number; end_time: number }[] | null,
outputDuration: number = 0,
): ScriptSentence[] { ): ScriptSentence[] {
const text = (scriptText || "").trim() const text = (scriptText || "").trim()
if (!text) return [] if (!text) return []
// 1. 后端返回了 sentence_timings:校验通过就直接用,跳过本地分句
// 校验条件放宽:只要是数组、至少1条、每条 start_time/end_time 是数字即可
// (不再强制要求条数相等——后端静音检测可能按停顿切出更多/更少边界,
// 比如文案用逗号连写时本地只分1句、后端按停顿切4句,后端的切法才是对的)
if (Array.isArray(sentenceTimings) && sentenceTimings.length > 0) {
const valid = sentenceTimings.every(
(t) =>
t &&
typeof t.start_time === "number" &&
typeof t.end_time === "number" &&
isFinite(t.start_time) &&
isFinite(t.end_time) &&
t.end_time >= t.start_time,
)
if (valid) {
let accChar = 0
return sentenceTimings.map((t, i) => {
const sentenceText = (t.text || "").trim() || `句子${i + 1}`
const charCount = sentenceText.replace(/\s/g, "").length
const sentence: ScriptSentence = {
index: typeof t.index === "number" ? t.index : i,
text: sentenceText,
charCount,
startChar: accChar,
startTime: round1(t.start_time),
endTime: round1(t.end_time),
}
accChar += charCount
return sentence
})
}
}
// 2. 本地分句 + 按字数比例估算(降级路径)
const rawParts = text const rawParts = text
.split(/[。!?!?;\n\r]+/) .split(SENTENCE_SPLIT_RE)
.map((part) => part.trim()) .map((part) => part.trim())
.filter((part) => part.length > 0) .filter((part) => part.length > 0)
@@ -0,0 +1,179 @@
/**
* AI数字人 — 标题 Canvas 渲染工具
*
* 把标题按前端预览的 HTML/CSS 效果画到透明背景 PNG 上(与视频同分辨率),
* 以 dataURL 形式传给后端,后端用 FFmpeg overlay 直接叠加图层,
* 彻底解决前端 HTML/CSS 预览 ≠ FFmpeg drawtext 成片的 WYSIWYG 问题。
*
* 约定:titleConfig.size 的语义是"720p 基准宽度下的字号(px",
* 按 videoWidth / 720 得到 scale,所有长度类参数乘以 scale,
* 保证 1080p / 4K 成片里标题视觉大小与预览一致。
*/
import type { AiAvatarTitleConfig } from "../types"
export interface RenderTitlePngOptions {
/** 标题配置 */
titleConfig: AiAvatarTitleConfig
/** 视频宽度(像素),默认 720 */
videoWidth?: number
/** 视频高度(像素),默认 1280 */
videoHeight?: number
}
/**
* 将标题渲染为透明背景 PNG 的 dataURLdata:image/png;base64,...
* Canvas 尺寸与视频一致,保证叠加时 1:1 像素对齐。
*
* 标题为空时返回 null。
*/
export function renderTitleToPngDataUrl(opts: RenderTitlePngOptions): string | null {
const { titleConfig, videoWidth = 720, videoHeight = 1280 } = opts
if (!titleConfig) return null
const rawTitle = (titleConfig.title || "").trim()
if (!rawTitle) return null
// 按 / 或 分割为多行
const lines = rawTitle
.split(/[/]/)
.map((l) => l.trim())
.filter((l) => l.length > 0)
if (lines.length === 0) return null
// 分辨率缩放系数:基准 720p,所有长度类参数乘以 scale
const scale = videoWidth / 720
const r = (v: number) => Math.round(v * scale)
const canvas = document.createElement("canvas")
canvas.width = videoWidth
canvas.height = videoHeight
const ctx = canvas.getContext("2d")
if (!ctx) return null
const baseSize = Math.max(12, Math.round(titleConfig.size || 48))
const size = r(baseSize)
const bold = !!titleConfig.bold
const italic = !!titleConfig.italic
const color = titleConfig.color || "#ffffff"
const stroke = !!titleConfig.stroke
const shadow = !!titleConfig.shadow
// 字体族 fallback 链:优先中文字体
const fontFamily =
'"Noto Sans CJK SC","Source Han Sans CN","PingFang SC","Microsoft YaHei",sans-serif'
const fontParts: string[] = []
if (italic) fontParts.push("italic")
if (bold) fontParts.push("bold")
fontParts.push(`${size}px`, fontFamily)
ctx.font = fontParts.join(" ")
ctx.fillStyle = color
ctx.textAlign = "center"
ctx.textBaseline = "middle"
// 阴影(shadow=true 时开启)——按 scale 缩放
if (shadow) {
ctx.shadowColor = "rgba(0,0,0,0.8)"
ctx.shadowBlur = r(4)
ctx.shadowOffsetX = 0
ctx.shadowOffsetY = r(2)
}
// 位置计算:与 PanelLipsyncPreview 的 CSS 对齐(按 scale 缩放 PAD
const PAD = r(16)
let centerX = videoWidth / 2
const position = titleConfig.position || "bottom"
const lineGap = size * 1.2
const totalTextH = lines.length * lineGap - (lineGap - size) // 所有行的总高度
// 文本块顶部 ytextBaseline=middle 时首行基线)
let firstLineY: number
if (
position === "custom" &&
typeof titleConfig.pos_x === "number" &&
typeof titleConfig.pos_y === "number"
) {
centerX = (Math.max(0, Math.min(100, titleConfig.pos_x)) / 100) * videoWidth
const centerY = (Math.max(0, Math.min(100, titleConfig.pos_y)) / 100) * videoHeight
firstLineY = centerY - totalTextH / 2 + size / 2
} else if (position === "top") {
// 顶部:y = size/2 + PAD
firstLineY = size / 2 + PAD
} else if (position === "center") {
firstLineY = videoHeight / 2 - totalTextH / 2 + size / 2
} else {
// bottom(默认)
firstLineY = videoHeight - totalTextH - PAD + size / 2
}
// 描边参数:描边 lineWidth 按 scale 缩放(基准 size * 0.06,最小 2px @720p
const doStroke = stroke
const strokeWidth = Math.max(r(2), Math.round(size * 0.06))
// 逐行绘制
lines.forEach((line, idx) => {
const y = firstLineY + idx * lineGap
if (doStroke) {
const prevShadowColor = ctx.shadowColor
const prevShadowBlur = ctx.shadowBlur
// 描边不要带阴影(避免黑色描边发虚)
ctx.shadowColor = "rgba(0,0,0,0)"
ctx.shadowBlur = 0
ctx.lineWidth = strokeWidth
ctx.strokeStyle = "#000000"
ctx.lineJoin = "round"
ctx.strokeText(line, centerX, y)
// 恢复阴影
if (shadow) {
ctx.shadowColor = "rgba(0,0,0,0.8)"
ctx.shadowBlur = r(4)
} else {
ctx.shadowColor = prevShadowColor
ctx.shadowBlur = prevShadowBlur
}
}
ctx.fillText(line, centerX, y)
})
try {
return canvas.toDataURL("image/png")
} catch {
return null
}
}
/**
* 获取视频真实分辨率(HTMLVideoElement + loadedmetadata,超时 3 秒兜底 720×1280)。
*/
export function getVideoResolution(
videoUrl: string,
timeoutMs = 3000,
): Promise<{ width: number; height: number }> {
return new Promise((resolve) => {
if (!videoUrl) {
resolve({ width: 720, height: 1280 })
return
}
const video = document.createElement("video")
video.preload = "metadata"
video.muted = true
video.playsInline = true
video.crossOrigin = "anonymous"
let settled = false
const done = (w: number, h: number) => {
if (settled) return
settled = true
video.removeAttribute("src")
video.load()
resolve({ width: w, height: h })
}
const timer = window.setTimeout(() => done(720, 1280), timeoutMs)
video.onloadedmetadata = () => {
window.clearTimeout(timer)
const w = video.videoWidth || 720
const h = video.videoHeight || 1280
done(w, h)
}
video.onerror = () => {
window.clearTimeout(timer)
done(720, 1280)
}
video.src = videoUrl
})
}
+77 -25
View File
@@ -1,17 +1,22 @@
/** /**
* 成片库页面 — V21 设计系统 * 成片库页面 — V21 设计系统
* 卡片网格布局,支持视频内联播放/下载/分享、批量操作、筛选 * 卡片网格布局,支持视频内联播放/下载/分享、批量操作、筛选、无限滚动分页
* *
* 主组件仅保留 Hook 组装与整体布局 * 主组件仅保留 Hook 组装与整体布局
* 列表查询 → hooks/useProductList * 列表查询 → hooks/useProductListuseInfiniteQuery 分页)
* 操作逻辑 → hooks/useProductActions * 操作逻辑 → hooks/useProductActions
* 筛选栏 → components/ProductFilterBar * 筛选栏 → components/ProductFilterBar
* 批量操作栏 → components/ProductBatchBar * 批量操作栏 → components/ProductBatchBar
* 空状态 → components/ProductEmptyState * 空状态 → components/ProductEmptyState
* 产品卡片 → components/ProductCard(内联视频播放) * 产品卡片 → components/ProductCard(内联视频播放)
*/ */
import React from "react" import React, { useEffect, useRef } from "react"
import { VideoCameraOutlined, DownloadOutlined, ReloadOutlined } from "@ant-design/icons" import {
VideoCameraOutlined,
DownloadOutlined,
ReloadOutlined,
LoadingOutlined,
} from "@ant-design/icons"
import { Button } from "@/components/ui" import { Button } from "@/components/ui"
import { ProductCard } from "./components/ProductCard" import { ProductCard } from "./components/ProductCard"
import { ProductFilterBar } from "./components/ProductFilterBar" import { ProductFilterBar } from "./components/ProductFilterBar"
@@ -24,11 +29,13 @@ import "./products.css"
const ProductLibrary: React.FC = () => { const ProductLibrary: React.FC = () => {
const { const {
products,
filteredProducts, filteredProducts,
isLoading, isLoading,
isFetchingNextPage,
isError, isError,
error, error,
hasNextPage,
fetchNextPage,
refetch, refetch,
searchText, searchText,
setSearchText, setSearchText,
@@ -64,19 +71,40 @@ const ProductLibrary: React.FC = () => {
} = useProductActions({ } = useProductActions({
selectedIds, selectedIds,
clearSelection, clearSelection,
products, products: filteredProducts,
setPlayingProduct: () => {}, // 不再使用弹窗播放 setPlayingProduct: () => {}, // 不再使用弹窗播放
}) })
const { recomputeDedup, isRecomputing } = useRecomputeDedup() const { recomputeDedup, isRecomputing } = useRecomputeDedup()
// ── Loading 状态 ── /* ── 无限滚动:IntersectionObserver 监听底部哨兵元素 ── */
if (isLoading) { const sentinelRef = useRef<HTMLDivElement>(null)
useEffect(() => {
const el = sentinelRef.current
if (!el) return
// 已有数据但正在加载中/没有更多页时不触发
if (isFetchingNextPage || !hasNextPage) return
const observer = new IntersectionObserver(
(entries) => {
if (entries[0]?.isIntersecting) {
void fetchNextPage()
}
},
{ rootMargin: "200px" },
)
observer.observe(el)
return () => observer.disconnect()
}, [fetchNextPage, hasNextPage, isFetchingNextPage])
// ── Loading 状态(仅首次加载)──
if (isLoading && filteredProducts.length === 0) {
return <ProductEmptyState type="loading" /> return <ProductEmptyState type="loading" />
} }
// ── Error 状态 ── // ── Error 状态 ──
if (isError) { if (isError && filteredProducts.length === 0) {
console.error("[ProductLibrary] 加载失败:", error) console.error("[ProductLibrary] 加载失败:", error)
const errorMsg = error?.message || "加载失败" const errorMsg = error?.message || "加载失败"
const is404 = errorMsg.includes("404") || errorMsg.includes("Not Found") const is404 = errorMsg.includes("404") || errorMsg.includes("Not Found")
@@ -143,22 +171,46 @@ const ProductLibrary: React.FC = () => {
{/* 卡片网格 */} {/* 卡片网格 */}
{filteredProducts.length > 0 ? ( {filteredProducts.length > 0 ? (
<div className="xx-products-grid"> <>
{filteredProducts.map((product) => ( <div className="xx-products-grid">
<ProductCard {filteredProducts.map((product) => (
key={product.id} <ProductCard
product={product} key={product.id}
isSelected={selectedIds.has(product.id)} product={product}
batchMode={batchMode} isSelected={selectedIds.has(product.id)}
onToggleSelect={handleToggleSelect} batchMode={batchMode}
onDownload={handleDownload} onToggleSelect={handleToggleSelect}
onShare={handleShare} onDownload={handleDownload}
onDelete={handleDelete} onShare={handleShare}
onPublish={handlePublish} onDelete={handleDelete}
onReviewStatusChange={handleReviewStatusChange} onPublish={handlePublish}
/> onReviewStatusChange={handleReviewStatusChange}
))} />
</div> ))}
</div>
{/* 底部哨兵 + 状态提示 */}
<div
ref={sentinelRef}
style={{
gridColumn: "1 / -1",
textAlign: "center",
padding: "24px 0",
fontSize: 13,
color: "#8c8ca1",
}}
>
{isFetchingNextPage ? (
<>
<LoadingOutlined />
</>
) : hasNextPage ? (
<span style={{ opacity: 0 }}></span>
) : (
<span> </span>
)}
</div>
</>
) : ( ) : (
<ProductEmptyState type="empty" /> <ProductEmptyState type="empty" />
)} )}
@@ -1,28 +1,53 @@
import { useMemo } from "react" import { useMemo } from "react"
import { useQuery } from "@tanstack/react-query" import { useInfiniteQuery } from "@tanstack/react-query"
import { getProducts, type ProductItem as ApiProductItem } from "@/api/products" import { getProducts, type ProductItem as ApiProductItem } from "@/api/products"
import { mapApiProduct } from "../../utils" import { mapApiProduct } from "../../utils"
import type { ProductItem } from "../../types"
import { useProductFiltering } from "./useProductFiltering" import { useProductFiltering } from "./useProductFiltering"
import { useBatchSelection } from "./useBatchSelection" import { useBatchSelection } from "./useBatchSelection"
export type { Filters } from "./useProductFiltering" export type { Filters } from "./useProductFiltering"
const PAGE_SIZE = 20
export const useProductList = () => { export const useProductList = () => {
/* ── 获取成品列表 ── */ /* ── 无限滚动获取成品列表(每页 20 条) ── */
const { const {
data: apiProducts = [], data,
isLoading, isLoading,
isFetchingNextPage,
isError, isError,
error, error,
hasNextPage,
fetchNextPage,
refetch, refetch,
} = useQuery<ApiProductItem[], Error>({ } = useInfiniteQuery<
{
items: ApiProductItem[]
total: number
page: number
page_size: number
},
Error
>({
queryKey: ["products"], queryKey: ["products"],
queryFn: () => getProducts(), queryFn: async ({ pageParam = 1 }) =>
getProducts({ page: pageParam as number, page_size: PAGE_SIZE }),
initialPageParam: 1,
getNextPageParam: (lastPage) => {
const loadedCount = lastPage.page * lastPage.page_size
return loadedCount < lastPage.total ? lastPage.page + 1 : undefined
},
staleTime: 30_000, staleTime: 30_000,
}) })
// 映射为前端类型,按创建时间倒序排列,防御非数组返回 // 将所有页拼接为一维数组,再做前端映射+排序
const products = useMemo( const apiProducts = useMemo<ApiProductItem[]>(() => {
if (!data?.pages) return []
return data.pages.flatMap((p) => p.items)
}, [data])
const products = useMemo<ProductItem[]>(
() => () =>
(Array.isArray(apiProducts) ? apiProducts : []).map(mapApiProduct).sort((a, b) => { (Array.isArray(apiProducts) ? apiProducts : []).map(mapApiProduct).sort((a, b) => {
if (!a.date || a.date === "—") return 1 if (!a.date || a.date === "—") return 1
@@ -65,8 +90,11 @@ export const useProductList = () => {
products, products,
filteredProducts, filteredProducts,
isLoading, isLoading,
isFetchingNextPage,
isError, isError,
error, error,
hasNextPage,
fetchNextPage,
refetch, refetch,
// 筛选 // 筛选
searchText, searchText,
@@ -703,6 +703,9 @@ class LipsyncJobModel(Base):
error_message = Column(Text, nullable=False, default="") error_message = Column(Text, nullable=False, default="")
error_code = Column(String(100), nullable=False, default="") error_code = Column(String(100), nullable=False, default="")
# 精确句子时间戳(TTS 合成后由 silencedetect 计算,用于 B-roll 精确定位)
sentence_timings = Column(JSON, nullable=True) # list[{index,text,start_time,end_time}]
# 时间戳 # 时间戳
submitted_at = Column(DateTime, nullable=True) submitted_at = Column(DateTime, nullable=True)
completed_at = Column(DateTime, nullable=True) completed_at = Column(DateTime, nullable=True)
+7 -9
View File
@@ -49,19 +49,17 @@ class GeneratedVideo:
thumbnail_url: str | None = None, thumbnail_url: str | None = None,
generation_params: dict[str, Any] | None = None, generation_params: dict[str, Any] | None = None,
) -> "GeneratedVideo": ) -> "GeneratedVideo":
if not project_id.strip(): # project_id / generation_task_id 允许为空:AI数字人等无项目场景下,前端可能不传 project_id;
raise ValueError("project_id cannot be empty") # lipsync 路径下 generation_task_id 也可能暂时为空。空串会被下面统一兜底为 "" 入库。
if not generation_task_id.strip(): if not name or not name.strip():
raise ValueError("generation_task_id cannot be empty")
if not name.strip():
raise ValueError("name cannot be empty") raise ValueError("name cannot be empty")
if not file_url.strip(): if not file_url or not file_url.strip():
raise ValueError("file_url cannot be empty") raise ValueError("file_url cannot be empty")
return cls( return cls(
id=uuid4().hex, id=uuid4().hex,
project_id=project_id.strip(), project_id=(project_id or "").strip(),
user_id=user_id.strip(), user_id=(user_id or "").strip(),
generation_task_id=generation_task_id.strip(), generation_task_id=(generation_task_id or "").strip(),
name=name.strip(), name=name.strip(),
file_url=file_url.strip(), file_url=file_url.strip(),
file_size=file_size, file_size=file_size,
+205
View File
@@ -0,0 +1,205 @@
"""共享的句子时间戳计算工具 — 供 Celery TTS 任务和 /lipsync/tts-preview 同步接口复用.
- `_split_script_into_sentences`: 按标点分句(中英文逗号/句号/问号/感叹号/分号/换行)
- `_estimate_sentence_timings_by_chars`: 按字数比例估算(静音检测失败时降级)
- `_probe_audio_duration`: ffprobe 读取音频时长
- `compute_sentence_timings`: 基于 ffmpeg silencedetect 精确计算每句起止时间
"""
from __future__ import annotations
import logging
import os
import re
import subprocess
import tempfile
from typing import Optional
logger = logging.getLogger(__name__)
def split_script_into_sentences(script_text: str) -> list[str]:
"""按句号/问号/感叹号/分号/逗号/换行分句(与前端 SENTENCE_SPLIT_RE 一致).
中文短视频文案习惯用「,」断小句(如"卖花的叫花无缺,卖姜的叫姜子牙"),
必须把逗号也纳入分隔符,否则多句文案会被识别成一整句,导致 B-roll 时间戳错位。
"""
text = (script_text or "").strip()
if not text:
return []
parts = re.split(r"[。!?!??!;,\n\r]+", text)
return [p.strip() for p in parts if p.strip()]
def estimate_sentence_timings_by_chars(sentences: list[str], total_duration: float) -> list[dict]:
"""降级方案:按字数比例估算句子时间(与原前端逻辑一致)."""
if not sentences or total_duration <= 0:
return []
total_chars = sum(len(s.replace(r"\s", "")) for s in sentences)
if total_chars == 0:
return []
timings = []
acc = 0
for i, sent in enumerate(sentences):
chars = len(sent.replace(r"\s", ""))
start = (acc / total_chars) * total_duration
end = ((acc + chars) / total_chars) * total_duration
timings.append(
{
"index": i,
"text": sent,
"start_time": round(start, 2),
"end_time": round(end, 2),
}
)
acc += chars
return timings
def probe_audio_duration(audio_data: bytes, timeout: int = 10) -> float:
"""用 ffprobe 读取音频字节流的时长(秒).
Returns:
时长(秒),失败返回 0.0
"""
if not audio_data:
return 0.0
tmp_path: Optional[str] = None
try:
with tempfile.NamedTemporaryFile(suffix=".mp3", delete=False) as tmp:
tmp.write(audio_data)
tmp_path = tmp.name
result = subprocess.run(
[
"ffprobe",
"-v",
"error",
"-show_entries",
"format=duration",
"-of",
"default=noprint_wrappers=1:nokey=1",
tmp_path,
],
capture_output=True,
text=True,
timeout=timeout,
)
stdout = (result.stdout or "").strip()
if not stdout:
logger.warning("[sentence_timings] ffprobe 无输出: stderr=%s", (result.stderr or "")[:200])
return 0.0
return float(stdout)
except Exception as exc:
logger.warning("[sentence_timings] ffprobe 时长探测失败: %s", exc)
return 0.0
finally:
if tmp_path:
try:
os.unlink(tmp_path)
except Exception:
pass
def compute_sentence_timings(audio_data: bytes, script_text: str, total_duration: float) -> list[dict]:
"""基于 TTS 音频的静音检测,精确计算每句文案的起止时间.
使用 ffmpeg silencedetect 检测静音段,将静音点与句子边界对齐。
比字数比例估算准确得多。
Args:
audio_data: TTS 音频二进制数据(MP3
script_text: 文案全文
total_duration: 音频总时长(秒)
Returns:
list[{"index": int, "text": str, "start_time": float, "end_time": float}]
"""
sentences = split_script_into_sentences(script_text)
if not sentences:
return []
tmp_path: Optional[str] = None
try:
with tempfile.NamedTemporaryFile(suffix=".mp3", delete=False) as tmp:
tmp.write(audio_data)
tmp_path = tmp.name
result = subprocess.run(
[
"ffmpeg",
"-i",
tmp_path,
"-af",
"silencedetect=noise=-25dB:d=0.3",
"-f",
"null",
"-",
],
capture_output=True,
text=True,
timeout=30,
)
stderr = result.stderr or ""
silence_ends = []
for match in re.finditer(r"silence_end:\s*([\d.]+)", stderr):
t = float(match.group(1))
if 0 < t < total_duration:
silence_ends.append(t)
if len(silence_ends) < len(sentences) - 1:
logger.warning(
"[sentence_timings] 静音点不足(%d < %d),降级为字数比例估算",
len(silence_ends),
len(sentences) - 1,
)
return estimate_sentence_timings_by_chars(sentences, total_duration)
n_boundaries = len(sentences) - 1
boundaries = []
used_indices = set()
for i in range(n_boundaries):
expected_pos = (i + 1) / len(sentences) * total_duration
best_idx = None
best_dist = float("inf")
for j, t in enumerate(silence_ends):
if j in used_indices:
continue
dist = abs(t - expected_pos)
if dist < best_dist:
best_dist = dist
best_idx = j
if best_idx is not None:
used_indices.add(best_idx)
boundaries.append(silence_ends[best_idx])
boundaries.sort()
timings = []
prev_end = 0.0
for i, sent in enumerate(sentences):
start = prev_end
end = boundaries[i] if i < len(boundaries) else total_duration
timings.append(
{
"index": i,
"text": sent,
"start_time": round(start, 2),
"end_time": round(end, 2),
}
)
prev_end = end
return timings
except Exception as exc:
logger.warning("[sentence_timings] 静音检测异常,降级为字数比例估算: %s", exc)
return estimate_sentence_timings_by_chars(sentences, total_duration)
finally:
if tmp_path:
try:
os.unlink(tmp_path)
except Exception:
pass
+209 -90
View File
@@ -377,26 +377,30 @@ def _append_audio_concat(parts: list[str], clip_chains: list[ClipFilterChain]) -
# ── 标题 drawtext 滤镜构建(#1789)───────────────────────────────────────────── # ── 标题 drawtext 滤镜构建(#1789)─────────────────────────────────────────────
# drawtext 字体搜索路径:按优先级列出常见安装位置 # drawtext 字体搜索路径:按优先级从高到低排
# 服务器使用 Noto Sans SC(思源黑体)作为默认字体 # 服务器使用 Noto Sans SC(思源黑体)作为默认字体
# - NotoSansSC-VF.ttf 是 worker-base.Dockerfile 中 COPY 的 VF 字体(含所有字重,无 Mono 变体),优先级最高
# - .ttc 系列为 fonts-noto-cjk 包预装字体(Dockerfile 已删除含 Mono 变体的旧 .ttc,存在时作为 fallback
# - DejaVuSans 仅含拉丁字符不支持中文,已移除
DRAWTEXT_FONT_SEARCH_PATHS: list[str] = [ DRAWTEXT_FONT_SEARCH_PATHS: list[str] = [
"/usr/share/fonts/opentype/noto/NotoSansSC-VF.ttf",
"/usr/share/fonts/opentype/noto/NotoSansCJK-Regular.ttc", "/usr/share/fonts/opentype/noto/NotoSansCJK-Regular.ttc",
"/usr/share/fonts/opentype/noto/NotoSansCJK-Bold.ttc",
"/usr/share/fonts/noto-cjk/NotoSansCJK-Regular.ttc", "/usr/share/fonts/noto-cjk/NotoSansCJK-Regular.ttc",
"/usr/share/fonts/google-noto-cjk/NotoSansCJK-Regular.ttc", "/usr/share/fonts/google-noto-cjk/NotoSansCJK-Regular.ttc",
"/usr/share/fonts/truetype/noto/NotoSansSC-Regular.ttf", "/usr/share/fonts/truetype/noto/NotoSansSC-Regular.ttf",
"/usr/share/fonts/noto/NotoSansSC-Regular.ttf", "/usr/share/fonts/noto/NotoSansSC-Regular.ttf",
"/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf",
] ]
# 前端字体名 → drawtext 字体搜索关键字 # 前端字体名 → drawtext 字体搜索关键字(匹配 DRAWTEXT_FONT_SEARCH_PATHS 中的文件名关键字)
DRAWTEXT_FONT_MAP: dict[str, str] = { DRAWTEXT_FONT_MAP: dict[str, str] = {
"思源黑体": "NotoSansCJK", "思源黑体": "NotoSansSC",
"思源宋体": "NotoSerifCJK", "思源宋体": "NotoSerifCJK",
"苹方": "NotoSansCJK", "苹方": "NotoSansSC",
"PingFang": "NotoSansCJK", "PingFang": "NotoSansSC",
"微软雅黑": "NotoSansCJK", "微软雅黑": "NotoSansSC",
"楷体": "NotoSerifCJK", "楷体": "NotoSerifCJK",
"华康俪金黑": "NotoSansCJK", "华康俪金黑": "NotoSansSC",
} }
@@ -416,6 +420,11 @@ def _escape_drawtext_text(text: str) -> str:
return result return result
# 粗体字体已由前端 Canvas 直接渲染(Canvas 使用浏览器原生粗体 glyph),
# FFmpeg 侧不再需要查找 Bold 字体文件;drawtext 仅作为旧版前端的降级路径,
# 通过 borderw 黑色细描边模拟粗体(见 build_title_drawtext_filter)。
def _resolve_font_path(font_name: str) -> str: def _resolve_font_path(font_name: str) -> str:
"""解析字体名到服务器实际字体文件路径。 """解析字体名到服务器实际字体文件路径。
@@ -423,6 +432,9 @@ def _resolve_font_path(font_name: str) -> str:
1. 通过 DRAWTEXT_FONT_MAP 映射前端字体名到服务器关键字 1. 通过 DRAWTEXT_FONT_MAP 映射前端字体名到服务器关键字
2. 在 DRAWTEXT_FONT_SEARCH_PATHS 中查找匹配路径 2. 在 DRAWTEXT_FONT_SEARCH_PATHS 中查找匹配路径
3. 未找到则返回空字符串(drawtext 使用内置默认字体) 3. 未找到则返回空字符串(drawtext 使用内置默认字体)
注:粗体已由前端 Canvas 渲染时直接用浏览器 bold glyph 绘制,
此处仅作为旧版前端降级路径,无需切换 Bold 字体文件。
""" """
keyword = DRAWTEXT_FONT_MAP.get(font_name, font_name) keyword = DRAWTEXT_FONT_MAP.get(font_name, font_name)
import os import os
@@ -466,8 +478,10 @@ def build_title_drawtext_filter(
if not title_config or not isinstance(title_config, dict): if not title_config or not isinstance(title_config, dict):
return None return None
# 字段名归一化:兼容 content/text、font_preset/font 两套命名 # 字段名归一化:兼容 content/text/title 三套命名
text = (title_config.get("text") or title_config.get("content") or "").strip() text = (
title_config.get("text") or title_config.get("content") or title_config.get("title") or ""
).strip()
if not text: if not text:
return None return None
@@ -477,13 +491,13 @@ def build_title_drawtext_filter(
# ── 样式参数 ── # ── 样式参数 ──
font_name = title_config.get("font") or title_config.get("font_preset") or "思源黑体" font_name = title_config.get("font") or title_config.get("font_preset") or "思源黑体"
font_size = int(title_config.get("font_size") or title_config.get("size") or 36) font_size = int(title_config.get("font_size") or title_config.get("size") or 48)
font_color = title_config.get("font_color") or title_config.get("color") or "#ffffff" font_color = title_config.get("font_color") or title_config.get("color") or "#ffffff"
# 去掉 # 前缀(drawtext 用纯 hex 或颜色名) # 去掉 # 前缀(drawtext 用纯 hex 或颜色名)
if font_color.startswith("#"): if font_color.startswith("#"):
font_color = font_color[1:] font_color = font_color[1:]
position = title_config.get("position", "top") position = title_config.get("position") or "bottom"
bold = bool(title_config.get("bold", True)) bold = bool(title_config.get("bold", True))
stroke = title_config.get("stroke") stroke = title_config.get("stroke")
shadow = title_config.get("shadow") shadow = title_config.get("shadow")
@@ -491,7 +505,7 @@ def build_title_drawtext_filter(
# ── 构建 drawtext 参数 ── # ── 构建 drawtext 参数 ──
params: list[str] = [] params: list[str] = []
# 字体文件 # 字体文件(drawtext 降级路径:粗体通过 borderw 黑色描边模拟)
font_path = _resolve_font_path(font_name) font_path = _resolve_font_path(font_name)
if font_path: if font_path:
escaped_path = font_path.replace("\\", "\\\\").replace(":", "\\\\:").replace("'", "\\\\'") escaped_path = font_path.replace("\\", "\\\\").replace(":", "\\\\:").replace("'", "\\\\'")
@@ -504,26 +518,28 @@ def build_title_drawtext_filter(
params.append(f"fontsize={font_size}") params.append(f"fontsize={font_size}")
params.append(f"fontcolor={font_color}") params.append(f"fontcolor={font_color}")
# 粗体:bold 在 drawtext 中通过 font 的 Bold 变体实现
# 若字体有 Bold 变体可用 fontfont=bold;否则通过 borderw 模拟
if bold:
# 使用 font 参数尝试加载 Bold 变体(Noto Sans SC 有 Bold 变体文件)
params.append("font=bold")
# 描边(borderw 需要 libfreetype 支持) # 描边(borderw 需要 libfreetype 支持)
# 之前用 borderw=3 + font_color 同色描边模拟粗体,会在小字号/竖屏视频上造成
# 字形偏移、边缘重影,看起来像文字被打印了两次(用户截图中的标题"曝光曝光…")。
# 修复:粗体改用黑色细描边(borderw=2, 黑色),视觉上清晰加粗且不产生偏移。
# 用户显式开启 stroke 时按用户配置走;粗体+无stroke 默认黑色细描边。
border_width = 0
border_color = "000000"
if stroke: if stroke:
if isinstance(stroke, bool): if isinstance(stroke, bool):
border_width = 2 border_width = 2
border_color = "black" border_color = "000000"
elif isinstance(stroke, dict): elif isinstance(stroke, dict):
border_width = int(stroke.get("width", 2)) if stroke.get("enabled", True) else 0 if stroke.get("enabled", True):
border_color = (stroke.get("color") or "#000000").lstrip("#") border_width = int(stroke.get("width", 2))
else: border_color = (stroke.get("color") or "#000000").lstrip("#")
border_width = 0 elif bold:
border_color = "black" # 粗体模式且未配描边:黑色细描边,模拟粗体同时保证不重影
if border_width > 0: border_width = 2
params.append(f"borderw={border_width}") border_color = "000000"
params.append(f"bordercolor={border_color}") if border_width > 0:
params.append(f"borderw={border_width}")
params.append(f"bordercolor={border_color}")
# 阴影(shadowcolor + shadowx/y # 阴影(shadowcolor + shadowx/y
if shadow: if shadow:
@@ -548,8 +564,13 @@ def build_title_drawtext_filter(
and not isinstance(pos_x, bool) and not isinstance(pos_x, bool)
and not isinstance(pos_y, bool) and not isinstance(pos_y, bool)
): ):
params.append(f"x={int(pos_x)}") # pos_x/pos_y 为百分比坐标(0-100),转换为 drawtext 表达式
params.append(f"y={int(pos_y)}") # 例如 pos_x=50 → x=(w-text_w)*0.50(水平居中偏50%
# pos_y=30 → y=(h-text_h)*0.30
pct_x = max(0.0, min(100.0, float(pos_x))) / 100.0
pct_y = max(0.0, min(100.0, float(pos_y))) / 100.0
params.append(f"x=(w-text_w)*{pct_x:.4f}")
params.append(f"y=(h-text_h)*{pct_y:.4f}")
else: else:
# 三档预设位置:top / center / bottom # 三档预设位置:top / center / bottom
# x 始终水平居中:(w-text_w)/2 # x 始终水平居中:(w-text_w)/2
@@ -565,6 +586,43 @@ def build_title_drawtext_filter(
return "drawtext=" + ":".join(params) return "drawtext=" + ":".join(params)
def build_title_overlay_filter(
title_config: dict[str, Any],
output_width: int, # noqa: ARG001 - 保留参数签名,PNG 已按视频分辨率绘制
output_height: int, # noqa: ARG001
title_png_path: str,
*,
title_input_label: str = "[1:v]",
base_label: str = "[0:v]",
output_label: str = "vout_titled",
) -> str | None:
"""构建标题 PNG 图层 overlay 滤镜(WYSIWYG 路径)。
前端用 Canvas 把标题画成与视频同分辨率的透明 PNG(所见即所得),
后端直接 overlay=0:0 叠加即可,PNG 透明区域不遮挡视频。
Args:
title_config: 标题配置 dict(仅用来判断降级)
output_width: 输出宽度(未使用,PNG 已按该分辨率绘制)
output_height: 输出高度(未使用)
title_png_path: 已保存到本地的标题 PNG 文件路径
title_input_label: 标题 PNG 在 filter_complex 中的输入标签(默认 "[1:v]"
base_label: 前序滤镜输出标签(如 B-roll 输出 "[vout]"
output_label: overlay 输出标签名
Returns:
overlay 滤镜字符串;title_png_path 为空/文件不存在时返回 None(降级到 drawtext
"""
import os
if not title_png_path or not os.path.isfile(title_png_path):
return None
if not title_config or not isinstance(title_config, dict):
return None
return f"{base_label}{title_input_label}overlay=0:0[{output_label}]"
# ── B-roll 叠加滤镜 ───────────────────────────────────────────────────────── # ── B-roll 叠加滤镜 ─────────────────────────────────────────────────────────
@@ -573,7 +631,7 @@ def build_broll_overlay_filter(
video_duration: float, video_duration: float,
output_width: int = DEFAULT_OUTPUT_WIDTH, output_width: int = DEFAULT_OUTPUT_WIDTH,
output_height: int = DEFAULT_OUTPUT_HEIGHT, output_height: int = DEFAULT_OUTPUT_HEIGHT,
) -> str: ) -> tuple[str, str | None]:
"""构建 B-roll 叠加滤镜链。 """构建 B-roll 叠加滤镜链。
支持两种模式: 支持两种模式:
@@ -581,121 +639,182 @@ def build_broll_overlay_filter(
- pip: 在对口型视频上叠加画中画 B-roll - pip: 在对口型视频上叠加画中画 B-roll
Args: Args:
b_roll_segments: B-roll 片段配置列表 b_roll_segments: B-roll 片段配置列表(原始顺序,决定 FFmpeg -i 输入顺序)
video_duration: 对口型视频总时长(秒) video_duration: 对口型视频总时长(秒)
output_width: 输出宽度 output_width: 输出宽度(默认 1280;AI 数字人竖屏传 720)
output_height: 输出高度 output_height: 输出高度(默认 720;AI 数字人竖屏传 1280)
Returns: Returns:
FFmpeg filter_complex 滤镜字符串片段 (filter_complex_str, final_label)
- filter_complex_str: filter_complex 片段字符串(末尾无分号)
- final_label: 最终输出 pad 标签名,如 "vout";无 B-roll 时返回 None
""" """
if not b_roll_segments: if not b_roll_segments:
return "" return "", None
# 建立原始列表下标 → FFmpeg 输入下标的映射:
# cmd 中 [0:v] 是主视频,随后按 b_roll_segments 原始顺序追加 -i
# 因此第 i 个 segment 的输入是 [{i+1}:v]
def _input_label(seg: dict[str, Any]) -> str:
# seg 必须来自 b_roll_segments;通过 id() 在原列表中查找
for i, s in enumerate(b_roll_segments):
if s is seg:
return f"[{i + 1}:v]"
# fallback: 找不到时不应发生,保守返回
return "[1:v]"
parts: list[str] = [] parts: list[str] = []
sorted_segments = sorted(b_roll_segments, key=lambda s: s.get("start_time", 0)) sorted_segments = sorted(b_roll_segments, key=lambda s: s.get("start_time", 0))
# 按模式分组处理 # 按模式分组
fullscreen_segments = [s for s in sorted_segments if s.get("mode") == "fullscreen"] fullscreen_segments = [s for s in sorted_segments if s.get("mode") == "fullscreen"]
pip_segments = [s for s in sorted_segments if s.get("mode") == "pip"] pip_segments = [s for s in sorted_segments if s.get("mode") == "pip"]
final_label = None
# ── fullscreen 模式: 切分 + concat ── # ── fullscreen 模式: 切分 + concat ──
if fullscreen_segments: if fullscreen_segments:
parts.append(_build_fullscreen_filters(fullscreen_segments, video_duration, output_width, output_height)) fs_filter, fs_label = _build_fullscreen_filters(
fullscreen_segments, b_roll_segments, video_duration, output_width, output_height, _input_label
)
parts.append(fs_filter)
final_label = fs_label
else:
fs_label = None
# ── pip 模式: overlay 滤镜 ── # ── pip 模式: overlay 滤镜 ──
if pip_segments: if pip_segments:
for idx, seg in enumerate(pip_segments): pip_filter, pip_label = _build_pip_filters(
start = seg.get("start_time", 0) pip_segments, output_width, output_height, _input_label, base_label=fs_label
end = seg.get("end_time", video_duration) )
scale = seg.get("pip_scale", 0.3) parts.append(pip_filter)
position = seg.get("pip_position", "bottom_right") final_label = pip_label
pip_w = int(output_width * scale)
pip_h = int(output_height * scale)
# 位置映射
pos_map = {
"top_left": "10:10",
"top_right": "W-w-10:10",
"bottom_left": "10:H-h-10",
"bottom_right": "W-w-10:H-h-10",
"center": "(W-w)/2:(H-h)/2",
}
pos_expr = pos_map.get(position, pos_map["bottom_right"])
broll_input_idx = len(sorted_segments) # placeholder for input index
parts.append(
f"[{broll_input_idx + idx}:v]scale={pip_w}:{pip_h}," f"enable='between(t,{start},{end})'[pip{idx}];"
)
# overlay onto main stream
if idx == 0:
base_label = "[vout]" if fullscreen_segments else "[0:v]"
else:
base_label = f"[pip{idx - 1}]"
parts.append(f"{base_label}[pip{idx}]overlay={pos_expr}:enable='between(t,{start},{end})'[vout{idx}];")
result = "".join(parts) result = "".join(parts)
# 清理末尾多余分号 # 清理末尾多余分号
if result.endswith(";"): if result.endswith(";"):
result = result[:-1] result = result[:-1]
return result return result, final_label
def _build_fullscreen_filters( def _build_fullscreen_filters(
segments: list[dict[str, Any]], sorted_fs_segments: list[dict[str, Any]],
all_segments: list[dict[str, Any]],
video_duration: float, video_duration: float,
output_width: int, output_width: int,
output_height: int, output_height: int,
) -> str: input_label_fn,
"""构建 fullscreen 模式的切分 + concat 滤镜. ) -> tuple[str, str]:
"""构建 fullscreen 模式的切分 + concat 滤镜。
对口型视频按 B-roll 时间段切分,然后用 concat 拼接 B-roll 片段。 视频按 B-roll 时间段切分,然后用 concat 拼接主视频片段和 B-roll 片段。
Returns:
(filter_str, final_label) 其中 final_label 是 concat 输出的 pad 标签
""" """
parts: list[str] = [] parts: list[str] = []
prev_end = 0.0 prev_end = 0.0
for idx, seg in enumerate(segments): # 注意:这里的 idx 是 sorted_fs_segments 中的下标;
# 实际 FFmpeg 输入下标必须通过 input_label_fn 查询
for idx, seg in enumerate(sorted_fs_segments):
start = seg.get("start_time", 0) start = seg.get("start_time", 0)
end = seg.get("end_time", video_duration) end = seg.get("end_time", video_duration)
# 保持原视频片段(B-roll 之前的部分 # 视频片段(B-roll 之前)
if prev_end < start: if prev_end < start:
parts.append(f"[0:v]trim=start={prev_end}:end={start},setpts=PTS-STARTPTS[main{idx}];") parts.append(f"[0:v]trim=start={prev_end}:end={start},setpts=PTS-STARTPTS[main{idx}];")
# B-roll 片段:缩放至目标分辨率 # B-roll 片段:缩放到输出分辨率并裁到对应时长
in_lbl = input_label_fn(seg)
parts.append( parts.append(
f"[{idx + 1}:v]scale={output_width}:{output_height}" f"{in_lbl}scale={output_width}:{output_height}"
f":force_original_aspect_ratio=decrease," f":force_original_aspect_ratio=decrease,"
f"pad={output_width}:{output_height}:(ow-iw)/2:(oh-ih)/2," f"pad={output_width}:{output_height}:(ow-iw)/2:(oh-ih)/2,"
f"trim=start=0:end={end - start},setpts=PTS-STARTPTS[br{idx}];" f"trim=start=0:end={end - start},setpts=PTS-STARTPTS[br{idx}];"
) )
prev_end = end prev_end = end
# 尾部片段 # 尾部主视频片段
if prev_end < video_duration: if prev_end < video_duration:
last_idx = len(segments) last_idx = len(sorted_fs_segments)
parts.append(f"[0:v]trim=start={prev_end}:end={video_duration},setpts=PTS-STARTPTS[main{last_idx}];") parts.append(f"[0:v]trim=start={prev_end}:end={video_duration},setpts=PTS-STARTPTS[main{last_idx}];")
# concat 所有片段 # concat 所有片段
segment_labels = [] segment_labels: list[str] = []
for idx in range(len(segments)): for idx, seg in enumerate(sorted_fs_segments):
start = segments[idx].get("start_time", 0) start = seg.get("start_time", 0)
if (idx == 0 and segments[0].get("start_time", 0) > 0) or idx > 0: # 每段 B-roll 之前是否有主视频片段?
prev_end_prev = segments[idx - 1].get("end_time", 0) if idx > 0 else 0 has_main_before = (idx == 0 and start > 0) or (
if prev_end_prev < start: idx > 0 and sorted_fs_segments[idx - 1].get("end_time", 0) < start
segment_labels.append(f"[main{idx}]") )
if has_main_before:
segment_labels.append(f"[main{idx}]")
segment_labels.append(f"[br{idx}]") segment_labels.append(f"[br{idx}]")
if prev_end < video_duration: if prev_end < video_duration:
segment_labels.append(f"[main{len(segments)}]") segment_labels.append(f"[main{len(sorted_fs_segments)}]")
final_lbl = "vout_fs"
n = len(segment_labels) n = len(segment_labels)
if n > 0: if n > 0:
concat_inputs = "".join(segment_labels) concat_inputs = "".join(segment_labels)
parts.append(f"{concat_inputs}concat=n={n}:v=1:a=0[vout];") parts.append(f"{concat_inputs}concat=n={n}:v=1:a=0[{final_lbl}];")
return "".join(parts) return "".join(parts), final_lbl
def _build_pip_filters(
pip_segments: list[dict[str, Any]],
output_width: int,
output_height: int,
input_label_fn,
base_label: str | None,
) -> tuple[str, str]:
"""构建 PIP(画中画)overlay 滤镜链。
Args:
pip_segments: 按时间排序的 pip 片段
output_width: 输出宽度
output_height: 输出高度
input_label_fn: 片段 → 输入标签的映射函数
base_label: 前序滤镜链输出的标签(如 fullscreen 的 vout_fs),为 None 则基于 [0:v]
Returns:
(filter_str, final_label)
"""
parts: list[str] = []
cur_label = base_label # 当前叠加到的标签
pos_map = {
"top_left": "10:10",
"top_right": "W-w-10:10",
"bottom_left": "10:H-h-10",
"bottom_right": "W-w-10:H-h-10",
"center": "(W-w)/2:(H-h)/2",
}
for idx, seg in enumerate(pip_segments):
start = seg.get("start_time", 0)
end = seg.get("end_time", 0)
scale = seg.get("pip_scale", 0.3)
position = seg.get("pip_position", "bottom_right")
pos_expr = pos_map.get(position, pos_map["bottom_right"])
pip_w = max(1, int(output_width * scale))
pip_h = max(1, int(output_height * scale))
enable_expr = f"enable='between(t,{start},{end})'"
in_lbl = input_label_fn(seg)
pip_scaled = f"pip{idx}"
parts.append(f"{in_lbl}scale={pip_w}:{pip_h},{enable_expr}[{pip_scaled}];")
# overlay onto the current base
base = f"[{cur_label}]" if cur_label else "[0:v]"
out_lbl = f"vout_pip{idx}" if idx < len(pip_segments) - 1 else "vout"
parts.append(f"{base}[{pip_scaled}]overlay={pos_expr}:{enable_expr}[{out_lbl}];")
cur_label = out_lbl
return "".join(parts), cur_label or "vout"
def build_cover_extract_command( def build_cover_extract_command(
+4
View File
@@ -13,3 +13,7 @@ pytest-cov==6.0.0
# 工具 # 工具
python-dotenv==1.0.1 python-dotenv==1.0.1
# AI 数字人封面智能选帧(cover_frame_scorer 用 cv2/numpy 做清晰度/亮度/色彩评分)
numpy==1.26.4
opencv-python-headless==4.10.0.84
+9 -18
View File
@@ -87,28 +87,19 @@ class TestGeneratedVideoCreate:
assert v.file_url == "http://x/v" assert v.file_url == "http://x/v"
def test_create_empty_project_id(self): def test_create_empty_project_id(self):
"""空 project_id 无效.""" """空 project_id 允许(AI数字人无项目场景)."""
try: v = GeneratedVideo.create("", "t1", "v", "http://x/v")
GeneratedVideo.create("", "t1", "v", "http://x/v") assert v.project_id == ""
assert False
except ValueError as e:
assert "project_id" in str(e)
def test_create_whitespace_project_id(self): def test_create_whitespace_project_id(self):
"""纯空白 project_id 无效.""" """纯空白 project_id 归一化为空串."""
try: v = GeneratedVideo.create(" ", "t1", "v", "http://x/v")
GeneratedVideo.create(" ", "t1", "v", "http://x/v") assert v.project_id == ""
assert False
except ValueError as e:
assert "project_id" in str(e)
def test_create_empty_task_id(self): def test_create_empty_task_id(self):
"""空 generation_task_id 无效.""" """空 generation_task_id 允许."""
try: v = GeneratedVideo.create("p1", "", "v", "http://x/v")
GeneratedVideo.create("p1", "", "v", "http://x/v") assert v.generation_task_id == ""
assert False
except ValueError as e:
assert "generation_task_id" in str(e)
def test_create_empty_name(self): def test_create_empty_name(self):
"""空 name 无效.""" """空 name 无效."""
@@ -262,8 +262,8 @@ def test_smart_cover_selects_best_frame_and_persists():
score_patch.assert_called_once() score_patch.assert_called_once()
# 验证使用了增大的轮询参数 # 验证使用了增大的轮询参数
call_kwargs = mk.extract_frames.call_args call_kwargs = mk.extract_frames.call_args
assert call_kwargs.kwargs.get("poll_interval") == 3.0 or call_kwargs[1].get("poll_interval") == 3.0 assert call_kwargs.kwargs.get("poll_interval") == 1.0 or call_kwargs[1].get("poll_interval") == 1.0
assert call_kwargs.kwargs.get("max_poll_attempts") == 20 or call_kwargs[1].get("max_poll_attempts") == 20 assert call_kwargs.kwargs.get("max_poll_attempts") == 15 or call_kwargs[1].get("max_poll_attempts") == 15
def test_smart_cover_returns_empty_when_mediakit_unavailable(): def test_smart_cover_returns_empty_when_mediakit_unavailable():
@@ -334,8 +334,8 @@ def test_extract_frames_uses_extended_poll_params():
cov.select_best_cover_frame("https://other/avatar.mp4", max_frames=3) cov.select_best_cover_frame("https://other/avatar.mp4", max_frames=3)
call_kwargs = mk.extract_frames.call_args call_kwargs = mk.extract_frames.call_args
assert call_kwargs.kwargs.get("poll_interval") == 3.0 or call_kwargs[1].get("poll_interval") == 3.0 assert call_kwargs.kwargs.get("poll_interval") == 1.0 or call_kwargs[1].get("poll_interval") == 1.0
assert call_kwargs.kwargs.get("max_poll_attempts") == 20 or call_kwargs[1].get("max_poll_attempts") == 20 assert call_kwargs.kwargs.get("max_poll_attempts") == 15 or call_kwargs[1].get("max_poll_attempts") == 15
assert call_kwargs.kwargs.get("max_retries") == 1 or call_kwargs[1].get("max_retries") == 1 assert call_kwargs.kwargs.get("max_retries") == 1 or call_kwargs[1].get("max_retries") == 1
+123 -3
View File
@@ -258,8 +258,9 @@ class TestBrollOverlayFilter:
def test_empty_segments_returns_empty(self): def test_empty_segments_returns_empty(self):
from packages.domain.video_filter_builder import build_broll_overlay_filter from packages.domain.video_filter_builder import build_broll_overlay_filter
result = build_broll_overlay_filter([], 30.0) result, label = build_broll_overlay_filter([], 30.0)
assert result == "" assert result == ""
assert label is None
def test_pip_mode_generates_overlay(self): def test_pip_mode_generates_overlay(self):
from packages.domain.video_filter_builder import build_broll_overlay_filter from packages.domain.video_filter_builder import build_broll_overlay_filter
@@ -275,8 +276,9 @@ class TestBrollOverlayFilter:
"pip_scale": 0.3, "pip_scale": 0.3,
} }
] ]
result = build_broll_overlay_filter(segments, 30.0) result, label = build_broll_overlay_filter(segments, 30.0)
assert "overlay" in result or "scale=" in result assert "overlay" in result or "scale=" in result
assert label == "vout"
def test_fullscreen_mode_generates_concat(self): def test_fullscreen_mode_generates_concat(self):
from packages.domain.video_filter_builder import build_broll_overlay_filter from packages.domain.video_filter_builder import build_broll_overlay_filter
@@ -290,8 +292,9 @@ class TestBrollOverlayFilter:
"end_time": 10.0, "end_time": 10.0,
} }
] ]
result = build_broll_overlay_filter(segments, 30.0) result, label = build_broll_overlay_filter(segments, 30.0)
assert "trim" in result or "concat" in result assert "trim" in result or "concat" in result
assert label == "vout_fs"
def test_cover_extract_command(self): def test_cover_extract_command(self):
from packages.domain.video_filter_builder import build_cover_extract_command from packages.domain.video_filter_builder import build_cover_extract_command
@@ -315,3 +318,120 @@ class TestBrollOverlayFilter:
"/tmp/cover.jpg", "/tmp/cover.jpg",
) )
assert "scale=" in cmd assert "scale=" in cmd
def _make_mock_auth_user(user_id="user-1"):
"""构造 AuthenticatedUsercurrent_user.user.id."""
auth = MagicMock()
auth.user.id = user_id
return auth
class TestRenderSmartCoverRoute:
"""POST /renders/{job_id}/smart-cover — 从成片智能抽封面(步骤②)."""
def test_smart_cover_job_not_found_returns_404(self):
"""渲染任务不存在 → 404."""
from app.api.routes.ai_avatar_render import generate_render_smart_cover
from fastapi import HTTPException
mock_service = MagicMock()
mock_service.get_render_job.return_value = None
mock_db = MagicMock()
mock_user = _make_mock_auth_user()
# 函数内部 `from app.services.ai_avatar_render_service import AiAvatarRenderService`
with patch("app.services.ai_avatar_render_service.AiAvatarRenderService", return_value=mock_service):
with pytest.raises(HTTPException) as exc_info:
generate_render_smart_cover(job_id="render-missing", current_user=mock_user, db=mock_db)
assert exc_info.value.status_code == 404
assert "不存在" in exc_info.value.detail
mock_service.get_render_job.assert_called_once_with("render-missing", "user-1")
def test_smart_cover_job_not_completed_returns_400(self):
"""任务未 completed(如 processing)→ 400."""
from app.api.routes.ai_avatar_render import generate_render_smart_cover
from fastapi import HTTPException
mock_service = MagicMock()
mock_job = _make_mock_render_job(status="processing", output_video_url="https://oss/video.mp4")
mock_service.get_render_job.return_value = mock_job
mock_db = MagicMock()
mock_user = _make_mock_auth_user()
with patch("app.services.ai_avatar_render_service.AiAvatarRenderService", return_value=mock_service):
with pytest.raises(HTTPException) as exc_info:
generate_render_smart_cover(job_id="render-1", current_user=mock_user, db=mock_db)
assert exc_info.value.status_code == 400
assert "先完成视频生成" in exc_info.value.detail
def test_smart_cover_empty_video_url_returns_400(self):
"""已 completed 但 output_video_url 为空/空白 → 400."""
from app.api.routes.ai_avatar_render import generate_render_smart_cover
from fastapi import HTTPException
mock_service = MagicMock()
mock_job = _make_mock_render_job(status="completed", output_video_url=" ")
mock_service.get_render_job.return_value = mock_job
mock_db = MagicMock()
mock_user = _make_mock_auth_user()
with patch("app.services.ai_avatar_render_service.AiAvatarRenderService", return_value=mock_service):
with pytest.raises(HTTPException) as exc_info:
generate_render_smart_cover(job_id="render-1", current_user=mock_user, db=mock_db)
assert exc_info.value.status_code == 400
assert "URL 为空" in exc_info.value.detail
def test_smart_cover_success_updates_db_and_returns_url(self):
"""抽帧成功 → 更新 job.cover_config / output_cover_url 并 commit,返回 completed."""
from app.api.routes.ai_avatar_render import generate_render_smart_cover
mock_service = MagicMock()
mock_job = _make_mock_render_job(
status="completed",
output_video_url="https://oss/final.mp4",
)
mock_job.cover_config = {"mode": "manual"}
mock_service.get_render_job.return_value = mock_job
mock_db = MagicMock()
mock_user = _make_mock_auth_user()
with (
patch("app.services.ai_avatar_render_service.AiAvatarRenderService", return_value=mock_service),
patch(
"app.api.routes.ai_avatar_render.generate_smart_cover", return_value="https://oss/cover.jpg"
) as mock_gen,
):
result = generate_render_smart_cover(job_id="render-1", current_user=mock_user, db=mock_db)
mock_gen.assert_called_once_with("https://oss/final.mp4", job_id="render-1", max_frames=5)
assert result.status == "completed"
assert result.cover_url == "https://oss/cover.jpg"
assert mock_job.output_cover_url == "https://oss/cover.jpg"
assert mock_job.cover_config["mode"] == "auto_frame"
assert mock_job.cover_config["url"] == "https://oss/cover.jpg"
mock_db.commit.assert_called_once()
def test_smart_cover_extract_failure_returns_fallback_failed(self):
"""generate_smart_cover 抛异常 → fallback_failed,不抛错不写 DB."""
from app.api.routes.ai_avatar_render import generate_render_smart_cover
mock_service = MagicMock()
mock_job = _make_mock_render_job(status="completed", output_video_url="https://oss/final.mp4")
mock_service.get_render_job.return_value = mock_job
mock_db = MagicMock()
mock_user = _make_mock_auth_user()
with (
patch("app.services.ai_avatar_render_service.AiAvatarRenderService", return_value=mock_service),
patch("app.api.routes.ai_avatar_render.generate_smart_cover", side_effect=RuntimeError("mediakit down")),
):
result = generate_render_smart_cover(job_id="render-1", current_user=mock_user, db=mock_db)
assert result.status == "fallback_failed"
assert result.cover_url == ""
# 失败时不写 cover_config / 不 commit
mock_db.commit.assert_not_called()
+63 -2
View File
@@ -547,7 +547,7 @@ class TestAiAvatarRenderService:
with ( with (
patch.object(svc, "_download_video", return_value="/tmp/video.mp4"), patch.object(svc, "_download_video", return_value="/tmp/video.mp4"),
patch.object(svc, "_upload_to_oss", side_effect=lambda path, key: f"https://oss/{key}"), patch.object(svc, "_upload_to_oss", side_effect=lambda path, key: f"https://oss/{key}"),
patch("os.system", return_value=0), patch("subprocess.run") as mock_run,
patch("tempfile.TemporaryDirectory") as tmpdir_mock, patch("tempfile.TemporaryDirectory") as tmpdir_mock,
patch( patch(
"app.services.ai_avatar_cover_service.generate_smart_cover", return_value="https://oss/smart_cover.jpg" "app.services.ai_avatar_cover_service.generate_smart_cover", return_value="https://oss/smart_cover.jpg"
@@ -557,6 +557,9 @@ class TestAiAvatarRenderService:
"packages.adapters.sqlalchemy_impl.generated_video_repository.SQLAlchemyGeneratedVideoRepository" "packages.adapters.sqlalchemy_impl.generated_video_repository.SQLAlchemyGeneratedVideoRepository"
) as repo_cls, ) as repo_cls,
): ):
import subprocess as _sp
mock_run.return_value = _sp.CompletedProcess(args=[], returncode=0, stdout="", stderr="")
import tempfile as _tf import tempfile as _tf
tmpdir_mock.return_value.__enter__ = MagicMock(return_value="/tmp/testdir") tmpdir_mock.return_value.__enter__ = MagicMock(return_value="/tmp/testdir")
@@ -605,10 +608,13 @@ class TestAiAvatarRenderService:
with ( with (
patch.object(svc, "_download_video", return_value="/tmp/video.mp4"), patch.object(svc, "_download_video", return_value="/tmp/video.mp4"),
patch.object(svc, "_upload_to_oss", side_effect=lambda path, key: f"https://oss/{key}"), patch.object(svc, "_upload_to_oss", side_effect=lambda path, key: f"https://oss/{key}"),
patch("os.system", return_value=0), patch("subprocess.run") as mock_run,
patch("tempfile.TemporaryDirectory") as tmpdir_mock, patch("tempfile.TemporaryDirectory") as tmpdir_mock,
patch("app.services.ai_avatar_cover_service.generate_smart_cover", side_effect=RuntimeError("DB error")), patch("app.services.ai_avatar_cover_service.generate_smart_cover", side_effect=RuntimeError("DB error")),
): ):
import subprocess as _sp
mock_run.return_value = _sp.CompletedProcess(args=[], returncode=0, stdout="", stderr="")
tmpdir_mock.return_value.__enter__ = MagicMock(return_value="/tmp/testdir") tmpdir_mock.return_value.__enter__ = MagicMock(return_value="/tmp/testdir")
tmpdir_mock.return_value.__exit__ = MagicMock(return_value=False) tmpdir_mock.return_value.__exit__ = MagicMock(return_value=False)
svc.execute_render("render-clip-fail") svc.execute_render("render-clip-fail")
@@ -622,3 +628,58 @@ class TestAiAvatarRenderService:
err = AiAvatarRenderError("测试错误", code="TestCode") err = AiAvatarRenderError("测试错误", code="TestCode")
assert err.code == "TestCode" assert err.code == "TestCode"
assert str(err) == "测试错误" assert str(err) == "测试错误"
class TestAiAvatarRenderCoverPassthrough:
"""execute_render 中封面透传逻辑(320~329 行):cover_config 含 url/imageUrl/cover_url 时直接透传到 output_cover_url."""
def _run_execute(self, mock_job, mock_lipsync_job):
"""驱动 execute_render 跑到完成阶段的通用脚手架(mock IO 部分)."""
from app.services.ai_avatar_render_service import AiAvatarRenderService
mock_db = _make_mock_db()
mock_filter = MagicMock()
# query.filter 返回同一个 filter 两次(render_job 查询、lipsync 查询)
mock_filter.first.side_effect = [mock_job, mock_lipsync_job]
mock_query = MagicMock()
mock_query.filter.return_value = mock_filter
mock_db.query.return_value = mock_query
svc = AiAvatarRenderService(mock_db)
with (
patch.object(svc, "_download_video", return_value="/tmp/video.mp4"),
patch.object(svc, "_upload_to_oss", side_effect=lambda path, key: f"https://oss/{key}"),
patch("subprocess.run") as mock_run,
patch("tempfile.TemporaryDirectory") as tmpdir_mock,
patch("app.services.ai_avatar_cover_service.generate_smart_cover", return_value=""),
patch("packages.domain.generated_video.GeneratedVideo.create", return_value=MagicMock()),
patch(
"packages.adapters.sqlalchemy_impl.generated_video_repository.SQLAlchemyGeneratedVideoRepository"
) as repo_cls,
):
import subprocess as _sp
mock_run.return_value = _sp.CompletedProcess(args=[], returncode=0, stdout="", stderr="")
import tempfile as _tf
tmpdir_mock.return_value.__enter__ = MagicMock(return_value="/tmp/testdir")
tmpdir_mock.return_value.__exit__ = MagicMock(return_value=False)
repo_cls.return_value = MagicMock()
svc.execute_render(mock_job.id)
return mock_db, mock_job
def test_cover_url_in_cover_config_passthrough_to_output_cover(self):
"""cover_config.url 存在 → 透传到 output_cover_url."""
mock_job = _make_mock_render_job(job_id="render-cov-1", status="pending")
mock_job.cover_config = {"mode": "upload", "url": "https://oss/user-cover.jpg"}
mock_lipsync_job = _make_mock_lipsync_job(status="completed", output_duration=10.0)
_, job = self._run_execute(mock_job, mock_lipsync_job)
assert job.output_cover_url == "https://oss/user-cover.jpg"
def test_cover_imageurl_fallback_also_passthrough(self):
"""cover_config.imageUrl(老字段)存在 → 也透传到 output_cover_url."""
mock_job = _make_mock_render_job(job_id="render-cov-2", status="pending")
mock_job.cover_config = {"mode": "upload", "imageUrl": "https://oss/user-cover2.jpg"}
mock_lipsync_job = _make_mock_lipsync_job(status="completed", output_duration=10.0)
_, job = self._run_execute(mock_job, mock_lipsync_job)
assert job.output_cover_url == "https://oss/user-cover2.jpg"
+5 -4
View File
@@ -43,7 +43,7 @@ class TestScoreFrame:
@requires_cv2 @requires_cv2
def test_clear_image_high_score(self): def test_clear_image_high_score(self):
"""清晰、亮度适中、色彩丰富的图像应得高分.""" """清晰、亮度适中、色彩丰富的图像应得高分."""
# 创建一个清晰的渐变图像(色彩丰富、亮度适中) # 创建一个清晰的渐变图像(色彩丰富、亮度适中)
img = np.zeros((100, 100, 3), dtype=np.uint8) img = np.zeros((100, 100, 3), dtype=np.uint8)
for i in range(100): for i in range(100):
@@ -53,7 +53,8 @@ class TestScoreFrame:
from packages.shared.cover_frame_scorer import score_frame from packages.shared.cover_frame_scorer import score_frame
score = score_frame(img) score = score_frame(img)
assert 50.0 <= score <= 100.0, f"清晰图像应得高分,实际: {score}" # 渐变图清晰度中等+亮度尚可+色彩有变化,分数应明显高于模糊/全黑/全白
assert 40.0 <= score <= 100.0, f"清晰图像应得较高分,实际: {score}"
@requires_cv2 @requires_cv2
def test_blurry_image_low_clarity(self): def test_blurry_image_low_clarity(self):
@@ -76,8 +77,8 @@ class TestScoreFrame:
from packages.shared.cover_frame_scorer import score_frame from packages.shared.cover_frame_scorer import score_frame
score = score_frame(img) score = score_frame(img)
# 全黑:清晰度 0,亮度 0,色彩 0 # 全黑:清晰度 0,亮度偏离130扣约24分,色彩 0 → 得分约0~7,允许cv2内部微小浮点差异
assert score <= 5.0, f"全黑图像应接近 0 分,实际: {score}" assert score <= 10.0, f"全黑图像应接近 0 分,实际: {score}"
@requires_cv2 @requires_cv2
def test_bright_image_low_brightness(self): def test_bright_image_low_brightness(self):
+6 -8
View File
@@ -180,28 +180,26 @@ class TestDetectKeyframeTimestamps:
def test_cannot_open_video_raises(self): def test_cannot_open_video_raises(self):
"""无法打开视频时抛出 RuntimeError.""" """无法打开视频时抛出 RuntimeError."""
cv2_mock = _dedup_mod.cv2
mock_cap = MagicMock() mock_cap = MagicMock()
mock_cap.isOpened.return_value = False mock_cap.isOpened.return_value = False
cv2_mock.VideoCapture.return_value = mock_cap
import pytest import pytest
with pytest.raises(RuntimeError, match="Cannot open video"): with patch.object(_dedup_mod.cv2, "VideoCapture", return_value=mock_cap):
detect_keyframe_timestamps("/fake/path.mp4") with pytest.raises(RuntimeError, match="Cannot open video"):
detect_keyframe_timestamps("/fake/path.mp4")
def test_zero_duration_returns_empty(self): def test_zero_duration_returns_empty(self):
"""视频时长为 0 时返回空列表.""" """视频时长为 0 时返回空列表."""
cv2_mock = _dedup_mod.cv2
mock_cap = MagicMock() mock_cap = MagicMock()
mock_cap.isOpened.return_value = True mock_cap.isOpened.return_value = True
# cv2.CAP_PROP_FPS etc. are Mock objects; configure get() to return 0 for frame_count # cv2.CAP_PROP_FPS etc. are Mock objects; configure get() to return 0 for frame_count
mock_cap.get.return_value = 0 mock_cap.get.return_value = 0
mock_cap.read.return_value = (False, None) mock_cap.read.return_value = (False, None)
cv2_mock.VideoCapture.return_value = mock_cap
result = detect_keyframe_timestamps("/fake/zero.mp4") with patch.object(_dedup_mod.cv2, "VideoCapture", return_value=mock_cap):
assert result == [] result = detect_keyframe_timestamps("/fake/zero.mp4")
assert result == []
def test_function_signature(self): def test_function_signature(self):
"""验证函数签名和默认参数.""" """验证函数签名和默认参数."""
+27 -24
View File
@@ -47,32 +47,35 @@ class TestGeneratedVideoCreate:
assert video.file_url == "https://example.com/video.mp4" assert video.file_url == "https://example.com/video.mp4"
assert video.user_id == "user1" assert video.user_id == "user1"
def test_create_empty_project_id_raises(self): def test_create_empty_project_id_allowed(self):
with pytest.raises(ValueError, match="project_id cannot be empty"): """project_id 允许为空(AI数字人等无项目场景)。"""
GeneratedVideo.create( video = GeneratedVideo.create(
project_id="", project_id="",
generation_task_id="task1", generation_task_id="task1",
name="视频", name="视频",
file_url="https://example.com/v.mp4", file_url="https://example.com/v.mp4",
) )
assert video.project_id == ""
def test_create_whitespace_project_id_raises(self): def test_create_whitespace_project_id_normalized_to_empty(self):
with pytest.raises(ValueError, match="project_id cannot be empty"): """project_id 纯空白会被 strip 为空串,不抛异常。"""
GeneratedVideo.create( video = GeneratedVideo.create(
project_id=" ", project_id=" ",
generation_task_id="task1", generation_task_id="task1",
name="视频", name="视频",
file_url="https://example.com/v.mp4", file_url="https://example.com/v.mp4",
) )
assert video.project_id == ""
def test_create_empty_generation_task_id_raises(self): def test_create_empty_generation_task_id_allowed(self):
with pytest.raises(ValueError, match="generation_task_id cannot be empty"): """generation_task_id 允许为空(兼容部分异步链路)。"""
GeneratedVideo.create( video = GeneratedVideo.create(
project_id="proj1", project_id="proj1",
generation_task_id="", generation_task_id="",
name="视频", name="视频",
file_url="https://example.com/v.mp4", file_url="https://example.com/v.mp4",
) )
assert video.generation_task_id == ""
def test_create_empty_name_raises(self): def test_create_empty_name_raises(self):
with pytest.raises(ValueError, match="name cannot be empty"): with pytest.raises(ValueError, match="name cannot be empty"):
+27 -24
View File
@@ -75,32 +75,35 @@ class TestGeneratedVideoCreate:
assert video.file_url == "https://example.com/out.mp4" assert video.file_url == "https://example.com/out.mp4"
assert video.user_id == "user_003" assert video.user_id == "user_003"
def test_create_empty_project_id_raises(self): def test_create_empty_project_id_allowed(self):
with pytest.raises(ValueError, match="project_id"): """project_id 允许为空(AI数字人等无项目场景)。"""
GeneratedVideo.create( video = GeneratedVideo.create(
project_id="", project_id="",
generation_task_id="t", generation_task_id="t",
name="n", name="n",
file_url="u", file_url="u",
) )
assert video.project_id == ""
def test_create_whitespace_project_id_raises(self): def test_create_whitespace_project_id_normalized(self):
with pytest.raises(ValueError, match="project_id"): """project_id 纯空白归一化为空串。"""
GeneratedVideo.create( video = GeneratedVideo.create(
project_id=" ", project_id=" ",
generation_task_id="t", generation_task_id="t",
name="n", name="n",
file_url="u", file_url="u",
) )
assert video.project_id == ""
def test_create_empty_generation_task_id_raises(self): def test_create_empty_generation_task_id_allowed(self):
with pytest.raises(ValueError, match="generation_task_id"): """generation_task_id 允许为空。"""
GeneratedVideo.create( video = GeneratedVideo.create(
project_id="p", project_id="p",
generation_task_id="", generation_task_id="",
name="n", name="n",
file_url="u", file_url="u",
) )
assert video.generation_task_id == ""
def test_create_empty_name_raises(self): def test_create_empty_name_raises(self):
with pytest.raises(ValueError, match="name"): with pytest.raises(ValueError, match="name"):
@@ -45,25 +45,25 @@ class TestGeneratedVideo:
assert video.duplicate_of is None assert video.duplicate_of is None
assert video.generation_params == {} assert video.generation_params == {}
def test_create_empty_project_id_raises(self): def test_create_empty_project_id_allowed(self):
"""project_id抛异常.""" """project_id 允许为空(AI数字人场景),空白归一化为空串."""
with pytest.raises(ValueError, match="project_id"): video = GeneratedVideo.create(
GeneratedVideo.create( project_id=" ",
project_id=" ", generation_task_id="t1",
generation_task_id="t1", name="v.mp4",
name="v.mp4", file_url="https://x.com/v.mp4",
file_url="https://x.com/v.mp4", )
) assert video.project_id == ""
def test_create_empty_task_id_raises(self): def test_create_empty_task_id_allowed(self):
"""generation_task_id抛异常.""" """generation_task_id 允许为空."""
with pytest.raises(ValueError, match="generation_task_id"): video = GeneratedVideo.create(
GeneratedVideo.create( project_id="p1",
project_id="p1", generation_task_id="",
generation_task_id="", name="v.mp4",
name="v.mp4", file_url="https://x.com/v.mp4",
file_url="https://x.com/v.mp4", )
) assert video.generation_task_id == ""
def test_create_empty_name_raises(self): def test_create_empty_name_raises(self):
"""空name抛异常.""" """空name抛异常."""
+2 -1
View File
@@ -159,7 +159,8 @@ class TestSchemaValidation:
voice_id="longxiaochun_v3", voice_id="longxiaochun_v3",
script_text="测试文本", script_text="测试文本",
) )
assert req.enable_video_loop is False # AI数字人场景文案长度不可控,默认开启视频循环,防止音频长于视频时被截断
assert req.enable_video_loop is True
def test_video_url_strip_query_params(self): def test_video_url_strip_query_params(self):
"""视频 URL 含查询参数时,扩展名检查应忽略 ? 后面的部分.""" """视频 URL 含查询参数时,扩展名检查应忽略 ? 后面的部分."""
+55 -7
View File
@@ -35,7 +35,10 @@ class TestFFmpegPresetOptimization:
final_label=None, final_label=None,
output_path="/tmp/output.mp4", output_path="/tmp/output.mp4",
) )
assert "-preset veryfast" in cmd, f"期望 -preset veryfast,实际命令: {cmd}" # cmd 现在是 list[str]preset 与值是相邻两个元素
assert "-preset" in cmd, f"期望包含 -preset,实际命令: {cmd}"
preset_idx = cmd.index("-preset")
assert cmd[preset_idx + 1] == "veryfast", f"期望 veryfast,实际: {cmd}"
def test_preset_veryfast_with_filter(self): def test_preset_veryfast_with_filter(self):
"""带滤镜场景下也必须使用 veryfast.""" """带滤镜场景下也必须使用 veryfast."""
@@ -49,7 +52,8 @@ class TestFFmpegPresetOptimization:
final_label="[v]", final_label="[v]",
output_path="/tmp/output.mp4", output_path="/tmp/output.mp4",
) )
assert "-preset veryfast" in cmd assert "-preset" in cmd
assert cmd[cmd.index("-preset") + 1] == "veryfast"
assert "-filter_complex" in cmd assert "-filter_complex" in cmd
def test_preset_not_fast(self): def test_preset_not_fast(self):
@@ -65,11 +69,11 @@ class TestFFmpegPresetOptimization:
output_path="/tmp/output.mp4", output_path="/tmp/output.mp4",
) )
# 确保是 veryfast 而不是 fast # 确保是 veryfast 而不是 fast
assert "-preset veryfast" in cmd assert "-preset" in cmd
# 排除 "fast" 单独出现(veryfast 包含 fast 子串,需精确判断) preset_idx = cmd.index("-preset")
parts = cmd.split() assert cmd[preset_idx + 1] == "veryfast"
preset_idx = parts.index("-preset") # 禁止 fast 单独作为 preset 值(veryfast 包含 "fast" 子串,不影响)
assert parts[preset_idx + 1] == "veryfast" assert cmd[preset_idx + 1] != "fast"
# ═══════════════════════════════════════════════════════════════════════════════ # ═══════════════════════════════════════════════════════════════════════════════
@@ -284,3 +288,47 @@ class TestCancelJobTtsProcessing:
result = svc.cancel_job("job-1", "user-1") result = svc.cancel_job("job-1", "user-1")
assert result.status == "cancelled" assert result.status == "cancelled"
class TestCreateJobCommitOrder:
"""验证事务顺序修复:create_job 必须先 commit 再发 Celery 任务,避免 worker 消费时 job 不可见。"""
def test_commit_called_before_apply_async_in_tts_mode(self):
"""TTS 模式:db.commit() 必须在 apply_async() 之前调用,防止 worker 查不到 job 永远卡在 tts_processing。"""
svc, client, cosy = _make_service_with_mocks()
call_order: list[str] = []
def track_commit():
call_order.append("commit")
def track_apply_async(*args, **kwargs):
call_order.append("apply_async")
svc.db.commit.side_effect = track_commit
with patch("app.services.lipsync_service.tts_synthesize_and_submit") as mock_task:
mock_task.apply_async = MagicMock(side_effect=track_apply_async)
svc.create_job(
user_id="user-1",
video_url="https://example.com/video.mp4",
voice_id="v-1",
script_text="测试",
)
# 至少有一次 commit 在 apply_async 之前
assert "commit" in call_order, "db.commit 必须被调用"
assert "apply_async" in call_order, "apply_async 必须被调用"
assert call_order.index("commit") < call_order.index(
"apply_async"
), f"事务顺序错误:commit 必须在 apply_async 之前,实际顺序 {call_order}"
def test_job_not_found_retry_mechanism_exists(self):
"""worker 侧 job not found 必须有重试机制(self.retry),而不是静默 return。"""
import inspect
from app.tasks.lipsync_tts import tts_synthesize_and_submit
source = inspect.getsource(tts_synthesize_and_submit.run)
assert (
"self.retry" in source or "retry" in source
), "tts_synthesize_and_submit 在 job not found 时必须重试,防止静默失败"
+240 -1
View File
@@ -185,7 +185,10 @@ class TestTtsSynthesizeAndSubmit:
mk_client.submit_lipsync.assert_called_once() mk_client.submit_lipsync.assert_called_once()
call_kwargs = mk_client.submit_lipsync.call_args.kwargs call_kwargs = mk_client.submit_lipsync.call_args.kwargs
assert call_kwargs["client_token"] == "job-1" assert call_kwargs["client_token"] == "job-1"
assert call_kwargs["audio_url"].endswith("?signed") # CosyVoice 临时 URL 经 _sign_media_url 透传(mock 统一追加 ?signed),
# 自家 OSS 才会被重签,外部 URL 原样透传;job.audio_url 存原始临时 URL
assert call_kwargs["audio_url"] == "https://tts/raw.mp3?signed"
assert job.audio_url == "https://tts/raw.mp3"
session.commit.assert_called() session.commit.assert_called()
session.close.assert_called_once() session.close.assert_called_once()
@@ -387,3 +390,239 @@ class TestSignMediaUrl:
assert result == "https://anything.example.com/a.mp3" assert result == "https://anything.example.com/a.mp3"
fake_storage.get_download_url.assert_not_called() fake_storage.get_download_url.assert_not_called()
class TestPersistOutputVideoTask:
"""persist_output_video_task:下载 MediaKit 临时视频 → 上传自有 OSS → 更新 DB."""
def _make_persist_job(self, **kwargs):
job = MagicMock()
job.id = kwargs.get("job_id", "job-1")
job.user_id = kwargs.get("user_id", "user-1")
job.output_video_url = kwargs.get("output_video_url", "https://temp.mk/output.mp4")
job.updated_at = None
return job
def _persist_patches(self, *, job, video_bytes=b"FAKEMP4", download_side_effect=None, upload_url=None):
"""统一 patchSessionLocal、httpx.Client、storage、_sign_media_url."""
fake_app_db = ModuleType("app.db")
fake_worker_db = ModuleType("worker_app.db")
session, factory = _build_session(job)
fake_app_db.SessionLocal = factory
fake_worker_db.SessionLocal = factory
# httpx.Client 上下文管理器
fake_response = MagicMock()
fake_response.content = video_bytes
fake_response.raise_for_status = MagicMock()
fake_client = MagicMock()
fake_client.get.return_value = fake_response
fake_client_cm = MagicMock()
fake_client_cm.__enter__ = MagicMock(return_value=fake_client)
fake_client_cm.__exit__ = MagicMock(return_value=False)
FakeHttpxClient = MagicMock(return_value=fake_client_cm)
if download_side_effect is not None:
fake_client.get.side_effect = download_side_effect
# storage
storage = MagicMock()
storage.public_url = "https://oss.example.com/"
storage.upload_file.return_value = upload_url or "https://oss.example.com/lipsync-outputs/user-1/job-1.mp4"
# _sign_media_url 内部会调 storage.get_download_url,必须mock返回字符串
_upload_url = upload_url or "https://oss.example.com/lipsync-outputs/user-1/job-1.mp4"
storage.get_download_url.return_value = _upload_url + "?signed"
fake_httpx = ModuleType("httpx")
fake_httpx.Client = FakeHttpxClient
patches = [
patch.dict(
sys.modules,
{"app.db": fake_app_db, "worker_app.db": fake_worker_db, "httpx": fake_httpx},
),
patch("packages.shared.storage.get_shared_storage_service", return_value=storage),
patch("app.tasks.lipsync_tts._sign_media_url", side_effect=lambda url: url + "?signed" if url else url),
]
return session, fake_client, storage, patches
def test_success_download_upload_updates_db(self):
"""正常路径:下载 temp_url → 上传 OSS → 签名 → 写回 DB commit."""
from app.tasks.lipsync_tts import persist_output_video_task
job = self._make_persist_job(output_video_url="https://temp.mk/x.mp4")
session, fake_client, storage, patches = self._persist_patches(
job=job, video_bytes=b"VIDEODATA", upload_url="https://oss.example.com/lipsync-outputs/user-1/job-1.mp4"
)
entered = [p.__enter__() for p in patches]
try:
persist_output_video_task("job-1", "user-1", "https://temp.mk/x.mp4")
finally:
for p in reversed(patches):
p.__exit__(None, None, None)
fake_client.get.assert_called_once_with("https://temp.mk/x.mp4")
storage.upload_file.assert_called_once()
call_args = storage.upload_file.call_args.args
# 上传的 key 必须是 lipsync-outputs/{user_id}/{job_id}.mp4
assert call_args[1] == "lipsync-outputs/user-1/job-1.mp4"
# upload_file 返回永久 URL,再被 _sign_media_url 追加 ?signed
assert job.output_video_url == "https://oss.example.com/lipsync-outputs/user-1/job-1.mp4?signed"
assert job.updated_at is not None
session.commit.assert_called_once()
session.close.assert_called_once()
def test_download_failure_keeps_temp_url_no_commit(self):
"""下载失败(raise)→ 记录 warning、保留 temp_url、不抛异常."""
from app.tasks.lipsync_tts import persist_output_video_task
job = self._make_persist_job(output_video_url="https://temp.mk/x.mp4")
session, fake_client, storage, patches = self._persist_patches(
job=job, download_side_effect=RuntimeError("network down")
)
entered = [p.__enter__() for p in patches]
try:
persist_output_video_task("job-1", "user-1", "https://temp.mk/x.mp4")
finally:
for p in reversed(patches):
p.__exit__(None, None, None)
storage.upload_file.assert_not_called()
# output_video_url 保持原值(temp_url
assert job.output_video_url == "https://temp.mk/x.mp4"
# 内层 except 不会 commit
# 注:若内部发生 commit 说明测试失败
session.close.assert_called_once()
def test_empty_temp_url_skips_persist(self):
"""temp_url 为空 → 直接返回,不下载不上传."""
from app.tasks.lipsync_tts import persist_output_video_task
job = self._make_persist_job(output_video_url="")
session, fake_client, storage, patches = self._persist_patches(job=job)
entered = [p.__enter__() for p in patches]
try:
persist_output_video_task("job-1", "user-1", "")
finally:
for p in reversed(patches):
p.__exit__(None, None, None)
fake_client.get.assert_not_called()
storage.upload_file.assert_not_called()
session.commit.assert_not_called()
session.close.assert_called_once()
def test_job_not_found_returns_early(self):
"""DB 中找不到 job → 直接返回,不抛错."""
from app.tasks.lipsync_tts import persist_output_video_task
session, fake_client, storage, patches = self._persist_patches(job=None)
entered = [p.__enter__() for p in patches]
try:
persist_output_video_task("missing", "user-1", "https://temp.mk/x.mp4")
finally:
for p in reversed(patches):
p.__exit__(None, None, None)
fake_client.get.assert_not_called()
storage.upload_file.assert_not_called()
session.commit.assert_not_called()
session.close.assert_called_once()
class TestLipsyncServiceRefreshCompletedAsyncPersist:
"""refresh_job_status 在 completed 分支异步转存的单元测试(补 0% 覆盖的 316~335 行)."""
def test_refresh_completed_dispatches_persist_task(self):
"""completed 分支:设置 temp_url → commit → dispatch persist_output_video_task.apply_async."""
from app.services.lipsync_service import LipsyncService
mock_job = MagicMock()
mock_job.id = "job-1"
mock_job.user_id = "user-1"
mock_job.mediakit_task_id = "mk-1"
mock_job.status = "submitted"
mock_job.output_video_url = ""
mock_job.output_duration = 0.0
mock_db = MagicMock()
mock_query = MagicMock()
mock_filter = MagicMock()
mock_filter.first.return_value = mock_job
mock_query.filter.return_value = mock_filter
mock_db.query.return_value = mock_query
mock_client = MagicMock()
mock_client.get_task_status.return_value = {
"status": "completed",
"result": {"video_url": "https://temp.mk/out.mp4", "duration": 25.5},
}
fake_persist_task = MagicMock()
svc = LipsyncService(mock_db, client=mock_client, cosyvoice_service=MagicMock())
with patch.dict("sys.modules", {}):
# 直接 patch 懒 import 路径
with patch("app.tasks.lipsync_tts.persist_output_video_task", fake_persist_task, create=False):
# 但懒 import 发生在函数内部 from app.tasks.lipsync_tts import persist_output_video_task
# 通过 patch sys.modules 的方式提供
import sys as _sys
fake_mod = MagicMock()
fake_mod.persist_output_video_task = fake_persist_task
_sys.modules["app.tasks.lipsync_tts"] = fake_mod
try:
result = svc.refresh_job_status("job-1", "user-1")
finally:
_sys.modules.pop("app.tasks.lipsync_tts", None)
assert result.status == "completed"
assert result.output_video_url == "https://temp.mk/out.mp4"
assert result.output_duration == 25.5
mock_db.commit.assert_called()
# 必须在 commit 之后 dispatch
fake_persist_task.apply_async.assert_called_once()
kwargs = fake_persist_task.apply_async.call_args.kwargs
assert kwargs["args"] == ("job-1", "user-1", "https://temp.mk/out.mp4")
def test_refresh_completed_dispatch_exception_does_not_break_return(self):
"""apply_async 抛异常(如 Celery 不可用)→ 捕获 warning,仍返回 completed job."""
from app.services.lipsync_service import LipsyncService
mock_job = MagicMock()
mock_job.id = "job-2"
mock_job.user_id = "user-1"
mock_job.mediakit_task_id = "mk-2"
mock_job.status = "submitted"
mock_job.output_video_url = ""
mock_job.output_duration = 0.0
mock_db = MagicMock()
mock_query = MagicMock()
mock_filter = MagicMock()
mock_filter.first.return_value = mock_job
mock_query.filter.return_value = mock_filter
mock_db.query.return_value = mock_query
mock_client = MagicMock()
mock_client.get_task_status.return_value = {
"status": "completed",
"result": {"video_url": "https://temp.mk/out2.mp4", "duration": 10.0},
}
fake_persist_task = MagicMock()
fake_persist_task.apply_async.side_effect = ConnectionError("celery down")
svc = LipsyncService(mock_db, client=mock_client, cosyvoice_service=MagicMock())
import sys as _sys
fake_mod = MagicMock()
fake_mod.persist_output_video_task = fake_persist_task
_sys.modules["app.tasks.lipsync_tts"] = fake_mod
try:
result = svc.refresh_job_status("job-2", "user-1")
finally:
_sys.modules.pop("app.tasks.lipsync_tts", None)
# 即便 dispatch 失败,主流程不受影响:仍然返回 completed + temp_url
assert result.status == "completed"
assert result.output_video_url == "https://temp.mk/out2.mp4"
fake_persist_task.apply_async.assert_called_once()
+169
View File
@@ -0,0 +1,169 @@
"""AI 数字人 对口型 TTS 预合成接口(#1845)单元测试 — 覆盖 LipsyncService.preview_tts 成功/失败路径.
直接调用 LipsyncService.preview_tts()mock CosyVoiceService / safe_download_bytes / ffprobe
验证返回结构、错误码、与共享 sentence_timings 工具的协作。
"""
import os
from unittest.mock import MagicMock, patch
import pytest
os.environ.setdefault("JWT_SECRET_KEY", "dev-secret-key-for-testing")
def _make_service(
*,
cosyvoice=None,
download_bytes=b"FAKE_MP3_DATA",
download_error=None,
ffprobe_duration=5.0,
timings_result=None,
):
"""构造 LipsyncService 并把 CosyVoiceService/safe_download_bytes/probe/compute 全部 mock 掉。"""
from app.services.lipsync_service import LipsyncService
db = MagicMock()
# 构造唯一的 cosyvoice mock 实例,便于断言
_cosy_inst = MagicMock()
if cosyvoice is None:
_cosy_inst.submit_synthesize_task.return_value = {"audio_url": "https://cosy.example.com/tts.mp3"}
elif isinstance(cosyvoice, Exception):
_cosy_inst.submit_synthesize_task.side_effect = cosyvoice
else:
_cosy_inst.submit_synthesize_task.return_value = cosyvoice
def _fake_get_cosyvoice(self): # noqa: ARG001
return _cosy_inst
def _fake_resolve_voice_id(self, voice_id, user_id): # noqa: ARG001
return voice_id
svc = LipsyncService(db=db, client=MagicMock(), voice_clone_repo=MagicMock())
svc._cosyvoice = _cosy_inst
patch.object(LipsyncService, "_get_cosyvoice", _fake_get_cosyvoice).start()
patch.object(LipsyncService, "_resolve_voice_id", _fake_resolve_voice_id).start()
# mock safe_download_bytes
if download_error is not None:
patch(
"app.services.lipsync_service.safe_download_bytes",
side_effect=download_error,
).start()
else:
patch(
"app.services.lipsync_service.safe_download_bytes",
return_value=download_bytes,
).start()
# mock probe_audio_durationpatch 到 lipsync_service 模块的命名空间)
patch(
"app.services.lipsync_service.probe_audio_duration",
return_value=ffprobe_duration,
).start()
# mock compute_sentence_timings
default_timings = [
{"index": 0, "text": "你好", "start_time": 0.0, "end_time": 1.5},
{"index": 1, "text": "世界", "start_time": 1.5, "end_time": 5.0},
]
patch(
"app.services.lipsync_service.compute_sentence_timings",
return_value=timings_result if timings_result is not None else default_timings,
).start()
svc.__dict__["_test_cosy"] = _cosy_inst
return svc
def test_preview_tts_success():
"""正常路径:TTS 合成成功 → 下载 → ffprobe → 计算 timings,返回完整结构。"""
svc = _make_service(ffprobe_duration=5.0)
try:
result = svc.preview_tts(
user_id="user-1",
voice_id="longxiaochun",
script_text="你好,世界",
speed=1.0,
emotion="natural",
)
assert result["audio_url"] == "https://cosy.example.com/tts.mp3"
assert result["duration"] == 5.0
assert isinstance(result["sentence_timings"], list)
assert len(result["sentence_timings"]) == 2
assert result["sentence_timings"][0]["text"] == "你好"
cosy = svc.__dict__["_test_cosy"]
cosy.submit_synthesize_task.assert_called_once()
kwargs = cosy.submit_synthesize_task.call_args.kwargs
assert kwargs["text"] == "你好,世界"
assert kwargs["voice_id"] == "longxiaochun"
finally:
patch.stopall()
def test_preview_tts_cosyvoice_error():
"""CosyVoice 抛错:应该包装成 MediaKitError 抛出。"""
from app.services.mediakit_client import MediaKitError
from packages.application.cosyvoice_service import CosyVoiceError
svc = _make_service(cosyvoice=CosyVoiceError("cosyvoice down"))
try:
with pytest.raises(MediaKitError):
svc.preview_tts(
user_id="user-1",
voice_id="longxiaochun",
script_text="你好",
)
finally:
patch.stopall()
def test_preview_tts_download_fail_still_returns_url():
"""音频下载失败:不抛错,返回 audio_url + 空 timings,前端仍能继续(降级)。"""
svc = _make_service(download_error=RuntimeError("network down"))
try:
result = svc.preview_tts(
user_id="user-1",
voice_id="longxiaochun",
script_text="你好,世界",
)
assert result["audio_url"] == "https://cosy.example.com/tts.mp3"
assert result["duration"] == 0.0
assert result["sentence_timings"] == []
finally:
patch.stopall()
def test_preview_tts_ffprobe_zero_duration():
"""ffprobe 返回 0timings 为空,不抛错。"""
svc = _make_service(ffprobe_duration=0.0)
try:
result = svc.preview_tts(
user_id="user-1",
voice_id="longxiaochun",
script_text="你好",
)
assert result["audio_url"]
assert result["duration"] == 0.0
assert result["sentence_timings"] == []
finally:
patch.stopall()
def test_preview_tts_no_audio_url_in_response():
"""CosyVoice 返回无 audio_url:抛 MediaKitError TTSNoAudio。"""
from app.services.mediakit_client import MediaKitError
svc = _make_service(cosyvoice={"audio_url": ""})
try:
with pytest.raises(MediaKitError) as exc_info:
svc.preview_tts(
user_id="user-1",
voice_id="longxiaochun",
script_text="你好",
)
assert exc_info.value.code == "TTSNoAudio"
finally:
patch.stopall()
+169
View File
@@ -0,0 +1,169 @@
"""Tests for sentence timing functions (now in packages/domain/sentence_timings.py)."""
import os
import subprocess
import tempfile
import unittest
from unittest.mock import MagicMock, patch
from packages.domain.sentence_timings import compute_sentence_timings as _compute_sentence_timings
from packages.domain.sentence_timings import estimate_sentence_timings_by_chars as _estimate_sentence_timings_by_chars
from packages.domain.sentence_timings import split_script_into_sentences as _split_script_into_sentences
class TestSplitScriptIntoSentences(unittest.TestCase):
"""Tests for _split_script_into_sentences."""
def test_empty_string(self):
self.assertEqual(_split_script_into_sentences(""), [])
def test_none(self):
self.assertEqual(_split_script_into_sentences(None), [])
def test_whitespace_only(self):
self.assertEqual(_split_script_into_sentences(" \n "), [])
def test_single_sentence(self):
self.assertEqual(_split_script_into_sentences("你好世界。"), ["你好世界"])
def test_multiple_sentences_chinese(self):
result = _split_script_into_sentences("第一句。第二句!第三句?")
self.assertEqual(result, ["第一句", "第二句", "第三句"])
def test_english_punctuation(self):
result = _split_script_into_sentences("Hello World! How are you?")
self.assertEqual(result, ["Hello World", "How are you"])
def test_semicolons(self):
result = _split_script_into_sentences("第一部分;第二部分;第三部分")
self.assertEqual(result, ["第一部分", "第二部分", "第三部分"])
def test_newlines(self):
result = _split_script_into_sentences("第一行\n第二行\n第三行")
self.assertEqual(result, ["第一行", "第二行", "第三行"])
def test_no_trailing_punctuation(self):
result = _split_script_into_sentences("没有标点的句子")
self.assertEqual(result, ["没有标点的句子"])
class TestEstimateSentenceTimingsByChars(unittest.TestCase):
"""Tests for _estimate_sentence_timings_by_chars."""
def test_empty_sentences(self):
self.assertEqual(_estimate_sentence_timings_by_chars([], 10.0), [])
def test_zero_duration(self):
self.assertEqual(_estimate_sentence_timings_by_chars(["hello"], 0), [])
def test_negative_duration(self):
self.assertEqual(_estimate_sentence_timings_by_chars(["hello"], -5.0), [])
def test_single_sentence(self):
result = _estimate_sentence_timings_by_chars(["hello"], 10.0)
self.assertEqual(len(result), 1)
self.assertAlmostEqual(result[0]["start_time"], 0.0)
self.assertAlmostEqual(result[0]["end_time"], 10.0)
def test_two_equal_sentences(self):
result = _estimate_sentence_timings_by_chars(["你好", "世界"], 10.0)
self.assertEqual(len(result), 2)
self.assertAlmostEqual(result[0]["start_time"], 0.0)
self.assertAlmostEqual(result[0]["end_time"], 5.0)
self.assertAlmostEqual(result[1]["start_time"], 5.0)
self.assertAlmostEqual(result[1]["end_time"], 10.0)
def test_unequal_char_distribution(self):
result = _estimate_sentence_timings_by_chars(["ABCD", "EF"], 9.0)
self.assertEqual(len(result), 2)
self.assertAlmostEqual(result[0]["start_time"], 0.0)
self.assertAlmostEqual(result[0]["end_time"], 6.0) # 4/6 * 9 = 6
self.assertAlmostEqual(result[1]["start_time"], 6.0)
self.assertAlmostEqual(result[1]["end_time"], 9.0)
def test_timing_structure(self):
result = _estimate_sentence_timings_by_chars(["句子一", "句子二"], 6.0)
for item in result:
self.assertIn("index", item)
self.assertIn("text", item)
self.assertIn("start_time", item)
self.assertIn("end_time", item)
class TestComputeSentenceTimings(unittest.TestCase):
"""Tests for _compute_sentence_timings."""
def test_empty_script_returns_empty(self):
self.assertEqual(_compute_sentence_timings(b"fake_audio", "", 10.0), [])
def test_none_script_returns_empty(self):
self.assertEqual(_compute_sentence_timings(b"fake_audio", None, 10.0), [])
@patch("os.unlink")
@patch.object(tempfile, "NamedTemporaryFile")
@patch.object(subprocess, "run")
def test_silence_detection_insufficient_fallback(self, mock_run, mock_tmpfile, mock_unlink):
"""When silence detection finds too few points, fallback to char estimation."""
mock_run.return_value = MagicMock(stderr="", returncode=0)
mock_tmp = MagicMock()
mock_tmp.name = "/tmp/fake.mp3"
mock_tmp.__enter__ = MagicMock(return_value=mock_tmp)
mock_tmp.__exit__ = MagicMock(return_value=False)
mock_tmpfile.return_value = mock_tmp
result = _compute_sentence_timings(b"fake_audio", "第一句。第二句。第三句。", 10.0)
# Should fallback to char estimation with 3 sentences
self.assertEqual(len(result), 3)
self.assertAlmostEqual(result[0]["start_time"], 0.0)
@patch("os.unlink")
@patch.object(tempfile, "NamedTemporaryFile")
@patch.object(subprocess, "run")
def test_silence_detection_with_enough_points(self, mock_run, mock_tmpfile, mock_unlink):
"""When silence detection finds enough points, use them for boundaries."""
mock_run.return_value = MagicMock(
stderr="[silencedetect] silence_end: 3.5 | silence_duration: 0.4\n"
"[silencedetect] silence_end: 7.0 | silence_duration: 0.3\n",
returncode=0,
)
mock_tmp = MagicMock()
mock_tmp.name = "/tmp/fake.mp3"
mock_tmp.__enter__ = MagicMock(return_value=mock_tmp)
mock_tmp.__exit__ = MagicMock(return_value=False)
mock_tmpfile.return_value = mock_tmp
result = _compute_sentence_timings(b"fake_audio", "第一句。第二句。第三句。", 10.0)
self.assertEqual(len(result), 3)
self.assertAlmostEqual(result[0]["start_time"], 0.0)
self.assertAlmostEqual(result[0]["end_time"], 3.5)
self.assertAlmostEqual(result[1]["start_time"], 3.5)
self.assertAlmostEqual(result[1]["end_time"], 7.0)
self.assertAlmostEqual(result[2]["start_time"], 7.0)
self.assertAlmostEqual(result[2]["end_time"], 10.0)
@patch("os.unlink")
@patch.object(tempfile, "NamedTemporaryFile")
@patch.object(subprocess, "run")
def test_ffmpeg_exception_fallback(self, mock_run, mock_tmpfile, mock_unlink):
"""When ffmpeg raises an exception, fallback to char estimation."""
mock_run.side_effect = Exception("ffmpeg not found")
mock_tmp = MagicMock()
mock_tmp.name = "/tmp/fake.mp3"
mock_tmp.__enter__ = MagicMock(return_value=mock_tmp)
mock_tmp.__exit__ = MagicMock(return_value=False)
mock_tmpfile.return_value = mock_tmp
result = _compute_sentence_timings(b"fake_audio", "句子一。句子二。", 6.0)
# Should fallback to char estimation
self.assertEqual(len(result), 2)
self.assertAlmostEqual(result[0]["start_time"], 0.0)
self.assertAlmostEqual(result[0]["end_time"], 3.0)
self.assertAlmostEqual(result[1]["start_time"], 3.0)
self.assertAlmostEqual(result[1]["end_time"], 6.0)
if __name__ == "__main__":
unittest.main()
+147 -7
View File
@@ -13,6 +13,7 @@
from __future__ import annotations from __future__ import annotations
import os
import unittest import unittest
from dataclasses import FrozenInstanceError from dataclasses import FrozenInstanceError
from unittest.mock import patch from unittest.mock import patch
@@ -29,10 +30,12 @@ from packages.domain.video_filter_builder import (
ClipFilterChain, ClipFilterChain,
_escape_drawtext_text, _escape_drawtext_text,
_resolve_font_path, _resolve_font_path,
build_broll_overlay_filter,
build_clip_filter, build_clip_filter,
build_concat_filter, build_concat_filter,
build_filter_complex, build_filter_complex,
build_title_drawtext_filter, build_title_drawtext_filter,
build_title_overlay_filter,
build_xfade_filter, build_xfade_filter,
chain_filters, chain_filters,
has_audio, has_audio,
@@ -902,9 +905,10 @@ class TestResolveFontPath(unittest.TestCase):
@patch("os.path.isfile") @patch("os.path.isfile")
def test_unknown_font_fallback(self, mock_isfile): def test_unknown_font_fallback(self, mock_isfile):
mock_isfile.side_effect = lambda p: "DejaVu" in p # DejaVuSans 已从 fallback 列表移除(不支持 CJK),用 VF 路径模拟
mock_isfile.side_effect = lambda p: "NotoSansSC-VF" in p
result = _resolve_font_path("UnknownFont") result = _resolve_font_path("UnknownFont")
self.assertIn("DejaVu", result) self.assertIn("NotoSansSC-VF", result)
@patch("os.path.isfile") @patch("os.path.isfile")
def test_no_fonts_available(self, mock_isfile): def test_no_fonts_available(self, mock_isfile):
@@ -927,9 +931,11 @@ class TestResolveFontPath(unittest.TestCase):
@patch("os.path.isfile") @patch("os.path.isfile")
def test_font_fallback_skips_nonexistent(self, mock_isfile): def test_font_fallback_skips_nonexistent(self, mock_isfile):
mock_isfile.side_effect = lambda p: "DejaVu" in p # 所有中文字体路径都不存在时,fallback 返回第一个存在的文件;
# DejaVuSans 已从列表移除(不支持 CJK),使用 VF 字体路径模拟存在文件
mock_isfile.side_effect = lambda p: "NotoSansSC-VF" in p
result = _resolve_font_path("不存在字体") result = _resolve_font_path("不存在字体")
self.assertIn("DejaVu", result) self.assertIn("NotoSansSC-VF", result)
class TestDrawtextFontFileIncluded(unittest.TestCase): class TestDrawtextFontFileIncluded(unittest.TestCase):
@@ -1028,6 +1034,37 @@ class TestDrawtextBoldFalse(unittest.TestCase):
self.assertIsNotNone(result) self.assertIsNotNone(result)
self.assertNotIn("font=bold", result) self.assertNotIn("font=bold", result)
def test_bold_true_does_not_use_font_bold_param(self):
"""粗体模式不得使用 `font=bold`——该参数无效,会导致 filter_complex 解析失败(exit 234)。"""
result = build_title_drawtext_filter({"text": "标题", "bold": True})
self.assertIsNotNone(result)
self.assertNotIn("font=bold", result)
# 粗体应通过 borderw 实现
self.assertIn("borderw=", result)
@patch("packages.domain.video_filter_builder._resolve_font_path")
def test_bold_default_uses_black_stroke_when_no_bold_font(self, mock_font):
"""默认 bold=true 且无 Bold 字体文件时,使用黑色细描边(borderw=2 + 黑),
不得使用与文字同色的 borderw>=3(否则会造成竖屏小字号重影)。"""
mock_font.return_value = "" # 无粗体字体
result = build_title_drawtext_filter({"text": "标题"})
self.assertIsNotNone(result)
self.assertIn("borderw=2", result)
# 黑描边:要么是 black 关键字,要么是 000000
self.assertTrue("bordercolor=black" in result or "bordercolor=000000" in result)
self.assertNotIn("borderw=3", result)
@patch("packages.domain.video_filter_builder._resolve_font_path")
def test_bold_with_user_stroke_preserves_user_color(self, mock_font):
"""用户显式开启 stroke 时,stroke 颜色/宽度优先于默认粗体黑边。"""
mock_font.return_value = ""
result = build_title_drawtext_filter(
{"text": "标题", "bold": True, "stroke": {"width": 4, "color": "#ffffff"}}
)
self.assertIsNotNone(result)
self.assertIn("borderw=4", result)
self.assertIn("bordercolor=ffffff", result) # 去掉 # 前缀
class TestDrawtextPositionBranches(unittest.TestCase): class TestDrawtextPositionBranches(unittest.TestCase):
"""位置相关分支覆盖。""" """位置相关分支覆盖。"""
@@ -1054,12 +1091,23 @@ class TestDrawtextPositionBranches(unittest.TestCase):
self.assertIn("y=h-text_h-50", result) self.assertIn("y=h-text_h-50", result)
@patch("packages.domain.video_filter_builder._resolve_font_path") @patch("packages.domain.video_filter_builder._resolve_font_path")
def test_position_custom_with_float_coords(self, mock_font): def test_position_custom_with_percentage_coords(self, mock_font):
"""自定义位置:百分比坐标转换为 drawtext 表达式."""
mock_font.return_value = ""
# pos_x=50, pos_y=30 → x=(w-text_w)*0.5000, y=(h-text_h)*0.3000
result = build_title_drawtext_filter({"text": "标题", "position": "custom", "pos_x": 50, "pos_y": 30})
self.assertIsNotNone(result)
self.assertIn("x=(w-text_w)*0.5000", result)
self.assertIn("y=(h-text_h)*0.3000", result)
@patch("packages.domain.video_filter_builder._resolve_font_path")
def test_position_custom_clamped_to_100(self, mock_font):
"""自定义位置:超过100的坐标被截断到100%."""
mock_font.return_value = "" mock_font.return_value = ""
result = build_title_drawtext_filter({"text": "标题", "position": "custom", "pos_x": 100.7, "pos_y": 200.3}) result = build_title_drawtext_filter({"text": "标题", "position": "custom", "pos_x": 100.7, "pos_y": 200.3})
self.assertIsNotNone(result) self.assertIsNotNone(result)
self.assertIn("x=100", result) self.assertIn("x=(w-text_w)*1.0000", result)
self.assertIn("y=200", result) self.assertIn("y=(h-text_h)*1.0000", result)
@patch("packages.domain.video_filter_builder._resolve_font_path") @patch("packages.domain.video_filter_builder._resolve_font_path")
def test_position_custom_bool_coords_fallback(self, mock_font): def test_position_custom_bool_coords_fallback(self, mock_font):
@@ -1116,5 +1164,97 @@ class TestDrawtextNotDictConfig(unittest.TestCase):
self.assertIsNone(build_title_drawtext_filter([1, 2, 3])) self.assertIsNone(build_title_drawtext_filter([1, 2, 3]))
class TestTitleOverlay(unittest.TestCase):
"""build_title_overlay_filter 单元测试(WYSIWYG PNG 叠加路径)。"""
def test_overlay_filter_format(self):
"""PNG 文件存在时返回正确的 overlay 滤镜字符串。"""
import tempfile
with tempfile.NamedTemporaryFile(suffix=".png", delete=False) as tmp:
tmp.write(b"\x89PNG\r\n\x1a\n")
tmp_path = tmp.name
try:
result = build_title_overlay_filter(
{"text": "标题"},
output_width=720,
output_height=1280,
title_png_path=tmp_path,
title_input_label="[2:v]",
base_label="[vout]",
output_label="vout_titled",
)
self.assertIsNotNone(result)
self.assertIn("[vout][2:v]overlay=0:0[vout_titled]", result)
finally:
os.unlink(tmp_path)
def test_overlay_default_labels(self):
"""不传 label 参数时使用默认 [0:v] / [1:v] / vout_titled。"""
import tempfile
with tempfile.NamedTemporaryFile(suffix=".png", delete=False) as tmp:
tmp.write(b"\x89PNG\r\n\x1a\n")
tmp_path = tmp.name
try:
result = build_title_overlay_filter(
{"text": "标题"},
output_width=720,
output_height=1280,
title_png_path=tmp_path,
)
self.assertEqual(result, "[0:v][1:v]overlay=0:0[vout_titled]")
finally:
os.unlink(tmp_path)
def test_overlay_returns_none_when_png_missing(self):
"""PNG 文件不存在时返回 None,供调用方降级到 drawtext。"""
result = build_title_overlay_filter(
{"text": "标题"},
output_width=720,
output_height=1280,
title_png_path="/nonexistent/path/title.png",
)
self.assertIsNone(result)
def test_overlay_returns_none_for_empty_config(self):
"""title_config 为空/非 dict 时返回 None。"""
import tempfile
with tempfile.NamedTemporaryFile(suffix=".png", delete=False) as tmp:
tmp.write(b"\x89PNG\r\n\x1a\n")
tmp_path = tmp.name
try:
self.assertIsNone(
build_title_overlay_filter(
None,
output_width=720,
output_height=1280,
title_png_path=tmp_path,
)
)
self.assertIsNone(
build_title_overlay_filter(
"not a dict",
output_width=720,
output_height=1280,
title_png_path=tmp_path,
)
)
finally:
os.unlink(tmp_path)
def test_overlay_returns_none_for_empty_path(self):
"""title_png_path 为空字符串时返回 None。"""
self.assertIsNone(
build_title_overlay_filter(
{"text": "标题"},
output_width=720,
output_height=1280,
title_png_path="",
)
)
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()