902 lines
26 KiB
Markdown
902 lines
26 KiB
Markdown
# 图片内容审核系统升级方案
|
||
## 从混合同步/异步架构升级到统一异步架构
|
||
|
||
---
|
||
|
||
## 🎯 升级概述
|
||
|
||
本升级方案旨在将现有的图片内容审核系统从 **混合同步/异步架构** 升级为 **统一异步架构**，解决同步审核阻塞异步模板执行的核心问题，并实现平台差异抹平。
|
||
|
||
### 🚨 核心问题
|
||
|
||
**现状**：抖音图片审核失败 `抖音图片审核失败: 抖音审核API调用失败: 无效的图片URL`
|
||
**根因**：
|
||
1. 图片URL验证逻辑过于严格（已修复）
|
||
2. **同步审核与异步模板执行的架构矛盾**（本次升级重点）
|
||
3. 平台差异导致的不一致用户体验
|
||
|
||
---
|
||
|
||
## 📊 现状分析
|
||
|
||
### ✅ 已实现功能
|
||
|
||
1. **基础内容审核架构**：
|
||
- ✅ `UnifiedContentService` 统一服务
|
||
- ✅ `ContentAdapterFactory` 适配器工厂
|
||
- ✅ `DouyinContentAdapter` 抖音适配器
|
||
- ✅ `WechatContentAdapter` 微信适配器
|
||
- ✅ `BaseContentAdapter` 基础适配器
|
||
- ✅ 审核日志记录 `ContentAuditLogEntity`
|
||
|
||
2. **模板执行系统**：
|
||
- ✅ `TemplateController.executeTemplateByCode`
|
||
- ✅ `TemplateExecutionEntity` 执行记录
|
||
- ✅ 基础状态管理
|
||
|
||
3. **环境配置**：
|
||
- ✅ 审核回调URL配置
|
||
- ✅ 平台API配置
|
||
|
||
### ❌ 存在问题
|
||
|
||
1. **架构问题**：
|
||
```typescript
|
||
// 🚨 问题：同步等待异步审核结果
|
||
const auditResult = await this.unifiedContentService.auditImage(platform, auditData);
|
||
if (auditResult.conclusion !== AuditConclusion.PASS) {
|
||
throw new HttpException('图片审核未通过', HttpStatus.FORBIDDEN);
|
||
}
|
||
// 继续执行模板...
|
||
```
|
||
|
||
2. **状态枚举缺失**：
|
||
```typescript
|
||
// 当前状态枚举
|
||
enum ExecutionStatus {
|
||
PENDING = 'pending',
|
||
PROCESSING = 'processing',
|
||
COMPLETED = 'completed',
|
||
FAILED = 'failed',
|
||
CANCELLED = 'cancelled',
|
||
}
|
||
|
||
// ❌ 缺失：PENDING_AUDIT, AUDIT_FAILED
|
||
```
|
||
|
||
3. **实体字段缺失**：
|
||
```typescript
|
||
// TemplateExecutionEntity 缺失字段
|
||
// ❌ auditTaskId?: string; // 审核任务ID关联
|
||
```
|
||
|
||
4. **平台差异未抹平**：
|
||
- 微信同步API直接返回结果
|
||
- 抖音异步API需要回调处理
|
||
- 业务层需要区别处理
|
||
|
||
---
|
||
|
||
## 🚀 升级目标架构
|
||
|
||
### 🎯 统一异步执行流程
|
||
|
||
```mermaid
|
||
graph TD
|
||
A[用户提交] --> B[提交审核<br/>所有平台返回PROCESSING]
|
||
B --> C[创建执行记录<br/>PENDING_AUDIT状态]
|
||
C --> D[立即返回executionId]
|
||
|
||
B --> E[微信同步平台<br/>setImmediate触发回调]
|
||
B --> F[抖音异步平台<br/>外部回调]
|
||
|
||
E --> G[审核完成回调]
|
||
F --> G
|
||
|
||
G --> H{审核结果}
|
||
H -->|PASS| I[更新状态PROCESSING<br/>开始执行模板]
|
||
H -->|REJECT| J[更新状态AUDIT_FAILED]
|
||
|
||
I --> K[执行完成<br/>COMPLETED状态]
|
||
|
||
D --> L[用户轮询状态]
|
||
L --> M[返回最新状态]
|
||
```
|
||
|
||
---
|
||
|
||
## 📋 升级计划
|
||
|
||
### 阶段一：核心架构升级 (1-2天)
|
||
|
||
#### 1.1 扩展执行状态枚举
|
||
```typescript
|
||
// 📁 src/entities/template-execution.entity.ts
|
||
export enum ExecutionStatus {
|
||
PENDING = 'pending',
|
||
PENDING_AUDIT = 'pending_audit', // 🆕 待审核
|
||
AUDIT_FAILED = 'audit_failed', // 🆕 审核失败
|
||
PROCESSING = 'processing',
|
||
COMPLETED = 'completed',
|
||
FAILED = 'failed',
|
||
CANCELLED = 'cancelled',
|
||
}
|
||
```
|
||
|
||
#### 1.2 扩展模板执行实体
|
||
```typescript
|
||
// 📁 src/entities/template-execution.entity.ts
|
||
@Entity('template_executions')
|
||
export class TemplateExecutionEntity {
|
||
// 现有字段...
|
||
|
||
/** 🆕 审核任务ID - 关联审核任务，用于回调匹配 */
|
||
@Column({ name: 'audit_task_id', nullable: true })
|
||
auditTaskId?: string;
|
||
|
||
/** 🔄 修改默认状态 */
|
||
@Column({
|
||
type: 'enum',
|
||
enum: ExecutionStatus,
|
||
default: ExecutionStatus.PENDING_AUDIT, // 🔄 改为待审核
|
||
})
|
||
status: ExecutionStatus;
|
||
}
|
||
```
|
||
|
||
#### 1.3 创建增强基础适配器
|
||
```typescript
|
||
// 📁 src/content-moderation/adapters/enhanced-base-content.adapter.ts
|
||
export abstract class EnhancedBaseContentAdapter extends BaseContentAdapter {
|
||
abstract readonly isSyncPlatform: boolean;
|
||
|
||
async auditImage(auditData: ImageAuditRequest): Promise<ContentAuditResult> {
|
||
// 1. 调用平台API
|
||
const platformResult = await this.callPlatformAuditAPI(auditData);
|
||
|
||
// 2. 🎯 统一返回PROCESSING状态
|
||
const unifiedResult = {
|
||
taskId: auditData.taskId,
|
||
status: AuditStatus.PROCESSING,
|
||
conclusion: AuditConclusion.UNCERTAIN,
|
||
};
|
||
|
||
// 3. 同步平台立即触发回调
|
||
if (this.isSyncPlatform) {
|
||
setImmediate(() => this.simulateCallback(platformResult));
|
||
}
|
||
|
||
return unifiedResult;
|
||
}
|
||
|
||
protected abstract callPlatformAuditAPI(auditData: ImageAuditRequest): Promise<any>;
|
||
protected abstract formatCallbackData(platformResult: any): any;
|
||
}
|
||
```
|
||
|
||
#### 1.4 创建增强微信适配器
|
||
```typescript
|
||
// 📁 src/content-moderation/adapters/enhanced-wechat-content.adapter.ts
|
||
export class EnhancedWechatContentAdapter extends EnhancedBaseContentAdapter {
|
||
readonly isSyncPlatform = true;
|
||
|
||
protected async callPlatformAuditAPI(auditData: ImageAuditRequest) {
|
||
// 调用微信同步API
|
||
const response = await this.callWechatSyncAPI(auditData);
|
||
return response;
|
||
}
|
||
|
||
protected formatCallbackData(platformResult: any) {
|
||
// 格式化为标准回调格式
|
||
return {
|
||
task_id: platformResult.taskId,
|
||
status: 1, // 完成
|
||
conclusion: platformResult.isPass ? 1 : 2,
|
||
// ...
|
||
};
|
||
}
|
||
}
|
||
```
|
||
|
||
### 阶段二：控制器升级 (1天)
|
||
|
||
#### 2.1 创建增强模板控制器
|
||
```typescript
|
||
// 📁 src/controllers/enhanced-template.controller.ts
|
||
@Controller('enhanced/templates')
|
||
export class EnhancedTemplateController {
|
||
|
||
@Post('code/:code/execute')
|
||
async executeTemplateByCode(
|
||
@Param('code') code: string,
|
||
@Body() body: { imageUrl: string },
|
||
@Request() req,
|
||
) {
|
||
// 1. 🎯 提交审核（统一返回PROCESSING）
|
||
const auditResult = await this.unifiedContentService.auditImage(platform, auditData);
|
||
|
||
// 2. 🎯 创建执行记录（PENDING_AUDIT状态）
|
||
const execution = await this.executionRepository.save({
|
||
templateId: templateConfig.id,
|
||
userId,
|
||
auditTaskId: auditResult.taskId, // 🆕 关联审核任务
|
||
status: ExecutionStatus.PENDING_AUDIT, // 🆕 待审核状态
|
||
// ...
|
||
});
|
||
|
||
// 3. 🎯 立即返回，不等待审核结果
|
||
return ResponseUtil.success({
|
||
executionId: execution.id,
|
||
auditTaskId: auditResult.taskId,
|
||
status: 'pending_audit',
|
||
}, '模板执行已提交，正在进行图片审核');
|
||
}
|
||
|
||
// 🎯 审核完成回调处理
|
||
async handleAuditComplete(auditTaskId: string, auditResult: ContentAuditResult) {
|
||
const execution = await this.findByAuditTaskId(auditTaskId);
|
||
|
||
if (auditResult.conclusion === AuditConclusion.PASS) {
|
||
await this.startTemplateExecution(execution);
|
||
} else {
|
||
await this.updateStatus(execution.id, ExecutionStatus.AUDIT_FAILED);
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
#### 2.2 增强状态查询接口
|
||
```typescript
|
||
@Get('executions/:executionId/status')
|
||
async getExecutionStatus(@Param('executionId') executionId: number) {
|
||
const execution = await this.executionRepository.findOne({
|
||
where: { id: executionId },
|
||
relations: ['template'],
|
||
});
|
||
|
||
// 🎯 如果是待审核状态，主动查询最新审核结果
|
||
if (execution.status === ExecutionStatus.PENDING_AUDIT) {
|
||
const auditResult = await this.unifiedContentService.queryAuditResult(
|
||
execution.platform,
|
||
execution.auditTaskId
|
||
);
|
||
|
||
if (auditResult.status === AuditStatus.COMPLETED) {
|
||
await this.handleAuditComplete(execution.auditTaskId, auditResult);
|
||
// 重新查询更新后的状态
|
||
}
|
||
}
|
||
|
||
return ResponseUtil.success({
|
||
executionId: execution.id,
|
||
status: execution.status,
|
||
statusDescription: this.getStatusDescription(execution.status),
|
||
// ...
|
||
});
|
||
}
|
||
```
|
||
|
||
### 阶段三：回调集成升级 (1天)
|
||
|
||
#### 3.1 增强回调处理
|
||
```typescript
|
||
// 📁 src/content-moderation/adapters/enhanced-wechat-content.adapter.ts
|
||
async handleAuditCallback(callbackData: any): Promise<void> {
|
||
try {
|
||
// 1. 解析回调数据
|
||
const auditResult = this.parseCallbackData(callbackData);
|
||
|
||
// 2. 更新审核日志
|
||
await this.updateAuditLog(auditResult.taskId, auditResult);
|
||
|
||
// 3. 🎯 通知模板执行系统
|
||
await this.notifyTemplateExecution(auditResult.taskId, auditResult);
|
||
|
||
} catch (error) {
|
||
console.error('处理审核回调失败:', error);
|
||
}
|
||
}
|
||
|
||
private async notifyTemplateExecution(taskId: string, auditResult: ContentAuditResult) {
|
||
// 🎯 集成点：调用模板执行控制器的回调处理
|
||
const templateController = Container.get(EnhancedTemplateController);
|
||
await templateController.handleAuditComplete(taskId, auditResult);
|
||
}
|
||
```
|
||
|
||
#### 3.2 适配器工厂升级
|
||
```typescript
|
||
// 📁 src/content-moderation/services/content-adapter.factory.ts
|
||
@Injectable()
|
||
export class ContentAdapterFactory {
|
||
|
||
getAdapter(platform: PlatformType): IContentModerationAdapter {
|
||
switch (platform) {
|
||
case PlatformType.BYTEDANCE:
|
||
return this.douyinAdapter; // 原生异步
|
||
case PlatformType.WECHAT:
|
||
return this.enhancedWechatAdapter; // 🆕 使用增强版
|
||
default:
|
||
throw new BadRequestException(`Unsupported platform: ${platform}`);
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
### 阶段四：数据库迁移 (0.5天)
|
||
|
||
#### 4.1 MySQL数据库迁移脚本
|
||
|
||
**📁 创建 TypeORM 迁移文件：**
|
||
```typescript
|
||
// migrations/1726000000000-UpgradeToUnifiedAsyncArchitecture.ts
|
||
import { MigrationInterface, QueryRunner } from 'typeorm';
|
||
|
||
/**
|
||
* 升级到统一异步架构
|
||
* 1. 添加审核任务ID字段
|
||
* 2. 扩展状态枚举支持审核状态
|
||
* 3. 修改默认状态为待审核
|
||
* 4. 添加相关索引
|
||
*/
|
||
export class UpgradeToUnifiedAsyncArchitecture1726000000000 implements MigrationInterface {
|
||
name = 'UpgradeToUnifiedAsyncArchitecture1726000000000';
|
||
|
||
public async up(queryRunner: QueryRunner): Promise<void> {
|
||
// 1. 添加审核任务ID字段
|
||
await queryRunner.query(`
|
||
ALTER TABLE \`template_executions\`
|
||
ADD COLUMN \`audit_task_id\` VARCHAR(255) NULL
|
||
COMMENT '审核任务ID，用于关联审核记录'
|
||
`);
|
||
|
||
// 2. 扩展状态枚举 - MySQL需要重新定义整个ENUM
|
||
await queryRunner.query(`
|
||
ALTER TABLE \`template_executions\`
|
||
MODIFY COLUMN \`status\` ENUM(
|
||
'pending',
|
||
'pending_audit',
|
||
'audit_failed',
|
||
'processing',
|
||
'completed',
|
||
'failed',
|
||
'cancelled'
|
||
) NOT NULL DEFAULT 'pending_audit'
|
||
`);
|
||
|
||
// 3. 更新现有记录的状态：将pending改为pending_audit
|
||
await queryRunner.query(`
|
||
UPDATE \`template_executions\`
|
||
SET \`status\` = 'pending_audit'
|
||
WHERE \`status\` = 'pending'
|
||
`);
|
||
|
||
// 4. 添加索引 - 使用反引号包围字段名
|
||
await queryRunner.query(`
|
||
CREATE INDEX \`IDX_template_executions_audit_task_id\`
|
||
ON \`template_executions\`(\`audit_task_id\`)
|
||
`);
|
||
|
||
await queryRunner.query(`
|
||
CREATE INDEX \`IDX_template_executions_status_updated\`
|
||
ON \`template_executions\`(\`status\`, \`updatedAt\`)
|
||
`);
|
||
|
||
// 5. 添加复合索引以优化查询性能
|
||
await queryRunner.query(`
|
||
CREATE INDEX \`IDX_template_executions_user_status\`
|
||
ON \`template_executions\`(\`userId\`, \`status\`, \`createdAt\`)
|
||
`);
|
||
}
|
||
|
||
public async down(queryRunner: QueryRunner): Promise<void> {
|
||
// 回滚步骤：删除索引和字段，恢复原始状态枚举
|
||
|
||
// 1. 删除新增的索引
|
||
await queryRunner.query(`DROP INDEX \`IDX_template_executions_audit_task_id\` ON \`template_executions\``);
|
||
await queryRunner.query(`DROP INDEX \`IDX_template_executions_status_updated\` ON \`template_executions\``);
|
||
await queryRunner.query(`DROP INDEX \`IDX_template_executions_user_status\` ON \`template_executions\``);
|
||
|
||
// 2. 将pending_audit状态改回pending（数据迁移）
|
||
await queryRunner.query(`
|
||
UPDATE \`template_executions\`
|
||
SET \`status\` = 'pending'
|
||
WHERE \`status\` IN ('pending_audit', 'audit_failed')
|
||
`);
|
||
|
||
// 3. 恢复原始状态枚举
|
||
await queryRunner.query(`
|
||
ALTER TABLE \`template_executions\`
|
||
MODIFY COLUMN \`status\` ENUM(
|
||
'pending',
|
||
'processing',
|
||
'completed',
|
||
'failed',
|
||
'cancelled'
|
||
) NOT NULL DEFAULT 'pending'
|
||
`);
|
||
|
||
// 4. 删除审核任务ID字段
|
||
await queryRunner.query(`ALTER TABLE \`template_executions\` DROP COLUMN \`audit_task_id\``);
|
||
}
|
||
}
|
||
```
|
||
|
||
**📁 备用手动SQL脚本（如需要）：**
|
||
```sql
|
||
-- migrations/manual-upgrade-unified-async.sql
|
||
-- 手动执行时使用（建议使用TypeORM迁移）
|
||
|
||
-- 备份现有数据
|
||
CREATE TABLE template_executions_backup_20241205 AS
|
||
SELECT * FROM template_executions;
|
||
|
||
-- 添加审核任务ID字段
|
||
ALTER TABLE `template_executions`
|
||
ADD COLUMN `audit_task_id` VARCHAR(255) NULL
|
||
COMMENT '审核任务ID，用于关联审核记录';
|
||
|
||
-- 扩展状态枚举
|
||
ALTER TABLE `template_executions`
|
||
MODIFY COLUMN `status` ENUM(
|
||
'pending',
|
||
'pending_audit',
|
||
'audit_failed',
|
||
'processing',
|
||
'completed',
|
||
'failed',
|
||
'cancelled'
|
||
) NOT NULL DEFAULT 'pending_audit';
|
||
|
||
-- 更新现有数据
|
||
UPDATE `template_executions`
|
||
SET `status` = 'pending_audit'
|
||
WHERE `status` = 'pending';
|
||
|
||
-- 添加索引
|
||
CREATE INDEX `IDX_template_executions_audit_task_id`
|
||
ON `template_executions`(`audit_task_id`);
|
||
|
||
CREATE INDEX `IDX_template_executions_status_updated`
|
||
ON `template_executions`(`status`, `updatedAt`);
|
||
|
||
CREATE INDEX `IDX_template_executions_user_status`
|
||
ON `template_executions`(`userId`, `status`, `createdAt`);
|
||
|
||
-- 验证迁移结果
|
||
SELECT
|
||
COUNT(*) as total_records,
|
||
COUNT(CASE WHEN status = 'pending_audit' THEN 1 END) as pending_audit_count,
|
||
COUNT(CASE WHEN audit_task_id IS NOT NULL THEN 1 END) as with_audit_task_id
|
||
FROM template_executions;
|
||
```
|
||
|
||
#### 4.2 数据迁移脚本
|
||
```typescript
|
||
// 📁 scripts/migrate-existing-executions.ts
|
||
import { getRepository } from 'typeorm';
|
||
import { TemplateExecutionEntity, ExecutionStatus } from '../src/entities/template-execution.entity';
|
||
|
||
async function migrateExistingExecutions() {
|
||
const repository = getRepository(TemplateExecutionEntity);
|
||
|
||
// 将现有的 PENDING 状态改为 PENDING_AUDIT
|
||
await repository.update(
|
||
{ status: ExecutionStatus.PENDING as any },
|
||
{ status: ExecutionStatus.PENDING_AUDIT }
|
||
);
|
||
|
||
console.log('迁移完成：现有执行记录状态已更新');
|
||
}
|
||
```
|
||
|
||
### 阶段五：测试和部署 (1天)
|
||
|
||
#### 5.1 单元测试升级
|
||
```typescript
|
||
// 📁 src/controllers/__tests__/enhanced-template.controller.spec.ts
|
||
describe('EnhancedTemplateController', () => {
|
||
it('should return pending_audit status immediately', async () => {
|
||
const response = await controller.executeTemplateByCode('photo_restore_v1', {
|
||
imageUrl: 'https://example.com/test.jpg'
|
||
});
|
||
|
||
expect(response.data.status).toBe('pending_audit');
|
||
expect(response.data.executionId).toBeDefined();
|
||
expect(response.data.auditTaskId).toBeDefined();
|
||
});
|
||
|
||
it('should handle audit completion callback', async () => {
|
||
const auditResult = {
|
||
taskId: 'audit_task_123',
|
||
conclusion: AuditConclusion.PASS,
|
||
};
|
||
|
||
await controller.handleAuditComplete('audit_task_123', auditResult);
|
||
|
||
// 验证执行记录状态更新为 PROCESSING
|
||
const execution = await repository.findByAuditTaskId('audit_task_123');
|
||
expect(execution.status).toBe(ExecutionStatus.PROCESSING);
|
||
});
|
||
});
|
||
```
|
||
|
||
#### 5.2 集成测试
|
||
```typescript
|
||
// 📁 test/integration/unified-async-flow.e2e-spec.ts
|
||
describe('Unified Async Flow (E2E)', () => {
|
||
it('should complete full async execution flow', async () => {
|
||
// 1. 提交模板执行
|
||
const submitResponse = await request(app.getHttpServer())
|
||
.post('/enhanced/templates/code/photo_restore_v1/execute')
|
||
.send({ imageUrl: 'https://example.com/test.jpg' })
|
||
.expect(200);
|
||
|
||
const { executionId, auditTaskId } = submitResponse.body.data;
|
||
|
||
// 2. 立即查询状态 - 应该是待审核
|
||
const statusResponse1 = await request(app.getHttpServer())
|
||
.get(`/enhanced/templates/executions/${executionId}/status`)
|
||
.expect(200);
|
||
|
||
expect(statusResponse1.body.data.status).toBe('pending_audit');
|
||
|
||
// 3. 模拟审核回调
|
||
await request(app.getHttpServer())
|
||
.post('/api/v1/content-moderation/wechat/callback')
|
||
.send({
|
||
task_id: auditTaskId,
|
||
status: 1,
|
||
conclusion: 1,
|
||
confidence: 95,
|
||
})
|
||
.expect(200);
|
||
|
||
// 4. 再次查询状态 - 应该是处理中或已完成
|
||
await new Promise(resolve => setTimeout(resolve, 1000)); // 等待异步处理
|
||
|
||
const statusResponse2 = await request(app.getHttpServer())
|
||
.get(`/enhanced/templates/executions/${executionId}/status`)
|
||
.expect(200);
|
||
|
||
expect(['processing', 'completed']).toContain(statusResponse2.body.data.status);
|
||
});
|
||
});
|
||
```
|
||
|
||
---
|
||
|
||
## 📋 升级清单
|
||
|
||
### 🔧 代码变更清单
|
||
|
||
#### 新增文件
|
||
- [ ] `src/content-moderation/adapters/enhanced-base-content.adapter.ts`
|
||
- [ ] `src/content-moderation/adapters/enhanced-wechat-content.adapter.ts`
|
||
- [ ] `src/controllers/enhanced-template.controller.ts`
|
||
- [ ] `migrations/xxx-add-audit-task-id-and-status.ts`
|
||
- [ ] `scripts/migrate-existing-executions.ts`
|
||
|
||
#### 修改文件
|
||
- [ ] `src/entities/template-execution.entity.ts` - 扩展状态枚举和字段
|
||
- [ ] `src/content-moderation/services/content-adapter.factory.ts` - 适配器选择逻辑
|
||
- [ ] `src/content-moderation/adapters/douyin-content.adapter.ts` - 回调通知集成
|
||
- [ ] `src/content-moderation/adapters/wechat-content.adapter.ts` - 回调通知集成
|
||
|
||
#### 测试文件
|
||
- [ ] `src/controllers/__tests__/enhanced-template.controller.spec.ts`
|
||
- [ ] `test/integration/unified-async-flow.e2e-spec.ts`
|
||
- [ ] 更新现有测试用例
|
||
|
||
### 🗄️ 数据库变更清单
|
||
|
||
- [ ] 添加 `audit_task_id` 字段
|
||
- [ ] 扩展 `status` 枚举值
|
||
- [ ] 修改默认状态值
|
||
- [ ] 添加相关索引
|
||
- [ ] 执行数据迁移脚本
|
||
|
||
### 🌐 环境配置清单
|
||
|
||
- [ ] 验证 `AUDIT_CALLBACK_URL` 配置
|
||
- [ ] 验证 `BYTEDANCE_AUDIT_API_URL` 配置
|
||
- [ ] 验证 `WECHAT_AUDIT_API_URL` 配置
|
||
- [ ] 确保回调URL的可访问性
|
||
|
||
---
|
||
|
||
## 🚀 部署方案
|
||
|
||
### MySQL 数据库升级步骤
|
||
|
||
1. **Phase 1 - 数据库备份和升级**：
|
||
```bash
|
||
# 1. 创建完整数据库备份
|
||
mysqldump -u root -p \
|
||
--routines \
|
||
--triggers \
|
||
--events \
|
||
--single-transaction \
|
||
--lock-tables=false \
|
||
nano_camera_miniapp > backup_$(date +%Y%m%d_%H%M%S).sql
|
||
|
||
# 2. 验证备份文件
|
||
ls -lh backup_*.sql
|
||
|
||
# 3. 创建TypeORM迁移文件
|
||
npm run migration:generate -- -n UpgradeToUnifiedAsyncArchitecture
|
||
|
||
# 4. 执行迁移（建议在低峰时段）
|
||
npm run migration:run
|
||
|
||
# 5. 验证迁移结果
|
||
mysql -u root -p -e "
|
||
USE nano_camera_miniapp;
|
||
DESCRIBE template_executions;
|
||
SELECT status, COUNT(*) FROM template_executions GROUP BY status;
|
||
SHOW INDEX FROM template_executions;
|
||
"
|
||
```
|
||
|
||
2. **Phase 1.5 - MySQL性能优化**：
|
||
```sql
|
||
-- 检查表大小和索引效率
|
||
SELECT
|
||
table_name,
|
||
table_rows,
|
||
ROUND(((data_length + index_length) / 1024 / 1024), 2) AS 'Size (MB)'
|
||
FROM information_schema.TABLES
|
||
WHERE table_schema = 'nano_camera_miniapp'
|
||
AND table_name = 'template_executions';
|
||
|
||
-- 检查索引使用情况（运行一段时间后执行）
|
||
SHOW INDEX FROM template_executions;
|
||
|
||
-- 分析查询性能
|
||
EXPLAIN SELECT * FROM template_executions
|
||
WHERE userId = 'test_user' AND status = 'pending_audit'
|
||
ORDER BY createdAt DESC LIMIT 10;
|
||
```
|
||
|
||
2. **Phase 2 - 代码部署**：
|
||
```bash
|
||
# 1. 部署新代码
|
||
git checkout main
|
||
git pull origin main
|
||
npm install
|
||
npm run build
|
||
|
||
# 2. 重启应用
|
||
pm2 restart app
|
||
```
|
||
|
||
3. **Phase 3 - 功能验证**：
|
||
```bash
|
||
# 1. 运行集成测试
|
||
npm run test:e2e
|
||
|
||
# 2. 手动测试关键流程
|
||
curl -X POST /enhanced/templates/code/photo_restore_v1/execute
|
||
```
|
||
|
||
### 灰度方案
|
||
|
||
```typescript
|
||
// 环境变量控制灰度
|
||
const USE_ENHANCED_CONTROLLER = process.env.USE_ENHANCED_CONTROLLER === 'true';
|
||
|
||
@Controller(USE_ENHANCED_CONTROLLER ? 'enhanced/templates' : 'templates')
|
||
export class TemplateController {
|
||
// 原有逻辑保持不变，确保向后兼容
|
||
}
|
||
```
|
||
|
||
### 回滚方案
|
||
|
||
1. **代码回滚**：
|
||
```bash
|
||
git checkout [previous_commit_hash]
|
||
npm run build
|
||
pm2 restart app
|
||
```
|
||
|
||
2. **MySQL数据库回滚**：
|
||
```bash
|
||
# 1. 如果使用TypeORM迁移，直接回滚
|
||
npm run migration:revert
|
||
|
||
# 2. 如果需要手动回滚
|
||
mysql -u root -p nano_camera_miniapp << 'EOF'
|
||
-- 删除新增索引
|
||
DROP INDEX `IDX_template_executions_audit_task_id` ON `template_executions`;
|
||
DROP INDEX `IDX_template_executions_status_updated` ON `template_executions`;
|
||
DROP INDEX `IDX_template_executions_user_status` ON `template_executions`;
|
||
|
||
-- 更新状态数据
|
||
UPDATE `template_executions`
|
||
SET `status` = 'pending'
|
||
WHERE `status` IN ('pending_audit', 'audit_failed');
|
||
|
||
-- 恢复原始ENUM
|
||
ALTER TABLE `template_executions`
|
||
MODIFY COLUMN `status` ENUM('pending','processing','completed','failed','cancelled')
|
||
NOT NULL DEFAULT 'pending';
|
||
|
||
-- 删除字段
|
||
ALTER TABLE `template_executions` DROP COLUMN `audit_task_id`;
|
||
EOF
|
||
|
||
# 3. 验证回滚结果
|
||
mysql -u root -p -e "
|
||
USE nano_camera_miniapp;
|
||
DESCRIBE template_executions;
|
||
SELECT status, COUNT(*) FROM template_executions GROUP BY status;
|
||
"
|
||
```
|
||
|
||
### 🔧 MySQL特有注意事项
|
||
|
||
#### 版本兼容性
|
||
```bash
|
||
# 检查MySQL版本
|
||
mysql --version
|
||
|
||
# 确保版本兼容性：
|
||
# ✅ MySQL 5.7+ : 支持JSON字段类型
|
||
# ✅ MySQL 8.0+ : 更好的索引性能，推荐
|
||
# ❌ MySQL 5.6及以下 : 不支持JSON字段，需要升级
|
||
|
||
# 检查当前实例特性
|
||
mysql -u root -p -e "
|
||
SELECT VERSION() as mysql_version;
|
||
SHOW VARIABLES LIKE 'innodb_version';
|
||
SHOW VARIABLES LIKE 'default_storage_engine';
|
||
"
|
||
```
|
||
|
||
#### ENUM类型处理
|
||
```sql
|
||
-- ⚠️ MySQL ENUM修改注意事项：
|
||
-- 1. MySQL不能直接添加ENUM值到中间位置，需要重新定义整个ENUM
|
||
-- 2. 修改ENUM会锁表，建议在低峰时段操作
|
||
-- 3. 大表修改ENUM可能耗时较长
|
||
|
||
-- 检查当前ENUM定义
|
||
SELECT COLUMN_TYPE
|
||
FROM INFORMATION_SCHEMA.COLUMNS
|
||
WHERE TABLE_SCHEMA = 'nano_camera_miniapp'
|
||
AND TABLE_NAME = 'template_executions'
|
||
AND COLUMN_NAME = 'status';
|
||
```
|
||
|
||
#### 索引优化
|
||
```sql
|
||
-- MySQL索引命名约定（项目已遵循）
|
||
-- 前缀: IDX_表名_字段名
|
||
-- 示例: IDX_template_executions_audit_task_id
|
||
|
||
-- 检查索引碎片化
|
||
SELECT
|
||
table_name,
|
||
index_name,
|
||
cardinality,
|
||
pages,
|
||
avg_page_frag
|
||
FROM mysql.innodb_index_stats
|
||
WHERE database_name = 'nano_camera_miniapp'
|
||
AND table_name = 'template_executions';
|
||
|
||
-- 如果碎片化严重，重建索引
|
||
-- ALTER TABLE template_executions ENGINE=InnoDB;
|
||
```
|
||
|
||
#### 性能监控
|
||
```sql
|
||
-- 监控慢查询（my.cnf配置）
|
||
-- slow_query_log = 1
|
||
-- slow_query_log_file = /var/log/mysql-slow.log
|
||
-- long_query_time = 2
|
||
|
||
-- 查看执行中的查询
|
||
SHOW PROCESSLIST;
|
||
|
||
-- 查看表锁定情况
|
||
SHOW OPEN TABLES WHERE In_use > 0;
|
||
```
|
||
|
||
#### MySQL配置优化建议
|
||
```ini
|
||
# my.cnf 或 my.ini 配置优化
|
||
[mysqld]
|
||
# 基础性能优化
|
||
innodb_buffer_pool_size = 1G # 根据可用内存调整
|
||
innodb_log_file_size = 256M # 事务日志大小
|
||
innodb_flush_log_at_trx_commit = 1 # 事务安全性
|
||
innodb_file_per_table = 1 # 每个表独立表空间
|
||
|
||
# 连接和查询优化
|
||
max_connections = 500 # 最大连接数
|
||
wait_timeout = 300 # 连接超时
|
||
interactive_timeout = 300 # 交互超时
|
||
|
||
# 索引和排序优化
|
||
sort_buffer_size = 2M # 排序缓冲区
|
||
read_buffer_size = 1M # 读缓冲区
|
||
tmp_table_size = 64M # 临时表大小
|
||
max_heap_table_size = 64M # 内存表大小
|
||
|
||
# 慢查询日志（监控性能）
|
||
slow_query_log = 1
|
||
slow_query_log_file = /var/log/mysql/slow.log
|
||
long_query_time = 2
|
||
|
||
# 二进制日志（数据安全）
|
||
log_bin = mysql-bin
|
||
binlog_format = ROW
|
||
expire_logs_days = 7
|
||
```
|
||
|
||
#### 大表迁移策略
|
||
```bash
|
||
# 如果 template_executions 表很大（>100万记录），使用在线迁移工具
|
||
# 1. 安装 pt-online-schema-change（Percona Toolkit）
|
||
# 2. 使用在线迁移避免锁表
|
||
|
||
# 示例：在线添加字段（如果表很大）
|
||
pt-online-schema-change \
|
||
--alter "ADD COLUMN audit_task_id VARCHAR(255) NULL" \
|
||
--host=localhost \
|
||
--user=root \
|
||
--ask-pass \
|
||
--execute \
|
||
D=nano_camera_miniapp,t=template_executions
|
||
|
||
# 3. 在线修改ENUM（大表推荐）
|
||
pt-online-schema-change \
|
||
--alter "MODIFY COLUMN status ENUM('pending','pending_audit','audit_failed','processing','completed','failed','cancelled') NOT NULL DEFAULT 'pending_audit'" \
|
||
--host=localhost \
|
||
--user=root \
|
||
--ask-pass \
|
||
--execute \
|
||
D=nano_camera_miniapp,t=template_executions
|
||
```
|
||
|
||
---
|
||
|
||
## 📈 预期效果
|
||
|
||
### 🎯 问题解决
|
||
|
||
1. **同步审核阻塞问题** → ✅ 统一异步，立即响应
|
||
2. **平台差异体验** → ✅ 一致的异步体验
|
||
3. **图片URL验证失败** → ✅ 已优化验证逻辑
|
||
4. **状态追踪不清晰** → ✅ 完整的状态流转
|
||
|
||
### 📊 性能提升
|
||
|
||
- **响应时间**：从审核时间(2-5秒) → 200ms内
|
||
- **用户体验**：阻塞等待 → 异步轮询状态
|
||
- **系统吞吐**：串行执行 → 并行处理
|
||
- **错误恢复**：审核失败阻断 → 独立状态管理
|
||
|
||
### 🔮 扩展能力
|
||
|
||
- **新平台接入**：只需实现适配器，业务层无感知
|
||
- **批量处理**：天然支持大规模并发审核
|
||
- **监控告警**：完整的状态和指标监控
|
||
- **智能重试**：基于状态的重试机制
|
||
|
||
---
|
||
|
||
## 🎉 总结
|
||
|
||
本升级方案通过 **统一异步架构** 和 **平台差异抹平技术**，彻底解决了同步审核与异步模板执行的架构矛盾。升级后系统将具备：
|
||
|
||
- 🚀 **一致的异步体验**：所有平台统一异步模式
|
||
- 🔧 **优雅的架构设计**：清晰的状态流转和回调机制
|
||
- 📈 **更好的性能**：非阻塞处理，提升吞吐量
|
||
- 🛡️ **更强的扩展性**：新平台接入成本低
|
||
- 🎯 **完善的监控**：全链路状态跟踪
|
||
|
||
**预计升级周期：3-4个工作日**
|
||
**预计收益：彻底解决现有架构问题，为未来扩展奠定坚实基础** |