[Enhancement] Face tracking lacks IOU check causing tracking errors in multi-person scenarios

[suntp]

问题描述 / Problem Description

中文: 在人脸追踪过程中缺少 IOU（Intersection over Union）检查，在多人场景或快速运动时可能追踪错误的人脸或丢失目标。

English: The face tracking process lacks IOU (Intersection over Union) checks, which may cause tracking of wrong faces or target loss in multi-person scenarios or during fast movements.

问题位置 / Location

文件 / File: src/utils/cropper.py
行号 / Line: 238

问题代码 / Problematic Code

# TODO: add IOU check for tracking

影响 / Impact

中文:

1. 多人场景: 可能从追踪人物A切换到人物B
2. 快速运动: 目标快速移动时可能丢失追踪
3. 遮挡情况: 遮挡后可能追踪到错误的人脸
4. 视频质量: 导致生成的动画不一致、跳变

English:

1. Multi-person scenarios: May switch from tracking person A to person B
2. Fast movements: May lose tracking when target moves quickly
3. Occlusion: May track wrong face after occlusion
4. Video quality: Causes inconsistent and jumpy animations

场景示例 / Scenario Examples

场景1: 多人场景 / Scenario 1: Multi-person Scene

帧1 / Frame 1: 检测到人脸A（主角）/ Detect face A (main character)
帧2 / Frame 2: 人脸B（路人）进入画面 / Face B (passerby) enters frame
帧3 / Frame 3: 追踪切换到人脸B ❌ / Tracking switches to face B ❌

场景2: 快速运动 / Scenario 2: Fast Movement

帧1 / Frame 1: 人脸在位置A / Face at position A
帧2 / Frame 2: 快速移动到位置B / Fast move to position B
帧3 / Frame 3: 追踪丢失 ❌ / Tracking lost ❌

建议修复 / Suggested Fixes

添加 IOU 检查 / Add IOU Check

def calculate_iou(box1, box2):
    """计算两个边界框的IOU / Calculate IOU between two bounding boxes"""
    x1 = max(box1[0], box2[0])
    y1 = max(box1[1], box2[1])
    x2 = min(box1[2], box2[2])
    y2 = min(box1[3], box2[3])
    
    intersection = max(0, x2 - x1) * max(0, y2 - y1)
    area1 = (box1[2] - box1[0]) * (box1[3] - box1[1])
    area2 = (box2[2] - box2[0]) * (box2[3] - box2[1])
    union = area1 + area2 - intersection
    
    return intersection / union if union > 0 else 0

# 在追踪时使用 / Use during tracking
if trajectory.lmk_lst:
    prev_bbox = get_bbox_from_lmk(trajectory.lmk_lst[-1])
    curr_bbox = get_bbox_from_lmk(lmk)
    iou = calculate_iou(prev_bbox, curr_bbox)
    
    if iou < 0.3:  # IOU阈值 / IOU threshold
        log(f"Tracking lost at frame #{idx}, IOU={iou:.2f}")
        # 处理追踪丢失 / Handle tracking loss

添加关键点距离检查 / Add Landmark Distance Check

def check_landmark_consistency(prev_lmk, curr_lmk, threshold=50):
    """检查关键点移动距离是否合理 / Check if landmark movement is reasonable"""
    distances = np.linalg.norm(prev_lmk - curr_lmk, axis=1)
    max_distance = np.max(distances)
    return max_distance < threshold

# 使用 / Usage
if trajectory.lmk_lst:
    if not check_landmark_consistency(trajectory.lmk_lst[-1], lmk):
        log(f"Abnormal movement detected at frame #{idx}")
        # 使用前一帧或重新检测 / Use previous frame or re-detect

配置建议 / Configuration Suggestions

class CropConfig:
    # ...
    tracking_iou_threshold: float = 0.3
    tracking_distance_threshold: float = 50.0
    enable_tracking_validation: bool = True

优先级 / Priority

P1 - 建议短期修复 / Recommend short-term fix

相关信息 / Related Information

• 发现时间 / Discovered: 2026-03-05
• 相关 TODO: TODO: add IOU check for tracking
• 影响 / Impact: 多人场景、快速运动场景 / Multi-person scenarios, fast movement scenarios