Visual Replay Tester (VRT)

简体中文 | English

Visual Replay Tester (VRT) is a next-generation End-to-End (E2E) desktop automation tool integrating Playwright, Electron, and the Model Context Protocol (MCP).

Our core philosophy is "Autonomous Driving for Testing". VRT goes beyond traditional record-and-playback tools; it is an intelligent system capable of autonomous exploration, planning, and generating permanent testing assets.

Core Value: "One-time AI Exploration, Permanent Script Assets." Transform expensive AI reasoning into low-cost, highly stable engineering scripts.

---

🌟 Core Concepts & Architecture

VRT is designed to achieve a "Given a URL, Return a Report" closed-loop automation workflow.

1. 👁️ Perception & Planning

The system understands pages like a human engineer:

• Full-Screen VLM Scan: Uses Visual Language Models (VLM) to identify page types (e.g., Login, Dashboard) and core functional areas.
• Semantic Extraction: Understands the business meaning of UI components, not just the DOM structure.
• Strategy Generation: Automatically generates test paths (e.g., "Identify Login Box -> Plan Username/Password Input -> Click Login").

2. 🧠 Adaptive Execution Engine

Powered by a Hybrid MCP + VLM Driver to ensure stability across all scenarios:

• MCP Highway (The Standard Path):
  • Built on @playwright/mcp and Chrome DevTools Protocol (CDP).
  • Uses Accessibility Tree and Ref IDs for precise control.
  • Handles 80% of standard DOM interactions with high speed and stability.
• Vision Off-road (The Fallback):
  • Takes over when encountering Canvas, Shadow DOM, or heavily obfuscated code.
  • Simulates human visual judgment and physical clicks.

3. 📝 Asset Crystallization (Script Generation)

Agent actions are never wasted. Every exploration step is compiled into a Standard Automation Script.

• Dynamic to Static: Converts dynamic AI decisions into fixed Playwright/VRT scripts.
• Visual Anchors: Automatically generates visual fingerprints for VLM-driven steps to enable future playback.
• Zero-Cost Regression: Generated scripts can be run by traditional CI/CD pipelines without consuming further AI tokens.

---

⚡ Key Features

🤖 Intelligent MCP Automation

• Standard Protocol: Fully compatible with the Model Context Protocol.
• Natural Language Control: Tell the AI "Test the search function", and it will plan and execute the task.
• Precise Targeting: Say goodbye to fragile CSS selectors; use semantic targeting.

📹 Embedded Recording Studio

• WYSIWYG: Record and debug directly inside the Electron application.
• Visual Timeline: Drag-and-drop editor for test steps.
• Keyframe Snapshots: Automatically capture visual states at critical nodes for pixel-perfect regression testing.

---

🚀 Quick Start

Prerequisites

• Node.js: v20+ (Required)
• OS: macOS / Windows / Linux

Installation

1. Clone the repository and install dependencies:

   git clone [repository-url]
   cd Visual-Replay-Tester
   npm install
   nvm use 20

2. Install browser binaries:

   npx playwright install chromium

3. Start the development server:

   npm run dev

   The app will launch and automatically start the local CDP server on port 9222.

AI Configuration

Before using AI features, please configure your models:

1. Click Settings in the navigation bar.
2. Configure your Model Provider (OpenAI, Anthropic, Gemini, or Custom SiliconFlow/Together AI).
3. Enter your API Key and enable the model.
4. Assign the model to "MCP Automation" and "Visual AI" in the Feature Mapping section.

---

📖 Usage Scenarios

Scenario 1: Natural Language Exploration (Agent Mode)

1. Go to the MCP Automation tab.
2. Enter the target URL.
3. Type your task in the chat: "Test the login flow with username admin".
4. Watch the Agent analyze the page, plan steps, and execute them.

Scenario 2: Traditional Recording & Visual Regression

1. Click + New Recording.
2. Interact with the page manually; the system generates steps automatically.
3. Click Capture Keyframe at critical points.
4. Save the script. Future runs will automatically compare new screenshots against the baseline.

---

🏗️ Technical Architecture

graph TD
    User[User] -->|URL/Command| Agent[AI Agent]
    
    subgraph "VRT Engine"
        Agent -->|Visual Perception| VLM[VLM Model]
        Agent -->|Action Instructions| Hybrid[Hybrid Executor]
        
        Hybrid -->|Standard Actions| MCP[MCP Client]
        Hybrid -->|Complex Interactions| VIS[Visual Driver]
        
        MCP -->|CDP/Accessibility| Browser[Internal Browser]
        VIS -->|Coordinate Sim| Browser
    end
    
    Browser -->|Feedback/Screenshots| Agent
    Hybrid -->|Compile| Script[Automation Script]
    Script -->|CI/CD| Report[Test Report]

Loading

• Frontend: React + TypeScript + TailwindCSS
• Backend/Main: Electron + Node.js
• Protocol: Model Context Protocol (MCP)

---

🛠️ Development Tools

• Build: npm run build
• Lint: npm run lint

📄 License

MIT