Conversation
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new vnode data repair tool (dmRepairCopy) for data recovery via SSH or local copy, alongside a meta-reader utility for manual metadata inspection and repair. It implements enhanced table-level privilege filtering for system table queries, addresses client-side double-free vulnerabilities, and fixes DST-related inaccuracies in time truncation. Other significant changes include the addition of a flexible deployment mode, optimized log rotation, and expanded test coverage for subquery pruning and CASE WHEN conditions. Review feedback highlights several critical issues in the new repair tool, including the incorrect use of dot notation for nested JSON lookups, potential buffer overflows when handling shell-quoted paths, and fragile parsing of remote directory listings.
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
localvar
force-pushed
the
feat/copy-mode-revovery
branch
from
10f8581 to
4508164
Compare
localvar
force-pushed
the
feat/copy-mode-revovery
branch
from
4508164 to
15c9782
Compare
There was a problem hiding this comment.
Pull request overview
This PR introduces a new taosd repair “copy” mode to recover vnode data by copying TSDB/non-TSDB files from a source node (local or via SSH), regenerating current.json, updating sync indexes, and cleaning sync state. It also adds CI tests to validate multiple copy/repair scenarios, including multi-tier and multi-disk remapping.
Changes:
- Add
-r --mode copyCLI workflow intaosdand route execution todmRepairCopyMode(). - Implement copy-mode repair logic (source cfg parsing, vnode backup/rollback, file copy/link,
current.jsonregeneration, sync index updates, cleanup). - Add a comprehensive pytest suite and register it in CI tasks.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| test/ci/cases.task | Adds the new repair-copy pytest case to CI execution. |
| test/cases/50-Others/02-Repair/test_copy_repair.py | New end-to-end tests for local/remote copy-mode repair and disk/tier remapping. |
| source/dnode/mgmt/exe/dmMain.c | Adds CLI parsing for copy-mode repair and invokes dmRepairCopyMode() when enabled. |
| source/common/src/dmRepairCopy.c | New implementation of copy-mode repair (copy/link/remap/regenerate/cleanup with rollback). |
| source/common/CMakeLists.txt | Adds include paths needed to build the new common repair-copy implementation. |
| include/common/dmRepairCopy.h | New public header defining options and entry points for repair-copy mode. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
localvar
changed the title
| char filePath[PATH_MAX]; | ||
| dmBuildTsdbFilePath(diskPath, vnodeId, lf, filePath, sizeof(filePath)); | ||
|
|
||
| int64_t actualSize = 0; | ||
| if (taosStatFile(filePath, &actualSize, NULL, NULL) != 0 || actualSize <= 0) { | ||
| allExist = false; | ||
| break; | ||
| } | ||
| } | ||
| if (allExist && nLocalFiles > 0) retained = true; |
| } | ||
| uInfo("repair: vnode%d renamed %s to .bak", vnodeId, srcPath); | ||
| } else { | ||
| if (taosMkDir(dstPath) != 0) { |
|
|
||
| SDiskSize diskSize = {0}; | ||
| if (taosGetDiskSize((char *)diskPath, &diskSize) != 0) continue; | ||
| if (diskSize.avail < fileSize + TFS_MIN_DISK_FREE_SIZE) continue; |
Description
implement copy mode data recovery.
Issue(s)
Checklist
Please check the items in the checklist if applicable.