diff --git a/.github/workflows/auto-pr-review.yml b/.github/workflows/auto-pr-review.yml index 6a585355f..8e4f3d7f7 100644 --- a/.github/workflows/auto-pr-review.yml +++ b/.github/workflows/auto-pr-review.yml @@ -15,7 +15,7 @@ # specific language governing permissions and limitations # under the License. -name: "Auto PR Commenter" +name: "Auto PR AI Review" on: pull_request_target: diff --git a/content/cn/docs/guides/backup-restore.md b/content/cn/docs/guides/backup-restore.md index 495587f0f..93e5b06bb 100644 --- a/content/cn/docs/guides/backup-restore.md +++ b/content/cn/docs/guides/backup-restore.md @@ -1,7 +1,7 @@ --- title: "Backup Restore" linkTitle: "备份 & 恢复" -weight: 4 +weight: 5 --- ## 描述 diff --git a/content/cn/docs/guides/faq.md b/content/cn/docs/guides/faq.md index 45986bf58..d658fdddc 100644 --- a/content/cn/docs/guides/faq.md +++ b/content/cn/docs/guides/faq.md @@ -1,7 +1,7 @@ --- title: "FAQ" linkTitle: "FAQ" -weight: 5 +weight: 6 --- - 如何选择后端存储? 选 RocksDB 还是 Cassandra 还是 Hbase 还是 Mysql? diff --git a/content/cn/docs/guides/security.md b/content/cn/docs/guides/security.md index 0f71ff169..a1bdf20c5 100644 --- a/content/cn/docs/guides/security.md +++ b/content/cn/docs/guides/security.md @@ -1,7 +1,7 @@ --- title: "报告安全问题" linkTitle: "安全公告" -weight: 6 +weight: 7 --- ## 报告 Apache HugeGraph 的安全问题 diff --git a/content/cn/docs/guides/toolchain-local-test.md b/content/cn/docs/guides/toolchain-local-test.md new file mode 100644 index 000000000..b2d4e29fb --- /dev/null +++ b/content/cn/docs/guides/toolchain-local-test.md @@ -0,0 +1,443 @@ +--- +title: "HugeGraph工具链本地测试指南" +linkTitle: "Toolchain本地测试" +weight: 4 +--- + +本指南帮助开发者在本地运行 HugeGraph 工具链测试。 + +## 1. 核心概念 + +### 1.1 核心依赖：HugeGraph Server + +**工具链的集成测试和功能测试都依赖 HugeGraph Server**，包括 Client、Loader、Hubble、Spark Connector、Tools 等组件。 + +### 1.2 测试类型 + +- **单元测试 (Unit Tests)**：测试单个函数/方法，不依赖外部服务 +- **API 测试 (ApiTestSuite)**：测试 API 接口，需要运行中的 HugeGraph Server +- **功能测试 (FuncTestSuite)**：端到端测试，需要完整的系统环境 + +## 2. 环境准备 + +### 2.1 系统要求 + +- **操作系统**：Linux / macOS（Windows 使用 WSL2） +- **JDK**：>= 11，配置好 `JAVA_HOME` +- **Maven**：>= 3.5 +- **Python**：>= 3.11（仅 Hubble 测试需要） + +### 2.2 克隆代码 + +```bash +git clone https://github.com/${GITHUB_USER_NAME}/hugegraph-toolchain.git +cd hugegraph-toolchain +``` + +## 3. 部署测试环境 + +### 方式选择 + +- **脚本部署（推荐）**：通过指定 Commit ID 精确控制 Server 版本，避免接口不兼容 +- **Docker 部署**：快速启动，但可能版本滞后导致测试失败 + +> 详细安装说明参考 [社区文档](https://hugegraph.apache.org/cn/docs/quickstart/hugegraph/hugegraph-server/) + +### 3.1 脚本部署（推荐） + +#### 参数说明 + +- **`$COMMIT_ID`**：指定 Server 源码的 Git Commit ID +- **`$DB_DATABASE` / `$DB_PASS`**：Loader JDBC 测试用的 MySQL 数据库名和密码 + +#### 部署步骤 + +**1. 安装 HugeGraph Server** + +```bash +# 设置版本 +export COMMIT_ID="master" # 或特定 commit hash，如 "8b90977" + +# 执行安装（脚本位于 /assembly/travis/ 目录） +hugegraph-client/assembly/travis/install-hugegraph-from-source.sh $COMMIT_ID +``` + +- 默认端口：http 8080, https 8443 +- 确保端口未被占用 + +**2. 安装可选依赖** + +```bash +# Hadoop (仅 Loader HDFS 测试需要) +hugegraph-loader/assembly/travis/install-hadoop.sh + +# MySQL (仅 Loader JDBC 测试需要) +hugegraph-loader/assembly/travis/install-mysql.sh $DB_DATABASE $DB_PASS +``` + +**3. 健康检查** + +```bash +curl http://localhost:8080/graphs +# 返回 {"graphs":["hugegraph"]} 表示成功 +``` + +### 3.2 Docker 部署 + +> **注意**：Docker 镜像可能版本滞后，如遇兼容性问题请使用脚本部署 + +#### 快速启动 + +```bash +docker network create hugegraph-net +docker run -itd --name=server -p 8080:8080 --network hugegraph-net hugegraph/hugegraph:latest +``` + +#### docker-compose 配置（可选） + +完整配置示例，包含 Server、MySQL、Hadoop 服务（需要 Docker Compose V2）： + +```yaml +version: '3.8' + +services: + hugegraph-server: + image: hugegraph/hugegraph:latest # 可以替换为特定版本，或构建自己的镜像 + container_name: hugegraph-server + ports: + - "8080:8080" # HugeGraph Server HTTP 端口 + environment: + # 根据需要配置HugeGraph Server的参数，例如后端存储 + - HUGEGRAPH_SERVER_OPTIONS="-Dstore.backend=rocksdb" + volumes: + # 如果需要持久化数据或挂载配置文件，可以在这里添加卷 + # - ./hugegraph-data:/opt/hugegraph/data + healthcheck: + test: ["CMD-SHELL", "curl -f http://localhost:8080/graphs || exit 1"] + interval: 5s + timeout: 3s + retries: 5 + networks: + - hugegraph-net + + # 如果需要hugegraph-loader的JDBC测试，可以添加以下服务 + # mysql: + # image: mysql:5.7 + # container_name: mysql-db + # environment: + # MYSQL_ROOT_PASSWORD: ${DB_PASS:-your_mysql_root_password} # 从环境变量读取，或使用默认值 + # MYSQL_DATABASE: ${DB_DATABASE:-hugegraph_test_db} # 从环境变量读取，或使用默认值 + # ports: + # - "3306:3306" + # volumes: + # - ./mysql-data:/var/lib/mysql # 数据持久化 + # healthcheck: + # test: ["CMD", "mysqladmin", "ping", "-h", "localhost", "-p${DB_PASS:-your_mysql_root_password}"] + # interval: 5s + # timeout: 3s + # retries: 5 + # networks: + # - hugegraph-net + + # 如果需要hugegraph-loader的Hadoop/HDFS测试，可以添加以下服务 + # namenode: + # image: johannestang/hadoop-namenode:2.0.0-hadoop2.8.5-java8 + # container_name: namenode + # ports: + # - "0.0.0.0:9870:9870" + # - "0.0.0.0:8020:8020" + # environment: + # - CLUSTER_NAME=test-cluster + # - HDFS_NAMENODE_USER=root + # - HADOOP_CONF_DIR=/hadoop/etc/hadoop + # volumes: + # - ./config/core-site.xml:/hadoop/etc/hadoop/core-site.xml + # - ./config/hdfs-site.xml:/hadoop/etc/hadoop/hdfs-site.xml + # - namenode_data:/hadoop/dfs/name + # command: bash -c "if [ ! -d /hadoop/dfs/name/current ]; then hdfs namenode -format; fi && /entrypoint.sh" + # healthcheck: + # test: ["CMD", "hdfs", "dfsadmin", "-report"] + # interval: 5s + # timeout: 3s + # retries: 5 + # networks: + # - hugegraph-net + + # datanode: + # image: johannestang/hadoop-datanode:2.0.0-hadoop2.8.5-java8 + # container_name: datanode + # depends_on: + # - namenode + # environment: + # - CLUSTER_NAME=test-cluster + # - HDFS_DATANODE_USER=root + # - HADOOP_CONF_DIR=/hadoop/etc/hadoop + # volumes: + # - ./config/core-site.xml:/hadoop/etc/hadoop/core-site.xml + # - ./config/hdfs-site.xml:/hadoop/etc/hadoop/hdfs-site.xml + # - datanode_data:/hadoop/dfs/data + # healthcheck: + # test: ["CMD", "hdfs", "dfsadmin", "-report"] + # interval: 5s + # timeout: 3s + # retries: 5 + # networks: + # - hugegraph-net + +networks: + hugegraph-net: + driver: bridge +volumes: + namenode_data: + datanode_data: +``` + +#### Hadoop 配置挂载 +在与 `docker-compose.yml` 相同的目录下创建 `./config` 文件夹用于挂载 Hadoop 配置文件。如果不需要 HDFS 测试，可以跳过此步骤。 + +📁 ./config/core-site.xml 内容： + +```xml + + + fs.defaultFS + hdfs://namenode:8020 + + +``` + +📁 ./config/hdfs-site.xml 内容： + +```xml + + + dfs.namenode.name.dir + /hadoop/hdfs/name + + + dfs.datanode.data.dir + /hadoop/hdfs/data + + + dfs.permissions.superusergroup + hadoop + + + dfs.support.append + true + + +``` + +#### Docker 操作 + +```bash +# 启动服务 +docker compose up -d + +# 检查状态 +docker compose ps +lsof -i:8080 # Server +lsof -i:8020 # Hadoop +lsof -i:3306 # MySQL + +# 停止服务 +docker compose down +``` + +## 4. 运行测试 + +各工具的测试流程： + +
+ HugeGraph工具链测试流程图 +
+ +### 4.1 hugegraph-client + +#### 编译 + +```bash +mvn -e compile -pl hugegraph-client -Dmaven.javadoc.skip=true -ntp +``` + +#### 依赖服务 + +启动 HugeGraph Server（参考 [第3节](#3-部署测试环境)） + +##### Server 鉴权配置 + +> **注意**：Docker 镜像 <= 1.5.0 不支持鉴权测试，需 1.6.0+ + +ApiTest 需要鉴权配置，使用脚本安装可跳过。使用 Docker 需手动配置： + +```bash +# 1. 修改鉴权模式 +cp conf/rest-server.properties conf/rest-server.properties.backup +sed -i '/^auth.authenticator=/c\auth.authenticator=org.apache.hugegraph.auth.StandardAuthenticator' conf/rest-server.properties +grep auth.authenticator conf/rest-server.properties + +# 2. 设置密码 +# 注：测试代码中默认使用 "pa" 作为密码，设置时需与测试保持一致 +bin/stop-hugegraph.sh +export PASSWORD="pa" # 设置为测试默认密码 +echo -e "${PASSWORD}" | bin/init-store.sh +bin/start-hugegraph.sh +``` + +#### 运行测试 + +```bash +# 检查环境 +curl http://localhost:8080/graphs # 应返回 {"graphs":["hugegraph"]} +curl -u admin:pa http://localhost:8080/graphs # 鉴权测试（密码 pa 是测试默认值） + +# 运行测试 +cd hugegraph-client +mvn test -Dtest=UnitTestSuite -ntp # 单元测试 +mvn test -Dtest=ApiTestSuite -ntp # API测试（需 Server） +mvn test -Dtest=FuncTestSuite -ntp # 功能测试（需 Server） +``` + +> 测试失败时检查 Server 日志：`logs/hugegraph-server.log` + +### 4.2 hugegraph-loader + +#### 编译 + +```bash +mvn install -pl hugegraph-client,hugegraph-loader -am -Dmaven.javadoc.skip=true -DskipTests -ntp +``` + +#### 依赖服务 + +- **必需**：HugeGraph Server +- **可选**：Hadoop (HDFS 测试)、MySQL (JDBC 测试) + +#### 运行测试 + +```bash +cd hugegraph-loader +mvn test -P unit -ntp # 单元测试 +mvn test -P file -ntp # 文件测试（需 Server） +mvn test -P hdfs -ntp # HDFS测试（需 Server + Hadoop） +mvn test -P jdbc -ntp # JDBC测试（需 Server + MySQL） +mvn test -P kafka -ntp # Kafka测试（需 Server） +``` + +### 4.3 hugegraph-hubble + +#### 编译 + +```bash +mvn install -pl hugegraph-client,hugegraph-loader -am -Dmaven.javadoc.skip=true -DskipTests -ntp +cd hugegraph-hubble +mvn -e compile -Dmaven.javadoc.skip=true -ntp +``` + +#### 依赖服务 + +**1. 启动 Server**（参考 [第3节](#3-部署测试环境)） + +**2. Python 环境** + +```bash +python -m venv venv +source venv/bin/activate # Windows: venv\Scripts\activate +python -m pip install -r hubble-dist/assembly/travis/requirements.txt +``` + +**3. 构建并验证** + +```bash +mvn package -Dmaven.test.skip=true +# 可选：启动验证 +cd apache-hugegraph-hubble-incubating-*/bin +./start-hubble.sh -d && sleep 10 +curl http://localhost:8088/api/health +./stop-hubble.sh +``` + +#### 运行测试 + +```bash +# 单元测试 +mvn test -P unit-test -pl hugegraph-hubble/hubble-be -ntp + +# API测试（需 Server + Hubble 运行） +curl http://localhost:8080/graphs # 检查 Server +curl http://localhost:8088/api/health # 检查 Hubble +cd hugegraph-hubble/hubble-dist +./assembly/travis/run-api-test.sh +``` + +### 4.4 hugegraph-spark-connector + +#### 编译 + +```bash +mvn install -pl hugegraph-client,hugegraph-spark-connector -am -Dmaven.javadoc.skip=true -DskipTests -ntp +``` + +#### 运行测试 + +```bash +cd hugegraph-spark-connector +mvn test -ntp # 需 Server 运行 +``` + +### 4.5 hugegraph-tools + +#### 编译 + +```bash +mvn install -pl hugegraph-client,hugegraph-tools -am -Dmaven.javadoc.skip=true -DskipTests -ntp +``` + +#### 运行测试 + +```bash +cd hugegraph-tools +mvn test -Dtest=FuncTestSuite -ntp # 需 Server 运行 +``` + +## 5. 常见问题 + +### 服务连接问题 + +**症状**：无法连接 Server/MySQL/Hadoop + +**排查**： +- 确认服务已启动（Server 必须在 8080 端口） +- 检查端口占用：`lsof -i:8080` +- Docker 检查：`docker compose ps` 和 `docker compose logs` + +### 配置问题 + +**症状**：找不到文件、参数错误 + +**排查**： +- 检查环境变量：`echo $COMMIT_ID` +- 脚本权限：`chmod +x hugegraph-*/assembly/travis/*.sh` + +### HDFS 测试失败 + +**排查**： +- 确认 NameNode/DataNode 运行正常 +- 检查 Hadoop 日志 +- 验证 HDFS 连接：`hdfs dfsadmin -report` + +### JDBC 测试失败 + +**排查**： +- 确认 MySQL 运行正常 +- 验证数据库连接：`mysql -u root -p$DB_PASS` +- 检查 MySQL 日志 + +## 6. 参考资料 + +* **HugeGraph GitHub 仓库**：[https://github.com/apache/hugegraph](https://github.com/apache/hugegraph) +* **HugeGraph 工具链 GitHub 仓库**：[https://github.com/apache/hugegraph-toolchain](https://github.com/apache/hugegraph-toolchain) +* **HugeGraph Server 官方文档**：[https://hugegraph.apache.org/cn/docs/quickstart/hugegraph/hugegraph-server/](https://hugegraph.apache.org/cn/docs/quickstart/hugegraph/hugegraph-server/) +* **CI 脚本路径**：`.github/workflows/*-ci.yml`（HugeGraph 工具链项目中的 CI 配置文件，可作为参考） +* **依赖服务安装脚本**：`hugegraph-*/assembly/travis/`（HugeGraph 工具链项目中用于 CI 和本地测试的安装脚本，可直接使用或作为参考） diff --git a/content/cn/docs/images/toolchain-test-mermaid-1.png b/content/cn/docs/images/toolchain-test-mermaid-1.png new file mode 100644 index 000000000..2ce70294b Binary files /dev/null and b/content/cn/docs/images/toolchain-test-mermaid-1.png differ diff --git a/content/cn/docs/images/toolchain-test-mermaid-2.png b/content/cn/docs/images/toolchain-test-mermaid-2.png new file mode 100644 index 000000000..9904df7dc Binary files /dev/null and b/content/cn/docs/images/toolchain-test-mermaid-2.png differ diff --git a/content/cn/docs/quickstart/toolchain/_index.md b/content/cn/docs/quickstart/toolchain/_index.md index f00f7ed12..8033e0e72 100644 --- a/content/cn/docs/quickstart/toolchain/_index.md +++ b/content/cn/docs/quickstart/toolchain/_index.md @@ -3,3 +3,5 @@ title: "HugeGraph ToolChain" linkTitle: "HugeGraph 工具链" weight: 2 --- + +> **测试指南**：如需在本地运行工具链测试，请参考 [HugeGraph 工具链本地测试指南](/cn/docs/guides/toolchain-local-test) diff --git a/content/cn/docs/quickstart/toolchain/hugegraph-hubble.md b/content/cn/docs/quickstart/toolchain/hugegraph-hubble.md index 64871ca1f..2167c0a72 100644 --- a/content/cn/docs/quickstart/toolchain/hugegraph-hubble.md +++ b/content/cn/docs/quickstart/toolchain/hugegraph-hubble.md @@ -8,6 +8,8 @@ weight: 1 > **特别注意:** 当前版本的 Hubble 还没有添加 Auth/Login 相关界面和接口和单独防护, 在下一个 Release 版 (＞ 1.5) 会加入, > 请留意避免把它暴露在公网环境或不受信任的网络中，以免引起相关 SEC 问题 (另外也可以使用 **IP & 端口**白名单 + HTTPS) +> +> **测试指南**：如需在本地运行 Hubble 测试，请参考 [工具链本地测试指南](/cn/docs/guides/toolchain-local-test) **HugeGraph-Hubble** 是 HugeGraph 的一站式可视化分析平台，平台涵盖了从数据建模，到数据快速导入， 再到数据的在线、离线分析、以及图的统一管理的全过程，实现了图应用的全流程向导式操作，旨在提升用户的使用流畅度， diff --git a/content/cn/docs/quickstart/toolchain/hugegraph-loader.md b/content/cn/docs/quickstart/toolchain/hugegraph-loader.md index b88a28ef5..0ea2b729c 100644 --- a/content/cn/docs/quickstart/toolchain/hugegraph-loader.md +++ b/content/cn/docs/quickstart/toolchain/hugegraph-loader.md @@ -18,7 +18,9 @@ HugeGraph-Loader 是 HugeGraph 的数据导入组件，能够将多种数据源 后面会具体说明。 -> 注意：使用 HugeGraph-Loader 需要依赖 HugeGraph Server 服务，下载和启动 Server 请参考 [HugeGraph-Server Quick Start](/cn/docs/quickstart/hugegraph/hugegraph-server) +> **注意**：使用 HugeGraph-Loader 需要依赖 HugeGraph Server 服务，下载和启动 Server 请参考 [HugeGraph-Server Quick Start](/cn/docs/quickstart/hugegraph/hugegraph-server) +> +> **测试指南**：如需在本地运行 Loader 测试，请参考 [工具链本地测试指南](/cn/docs/guides/toolchain-local-test) ### 2 获取 HugeGraph-Loader diff --git a/content/cn/docs/quickstart/toolchain/hugegraph-tools.md b/content/cn/docs/quickstart/toolchain/hugegraph-tools.md index 7ee73208d..cd2414ed9 100644 --- a/content/cn/docs/quickstart/toolchain/hugegraph-tools.md +++ b/content/cn/docs/quickstart/toolchain/hugegraph-tools.md @@ -8,6 +8,8 @@ weight: 3 HugeGraph-Tools 是 HugeGraph 的自动化部署、管理和备份/还原组件。 +> **测试指南**：如需在本地运行 Tools 测试，请参考 [工具链本地测试指南](/cn/docs/guides/toolchain-local-test) + ### 2 获取 HugeGraph-Tools 有两种方式可以获取 HugeGraph-Tools：(它被包含子 Toolchain 中) diff --git a/content/en/docs/guides/backup-restore.md b/content/en/docs/guides/backup-restore.md index 9506a54ce..f8ef78f5f 100644 --- a/content/en/docs/guides/backup-restore.md +++ b/content/en/docs/guides/backup-restore.md @@ -1,7 +1,7 @@ --- title: "Backup and Restore" linkTitle: "Backup Restore" -weight: 4 +weight: 5 --- ## Description diff --git a/content/en/docs/guides/faq.md b/content/en/docs/guides/faq.md index 822920881..5dd41caf8 100644 --- a/content/en/docs/guides/faq.md +++ b/content/en/docs/guides/faq.md @@ -1,7 +1,7 @@ --- title: "FAQ" linkTitle: "FAQ" -weight: 5 +weight: 6 --- - How to choose the back-end storage? Choose RocksDB, Cassandra, ScyllaDB, Hbase or Mysql? diff --git a/content/en/docs/guides/security.md b/content/en/docs/guides/security.md index b24b3ef3d..5fde71379 100644 --- a/content/en/docs/guides/security.md +++ b/content/en/docs/guides/security.md @@ -1,7 +1,7 @@ --- title: "Security Report" linkTitle: "Security" -weight: 6 +weight: 7 --- ## Reporting New Security Problems with Apache HugeGraph diff --git a/content/en/docs/guides/toolchain-local-test.md b/content/en/docs/guides/toolchain-local-test.md new file mode 100644 index 000000000..df4661825 --- /dev/null +++ b/content/en/docs/guides/toolchain-local-test.md @@ -0,0 +1,444 @@ +--- +title: "HugeGraph Toolchain Local Testing Guide" +linkTitle: "Toolchain Local Testing" +weight: 4 +--- + +This guide helps developers run HugeGraph toolchain tests locally. + +## 1. Core Concepts + +### 1.1 Core Dependency: HugeGraph Server + +**Integration and functional tests of the toolchain depend on HugeGraph Server**, including Client, Loader, Hubble, Spark Connector, Tools, and other components. + +### 1.2 Test Types + +- **Unit Tests**: Test individual functions/methods, no external dependencies required +- **API Tests (ApiTestSuite)**: Test API interfaces, requires running HugeGraph Server +- **Functional Tests (FuncTestSuite)**: End-to-end tests, require complete system environment + +## 2. Environment Setup + +### 2.1 System Requirements + +- **Operating System**: Linux / macOS (Windows use WSL2) +- **JDK**: >= 11, configure `JAVA_HOME` +- **Maven**: >= 3.5 +- **Python**: >= 3.11 (only required for Hubble tests) + +### 2.2 Clone Code + +```bash +git clone https://github.com/${GITHUB_USER_NAME}/hugegraph-toolchain.git +cd hugegraph-toolchain +``` + +## 3. Deploy Test Environment + +### Deployment Options + +- **Script Deployment (Recommended)**: Precisely control Server version by specifying Commit ID, avoid interface incompatibility +- **Docker Deployment**: Quick start, but may have version lag causing test failures + +> For detailed installation instructions, refer to [Community Documentation](https://hugegraph.apache.org/docs/quickstart/hugegraph/hugegraph-server/) + +### 3.1 Script Deployment (Recommended) + +#### Parameter Description + +- **`$COMMIT_ID`**: Specify Server source code Git Commit ID +- **`$DB_DATABASE` / `$DB_PASS`**: MySQL database name and password for Loader JDBC tests + +#### Deployment Steps + +**1. Install HugeGraph Server** + +```bash +# Set version +export COMMIT_ID="master" # Or specific commit hash, e.g. "8b90977" + +# Execute installation (script located in /assembly/travis/ directory) +hugegraph-client/assembly/travis/install-hugegraph-from-source.sh $COMMIT_ID +``` + +- Default ports: http 8080, https 8443 +- Ensure ports are not occupied + +**2. Install Optional Dependencies** + +```bash +# Hadoop (only required for Loader HDFS tests) +hugegraph-loader/assembly/travis/install-hadoop.sh + +# MySQL (only required for Loader JDBC tests) +hugegraph-loader/assembly/travis/install-mysql.sh $DB_DATABASE $DB_PASS +``` + +**3. Health Check** + +```bash +curl http://localhost:8080/graphs +# Returns {"graphs":["hugegraph"]} indicates success +``` + +### 3.2 Docker Deployment + +> **Note**: Docker images may have version lag, use script deployment if encountering compatibility issues + +#### Quick Start + +```bash +docker network create hugegraph-net +docker run -itd --name=server -p 8080:8080 --network hugegraph-net hugegraph/hugegraph:latest +``` + +#### docker-compose Configuration (Optional) + +Complete configuration example including Server, MySQL, Hadoop services (requires Docker Compose V2): + +```yaml +version: '3.8' + +services: + hugegraph-server: + image: hugegraph/hugegraph:latest # Can be replaced with a specific version, or build your own image + container_name: hugegraph-server + ports: + - "8080:8080" # HugeGraph Server HTTP port + environment: + # Configure HugeGraph Server parameters as needed, e.g., backend storage + - HUGEGRAPH_SERVER_OPTIONS="-Dstore.backend=rocksdb" + volumes: + # If you need to persist data or mount configuration files, add volumes here + # - ./hugegraph-data:/opt/hugegraph/data + healthcheck: + test: ["CMD-SHELL", "curl -f http://localhost:8080/graphs || exit 1"] + interval: 10s + timeout: 3s + retries: 5 + networks: + - hugegraph-net + + # If you need JDBC tests for hugegraph-loader, you can add the following service + # mysql: + # image: mysql:5.7 + # container_name: mysql-db + # environment: + # MYSQL_ROOT_PASSWORD: ${DB_PASS:-your_mysql_root_password} # Read from environment variable, or use default + # MYSQL_DATABASE: ${DB_DATABASE:-hugegraph_test_db} # Read from environment variable, or use default + # ports: + # - "3306:3306" + # volumes: + # - ./mysql-data:/var/lib/mysql # Data persistence + # healthcheck: + # test: ["CMD", "mysqladmin", "ping", "-h", "localhost", "-p${DB_PASS:-your_mysql_root_password}"] + # interval: 5s + # timeout: 3s + # retries: 5 + # networks: + # - hugegraph-net + + # If you need Hadoop/HDFS tests for hugegraph-loader, you can add the following services + # namenode: + # image: johannestang/hadoop-namenode:2.0.0-hadoop2.8.5-java8 + # container_name: namenode + # ports: + # - "0.0.0.0:9870:9870" + # - "0.0.0.0:8020:8020" + # environment: + # - CLUSTER_NAME=test-cluster + # - HDFS_NAMENODE_USER=root + # - HADOOP_CONF_DIR=/hadoop/etc/hadoop + # volumes: + # - ./config/core-site.xml:/hadoop/etc/hadoop/core-site.xml + # - ./config/hdfs-site.xml:/hadoop/etc/hadoop/hdfs-site.xml + # - namenode_data:/hadoop/dfs/name + # command: bash -c "if [ ! -d /hadoop/dfs/name/current ]; then hdfs namenode -format; fi && /entrypoint.sh" + # healthcheck: + # test: ["CMD", "hdfs", "dfsadmin", "-report"] + # interval: 5s + # timeout: 3s + # retries: 5 + # networks: + # - hugegraph-net + + # datanode: + # image: johannestang/hadoop-datanode:2.0.0-hadoop2.8.5-java8 + # container_name: datanode + # depends_on: + # - namenode + # environment: + # - CLUSTER_NAME=test-cluster + # - HDFS_DATANODE_USER=root + # - HADOOP_CONF_DIR=/hadoop/etc/hadoop + # volumes: + # - ./config/core-site.xml:/hadoop/etc/hadoop/core-site.xml + # - ./config/hdfs-site.xml:/hadoop/etc/hadoop/hdfs-site.xml + # - datanode_data:/hadoop/dfs/data + # healthcheck: + # test: ["CMD", "hdfs", "dfsadmin", "-report"] + # interval: 5s + # timeout: 3s + # retries: 5 + # networks: + # - hugegraph-net + +networks: + hugegraph-net: + driver: bridge +volumes: + namenode_data: + datanode_data: +``` + +#### Hadoop Configuration Mounts +Create a `./config` folder in the same directory as `docker-compose.yml` to mount Hadoop configuration files. You can skip this step if HDFS testing is not required. + +📁 `./config/core-site.xml` content: + +```xml + + + fs.defaultFS + hdfs://namenode:8020 + + +``` + +📁 `./config/hdfs-site.xml` content: + +```xml + + + dfs.namenode.name.dir + /hadoop/hdfs/name + + + dfs.datanode.data.dir + /hadoop/hdfs/data + + + dfs.permissions.superusergroup + hadoop + + + dfs.support.append + true + + +``` + +#### Docker Operations + +```bash +# Start services +docker compose up -d + +# Check status +docker compose ps +lsof -i:8080 # Server +lsof -i:8020 # Hadoop +lsof -i:3306 # MySQL + +# Stop services +docker compose down +``` + +## 4. Run Tests + +Test process for each tool: + +
+ HugeGraph Toolchain Testing Process +
+ + +### 4.1 hugegraph-client + +#### Compile + +```bash +mvn -e compile -pl hugegraph-client -Dmaven.javadoc.skip=true -ntp +``` + +#### Dependent Services + +Start HugeGraph Server (refer to [Section 3](#3-deploy-test-environment)) + +##### Server Authentication Configuration + +> **Note**: Docker images <= 1.5.0 don't support authentication tests, need 1.6.0+ + +ApiTest requires authentication configuration. Skip this if using script installation. Manual configuration needed for Docker: + +```bash +# 1. Modify authentication mode +cp conf/rest-server.properties conf/rest-server.properties.backup +sed -i '/^auth.authenticator=/c\auth.authenticator=org.apache.hugegraph.auth.StandardAuthenticator' conf/rest-server.properties +grep auth.authenticator conf/rest-server.properties + +# 2. Set password +# Note: Test code uses "pa" as default password, must match for tests to work +bin/stop-hugegraph.sh +export PASSWORD="pa" # Set to test default password +echo -e "${PASSWORD}" | bin/init-store.sh +bin/start-hugegraph.sh +``` + +#### Run Tests + +```bash +# Check environment +curl http://localhost:8080/graphs # Should return {"graphs":["hugegraph"]} +curl -u admin:pa http://localhost:8080/graphs # Authentication test (pa is test default password) + +# Run tests +cd hugegraph-client +mvn test -Dtest=UnitTestSuite -ntp # Unit tests +mvn test -Dtest=ApiTestSuite -ntp # API tests (requires Server) +mvn test -Dtest=FuncTestSuite -ntp # Functional tests (requires Server) +``` + +> Check Server log if tests fail: `logs/hugegraph-server.log` + +### 4.2 hugegraph-loader + +#### Compile + +```bash +mvn install -pl hugegraph-client,hugegraph-loader -am -Dmaven.javadoc.skip=true -DskipTests -ntp +``` + +#### Dependent Services + +- **Required**: HugeGraph Server +- **Optional**: Hadoop (HDFS tests), MySQL (JDBC tests) + +#### Run Tests + +```bash +cd hugegraph-loader +mvn test -P unit -ntp # Unit tests +mvn test -P file -ntp # File tests (requires Server) +mvn test -P hdfs -ntp # HDFS tests (requires Server + Hadoop) +mvn test -P jdbc -ntp # JDBC tests (requires Server + MySQL) +mvn test -P kafka -ntp # Kafka tests (requires Server) +``` + +### 4.3 hugegraph-hubble + +#### Compile + +```bash +mvn install -pl hugegraph-client,hugegraph-loader -am -Dmaven.javadoc.skip=true -DskipTests -ntp +cd hugegraph-hubble +mvn -e compile -Dmaven.javadoc.skip=true -ntp +``` + +#### Dependent Services + +**1. Start Server** (refer to [Section 3](#3-deploy-test-environment)) + +**2. Python Environment** + +```bash +python -m venv venv +source venv/bin/activate # Windows: venv\Scripts\activate +python -m pip install -r hubble-dist/assembly/travis/requirements.txt +``` + +**3. Build and Verify** + +```bash +mvn package -Dmaven.test.skip=true +# Optional: Start and verify +cd apache-hugegraph-hubble-incubating-*/bin +./start-hubble.sh -d && sleep 10 +curl http://localhost:8088/api/health +./stop-hubble.sh +``` + +#### Run Tests + +```bash +# Unit tests +mvn test -P unit-test -pl hugegraph-hubble/hubble-be -ntp + +# API tests (requires Server + Hubble running) +curl http://localhost:8080/graphs # Check Server +curl http://localhost:8088/api/health # Check Hubble +cd hugegraph-hubble/hubble-dist +./assembly/travis/run-api-test.sh +``` + +### 4.4 hugegraph-spark-connector + +#### Compile + +```bash +mvn install -pl hugegraph-client,hugegraph-spark-connector -am -Dmaven.javadoc.skip=true -DskipTests -ntp +``` + +#### Run Tests + +```bash +cd hugegraph-spark-connector +mvn test -ntp # Requires Server running +``` + +### 4.5 hugegraph-tools + +#### Compile + +```bash +mvn install -pl hugegraph-client,hugegraph-tools -am -Dmaven.javadoc.skip=true -DskipTests -ntp +``` + +#### Run Tests + +```bash +cd hugegraph-tools +mvn test -Dtest=FuncTestSuite -ntp # Requires Server running +``` + +## 5. Common Issues + +### Service Connection Issues + +**Symptoms**: Cannot connect to Server/MySQL/Hadoop + +**Troubleshooting**: +- Confirm services are running (Server must be on port 8080) +- Check port usage: `lsof -i:8080` +- Docker check: `docker compose ps` and `docker compose logs` + +### Configuration Issues + +**Symptoms**: File not found, parameter errors + +**Troubleshooting**: +- Check environment variables: `echo $COMMIT_ID` +- Script permissions: `chmod +x hugegraph-*/assembly/travis/*.sh` + +### HDFS Test Failures + +**Troubleshooting**: +- Confirm NameNode/DataNode running normally +- Check Hadoop logs +- Verify HDFS connection: `hdfs dfsadmin -report` + +### JDBC Test Failures + +**Troubleshooting**: +- Confirm MySQL running normally +- Verify database connection: `mysql -u root -p$DB_PASS` +- Check MySQL logs + +## 6. References + +* **HugeGraph GitHub Repository**: [https://github.com/apache/hugegraph](https://github.com/apache/hugegraph) +* **HugeGraph Toolchain GitHub Repository**: [https://github.com/apache/hugegraph-toolchain](https://github.com/apache/hugegraph-toolchain) +* **HugeGraph Server Official Documentation**: [https://hugegraph.apache.org/docs/quickstart/hugegraph/hugegraph-server/](https://hugegraph.apache.org/docs/quickstart/hugegraph/hugegraph-server/) +* **CI Script Path**: `.github/workflows/*-ci.yml` (CI configuration files in the HugeGraph toolchain project, which can be used as a reference) +* **Dependent Service Installation Scripts**: `hugegraph-*/assembly/travis/` (Installation scripts for CI and local testing in the HugeGraph toolchain project, can be used directly or as a reference) diff --git a/content/en/docs/images/toolchain-test-mermaid-1.png b/content/en/docs/images/toolchain-test-mermaid-1.png new file mode 100644 index 000000000..d7dd096ab Binary files /dev/null and b/content/en/docs/images/toolchain-test-mermaid-1.png differ diff --git a/content/en/docs/images/toolchain-test-mermaid-2.png b/content/en/docs/images/toolchain-test-mermaid-2.png new file mode 100644 index 000000000..d8e38c1b5 Binary files /dev/null and b/content/en/docs/images/toolchain-test-mermaid-2.png differ diff --git a/content/en/docs/quickstart/toolchain/_index.md b/content/en/docs/quickstart/toolchain/_index.md index 45f468c46..6c06a88cf 100644 --- a/content/en/docs/quickstart/toolchain/_index.md +++ b/content/en/docs/quickstart/toolchain/_index.md @@ -3,3 +3,5 @@ title: "HugeGraph ToolChain" linkTitle: "HugeGraph ToolChain" weight: 2 --- + +> **Testing Guide**: For running toolchain tests locally, please refer to [HugeGraph Toolchain Local Testing Guide](/docs/guides/toolchain-local-test) diff --git a/content/en/docs/quickstart/toolchain/hugegraph-hubble.md b/content/en/docs/quickstart/toolchain/hugegraph-hubble.md index 26b1a7d3d..d73403f0c 100644 --- a/content/en/docs/quickstart/toolchain/hugegraph-hubble.md +++ b/content/en/docs/quickstart/toolchain/hugegraph-hubble.md @@ -11,6 +11,8 @@ weight: 1 > Please be careful not to expose it in a public network environment or untrusted networks to > avoid related SEC issues (you can also use IP & port **whitelist** + HTTPS) +> **Testing Guide**: For running HugeGraph-Hubble tests locally, please refer to [HugeGraph Toolchain Local Testing Guide](/docs/guides/toolchain-local-test) + **HugeGraph-Hubble** is HugeGraph's one-stop visual analysis platform. The platform covers the whole process from data modeling, to efficient data import, to real-time and offline analysis of data, and unified management of graphs, realizing the whole process wizard of graph application. It is designed diff --git a/content/en/docs/quickstart/toolchain/hugegraph-loader.md b/content/en/docs/quickstart/toolchain/hugegraph-loader.md index 8c625f36d..a8ec5ec4d 100644 --- a/content/en/docs/quickstart/toolchain/hugegraph-loader.md +++ b/content/en/docs/quickstart/toolchain/hugegraph-loader.md @@ -19,6 +19,8 @@ It will be explained in detail below. > Note: HugeGraph-Loader requires HugeGraph Server service, please refer to [HugeGraph-Server Quick Start](/docs/quickstart/hugegraph/hugegraph-server) to download and start Server +> **Testing Guide**: For running HugeGraph-Loader tests locally, please refer to [HugeGraph Toolchain Local Testing Guide](/docs/guides/toolchain-local-test) + ### 2 Get HugeGraph-Loader There are two ways to get HugeGraph-Loader: diff --git a/content/en/docs/quickstart/toolchain/hugegraph-tools.md b/content/en/docs/quickstart/toolchain/hugegraph-tools.md index 05f566b5b..c55199f8e 100644 --- a/content/en/docs/quickstart/toolchain/hugegraph-tools.md +++ b/content/en/docs/quickstart/toolchain/hugegraph-tools.md @@ -8,6 +8,8 @@ weight: 3 HugeGraph-Tools is an automated deployment, management and backup/restore component of HugeGraph. +> **Testing Guide**: For running HugeGraph-Tools tests locally, please refer to [HugeGraph Toolchain Local Testing Guide](/docs/guides/toolchain-local-test) + ### 2 Get HugeGraph-Tools There are two ways to get HugeGraph-Tools: