V6 support v0.3.7.post2

.github/pull_request_template.md

@@ -0,0 +1,29 @@
+## Description
+
+<!-- A clear and concise description of the changes. Link to any relevant issues. -->
+
+## Type of Change
+
+* Types
+  - [ ] Bug fix
+  - [ ] New feature
+    - [ ] Transfer Engine
+    - [ ] Mooncake Store
+    - [ ] Mooncake EP
+    - [ ] Integration
+    - [ ] P2P Store
+    - [ ] Python Wheel
+  - [ ] Breaking change
+  - [ ] CI/CD
+  - [ ] Documentation update
+  - [ ] Other
+
+## How Has This Been Tested?
+
+<!-- Please describe the tests you've run to verify your changes. -->
+
+## Checklist
+
+- [ ] I have performed a self-review of my own code.
+- [ ] I have updated the documentation.
+- [ ] I have added tests to prove my changes are effective.

.github/workflows/ci.yml

@@ -166,6 +166,28 @@ jobs:
         deactivate
       shell: bash
 
+    - name: Start Mooncake Master
+      run: |
+        source test_env/bin/activate
+        mkdir -p /tmp/mooncake_storage
+        mooncake_master \
+          --eviction_high_watermark_ratio=0.95 \
+          --cluster_id=ci_test_cluster \
+          --port 50051 &
+        sleep 3
+      shell: bash
+
+    - name: Run Python Tensor API Performance Test (CI check)
+      env:
+        MOONCAKE_MASTER: "127.0.0.1:50051"
+        MOONCAKE_TE_META_DATA_SERVER: "http://127.0.0.1:8080/metadata"
+        MOONCAKE_PROTOCOL: "tcp"
+        LOCAL_HOSTNAME: "127.0.0.1"
+      run: |
+        source test_env/bin/activate
+        python scripts/test_tensor_api.py -n 1
+      shell: bash
+
   build-flags:
     runs-on: ubuntu-22.04
     strategy:

CMakeLists.txt

@@ -12,6 +12,7 @@ if (BUILD_UNIT_TESTS)
   enable_testing()
 endif()
 
+option(WITH_TE "build mooncake transfer engine and sample code" ON)
 option(WITH_STORE "build mooncake store library and sample code" ON)
 option(WITH_P2P_STORE "build p2p store library and sample code" OFF)
 option(WITH_RUST_EXAMPLE "build the Rust interface and sample code for the transfer engine" OFF)
@@ -45,8 +46,10 @@ add_subdirectory(mooncake-common)
 include_directories(mooncake-common/etcd)
 include_directories(mooncake-common/include)
 
-add_subdirectory(mooncake-transfer-engine)
-include_directories(mooncake-transfer-engine/include)
+if (WITH_TE)
+  add_subdirectory(mooncake-transfer-engine)
+  include_directories(mooncake-transfer-engine/include)
+endif()
 
 if (WITH_STORE)
   message(STATUS "Mooncake Store will be built")

CODE_OF_CONDUCT.md

@@ -0,0 +1,128 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our
+community a harassment-free experience for everyone, regardless of age, body
+size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity and expression, level of experience, education, socio-economic status,
+nationality, personal appearance, race, religion, or sexual identity
+and orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming,
+diverse, inclusive, and healthy community.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment for our
+community include:
+
+* Demonstrating empathy and kindness toward other people
+* Being respectful of differing opinions, viewpoints, and experiences
+* Giving and gracefully accepting constructive feedback
+* Accepting responsibility and apologizing to those affected by our mistakes,
+  and learning from the experience
+* Focusing on what is best not just for us as individuals, but for the
+  overall community
+
+Examples of unacceptable behavior include:
+
+* The use of sexualized language or imagery, and sexual attention or
+  advances of any kind
+* Trolling, insulting or derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or email
+  address, without their explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Enforcement Responsibilities
+
+Community leaders are responsible for clarifying and enforcing our standards of
+acceptable behavior and will take appropriate and fair corrective action in
+response to any behavior that they deem inappropriate, threatening, offensive,
+or harmful.
+
+Community leaders have the right and responsibility to remove, edit, or reject
+comments, commits, code, wiki edits, issues, and other contributions that are
+not aligned to this Code of Conduct, and will communicate reasons for moderation
+decisions when appropriate.
+
+## Scope
+
+This Code of Conduct applies within all community spaces, and also applies when
+an individual is officially representing the community in public spaces.
+Examples of representing our community include using an official e-mail address,
+posting via an official social media account, or acting as an appointed
+representative at an online or offline event.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported to the community leaders responsible for enforcement at
+.
+All complaints will be reviewed and investigated promptly and fairly.
+
+All community leaders are obligated to respect the privacy and security of the
+reporter of any incident.
+
+## Enforcement Guidelines
+
+Community leaders will follow these Community Impact Guidelines in determining
+the consequences for any action they deem in violation of this Code of Conduct:
+
+### 1. Correction
+
+**Community Impact**: Use of inappropriate language or other behavior deemed
+unprofessional or unwelcome in the community.
+
+**Consequence**: A private, written warning from community leaders, providing
+clarity around the nature of the violation and an explanation of why the
+behavior was inappropriate. A public apology may be requested.
+
+### 2. Warning
+
+**Community Impact**: A violation through a single incident or series
+of actions.
+
+**Consequence**: A warning with consequences for continued behavior. No
+interaction with the people involved, including unsolicited interaction with
+those enforcing the Code of Conduct, for a specified period of time. This
+includes avoiding interactions in community spaces as well as external channels
+like social media. Violating these terms may lead to a temporary or
+permanent ban.
+
+### 3. Temporary Ban
+
+**Community Impact**: A serious violation of community standards, including
+sustained inappropriate behavior.
+
+**Consequence**: A temporary ban from any sort of interaction or public
+communication with the community for a specified period of time. No public or
+private interaction with the people involved, including unsolicited interaction
+with those enforcing the Code of Conduct, is allowed during this period.
+Violating these terms may lead to a permanent ban.
+
+### 4. Permanent Ban
+
+**Community Impact**: Demonstrating a pattern of violation of community
+standards, including sustained inappropriate behavior,  harassment of an
+individual, or aggression toward or disparagement of classes of individuals.
+
+**Consequence**: A permanent ban from any sort of public interaction within
+the community.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 2.0, available at
+https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.
+
+Community Impact Guidelines were inspired by [Mozilla's code of conduct
+enforcement ladder](https://github.com/mozilla/diversity).
+
+[homepage]: https://www.contributor-covenant.org
+
+For answers to common questions about this code of conduct, see the FAQ at
+https://www.contributor-covenant.org/faq. Translations are available at
+https://www.contributor-covenant.org/translations.

README.md

@@ -8,7 +8,7 @@
   | <a href="FAST25-release/traces" target="_blank"><strong>Traces</strong></a>
   | <a href="https://arxiv.org/abs/2407.00079" target="_blank"><strong>Technical Report</strong></a>
   | <a href="https://kvcache-ai.github.io/Mooncake/" target="_blank"><strong>Blog</strong></a>
-  | <a href="https://join.slack.com/t/mooncake-project/shared_invite/zt-365h0e2tz-rfXTibSuGKIzik3GUYBJbg" target="_blank"><strong>Slack</strong></a>
+  | <a href="https://join.slack.com/t/mooncake-project/shared_invite/zt-3ig4fjai8-KH1zIm3x8Vm8WqyH0i_JaA" target="_blank"><strong>Slack</strong></a>
   <br />
   <br />
 
@@ -18,6 +18,7 @@
   [![PyPI - Downloads](https://img.shields.io/pypi/dm/mooncake-transfer-engine)](https://pypi.org/project/mooncake-transfer-engine)
   [![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/kvcache-ai/Mooncake)
   [![GitHub commit activity](https://img.shields.io/github/commit-activity/w/kvcache-ai/Mooncake)](https://github.com/kvcache-ai/Mooncake/graphs/commit-activity)
+  [![license](https://img.shields.io/github/license/kvcache-ai/mooncake.svg)](https://github.com/kvcache-ai/Mooncake/blob/main/LICENSE-APACHE)
 
 </div>
 <br/>

doc/en/mooncake-store.md

@@ -488,6 +488,18 @@ There are two startup parameters in `master_service` related to the soft pin mec
 
 Notably, soft pinned objects can still be removed using APIs such as `Remove` or `RemoveAll`.
 
+### Zombie Object Cleanup
+
+If a Client crashes or experiences a network failure after sending a `PutStart` request but before it can send the corresponding `PutEnd` or `PutRevoke` request to the Master, the object initiated by `PutStart` enters a "zombie" state—rendering it neither usable nor deletable. The existence of such "zombie objects" not only consumes storage space but also prevents subsequent `Put` operations on the same keys. To mitigate these issues, the Master records the start time of each `PutStart` request and employs two timeout thresholds—`put_start_discard_timeout` and `put_start_release_timeout`—to clean up zombie objects.
+
+#### `PutStart` Preemption
+
+If an object receives neither a `PutEnd` nor a `PutRevoke` request within `put_start_discard_timeout` (default: 30 seconds) after its `PutStart`, any subsequent `PutStart` request for the same object will be allowed to "preempt" the previous `PutStart`. This enables the new request to proceed with writing the object, thereby preventing a single faulty Client from permanently blocking access to that object. Note that during such preemption, the storage space allocated by the old `PutStart` is not reused; instead, new space is allocated for the preempting `PutStart`. The space previously allocated by the old `PutStart` will be reclaimed via the mechanism described below.
+
+#### Space Reclaim
+
+Replica space allocated during a `PutStart` is considered releasable by the Master if the write operation is neither completed (via `PutEnd`) nor canceled (via `PutRevoke`) within `put_start_release_timeout` (default: 10 minutes) after the `PutStart`. When object eviction is triggered—either due to allocation failures or because storage utilization exceeds the configured threshold—these releasable replica spaces are prioritized for release to reclaim storage capacity.
+
 ### Preferred Segment Allocation
 
 Mooncake Store provides a **preferred segment allocation** feature that allows users to specify a preferred storage segment (node) for object allocation. This feature is particularly useful for optimizing data locality and reducing network overhead in distributed scenarios.

doc/en/transfer-engine.md

@@ -443,3 +443,4 @@ For advanced users, TransferEngine provides the following advanced runtime optio
 - `MC_MIN_PRC_PORT` Specifies the minimum port number for RPC service. The default value is 15000.
 - `MC_MAX_PRC_PORT` Specifies the maximum port number for RPC service. The default value is 17000.
 - `MC_PATH_ROUNDROBIN` Use round-robin mode in the RDMA path selection. This may be beneficial for transferring large bulks.
+- `MC_ENDPOINT_STORE_TYPE` Choose FIFO Endpoint Store (`FIFO`) or Sieve Endpoint Store (`SIEVE`), default is `SIEVE`.

doc/en/troubleshooting.md

@@ -64,6 +64,13 @@ In addition, if the error `Failed to get description of XXX` is displayed, it in
 
 ## SGLang Common Questions
 
+### Do I need RDMA to run SGLang and Mooncake?
+
+When using Mooncake for KV cache transfer in SGLang PD disaggregation deployments, GPUDirect RDMA (GDR) is required.
+
+When using Mooncake as a KV cache storage backend in SGLang HiCache, RDMA is recommended for better performance.
+However, if RDMA NICs are not available, the TCP protocol is also supported.
+
 ### How to make sure GPUDirect RDMA (GDR) is supported
 
 1. Verify the presence of an RDMA-capable NIC (e.g., Mellanox, ERDMA) and drivers.
@@ -84,7 +91,7 @@ lsmod | grep peer_mem
 lsmod | grep nvidia_peer_mem
 ```
 
-3. If you use container to run SGLang, please make sure RDMA and GDR driver are installed in the container and run container in previledge mode. Requirements: (1) privileged mode must be enabled. (2) RDMA devices/NVIDIA devices mounted into container
+3. If you use container to run SGLang, please make sure RDMA and GDR driver are installed in the container and run container in privileged mode. Requirements: (1) privileged mode must be enabled. (2) RDMA devices/NVIDIA devices mounted into container
 
 4. Check the connectivity
 Benchmark end-to-end performance using ib_write_bw.
@@ -93,7 +100,6 @@ apt install perftest
 # server side
 ib_write_bw -d [rdma_device] -R -x gdr
 # client side
-# server side
 ib_write_bw -d [rdma_device] -R -x gdr [server_ip]
 ```
 Expected Output:

doc/zh/mooncake-store.md

@@ -492,6 +492,18 @@ virtual std::unique_ptr<AllocatedBuffer> Allocate(
 
 被软固定的对象仍然可以通过 `Remove`、`RemoveAll` 等 API 主动删除。
 
+### 僵尸对象清理机制
+
+如果Client因为进程崩溃或网络故障等原因，在发送完`PutStart`请求后无法向Master发送对应的`PutEnd`或`PutRevoke`请求，就会导致`PutStart`的对象处于无法使用也无法删除的“僵尸”状态。“僵尸对象”的存在不仅会占用存储空间，还会导致后续对相同key的`Put`操作无法进行。为了避免这些问题，Master会记录每个对象`PutStart`请求的开始时间，并基于两个超时时间：`put_start_discard_timeout`和`put_start_release_timeout`，对僵尸对象进行清理。
+
+#### `PutStart`顶替
+
+如果一个对象在`PutStart`后的`put_start_discard_timeout`（默认为30秒）时间内没有收到任何的`PutEnd`或是`PutRevoke`请求，那么后续新来的对该对象的`PutStart`操作将能够“顶替”旧的`PutStart`操作，继续进行对该对象的写入，从而避免单个Client的故障导致一些对象永远无法使用。需要注意的是，在发生“顶替”时，不会复用旧`PutStart`分配的空间，而是会重新分配空间供新`PutStart`使用，旧`PutStart`分配的空间将通过下述机制进行回收。
+
+#### 空间回收
+
+在`PutStart`中为对象分配的副本空间，如果在`PutStart`后的`put_start_release_timeout`（默认为10分钟）时间内没有完成写入（收到`PutEnd`）或被撤销（收到`PutRevoke`），将会被Master视为是可释放的。在因空间分配失败或空间使用率高于设定水位而触发对象淘汰时，这些可释放的对象副本空间将会被优先释放以回收存储空间。
+
 ### 首选段分配
 
 Mooncake Store 提供了**首选段分配**功能，允许用户为对象分配指定首选的存储段（节点）。此功能特别适用于优化数据局部性和减少分布式场景中的网络开销。

doc/zh/transfer-engine.md

@@ -415,4 +415,5 @@ int init(const std::string &metadata_conn_string,
 - `MC_FORCE_TCP` 强制使用 TCP 作为主要传输方式，无论是否安装了有效的 RDMA 网卡
 - `MC_MIN_PRC_PORT` 指定 RPC 服务使用的最小端口号。默认值为 15000。
 - `MC_MAX_PRC_PORT` 指定 RPC 服务使用的最大端口号。默认值为 17000。
-- `MC_PATH_ROUNDROBIN` 指定 RDMA 路径选择使用 Round Robin 模式，这对于传输大块数据可能有利。
+- `MC_PATH_ROUNDROBIN` 指定 RDMA 路径选择使用 Round Robin 模式，这对于传输大块数据可能有利。
+- `MC_ENDPOINT_STORE_TYPE` 选择 FIFO Endpoint Store (`FIFO`) 或者 Sieve Endpoint Store (`SIEVE`)，模式是 `SIEVE`。

docs/source/design/mooncake-store.md

@@ -488,6 +488,18 @@ There are two startup parameters in `master_service` related to the soft pin mec
 
 Notably, soft pinned objects can still be removed using APIs such as `Remove` or `RemoveAll`.
 
+### Zombie Object Cleanup
+
+If a Client crashes or experiences a network failure after sending a `PutStart` request but before it can send the corresponding `PutEnd` or `PutRevoke` request to the Master, the object initiated by `PutStart` enters a "zombie" state—rendering it neither usable nor deletable. The existence of such "zombie objects" not only consumes storage space but also prevents subsequent `Put` operations on the same keys. To mitigate these issues, the Master records the start time of each `PutStart` request and employs two timeout thresholds—`put_start_discard_timeout` and `put_start_release_timeout`—to clean up zombie objects.
+
+#### `PutStart` Preemption
+
+If an object receives neither a `PutEnd` nor a `PutRevoke` request within `put_start_discard_timeout` (default: 30 seconds) after its `PutStart`, any subsequent `PutStart` request for the same object will be allowed to "preempt" the previous `PutStart`. This enables the new request to proceed with writing the object, thereby preventing a single faulty Client from permanently blocking access to that object. Note that during such preemption, the storage space allocated by the old `PutStart` is not reused; instead, new space is allocated for the preempting `PutStart`. The space previously allocated by the old `PutStart` will be reclaimed via the mechanism described below.
+
+#### Space Reclaim
+
+Replica space allocated during a `PutStart` is considered releasable by the Master if the write operation is neither completed (via `PutEnd`) nor canceled (via `PutRevoke`) within `put_start_release_timeout` (default: 10 minutes) after the `PutStart`. When object eviction is triggered—either due to allocation failures or because storage utilization exceeds the configured threshold—these releasable replica spaces are prioritized for release to reclaim storage capacity.
+
 ### Preferred Segment Allocation
 
 Mooncake Store provides a **preferred segment allocation** feature that allows users to specify a preferred storage segment (node) for object allocation. This feature is particularly useful for optimizing data locality and reducing network overhead in distributed scenarios.