diff --git a/buffer/README.md b/buffer/README.md index e2e863dc..989d44b4 100644 --- a/buffer/README.md +++ b/buffer/README.md @@ -77,7 +77,37 @@ Fluentd will abort the attempt to transfer the failing chunks on the following c \(default: `72h`\) -In these events, **all chunks in the queue are discarded.** If you want to avoid this, you can enable `retry_forever` to make Fluentd retry indefinitely. +Before v1.19.0 (fluent-package v6), **all chunks in the queue are discarded** in these events. +If you want to avoid this, you can enable `retry_forever` to make Fluentd retry indefinitely or enable `secondary` output. + +Since v1.19.0 (fluent-package v6), the file buffer plugins ([`buf_file`](file.md) and [`buf_file_single`](file_single.md)) provide a safer and more user-friendly feature. +They evacuate the chunk files in the queue to the following directory before clearing the queue. + +```text +${root_dir}/buffer/${plugin_id}/ +``` + +`${root_dir}` is determined by the parameter `root_dir` in ``. +If you do not configure this parameter, it will be `/tmp/fluent/`. + +After the issues, such as network failures, are resolved, you can recover these chunks by moving the evacuated chunk files back to the plugin's buffer directory and restarting Fluentd. + +{% hint style='info' %} +If the plugin runs with multi-worker, the original buffer directory should be separated per worker. The evacuated chunk files can be restored to any worker's directory. +{% endhint %} + +{% hint style='info' %} +You can use [zero-downtime restart](../deployment/zero-downtime-restart.md) to prevent downtime of some input plugins like `in_tcp`. +{% endhint %} + +{% hint style='info' %} +For third-party buffer plugins, it is necessary to implement a function to support this feature. +Please see [How to Write Buffer Plugin](../plugin-development/api-plugin-buffer.md) for details. +{% endhint %} + +{% hint style='warning' %} +Please note that [`buf_memory`](memory.md) does not support this feature. +{% endhint %} ### Handling Unrecoverable Errors diff --git a/plugin-development/api-plugin-buffer.md b/plugin-development/api-plugin-buffer.md index 7992167f..6afe8bef 100644 --- a/plugin-development/api-plugin-buffer.md +++ b/plugin-development/api-plugin-buffer.md @@ -2,5 +2,28 @@ TODO: Write +## Methods + +### evacuate_chunk + +You can override this method to add feature to evacuate chunks before clearing the queue when reaching the retry limit. +See [Buffer - Handling Successive Failures](../buffer/README.md#handling-successive-failures) for details. + +```ruby +def evacuate_chunk(chunk) + unless chunk.is_a?(Fluent::Plugin::Buffer::FileChunk) + raise ArgumentError, "The chunk must be FileChunk, but it was #{chunk.class}." + end + + backup_dir = File.join(backup_base_dir, 'buffer', safe_owner_id) + FileUtils.mkdir_p(backup_dir, mode: system_config.dir_permission || Fluent::DEFAULT_DIR_PERMISSION) unless Dir.exist?(backup_dir) + + FileUtils.copy([chunk.path, chunk.meta_path], backup_dir) + log.warn "chunk files are evacuated to #{backup_dir}.", chunk_id: dump_unique_id_hex(chunk.unique_id) +rescue => e + log.error "unexpected error while evacuating chunk files.", error: e +end +``` + If this article is incorrect or outdated, or omits critical information, please [let us know](https://github.com/fluent/fluentd-docs-gitbook/issues?state=open). [Fluentd](http://www.fluentd.org/) is an open-source project under [Cloud Native Computing Foundation \(CNCF\)](https://cncf.io/). All components are available under the Apache 2 License.