Introduce Configurable Upper-Limits/default values for max retries and retry Interval in Error Handling(#12787)

[kazuhisa-wada]

Summary

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Close #12787

Screenshots

Before change

Default retry config in dify cloud version

LLM

• default is disable
• if enabled, default max_retry=3, retry_interval=1000ms
• important note: left side is default i.e. status immediately after node is created. right side is status immediately after enabled.

• upper limit is max_retry=10, retry_interval=5000ms

HTTP

• default is enabled
• if enabled, default max_retry=3, retry_interval=100ms
• This image is the one immediately after HTTP node is created.

• upper limit is max_retry=10, retry_interval=5000ms

Tool node

• default is disable
• if enabled, default max_retry=3, retry_interval=1000ms
• important note: left side is default i.e. status immediately after node is created. right side is status immediately after enabled.

• upper limit is max_retry=10, retry_interval=5000ms

After change

to test if behavior with commited code is the same as that before the change

LLM

• default is disable
• if enabled, default max_retry=3, retry_interval=1000ms
• important note: left side is default i.e. status immediately after node is created. right side is status immediately after enabled.

• upper limit is max_retry=10, retry_interval=5000ms

HTTP

• default is enabled, default max_retry=3, retry_interval=100ms
• This image is the one immediately after HTTP node is created.
• upper limit is max_retry=10, retry_interval=5000ms

left is to to check default, right is to to check upper limit

Tool node

• default is disable
• if enabled, default max_retry=3, retry_interval=1000ms
• important note: left side is default i.e. status immediately after node is created. right side is status immediately after enabled.

• upper limit is max_retry=10, retry_interval=5000ms

to test if config file works as expected

LLM

• retry is disabled, default max_retry=2, retry_interval=200ms as default
• upper limit: max_retry=8, retry_interval=2000

// Retries related parameters of LLM Node
export const MAX_RETRIES_DEFAULT_LLM_NODE = 2
export const RETRY_INTERVAL_DEFAULT_LLM_NODE = 200
export const RETRY_ENABLED_DEFAULT_LLM_NODE = false
export const MAX_RETRIES_UPPER_BOUND_LLM_NODE = 10
export const RETRY_INTERVAL_UPPER_BOUND_LLM_NODE = 2000

HTTP

• retry is disabled, default max_retry=1, retry_interval=900ms as default
• upper limit: max_retry=4, retry_interval=9000

// Retries related parameters of HTTP Node
export const MAX_RETRIES_DEFAULT_HTTP_NODE = 1
export const RETRY_INTERVAL_DEFAULT_HTTP_NODE = 900
export const RETRY_ENABLED_DEFAULT_HTTP_NODE = false
export const MAX_RETRIES_UPPER_BOUND_HTTP_NODE = 4
export const RETRY_INTERVAL_UPPER_BOUND_HTTP_NODE = 9000

Tool

• retry is disabled, default max_retry=6, retry_interval=600ms as default
• upper limit: max_retry=7, retry_interval=6666

// Retries related parameters of Tool Node
export const MAX_RETRIES_DEFAULT_TOOL_NODE = 6
export const RETRY_INTERVAL_DEFAULT_TOOL_NODE = 600
export const RETRY_ENABLED_DEFAULT_TOOL_NODE = false
export const MAX_RETRIES_UPPER_BOUND_TOOL_NODE = 7
export const RETRY_INTERVAL_UPPER_BOUND_TOOL_NODE = 6666

Checklist

Important

Please review the checklist below before submitting your pull request.

• This change requires a documentation update, included: Dify Document
• I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
• I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
• I've updated the documentation accordingly.
• I ran dev/reformat(backend) and cd web && npx lint-staged(frontend) to appease the lint gods

[crazywoola]

This is a good idea. However I would recommend moving these parameters into docker-compose-template, see this below

dify/docker/docker-compose-template.yaml

Line 57 in 6e0fb05

TEXT_GENERATION_TIMEOUT_MS: ${TEXT_GENERATION_TIMEOUT_MS:-60000}

We have an environment config regarding the timeout settings for frontend images, your approach won't take effect unless it is rebuilt again or start from the source code. By moving those into template will make sure they can be changed every time.

[kazuhisa-wada]

@crazywoola
100% Agree, thanks for your review.
Let me do additional these changes. I will push those changes later once I implement it and test it.

Thanks!

[crazywoola]

FYI https://github.com/langgenius/dify/pull/12925/files