Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce Configurable Upper-Limits/default values for max retries and retry Interval in Error Handling(#12787) #12788

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
+81 −20
Open

Introduce Configurable Upper-Limits/default values for max retries and retry Interval in Error Handling(#12787) #12788

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
c4ca97c
add flexibility on upper bounds of max_retry and retry intervals
kazuhisa-wada Jan 15, 2025
7a6f56e
add flexibility on upper bounds of max_retry and retry intervals for …
kazuhisa-wada Jan 15, 2025
109ef7d
add constraints related parameters in config about retry
kazuhisa-wada Jan 15, 2025
71c7109
changing initialNode on retry related initiation in response to chang…
kazuhisa-wada Jan 15, 2025
6f7c91b
optimize config file so behavior is exatly the same as-is
kazuhisa-wada Jan 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 25 additions & 17 deletions web/app/components/workflow/nodes/_base/components/retry/retry-on-panel.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import type {
Node,
} from '@/app/components/workflow/types'
import Split from '@/app/components/workflow/nodes/_base/components/split'
import { MAX_RETRIES_DEFAULT, MAX_RETRIES_UPPER_BOUND_DEFAULT, RETRY_INTERVAL_DEFAULT, RETRY_INTERVAL_UPPER_BOUND_DEFAULT } from '@/config'

type RetryOnPanelProps = Pick<Node, 'id' | 'data'>
const RetryOnPanel = ({
Expand All @@ -21,32 +22,39 @@ const RetryOnPanel = ({
const handleRetryEnabledChange = (value: boolean) => {
handleRetryConfigChange({
retry_enabled: value,
max_retries: retry_config?.max_retries || 3,
retry_interval: retry_config?.retry_interval || 1000,
max_retries: retry_config?.max_retries || MAX_RETRIES_DEFAULT,
retry_interval: retry_config?.retry_interval || RETRY_INTERVAL_DEFAULT,
max_retries_upper_bound: retry_config?.max_retries_upper_bound || MAX_RETRIES_UPPER_BOUND_DEFAULT,
retry_interval_upper_bound: retry_config?.retry_interval_upper_bound || RETRY_INTERVAL_UPPER_BOUND_DEFAULT,
})
}

const handleMaxRetriesChange = (value: number) => {
if (value > 10)
value = 10
if (value > (retry_config?.max_retries_upper_bound || MAX_RETRIES_UPPER_BOUND_DEFAULT))
value = (retry_config?.max_retries_upper_bound || MAX_RETRIES_UPPER_BOUND_DEFAULT)
else if (value < 1)
value = 1
handleRetryConfigChange({
retry_enabled: true,
max_retries: value,
retry_interval: retry_config?.retry_interval || 1000,
retry_interval: retry_config?.retry_interval || RETRY_INTERVAL_DEFAULT,
max_retries_upper_bound: retry_config?.max_retries_upper_bound || MAX_RETRIES_UPPER_BOUND_DEFAULT,
retry_interval_upper_bound: retry_config?.retry_interval_upper_bound || RETRY_INTERVAL_UPPER_BOUND_DEFAULT,
})
}

const handleRetryIntervalChange = (value: number) => {
if (value > 5000)
value = 5000
// リトライインターバルの制限をデフォルト値に基づき変更
if (value > (retry_config?.retry_interval_upper_bound || RETRY_INTERVAL_UPPER_BOUND_DEFAULT))
value = (retry_config?.retry_interval_upper_bound || RETRY_INTERVAL_UPPER_BOUND_DEFAULT)
else if (value < 100)
value = 100
handleRetryConfigChange({
retry_enabled: true,
max_retries: retry_config?.max_retries || 3,
max_retries: retry_config?.max_retries || MAX_RETRIES_DEFAULT,
retry_interval: value,
max_retries_upper_bound: retry_config?.max_retries_upper_bound || MAX_RETRIES_UPPER_BOUND_DEFAULT,
retry_interval_upper_bound: retry_config?.retry_interval_upper_bound || RETRY_INTERVAL_UPPER_BOUND_DEFAULT,
})
}

Expand All @@ -58,7 +66,7 @@ const RetryOnPanel = ({
<div className='mr-0.5 system-sm-semibold-uppercase text-text-secondary'>{t('workflow.nodes.common.retry.retryOnFailure')}</div>
</div>
<Switch
defaultValue={retry_config?.retry_enabled}
defaultValue={retry_config?.retry_enabled ?? false}
onChange={v => handleRetryEnabledChange(v)}
/>
</div>
Expand All @@ -69,18 +77,18 @@ const RetryOnPanel = ({
<div className='grow mr-2 system-xs-medium-uppercase'>{t('workflow.nodes.common.retry.maxRetries')}</div>
<Slider
className='mr-3 w-[108px]'
value={retry_config?.max_retries || 3}
value={retry_config?.max_retries || MAX_RETRIES_DEFAULT}
onChange={handleMaxRetriesChange}
min={1}
max={10}
max={retry_config?.max_retries_upper_bound || MAX_RETRIES_UPPER_BOUND_DEFAULT}
/>
<Input
type='number'
wrapperClassName='w-[80px]'
value={retry_config?.max_retries || 3}
value={retry_config?.max_retries || MAX_RETRIES_DEFAULT}
onChange={e => handleMaxRetriesChange(e.target.value as any)}
min={1}
max={10}
max={retry_config?.max_retries_upper_bound || MAX_RETRIES_UPPER_BOUND_DEFAULT}
unit={t('workflow.nodes.common.retry.times') || ''}
className={s.input}
/>
Expand All @@ -89,18 +97,18 @@ const RetryOnPanel = ({
<div className='grow mr-2 system-xs-medium-uppercase'>{t('workflow.nodes.common.retry.retryInterval')}</div>
<Slider
className='mr-3 w-[108px]'
value={retry_config?.retry_interval || 1000}
value={retry_config?.retry_interval || RETRY_INTERVAL_DEFAULT}
onChange={handleRetryIntervalChange}
min={100}
max={5000}
max={retry_config?.retry_interval_upper_bound || RETRY_INTERVAL_UPPER_BOUND_DEFAULT}
/>
<Input
type='number'
wrapperClassName='w-[80px]'
value={retry_config?.retry_interval || 1000}
value={retry_config?.retry_interval || RETRY_INTERVAL_DEFAULT}
onChange={e => handleRetryIntervalChange(e.target.value as any)}
min={100}
max={5000}
max={retry_config?.retry_interval_upper_bound || RETRY_INTERVAL_UPPER_BOUND_DEFAULT}
unit={t('workflow.nodes.common.retry.ms') || ''}
className={s.input}
/>
Expand Down
2 changes: 2 additions & 0 deletions web/app/components/workflow/nodes/_base/components/retry/types.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,6 @@ export type WorkflowRetryConfig = {
max_retries: number
retry_interval: number
retry_enabled: boolean
max_retries_upper_bound?: number
retry_interval_upper_bound?: number
}
9 changes: 6 additions & 3 deletions web/app/components/workflow/nodes/http/default.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import {
ALL_CHAT_AVAILABLE_BLOCKS,
ALL_COMPLETION_AVAILABLE_BLOCKS,
} from '@/app/components/workflow/constants'
import { MAX_RETRIES_DEFAULT_HTTP_NODE, MAX_RETRIES_UPPER_BOUND_HTTP_NODE, RETRY_ENABLED_DEFAULT_HTTP_NODE, RETRY_INTERVAL_DEFAULT_HTTP_NODE, RETRY_INTERVAL_UPPER_BOUND_HTTP_NODE } from '@/config'

const nodeDefault: NodeDefault<HttpNodeType> = {
defaultValue: {
Expand All @@ -28,9 +29,11 @@ const nodeDefault: NodeDefault<HttpNodeType> = {
max_write_timeout: 0,
},
retry_config: {
retry_enabled: true,
max_retries: 3,
retry_interval: 100,
retry_enabled: RETRY_ENABLED_DEFAULT_HTTP_NODE,
max_retries: MAX_RETRIES_DEFAULT_HTTP_NODE,
retry_interval: RETRY_INTERVAL_DEFAULT_HTTP_NODE,
max_retries_upper_bound: MAX_RETRIES_UPPER_BOUND_HTTP_NODE,
retry_interval_upper_bound: RETRY_INTERVAL_UPPER_BOUND_HTTP_NODE,
},
},
getAvailablePrevNodes(isChatMode: boolean) {
Expand Down
8 changes: 8 additions & 0 deletions web/app/components/workflow/nodes/llm/default.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { BlockEnum, EditionType } from '../../types'
import { type NodeDefault, type PromptItem, PromptRole } from '../../types'
import type { LLMNodeType } from './types'
import { ALL_CHAT_AVAILABLE_BLOCKS, ALL_COMPLETION_AVAILABLE_BLOCKS } from '@/app/components/workflow/constants'
import { MAX_RETRIES_DEFAULT_LLM_NODE, MAX_RETRIES_UPPER_BOUND_LLM_NODE, RETRY_ENABLED_DEFAULT_LLM_NODE, RETRY_INTERVAL_DEFAULT_LLM_NODE, RETRY_INTERVAL_UPPER_BOUND_LLM_NODE } from '@/config'

const i18nPrefix = 'workflow.errorMsg'

Expand All @@ -26,6 +27,13 @@ const nodeDefault: NodeDefault<LLMNodeType> = {
vision: {
enabled: false,
},
retry_config: {
retry_enabled: RETRY_ENABLED_DEFAULT_LLM_NODE,
max_retries: MAX_RETRIES_DEFAULT_LLM_NODE,
retry_interval: RETRY_INTERVAL_DEFAULT_LLM_NODE,
max_retries_upper_bound: MAX_RETRIES_UPPER_BOUND_LLM_NODE,
retry_interval_upper_bound: RETRY_INTERVAL_UPPER_BOUND_LLM_NODE,
},
},
getAvailablePrevNodes(isChatMode: boolean) {
const nodes = isChatMode
Expand Down
9 changes: 9 additions & 0 deletions web/app/components/workflow/nodes/tool/default.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,22 @@ import type { NodeDefault } from '../../types'
import type { ToolNodeType } from './types'
import { VarType as VarKindType } from '@/app/components/workflow/nodes/tool/types'
import { ALL_CHAT_AVAILABLE_BLOCKS, ALL_COMPLETION_AVAILABLE_BLOCKS } from '@/app/components/workflow/constants'
import { MAX_RETRIES_DEFAULT_TOOL_NODE, MAX_RETRIES_UPPER_BOUND_TOOL_NODE, RETRY_ENABLED_DEFAULT_TOOL_NODE, RETRY_INTERVAL_DEFAULT_TOOL_NODE, RETRY_INTERVAL_UPPER_BOUND_TOOL_NODE } from '@/config'

const i18nPrefix = 'workflow.errorMsg'

const nodeDefault: NodeDefault<ToolNodeType> = {
defaultValue: {
tool_parameters: {},
tool_configurations: {},
retry_config: {
retry_enabled: RETRY_ENABLED_DEFAULT_TOOL_NODE,
max_retries: MAX_RETRIES_DEFAULT_TOOL_NODE,
retry_interval: RETRY_INTERVAL_DEFAULT_TOOL_NODE,
max_retries_upper_bound: MAX_RETRIES_UPPER_BOUND_TOOL_NODE,
retry_interval_upper_bound: RETRY_INTERVAL_UPPER_BOUND_TOOL_NODE,
},

},
getAvailablePrevNodes(isChatMode: boolean) {
const nodes = isChatMode
Expand Down
3 changes: 3 additions & 0 deletions web/app/components/workflow/utils.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ import type { ToolNodeType } from './nodes/tool/types'
import type { IterationNodeType } from './nodes/iteration/types'
import { CollectionType } from '@/app/components/tools/types'
import { toolParametersToFormSchemas } from '@/app/components/tools/utils/to-form-schema'
import { MAX_RETRIES_UPPER_BOUND_DEFAULT, RETRY_INTERVAL_UPPER_BOUND_DEFAULT } from '@/config'

const WHITE = 'WHITE'
const GRAY = 'GRAY'
Expand Down Expand Up @@ -286,6 +287,8 @@ export const initialNodes = (originNodes: Node[], originEdges: Edge[]) => {
retry_enabled: true,
max_retries: DEFAULT_RETRY_MAX,
retry_interval: DEFAULT_RETRY_INTERVAL,
max_retries_upper_bound: MAX_RETRIES_UPPER_BOUND_DEFAULT,
retry_interval_upper_bound: RETRY_INTERVAL_UPPER_BOUND_DEFAULT,
}
}

Expand Down
28 changes: 28 additions & 0 deletions web/config/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -260,3 +260,31 @@ export const TEXT_GENERATION_TIMEOUT_MS = textGenerationTimeoutMs
export const DISABLE_UPLOAD_IMAGE_AS_ICON = process.env.NEXT_PUBLIC_DISABLE_UPLOAD_IMAGE_AS_ICON === 'true'

export const FULL_DOC_PREVIEW_LENGTH = 50

// System default upper bounds of max retry and retry intervals
export const MAX_RETRIES_DEFAULT = 3
Copy link
Member

@crazywoola crazywoola Jan 21, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good idea. However I would recommend moving these parameters into docker-compose-template, see this below

dify/docker/docker-compose-template.yaml

Line 57 in 6e0fb05

TEXT_GENERATION_TIMEOUT_MS: ${TEXT_GENERATION_TIMEOUT_MS:-60000}

We have an environment config regarding the timeout settings for frontend images, your approach won't take effect unless it is rebuilt again or start from the source code. By moving those into template will make sure they can be changed every time.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@crazywoola
100% Agree, thanks for your review.
Let me do additional these changes. I will push those changes later once I implement it and test it.

Thanks!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI https://github.com/langgenius/dify/pull/12925/files

export const RETRY_INTERVAL_DEFAULT = 1000
export const RETRY_ENABLED_DEFAULT = false
export const MAX_RETRIES_UPPER_BOUND_DEFAULT = 10
export const RETRY_INTERVAL_UPPER_BOUND_DEFAULT = 5000

// Retries related parameters of LLM Node
export const MAX_RETRIES_DEFAULT_LLM_NODE = 3
export const RETRY_INTERVAL_DEFAULT_LLM_NODE = 1000
export const RETRY_ENABLED_DEFAULT_LLM_NODE = false
export const MAX_RETRIES_UPPER_BOUND_LLM_NODE = 10
export const RETRY_INTERVAL_UPPER_BOUND_LLM_NODE = 5000

// Retries related parameters of HTTP Node
export const MAX_RETRIES_DEFAULT_HTTP_NODE = 3
export const RETRY_INTERVAL_DEFAULT_HTTP_NODE = 100
export const RETRY_ENABLED_DEFAULT_HTTP_NODE = false
export const MAX_RETRIES_UPPER_BOUND_HTTP_NODE = 10
export const RETRY_INTERVAL_UPPER_BOUND_HTTP_NODE = 5000

// Retries related parameters of Tool Node
export const MAX_RETRIES_DEFAULT_TOOL_NODE = 3
export const RETRY_INTERVAL_DEFAULT_TOOL_NODE = 1000
export const RETRY_ENABLED_DEFAULT_TOOL_NODE = false
export const MAX_RETRIES_UPPER_BOUND_TOOL_NODE = 10
export const RETRY_INTERVAL_UPPER_BOUND_TOOL_NODE = 5000
Loading