-
Notifications
You must be signed in to change notification settings - Fork 1.1k
[wrangler] Add automatic queue provisioning to resource provisioning #11788
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
…system Extends the existing resource provisioning feature to support Cloudflare Queues. When users deploy with queue bindings configured, missing queues are now automatically created, providing the same seamless experience as KV, D1, and R2. Changes: - Add QueueHandler class following the R2Handler pattern (name-based, no ID) - Add queue provisioning helper functions using ComplianceConfig - Collect and de-duplicate queues from both producers and consumers - Skip ensureQueuesExistByConfig when provisioning is enabled - Include delivery_delay in queue binding transformation - Update deploy tests to use --x-provision=false for legacy behavior tests The feature is controlled by the existing --x-provision flag (enabled by default) and --x-auto-create flag for interactive vs automatic creation.
🦋 Changeset detectedLatest commit: b0ac2a1 The changes in this PR will be included in the next version bump. Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
Claude finished @irvinebroque's task —— View job Changeset ReviewReviewing changeset files... Tasks:
The changeset Markdown Headers: The changeset uses code blocks with JSONC format which is good, but there's a comment in the review from petebacondarwin noting "We prefer JSONC config nowadays" on this file. Looking at the changeset, it already uses However, the changeset has one minor formatting issue: Line 11: The comment syntax in the example uses Other validation results: The changeset is well-written overall with good examples and clear explanations. The only suggestion is to clarify the configuration format being shown or use the more standard TOML format. |
create-cloudflare
@cloudflare/kv-asset-handler
miniflare
@cloudflare/pages-shared
@cloudflare/unenv-preset
@cloudflare/vite-plugin
@cloudflare/vitest-pool-workers
@cloudflare/workers-editor-shared
@cloudflare/workers-utils
wrangler
commit: |
|
@irvinebroque - was this PR generated by AI? If so, although you said you have reviewed it, can you mention this for transparency in the PR description? |
c34bbe1 to
3ef2e35
Compare
| // Skip queues - they use queue name from config, no ID write-back needed | ||
| if (resourceType === "queues") { | ||
| continue; | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not obvious to me why this ended up being necessary, given that Queues should behave like R2 and I don't see equivalent here for R2
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think these are noops since the binding will be identical and the patch function below will find no difference.
So I think you can remove this check and the one below to simplify this code.
petebacondarwin
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Exciting to see this implemented but I think we can be a bit cleaner with the approach.
| return mock; | ||
| } | ||
|
|
||
| // Legacy helpers for backwards compatibility |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not sure this comment is accurate. These helpers are not legacy nor related to backward compatibility. I would just delete this comment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| // Skip queues - they use queue name from config, no ID write-back needed | ||
| if (resourceType === "queues") { | ||
| continue; | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think these are noops since the binding will be identical and the patch function below will find no difference.
So I think you can remove this check and the one below to simplify this code.
| bindings: CfWorkerInit["bindings"], | ||
| requireRemote: boolean | ||
| requireRemote: boolean, | ||
| config?: Config |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rather than add a new parameter, just repurpose the complianceConfig parameter above. The actual object that is passed in there is always a Config anyway...
async function collectPendingResources(
config: Config,
accountId: string,
scriptName: string,
bindings: CfWorkerInit["bindings"],
requireRemote: boolean
}
| const resources = | ||
| resourceType === "queues" && config | ||
| ? collectUniqueQueueBindings(config) | ||
| : bindings[resourceType] ?? []; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rather than special casing here, I would rather we use polymorphism by adding a new method to the ProvisionResourceHandler class that returns the binding:
getResourcesFromConfig(config: Config | undefined, bindings: B[]) {
return bindings;
}and then in the QueueHandler class:
getResourcesFromConfig(config: Config | undefined, bindings: B[]) {
assert(config);
return collectUniqueQueueBindings(config);
}and the call site would look like:
const resources = HANDLERS[resourceType].getResourcesFromConfig(config, bindings[resourceType] ?? []);Ensures queue producer bindings include the `remote` field, matching the pattern used by R2 and other bindings that support remote mode.
Rather than special-casing queues in collectPendingResources, add a getResourcesFromConfig method to each handler in HANDLERS that can be overridden per resource type. QueueHandler's method calls collectUniqueQueueBindings to collect both producers and consumers. Also simplifies the function signature by using Config directly instead of ComplianceConfig, since Config is always passed in practice.
Clarify that the queue skipping in config patching is required because: 1. config.queues has a different shape (producers/consumers) than other bindings which are flat arrays 2. Queues use name-based identification so no ID write-back is needed
The mockListQueues and mockCreateQueue helpers are standard test utilities, not legacy helpers for backwards compatibility.
| getResourcesFromConfig: ( | ||
| _config: Config, | ||
| bindings: CfKvNamespace[] | ||
| ): CfKvNamespace[] => bindings, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My suggestion was for this to be a method on the handler (e.g. here KVHandler. The "standard" implementation will be on ProvisionResourceHandler and only the QueueHandler needs the method to be overridden.
Moreover, I believe that this method can use a type parameter to avoid having to cast the return values.
Extends the existing resource provisioning feature to support Cloudflare Queues. When users deploy with queue bindings configured, missing queues are now automatically created, just like KV, D1, and R2.
Changes:
The feature is controlled by the existing
--x-provision flag(enabled by default) and--x-auto-createflag for interactive vs automatic creation.I've tested this out locally and deployed successfully. And reviewed code, left comments. Written with opencode.
Questions