Skip to content

DR-6663 Added rate limiter to create import #73

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
aidankmcalister wants to merge 3 commits into
base: main
Choose a base branch
from
Open

DR-6663 Added rate limiter to create import #73

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c0b1b65
feat: rate limiter added
aidankmcalister Dec 18, 2025
bbe7180
fix: rate limiter moved to create-db-worker
aidankmcalister Dec 18, 2025
46b9bf2
fix: rate limit changed to 5 per minute
aidankmcalister Dec 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 47 additions & 1 deletion create-db-worker/src/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ interface Env {
DELETE_DB_WORKFLOW: Workflow;
DELETE_STALE_WORKFLOW: Workflow;
CREATE_DB_RATE_LIMITER: RateLimit;
PROGRAMMATIC_RATE_LIMITER: RateLimit;
CREATE_DB_DATASET: AnalyticsEngineDataset;
POSTHOG_API_KEY?: string;
POSTHOG_API_HOST?: string;
Expand Down Expand Up @@ -130,6 +131,7 @@ export default {
name?: string;
analytics?: { eventName?: string; properties?: Record<string, unknown> };
userAgent?: string;
source?: 'programmatic' | 'cli';
};

let body: CreateDbBody = {};
Expand All @@ -140,7 +142,51 @@ export default {
return new Response('Invalid JSON body', { status: 400 });
}

const { region, name, analytics: analyticsData, userAgent } = body;
const { region, name, analytics: analyticsData, userAgent, source } = body;

// Apply stricter rate limiting for programmatic requests
if (source === 'programmatic') {
const programmaticKey = `programmatic:${clientIP}`;
try {
const res = await env.PROGRAMMATIC_RATE_LIMITER.limit({
key: programmaticKey,
});

if (!res.success) {
return new Response(
JSON.stringify({
error: 'RATE_LIMIT_EXCEEDED',
message: 'Rate limit exceeded for programmatic database creation. You can create up to 1 database per minute. Please try again later.',
rateLimitInfo: {
retryAfterMs: 60000, // Approximate - Cloudflare doesn't expose exact timing
currentCount: 1,
maxRequests: 1,
},
}),
{
status: 429,
headers: {
'Content-Type': 'application/json',
'Retry-After': '60',
},
},
);
}
Comment on lines +155 to +174
Copy link

@coderabbitai coderabbitai bot Dec 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Inconsistent rate limit values in error response.

The error message says "1 database per minute" and rateLimitInfo reports maxRequests: 1, but the actual rate limiter config in wrangler.jsonc is limit: 5 per 60 seconds. This mismatch will confuse API consumers and may cause incorrect retry behavior.

🔎 Apply this diff to fix the inconsistency: 
 					if (!res.success) {
 						return new Response(
 							JSON.stringify({
 								error: 'RATE_LIMIT_EXCEEDED',
-								message: 'Rate limit exceeded for programmatic database creation. You can create up to 1 database per minute. Please try again later.',
+								message: 'Rate limit exceeded for programmatic database creation. You can create up to 5 databases per minute. Please try again later.',
 								rateLimitInfo: {
 									retryAfterMs: 60000, // Approximate - Cloudflare doesn't expose exact timing
-									currentCount: 1,
-									maxRequests: 1,
+									maxRequests: 5,
 								},
 							}),

Also consider removing currentCount since it's hardcoded and doesn't reflect the actual request count.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if (!res.success) {
return new Response(
JSON.stringify({
error: 'RATE_LIMIT_EXCEEDED',
message: 'Rate limit exceeded for programmatic database creation. You can create up to 1 database per minute. Please try again later.',
rateLimitInfo: {
retryAfterMs: 60000, // Approximate - Cloudflare doesn't expose exact timing
currentCount: 1,
maxRequests: 1,
},
}),
{
status: 429,
headers: {
'Content-Type': 'application/json',
'Retry-After': '60',
},
},
);
}
if (!res.success) {
return new Response(
JSON.stringify({
error: 'RATE_LIMIT_EXCEEDED',
message: 'Rate limit exceeded for programmatic database creation. You can create up to 5 databases per minute. Please try again later.',
rateLimitInfo: {
retryAfterMs: 60000, // Approximate - Cloudflare doesn't expose exact timing
maxRequests: 5,
},
}),
{
status: 429,
headers: {
'Content-Type': 'application/json',
'Retry-After': '60',
},
},
);
}
🤖 Prompt for AI Agents 
In create-db-worker/src/index.ts around lines 155 to 174, the rate-limit error
response hardcodes "1 database per minute", maxRequests: 1 and a currentCount: 1
while the actual rate limiter in wrangler.jsonc is limit: 5 per 60 seconds;
update the response to reflect the real limiter (set message to "5 databases per
minute" or better: derive values from the rate-limit config/constants, set
rateLimitInfo.maxRequests to 5, remove the hardcoded currentCount field, and
keep retryAfterMs/Retry-After in sync with the configured window (60 seconds));
ensure these values are read from the same config/constants used by the limiter
rather than duplicated literals.

} catch (e) {
console.error('Programmatic rate limiter error:', e);
// Fail closed for programmatic requests
return new Response(
JSON.stringify({
error: 'rate_limiter_error',
message: 'Rate limiter temporarily unavailable. Please try again later.',
}),
{
status: 503,
headers: { 'Content-Type': 'application/json' },
},
);
}
}
if (!region || !name) {
return new Response('Missing region or name in request body', { status: 400 });
}
Expand Down
9 changes: 9 additions & 0 deletions create-db-worker/wrangler.jsonc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,15 @@
"period": 60,
},
},
{
"name": "PROGRAMMATIC_RATE_LIMITER",
"type": "ratelimit",
"namespace_id": "1006",
"simple": {
"limit": 5,
"period": 60,
},
},
],
},
}
21 changes: 20 additions & 1 deletion create-db/src/database.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@ export async function createDatabaseCore(
createDbWorkerUrl: string,
claimDbWorkerUrl: string,
userAgent?: string,
cliRunId?: string
cliRunId?: string,
source?: "programmatic" | "cli"
): Promise<CreateDatabaseResult> {
const name = new Date().toISOString();
const runId = cliRunId ?? randomUUID();
Expand All @@ -27,6 +28,7 @@ export async function createDatabaseCore(
name,
utm_source: getCommandName(),
userAgent,
source: source || "cli",
}),
});

Expand All @@ -37,6 +39,23 @@ export async function createDatabaseCore(
runId,
createDbWorkerUrl
);

// Try to parse the rate limit response from the server
try {
const errorData = await resp.json();
if (errorData.error === "RATE_LIMIT_EXCEEDED" && errorData.rateLimitInfo) {
return {
success: false,
error: "RATE_LIMIT_EXCEEDED",
message: errorData.message,
rateLimitInfo: errorData.rateLimitInfo,
status: 429,
};
}
} catch {
// If parsing fails, fall through to generic message
}

return {
success: false,
error: "rate_limit_exceeded",
Expand Down
10 changes: 7 additions & 3 deletions create-db/src/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,14 +77,16 @@ const validateRegionWithUrl = (region: string) =>
const createDatabaseCoreWithUrl = (
region: string,
userAgent?: string,
cliRunId?: string
cliRunId?: string,
source?: "programmatic" | "cli"
) =>
createDatabaseCore(
region,
CREATE_DB_WORKER_URL,
CLAIM_DB_WORKER_URL,
userAgent,
cliRunId
cliRunId,
source
);

const router = os.router({
Expand Down Expand Up @@ -392,7 +394,9 @@ export async function create(
): Promise<CreateDatabaseResult> {
return createDatabaseCoreWithUrl(
options?.region || "us-east-1",
options?.userAgent
options?.userAgent,
undefined,
"programmatic"
);
}

Expand Down
5 changes: 5 additions & 0 deletions create-db/src/types.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,11 @@ export interface DatabaseError {
raw?: string;
details?: unknown;
status?: number;
rateLimitInfo?: {
retryAfterMs: number;
currentCount: number;
maxRequests: number;
};
}

export type CreateDatabaseResult = DatabaseResult | DatabaseError;
Expand Down