Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AWS S3 PutObjectCommand extremely slow #27841

Open
HuakunShen opened this issue Jan 27, 2025 · 2 comments
Open

AWS S3 PutObjectCommand extremely slow #27841

HuakunShen opened this issue Jan 27, 2025 · 2 comments
Labels
bug Something isn't working correctly node compat

Comments

@HuakunShen
Copy link

Version: Deno 2.1.7

I am using "@aws-sdk/client-s3": "npm:@aws-sdk/[email protected]", to upload files to S3.

Here is sample code, uploading a 4KB file takes 6 seconds. The time is taken on step s3Client.send(command);.

I also tried uploading a 100MB file, took ~7.5 second. So the upload speed seems fine. Could it be issue with connection establish?

I ran the same code with bun, took 0.8ms to upload a 4KB file.

import { S3Client, PutObjectCommand } from '@aws-sdk/client-s3';

// Configure the S3 client
const s3Client = new S3Client({
	region: Deno.env.get('AWS_REGION') || 'us-east-1',
	credentials: {
		accessKeyId: Deno.env.get('AWS_ACCESS_KEY_ID') || '',
		secretAccessKey: Deno.env.get('AWS_SECRET_ACCESS_KEY') || ''
	}
});

// Function to upload a file to S3
async function uploadFileToS3(bucketName: string, key: string, filePath: string) {
	try {
		// Read the file
		const fileContent = await Deno.readFile(filePath);

		const command = new PutObjectCommand({
			Bucket: bucketName,
			Key: key,
			Body: fileContent
		});
		console.time('send');
		const response = await s3Client.send(command);
		console.timeEnd('send');
		console.log('File uploaded successfully:', response);
	} catch (error) {
		console.error('Error uploading file:', error);
		throw error; // Re-throw the error for better error handling
	}
}

// Example usage
const bucketName = Deno.env.get('AWS_BUCKET_NAME') || '';
await uploadFileToS3(
	bucketName,
	'wacv24-2686.mp4',
	'/Users/hk/Downloads/wacv24-2686.mp4'
);
@HuakunShen
Copy link
Author

Reproducible on M4 pro Mac, and x86 Ubuntu 22, didn't try Windows yet.

@lucacasonato lucacasonato added bug Something isn't working correctly node compat labels Jan 28, 2025
@bartlomieju
Copy link
Member

We believe this was the same problem as in #27239 which should be now fixed by #27381. Please try out latest canary (deno upgrade --canary) and let me know if the problem persists.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working correctly node compat
Projects
None yet
Development

No branches or pull requests

3 participants