Uploading files directly to S3 bucket using Presigned URLs

  • img
    Justin Joseph
  • March 2,2021

Uploading files to S3 directly from the browser is a great way to increase performance rather than file send to server then upload.

Generally, API allows two applications to communicate with each other. API transfer files, access, and process database. Nowadays, many individuals and companies are struggling to find solutions to store a massive amount of data. So, they depend on third-party storage solutions like S3 to solve their storage problem. Here API server gains additional resource cost by transferring the files to S3. Therefore, the clients could avoid the expense of RAM, CPU, etc., by uploading their files directly to S3. Pre-signed PUT data is a more comfortable and better way to upload files. By using pre-signed PUT data S3 helps us to upload files directly to it. If you want to create a thumbnail generation, you can create a lambda function to read the source's image and create a thumbnail image.

Summary

The client requests an endpoint that responds with a URL and pre-signed post data.

The client then uses the provided URL and pre-signed PUT data to form a request containing the file to be uploaded directly to S3.

Avoid CORS error

Assuming you already have an S3 bucket you'd like to upload your files to, the next step is to modify the bucket's CORS configuration to allow PUT requests.

[
    {
        "AllowedHeaders": [
            "*"
        ],
        "AllowedMethods": [
            "PUT",
            "POST",
            "DELETE",
            "GET",
            "HEAD"
        ],
        "AllowedOrigins": [
            "*"
        ],
        "ExposeHeaders": [
            "x-amz-server-side-encryption",
            "x-amz-request-id",
            "x-amz-id-2"
        ],
        "MaxAgeSeconds": 3000
    }
]

After that, we can create our pre-signed PUT request

Creating the pre-signed PUT request

We can write the endpoint function on lambda or node functions code similar to the following.

import aws from 'aws-sdk';

  const s3 = new aws.S3({ signatureVersion: 'v4' });
  
  s3.config.update({
    signatureVersion: 'v4',
    region: aws-region,
    accessKeyId: acess_key_id,
    secretAccessKey: secret_access_key
  });

  const fileType = contentType   // image/jpeg
    let presignedPUTURL = await s3.getSignedUrl('putObject', {
      Bucket: S3_BUCKET_Name,
      Key: fileName,
      Expires: 1000,
      ContentType: fileType,
    });
      
    return presignedPUTURL;

We verify that the file size is less than 100 megabytes and that the content-type of the file begins with 'image/'.

The result of getSignedUrl would look something like the following:

https://presignedurldemo.s3.eu-west-2.amazonaws.com/image.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJJWZ7B6WCRGMKFGQ%2F20180210%2Feu-west-2%2Fs3%2Faws4_request&X-Amz-Date=20180210T171315Z&X-Amz-Expires=1800&X-Amz-Signature=12b74b0788aa036bc7c3d03b3f20c61f1f91cc9ad8873e3314255dc479a25351&X-Amz-SignedHeaders=host

Uploading the file to S3

Once we've written the code to create a pre-signed PUT request, it's time to create the request itself and upload the file to S3.

const setImageUploadUsing = async() => {
    const awsS3ImageUpload = await axios.put(presignedUrl, file);
    return awsS3ImageUpload;
  }

Note: The file must be the full file, not FormData.

Subscribe to newsletter
Need more tech news? Tune in to our weekly newsletter to get the latest updates