Using AWS CloudFront Path-Based behaviour for uploading to multiple S3 Buckets with CF and S3 resigned URLs
TLDR
- Replace multiple S3 endpoint with a single custom CloudFront domain (e.g., for branding reasons)
- Use Path-Based behaviour in CloudFront to handle destination bucket (e.g., for app compatibility reasons)
- Combine the following:
- CloudFront Presigned URL - an "outer auth" for the request to be let in
- S3 Presigned Post - an "inner auth" against S3 for to POST request to be processed
Design Diagram
- S3 Presigned Post URL
- The S3 endpoint generated is discarded
- The POST field containing signature (S3 Auth), policy, bucket name and key related to the upload is retained
- CloudFront Signed URL
- The URL is made up of CloudFront custom domain + Path of each bucket's associated CloudFront behaviour
- CloudFront signer adds query string to the URI with signatures, expirations, etc
- The Signed URL forms the outer credentials (CloudFront auth) to access the CloudFront behaviour
- These two are then combined and sent to the client, and the client POST the content to the CloudFront distribution
- See flow above for how the request is combined
- On the CloudFront side, a Javascript based CloudFront function is placed before the origin to strip out the URI (e.g., `/upload/bucket1`).
This is necessary so that S3 endpoint process the request as a S3 REST API call, instead of acting as a Website Endpoint.
Considerations
The main purpose of this design is mostly enterprise branding and for endpoint to be consolidated.
Performance might take a small impact, but bandwidth cost will be a major consideration.
During my super non-scientific testing, upload through CloudFront will never exceed that of direct to S3 endpoint, except for endpoints that is on the other side of the planet. (Tested from a domestic FTTP multi-gig connection.)
You will have to pay for all the bandwidth of the upload, which might be significant depending on your use case.
Comments
Post a Comment