Building an S3 File Browser with Cognito Authentication

"How hard could it be?" Famous last words. Building a web file browser for S3 sounds simple: list objects, render them as folders, let users click to download. Then you add authentication. Then authorization. Then you remember the browser is involved, which means CORS, token refresh, and a half-dozen other things that turn a weekend project into a quarter.

This is a real walkthrough of what it takes. The architecture, the parts that bite, and the design calls that matter.

The shape of it

A reasonable design has five layers:

Browser (React SPA)
  ↓
CloudFront (CDN + routing)
  ↓
API Gateway (REST)
  ↓
Lambda (logic)
  ↓
S3 (files) + DynamoDB (config) + Cognito (auth)

CloudFront pulls double duty: it serves the static React app and proxies API requests to API Gateway under the same domain. Same-origin keeps CORS out of your life, which is worth a lot.

API Gateway with a Cognito authorizer validates JWTs on every request. If a token is missing, expired, or wrong, the request never reaches your Lambda. That's one less thing to think about.

Lambda is where the real work happens: list files, generate pre-signed URLs, enforce who can see what. It looks up permissions from DynamoDB based on the user's Cognito groups.

Gotcha #1: CloudFront in front of API Gateway

Wiring CloudFront to two origins (S3 for the app, API Gateway for the API) is the first place people get stuck. You need behaviors that say "send /api/* over here, send everything else over there."

The non-obvious part: API Gateway is fussy about headers and paths. CloudFront strips the Authorization header by default, so you have to forward it explicitly, or the Cognito authorizer will reject every request because it never sees the token. You also need an origin path that strips the /api prefix before it hits API Gateway, plus the API Gateway stage name in the URL.

If you're on CDK, that means an HttpOrigin for API Gateway with the right origin path and a behavior that allows the headers you need. Get any piece of this wrong and you'll spend an afternoon staring at 403s wondering why local tests pass and production doesn't.

Gotcha #2: Cognito tokens in the browser

Cognito hands you three tokens. ID, access, refresh. You have to manage all of them.

The ID token has the user's identity claims (email, name, groups). The access token is what you actually send to the API. The refresh token is what gets you new ones when the others expire, which they will, after an hour, in the middle of someone's session.

The annoying details:

• Expiry handling. Your API client needs to catch 401s, refresh transparently, and retry the original request. Users should never see this happening.
• Where to store them. amazon-cognito-identity-js stashes tokens in localStorage by default. Convenient, but XSS-vulnerable. If you care, you'll need a backend token exchange that puts them in httpOnly cookies instead.
• Group claims. Groups land in the ID token under cognito:groups. The Cognito authorizer in API Gateway validates the token, but it does not enforce group membership. That check has to happen in your Lambda.

Gotcha #3: Listing S3 the way users expect

ListObjectsV2 caps out at 1,000 objects per call. Pagination, fine. Use the ContinuationToken, not interesting.

The interesting part is that nobody actually wants a flat dump of every object in a bucket. They want folders. S3 doesn't have folders. It just has keys with slashes. But you can fake it with Delimiter='/' and Prefix. That returns CommonPrefixes (your folders) plus the objects sitting at the current level. Walk into a "folder" by appending it to the prefix and querying again.

Your Lambda takes a prefix, returns two lists (folders, files), and the frontend renders the obvious tree.

Watch out for the small stuff: S3 returns full keys, so strip the current prefix before showing filenames. Empty folders sometimes show up as zero-byte objects with a trailing slash because someone "created a folder" in the console. You'll want to filter those out or handle them gracefully, depending on your UX.

Gotcha #4: Pre-signed URLs and the download UX

When someone clicks "download," your Lambda generates a pre-signed URL and returns it. The browser hits S3 directly. Files don't flow through your stack, which is great.

What you have to think about:

• Expiry. Short is safer. 5 to 15 minutes is the usual range. Too short and a slow download fails midway. Too long and a forwarded URL is a problem.
• Content-Disposition. Without setting ResponseContentDisposition on the pre-signed URL, browsers will happily render PDFs and images inline instead of downloading them. Almost never what you want.
• Filename encoding. Spaces, unicode, anything outside basic ASCII. They all need careful encoding in both the S3 key and the Content-Disposition header. This is the bug that hides until a German customer with umlauts files a ticket.

And critically: your Lambda has to verify the user's group can access this specific file before generating the URL. The URL itself is a delivery mechanism, not an access control. If you forget that check, you've built a pretty tunnel into your bucket.

Gotcha #5: Group-based permissions

This is where the design gets interesting. You need a data model that maps Cognito groups to bucket/prefix permissions, and an admin UI for managing those mappings.

A DynamoDB single-table design fits this well. One record per (group, bucket, prefix). Your list and download Lambdas query it to figure out what to allow.

The hard case is users in multiple groups. If group A has access to reports/ and group B has access to reports/confidential/, a user who's in both should see reports/ with the confidential folder included. Not duplicated, not split across two views. Your listing endpoint has to merge permissions and hand back something coherent. It's not impossible, but it's the kind of code that needs tests.

Gotcha #6: The edges

Production file browsers hit a long list of edges:

• Cross-region buckets. Pre-signed URLs are region-specific. A us-east-1 signature doesn't fly for an eu-west-1 bucket.
• Big files. Multi-gigabyte downloads can time out or stall. You may need range-request handling for the really large stuff.
• Deleted between list and download. The file was there a second ago. Now it's not. Handle the 404.
• Versioned buckets. Show versions or only the latest? It's a product decision, not a technical one.
• Empty states. What does a user see when they're not in any group, or their groups have no buckets configured? "Welcome, here's nothing" is a bad first impression.

Build it or buy it?

If you have a real reason — specific compliance constraints, weird requirements, an existing system to integrate with — building from scratch is fine. The architecture above is well-trodden. None of it is mysterious.

The honest cost, though, is bigger than it looks. Infrastructure, Lambdas, the React frontend with auth, the admin UI for permissions, the testing, the hardening. That's three to six weeks of focused work for a senior engineer. And then it's yours, including the parts you didn't think about.

BucketDrive is exactly this architecture, packaged up. One CloudFormation stack, deploys in five minutes, into your AWS account. The frontend, backend, auth, access control, and audit logs are pre-built and tested. If you're trying to decide between building and buying, this post should at least give you a fair picture of what you're signing up for.