AWS MEN Stack File Upload Guide

Welcome, Future Developer!

Imagine you're building a photo-sharing website. Users want to upload pictures of their cats, their vacations, their food. But where do those pictures actually go? Your computer? The user's computer? The cloud? And what even is "the cloud"?

By the end of this guide, you'll understand exactly how file uploads work on the web, and you'll have built one yourself. Let's begin!

Understanding the Big Picture

Before we write a single line of code, let's understand what we're building and why.

The Restaurant Analogy

Think of your web application like a restaurant:

When a user uploads a photo:

1. They use the menu (EJS form) to select their file
2. The waiter (Express) takes the file and passes it to the kitchen
3. The kitchen (Node.js) processes it and sends it to the storage room (AWS S3)
4. The recipe book (MongoDB) records where the file was stored (the URL/link)
5. Later, when someone wants to see the photo, we look up the location in MongoDB and fetch it from S3

Why Not Store Files Directly in MongoDB?

Great question! Here's why:

1. Size Limits: MongoDB documents have a 16MB limit. A single high-resolution photo can exceed this.
2. Cost: Database storage is much more expensive than file storage (S3).
3. Performance: Databases are optimized for querying data, not serving large files.
4. Scalability: S3 can serve millions of files globally; your database would struggle.

The Solution: Store the address (URL) of the file in MongoDB, and the actual file in S3. It's like storing your friend's address in your phone instead of keeping their entire house in your pocket!

Phase 1: The Cloud Warehouse (AWS Setup)

What is AWS?

AWS (Amazon Web Services) is like a massive digital city owned by Amazon. In this city, you can rent:

• Storage rooms (S3)
• Computers (EC2)
• Databases (RDS)
• And hundreds of other services

You only pay for what you use, like a utility bill.

What is S3?

S3 (Simple Storage Service) is AWS's storage room service.

Analogy: Think of S3 like a self-storage facility (like Public Storage or U-Haul). You rent a storage unit (called a "bucket"), put your stuff (files) inside, and each item gets a unique location tag (URL) so you can find it later.

What is an S3 Bucket?

A bucket is a container for your files.

Analogy: If S3 is the storage facility, a bucket is your individual storage unit. You might have one bucket for profile pictures, another for documents, another for videos. Each bucket has a globally unique name.

What is IAM?

IAM (Identity and Access Management) is AWS's security guard system.

Analogy: Imagine the storage facility has a security office. You (the owner) have a master key that opens everything. But you also need to give a key to your assistant (your application). You don't want to give them the master key! Instead, you create a special key (IAM User) that only opens specific doors (permissions) they need. If that key is ever stolen, you can deactivate just that key without affecting anything else.

Step 1.1: Create an AWS Account

1. Go to: https://aws.amazon.com
2. Click: "Create an AWS Account" (top right)
3. Fill in: Email address and AWS account name
4. Verify your email
5. Set a strong password
6. Choose account type: Select "Personal"
7. Enter payment information (AWS requires a credit card, but we'll stay in the free tier)
8. Verify your phone number
9. Choose a support plan: Select "Basic support - Free"
10. Sign in to your new account

Don't worry! AWS has a "Free Tier" that gives you 5GB of S3 storage free for 12 months.

Step 1.2: Create an S3 Bucket

1. Search for "S3" in the AWS search bar
2. Click "Create bucket"
3. Choose a unique bucket name (e.g., my-photo-uploads-2024-yourname)
4. Choose an AWS Region closest to you
5. Object Ownership: Select "ACLs disabled (recommended)"
6. Block Public Access: UNCHECK "Block all public access" and acknowledge
7. Bucket Versioning: Leave disabled
8. Default encryption: Leave as default
9. Click "Create bucket"

Step 1.3: Configure Bucket Policy

We need to tell our bucket: "Anyone can READ (view) these files, but only authorized users can WRITE (upload) them."

1. Click on your bucket name to open it
2. Click the "Permissions" tab
3. Scroll to "Bucket policy" and click "Edit"
4. Paste this policy (replace YOUR-BUCKET-NAME):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PublicReadGetObject",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::YOUR-BUCKET-NAME/*"
        }
    ]
}

5. Click "Save changes"

Step 1.4: Configure CORS

What is CORS? Imagine you're at a restaurant, and you want food from the restaurant next door. The waiter says "I can only serve food from THIS restaurant." CORS is the permission slip that lets your website request files from S3.

1. Stay in the "Permissions" tab
2. Scroll to "Cross-origin resource sharing (CORS)" and click "Edit"
3. Paste this configuration:

[
    {
        "AllowedHeaders": ["*"],
        "AllowedMethods": ["GET", "PUT", "POST", "DELETE"],
        "AllowedOrigins": ["*"],
        "ExposeHeaders": []
    }
]

4. Click "Save changes"

Step 1.5: Create an IAM User

1. Search for "IAM" in the AWS search bar
2. Click "Users" in the left sidebar
3. Click "Create user"
4. Enter a username (e.g., photo-upload-app-user)
5. Click "Next"
6. Choose "Attach policies directly"
7. Search and check: AmazonS3FullAccess
8. Click "Next" then "Create user"

Step 1.6: Create Access Keys

1. Click on your new user's name
2. Click the "Security credentials" tab
3. Scroll to "Access keys" and click "Create access key"
4. Choose "Application running outside AWS"
5. Check acknowledgment and click "Next"
6. Add description tag: "Local development key"
7. Click "Create access key"

CRITICAL: Copy both the Access key ID and Secret access key. This is the only time you'll see the secret!

8. Click "Download .csv file" as backup
9. Click "Done"

Phase 2: The Foundation (Project Setup)

What is Node.js?

Analogy: Think of Node.js as a translator. Your computer speaks one language (binary), JavaScript speaks another. Node.js lets you run JavaScript code directly on your computer, outside of a web browser.

What is npm?

Analogy: npm (Node Package Manager) is like an app store for code. Instead of writing everything from scratch, you can download pre-made tools that other developers have shared.

Step 2.1: Install Node.js

Check if you already have it:

node --version

If not installed, go to https://nodejs.org and download the LTS version.

Step 2.2: Create Your Project

cd ~/Documents
mkdir photo-upload-app
cd photo-upload-app
npm init -y

Step 2.3: Install Dependencies

npm install express ejs mongoose multer multer-s3 @aws-sdk/client-s3 dotenv

Step 2.4: Create Project Structure

mkdir views public models
touch server.js .env .env.example .gitignore
touch views/upload.ejs views/success.ejs views/gallery.ejs
touch models/Image.js

Step 2.5: Configure .env File

Edit .env with your actual values:

# AWS Credentials
AWS_ACCESS_KEY_ID=your_access_key_here
AWS_SECRET_ACCESS_KEY=your_secret_key_here

# AWS S3 Configuration
AWS_REGION=us-east-1
AWS_BUCKET_NAME=your-bucket-name

# MongoDB Connection
MONGODB_URI=mongodb://localhost:27017/photo-upload-app

# Server Configuration
PORT=3000

Step 2.6: Configure .gitignore

.env
node_modules/
.DS_Store
*.log

Phase 3: The Middleman (Server Code)

What is Express?

Analogy: Express is like an experienced waiter. When a customer (web browser) makes a request, the waiter knows exactly where to go and what to bring back.

What is Middleware?

Analogy: Middleware is like checkpoints your food goes through. First, the kitchen prepares it. Then, quality control checks it. Each step processes the request before it moves on.

The Complete Server Code (server.js)

// Load environment variables first
require('dotenv').config();

const express = require('express');
const mongoose = require('mongoose');
const multer = require('multer');
const multerS3 = require('multer-s3');
const { S3Client } = require('@aws-sdk/client-s3');
const path = require('path');
const Image = require('./models/Image');

// Configure AWS S3 connection
const s3Client = new S3Client({
    region: process.env.AWS_REGION,
    credentials: {
        accessKeyId: process.env.AWS_ACCESS_KEY_ID,
        secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
    }
});

// Configure file upload handler
const upload = multer({
    storage: multerS3({
        s3: s3Client,
        bucket: process.env.AWS_BUCKET_NAME,
        metadata: function (req, file, cb) {
            cb(null, { fieldName: file.fieldname });
        },
        key: function (req, file, cb) {
            const uniqueName = Date.now().toString() + path.extname(file.originalname);
            cb(null, uniqueName);
        }
    }),
    limits: { fileSize: 5 * 1024 * 1024 },
    fileFilter: function (req, file, cb) {
        const allowedTypes = /jpeg|jpg|png|gif|webp/;
        const extname = allowedTypes.test(path.extname(file.originalname).toLowerCase());
        const mimetype = allowedTypes.test(file.mimetype);
        if (extname && mimetype) {
            cb(null, true);
        } else {
            cb(new Error('Only image files are allowed!'));
        }
    }
});

// Create Express application
const app = express();

// Configure middleware
app.set('view engine', 'ejs');
app.set('views', path.join(__dirname, 'views'));
app.use(express.static(path.join(__dirname, 'public')));
app.use(express.urlencoded({ extended: true }));
app.use(express.json());

// Connect to MongoDB
mongoose.connect(process.env.MONGODB_URI)
    .then(() => console.log('Connected to MongoDB!'))
    .catch((err) => {
        console.error('MongoDB connection error:', err.message);
        process.exit(1);
    });

// Routes
app.get('/', (req, res) => {
    res.render('upload', { error: null });
});

app.post('/upload', upload.single('image'), async (req, res) => {
    try {
        if (!req.file) {
            return res.render('upload', { error: 'Please select a file!' });
        }

        const newImage = new Image({
            imageUrl: req.file.location,
            s3Key: req.file.key,
            originalName: req.file.originalname,
            fileSize: req.file.size,
            mimeType: req.file.mimetype
        });

        await newImage.save();
        res.render('success', { imageUrl: req.file.location, imageId: newImage._id });
    } catch (error) {
        res.render('upload', { error: 'Upload failed: ' + error.message });
    }
});

app.get('/gallery', async (req, res) => {
    try {
        const images = await Image.find().sort({ createdAt: -1 });
        res.render('gallery', { images });
    } catch (error) {
        res.status(500).send('Error loading gallery');
    }
});

// Error handling
app.use((err, req, res, next) => {
    if (err instanceof multer.MulterError) {
        if (err.code === 'LIMIT_FILE_SIZE') {
            return res.render('upload', { error: 'File too large! Max 5MB.' });
        }
        return res.render('upload', { error: 'Upload error: ' + err.message });
    }
    if (err.message.includes('Only image files')) {
        return res.render('upload', { error: err.message });
    }
    res.render('upload', { error: 'Something went wrong.' });
});

// Start server
const PORT = process.env.PORT || 3000;
app.listen(PORT, () => {
    console.log(`Server running at http://localhost:${PORT}`);
});

Phase 4: The Interface (EJS Templates)

What is EJS?

Analogy: EJS (Embedded JavaScript) is like a "fill in the blanks" template. Imagine a form letter: "Dear _____, Thank you for your order of _____." EJS lets us create HTML with blanks that get filled in with real data.

The syntax <%= something %> means "print the value here."

The syntax <% code %> lets us write JavaScript logic.

Upload Form (views/upload.ejs)

See the starter code download for the complete file. Key points:

• Uses enctype="multipart/form-data" - CRITICAL for file uploads!
• The input name="image" must match upload.single('image')
• Shows error messages conditionally with <% if (error) %>

Success Page (views/success.ejs)

Displays the uploaded image using its S3 URL and provides navigation.

Gallery Page (views/gallery.ejs)

Loops through all images with <% images.forEach %> to display a grid.

Phase 5: The Librarian (MongoDB)

What is MongoDB?

Analogy: MongoDB is like a librarian with a very flexible filing system. Unlike traditional databases (like a filing cabinet with fixed-size drawers), MongoDB uses "documents" - think of them as index cards that can have any information you want on them.

What is Mongoose?

Analogy: Mongoose is your translator when talking to the MongoDB librarian. You speak JavaScript, the librarian speaks MongoDB Query Language. Mongoose also lets you define "schemas" - templates that say what fields each document should have.

Install MongoDB

Option A: Local Installation

1. Go to: mongodb.com/try/download/community
2. Download and install for your OS
3. Verify: mongosh --eval "db.version()"

Option B: MongoDB Atlas (Cloud)

1. Go to: mongodb.com/cloud/atlas
2. Create a free account and cluster
3. Get your connection string and update .env

Image Model (models/Image.js)

const mongoose = require('mongoose');

const imageSchema = new mongoose.Schema({
    imageUrl: {
        type: String,
        required: [true, 'Image URL is required'],
        trim: true
    },
    s3Key: {
        type: String,
        required: [true, 'S3 key is required'],
        trim: true
    },
    originalName: {
        type: String,
        required: [true, 'Original filename is required'],
        trim: true
    },
    fileSize: {
        type: Number,
        required: true,
        min: [0, 'File size cannot be negative']
    },
    mimeType: {
        type: String,
        required: true,
        trim: true
    },
    description: {
        type: String,
        trim: true,
        maxlength: [500, 'Description too long']
    },
    tags: {
        type: [String],
        default: []
    }
}, {
    timestamps: true,
    collection: 'images'
});

// Virtual for KB size
imageSchema.virtual('fileSizeKB').get(function() {
    return (this.fileSize / 1024).toFixed(2);
});

// Indexes for faster queries
imageSchema.index({ createdAt: -1 });
imageSchema.index({ tags: 1 });

module.exports = mongoose.model('Image', imageSchema);

Why URLs, Not Files?

Putting It All Together

Final Project Structure

photo-upload-app/
├── models/
│   └── Image.js
├── node_modules/
├── public/
├── views/
│   ├── upload.ejs
│   ├── success.ejs
│   └── gallery.ejs
├── .env
├── .env.example
├── .gitignore
├── package.json
├── package-lock.json
└── server.js

Final Checklist

AWS Setup

• Created AWS account
• Created S3 bucket
• Configured bucket policy (public read)
• Configured CORS
• Created IAM user with S3 access
• Saved access keys

Local Setup

• Node.js installed
• Project folder created
• Dependencies installed
• .env file configured
• MongoDB running

Testing Your Application

Step 1: Start MongoDB

# Mac with Homebrew
brew services start mongodb-community

# Windows (as administrator)
net start MongoDB

Step 2: Start Your Application

cd ~/Documents/photo-upload-app
node server.js

You should see:

Connected to MongoDB!
Server running at http://localhost:3000

Step 3: Test the Upload

1. Open browser: http://localhost:3000
2. Select an image file
3. Click "Upload Image"
4. See your image on the success page!

Step 4: Verify in AWS S3

Go to AWS Console → S3 → Your bucket and look for your uploaded file.

Step 5: Verify in MongoDB

mongosh
use photo-upload-app
db.images.find().pretty()

Troubleshooting Common Issues

"Cannot connect to MongoDB"

1. Make sure MongoDB is running: ps aux | grep mongod
2. Check your MONGODB_URI in .env

"Access Denied" uploading to S3

1. Double-check .env values (no spaces, exact copy)
2. Verify IAM user has AmazonS3FullAccess policy
3. Verify bucket region matches AWS_REGION

Image uploads but shows broken

1. Check bucket's Block Public Access is OFF
2. Verify bucket policy allows GetObject
3. Check browser network tab for CORS errors

"File too large" error

We set a 5MB limit. To increase:

limits: { fileSize: 10 * 1024 * 1024 } // 10 MB

Syntax errors on startup

1. Check Node.js version: node --version (need 14+)
2. Reinstall packages: rm -rf node_modules && npm install

Download Starter Code

Get the complete working code to start building immediately:

What's Included

• server.js - Complete Express server with S3 integration
• models/Image.js - MongoDB schema
• views/upload.ejs - Upload form
• views/success.ejs - Success page
• views/gallery.ejs - Image gallery
• .env.example - Environment template
• package.json - Dependencies

Quick Start

1. Copy .env.example to .env
2. Add your AWS credentials and MongoDB URI
3. Run npm install
4. Run node server.js
5. Open http://localhost:3000

The Complete Flow

User clicks "Upload" on webpage (EJS)
         |
         v
Browser sends file to Express server
         |
         v
Multer middleware receives the file
         |
         v
Multer-S3 uploads file to AWS S3
         |
         v
S3 returns the file URL
         |
         v
Express saves URL to MongoDB
         |
         v
Express renders success page with image
         |
         v
User sees their uploaded image!

Congratulations! You've learned how to build a complete file upload system with AWS S3, MongoDB, Express, and Node.js.

Next Steps

• Add user authentication
• Add image deletion
• Add drag-and-drop uploads
• Add multiple file uploads
• Add image resizing/thumbnails
• Deploy to production